When binary files (i.e. executables) are saved they usually have a format (e.g. ELF or .out) where we have a header containing pointers to where data or code is stored inside the file. But why don’t we store the binary files directly in the form of sequence of machine instructions.
Why do we need to store data separately from the code?
Secondly when the assembler creates a binary file is the file is among the above formats?
4
There are several use cases for metadata like this, some being “necessary” and others being nice to have.
- At the very bare minimum (if the code isn’t position-independent, which is still not the default), the loader needs to know at which address the code should go.
- On some platforms (including some in common use today, such as embedded micro controllers), there is separate memory for code and for data. The loader needs to be able to tell code from data to put the code into memory the CPU can execute from, and the data into memory the CPU can read and write.
- For binaries which aren’t executable files but rather libraries, you need a symbol table to enable code using that library to call the right function.
- Even if there isn’t physically separate memory for code and data, it’s beneficial to set up memory such that the data is not executable and the code is not writable. Sure, you could let the program do that itself in initialization, but that’s error prone, pointless code duplication for no good reason.
- Knowing the purpose of data sections can allow optimization. For example, if you have a separate
.bsssection for data that should be initialized to zero, a modern OS can lazily allocate zero’d pages to improve process creation time. If you have a.rodatasection for read-only data, this data can go into separate ROM (if your computer has ROM), freeing up RAM and possibly bringing other benefits. When running multiple instances of a program, the data assumed read-only (including code) can be shared among those processes.
I’m sure there are more reasons, I’m not very familiar with the design decisions that go into a loader.
2
Because we want our computers to do more than just execute one fixed program.
Having a stream of opcodes in memory and endlessly stepping through it is fine if you have a single-purpose machine. But on a real computer, you want to achieve things such as
- have one binary call another and transfer control to it
- have one binary call another and get control back after it terminates
- multi-task between many programs and make it appear as if they were running simultaneously
- give one program monitoring or administration rights over another program to limit the things the other program can do
- and much, much more!
For all of this, you need executable software that treats other executable software as its input data, which means that you have to be able to express things such as “this code owns that memory address” or “those two processes are really instances of the same program and can share their static data segment”.
A lot of the bytes in a binary file is, in fact, a string of machine code, but everything else has to be stored somehow as well, and a binary file format is just a spec saying how to do this.
1
Setting certain areas as data lets some architectures detect when trying to execute them (and then let the OS kill it).
Also there is an architecture design that splits code and data into separate address spaces so the file needs to make the difference between the 2 otherwise it wouldn’t be able to access the data. The restrictions of the executable format comes from trying to be cross compatible between them (assuming the same instruction sets).