I’m a complete beginner at Assembly, and my aim is to learn as much as I can to do with Assembly to one day I can reach expert level (I know I’m way off right now, but you never know). My only problem is this: I’ve got two books which both teach assembly, one on a Linux and the other on Windows. They are Jeff Duntemann’s Assembly Language Step By Step (the linux one) and Introduction to 80×86 Assembly Language and Computer Architecture (the windows version). If I want to get the best out of assembly, should I do this on linux and windows? Also, is the syntax the same on Windows and Linux or will I have teach my self again when learning on the other OS( which is my main concern, I want to be able to use assembly on windows and linux).
The one major difference between coding in ASM on Win/Linux/OSX is the calling convention. There isn’t one more difficult to use than the other, they’re just different. You also need to be careful about the register that need to be preserved between calls.
WINDOWS: RCX, RDX, R8, R9
The Microsoft x64 calling convention is followed on Windows and
pre-boot UEFI (for long mode on x86-64). It uses registers RCX, RDX,
R8, R9 for the first four integer or pointer arguments (in that
order), and XMM0, XMM1, XMM2, XMM3 are used for floating point
arguments. Additional arguments are pushed onto the stack (right to
left). Integer return values (similar to x86) are returned in RAX if
64 bits or less. Floating point return values are returned in XMM0.
Parameters less than 64 bits long are not zero extended; the high bits
are not zeroed.
LINUX/OSX: RDI, RSI, RDX, RCX, R8, R9
The calling convention of the System V AMD64 ABI is followed on
Solaris, Linux, FreeBSD, OS X, and other UNIX-like or POSIX-compliant
operating systems. The first six integer or pointer arguments are
passed in registers RDI, RSI, RDX, RCX, R8, and R9, while XMM0, XMM1,
XMM2, XMM3, XMM4, XMM5, XMM6 and XMM7 are used for floating point
arguments. For system calls, R10 is used instead of RCX. As in the
Microsoft x64 calling convention, additional arguments are passed on
the stack and the return value is stored in RAX.
TOOLS :
You’ll find pretty much the same tools on Windows & Linux.
My personal opinion :
- FASM & NASM are awesome 🙂
- I often use Purebasic, it’s a closed-source commercial (cheap, with a freeware limited version) basic that produce nice readable/commented FASM code and accept Inline Assembler. It doesn’t produce the fasted code ever, but it’s a tool i like. You can do without it of course, and just write pure ASM from scratch.
- DON’T BOTHER with 386, jump to x86_64 ASM. Much more instructions (beware : MMX/SSE packed instruction are weird, (very?) hard to use) and much much more registers (awesome!) !
- Hardcore option : Install BOCHS or QEMU (i use QEMU) and start writing ASM without operating system. Start with the MBR, then bootloader Stage 1, Stage2, a kernel, and so on… Feel free to take a look at my heavily documented MBR/Bootloader (functional but incomplete) https://github.com/ker2x/zymology/tree/master/bootloader/Oak64/src if you’re in the mood 🙂
1
It depends on what you want to write.
If all you want to write is a specific function that will be called from a “driver” program written in C, and if you do not need to perform a system call, then the C compiler you use is much more important than the architecture, because it prescribes how you have to expose your functions.
If you are instead interested in writing full fledged applications in assembly and making use of system calls — or writing device drivers — then the operating system is very relevant as well.
So basically, pick the one you want to write programs for.
In FreeBSD’s developers handbook, there is a chapter devoted to assembly programming. It is short and well written, and contains interesting examples, culminating with a simple CGI script(!). You can read it to get a feeling:
http://www.freebsd.org/doc/en/books/developers-handbook/x86.html
P.S.: Assembly is one of the very first languages I learned in the middle of the nineties. I learned it on an MS-DOS 6.22 system, using Isaacson’s shareware assembler and Ralf Brown’s interrupt list… thanks to Virtual Box you can probably get that feeling again! 🙂
1
Assembly differs between platforms, and even between different assemblers on the same platform. System calls and interrupts will be different, and there’s the matter of AT&T vs. Intel syntax. However, the difference between Windows and Linux assembly is much smaller than the difference between x86 and PIC assembly, for example. If you know one, it’s trivial to learn the other.
It’s much more useful to be able to read assembly than to write it, and you get better at reading it by trying several different architectures. Don’t limit yourself. In my job, x86 assembly never comes up, but I occasionally debug in PowerPC assembly.
Linux tends to favour AT&T style x86 assembly code for some reason. It’s quite awful. For doing Windows development you probably use Visual Studio which supports the far more readable Intel style syntax. Sample code from Intel uses this style too generally. Note that you cannot write inline assembly for 64 bits apps with Visual Studio, it works only for 32 bit apps. 64 bit apps are expected to use intrinsics instead. I don’t believe gcc has this restriction.
3