6

I was working on memorizing the order of the Linux system calls so I could more easily identify this. And, then I found this paper here, and it says,

To make a system call in 32-bit Linux, place the system call number in eax, then its arguments, in order, in ebx, ecx, edx, esi, edi, and ebp, then invoke int 0x80.

And, then,

To make a system call in 64-bit Linux, place the system call number in rax, then its arguments, in order, in rdi, rsi, rdx, r10, r8, and r9, then invoke syscall.

How come the order gets so mutilated between 64 bit and 32 bit? I know this question may be historical rather than technical.

This is totally decided by the kernel, right? Are there technical reasons to favor the newer convention?

Rui F Ribeiro
  • 56,709
  • 26
  • 150
  • 232
Evan Carroll
  • 30,763
  • 48
  • 183
  • 315

2 Answers2

8

It is largely up to the kernel, however there is a good reason to use the chosen calling convention on 64-bit x86: it matches the chosen user-space convention. The System V x86-64 ABI, which is what Linux uses, specifies that functions use registers %rdi, %rsi, %rdx, %rcx, %r8, and %r9 to pass arguments. The system call convention is very close to that: the only difference is that it uses %r10 instead of %rcx, mainly because SYSCALL, a new 64-bit instruction used to invoke system calls, needs %rcx for other purposes.

Stephen Kitt
  • 434,908
  • SYSENTER isn't even available on x86_64 though, right? Could you explain more about this. Sorry, I come back months later as I learn more =) – Evan Carroll Sep 30 '18 at 21:00
  • 1
    SYSENTER is available on 64-bit x86 as well as 32-bit x86 (since the Pentium II), but it doesn’t need ECX/RCX so it doesn’t have any impact on this discussion (I’ve updated my answer). SYSCALL is 64-bit-specific and overwrites RCX. – Stephen Kitt Oct 01 '18 at 07:27
0

The orderof registers in compiled code is a calling convention for Linux to facilitate generating and debugging code.

The amd-64 architecture has more registers than the 32-bit architecture, and the new calling convention was used to free the other registers for better optimizing code.

If you disassembly code between the two architectures, you also will notice in amd-64 you will frequently see more registers taking the place of local integer variables in functions. If I am not wrong, rebp is used to optimize the code generated for riding the stack of local variables before returning from a function.

Rui F Ribeiro
  • 56,709
  • 26
  • 150
  • 232