x86-64 calling convention by gcc
x86-64 calling convention in GCC
The x86-64 architecture on Linux follows the System V AMD64 ABI calling convention. Understanding this is essential when writing assembly, optimizing code, debugging, or interfacing with C libraries.
Register usage for arguments
Integer and pointer arguments use these registers in order:
RDI— first argumentRSI— second argumentRDX— third argumentRCX— fourth argumentR8— fifth argumentR9— sixth argument
Floating-point arguments use XMM registers:
XMM0throughXMM7— floating-point arguments
If a function takes both integer and floating-point arguments, they occupy their respective registers independently. The first integer argument still uses RDI even if floating-point arguments are present.
Additional arguments beyond the sixth integer or eighth floating-point argument are passed on the stack, in right-to-left order (the caller pushes them).
Return values
RAX— return value (alsoRDX:RAXfor 128-bit returns)XMM0andXMM1— floating-point return values
System calls
System calls follow a slightly different convention. The syscall number goes in RAX, and arguments use the same registers as normal calls except R10 is used instead of RCX for the fourth argument. This avoids clobbering RCX, which the syscall instruction uses internally.
Caller and callee saved registers
- Caller-saved (volatile):
RAX,RCX,RDX,RSI,RDI,R8,R9,R10,R11, and XMM0-XMM15 - Callee-saved (non-volatile):
RBX,RSP,RBP,R12,R13,R14,R15
The callee must preserve these registers if it uses them. The caller cannot assume them to survive a function call.
Stack alignment and red zone
The stack must be 16-byte aligned before a call instruction. Since call pushes an 8-byte return address, the stack is 8 bytes misaligned at function entry. Most code compensates by immediately pushing another 8-byte value (like rbp) to restore alignment.
The 128 bytes below RSP form a “red zone” that is safe from signal handlers. Functions can use this space for temporary data without allocating stack space, though signal handlers will overwrite it.
Practical example
#include <stdio.h>
long add_five(long a, long b, long c, long d, long e, long f, long g) {
return a + b + c + d + e + f + g;
}
int main() {
long result = add_five(1, 2, 3, 4, 5, 6, 7);
printf("%ld\n", result);
return 0;
}In this function:
athroughfarrive inRDI,RSI,RDX,RCX,R8,R9g(the seventh argument) is passed on the stack- The result is returned in
RAX
Examine the calling convention in practice with objdump -d or gdb to verify how arguments are passed.
Variadic functions
Functions with variable arguments (like printf) are more complex. They must track how many floating-point arguments were passed in XMM registers via RAX. The System V ABI specifies that RAX should contain the number of vector registers used for floating-point arguments (0-8). This allows variadic functions to know whether to retrieve arguments from registers or the stack.