1.17 - Calling Conventions (Windows)

X86 (32-bit, Windows)

Cdecl

• Arguments: stack, right → left
• Return value: EAX
• Stack cleanup: caller
• Registers preserved (callee-saved): EBX, ESI, EDI, EBP

Used by: C library functions (e.g., printf)

---

Stdcall

• Arguments: stack, right → left
• Return value: EAX
• Stack cleanup: callee
• Registers preserved: same as cdecl

Used by: WinAPI functions (MessageBoxA, etc.)

---

Fastcall (Microsoft variant)

• Arguments:
  • First arg: ECX
  • Second arg: EDX
  • Remaining: stack, right → left
• Return value: EAX
• Stack cleanup: callee

Used by: some compiler optimizations, certain system functions

---

Thiscall (for C++ Member functions)

• Arguments:
  • this pointer in ECX
  • Remaining args: stack, right → left
• Return value: EAX
• Stack cleanup: callee (usually)

---

Syscalls (NT Kernel, x86)

• Older Windows used int 0x2e, newer use sysenter
• Syscall number in EAX, args on stack (not typically relevant unless kernel reversing)

---

x86-64 (Windows)

Microsoft X64 Calling Convention

• Arguments (register order): 1 → RCX 2 → RDX 3 → R8 4 → R9 Remaining args: stack, right → left
• Return value: RAX
• Stack cleanup: caller
• Shadow space: Caller must allocate 32 bytes of stack space before call (for callee to spill first 4 args if needed)
• Registers preserved (callee-saved): RBX, RBP, RDI, RSI, R12–R15, RSP
• Registers volatile (caller-saved): RAX, RCX, RDX, R8–R11

Stack layout (at call entry):

[RSP]      return address
[RSP+8]    shadow space (32 bytes reserved)
[RSP+40]   5th argument if needed

Example:

mov rcx, 1     ; arg1
mov rdx, 2     ; arg2
mov r8,  3     ; arg3
mov r9,  4     ; arg4
call func
; result in RAX

---

Syscalls (Windows x64)

• Windows does not use the System V convention.
• Userland syscalls use syscall with a syscall stub from ntdll.dll.
• Syscall number in RAX, args typically in RCX, RDX, R8, R9, stack for the rest.
• Return in RAX.

---

Quick Reference Table

Platform	Convention	Args order	Return	Stack cleanup	Notes
x86	cdecl	stack (right→left)	EAX	caller	C lib
x86	stdcall	stack (right→left)	EAX	callee	WinAPI
x86	fastcall	ECX, EDX, stack (right→left)	EAX	callee	Optimized calls
x86	thiscall	ECX=this, stack (right→left)	EAX	callee	C++ methods
x86-64	Microsoft	RCX, RDX, R8, R9, stack (right→left)	RAX	caller	Only convention on Win64

---

Tips for Reversing Windows

• On 32-bit, most WinAPI functions use stdcall → watch for ret N (callee cleans stack).
• On 64-bit, there’s only one convention (Microsoft x64). If you see RCX/RDX/R8/R9 being set, you’re in usermode function call land.
• Shadow space is a Windows-only concept: if you see sub rsp, 20h before a call, that’s the 32-byte home space.
• Syscalls are usually hidden behind ntdll.dll stubs, not called directly.

---

Resources

---

Link	Description
1.17 - Calling Conventions (Linux)