`-Zregparm` doesn't put 64-bit arguments in registers

[sulix]

The -Zregparm option on 32-bit x86 should pass arguments in registers where possible, but for 64-bit arguments, it will push them onto the stack, whereas C compilers (both gcc and clang) will put them in a pair of registers.

For example, the following rust code:

unsafe extern "C" {
    pub fn test(s: i64) -> i64;
}

#[unsafe(no_mangle)]
pub fn f1() {
    unsafe {
        let r = test(42);
    }
}

compiles to (with -Zregparm=3):

f1:
        sub     esp, 20
        push    0
        push    42
        call    test
        add     esp, 28
        ret

Note the 64-bit value is put on the stack.

However, the equivalent C code:

extern long long test(long long s);

void f1(void)
{
    long long r = test(42);
}

compiles to (gcc 15.2 with -mregparm=3, but clang produces something similar):

f1():
        push    ebp
        mov     ebp, esp
        sub     esp, 24
        mov     eax, 42
        mov     edx, 0
        call    test(long long)
        mov     DWORD PTR [ebp-16], eax
        mov     DWORD PTR [ebp-12], edx
        nop
        leave
        ret

Note that the 64-bit value is put in edx:eax.

This can cause crashes/incorrect results due to an ABI mismatch, particularly in Rust-for-Linux on 32-bit x86.

Note that 64-bit return values seem to be functioning correctly, and are placed in edx:eax.

Compiler explorer link with the above examples: https://godbolt.org/z/feb7addG7

Rust-for-Linux tracking issue for 32-bit x86: Rust-for-Linux/linux#78
Tracking issue for -Zregparm: #131749

[ojeda]

Cc @azhogin.

[bjorn3]

Maybe the size <= 32 at

rust/compiler/rustc_target/src/callconv/x86.rs

Lines 186 to 188 in 57e620e

	if arg.layout.size.bits() <= 32 && unit.kind == RegKind::Integer {
	attrs.set(ArgAttribute::InReg);
	}

should be removed or replaced by size <= 64?

[sulix]

Looking at what clang does, it seems that regparm has a pretty different set of ABI rules to fastcall/vectorcall/regcall. In particular:

• Unlike those ABIs, regparm isn't limited to primitive types (pointer/int).
• The 32bit limit seems to be correct for fastcall/vectorcall/regcall, but not for regparm.

At the very least, -mregparm ought to pass small structs in registers (but only, afaict, if the whole struct fits, it won't split a struct up between registers and the stack). See: https://godbolt.org/z/EYTszP457

Of course, this seems to contradict the gcc docs, which state that regparm causes the:

compiler to pass arguments number one to number if they are of integral type

So, unless "integral type" includes structs (even with floating-point members), I suspect the documentation is being misleading here, and the fix should be more like

if opts.flavor != Flavor::FastcallOrVectorcall || arg.layout.size.bits() <= 32 && unit.kind == RegKind::Integer { 
    attrs.set(ArgAttribute::InReg); 
}

But I haven't tried testing fastcall or vectorcall, so don't take my word on that blindly.

[workingjubilee]

It seems we cannot have fastcall and regparm share as much logic as we thought.