Skip to content

cg_clif: f128-to-small-int cast uses to_ty instead of ret_ty for lib_call return ABI #450

Description

@SebTardif

Bug

In compiler/rustc_codegen_cranelift/src/codegen_f16_f128.rs:253-254, when converting f128 to I8 or I16, the code correctly computes ret_ty = I32 (since the C library functions __fixtfsi/__fixunstfsi return i32 per the compiler-rt ABI), but then passes to_ty (I8/I16) to lib_call instead of ret_ty:

let ret_ty = if to_ty.bits() < 32 { types::I32 } else { to_ty };
// ...
fx.lib_call(&name, vec![AbiParam::new(from_ty)], vec![AbiParam::new(to_ty)], &[from])
//                                                                   ^^^^^ should be ret_ty

The lib_call declares the function returns I8/I16, but the actual C function returns i32. This ABI mismatch can cause incorrect values or undefined behavior.

Fix

Change to_ty to ret_ty in the AbiParam::new() call.

Impact

f128-to-i8 and f128-to-i16 casts under the Cranelift backend produce calls with wrong return type ABI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-codegenArea: code generationI-wrongWrong result or data corruptionP-mediumMedium impact: affects specific usage patternsbugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions