Rust ABI uses memory instead of registers for SIMD and similar types

[nazar-pc]

I tried this code:

https://rust.godbolt.org/z/Yrr6h6ebq

I expected to see this happen: Rust ABI to be similar to C ABI in returning SIMD types via registers.

Instead, this happened:

On x86-64 Rust ABI returns native SIMD types like __m128i using memory pointer instead of SIMD registers on both Linux (where C and sysv64 use registers instead) and Windows (where C ABI, unfortunately, uses memory too).

On aarch64 a similar issue happens with both native SIMD types like float64x2_t and float aggregates like #[repr(C)] struct(f64, f64, f64), where C ABI uses registers and Rust ABI uses memory instead.

Similar issues exist for arguments too as shown in the reproduction.

While Rust ABI is unspecified and is free to do that, in performance-sensitive situations like tail call interpreters with every I/O register accounted for, the default ABI causes a major regression that would have been nice to not have. At least on platforms where C ABI has the expected behavior.

It is especially bad on x86-64 Linux where using all 6 GPRs for input arguments + SIMD for return value causes both return value and some input arguments to spill into memory.

I didn't find an existing identical issue and not sure if this should have been two or even three separate issues instead, so let me know if anything needs to change here.

Meta

rustc --version --verbose:

rustc 1.100.0-nightly (8fa1c96cf 2026-08-17)
binary: rustc
commit-hash: 8fa1c96cfd489e4c27654c144ae871ce2c4db6c6
commit-date: 2026-08-17
host: x86_64-unknown-linux-gnu
release: 1.100.0-nightly
LLVM version: 23.1.0

[ais523]

I'm pretty sure that doing this is intentional in order to avoid undefined behavior when linking code compiled for two different CPUs: see #44367 and #64609.

Imagine that you're compiling crate X for your native CPU, but its dependency crate Y was compiled for a generic CPU. If SIMD types were stored in registers, then crate X would attempt to store them in registers when calling functions defined in crate Y. However, being compiled for a generic CPU (which might not have the same set of vector registers), crate Y wouldn't actually have generated code to read or write the registers in question (it couldn't do this soundly because they might not exist), so it would end up reading garbage memory rather than the arguments that crate X had stored into the registers.

Fixing the problem would likely involve changing the Rust ABI based on target features, and also having crates somehow detect which specific version of the ABI their dependencies had used and adapting for them. This is quite difficult in practice (e.g. what if crate X is trying to convert a function defined in crate Y into a function pointer, and pass that as an argument to a function defined in crate Z?). The workaround of passing vector arguments in memory is much simpler, because any CPU can read them from there, regardless of what vector registers it might or might not have. (It doesn't, however, work for interoperability with C; that particular unsoundness is fixed by preventing the use of vectors in FFI (#116558).)

It would sometimes be possible to use vector registers internally within a single crate; this was attempted in the past, but ended up being reverted (#141309) because it ended up confusing LLVM's inliner too much and it actually ended up generating worse code than the in-memory version (#139029).

[nazar-pc]

That explanation makes sense abstractly and I did read a few issues related to features causing ABI issues (soft float stuff and such), but I'm not sure it is applicable here.

__m128i is mandatory for any x86-64 CPU to support, similarly to float64x2_t on aarch64. So regardless of what features you use, you can rely on them not being disabled unless I am missing something here. And using __m256i clearly indicates you're targeting AVX2 CPU.

Not using AVX2 registers for arguments, but crashing inside the function on AVX2 instructions anyway makes little sense to me in that case. It would make more sense with core::simd::Simd (which I don't think is possible to convince compiler to pass in vector registers), but not for concrete architecture-specific registers.

And C ABI already supports all that and you could link C libraries that are compiled with different features enabled, including into Rust binary, so I'm not sure what problem is Rust solving by intentionally not passing SIMD types in SIMD registers (if that is the case and not something that simple wasn't implemented yet).

[bjorn3]

__m128i is mandatory for any x86-64 CPU to support, similarly to float64x2_t on aarch64.

There are x86_64 targets with all simd support disabled. Though on the targets where it is unconditionally enabled, we could in principle pass __m128i in a register. We have a flag to pass vector types directly, but this doesn't currently support distinguishing between different vector sizes, so we did have to add support for that before we could use it for x86_64 as otherwise AVX types would also incorrectly be passed in registers.

And using __m256i clearly indicates you're targeting AVX2 CPU.

That doesn't mean that the function that called it actually has AVX2 support enabled.

// INVARIANT: The Avx variant is only created on systems with AVX support.
enum SimdAccumulator {
    Sse(__m128i),
    Avx(__m256i),
}

impl SimdAccumulator {
    fn add(&mut self, data: &[u64]) {
        match self {
            SimdAccumulator::Sse(ref mut acc) => unsafe { *acc = sse_acc(*acc, data) },
            // This call happens in a function without AVX support enabled, so if
            // __m256i were passed in registers, there would be no way to call it.
            // LLVM doesn't support enabling vector register usage on individual
            // call sites, only on whole functions.
            SimdAccumulator::Avx(ref mut acc) => unsafe { *acc = avx_acc(*acc, data) },
        }
    }
}

#[target_feature(enable = "avx")]
fn avx_acc(acc: __m256i, data: &[u64]) -> __m256i {
    todo!()
}

And C ABI already supports all that and you could link C libraries that are compiled with different features enabled, including into Rust binary, so I'm not sure what problem is Rust solving by intentionally not passing SIMD types in SIMD registers (if that is the case and not something that simple wasn't implemented yet).

That the C ABI is unsound, is not an argument for also making the Rust ABI unsound. We try to avoid the mistakes C has made where possible.

[nazar-pc]

That doesn't mean that the function that called it actually has AVX2 support enabled.

From my testing with extern "sysv64" only simple single-type return values are returned via registers (including transparent wrappers). That enum has a discriminant, which immediately disqualifies direct SIMD register usage. The same happens with struct { a: __m128i, b: __m128i } for some reason.

So I was only talking about cases where only a single SIMD type is returned on x86-64 to be at least as good as C ABI where there is no ambiguity in whether it is AVX-only function or not.

[bjorn3]

That enum has a discriminant, which immediately disqualifies direct SIMD register usage.

I'm not talking about the enum. I'm talking about the argument that is passed to avx_acc, which is a plain __m256i. SimdAccumulator::add can't pass __m256i arguments in registers as SimdAccumulator::add doesn't have AVX support enabled. LLVM simply doesn't allow using AVX registers at all when AVX is disabled for the caller, not even for calling a function that does have AVX enabled.

Also in the presence of function pointers it wouldn't even be possible to know if the callee has AVX supported in the first place, so for function pointers to work, the calling convention must not depend on on callee target features, which means the calling convention has to match what a callee without any non-default target features enabled would use as calling convention, aka using AVX registers is not possible.

fn sse(a: __m256i) {}
#[target_feature(enable = "avx")]
fn avx(a: __m256i) {}

let val = unsafe { std::mem::transmute([0u8; 32]) };
let f = if is_x86_feature_detected!("avx") { avx as unsafe fn(__m256i) } else { sse as _ };
unsafe { f(val) }; // We don't know if the callee has AVX support enabled, so __m256i can't be passed in a register.

[nazar-pc]

Interesting, but isn't having one of the arguments or return type being plain __m256i or transparent new type over it a sufficient reason to assume the function can only be called with AVX support available without extra annotations?

I mean there should be no way for it to be usable on CPU that physically doesn't support such a SIMD width and instructions that come with it.

[bjorn3]

Interesting, but isn't having one of the arguments or return type being plain __m256i or transparent new type over it a sufficient reason to assume the function can only be called with AVX support available without extra annotations?

Perhaps in an alternative timeline. But alas, it was chosen not to restrict these types to functions that enable certain target features. And you can both construct and use __m256i without AVX just fine by transmuting it.