Audit for binary IEEE 754-2008 compliance on relevant platforms

[auroranockert]

@thestinger filed #9987, and I thought we should get the ball rolling on documenting and testing IEEE 754-2008 compliance for Rust on non-embedded platforms.

I don't think that there isn't any formal decision that we should implement IEEE 754-2008 but it would really make sense for many applications of Rust.

• Scientific applications
• Games with synchronized or reproducible simulations (multiplayer, recordings)
• JS interpreters
• Anything that needs to be portable

The (binary part of the) standard covers

• Floating-point data format (for interchange and for computation)
• Basic operations (add, sub, mul, div, fma, sqrt, compare, &c.)
• Integer to floating-point conversion
• Floating-point to floating-point conversion
• Floating-point to string conversion
• Floating-point exceptions and handling (NaNs, exceptions, flags, &c.)

So all of this would need tests, and documentation. And my plan is to compile a suite of tests by slowly going through the standard page-by-page and write the basic conformance tests, then do another (even slower pass) that compiles another list of accuracy tests.

@bjz, @thestinger, @pcwalton, @catamorphism, @graydon and anyone interested: Any inputs on this? Is it a good idea at this point in time?

[auroranockert]

Some clarification on IRC, I don't want to take on the gargantuan task of implementing cross-platform implementations of the stuff that fails, yet. (sqrt and fma is the things I think may fail to be accurate / have wrong NaN handling on a few platforms)

dbaupp > jensnockert: that includes compensating for deficiencies/inaccuracies in LLVM/libc, right?
jensnockert > dbaupp: The idea is to find the deficiencies, another bug would be to fix them.
jensnockert > dbaupp: My idea is to build up a test suite.
jensnockert > So we know what is broken on what platforms.
dbaupp > jensnockert: yup; I meant, the goal would be full ieee compliance, not just saying "the platform is broken so we're broken too"
jensnockert > dbaupp: In the long run, that is what I want, yes.
dbaupp agrees
jensnockert > dbaupp: If we want to be able to support cross-platform reproducible floating-point, we need to do it. (And since a lot of people want to write games in Rust, that could be a killer feature)
dbaupp > jensnockert: apparently graydon was on some of the ieee floating point commities (random fact of the day :P )
dbaupp > jensnockert: yeah, having "guaranteed" reproducability across all platforms would be very nice
jensnockert > dbaupp: Yeah, he helped write IEEE 754-2008 apparently.
jensnockert > This probably means that Graydon is even more of my idol now.
dbaupp > jensnockert: heh

[thestinger]

I think the major deficiency in LLVM is the current lack of sNaN support by many of the instructions/intrinsics.

[auroranockert]

I agree, and the platforms affected shouldn't be too hard to find once we have the tests.

[thestinger]

The LLVM limitations are now part of issue #10186. #10185 and #10184 are relevant too.

[flaper87]

Triage bump. Still relevant, nothing to add.

[steveklabnik]

Triage bump: no change

[steveklabnik]

Triage: no change

[workingjubilee]

The current version, IEEE 754-2019, was published in July 2019.[1] It is a minor revision of the previous version, incorporating mainly clarifications, defect fixes and new recommended operations.

Should this be updated to audit against the new spec?

[workingjubilee]

Also: We definitely cannot obtain binary compliance currently on many platforms, even those that are not "embedded", without extensively patching our operations there, because there are platforms that have nonstandard behavior, even if it's just for the sole reason of "their silicon literally predates the standard of IEEE754-2008". So LLVM's problems are not necessarily the only gateway to pass.

For aarch64, x86-64, ppc64le, and a few other targets, this is comparatively minor as a concern.

[workingjubilee]

Some problems and possible solutions:

• Pre-IEEE754-2008 MIPS machines use a different sNaN/qNaN representation.
  • Don't care / flip the NaN bit if relevant?
• 32-bit x86 uses extended precision floats and parameter passing can trigger double rounding
  • Disable the x87 FPU entirely and use SSE2 for float parameter passing in Rust (only on certain x86 machines)
  • Always use soft float on x86 platforms without SSE2
  • Drop floats on x86 without SSE2
  • Define "fast math" semantics which allows double rounding (but not IEEE754 compliant)
• Some systems (e.g. Armv7 Neon registers) allow flushing denormals to zero, controlled by an FPCSR
  • Default to always setting the "don't flush denormals" bit where possible in Rust code (can be slower)
  • Define "fast math" semantics which allows denormal flushing (but not IEEE754 compliant)

On x86_64 with SSE2 and aarch64 with Neon, our f32 and f64 behave more-or-less "as expected" at runtime, barring LLVM compilation oddities.