1.26.0 - stdsimd/coresimd/x86/mod.rs test failures on non-x86 architectures

[infinity0]

Same failures on Debian and Fedora

test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m256d (line 32) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m128 (line 32) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m64 (line 39) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m256i (line 36) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m128i (line 39) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m128d (line 32) ... FAILED
test ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m256 (line 32) ... FAILED

Current master has diffs on top of 1.26.0 looking like this:

     /// ```
-    /// # #![feature(cfg_target_feature, target_feature, stdsimd)]
+    /// # #![cfg_attr(not(dox), feature(stdsimd))]
     /// # #![cfg_attr(not(dox), no_std)]

but on a first glance this doesn't look like it would fix the failures.

cc @cuviper

[cuviper]

I guess these need a target_arch config filter too?

[infinity0]

Is there a way to do that for the whole file, like the opposite of // ignore ?

[infinity0]

The error message says:

---- ../stdsimd/coresimd/x86/mod.rs - coresimd::x86::__m256d (line 32) stdout ----
	error: 
is_x86_feature_detected can only be used on x86 and x86_64 targets.
You can prevent it from being used in other architectures by
guarding it behind a cfg(target_arch) as follows:

    #[cfg(any(target_arch = "x86", target_arch = "x86_64"))] {
        if is_x86_feature_detected(...) { ... }
    }

So I'm going to try that and report back. Ironically this overly-strict macro is defined in the very same crate whose tests fail its strictness, in stdsimd/arch/detect/mod.rs.

What's curious is that we don't get the same errors for very similar tests defined in coresimd/x86/avx2.rs, which also use is_x86_feature_detected that are ungarded by a cfg(). There are other similarities and differences about these tests but none of them seem like they should be causing such a failure vs non-failure that we're seeing.

[infinity0]

cc @gnzlbg who last touched the detection macro

[gnzlbg]

I guess these need a target_arch config filter too?

The whole coresimd/x86 module has a cfg on the target_arch for x86 and x86_64 (https://github.com/rust-lang-nursery/stdsimd/blob/master/coresimd/mod.rs#L149).

Why is that module being compiled for your ppc builds? I don't know, our build bots for ppc, ppc64 (be), and ppc64le are not failing, but this doesn't help you much.

Maybe it has something to do with dox, but AFAIK dox is only defining when running cargo doc which does not run any tests. cc @QuietMisdreavus

[gnzlbg]

Digging into the logs, this is the command that fails

  Doc-tests core
     Running `/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/bootstrap/debug/rustdoc --test /<<BUILDDIR>>/rustc-1.26.0+dfsg1/src/libcore/lib.rs --crate-name core -L dependency=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/powerpc64le-unknown-linux-gnu/release/deps -L dependency=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/release/deps --extern core=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/powerpc64le-unknown-linux-gnu/release/deps/libcore-dfc03e5d45d15e08.rlib`
rustdoc command: "/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage2/bin/rustdoc" "--test" "/<<BUILDDIR>>/rustc-1.26.0+dfsg1/src/libcore/lib.rs" "--crate-name" "core" "-L" "dependency=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/powerpc64le-unknown-linux-gnu/release/deps" "-L" "dependency=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/release/deps" "--extern" "core=/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1-std/powerpc64le-unknown-linux-gnu/release/deps/libcore-dfc03e5d45d15e08.rlib" "--cfg" "stage1" "--cfg" "dox" "--sysroot" "/<<BUILDDIR>>/rustc-1.26.0+dfsg1/build/powerpc64le-unknown-linux-gnu/stage1" "-Z" "force-unstable-if-unmarked" "-Z" "unstable-options" "--crate-version" "1.26.0"

So it might well be that the current setup attempts to run all doc tests of all architectures on all architectures, but that was never going to work.

---

cc @alexcrichton : this looks like an issue with whitelisting those architectures with dox. The flag that we want here is one that imports those modules when generating documentation, but not when running documentation tests.

cc @GuillaumeGomez

[QuietMisdreavus]

The reason that dox flag is there is so that the standard library documentation shows information for as many platforms as it can at once, just like how std::os has information on both Linux and Windows. Rustdoc should avoid executing doctests if they're properly tagged with a #[doc(cfg(...))] attribute, as the public exports are. However, this makes me think that those tests are being run from the private modules, due to the path being shown in the failure text. Those aren't tagged with doc(cfg) because of how 64-bit chips extend the primitives of their 32-bit counterpart, so for example they're tagged as x86 or x86_64 based on where they're exported rather than where they're defined.

This makes me wonder whether we should move those attributes and use something like #[doc(cfg(any(target_arch = "x86", target_arch = "x86_64")))], which will add lots of noise to the docs but possibly make these tests run properly.

[gnzlbg]

However, this makes me think that those tests are being run from the private modules, due to the path being shown in the failure text. Those aren't tagged with doc(cfg) because of how 64-bit chips extend the primitives of their 32-bit counterpart, so for example they're tagged as x86 or x86_64 based on where they're exported rather than where they're defined.

Urgh :/

This makes me wonder whether we should move those attributes and use something like #[doc(cfg(any(target_arch = "x86", target_arch = "x86_64")))], which will add lots of noise to the docs but possibly make these tests run properly.

Let's just do that. There are only a bunch of those types.

[QuietMisdreavus]

It's possible that we can make the doc(cfg) handling a little more clever by reducing the condition if it gets something like all(a, any(a, b)) to just make it use a in that situation, and leave the doc(cfg) tags on the exports, which should leave the docs as-is, and also properly tag all the doctests! That sounds like it starts to lean into "CS research" territory tho. >_>

[QuietMisdreavus]

I've opened rust-lang/stdarch#466 to tag the private modules. We'll need to run doctests for libcore in a powerpc64 environment (or in Debian/Fedora's environment, if that doesn't work) to be sure it fixes the issue.

[infinity0]

Thanks, I've added this to Debian and uploaded, results will appear here in about 4-10 hours for version 1.26.0+dfsg1-1~exp4.