posix/test.zig: "sigset_t bits" test is flaky

[rootbeer]

Zig Version

0.15.0-dev.911+31bc6d5a9

Steps to Reproduce and Observed Behavior

Andy has seen flaky failures in the "sigset_t bits" posix test:

expected 1, found 0
1793/2920 posix.test.test.sigset_t bits...FAIL (TestExpectedEqual)
/home/andy/dev/zig/lib/std/testing.zig:109:17: 0x1f34c4d in expectEqualInner__anon_46282 (test)
                return error.TestExpectedEqual;
                ^
/home/andy/dev/zig/lib/std/posix/test.zig:1057:17: 0x2ee9c3f in test.sigset_t bits (test)
                try testing.expectEqual(1, S.handler_called_count);
                ^

The test is "native on my x86_64 host" and "it's rare; I've seen it twice in the last week across hundreds of runs".

Current theories are:

• signal delivery is delayed
• write to S.handler_called_count isn't visible after handler interrupts code
• some other signal handlers (child processes?) are creating conflicts with this test

Expected Behavior

Test is robust and doesn't flake.

[rootbeer]

I believe I've figured out the flakiness here (at least one source). This test case uses posix.system.getpid() to send a signal to the current thread and it uses sigprocmask calls to manipulate (the per-thread) signal masks. Mostly this test executes on the main thread, where the pid and the tid are the same, and thus the test passes. But, very rarely Zig runs this test on a different thread from the main thread and then the signal is sent to the wrong thread (where the signal isn't blocked) and the test fails.

Thankfully, when run under strace the timing is inverted for whatever reason, and the background thread runs the test more often.

I'm not exactly sure where the thread is coming from, but I assume this is expected and Zig tests should never assume they're running in the main thread. The POSIX tests like this one that depend on process-wide mutable state (CWD, signal handlers, fork, signal masks, process-level signals, etc) probably should not be driven from Zig tests, and should probably be run as "standalone" style tests. I'll look into moving this test (and some of the others like it) over there.

I am a bit surprised at how rare this failure is in the current implementation (it never hits in CI? And I had to run 10000+ iterations of test-std filtered to just this test to reproduce the failure.) Should something be changed so the non-main thread runs tests more often (or always)?