Fixed nontermination issue in par operators

[djspiewak]

Fixes #2238

This is a relatively high-priority fix, tbh, since par operations could be non-terminating in 3.2.2 with the current implementation. The performance impact is pretty real though:

Before:

[info] Benchmark                      (cpuTokens)  (size)   Mode  Cnt    Score    Error  Units
[info] ParallelBenchmark.parTraverse        10000    1000  thrpt    5  644.762 ± 49.298  ops/s
[info] ParallelBenchmark.traverse           10000    1000  thrpt    5  236.523 ±  9.855  ops/s

After

[info] Benchmark                      (cpuTokens)  (size)   Mode  Cnt    Score    Error  Units
[info] ParallelBenchmark.parTraverse        10000    1000  thrpt    5  596.967 ± 20.901  ops/s
[info] ParallelBenchmark.traverse           10000    1000  thrpt    5  237.569 ± 10.316  ops/s

So about 7.5% slower in a naively relative comparison. A better comparison is to look at the overhead costs by measuring relative to an idealized traverse. I have 16 physical threads, meaning that an (impossible) idealized overhead-free parTraverse implementation would get about 3784.368 ops/sec on the first run and 3801.104 ops/sec on the second. The actual throughput was 644.762 and 596.967, respectively, meaning the overhead accounts for 83.0% of the runtime before this change and 84.3% after, which is an increase of about 1.6%. That's unfortunate (also it sucks that the overhead is so high), but it's generally within the bounds of sanity.

In theory, an alternative implementation using two Deferreds (one for each fiber) and guaranteeCase within the body of each fiber would probably be slightly faster, but it would also require some more complex dynamic dispatch machinery similar to the original proposal in #2155. It's probably worth measuring that alternative implementation to see if it's a meaningful improvement.

[vasilmkd]

Another annoyance I just thought of is the fact that IO won't side-channel the second exception in the case that both spawned fibers fail.

[djspiewak]

Oh that's a good point. I'll think about how to resolve that.

[djspiewak]

Btw given the priority of this fix, I think we shouldn't hold it for the side channel fix. Let's file that and fix in a follow up.

[vasilmkd]

Yup, I'm just waiting for the CI.