Skip to content

fix(retry): let streaming requests reach the retry loop - #47

Open
shoemoney wants to merge 1 commit into
MiniMax-AI:mainfrom
shoemoney:fix/stream-retry
Open

fix(retry): let streaming requests reach the retry loop#47
shoemoney wants to merge 1 commit into
MiniMax-AI:mainfrom
shoemoney:fix/stream-retry

Conversation

@shoemoney

@shoemoney shoemoney commented Aug 5, 2026

Copy link
Copy Markdown

send_request retries up to --retries times (default 10), but streaming requests only ever
get one attempt.

_handle_stream_request wraps its whole body in a try and converts every exception into a normal
return value:

# verify.py:355-357
        except Exception as e:
            logger.error(f"Stream request failed: {e}")
            return "failed", {"error": str(e)}

Because verify.py:259 is a return, that value goes straight back out of send_request and the
except on line 263 never sees anything. The loop exits on its first iteration.

--stream was effectively dead code until #14 wired req["stream"] = True at verify.py:212-213,
which is why this hasn't bitten before.

Measured

Driving the real ValidatorRunner.send_request against a local server that always fails, counting
HTTP requests that actually arrive:

scenario HTTP attempts before after
HTTP 500, --retries=10, SDK retries off, non-stream 10 10
HTTP 500, --retries=10, SDK retries off, --stream 1 10
connection dropped mid-stream, --retries=10, non-stream 10 10
connection dropped mid-stream, --retries=10, --stream 1 10
HTTP 500, --retries=3, SDK defaults (real-world config), non-stream 12 12
HTTP 500, --retries=3, SDK defaults (real-world config), --stream 4 12

The SDK's own max_retries (set at verify.py:95) is a different layer — it applies to both modes
equally, and it cannot help the mid-stream case at all, since the 200 has already landed.

Why it matters beyond the retry count

all_request_count (verify.py:255) counts retry attempts and is reported as all_count
(verify.py:484), which scripts/calculate_batch_metrics.py:255 uses as the denominator of
Query-Success-Rate.

Since stream mode never retries, its all_count equals its request count, while non-stream inflates
on a flaky provider. The same provider scores a better Query-Success-Rate under --stream than
without it
— so the numbers aren't comparable across modes today.

The change

         except Exception as e:
             logger.error(f"Stream request failed: {e}")
-            return "failed", {"error": str(e)}
+            # Re-raise so send_request's retry loop can see the failure. Returning
+            # here would exit the loop after a single attempt.
+            raise

raise rather than deleting the try — it keeps the stream-specific log line, and dropping the
block would mean dedenting ~75 lines for no behavioural gain. The final-failure return shape is
unchanged: send_request already returns "failed", {"error": str(last_error)} at line 276.

Not included, deliberately

Spotted while tracing this, left alone: verify.py:612's help text says
"Number of retries on failure (default: 3)" while verify.py:611 is default=10. Separate
concern, happy to send it as its own one-word PR.

No test. There is no unit-test suite here — m3_format_check/ needs a live provider endpoint — and
adding test infrastructure isn't this PR's concern. The reproduction above is a standalone script
driving the real code path against a local failing server; happy to contribute it, or a proper test,
if you'd like the harness.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

_handle_stream_request converted every exception into a normal
("failed", ...) return value, so send_request's except clause never
fired for a streaming request and the loop exited after one attempt.
Re-raise instead; the final-failure return shape is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant