fix(security): stop exception messages reaching ops/system-agent responses (#1917) - #1959
Merged
Conversation
…onses (#1917) PR #1912 fixed `restart_fleet` after CodeQL flagged py/stack-trace-exposure, and deliberately left the sibling sites alone under the minimal-changes rule. This closes them with the same pattern — response carries platform-authored text or the exception CLASS NAME; the full message + traceback go to the backend log via exc_info=True. Sites (located by content, not the issue's line numbers — dev has moved): routers/ops.py fleet health probe, stop_fleet per-agent, _stop_agent_container (feeds emergency_stop), ops costs routers/system_agent.py health probe (open CodeQL alert #231), workspace cleanup, template copy, the re-initialize and restart 500s, and the terminal WebSocket error frame routers/agents.py start / stop / logs HTTPException details The health probe had `str(e)[:50]`. Truncation bounds a leak, it does not remove one: docker and httpx messages lead with the host, socket path or URL, so the first 50 characters are exactly the sensitive part. Two sites were not in the issue and are the same defect, found by grepping the named files rather than trusting the list: system_agent's re-initialize/restart 500 details, and its terminal WebSocket error frame — the only one here that reaches a browser rather than an admin API client. Guards: 8 tests, all 8 failing against pre-fix code. Each drives the real router function with a sentinel in the exception message and asserts BOTH that the sentinel is gone and that the class name survives — a response that dropped the error entirely would pass a sentinel-only check while making the failure undiagnosable. Plus a static ban on str(e) in the two routers where every occurrence was this defect (agents.py is deliberately excluded: it has many str(e) uses that never reach a response, so a blanket ban there would be false). The fleet-health test initially patched a `get_agent_context_info` that does not exist — `raising=False` made that silent, the probe never raised, and the test passed against the UNPATCHED router. The pre-fix run is what caught it; it now patches `get_agent_client` and asserts the probe actually ran. Same vacuous-negative class as #1932/#1951. Related to #1917 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dolho
force-pushed
the
fix/1917-stack-trace-exposure
branch
from
August 3, 2026 11:00
dd9562e to
09b5b70
Compare
vybe
approved these changes
Aug 3, 2026
vybe
left a comment
Contributor
There was a problem hiding this comment.
Validated via /validate-pr. Closes #1917 (closing keyword added so the issue auto-promotes to status-in-dev).
Security fix, no schema/API/config surface, no new top-level backend module, no secrets in the diff. Three things I checked specifically and liked:
- Sites located by content, not the issue's stale line numbers — and two sites that weren't in the issue (system_agent's re-init/restart 500 details, and the terminal WS error frame, the only one reaching a browser) were found by grepping the named files rather than trusting the list.
str(e)[:50]correctly treated as a leak, not a mitigation — docker/httpx messages lead with the socket path or internal host, so the first 50 chars are the sensitive part.- Both-halves assertion (
SENTINEL not in bodyAND"RuntimeError" in body) — a response that dropped the error entirely would pass a sentinel-only check while making the failure undebuggable. The disclosed vacuous first draft (patching aget_agent_context_infothat doesn't exist, silent underraising=False) is exactly the class worth catching, and 8/8 now fail pre-fix.
The static str(e) ban scoped to the two routers where every occurrence was this defect — with agents.py deliberately excluded and given targeted window checks instead — is the right call over a blanket rule that would be false there.
CodeQL alert #231 can only close on the next main analysis; noted, not blocking.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PR #1912 fixed
restart_fleetafter CodeQL flaggedpy/stack-trace-exposure, and deliberately left the sibling sites alone under the minimal-changes rule. This closes them with the same pattern: the response carries platform-authored text or the exception class name; the full message + traceback go to the backend log viaexc_info=True.Exposure is bounded — every endpoint here is admin-gated — but exception messages routinely embed internals. Docker and httpx errors lead with the socket path or the internal container host, and
learnings.md(2026-07-14) records git stderr carrying a PAT into operator-visible state. "Admin-only" is a blast-radius argument, not a reason for the string to be there.Sites
Located by content, not the issue's line numbers —
devhas moved since it was filed, and every cited line had drifted.routers/ops.pystop_fleetper-agent ·_stop_agent_container(feedsemergency_stop) · ops costsrouters/system_agent.pyrouters/agents.pyHTTPExceptiondetailsThe health probe had
str(e)[:50]. Truncation bounds a leak, it doesn't remove one — docker/httpx messages lead with the host, socket path or URL, so the first 50 characters are the sensitive part.Two sites weren't in the issue and are the same defect, found by grepping the named files rather than trusting the list: system_agent's re-initialize/restart 500 details, and its terminal WebSocket error frame — the only one here that reaches a browser rather than an admin API client.
Guards — 8 tests, all 8 failing against pre-fix code
Each drives the real router function with a sentinel planted in the exception message, and asserts both halves:
The second assertion matters: a response that dropped the error entirely would pass a sentinel-only check while making the failure impossible to debug.
Plus a static ban on
str(e)in the two routers where every occurrence was this defect.agents.pyis deliberately excluded from that ban — it's a 1000+ line router with manystr(e)uses that never reach a response, so a blanket rule there would be false; its three sites get a targeted window check instead.One test was vacuous, and the pre-fix run caught it
My first version of the fleet-health test patched a
get_agent_context_infothat does not exist.raising=Falsemade that silent, the probe never raised, and the test passed against the unpatched router — 7 of 8 failing pre-fix instead of 8. It now patches the real seam (get_agent_client(...).get_session()) and assertsawait_count == 1, so it cannot pass without the probe running. Same vacuous-negative class as #1932/#1951.Verification
Acceptance criteria
exc_info=Truemainanalysis — cannot be verified from a PR branch; alert Vectorized Episodic Memory #39 closes on its own once fix: fleet restart adopts rebuilt base images through the canonical lifecycle path (#1860) #1912 ships in a release (the issue says verify, don't re-fix — I have not touchedrestart_fleet)Closes #1917
🤖 Generated with Claude Code