feat(agent-core-v2): gate image formats and add media recovery resends - #1626
Merged
Conversation
- remove ping interval / pong timeout / socket.terminate() from both the
v1 (WsConnectionV1) and v2 (WsConnection) WebSocket connections
- v2 protocol: drop pong from the client message schema, remove PingMessage
and ReadyMessage.heartbeatMs; the ready frame is now bare { type: 'ready' }
- v1 protocol: remove buildPing/PingFrame and ServerHelloPayload.heartbeat_ms;
server_hello no longer advertises a heartbeat
- drop pingIntervalMs/pongTimeoutMs options from registerWs/registerWsV1
- add image-format-policy as the single source of truth for the provider-accepted image MIME set (PNG/JPEG/GIF/WebP), with MIME normalization, data-URL parsing, byte sniffing, and refusal notices - enforce the format gate at every ingestion point: ReadMediaFile refuses unsupported formats with per-OS conversion guidance, MCP tool results and prompt step requests drop them for a text notice, and kap-server prompt routes gate inline, uploaded, and remote-URL images on the sniffed bytes - resend once with every media part replaced by a text marker when the provider rejects an image's format, and keep later steps of the same turn on the media-stripped projection (v1 parity) - commit the WebP decoder wasm as a base64 module for the bundled CLI, with a regenerate script
- resend once with the media-degraded projection after an HTTP 413 body-size rejection: all but the two most recent media parts are replaced by text markers, and later steps of the turn stay degraded (stripped still wins over degraded) - generalize stripAllMediaParts into degradeOlderMediaParts; the media-stripped resend is now the keepRecent=0 case - re-encode non-animated WebP through the wasm decoder and the PNG/JPEG ladder instead of passing oversized WebP through (animated WebP still passes through whole) - stop lossless PNG rescaling at a 1000px floor and switch to the JPEG ladder below it, so small byte budgets stay readable - add the @jsquash/webp dependency
- add the faultInjection domain: a one-shot arm/take latch that raises a deterministic provider failure (HTTP 413 body-size or image-format 400) before the provider is contacted, so the media-degraded / media-stripped recovery resends can be exercised end-to-end against a real provider - gate arming behind the new fault-injection experimental flag (KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION), off by default - expose IAgentPromptService and IFaultInjectionService over the kap-server /api/v2 RPC channel - add a klient example that drives the REST ingestion gate and both recovery resends against a live server
🦋 Changeset detectedLatest commit: 5b0e014 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b0e01437b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
132
to
137
| buildServerHello({ | ||
| ws_connection_id: this.id, | ||
| protocol_version: WS_PROTOCOL_VERSION, | ||
| heartbeat_ms: this.pingIntervalMs, | ||
| max_event_buffer_size: this.maxBufferSize, | ||
| capabilities: { event_batching: false, compression: false }, | ||
| }), |
There was a problem hiding this comment.
When a /api/v1/ws client validates the initial server_hello against the published v1 protocol, this frame is now missing payload.heartbeat_ms; packages/protocol/src/ws-control.ts still requires heartbeat_ms in serverHelloPayloadSchema, so such clients reject the handshake before subscribing. Either keep emitting a positive heartbeat value while disabling pings, or update the shared protocol/client schemas in the same change.
Useful? React with 👍 / 👎.
Merged
7723qqq
pushed a commit
to 7723qqq/kimi-code
that referenced
this pull request
Jul 15, 2026
MoonshotAI#1626) * feat(kap-server): drop WS heartbeat; never terminate idle connections - remove ping interval / pong timeout / socket.terminate() from both the v1 (WsConnectionV1) and v2 (WsConnection) WebSocket connections - v2 protocol: drop pong from the client message schema, remove PingMessage and ReadyMessage.heartbeatMs; the ready frame is now bare { type: 'ready' } - v1 protocol: remove buildPing/PingFrame and ServerHelloPayload.heartbeat_ms; server_hello no longer advertises a heartbeat - drop pingIntervalMs/pongTimeoutMs options from registerWs/registerWsV1 * feat(agent-core-v2): gate image formats and add media-stripped resend - add image-format-policy as the single source of truth for the provider-accepted image MIME set (PNG/JPEG/GIF/WebP), with MIME normalization, data-URL parsing, byte sniffing, and refusal notices - enforce the format gate at every ingestion point: ReadMediaFile refuses unsupported formats with per-OS conversion guidance, MCP tool results and prompt step requests drop them for a text notice, and kap-server prompt routes gate inline, uploaded, and remote-URL images on the sniffed bytes - resend once with every media part replaced by a text marker when the provider rejects an image's format, and keep later steps of the same turn on the media-stripped projection (v1 parity) - commit the WebP decoder wasm as a base64 module for the bundled CLI, with a regenerate script * feat(agent-core-v2): add media-degraded 413 resend and WebP re-encoding - resend once with the media-degraded projection after an HTTP 413 body-size rejection: all but the two most recent media parts are replaced by text markers, and later steps of the turn stay degraded (stripped still wins over degraded) - generalize stripAllMediaParts into degradeOlderMediaParts; the media-stripped resend is now the keepRecent=0 case - re-encode non-animated WebP through the wasm decoder and the PNG/JPEG ladder instead of passing oversized WebP through (animated WebP still passes through whole) - stop lossless PNG rescaling at a 1000px floor and switch to the JPEG ladder below it, so small byte budgets stay readable - add the @jsquash/webp dependency * feat(agent-core-v2): add flag-gated fault injection for recovery testing - add the faultInjection domain: a one-shot arm/take latch that raises a deterministic provider failure (HTTP 413 body-size or image-format 400) before the provider is contacted, so the media-degraded / media-stripped recovery resends can be exercised end-to-end against a real provider - gate arming behind the new fault-injection experimental flag (KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION), off by default - expose IAgentPromptService and IFaultInjectionService over the kap-server /api/v2 RPC channel - add a klient example that drives the REST ingestion gate and both recovery resends against a live server
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
No linked issue — the problems are described below.
Problem
In the v2 engine, images in formats providers reject (AVIF, HEIC, BMP, TIFF, ICO) could enter the session history through any ingestion point — ReadMediaFile, MCP tool results, or REST prompt uploads. Because the history is re-sent on every request, one such image makes every subsequent request fail: the session is poisoned with no way to recover. Image-heavy sessions hit a second wall: HTTP 413 body-size rejections from accumulated base64 media, which token-driven compaction never recovers from (media is estimated at a small flat cost). Oversized WebP images were also passed through untouched because the default jimp build ships no WebP codec. Finally, none of these recovery paths could be exercised end-to-end, since a real provider cannot be asked to produce a specific rejection on demand.
Separately, kap-server's WebSocket ping/pong heartbeat could terminate connections that were merely idle, dropping healthy long-lived sessions.
What changed
Image-format gate (single source of truth). A new
image-format-policymodule defines the provider-accepted image set (PNG/JPEG/GIF/WebP) with MIME normalization, data-URL parsing, magic-byte sniffing (bytes are authoritative over declared labels), and URL-extension heuristics. Every ingestion point enforces it: ReadMediaFile refuses unsupported formats with per-OS conversion instructions the model can run; MCP tool results and prompt step requests drop them for a text notice; kap-server's REST prompt route gates inline, uploaded, and remote-URL images on the sniffed bytes.Recovery resends. When the provider still rejects a request, the requester resends once with a read-side projection rebuild: media-stripped (every media part replaced by a text marker) after an image-format 400, and media-degraded (all but the two most recent media parts replaced) after an HTTP 413. Once a step only succeeds via a recovery resend, later steps of the same turn build from that projection directly — v1 parity.
WebP re-encoding. The
@jsquash/webpdecoder wasm is committed as a base64 module (the bundled CLI has no on-disk assets), so non-animated WebP now re-encodes through the PNG/JPEG ladder instead of passing through; animated WebP still passes through whole. Downscaled PNGs switch to the JPEG ladder below a 1000px floor, keeping small byte budgets readable.Fault injection + e2e coverage. A new experimental
fault-injectionflag (KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION, off by default) allows arming a one-shot deterministic provider failure (413 or image-format 400) on the next LLM request, so the recovery resends can be exercised against a real provider.IAgentPromptServiceandIFaultInjectionServiceare exposed over the kap-server/api/v2RPC channel, and a klient example (examples/media-recovery.ts) drives the ingestion gate and both recovery resends against a live server.WebSocket heartbeat removal. kap-server no longer runs a ping/pong heartbeat and never terminates idle connections, on both the v1 and v2 WS protocols; liveness is left to TCP and the client.
Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.