Skip to content

[codex] reuse encoded Responses request bodies - #28324

Closed
jif-oai wants to merge 1 commit into
openai:mainfrom
jif-oai:jif/reuse-http-request-bytes
Closed

[codex] reuse encoded Responses request bodies#28324
jif-oai wants to merge 1 commit into
openai:mainfrom
jif-oai:jif/reuse-http-request-bytes

Conversation

@jif-oai

@jif-oai jif-oai commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Why

Responses HTTP requests were converted from ResponsesApiRequest into a full serde_json::Value. EndpointSession then deep-cloned that value for each retry, and the transport serialized and compressed it again before every send.

Large histories make those copies expensive. Retry attempts should reuse the same immutable request bytes.

What

  • Serialize standard Responses requests directly into a ref-counted EncodedJsonBody.
  • Preserve the Azure path that attaches item IDs before encoding.
  • Prepare JSON, compression, and derived content headers once before the retry loop.
  • Clone the prepared request per attempt so body clones only bump the Bytes reference count.
  • Keep auth inside the retry loop. Signing auth sees the exact final headers and body bytes that the transport sends.
  • Preserve request-body TRACE output. With TRACE plus compression, retain the original JSON bytes for logging; normal requests keep only the final wire bytes.
  • Leave non-Responses endpoint bodies on the existing Value path.

Performance

A temporary release-mode measurement used a 10 MiB JSON body and 10 retry preparations:

  • old Value clone + serialize path: 30 ms total
  • prepared shared-byte path: less than 1 ms total

That is about 3 ms avoided per retry for this payload on the test machine. Each retry also stops allocating another request-sized JSON tree and serialized buffer. Without TRACE, compressed requests retain only the final compressed wire bytes.

Validation

  • just test -p codex-client — 28 passed
  • just test -p codex-api — 125 passed
  • just fix -p codex-client
  • just fix -p codex-api

@jif-oai

jif-oai commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Replaced by #28327, opened directly from an openai/codex branch.

@jif-oai jif-oai closed this Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant