[codex] exec-server: stream files in chunks by pakrym-oai · Pull Request #28354 · openai/codex

pakrym-oai · 2026-06-15T17:49:18Z

Why

fs/readFile buffers the entire file in one response, which makes large remote reads expensive and prevents callers from applying backpressure. We need an opt-in streaming path with bounded block sizes while preserving the existing single-call API for small and sandboxed reads.

What changed

Add ExecServerClient::stream, returning a named FileReadStream that implements futures::Stream and yields immutable 1 MiB byte blocks.
Add internal fs/open, fs/readBlock, and fs/close RPCs. fs/readBlock accepts an explicit offset and length.
Keep unsandboxed files open between block reads, cap open handles per connection, and clean them up on EOF, error, stream drop, explicit close, or connection shutdown.
Reject platform-sandboxed streaming opens instead of turning the one-shot sandbox helper into a persistent server. Existing fs/readFile behavior is unchanged.

Testing

just test -p codex-exec-server
Integration coverage for 1 MiB chunking, exact block-boundary EOF, sandbox rejection, and continued reads from the opened file after path replacement.
Handle-manager coverage for non-sequential offsets, variable block lengths, the 128-handle limit, and capacity release after close.

pakrym-oai · 2026-06-15T18:16:38Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 741b2b9213

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

pakrym-oai · 2026-06-15T18:52:40Z

@codex review this

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ab7aba16a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…-full-ci

pakrym-oai · 2026-06-15T20:30:21Z

@codex review

chatgpt-codex-connector · 2026-06-15T20:34:05Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 701738caec

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

jif-oai

Happy to discuss those points but I like the direction that this took

jif-oai · 2026-06-16T08:56:17Z

+    ) -> FileSystemResult<tokio::fs::File> {
+        reject_sandbox_context(sandbox)?;
+        let path = path.to_abs_path()?;
+        if !tokio::fs::metadata(path.as_path()).await?.is_file() {


I guess you know it but this is still Windows unsafe right?
PathUri round-trips \\.\pipe\..., metadata() itself opens the path with default QOS, and is_file() does not prove FILE_TYPE_DISK
Thsi exposed to named-pipe impersonation etc

jif-oai · 2026-06-16T08:58:40Z

+                format!("file read handle `{handle_id}` already exists"),
+            ));
+        }
+        if handles.len() >= MAX_OPEN_FILE_READS {


Ultra nit: the 128-entries cap doesn’t bound memory because handleId is an unrestricted caller string. stdio has no message-size cap so a gigantic id remain resident
Since the client is using uuids anyway, we could simply enforce >= 16 bytes or something like this (even 32 if we want a margin)

jif-oai · 2026-06-16T09:20:28Z

+    ) -> FileSystemResult<tokio::fs::File> {
+        reject_sandbox_context(sandbox)?;
+        let path = path.to_abs_path()?;
+        if !tokio::fs::metadata(path.as_path()).await?.is_file() {


I also think there is something slightly racy here. metadata() validates one path target, then File::open() resolves it again but after an await

I don't think this is a big deal but it would be cleaner to open nonblocking and validated the opened descriptor instead of validating the path name (for unix ofc)

…-full-ci # Conflicts: # codex-rs/app-server/tests/suite/v2/external_agent_config.rs

exec-server: stream files in chunks

7964543

pakrym-oai added the code-reviewed label Jun 15, 2026

pakrym-oai added 3 commits June 15, 2026 11:01

exec-server: reuse request loop for file reads

8108e89

exec-server: test file read handles

ae37730

exec-server: test file reads over rpc

741b2b9

pakrym-oai marked this pull request as ready for review June 15, 2026 18:08

pakrym-oai added 2 commits June 15, 2026 11:18

exec-server: allow more open file reads

f2c9b00

exec-server: use positional file reads

66c1125

chatgpt-codex-connector Bot reviewed Jun 15, 2026

View reviewed changes

Comment thread codex-rs/exec-server/src/local_file_system.rs Outdated

Comment thread codex-rs/exec-server/src/client.rs Outdated

pakrym-oai added 2 commits June 15, 2026 11:39

exec-server: assign file handles on the client

6fb7925

exec-server: reject special files before streaming

5ab7aba

chatgpt-codex-connector Bot reviewed Jun 15, 2026

View reviewed changes

Comment thread codex-rs/exec-server/src/client.rs Outdated

Comment thread codex-rs/exec-server/src/client.rs Outdated

pakrym-oai added 2 commits June 15, 2026 12:25

exec-server: expose file streams through filesystem

8a5bf73

filesystem: require explicit stream support

1ca386e

pakrym-oai requested a review from a team as a code owner June 15, 2026 19:44

pakrym-oai added 5 commits June 15, 2026 12:45

exec-server: keep file handles open after eof

79533b0

exec-server: expose file streams only through filesystem

25f4daf

codex: expose file read protocol methods (#28354)

f4d3c13

Merge remote-tracking branch 'origin/main' into pakrym/fs-read-chunks…

0277fea

…-full-ci

codex: simplify file read streaming (#28354)

701738c

codex: fix CI failure on PR #28354

ca49b47

jif-oai reviewed Jun 16, 2026

View reviewed changes

pakrym-oai added 2 commits June 16, 2026 08:24

codex: bound file read handle IDs (#28354)

e43701d

codex: validate opened file handles (#28354)

699b25f

jif-oai approved these changes Jun 16, 2026

View reviewed changes

codex: fix CI failure on PR #28354

ee57b6d

Merge remote-tracking branch 'origin/main' into pakrym/fs-read-chunks…

408b105

…-full-ci # Conflicts: # codex-rs/app-server/tests/suite/v2/external_agent_config.rs

pakrym-oai merged commit a4711b8 into main Jun 16, 2026
61 of 74 checks passed

pakrym-oai deleted the pakrym/fs-read-chunks-full-ci branch June 16, 2026 16:51

github-actions Bot locked and limited conversation to collaborators Jun 16, 2026

Uh oh!

Conversation

pakrym-oai commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Testing

Uh oh!

pakrym-oai commented Jun 15, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

pakrym-oai commented Jun 15, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

pakrym-oai commented Jun 15, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 15, 2026

Uh oh!

jif-oai left a comment

Choose a reason for hiding this comment

Uh oh!

jif-oai Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

jif-oai Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

pakrym-oai Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

jif-oai Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pakrym-oai commented Jun 15, 2026 •

edited

Loading