fix(python-sdk): keep streamed uploads off the retrying transport

[mishushakov]

Stacked on #1659.

pyqwest.middleware.retry makes a streamed request body replayable by growing a bytearray copy of it as the body is sent, for every body that isn't already bytes. So an upload was streamed to the wire and mirrored whole in RAM. Measured on main with volume.write_file(path, file_object) and a 64 MiB file: Content-Length: 67108864 with no chunked encoding and the server reading incrementally — genuinely streamed — while 67,108,864 bytes landed in RetryingRequestContent._buffer, for a traced peak of 70 MB. The copy is a tee rather than a replacement for streaming, which is why nothing in volume_sync.py or the filesystem write path looks wrong.

The copy is what makes the retries work, so it can't just be removed. reqwest reads ahead into its own body channel while the connection is being established, so a connect error surfaces with the iterator already started and nothing to rewind. Measured against a refused port on pyqwest 0.9.0, chunks pulled from the body before ConnectionError:

body	sync	async
8 × 1 KiB	2, 2, 2	8, 8, 8 (all of it)
8 × 1 MiB	2, 2, 2	1, 2, 3
1 × 200 B (unary-RPC shaped)	1, 1, 1	1, 1, 1

That last row is why the retry layer has to stay for RPC traffic: connectrpc hands pyqwest a generator even for unary calls, and without the copy every envd RPC would silently lose its connect retries.

So the uploads move instead of the copy. get_upload_transport is the httpx adapter over the same connection pool with the retry layer left out, and the three upload paths take it when their body is streamed:

• envd files.write — a third sibling client next to the retrying and streaming ones (_envd_api_upload), used on both the octet-stream and multipart paths.
• volume.write_file — get_upload_api_client, the volume content client on that transport.
• template context uploads — which as a side effect stop building a reqwest pool per build, the workaround refactor(python-sdk): unify the pyqwest connection pools #1659 called out in its review notes.

Sharing the pool is the point: an upload still travels the sandbox's or the volume host's pooled connections, and a health probe after a failed RPC still lands on the same connection. What a streamed upload gives up is the connect retry, which fires before any of the body was written, so the caller sees the connection error intact and can retry the upload itself. Writes of in-memory data (files.write(path, "text"), bytes) stay on the retrying transport — a bytes body is replayed without a copy, so those retries cost nothing and are kept.

No user-facing API change, so there are no usage examples to add: same public surface, same timeouts, no new options. JS has no counterpart — undici streams request bodies without buffering and the retry middleware is pyqwest-only.

Closes SDK-332.

Verification

• tests/test_upload_transport.py (new, 12 tests): a 32 MiB streamed upload arrives whole with Content-Length framing and stays under 8 MiB of traced peak allocations (measured 1.5 MB, against 38 MB on the retrying transport); the upload transport wraps the very same pool object with no retry layer and is cached per proxy and read bound; the envd filesystem's three clients sit on the three transports; and a write of in-memory data still goes to the retrying client while a file-like one goes to the upload client, on both the octet-stream and multipart paths.
• tests/test_volume_client.py: the volume upload transport is the SDK-wide upload transport, distinct from the volume client's default one.
• Against prod: the full files/ suites, sync and async (123 tests, in-memory, streamed octet-stream and multipart writes), and template_sync/template_async test_build.py with force_upload=True, which puts a real build context through the presigned S3 PUT on the shared pool.
• python-sdk unit suite, ruff lint/format and ty typecheck are green. The JS checks were not run — no JS or TS files are touched.

Notes for review

• get_pyqwest_transport (the retrying stack, used by the envd RPC clients) is unchanged in behavior; it's now built on top of get_pool, which is the pool cache split out so the retry layer can be skipped without losing the connections.
• Alternative considered: bounding the middleware's copy instead (keep it up to ~1 MiB, drop it past that), which would have preserved retries for small streamed bodies. It needed a hand-rolled retry loop, since pyqwest's execute_sync is @final — more machinery than the problem deserves.
• Worth raising a max_buffered_body_size upstream in pyqwest's middleware anyway, so other consumers don't pay an unbounded copy.

🤖 Generated with Claude Code

[linear-code]

SDK-332

[cursor]

PR Summary

Medium Risk
Touches the shared HTTP transport stack and every large upload path; streamed uploads intentionally lose automatic connect retries (callers must retry), while envd RPC and in-memory writes keep the retrying client.

Overview
Fixes SDK-332 by stopping pyqwest’s connect-retry middleware from buffering entire streamed upload bodies in RAM (large files.write, volume.write_file, and template context uploads could peak near file size).

Transport layering is refactored in sync/async client_*: shared pools are created via get_pool(), connect retries wrap the pool for normal traffic, and a new get_upload_transport() / get_transport(..., for_upload=True) exposes an httpx adapter on the same pool without the retry layer. In-memory bodies still use the retrying stack because bytes replay without copying.

Call sites are wired to the upload path: sandbox filesystem gets _envd_api_upload and uses it only for streamed octet-stream and multipart writes; volume write_file uses get_upload_api_client; template upload_file drops per-build HTTPTransport in favor of the shared upload transport. Tests cover transport caching, client selection, routing, and tracemalloc peaks on ~32 MiB uploads.

^{Reviewed by Cursor Bugbot for commit 0d6033d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Package Artifacts

Built from 93f6c21. Download artifacts from this workflow run.

JS SDK (e2b@2.38.4-fix-retry-body-buffering.0):

npm install ./e2b-2.38.4-fix-retry-body-buffering.0.tgz

CLI (@e2b/cli@2.16.2-fix-retry-body-buffering.0):

npm install ./e2b-cli-2.16.2-fix-retry-body-buffering.0.tgz

Python SDK (e2b==2.38.0+fix.retry.body.buffering):

pip install ./e2b-2.38.0+fix.retry.body.buffering-py3-none-any.whl

[changeset-bot]

🦋 Changeset detected

Latest commit: 0d6033d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@e2b/python-sdk	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 0d6033d. Configure here.}

[cursor]

Volume writes drop in-memory retries

Medium Severity

write_file always uses get_upload_volume_api_client, including for str/bytes bodies that reach the transport as bytes and need no replay copy. Those calls lose connect retries, unlike files.write and contrary to the changeset note that in-memory writes keep them.

Additional Locations (1)

• packages/python-sdk/e2b/volume/volume_async.py#L624-L628

 

^{Reviewed by Cursor Bugbot for commit 0d6033d. Configure here.}