perf(chunk): hold ChunkPutRequest content as Bytes for zero-copy fan-out#1
Closed
jacderida wants to merge 1 commit into
Closed
perf(chunk): hold ChunkPutRequest content as Bytes for zero-copy fan-out#1jacderida wants to merge 1 commit into
jacderida wants to merge 1 commit into
Conversation
Replace `content: Vec<u8>` with `content: bytes::Bytes` on ChunkPutRequest. Wire format is unchanged — Bytes serialises as a byte sequence under postcard/serde, identical to Vec<u8> — but the in-memory representation is now refcounted, so callers that send the same chunk to multiple peers (notably close-group replication) share a single backing buffer instead of deep-copying the 4 MB payload per peer. Heaptrack against a 20 MB upload on a release ant binary showed the client's peak heap dominated by RawVecInner::finish_grow calls in ChunkPutRequest construction, with 168 MB consumed via ChunkMessage::encode → chunk_put_to_close_group. The fan-out path was running `content.to_vec()` once per recipient at chunk.rs:173, which the new Bytes-typed field eliminates from the caller side. This commit is the protocol-side half of the fix; the ant-client side drops the `to_vec()` in a follow-up PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closed
4 tasks
Owner
Author
|
Wrong target — reopening against WithAutonomi/ant-protocol. |
jacderida
added a commit
to jacderida/ant-client
that referenced
this pull request
May 11, 2026
`chunk_put_with_proof` was deep-copying every chunk's content into a fresh `Vec<u8>` via `Bytes::to_vec()` before stuffing it into a `ChunkPutRequest`. Since `chunk_put_to_close_group` spawns one of these per recipient (CLOSE_GROUP_MAJORITY ≈ 5) and the AIMD store controller caps concurrent chunks at 64, peak in-flight chunk content reached ~64 × 5 × 4 MB = 2.5 GB just from these copies — enough to OOM-kill `ant` on a 4 GB client VM partway through a 300 MB-or-larger upload. Heaptrack against a clean-exit 20 MB upload (release `ant` on a 4 GB VM) showed 285 MB peak in `alloc::raw_vec::RawVecInner::finish_grow`, with 168 MB of that consumed via `ChunkMessage::encode` → `chunk_put_to_close_group` — i.e. the per-recipient copy + encode loop. ant-protocol now holds `ChunkPutRequest::content` as `bytes::Bytes` (see jacderida/ant-protocol#1), so the caller can pass the refcounted `Bytes` straight through. Each peer's spawned task now shares the single 4 MB backing buffer instead of holding an independent copy. Wire format is unchanged. The patch.crates-io stanza pins ant-protocol to the perf branch commit so the build resolves the Bytes-typed field. The pin should be removed once a crates.io release containing the protocol change is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jacderida
added a commit
to WithAutonomi/ant-client
that referenced
this pull request
May 11, 2026
`chunk_put_with_proof` was deep-copying every chunk's content into a fresh `Vec<u8>` via `Bytes::to_vec()` before stuffing it into a `ChunkPutRequest`. Since `chunk_put_to_close_group` spawns one of these per recipient (CLOSE_GROUP_MAJORITY ≈ 5) and the AIMD store controller caps concurrent chunks at 64, peak in-flight chunk content reached ~64 × 5 × 4 MB = 2.5 GB just from these copies — enough to OOM-kill `ant` on a 4 GB client VM partway through a 300 MB-or-larger upload. Heaptrack against a clean-exit 20 MB upload (release `ant` on a 4 GB VM) showed 285 MB peak in `alloc::raw_vec::RawVecInner::finish_grow`, with 168 MB of that consumed via `ChunkMessage::encode` → `chunk_put_to_close_group` — i.e. the per-recipient copy + encode loop. ant-protocol now holds `ChunkPutRequest::content` as `bytes::Bytes` (see jacderida/ant-protocol#1), so the caller can pass the refcounted `Bytes` straight through. Each peer's spawned task now shares the single 4 MB backing buffer instead of holding an independent copy. Wire format is unchanged. The patch.crates-io stanza pins ant-protocol to the perf branch commit so the build resolves the Bytes-typed field. The pin should be removed once a crates.io release containing the protocol change is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jacderida
added a commit
to jacderida/ant-client
that referenced
this pull request
May 13, 2026
`chunk_put_with_proof` was deep-copying every chunk's content into a fresh `Vec<u8>` via `Bytes::to_vec()` before stuffing it into a `ChunkPutRequest`. Since `chunk_put_to_close_group` spawns one of these per recipient (CLOSE_GROUP_MAJORITY ≈ 5) and the AIMD store controller caps concurrent chunks at 64, peak in-flight chunk content reached ~64 × 5 × 4 MB = 2.5 GB just from these copies — enough to OOM-kill `ant` on a 4 GB client VM partway through a 300 MB-or-larger upload. Heaptrack against a clean-exit 20 MB upload (release `ant` on a 4 GB VM) showed 285 MB peak in `alloc::raw_vec::RawVecInner::finish_grow`, with 168 MB of that consumed via `ChunkMessage::encode` → `chunk_put_to_close_group` — i.e. the per-recipient copy + encode loop. ant-protocol now holds `ChunkPutRequest::content` as `bytes::Bytes` (see jacderida/ant-protocol#1), so the caller can pass the refcounted `Bytes` straight through. Each peer's spawned task now shares the single 4 MB backing buffer instead of holding an independent copy. Wire format is unchanged. The patch.crates-io stanza pins ant-protocol to the perf branch commit so the build resolves the Bytes-typed field. The pin should be removed once a crates.io release containing the protocol change is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jacderida
added a commit
to jacderida/ant-client
that referenced
this pull request
May 13, 2026
`chunk_put_with_proof` was deep-copying every chunk's content into a fresh `Vec<u8>` via `Bytes::to_vec()` before stuffing it into a `ChunkPutRequest`. Since `chunk_put_to_close_group` spawns one of these per recipient (CLOSE_GROUP_MAJORITY ≈ 5) and the AIMD store controller caps concurrent chunks at 64, peak in-flight chunk content reached ~64 × 5 × 4 MB = 2.5 GB just from these copies — enough to OOM-kill `ant` on a 4 GB client VM partway through a 300 MB-or-larger upload. Heaptrack against a clean-exit 20 MB upload (release `ant` on a 4 GB VM) showed 285 MB peak in `alloc::raw_vec::RawVecInner::finish_grow`, with 168 MB of that consumed via `ChunkMessage::encode` → `chunk_put_to_close_group` — i.e. the per-recipient copy + encode loop. ant-protocol now holds `ChunkPutRequest::content` as `bytes::Bytes` (see jacderida/ant-protocol#1), so the caller can pass the refcounted `Bytes` straight through. Each peer's spawned task now shares the single 4 MB backing buffer instead of holding an independent copy. Wire format is unchanged. The patch.crates-io stanza pins ant-protocol to the perf branch commit so the build resolves the Bytes-typed field. The pin should be removed once a crates.io release containing the protocol change is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jacderida
added a commit
to WithAutonomi/ant-client
that referenced
this pull request
May 13, 2026
`chunk_put_with_proof` was deep-copying every chunk's content into a fresh `Vec<u8>` via `Bytes::to_vec()` before stuffing it into a `ChunkPutRequest`. Since `chunk_put_to_close_group` spawns one of these per recipient (CLOSE_GROUP_MAJORITY ≈ 5) and the AIMD store controller caps concurrent chunks at 64, peak in-flight chunk content reached ~64 × 5 × 4 MB = 2.5 GB just from these copies — enough to OOM-kill `ant` on a 4 GB client VM partway through a 300 MB-or-larger upload. Heaptrack against a clean-exit 20 MB upload (release `ant` on a 4 GB VM) showed 285 MB peak in `alloc::raw_vec::RawVecInner::finish_grow`, with 168 MB of that consumed via `ChunkMessage::encode` → `chunk_put_to_close_group` — i.e. the per-recipient copy + encode loop. ant-protocol now holds `ChunkPutRequest::content` as `bytes::Bytes` (see jacderida/ant-protocol#1), so the caller can pass the refcounted `Bytes` straight through. Each peer's spawned task now shares the single 4 MB backing buffer instead of holding an independent copy. Wire format is unchanged. The patch.crates-io stanza pins ant-protocol to the perf branch commit so the build resolves the Bytes-typed field. The pin should be removed once a crates.io release containing the protocol change is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ChunkPutRequest.content: Vec<u8>withcontent: bytes::Bytes.bytescrate'sserdefeature soBytesserialises identically toVec<u8>on the wire (both becomeseq[u8]under postcard).ChunkPutRequest::new/with_paymentsignatures and the two existing tests.Why
A heaptrack capture against a release
antbinary uploading a 20 MB file shows peak heap pinned to ~285 MB inalloc::raw_vec::RawVecInner::finish_grow, with the dominant call chain landing atChunkMessage::encode→chunk_put_to_close_group. The fan-out path atant-client/ant-core/src/data/client/chunk.rs:173does:```rust
let request = ChunkPutRequest::with_payment(address, content.to_vec(), proof);
```
The
contentparameter is already a refcountedbytes::Bytes, butto_vec()deep-copies the entire 4 MB chunk into a freshVec<u8>— once per recipient. WithCLOSE_GROUP_MAJORITY ≈ 5peers and the AIMD store cap at 64, peak in-flight chunk-content can reach ~2.5 GB on a single client, which exceeds the 4 GB VM budget once DHT/QUIC/etc. overhead is added.Switching
contenttoBytesremoves the requirement for the caller to copy. The ant-client follow-up PR drops theto_vec()and passes theBytesstraight through, so each peer's spawned task shares a single 4 MB backing buffer via refcount instead of holding N independent copies.Wire compatibility
Identical. Under postcard + serde, both
Vec<u8>andbytes::Bytesare encoded as a varint length followed by the raw bytes. A mixed network of old and new clients/servers will interoperate.Test plan
cargo check --all-featurescleancargo test --lib chunk— all 17 chunk tests pass (the two ChunkPutRequest tests were updated to constructBytes::from_static(...))[patch.crates-io]and drops theto_vec()🤖 Generated with Claude Code