ref(relay): Limit maximum number of logs produced by expansion

[klochek]

No description provided.

[linear-code]

INGEST-1079

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 409e55f. Configure here.}

[cursor]

Debug quantity mismatch on limit

Medium Severity

When expansion hits TooManyExpandedLogs, produce has already called records.modify_by for each log emitted so far, but expand returns None and process::expand still finishes try_map with empty logs via unwrap_or_default. Debug RecordKeeper::success then fails quantity balancing and panics.

Additional Locations (1)

• relay-server/src/processing/logs/process.rs#L48-L52

 

^{Reviewed by Cursor Bugbot for commit 409e55f. Configure here.}

[sentry-warden]

max_expanded_log_count is only enforced for integrations, not log containers

The new max_expanded_log_count limit only applies to log integrations (OTel, Vercel, NEL) and not to LogItems::Container. A container with many small logs bypasses the count cap entirely, causing unbounded per-request CPU and memory usage during normalization, filtering, and scrubbing.

Evidence

• relay-server/src/processing/logs/process.rs:expand() receives max_expanded_log_count but only passes it to integrations::expand() for LogItems::Integration.
• For LogItems::Container, it calls expand_log_container() with no count parameter, so the cap is never checked.
• A container is bounded by max_container_size (default 12 MB), but with small logs that still allows tens or hundreds of thousands of items—far above the max_expanded_log_count default of 1000.
• Every log from the container then goes through validate::size, normalize, filter, scrub, and normalize_derived with no early count-based rejection, unlike the integration path where produce() returns Err(TooManyExpandedLogs) once the cap is exceeded.

_{Identified by Warden · wrdn-dos-review · MG4-3M7}

[Dav1dde]

I think this check isn't enough, as we're already deserializing (either via serde_json or prost) the entire array.

[klochek]

So, it's the entire array, but we've only expanded (with accompanying attributes) logs.len() full log entries. It's the amplification by the attributes that can make this size a lot larger than we'd like.

[Dav1dde]

Parsing is already too much. The amplification of attributes is just a bigger multiplier. The OurLog struct is 208 bytes, the minimum size for a valid log item is 2 bytes in JSON (3 if we count the ,, 2 in protobuf). That is a 100x memory multiplier without considering the attribute amplification. With a size limit of 12 MiB per container, that means in a parsed state this is ~1 GiB. Then we have additional overheads like WithHeader<OurLog> and over-allocation of the Vec (capacity). To fully address the issue we already need to limit the count during parsing.