perf: pre-compute string messagepack encoding#1948
Merged
paullegranddc merged 4 commits intoMay 12, 2026
Merged
Conversation
# Changes * Force pre-computation of static strings encoding in messagepack. * Bump rmp dependency to latest minor version in libdatadog # Motivation This makes the msgpack_encoder::v04 10% faster.
Contributor
📚 Documentation Check Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Contributor
🔒 Cargo Deny Results📦
|
Contributor
🎉 All green!❄️ No new flaky tests detected 🎯 Code Coverage (details) 🔗 Commit SHA: 02482bc | Docs | Datadog PR Page | Give us feedback! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1948 +/- ##
==========================================
+ Coverage 71.85% 72.60% +0.75%
==========================================
Files 434 450 +16
Lines 70454 74056 +3602
==========================================
+ Hits 50624 53771 +3147
- Misses 19830 20285 +455
🚀 New features to boost your workflow:
|
yannham
approved these changes
May 7, 2026
| use std::borrow::Borrow; | ||
|
|
||
| // Compute the encoding of a string to messagepack in a const manner | ||
| const fn msp_const_string_encoding(s: &str) -> ([u8; 256], usize) { |
Contributor
There was a problem hiding this comment.
It's probably useless given the size involved, but as a thought experiment, I wonder if you could use const generics to produce an array of the precise size required. I guess the only thing needed is to have a const-fn or macro to get the length of a static string at compile-time, and use that as a const integer parameter.
Contributor
Author
There was a problem hiding this comment.
It does work 😄
bantonsson
approved these changes
May 8, 2026
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
paullegranddc
deleted the
paullgdc/trace_utils/optmize_static_str_msgpack_encoding
branch
iunanua
added a commit
that referenced
this pull request
May 18, 2026
# Release proposal for libdd-sampling and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-tinybytes **Next version:** `1.1.1` **Semver bump:** `patch` **Tag:** `libdd-tinybytes-v1.1.1` ### Commits - perf: pre-compute string messagepack encoding (#1948) ## libdd-trace-protobuf **Next version:** `3.0.2` **Semver bump:** `patch` **Tag:** `libdd-trace-protobuf-v3.0.2` ### Commits - feat(obfuscation)!: feature parity on span obfuscation [APMSP-2671] (#1788) ## libdd-trace-utils **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v4.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^3.0.2 → ^4.1.0 ### Commits - feat(trace-utils)!: search all spans to populate tracer payload fields (#1954) - perf: pre-compute string messagepack encoding (#1948) - feat(capablities)!: sleep & spawn capabilities (#1873) - feat(libdd-trace-utils): check for empty value in header datadog-client-computed-stats (#1900) - chore(benchmarks): add allocation size tracking allocator (#1905) - fix(crypto): gate libdd-common TLS features in obfuscation and capabilities-impl (#1872) - fix(libdd-traceutils): Update cloud environment detection logic for Serverless [SVLS-8799] (#1857) - fix(spawn_worker): defer trampoline self-deletion to avoid Valgrind false positive (#1844) - feat(runtime)!: add shared runtime (#1602) - ci: compilation of libdd-data-pipeline to wasm32 (#1830) - feat(capabilities)!: trait architecture http (#1555) - feat(trace-export): map DD span resource to OTLP resource.name attribute (#1811) - feat(otel): add support for OTLP trace export (#1641) ## libdd-sampling **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-sampling-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. [APMSP-2671]: https://datadoghq.atlassian.net/browse/APMSP-2671?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SVLS-8799]: https://datadoghq.atlassian.net/browse/SVLS-8799?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: iunanua <18325288+iunanua@users.noreply.github.com>
iunanua
added a commit
that referenced
this pull request
May 18, 2026
# Release proposal for libdd-data-pipeline and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-dogstatsd-client **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-dogstatsd-client-v3.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^3.0.2 → ^4.1.0 ### Commits - fix(crypto): gate libdd-common TLS features in remaining internal crates + add CI guard (#1943) ## libdd-trace-obfuscation **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v3.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^3.0.2 → ^4.1.0 - `libdd-trace-utils`: ^3.0.1 → ^4.0.0 ### Commits - feat!: integrate obfuscation to the stats exporter [APMSP-2764] (#1819) - feat!: added regex-lite feature (#1939) - chore: clippy (#1889) - fix(crypto): gate libdd-common TLS features in obfuscation and capabilities-impl (#1872) - feat(obfuscation)!: feature parity on span obfuscation [APMSP-2671] (#1788) - feat(obfuscation/sql): feature parity on sql obfuscation [APMSP-2667] (#1708) ## libdd-trace-stats **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v3.0.0` ###⚠️ major bump forced due to: - `libdd-trace-utils`: ^3.0.1 → ^4.0.0 ### Commits - perf: pre-compute string messagepack encoding (#1948) - feat!: integrate obfuscation to the stats exporter [APMSP-2764] (#1819) - feat(capablities)!: sleep & spawn capabilities (#1873) - feat: use ip quantization when aggregating peer tags for trace stats (#1944) - fix(crypto): gate libdd-common TLS features in remaining internal crates + add CI guard (#1943) - feat(shared_runtime)!: allow worker to be stopped after fork (#1893) - feat(sidecar)!: Add stats computation via SHM (#1821) - feat(stats): propagate service source from span meta to client stats payload (#1803) - fix(stats): align with css spec (#1790) ## libdd-data-pipeline **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v4.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^3.0.2 → ^4.1.0 - `libdd-telemetry`: ^4.0.0 → ^5.0.0 - `libdd-trace-utils`: ^3.0.1 → ^4.0.0 ### Commits - fix(libdd-telemetry): restore previous Cargo.toml version (#1993) - fix(data-pipeline): remove default-features from of trace-obfuscation (#1981) - fix(trace_exporter: shared_runtime): unwrap_or being eager is not good (#1983) - perf: pre-compute string messagepack encoding (#1948) - feat!: integrate obfuscation to the stats exporter [APMSP-2764] (#1819) - feat(capablities)!: sleep & spawn capabilities (#1873) - fix(telemetry): avoid trigger loop in telemetry worker (#1950) - feat(telemetry)!: include dependencies and integrations in app-extended-heartbeat (#1962) - perf(trace-serializer): pre-allocate serialization buffer (#1949) - feat!: added regex-lite feature (#1939) - fix(crypto): gate libdd-common TLS features in remaining internal crates + add CI guard (#1943) - feat(telemetry): add session id support to trace export (#1822) - fix(path): missing bench path in data-pipeline (#1907) - feat(data-pipeline): port dd-trace-rs trace buffer implementation (#1826) - feat(info_fetcher): add timeout to info fetcher (#1890) - feat(shared_runtime)!: allow worker to be stopped after fork (#1893) - feat(sidecar)!: Add stats computation via SHM (#1821) - ci(libdd-shared-runtime): downgrade version so publish workflow succeeds (#1870) - feat(runtime)!: add shared runtime (#1602) - ci: compilation of libdd-data-pipeline to wasm32 (#1830) - feat(capabilities)!: trait architecture http (#1555) - feat(otel): add support for OTLP trace export (#1641) - fix(stats): align with css spec (#1790) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: iunanua <18325288+iunanua@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Motivation
This makes the msgpack_encoder::v04 benchmark 10% faster.