feat: Add context thread, enqueue all RUM and Logging operations for async processing by awforsythe · Pull Request #169 · DataDog/dd-sdk-cpp

awforsythe · 2026-03-06T21:19:43Z

This PR establishes a third background thread (in addition to the storage and upload threads) that's owned by the Core - the context thread - and it updates the FeatureScope interface such that a feature must offload work to this thread in order to 1.) read values from CoreContext, 2.) mutate CoreContext, or 3.) produce events.

This change brings the implementation of the C++ SDK more in line with the design of the iOS and Android SDKs. By design, any significant work done in response to an API call now happens on background threads. A few things have to be done synchronously on the calling thread, but thereafter, the feature enqueues a function on the context queue, and that function will be invoked on the context thread once it's consumed from the queue.

This approach has a couple of key benefits:

API calls block the calling thread no longer than absolutely necessary: the API can return as soon as it's pushed a function onto the context queue
Because the context thread executes all queued callbacks serially, it's now safe (generally speaking) to call any API function from any thread

The two feature implementations affected by this change are Logging and Rum. Both sets of API functions now perform the majority of their work (including event construction and JSON serialization) on the context thread.

Example per-thread activity on StartView

For example, in the previous implementation, here's what typically happened when the application called Rum::StartView from its main thread:

On the main thread, after the application calls Rum::StartView:
- impl::Rum constructs a StartView command
- impl::Rum dispatches that command to the root RumApplicationScope
- RumApplicationScope propagates command to RumSessionScope
- RumSessionScope handles view lifecycle logic; creates RumViewScope; propagates command
- RumViewScope updates internal state in response to command
- RumViewScope examines relatives in scope tree to build a snapshot current RUM state
- RumViewScope assembles a merged set of user attributes
- RumViewScope acquires a shared mutex for read and makes an immutable snapshot of CoreContext and retrieves os info, device info, etc.
- RumViewScope combines all of the data it's gathered into a RumViewEvent
- RumViewScope acquires a mutex in order to access a shared JSON buffer, then serializes the event to JSON, then copies the encoded payload onto the storage queue
- impl::Rum examines the updated scope tree to obtain a new snapshot of Rum application state
- impl::Rum acquires a shared mutex for write and mutates CoreContext with up-to-date Rum context
- Finally, Rum::StartView returns control to the application.
On the storage thread, once that payload is consumed from the storage queue:
- The raw JSON event is flushed to a batch file
On the upload thread, once that batch is ready for upload:
- The raw JSON event is read from the file, concatenated into the body of an HTTP request, and uploaded

By contrast, here's what happens as of this PR:

On the main thread, after the application calls Rum::StartView:
- impl::Rum constructs a StartView command
- impl::Rum constructs a std::weak_ptr to itself
- impl::Rum constructs a std::function, capturing both the weak_ptr and the command by value, and copies it onto the context queue
- Rum::StartView returns control to the application.
On the context thread, once that function is consumed from the context queue:
- the thunk that encloses the callback makes an immutable snapshot of CoreContext, then calls the callback
- the callback locks the std::weak_ptr, obtaining a stable std::shared_ptr<impl::Rum>
- the callback propagates the command to the root RumApplicationScope
- RumApplicationScope propagates command to RumSessionScope
- RumSessionScope handles view lifecycle logic; creates RumViewScope; propagates command
- RumViewScope updates internal state in response to command
- RumViewScope examines relatives in scope tree to build a snapshot current RUM state
- RumViewScope assembles a merged set of user attributes
- RumViewScope combines all of the data it's gathered, along with data passed in via the CoreContext arg, into a RumViewEvent
- RumViewScope serializes the event to JSON in a persistent buffer (only used by the context thread, so no lock needed), then copies the encoded payload onto the storage queue
- impl::Rum examines the updated scope tree to obtain a new snapshot of Rum application state
- impl::Rum enqueues a function to the context queue that will mutate CoreContext with up-to-date Rum context once it's processed by the context thread
On the storage thread, once that payload is consumed from the storage queue:
- The raw JSON event is flushed to a batch file
On the upload thread, once that batch is ready for upload:
- The raw JSON event is read from the file, concatenated into the body of an HTTP request, and uploaded

Takeaways from this quick comparison:

We return control to the caller much faster now, hooray
There's a bit more heap usage overall from copying things in more places, but overhead of allocations/locks/etc. is largely confined to background threads
Ordering of CoreContext changes is predictable and ordered: e.g. a log message will be properly enriched with the RUM Context that was current at the time the log call was made

Profiling results

I made a quick ad hoc repl benchmark that just creates 10 views, adds 5 custom actions per view, and makes 3 log calls for each action, for a total of 60 RUM API calls and 150 log calls, in as short a span of time as possible.

These graphs show the state of main before these changes (and before the accompanying std::string_view -> std::string change in RUM command types) in purple, and the latest changes in blue.

As you'd expect, peak memory usage is a bit higher, because we have an extra queue now, and we have to copy more values in order to shuttle data around between threads:

Likewise, we're performing more allocations per API call:

(This is something I'd like to profile more exhaustively and ultimately optimize, via approaches like CoW, string interning, inline buffers, pool allocators, etc.; but the focus for now remains on building critical SDK functionality.)

However, we can see a clear improvement in main-thread CPU time per API call:

Previously, a RUM API call would take somewhere around 3 microseconds, give or take 1μs. As of this PR, RUM API calls are consistently in the sub-microsecond range.

Additionally, looking at the first few API calls (which historically block for much longer than is typical, because they're performing initial up-front allocations etc.) clearly shows the benefits of offloading this work to the context thread: we no longer block the application on a cold start of the SDK.

It's only used on the context thread, which processes functions serially, so it does not require synchronization.

awforsythe · 2026-03-09T22:57:17Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5ae4aeb35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-09T23:04:01Z

+  _context_queue->Push([this, func]() {
+    DATADOG_ASSERT(_context_provider, "FeatureScope has no _context_provider");
+    const CoreContext context = _context_provider->Get();
+    func(context, _event_generated_func);


Avoid dangling this captures in context queue thunks

ExecuteOnContextThread enqueues a lambda that captures this, then dereferences _context_provider and _event_generated_func later on the context thread. During shutdown, Core::Stop() calls each feature's OnCoreStopping() (which resets/destroys FeatureScope) before it drains the context queue, so any queued thunk that has not run yet can execute with a dangling FeatureScope pointer and trigger use-after-free/crashes.

Useful? React with 👍 / 👎.

This is a fair point - fixed in 1dc05e9 by modifying the shutdown order such that the context queue is drained, then all features are stopped, then the storage and upload threads are stopped.

As a consequence, it's no longer possible for a feature implementation to do last-minute work (or produce last-minute events) in response to Feature::Stop() - see 77dfaa9. This is appropriate, as SDK stop is designed to be an immediate, non-blocking operation - in existing RUM SDKs, features are not traditionally given the opportunity to produce events in response to core.stop().

Note that in its current form, the C++ SDK still drains both the context queue and storage queue on shutdown, making Core::Stop() a blocking operation whose overhead scales with the amount of pending work enqueued by the application's API calls. This is nifty in that we can guarantee that if an API call produces an event, that event will be flushed to disk reliably, but it's inconsistent with the behavior of the iOS SDK, which treats shutdown as immediate and non-blocking, abandoning all pending work unless using a special integration-test-only shutdown method. I've created RUM-15042 to track this design discrepancy.

chatgpt-codex-connector · 2026-03-09T23:04:01Z

+  _context_queue->Push([this, callback]() {
+    DATADOG_ASSERT(_context_provider, "FeatureScope has no _context_provider");
+    _context_provider->Update(callback);
+  });


Update CoreContext immediately when already on context thread

UpdateContext now always re-enqueues the mutation, even when the caller is already running on the context thread. RUM command processing calls UpdateFeatureContext() from inside ExecuteOnContextThread, so the actual context write is deferred behind already-queued work; a subsequent log callback queued right after a RUM call can therefore run before the RUM context update and be enriched with stale/missing RUM IDs, regressing correlation accuracy.

Useful? React with 👍 / 👎.

This is a legitimate error in the implementation. As originally written, calling a RUM API function would enqueue a single function to run on the context thread. That function would perform its command-processing work (updating internal state and producing RUM events), then enqueue another function that would mutate the CoreContext to add the latest RUM UUIDs.

That second function would be enqueued at the end of the queue, causing a legitimate bug. e.g. if you made these three API calls back-to-back:

logger.Info("log1");

rum.StartView("my-view");

logger.Info("log2");

...then the second log event would be missing the RUM View ID set for my-view.

This is fixed in dac8a27: we now enqueue our two functions back-to-back, serially, in response to a RUM API call, such that in the example above, log2 will consistently see the updated View ID set by StartView.

datadog-official · 2026-03-10T19:12:28Z

✅ Code Quality ✅ Code Vulnerabilities ✅ Library Vulnerabilities ✅ Secrets

🎉 All green!

🛠️ No new code quality issues
🛡️ No new code vulnerabilities
📚 No new vulnerable libraries detected
🔑 No new secrets detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: fcbdf93 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

This function is used to tear down feature state; it may not be used to generate events or perform any work on the context thread (which is now stopped before the call to Feature::Stop()). This is consistent with the behavior of the iOS SDK, which treats Stop() as an immediate, non-blocking shutdown, abandoning all work on the context thread rather than draining the queue.

awforsythe · 2026-03-11T15:08:51Z

@codex review

Two P1 issues were flagged in a prior review - please verify that both are fixed:

Dangling this in context queue thunks (feature_scope.cpp:96): Shutdown was destroying FeatureScope before draining the queue, risking use-after-free. Fix: drain the context queue before stopping features.
UpdateContext always re-enqueues (feature_scope.cpp:77): RUM command processing called UpdateFeatureContext() from inside a context-thread thunk, appending the context mutation to the end of the queue and causing subsequent log events to see stale RUM IDs. Fix: enqueue the context mutation immediately after the command work, not as a trailing re-enqueue.

Please confirm these are correctly addressed.

We don't need two separate definitions of the same function type; it's clearer to just use `EventWriter` everywhere.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77dfaa9f1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-11T15:14:48Z

+  // thread or tear down feature state.
+  // TODO(RUM-15042): Other SDKs abandon the context thread and shut down in a
+  // non-blocking fashion, without draining the context queue
+  _context_queue->Stop();


Keep stop hooks able to mutate context before queue shutdown

Stopping the context queue before invoking OnCoreStopping() causes stop-time context mutations to be silently dropped, because FeatureScope::UpdateContext() enqueues via Push() and does not handle a false return once the queue is stopped. In this commit, Rum::Stop() still depends on UpdateContext() to clear ctx.rum, so after a stop/start cycle stale RUM IDs can remain in CoreContext until a later RUM command runs, and early log events may be enriched with incorrect session/view/action IDs.

Useful? React with 👍 / 👎.

Fixed in fcbdf93 - rather than requiring each feature to reset its own subset of CoreContext state on Stop() (which can no longer be done because Feature-induced CoreContext mutations are now asynchronous, and no further async work can be done on Stop()), we now just have Core::Start() preemptively reset CoreContext to a clean starting value.

Previously, each feature that stored custom state in `CoreContext` (in practice, just `Rum`) was responsible for clearing its own `CoreContext` state in `Feature::Stop()` so that, for example, if you stop and SDK instance and then restart it, you won't accidentally use stale RUM IDs from its past life. That was a simple enough approach when `CoreContext` mutations happened synchronously, but now that we're using a context thread (which has to be shut down before `Feature::Stop()`), this approach no longer makes sense. It's much simpler to just have the Core itself reset the `CoreContext` value each time it starts. This guarantees that the SDK will always use a clean `CoreContext` on `Start()`, regardless of whether it's stopped and restarted, and without requiring any feature-specific cleanup logic or tricky synchronization on shutdown.

awforsythe · 2026-03-11T15:58:27Z

@codex review

One P1 issue was flagged in a prior review:

Rum::Stop() called UpdateContext() to clear ctx.rum, but Core::Stop() drains and joins the context thread before calling OnCoreStopping(), so the enqueued mutation was silently dropped — leaving stale RUM IDs in CoreContext across a stop/start cycle. Fixed by moving context cleanup to Core::Start(), which resets all mutable feature context fields synchronously before any threads or features are started.

Please verify that the issue is fixed.

chatgpt-codex-connector · 2026-03-11T16:03:07Z

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

fuzzybinary

Codex caught way more than I would have on this one.

Changes LGTM.

fuzzybinary · 2026-03-13T18:10:33Z

+    const EventWriter& event_writer,
+    const DiagnosticLogger& in_diagnostic_logger,
+    FeatureScope::ExecutionMode mode,
+    Queue<std::function<void()>>* context_queue


I really wish C++ had a nice way to enforce single ownership with non-owning observers that didn't require shared_ptr.

fuzzybinary · 2026-03-13T18:18:14Z

  ) const;

-  void Dispatch(const RumCommand& command);
+  void DispatchAsync(const RumCommand& command);


I appreciate being more explicit here but we probably could have left this named Dispatch and it would have been fine.

Yeah, I agree - one of those cases of "the LLM made a silly style choice and I didn't get around to correcting it". I may fix it later 😅

awforsythe force-pushed the aforsythe/RUM-13165/rum-commands-own-strings branch from 394d726 to 8d8cf2c Compare March 9, 2026 20:35

awforsythe force-pushed the aforsythe/RUM-13164/context-queue branch from e992061 to 16d264d Compare March 9, 2026 20:35

Base automatically changed from aforsythe/RUM-13165/rum-commands-own-strings to main March 9, 2026 21:05

awforsythe added 11 commits March 9, 2026 17:44

feat: Define an entry point for the context thread

57b54d7

feat: Update Core to own a context queue and run a context thread

d74de51

feat: Update FeatureScope to allow deferred execution on context thread

511f69e

feat: Update RUM implementation to use context thread

c798ea3

fix: Make FeatureScope creation explicit about test-only mode

01af161

fix: Make UpdateContext queue context mutations for async execution

3ad3c53

chore: Fix comment to avoid "feature operation" confusion

dc4db37

chore: Remove Core::FlushContextQueue; it's not actually used

77d6641

chore: Update comments

4fcd192

chore: Remove vital_id references to reconcile

f840aef

fix: Update Logging implementation to use context thread

d5ae4ae

awforsythe force-pushed the aforsythe/RUM-13164/context-queue branch from 16d264d to d5ae4ae Compare March 9, 2026 21:45

awforsythe changed the title ~~feat: Add context thread, enqueue all RUM commands for async processing~~ feat: Add context thread, enqueue all RUM and Logging operations for async processing Mar 9, 2026

chore: Remove mutex on RUM event encode buffer

e1d8b3a

It's only used on the context thread, which processes functions serially, so it does not require synchronization.

chatgpt-codex-connector Bot reviewed Mar 9, 2026

View reviewed changes

awforsythe added 4 commits March 10, 2026 12:40

chore: Update FeatureScope tests to use a real context thread

3f579fb

chore: Refactor repeated mock os/device info into a common header

e743d19

chore: Add context_thread_test.cpp to complement feature_scope_test.cpp

0005f06

fix: Update RUM context immediately after command processing

dac8a27

awforsythe added 6 commits March 10, 2026 15:46

chore: Supplement FeatureScope tests for fixed context-mutation ordering

b00dd9a

chore: Add tentative context-queue-drain test

6c12f14

chore: Remove old GetContext() and WriteEvent() functions

101afbf

chore: Add tests for context-queue shutdown behavior

37d0946

fix: Fix shutdown ordering: stop context thread before stopping features

1dc05e9

awforsythe force-pushed the aforsythe/RUM-13164/context-queue branch from aa691a9 to 77dfaa9 Compare March 11, 2026 14:52

chore: Replace EventGeneratedFunc with EventWriter

4e679de

We don't need two separate definitions of the same function type; it's clearer to just use `EventWriter` everywhere.

chatgpt-codex-connector Bot reviewed Mar 11, 2026

View reviewed changes

awforsythe added 2 commits March 11, 2026 11:29

chore: Remove CoreContextProvider arg from ContextThreadMain

e19810f

awforsythe marked this pull request as ready for review March 11, 2026 16:15

awforsythe requested a review from a team as a code owner March 11, 2026 16:15

fuzzybinary approved these changes Mar 13, 2026

View reviewed changes

awforsythe merged commit eff8de3 into main Mar 13, 2026
18 checks passed

awforsythe deleted the aforsythe/RUM-13164/context-queue branch March 13, 2026 18:56

awforsythe mentioned this pull request Apr 8, 2026

Integrate dd-win-prof profiling feature into dd-sdk-cpp #185

Open

Uh oh!

Conversation

awforsythe commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example per-thread activity on StartView

Profiling results

Uh oh!

awforsythe commented Mar 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

awforsythe Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

awforsythe Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot commented Mar 10, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

awforsythe commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

awforsythe Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

awforsythe commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector Bot commented Mar 11, 2026

Uh oh!

fuzzybinary left a comment

Choose a reason for hiding this comment

Uh oh!

fuzzybinary Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

fuzzybinary Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

awforsythe Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

awforsythe commented Mar 6, 2026 •

edited

Loading

datadog-official Bot commented Mar 10, 2026 •

edited by datadog-prod-us1-4 Bot

Loading