[Bug]: Idle session reaper kills in-flight background agent work (dynamic workflows / subagents)

[ChamaruAmasara]

Before submitting

• I searched existing issues and did not find a duplicate.
• I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

1. Start a long-running background agent task in a thread — a dynamic workflow or a task that fans out subagents that keep working after the foreground turn settles.
2. Let the foreground turn complete so the thread has no active user turn, while the background work keeps running.
3. Leave the thread otherwise idle (no new sendTurn) for longer than the reaper's inactivity threshold (default 30 min; swept every 5 min).
4. Observe that ProviderSessionReaper calls stopSession for the thread and tears down the provider session while the background agents are still in flight.

Expected behavior

A session with live background agent work should not be reaped for inactivity. The reaper already skips sessions with an active foreground turn; the same protection should extend to in-flight background work.

Actual behavior

ProviderSessionReaper (apps/server/src/provider/Layers/ProviderSessionReaper.ts) stops any provider session after 30 min of last_seen_at inactivity. Its only busy-guard is the projection snapshot's activeTurnId. When the foreground turn completes, the adapter session goes ready with no active turn, and nothing refreshes last_seen_at from ongoing background activity. So a long-running background workflow with no foreground turn is torn down mid-run. The next interaction resumes the conversation from a transcript cursor as a fresh process, but the in-flight background orchestration (subagent work, queued steps) is gone, leaving the run half-completed.

Impact

Major degradation or frequent failure

Version or commit

main at 5d34f9ff2

Environment

T3 Code server, running background dynamic workflows / subagents.

Logs or stack traces

No exception is thrown. The reaper logs provider.session.reaped with reason: "inactivity_threshold" for a thread whose adapter session is still running background work.

Screenshots, recordings, or supporting files

A deterministic regression test is included in the linked fix.

Workaround

Send a message in the thread periodically (every <30 min) to refresh last_seen_at and keep the session alive while the background work runs.

Suggested fix

Refresh last_seen_at from runtime activity: bump the binding's timestamp when the adapter emits runtime events (e.g. task.progress from background subagents), throttled per thread so an event burst is at most one lightweight write per window. The adapter keeps emitting runtime events during background work even with no active foreground turn, so this keeps the session warm exactly when the reaper would otherwise reap it.

[Mooned8]

Same root cause here, different entry point: a background shell command (deploy watch) instead of subagents. The agent ends its turn saying it will report back, the reaper tears the session down (provider.session.reaped, reason: 'inactivity_threshold'), and on the next user message the session resumes with "No completion record was found for this background shell command from the previous session" — the promised follow-up is silently lost. Filed #4265 before finding this issue; closing it as a duplicate. One addition to the proposal: besides exempting sessions with in-flight background work, delivering the task result by respawning the session on completion would also repair the contract when a reap does happen. Observed on 0.0.28, self-hosted Ubuntu 24.04.