Skip to content

Fix tmux dashboard resilience#239

Merged
TraderSamwise merged 15 commits into
masterfrom
fix/tmux-dashboard-resilience
Jun 24, 2026
Merged

Fix tmux dashboard resilience#239
TraderSamwise merged 15 commits into
masterfrom
fix/tmux-dashboard-resilience

Conversation

@TraderSamwise

@TraderSamwise TraderSamwise commented Jun 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • make dashboard relinking non-destructive so stale dashboard slots do not kill or hide live agent windows
  • serialize restart/repair work with a global lock and timeout guarded TUI repairs so repair overlays cannot hang forever
  • harden the tmux dashboard fast path with project/build/owner validation and preserve client identity on reload fallback

Verification

  • yarn typecheck
  • yarn lint
  • yarn vitest --run
  • yarn build

Summary by CodeRabbit

  • New Features
    • Enhanced dashboard reload/open controls with additional client-session and client-tty options.
  • Bug Fixes
    • More reliably identifies valid tmux client sessions and ignores malformed ones across dashboard focusing, runtime reporting, and repairs.
    • Improved live dashboard target validation and dashboard relinking/recovery, including persistent build-stamp behavior.
    • Added hard timeout handling for runtime repair flows to prevent hangs and surface failures.
    • Improved restart coordination with safer concurrency and lock handling.
  • Tests
    • Updated and expanded coverage for the new session-identification and timeout behaviors.

@vercel

vercel Bot commented Jun 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
app Ready Ready Preview, Comment Jun 24, 2026 6:51am

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@TraderSamwise, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 13 minutes and 26 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fbef9aa6-c976-4868-a9bd-47b41e044e7a

📥 Commits

Reviewing files that changed from the base of the PR and between fa0d9ae and ec0c102.

📒 Files selected for processing (6)
  • src/multiplexer/dashboard-control.test.ts
  • src/multiplexer/dashboard-control.ts
  • src/runtime-restart.test.ts
  • src/runtime-restart.ts
  • src/tmux/runtime-manager.test.ts
  • src/tmux/runtime-manager.ts
📝 Walkthrough

Walkthrough

This PR tightens tmux client-session matching to exact -client-<8-hex> suffixes, updates dashboard targeting to stamp build metadata on sessions, refactors dashboard slot linking and restart handling, adds a repair timeout, and forwards client context through dashboard reload paths.

Changes

Tmux client sessions and dashboard lifecycle hardening

Layer / File(s) Summary
Session name helpers and client-session matching
src/tmux/session-names.ts, src/tmux/runtime-manager.ts, src/metadata-server.ts, src/runtime-coherence.ts, src/multiplexer/runtime-guard.ts, src/tmux/doctor.ts, src/dashboard/targets.ts, src/main.ts, src/meta-dashboard-model.test.ts, src/multiplexer/runtime-guard.test.ts, src/runtime-coherence.test.ts, src/metadata-server.test.ts
Strict client-session helper predicates replace prefix-based matching across session selection, runtime reporting, guard probing, and related tests.
Dashboard build stamp on sessions
src/dashboard/targets.ts, src/metadata-server.ts, src/dashboard/targets.test.ts, src/metadata-server.test.ts
Dashboard resolution now writes @aimux-dashboard-build at the session level before returning or reopening dashboard targets, and tests assert the session-level stamp.
Dashboard slot linking and unlinking
src/tmux/runtime-manager.ts, src/tmux/runtime-manager.test.ts
ensureLinkedWindow now uses link-window, swap-window/move-window, and unlink-window with updated slot and failure handling, with tests updated for malformed clients and slot replacement cases.
Filesystem restart lock and relink flow
src/runtime-restart.ts, src/runtime-restart.test.ts
Restart execution is wrapped in a filesystem lock with stale-lock handling, and dashboard relinking/restore flow now runs around the locked restart path with updated tests.
Runtime guard repair timeout
src/multiplexer/dashboard-control.ts, src/multiplexer/dashboard-control.test.ts
The repair flow now times out, sends SIGTERM, escalates to SIGKILL after a grace period, and tracks whether the repair lock is kept or released.
Dashboard reload context forwarding
scripts/tmux-control.sh, src/main.ts, src/tmux/control-script.test.ts
Dashboard reload now forwards client tty and current client session context, and the shell and CLI tests were updated to match the new arguments and session identifiers.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • TraderSamwise/aimux#126: Both PRs extend tmux client-target handling by threading explicit client context through dashboard opening and target resolution paths.
  • TraderSamwise/aimux#234: Both PRs modify src/multiplexer/dashboard-control.ts in the runtime-guard repair flow.
  • TraderSamwise/aimux#237: Both PRs change TmuxRuntimeManager.ensureLinkedWindow and related dashboard slot placement behavior.

Poem

🐇 I hop through sessions, neat and bright,
With eight-hex tails that match just right.
A build stamp glows on dashboard air,
And locks keep restarts from a pair.
I nibble bugs with careful cheer—
ثم hop away, the path is clear!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and clearly reflects the main theme of the changes: hardening tmux dashboard resilience.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/tmux-dashboard-resilience

Comment @coderabbitai help to get the list of available commands.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/multiplexer/dashboard-control.ts (1)

321-337: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Stop success verification once the timeout has settled the repair.

If the timeout fires while probeRuntimeGuard() is awaiting, succeed() can resume and still call refreshDashboardModelThroughApi() before the final settled check. Bail after each await so a timed-out repair cannot keep mutating dashboard state.

Proposed fix
     const probed = await probeRuntimeGuard(projectRoot);
+    if (settled) return;
     if (probed.kind !== "ok") {
       fail(`aimux repair completed but the control plane is still ${describeRuntimeGuardState(probed)}`);
       return;
     }
-    if (
-      isDashboardLifecycleCurrent(host, lifecycle) &&
-      !(await refreshDashboardModelThroughApi(host, { force: true, lifecycle }))
-    ) {
-      fail("aimux repair completed but dashboard data is still unavailable");
-      return;
+    if (isDashboardLifecycleCurrent(host, lifecycle)) {
+      const refreshed = await refreshDashboardModelThroughApi(host, { force: true, lifecycle });
+      if (settled) return;
+      if (!refreshed) {
+        fail("aimux repair completed but dashboard data is still unavailable");
+        return;
+      }
     }

Also applies to: 355-364

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/multiplexer/dashboard-control.ts` around lines 321 - 337, The succeed
flow in dashboard-control.ts can continue after the repair timeout has already
settled the operation, because it only checks settled before and after the full
sequence. Update succeed() to bail out immediately after each await (after
probeRuntimeGuard() and before/after refreshDashboardModelThroughApi()) so a
timed-out repair cannot call refreshDashboardModelThroughApi() or otherwise
mutate dashboard state; use the existing settled guard and
clearRepairTimeout/settled handling in the succeed path, and apply the same
pattern to the related timeout-guarded logic around the referenced repair flow.
src/dashboard/targets.ts (1)

82-85: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Backfill the session build stamp before returning a live dashboard.

The new shell validation reads @aimux-dashboard-build from the host session, but this early return skips the Line 107 setSessionOption(...). Existing dashboards that were already live before this change will keep failing validation until something forces a reload.

Suggested fix
   if (!options.forceReload) {
     const live = findLiveDashboardTarget(projectRoot, tmux);
-    if (live) return live;
+    if (live) {
+      tmux.setSessionOption(live.dashboardSession.sessionName, "`@aimux-dashboard-build`", dashboardBuildStamp);
+      return live;
+    }
   }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/dashboard/targets.ts` around lines 82 - 85, The early return in
findOrCreateDashboardTarget skips backfilling the host session build stamp for
already-live dashboards, so shell validation keeps failing on existing sessions.
Before returning the live target from findLiveDashboardTarget, make sure the
session option is initialized by calling setSessionOption with
`@aimux-dashboard-build` on the live session, while preserving the existing
forceReload behavior in findOrCreateDashboardTarget.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/tmux-control.sh`:
- Around line 206-207: The project-root validation in tmux-control.sh is
checking the wrong session metadata, which can reject valid client dashboards.
Update the target_project_root lookup in the validation flow to use the derived
host session variable from the earlier normalization logic, specifically the
validate_host_session path in the same block, so the `@aimux-project-root` check
reads from the host session instead of the original client session.

In `@src/runtime-restart.ts`:
- Around line 148-165: The steal-lock acquisition helper leaves an empty lock
directory behind if writing the owner file fails, which can block later restarts
until the stale timeout. Update tryAcquireRuntimeRestartStealLock so both
acquisition branches clean up the created stealPath when
writeFileSync(joinLockOwnerPath(...)) throws, and make sure the stale-lock retry
path in the same function applies the same cleanup before returning null.

In `@src/tmux/runtime-manager.ts`:
- Around line 386-389: The cleanup in `RuntimeManager`’s link/move failure path
is too broad and can unlink a pre-existing window that was not created by this
call. Update the catch block around the `move-window`/`swap-window` flow so it
only removes links that were newly created during the current operation, using
the relevant link-tracking variables in `runtime-manager.ts` (for example the
`linked` state and the window-linking logic around the catch path). Ensure
previously existing links in `clientSessionName` are preserved when repair
fails.

---

Outside diff comments:
In `@src/dashboard/targets.ts`:
- Around line 82-85: The early return in findOrCreateDashboardTarget skips
backfilling the host session build stamp for already-live dashboards, so shell
validation keeps failing on existing sessions. Before returning the live target
from findLiveDashboardTarget, make sure the session option is initialized by
calling setSessionOption with `@aimux-dashboard-build` on the live session, while
preserving the existing forceReload behavior in findOrCreateDashboardTarget.

In `@src/multiplexer/dashboard-control.ts`:
- Around line 321-337: The succeed flow in dashboard-control.ts can continue
after the repair timeout has already settled the operation, because it only
checks settled before and after the full sequence. Update succeed() to bail out
immediately after each await (after probeRuntimeGuard() and before/after
refreshDashboardModelThroughApi()) so a timed-out repair cannot call
refreshDashboardModelThroughApi() or otherwise mutate dashboard state; use the
existing settled guard and clearRepairTimeout/settled handling in the succeed
path, and apply the same pattern to the related timeout-guarded logic around the
referenced repair flow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c9ea20d6-75c3-4c9e-8185-650073f7d9fe

📥 Commits

Reviewing files that changed from the base of the PR and between 68cfbcd and 24728f6.

📒 Files selected for processing (20)
  • scripts/tmux-control.sh
  • src/dashboard/targets.test.ts
  • src/dashboard/targets.ts
  • src/main.ts
  • src/meta-dashboard-model.test.ts
  • src/metadata-server.test.ts
  • src/metadata-server.ts
  • src/multiplexer/dashboard-control.test.ts
  • src/multiplexer/dashboard-control.ts
  • src/multiplexer/runtime-guard.test.ts
  • src/multiplexer/runtime-guard.ts
  • src/runtime-coherence.test.ts
  • src/runtime-coherence.ts
  • src/runtime-restart.test.ts
  • src/runtime-restart.ts
  • src/tmux/control-script.test.ts
  • src/tmux/doctor.ts
  • src/tmux/runtime-manager.test.ts
  • src/tmux/runtime-manager.ts
  • src/tmux/session-names.ts

Comment thread scripts/tmux-control.sh Outdated
Comment thread src/runtime-restart.ts Outdated
Comment thread src/tmux/runtime-manager.ts Outdated
@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

Fixed the final independent reviewer findings in 8994bc4: dashboard relink fallback now fails if it cannot put the dashboard in slot 0, and timed-out repair children now escalate from SIGTERM to SIGKILL after a bounded grace period so the repair lock cannot be held indefinitely.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

Fixed the latest independent reviewer finding in fa0d9ae: SIGKILL no longer releases the repair lock immediately; the lock remains held until the child exit event or later dead-owner reclaim.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/multiplexer/dashboard-control.test.ts (1)

841-872: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Assert the owner PID liveness probe.

This test would still pass if stale repair locks were never reclaimed. Since the contract is “do not reclaim while the recorded owner is alive,” assert that the mocked process.kill probe is made before restoring the spy.

Proposed test tightening
     try {
       startRuntimeGuardRepair(host as never, { kind: "stale", reason: "service-mismatch" });
+      expect(killSpy).toHaveBeenCalledWith(987654, 0);
     } finally {
       killSpy.mockRestore();
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/multiplexer/dashboard-control.test.ts` around lines 841 - 872, The test
for startRuntimeGuardRepair currently only checks the final state, so it can
pass even if the owner-liveness check is never exercised. Tighten the assertion
by verifying the mocked process.kill probe is called with the recorded owner PID
from owner.json before restoring the spy, and keep the existing expectations
that spawn is not called and the lock remains in place. Use the existing
startRuntimeGuardRepair, killSpy, and owner.json setup to ensure the “owner
still alive” path is actually covered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime-restart.ts`:
- Around line 299-308: The append fallback in runtime-restart should not leave a
dashboard window behind when slot-0 relinking fails. Update the try/catch around
tmux.linkWindowToSession in runtime-restart.ts so that if the fallback link
succeeds at a non-zero index or throws, the newly linked dashboard window is
removed or moved back before recording the error. Use the existing sessionName,
dashboardTarget, linked.windowIndex, and errorMessage handling to locate the
logic and ensure failed relinks do not pollute the client session.

---

Nitpick comments:
In `@src/multiplexer/dashboard-control.test.ts`:
- Around line 841-872: The test for startRuntimeGuardRepair currently only
checks the final state, so it can pass even if the owner-liveness check is never
exercised. Tighten the assertion by verifying the mocked process.kill probe is
called with the recorded owner PID from owner.json before restoring the spy, and
keep the existing expectations that spawn is not called and the lock remains in
place. Use the existing startRuntimeGuardRepair, killSpy, and owner.json setup
to ensure the “owner still alive” path is actually covered.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e216abfd-0219-4901-a475-e85e81db59f9

📥 Commits

Reviewing files that changed from the base of the PR and between 24728f6 and fa0d9ae.

📒 Files selected for processing (10)
  • scripts/tmux-control.sh
  • src/dashboard/targets.test.ts
  • src/dashboard/targets.ts
  • src/multiplexer/dashboard-control.test.ts
  • src/multiplexer/dashboard-control.ts
  • src/runtime-restart.test.ts
  • src/runtime-restart.ts
  • src/tmux/control-script.test.ts
  • src/tmux/runtime-manager.test.ts
  • src/tmux/runtime-manager.ts
🚧 Files skipped from review as they are similar to previous changes (7)
  • scripts/tmux-control.sh
  • src/dashboard/targets.ts
  • src/multiplexer/dashboard-control.ts
  • src/tmux/runtime-manager.ts
  • src/tmux/runtime-manager.test.ts
  • src/runtime-restart.test.ts
  • src/tmux/control-script.test.ts

Comment thread src/runtime-restart.ts Outdated
@TraderSamwise

Copy link
Copy Markdown
Owner Author

Fixed the latest independent reviewer finding in 2143c22: dashboard repair stale-lock reclamation now uses a separate reclaim lock and revalidates the owner before deleting the main repair lock.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

Sub-agent review finding resolved in c90068f: TmuxRuntimeManager.unlinkWindow no longer targets mutable window indexes. It now unlinks by stable session:@windowId, with tests updated around stale dashboard cleanup.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

Sub-agent review findings resolved in ec0c102: dashboard relink move/swap now targets stable session:@windowId instead of a previously observed index, and newly linked dashboards are unlinked if post-replacement verification fails.

@TraderSamwise

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@TraderSamwise
TraderSamwise merged commit e51857e into master Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant