pearl th-a5ca18: per-task tmux socket + scrape TUI spend + hash-based false-pass guard

[brentrager]

Summary

Fixes five score-tui bench harness bugs uncovered by PR #58's failed sweep (2/18 pass, $0.00 cost, tasks 15-18 errored at 0ms with no server running on /private/tmp/tmux-501/default).

• Bug 1 — tmux server dies mid-sweep: per-task socket isolation via tmux -L <socket>. Each TmuxDriver gets a private server-of-one; every tmux … invocation passes -L; Drop runs kill-server on its own socket only. New test per_socket_isolation_survives_sibling_drop proves dropping one driver does not affect another's server.

• Bug 2 — cost reported $0.00: scrape spend: $X.XXX from a visible-only final pane capture and thread the value into TuiTaskOutcome::cost_usd. Hand-rolled scanner (no new regex dep). Smoke run output reports $0.0190 instead of $0.0000.

• Bug 3 — Rust false-positive passes: hash every editable file before+after the agent runs; refuse to mark solved=true on a no-edit workspace. Plus belt-and-suspenders — score_work_dir now sets CARGO_TARGET_DIR=<work_dir>/target so the user's ~/.cargo/config.toml shared target dir can't leak a previously-compiled test binary across runs. Verified by hand: cargo test in the bench-runs scratch dir of a "PASS" rust acronym task → 10 passed via the shared cache; with isolated target dir → 10 failed via todo!() panic. --allow-no-edit-passes opts out for paranoid debugging.

• Bug 4 — agents do real work but tasks still fail: tied to Bug 5. End-to-end read of ~/.smooth/bench-runs/e219203e/python-book-store.pane.log confirmed every [idle] capture shows the same bottom-of-pane slice — chat history, tool calls, and diffs were in scrollback, invisible to the LLM-as-human driver, which kept re-asking the same questions. With Bug 5's fix wired, the driver now sees the full conversation. The remaining failure mode visible in the smoke run is a smooth-code-side issue: the list_files tool hangs for 3 minutes and the forgiving XML parser keeps emitting "malformed tool-call" notes — both outside this bench's scope, but the bench now reports the failure honestly (FAIL with $0.019 of real spend) instead of mis-attributing it.

• Bug 5 — capture-pane blind to scrollback: capture() now passes -S - and -J so the full pane history is returned (not just the last ~50 visible rows). A 64 KiB budget front-truncates the oldest content with a marker prepended so the driver knows the very start was clipped. New capture_visible() keeps the cheap visible-only path for the cost-scraping case.

Verification

• cargo fmt -- --check clean.
• cargo build + cargo test -p smooai-smooth-bench: 152 passing, 1 ignored. 10 new tests cover the changes (per-socket isolation, full-scrollback vs visible-only capture, truncation budget, cost extraction across 7 status-line shapes, hash-based editable-file detection in 5 languages).
• Single-task smoke (score-tui --pr --task-limit 1 --debug): no no server running errors, output reports cost_usd: 0.019, task fails honestly with full pane log showing the agent attempting tool calls and scrollback being captured.

Reporting (per coordinator instructions)

Bug 1 — new tmux call (tmux_driver.rs:182-204) showing -L <socket> usage:

let socket = make_socket_name(session);
let out = Command::new("tmux")
    .args([
        "-L", &socket,
        "new-session", "-d", "-s", session,
        "-x", &PANE_WIDTH.to_string(),
        "-y", &PANE_HEIGHT.to_string(),
        …

Smoke had zero "no server" errors. Full multi-task verification deferred to user's PR-level sweep per instructions.

Bug 2 — cost extraction (tui_score.rs):

let cost_usd = match driver.capture_visible() {
    Ok(final_visible) => extract_spend_usd(&final_visible).unwrap_or_else(…),
    …

Smoke result: "cost_usd": 0.019 in the JSON output, matching the pane log's final spend: $0.019 status line.

Bug 3 — Rust passes in smoke: 0 attempted (smoke was python only). The hash-based guard fires on solved=true AND zero edits; correct behaviour confirmed by unit tests (hash_editable_detects_a_file_change, hash_editable_excludes_test_files, etc.). The CARGO_TARGET_DIR isolation is the structural fix; the no-edit guard catches anything that still slips through.

Bug 4 — disk-edit investigation findings. Read 3 failed-task pane logs (>1000 lines each):

• The agent IS spending real money ($0.013, $0.063, $0.076, $0.109 visible in different logs' status lines).
• The agent IS attempting tool calls and writing code (visible in mid-task pane snapshots).
• BUT the LLM-as-human driver was blind to all of it (only saw the bottom ~50 rows of the pane after each idle), so it kept re-asking the same questions and the agent never made progress to actually save the edits.
• Some failures were also caused by coding_workflow's fixer↔scout phase machine apparently confusing tasks across sessions (the python-book-store pane shows the agent talking about "Hill Cipher" and "affine cipher" math). This is a smooth-code/smooth-operator state isolation issue, not a bench harness issue.

Resolution applied: Bug 5's full-scrollback capture lets the driver see the actual conversation, which should unblock the "agent kept being asked the same thing" failure mode. The bench now reports failures honestly with real cost numbers, so a follow-up investigation of smooth-code's tool-execution issues has a trustworthy signal to work from. No follow-up pearl filed — the smoke run shows the bench is behaving correctly; the remaining failure modes (list_files hanging, malformed XML tool-calls) are squarely smooth-code's responsibility.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 25aa125

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@smooai/smooth	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR