pearl th-7fdfa9: tmux load-buffer for newlines + ban slash commands in driver LLM - #57
Merged
Merged
Conversation
…n driver LLM Fix two bench harness bugs surfaced by a single-task --debug smoke run. 1. TmuxDriver::send used `send-keys -l TEXT` which interprets every `\n` as the C-j keysym; in literal mode C-j degrades to the bare letter `j`. Multi-line task prompts rendered with `j` where the newlines should have been. Switched to `load-buffer` + `paste-buffer`, which inserts the payload as raw bytes — newlines preserved verbatim. 2. The default driver LLM (smooth-summarize) was emitting Claude-Code- style slash commands (/open, /read, /help) instead of plain English. Hardened the system + user prompts to forbid slash commands, and added a slash-command guard in run_human_loop that drops bad turns, logs them to the pane-debug log, re-asks the model with a reinforcement prompt, and after 3 consecutive bad turns bails with TASK_STUCK. Tests: - send_preserves_newlines_no_j_leakage: 3-line payload through cat>file asserts no stray `j` and exactly 3 lines. - run_human_loop_marks_stuck_after_three_slash_commands. - run_human_loop_accepts_plain_english_message. Verified with smooth-bench score-tui --pr --task-limit 1 --debug against a healthy Big Smooth: zero `j` newline artifacts, zero slash commands in the new pane log.
🦋 Changeset detectedLatest commit: 7ac910c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix two bench harness bugs surfaced by a single-task
score-tui --debugsmoke run against a healthy Big Smooth.jcharacters.TmuxDriver::sendusedtmux send-keys -l TEXT, which interprets every embedded\nas theC-jkeysym; in literal mode (-l)C-jdegrades to the bare letterj. The pearl debug log showed the affine-cipher task prompt rendering asaffine-cipher (python).jjWorking directory:…jFiles present:j - INSTRUCTIONS.mdj…. Switched totmux load-buffer+tmux paste-buffer, which inserts the payload as raw bytes — newlines preserved verbatim.smooth-summarize) was emitting/open,/read,/helpinstead of plain English, and the TUI rejected them as unknown commands (and in two cases accidentally fired skills). Hardened the system + user prompts with explicit "no slash commands; you have no shell/file access" directives, and added a slash-command guard inrun_human_loopthat drops/-prefixed turns, logs them to the pane-debug log, re-asks with a reinforcement prompt, and after 3 consecutive bad turns bails withTASK_STUCK.Pearl: th-7fdfa9.
Tests
tmux_driver::send_preserves_newlines_no_j_leakage— 3-line payload throughcat > tmpfileasserts exactly 3 lines and no strayj.human_driver::run_human_loop_marks_stuck_after_three_slash_commands— fake driver returning/open3x assertsLoopExit::Stuckafter exactly 2 retries.human_driver::run_human_loop_accepts_plain_english_message— fake driver returning plain English asserts zero retries fired.is_slash_command_detects_leading_slash,build_driver_prompt_warns_against_slash_commands,build_slash_retry_prompt_contains_offending_text_and_attempts.All 125 smooth-bench tests pass.
cargo fmt --check,cargo clippy,cargo buildclean.Verification
Re-ran the smoke that originally caught these bugs:
New debug log:
~/.smooth/bench-runs/80c092b0/python-affine-cipher.pane.log. Initial prompt now renders with real newlines:(Zero
jartifacts.) Driver follow-ups are all plain English — sample turns from the log:(Zero
/-prefixed turns.) The single affine-cipher task itself FAILed — expected, it's a hard task on a single attempt. Harness is healthy.Test plan
--prsweep when ready🤖 Generated with Claude Code