examples/codemode: put a human in front of the agent's writes - #21
Draft
AntoniTok wants to merge 13 commits into
Draft
examples/codemode: put a human in front of the agent's writes#21AntoniTok wants to merge 13 commits into
AntoniTok wants to merge 13 commits into
Conversation
added 12 commits
July 28, 2026 16:37
The agent could write to the workspace on its own say-so. Now a command the approval policy holds back does not run: the turn stops, the pending command goes on a queue, and it resumes only once a human answers. The gate is the AI SDK's own needsApproval on the exec tool, which excludes a call from execution and reports it as an approval request instead, so there is no provisional result and nothing to undo. A paused turn resumes by replaying the stored message history with a tool-approval-response attached. A rejection comes back to the model as execution-denied, which it can read and react to rather than an error that ends the turn. Approval is a table keyed by backend, because the backends differ in what they can reach. The two sandboxed backends run recognized reads unattended, while the container backend, with a full userland and a public network, is gated outright. Anything the matcher cannot classify needs a human, so an unknown verb or a state call reached through a computed access fails closed. Matching command strings is a heuristic suited to an example, and enforcement belongs at the capability layer, which is precisely why denying by default matters here. The state a pause needs outlives the request that created it, so it lives in AgentSession, a second durable object. The object that owns the filesystem stays a filesystem, holding no model state, and the object that records approval decisions holds no workspace stub and so cannot run a command. Two details follow from resuming by replay. The step budget now spans a whole turn rather than a single pass, so waiting for a human does not hand the model a fresh allowance. The transcript is collected from the tool rather than from the model's steps, because a command a human approved executes before the resumed pass makes its first model call and therefore belongs to no step.
Describe the pause and resume path, the per-backend policy table and the deny-by-default matcher behind it, and where a paused turn lives while it waits. Record why the example reaches for the AI SDK's approval hook rather than the approvals that ship with @cloudflare/codemode: the codemode backend here and the codemode runtime share a name without sharing a mechanism, and adopting the latter would mean giving up the backend choice this example exists to show. The smoke script gains an opt-in APPROVALS=1 pass that asks the agent for a write, checks the file is still absent while the approval waits, approves it, and checks that the write then landed. The absence is the part worth asserting.
The approval matcher allowlisted verbs but only blocklisted flags, so `find /workspace -mindepth 1 -delete` was classified as a read and ran unattended. The verb was on the read list, the line held no redirection or composition, and nothing looked further. Flags on a verb that can write are now allowlisted as well, and an unrecognized one gates. That is the shape the state.* check already used: name what is allowed and treat the rest as a question. A blocklist of writing flags would have to keep pace with every flag that happens to write, which is the same losing game the shell metacharacter list plays and wins only because that list is closed. Verbs whose writes cannot be read off their arguments are dropped from the read set rather than papered over. sed writes through -i and through a `w` command inside its script, uniq and tree take an output file as a positional argument, and date -s sets the clock. Listing them buys false confidence, not fewer approvals.
The HTTP surface answers an approval with a second request, which suits a UI but makes the feature hard to see: a turn pauses by returning JSON, and approving means copying an id into another curl. Nothing about that reads like a human being asked a question. script/agent closes the loop. It starts a turn and, each time the turn comes back waiting, prints the command with the reason it was held and asks y/n on the terminal, then posts the answer and picks the turn up where it left off. It defaults to a fresh workspace so the first write always has to be approved, takes AUTO_APPROVE=1 for a hands-off run, and accepts piped answers for scripted ones. It reads stdin a line at a time rather than through node:readline, because a piped answer arrives and ends the stream while the first model call is still in flight, and readline then rejects the question that follows.
The read-only rule gated every composed command, so listing files with `ls -1 /workspace | wc -l` stopped for a human. Both stages are reads and a pipe touches no files, which makes that a question with no useful answer, and asking it teaches whoever works the queue to stop reading the prompt. The reason the rule gated all composition was that classifying a composed line means classifying every element of it. That is worth doing rather than avoiding. Redirection, substitution, subshells and backgrounding still disqualify a line outright, since they either write or run something the matcher would have to parse to see. What is left is a pipeline, which now splits on its separators and passes only when every stage passes on its own. Nothing is given up by this. `ls; rm -rf /workspace` stops on its second stage, and `find /workspace -type f | xargs rm` stops on `xargs` even though that `find` would have passed alone. Add echo and printf to the read set while here. They were held back on the grounds that they write through their own syntax, which is true of awk and not of them: both write to stdout only, and aiming that at a file takes a redirect, which gates the line whatever the verb is.
The architecture page showed what the pieces are but described the calls between them as two prose lists, which is the wrong shape for explaining a flow that pauses in the middle. A reader following the approval path had to hold five participants and an ordering in their head at once. Add a sequence diagram per path, sharing lanes and opening rows so the two can be read against each other. The unattended turn and the gated turn are identical up to the policy check and diverge there, which puts the whole feature in one visual difference: one path reaches the Durable Object that owns the files, and the other stops short of it and parks the turn. Mark the pause explicitly, since that band is what the design is for and it is the part a prose list cannot show — the Worker holds no state across it and can die there without losing the turn. Note also that the policy check appears twice on the gated path, which is the reason it has to be a pure function.
The sequence diagrams walk one path each and so leave out everything that makes the route awkward to hold in the head: where the two entry points meet, which branch returns which status, and the fact that the route feeds back into itself. Add a control-flow diagram for it. The shape worth seeing is that POST /agent and POST /approvals/<id> converge on the same model loop, so a resumed turn can stop on a different command and hand back another awaiting-approval rather than a completion. Draw the loop as a loop. Put the gate inside the Worker box, upstream of the Durable Object that owns the files, since that placement is the reason a paused turn is safe to leave sitting: the command was never sent, so there is nothing in flight to keep alive. Note alongside it why an already-answered approval is a 404 and a partially-answered turn is a 202. Both look like failures next to the 200 that resumes a turn, and the difference is that answering twice would run the command again. Give the diagram its own arrow marker and label backing rather than borrowing the ones defined in the sequence diagram above it. Sharing them works, because an SVG style block applies document-wide, but it leaves the arrowheads silently dependent on a sibling figure staying where it is.
The page has grown four diagrams and is the quickest way to see how a turn moves through the Worker, the two Durable Objects and the model, but nothing referenced it, so finding it depended on browsing the directory. Point at it from the README, near the paragraph that introduces the approval flow it illustrates. Give the reader the open command as well. The file renders nowhere on GitHub, so a bare link invites reading seven hundred lines of hand written SVG instead of the picture it draws.
The gate sits in the exec tool, which puts it on the model's path and not on this route's. That is deliberate — approval exists to put a human in front of a command a model chose, and a caller posting here is already that human — but the consequence is easy to miss while reading the approval code, because nothing near the gate mentions that a second caller reaches the same method unchecked. Write it down at the route, including that this one will run `rm -rf /workspace` on request. Note also that it follows from where enforcement currently sits rather than from a decision to privilege this route, since that placement is the part of the design most likely to move.
The existing policy tests assert what the matcher says, which cannot find the defect that matters. The matcher guesses whether a command writes by reading its text, and the same hand writes the guess and the examples it is checked against, so both miss the same cases. That is not a hypothetical: `find -mindepth 1 -delete` ran unattended and a pipeline of two reads asked for approval, and neither turned up in a suite of hand-picked assertions. Both were found by running the agent and watching. Check the claim against the world instead. Generate a corpus by crossing every allowlisted verb with argument shapes that include the flags known to turn a read into a write, keep the commands the policy would run unattended, and execute each one under real just-bash against a filesystem that records every mutating call. The property runs one way only: a command allowed unattended must write nothing. A gated read is a nuisance and needs no test. Take the verbs from READ_ONLY_COMMANDS rather than a copy of it, so widening the policy later brings the new verb under test without anybody remembering to come here. Nonsense combinations are kept deliberately — a command the matcher allows must not write whether or not it makes sense. Prove the harness before trusting it. Four cases assert the recorder sees a write, sees a delete reached through find, stays quiet on a read, and fails when handed a policy that waves a write through. Without those, a recorder wired up wrong would make every command look like a read. Reintroducing the find hole fails the suite naming each file that would have been deleted. The shell is real but the storage is not, which is sound here: whether find reaches for a delete is a fact about just-bash and holds wherever its files live. Two gaps are written down rather than left to be discovered — the container backend runs GNU coreutils and can differ, which is another argument for gating it outright, and the codemode dialect is not exercised because reaching a live state namespace would drag workerd-only imports into this runner.
The page had the shape of a gated turn in two diagrams and a five-line prose summary, which is the right amount for deciding whether the design is sound and not enough for working on it. Questions the diagrams cannot answer: where the gate is evaluated, what the SDK leaves behind when it declines to run something, why the policy is consulted a second time, and what the resume has to put back. Add a section that walks it at the level of what each line does, split into the two sides of the pause. Splitting it is the point — a paused turn is two requests with an indefinite gap between them, and the things that are easy to get wrong sit either side of that gap: the first side has to leave enough behind to resume from, and the second has to reconstruct it without trusting the client. Call out the parts that read as surprises. The pause is discovered by scanning the last step's content after generateText returns normally rather than by catching anything. The gate runs again on resume and can downgrade an approval that has gone stale. An approved command executes in the pre-loop and belongs to no step, which is why the transcript comes from a closure instead of result.steps. Link it from the README next to the diagram it expands on.
set-versions.mjs rewrites the pinned wsd-linux-x64 image tag in the example Dockerfiles so a clone at any release tag pulls the matching image. The codemode example was never added to that list, though its Dockerfile carries the same pin and the same comment claiming the tag moves in lockstep. Nothing is broken today because the pin happens to match the current version. It would have gone stale at the next release, leaving that one example pulling an older wsd than everything around it, and the comment in the file would have argued it could not happen.
commit: |
The policy documentation treated the backend table and the read detection as one mechanism and then disowned both, which left the example arguing against its own centerpiece. They are not the same kind of thing. The backend table decides on what a backend can reach and needs no parsing to apply, so it is the boundary. The read detection classifies a command by reading it, fails closed, and exists to ask fewer questions. Say so in the README and in approval-policy.ts, and name approval-policy.effects.test.ts as what holds the second tier to its one-directional guarantee. Record why the mechanism is what it is. The pause is the AI SDK's needsApproval, the same field Think gates its own tools with. Think Actions, the Agents SDK, Workflows, fibers and the codemode runtime each cost more here for a different reason, and a reader a year from now will want that written down rather than rediscovered. Deciding which shell commands are safe has no equivalent in any of them, which is the part of this example that is actually new. Note that experimental_toolApprovalSecret stays off. Approval requests in the replayed message history carry no signature, so the setting throws. Nothing depends on it while the history stays on the server, and GET /agent/<turnId> strips it before answering. Cut the comment above handleExec to the distinction it was reaching for. Approval asks whether a person has seen a command the model chose. That route has no authorization check, which is a separate question with a separate fix.
Author
|
Notes on the approval design, for anyone reviewing the choice rather than the diff. The README now carries the durable version of this; the detail below is here so it does not have to live in the tree. The mechanism is the common one. The pause is the AI SDK's Behaviorally, What has no equivalent is the matcher. Think's built-in Bash tool runs on the same The capability argument has a precedent. Think protects files it did not mount during write-back, so a script cannot delete what it was never given. That is enforcement below the shell rather than in front of it, and it is the direction both open gaps close in. Alternatives weighed, and what each would costThink — closest fit, and it is a whole chat agent framework: memory, streaming, messengers, scheduled turns. Adopting it for the approval plumbing would replace what this example demonstrates. Agents SDK — Workflows — built for waits measured in months with per-step retries. This pause is minutes and holds nothing open, so a stored row is the smaller mechanism, and Fibers — codemode runtime — gates connector calls inside model-generated code and resumes by abort-and-replay. Different shape, and the name collides with the The two sides of a gated turnsequenceDiagram
participant H as human
participant W as Worker
participant M as Workers AI
participant S as AgentSession
participant D as workspace
H->>W: POST /agent { prompt }
W->>M: generateText
M-->>W: exec { command, backend }
Note over W: needsApproval → true<br/>call dropped before execution
W->>S: saveTurn (messages, pending, stepsUsed)
W-->>H: awaiting-approval + turnId + approvalId
Note over W,D: nothing ran: no connection,<br/>no dynamic worker, no container
sequenceDiagram
participant H as human
participant W as Worker
participant M as Workers AI
participant S as AgentSession
participant D as workspace
H->>W: POST /approvals/{id} { approved }
W->>S: resolveApproval
S-->>W: turn + ready + answers
W->>M: generateText (history + approval response)
Note over M: policy re-checked;<br/>a stale approval becomes a denial
M->>D: exec runs
D-->>M: result
M-->>W: text
W->>S: saveTurn (completed)
W-->>H: transcript
Every branch of POST /approvals/<approvalId>flowchart TD
A["POST /approvals/{id}"] --> B{"id outstanding?"}
B -- "no: unknown, or already answered" --> C["404 — nothing changed"]
B -- yes --> D["record the decision"]
D --> E{"any approvals left<br/>on this turn?"}
E -- yes --> F["202 — keep answering"]
E -- no --> G["resume the turn in this request"]
G --> H{"paused again?"}
H -- yes --> I["200 awaiting-approval"]
H -- no --> J["200 completed"]
|
Contributor
|
@AntoniTok thanks for opening this PR. It's pretty dense I need to sit with it once we've got the main codemode backend out of the door. In my head the key thing workspace needed to do was to provide a hook within an action with metadata that allowed a gate to be implemented in the tool layer, plus a hook after the action was completed for auditing. Similar to our observability implementation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The codemode example lets a model pick which of three backends runs each command, and until now it could write to the workspace on its own say-so. An example built to show a model driving real backends ought to show how you stop it, so this puts a person in front of anything that is not recognizably a read.
The check hangs off the
exectool rather than the workspace, which keeps the workspace unaware that a model exists. The tool declaresneedsApproval, and that consults a table keyed by backend. The two sandboxed backends let recognized reads run unattended. The container backend gets a full Linux userland and public network access, so it asks about everything. Anything the matcher cannot classify asks as well: failing closed turns an unfamiliar command into a question, while failing open turns it into an unreviewed write.A held-back command does not run at all — no shell, no dynamic worker, no container boot. The turn stops, and the Worker files the conversation so far into a second Durable Object,
AgentSession, keyed by the same workspace name. That object holds no workspace handle on purpose, so the thing remembering "nobody approved this yet" cannot itself run anything. Nothing is held open across the pause, so the Worker can be redeployed or crash and the turn survives as a row.sequenceDiagram participant H as human participant W as Worker participant M as Workers AI participant S as AgentSession participant D as workspace H->>W: POST /agent W->>M: run the turn M->>W: exec { command, backend } W->>W: policy says a person must see this W->>S: store the paused turn W->>H: awaiting-approval (nothing ran) H->>W: POST /approvals/id W->>S: read the turn back W->>M: replay with the answer W->>D: now run the command W->>H: completedAnswering the last outstanding approval resumes the turn in that same request. Answering one of several returns
202until the rest arrive, and answering one twice returns404rather than running the command again.To try it, run
npx wrangler dev --enable-containers=false, then./script/agent "Create /workspace/greeting.txt containing exactly: hello world"from the example directory. It prompts with the command, the backend, and why it stopped. Ask for a read instead, such as counting the files under/workspace, and it goes straight through.APPROVALS=1 ./script/runcovers the same ground without a person.Three test suites are ordinary: the policy decisions, the bookkeeping for a paused turn, and the pause-and-resume loop against a scripted model, which asserts a held-back command never reaches the workspace.
The fourth is the interesting one. The matcher guesses whether a command writes by reading its text, and the same hand writes the guess and the examples it gets checked against, so both miss the same cases. That is not hypothetical:
find /workspace -mindepth 1 -deleteran unattended, because the verb was on the allowed list and its flags were not. Soapproval-policy.effects.test.tschecks the guess against the world. It crosses every allowed verb with argument shapes that include the flags known to turn a read into a write, keeps the commands the policy would run unattended, and executes each under realjust-bashagainst a filesystem that records every change — 630 commands, 475 of them allowed, in under a second. Put thefindhole back and it fails naming each file that would have been deleted. The verbs come from the allowed list itself rather than a copy, so widening the policy later brings the new verb under test without anyone remembering to.What that failure looks like
The example README covers the approval flow, and its architecture page gains diagrams for the call sequence and for every branch of the approval route. That page is a standalone file with no build step and does not render on GitHub, so it is worth opening locally rather than reading in the diff.
The pause is not homegrown. It is the AI SDK's
needsApproval, one field on theexectool, and the same field Think gates its own tools with; Think Actions parks a turn on a human and resumes it later much as this does. Think, the Agents SDK, Workflows, fibers and the codemode runtime were each weighed and each costs more here than it saves, and the README says why for every one of them. What none of them answer is which shell commands are safe to run unattended, and that is the part of this change that is new.Two gaps remain. The matcher guesses a command's effect from its text, so it is the policy's second tier rather than its boundary — the backend table is the boundary, and the effects test is what holds the matcher to failing in one direction only. Separately,
POST /exechas no authorization check. Both close the same way: give the backend a read-only view of the workspace and let the filesystem refuse the write, which covers every caller instead of only the model's path and leaves the matcher as a way to ask fewer questions.