feat(cli): workforce invoke — fixture-replay simulation CLI (#186 P2)#188
Merged
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
CodeAnt AI is reviewing your PR. |
Contributor
|
Warning Review limit reached
More reviews will be available in 43 minutes and 24 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
codeant-ai
Bot
added
the
size:XL
P2 of workforce#186, on top of the simulateInvocation runtime (P1): - packages/cli/src/invoke-command.ts: `agentworkforce invoke <persona-path> --fixture <file>` — preflights the persona (same validation as deploy), stages the agent bundle under `.workforce/invoke-build/` inside the persona's tree (the bundle leaves @agentworkforce/runtime external, so resolution must walk up to a node_modules — same constraint as deploy's build dir), imports the handler with the same extraction the generated runner.mjs performs, and replays the fixture through simulateInvocation. - Fixture formats: single JSON envelope, JSON array, or NDJSON; entries are RawGatewayEnvelope (now exported from the runtime root alongside shimEnvelope). Unknown envelope types surface via the simulation's unsupported list, mirroring the runner's warn-and-continue. - Output: human summary → stderr; machine-readable Cloud-compatible run record (origin "local_dry_run") → stdout or --output <file>. Exit 0 when every dispatched envelope succeeded, 1 on any handler failure or usage/setup error. Sets process.exitCode (never process.exit) so streams flush and tests drive it directly. - Flags: --input KEY=value (persona inputs), --seed PATH=file (seed the simulated VFS with provider data), --workspace <id>. - cli.ts: `invoke` dispatch entry + USAGE block distinguishing it from deploy --dry-run. README: "Simulate an invocation" section covering the command, fixture format, run-record contract, and the dry-run-vs-simulate distinction. - Build dir is cleaned up after every run; bundle import is cache-busted so repeated invokes load the freshly staged bundle. 18 new node:test cases (parse, fixture parsing incl. malformed-line labeling, handler extraction parity, human summary, and 4 end-to-end tests through real preflight + esbuild staging + dynamic import). Full workspace `pnpm run check` green; CLI suite 229/229. Refs #186, AgentWorkforce/cloud#1783 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
Comment on lines
+71
to
+72
| } else if (a.startsWith('--fixture=')) { | ||
| fixturePath = expectInline('--fixture', a.slice('--fixture='.length)); |
There was a problem hiding this comment.
Suggestion: Appending a random query string to every dynamic import creates a unique ESM module cache entry per invocation, which accumulates indefinitely in long-lived processes (tests/watch flows). This causes unbounded memory growth; avoid per-run unique import URLs or isolate execution in a short-lived worker/process. [memory leak]
Severity Level: Major ⚠️
- ⚠️ Long-running `runInvoke` loops leak memory via ESM cache.
- ⚠️ Dev/watch flows using `invoke` may slow or crash over time.Steps of Reproduction ✅
1. In `runInvokeWithOptions` (`packages/cli/src/invoke-command.ts:45-92`), the persona is
preflighted and an agent bundle is staged to a temporary `buildDir` via
`bundleStager.stage` (`packages/cli/src/invoke-command.ts:63-67`), producing
`bundle.bundlePath`.
2. The code then constructs an import URL as `const bundleUrl =
\`${pathToFileURL(bundle.bundlePath).href}?invoke=${randomUUID()}\`;` and dynamically
imports it with `await import(bundleUrl)` (`packages/cli/src/invoke-command.ts:69-72`).
Because `randomUUID()` produces a fresh value each time, every call to
`runInvokeWithOptions` uses a unique module URL even when `bundle.bundlePath` is the same.
3. Node's ESM loader caches modules by their full URL (including query string). In a
long-lived process (for example, a watch-mode dev tool or test harness) that repeatedly
calls `runInvoke` with the same persona and fixture, each invocation stages a bundle and
imports it via a new `bundleUrl`. The ESM cache retains each imported module object
indefinitely, because each URL is unique.
4. Although `buildDir` is deleted in the `finally` block
(`packages/cli/src/invoke-command.ts:104-107`), the already-imported module instances
remain strongly referenced by the ESM module cache. Over hundreds or thousands of
invocations in a single process, this pattern causes monotonically increasing memory usage
from accumulated bundle modules, degrading performance and potentially leading to
out-of-memory conditions in long-running dev/watch flows.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** packages/cli/src/invoke-command.ts
**Line:** 71:72
**Comment:**
*Memory Leak: Appending a random query string to every dynamic import creates a unique ESM module cache entry per invocation, which accumulates indefinitely in long-lived processes (tests/watch flows). This causes unbounded memory growth; avoid per-run unique import URLs or isolate execution in a short-lived worker/process.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Comment on lines
+85
to
+92
| } else if (a === '--seed') { | ||
| addKeyValue('--seed', expectValue('--seed', args[++i]), seeds); | ||
| } else if (a.startsWith('--seed=')) { | ||
| addKeyValue('--seed', expectInline('--seed', a.slice('--seed='.length)), seeds); | ||
| } else if (a.startsWith('--')) { | ||
| throw new Error(`invoke: unknown flag "${a}"`); | ||
| } else if (!personaPath) { | ||
| personaPath = path.resolve(a); |
There was a problem hiding this comment.
Suggestion: --input values are forwarded directly without validating that the keys exist in the persona's declared inputs, so typos are silently ignored during simulation. This breaks parity with deploy input handling and can produce misleading simulation results; validate override keys against preflight.persona.inputs and fail on unknown keys. [api mismatch]
Severity Level: Major ⚠️
- ⚠️ `agentworkforce invoke` ignores misspelled `--input` overrides.
- ⚠️ Simulated runs may diverge from deploy-time input semantics.
- ⚠️ Developers may misdiagnose behavior using stale default inputs.Steps of Reproduction ✅
1. Create a persona JSON with a declared input key, e.g. `TOPICS`, and mark it optional or
give it a default so the persona is accepted without a deploy-time override. This persona
is parsed and validated by `preflightPersona` at `packages/deploy/src/preflight.ts:23-31`,
which populates `preflight.persona.inputs`.
2. Run the new CLI command through the binary defined in
`packages/cli/src/cli.ts:245-251`, e.g. `agentworkforce invoke ./persona.json --fixture
./event.json --input TOPIC=override`, intentionally misspelling the key as `TOPIC` instead
of `TOPICS`.
3. The CLI entry calls `runInvoke` (`packages/cli/src/invoke-command.ts:248-252`), which
parses arguments via `parseInvokeArgs` (`packages/cli/src/invoke-command.ts:57-85`). This
collects the flag into `opts.inputs = { TOPIC: 'override' }` even though `TOPIC` is not
declared on the persona.
4. `runInvokeWithOptions` (`packages/cli/src/invoke-command.ts:45-92`) then calls
`simulateInvocation` with `agent: { inputValues: opts.inputs }` at
`packages/cli/src/invoke-command.ts:85-91`. Inside `simulateInvocation`
(`packages/runtime/src/simulate/simulate.ts:60-88`), `buildCtx` is called, which in turn
calls `buildPersonaContext` (`packages/runtime/src/ctx.ts:447-47x`). `buildPersonaContext`
iterates only `persona.inputs` keys and ignores extra keys on `agentInputValues`, so the
`TOPIC` override is silently dropped. The simulation runs using the default/env value for
`TOPICS` with no error about the unknown override key, even though the CLI help
(`INVOKE_USAGE` at `packages/cli/src/invoke-command.ts:13-41`) and deploy design docs
(`docs/plans/deploy-v1.md:159-164`) state that only declared persona inputs should be
accepted.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** packages/cli/src/invoke-command.ts
**Line:** 85:92
**Comment:**
*Api Mismatch: `--input` values are forwarded directly without validating that the keys exist in the persona's declared inputs, so typos are silently ignored during simulation. This breaks parity with deploy input handling and can produce misleading simulation results; validate override keys against `preflight.persona.inputs` and fail on unknown keys.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
|
CodeAnt AI finished reviewing your PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User description
Summary
P2 of #186, stacked on #187 (base =
feat/invocation-simulation; GitHub retargets tomainwhen #187 merges — I'll rebase if the squash requires it).Adds the developer-facing surface for the P1 simulation runtime:
preflightPersona. Stages the agent bundle with the realbundleStagerunder.workforce/invoke-build/inside the persona's tree — the bundle externalizes@agentworkforce/runtime, so Node must walk up to anode_modules, exactly deploy's build-dir constraint. Build dir cleaned up after every run; bundle import is cache-busted.runner.mjs(defineAgent default export →.handler;{handler}object; bare-function fallback; same error hint).RawGatewayEnvelopes (type now exported from the runtime root withshimEnvelope). Unknown envelope types surface via the simulation'sunsupportedlist (runner's warn-and-continue mirrored).origin:"local_dry_run") → stdout or--output. Exit 0 = all dispatched envelopes succeeded; 1 = any handler failure or usage/setup error. Usesprocess.exitCode, neverprocess.exit.deploy --dry-run(validate-only) vsinvoke(execute-with-recorded-side-effects) distinction. CLIUSAGEupdated likewise.Issue acceptance mapping
logs.stdout+summary+simulation.sideEffects)origin: local_dry_run(P1, exercised end-to-end here)--dry-runTests
18 new
node:testcases: arg parsing (repeatable flags, inline forms, error messages), fixture parsing (object/array/NDJSON + malformed-line labeling), handler-extraction parity (all four shapes), human-summary rendering, and 4 end-to-end tests running the real preflight → real esbuild staging → dynamic import →simulateInvocation(success record, failure isolation + exit 1,--outputfile, usage-error path). E2E temp personas live insidepackages/cliso the externalized runtime resolves — same reason deploy stages in-tree.Full workspace
pnpm run checkgreen; CLI suite 229/229 (211 pre-existing + 18 new).Refs #186, AgentWorkforce/cloud#1783
🤖 Generated with Claude Code
CodeAnt-AI Description
Add a local invocation command for replaying fixture events
What Changed
agentworkforce invoketo run a persona against fixture event envelope(s) and record side effects instead of executing them.deploy --dry-run.Impact
✅ Local event replay without cloud side effects✅ Clearer run summaries and error output✅ Easier validation of persona behavior before deployment💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.