Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,33 @@ Useful flags: `--input KEY=value` overrides declared persona inputs;
filesystem with provider data the handler reads. Exit code is 0 when every
envelope succeeded, 1 when any handler invocation failed.

### Where fixtures come from

The primary path is exporting a **real fire** — the fixture is then cloud's
exact normalized output, so replay cannot drift from production:

```bash
agentworkforce runs export <runId> --fixture event.json
agentworkforce invoke ./persona.json --fixture event.json
```

(`runs export` reads the captured envelope from the run's
`/runs/:runId/envelope` endpoint; pass `--agent <deployedName>` to skip the
workspace-wide run lookup. Runs that predate envelope capture, or whose
envelope was too large to store, are reported with next steps — oversized
envelopes are omitted, never truncated.)

Before any real fire exists, scaffold a skeleton:

```bash
agentworkforce invoke --scaffold github.pull_request.opened --output event.json
```

The frame is filled in; for provider events the `resource` payload is an
explicit TODO hole (its shape is decided by adapter normalization + cloud's
buildEnvelope and shouldn't be guessed). `<type>` is validated against the
trigger catalog with a warn-don't-block stance.

## Persona vs agent

A deployable agent is two files. The **persona** says *what the agent is*
Expand Down
15 changes: 15 additions & 0 deletions packages/cli/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ import {
import ora, { type Ora } from 'ora';
import { runDeploy, runLogin, runLogout } from './deploy-command.js';
import { runInvoke } from './invoke-command.js';
import { runRuns } from './runs-command.js';
import { runDestroy } from './destroy-command.js';
import { runDeploymentList, runDeploymentLogs } from './list-command.js';
import {
Expand Down Expand Up @@ -258,6 +259,15 @@ Commands:
--input KEY=value override a declared persona input
--seed PATH=file seed the simulated filesystem
--workspace <id> workspace id for the simulated ctx
--scaffold <type> emit a fixture skeleton for an
event type instead of running
(no persona needed)
runs export <runId> [flags]
Export the gateway envelope cloud delivered to a run
as a replayable invoke fixture. Flags:
--agent <selector> agentId/deployedName owning the
run (otherwise all are checked)
--fixture <file> write the fixture to a file
deployments list List deployed cloud agents in the active workspace.
deployments logs Show structured logs for a deployed cloud agent.
destroy <persona-or-agent-id> [flags]
Expand Down Expand Up @@ -4388,6 +4398,11 @@ export async function main(): Promise<void> {
return;
}

if (subcommand === 'runs') {
await runRuns(rest);
return;
}

if (subcommand === 'deployments') {
const [action, ...extra] = rest;
if (!action || action === '-h' || action === '--help') {
Expand Down
83 changes: 81 additions & 2 deletions packages/cli/src/invoke-command.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import {
parseInvokeArgs,
renderHumanSummary,
runInvoke,
scaffoldFixture,
type RunInvokeIO
} from './invoke-command.js';
import type { SimulationResult } from '@agentworkforce/runtime';
Expand All @@ -30,8 +31,8 @@ test('parseInvokeArgs: persona + fixture + repeatable flags', () => {
'--output',
'./run.json'
]);
assert.ok(!('help' in parsed));
if ('help' in parsed) return;
assert.ok(!('help' in parsed) && !('scaffold' in parsed));
if ('help' in parsed || 'scaffold' in parsed) return;
assert.equal(path.basename(parsed.personaPath), 'persona.json');
assert.equal(path.basename(parsed.fixturePath), 'event.json');
assert.equal(path.basename(parsed.outputPath ?? ''), 'run.json');
Expand Down Expand Up @@ -326,3 +327,81 @@ test('renderHumanSummary: lists runs with status, side-effect count, skips', ()
assert.match(summary, /error: kaboom/);
assert.match(summary, /\[skip\] unsupported envelope e9 \(mystery\.event\)/);
});

// ---------------------------------------------------------------------------
// --scaffold (workforce#189)

test('parseInvokeArgs: --scaffold needs no persona or fixture', () => {
const parsed = parseInvokeArgs(['--scaffold', 'cron.tick', '--output', './event.json']);
assert.ok('scaffold' in parsed);
if (!('scaffold' in parsed)) return;
assert.equal(parsed.scaffold, 'cron.tick');
assert.equal(path.basename(parsed.outputPath ?? ''), 'event.json');
});

test('parseInvokeArgs: --scaffold rejects invoke-only args instead of ignoring them', () => {
assert.throws(
() => parseInvokeArgs(['./persona.json', '--scaffold', 'cron.tick']),
/only accepts --output.*<persona-path>/
);
assert.throws(
() => parseInvokeArgs(['--scaffold', 'cron.tick', '--fixture', './event.json']),
/only accepts --output.*--fixture/
);
assert.throws(
() => parseInvokeArgs(['--scaffold', 'cron.tick', '--input', 'FOO=bar', '--seed', '/x=./x.json']),
/only accepts --output.*--input, --seed/
);
});

test('scaffoldFixture: cron.tick emits a complete frame with name/cron and no warnings', () => {
const { fixture, warnings } = scaffoldFixture('cron.tick');
assert.deepEqual(warnings, []);
assert.equal(fixture.type, 'cron.tick');
assert.ok(typeof fixture.occurredAt === 'string');
assert.ok(String(fixture.name).includes('TODO'));
assert.ok(typeof fixture.cron === 'string');
assert.ok(!('resource' in fixture));

const named = scaffoldFixture('cron.daily-report');
assert.equal(named.fixture.type, 'cron.daily-report');
});

test('scaffoldFixture: known provider event emits frame + explicit resource TODO hole', () => {
const { fixture, warnings } = scaffoldFixture('github.pull_request.opened');
assert.deepEqual(warnings, []);
assert.equal(fixture.type, 'github.pull_request.opened');
const resource = fixture.resource as Record<string, unknown>;
assert.ok(String(resource.TODO).includes('runs export'));
});

test('scaffoldFixture: unknown provider/event warns but never blocks (lintTriggers stance)', () => {
const unknownProvider = scaffoldFixture('notaprovider.something');
assert.equal(unknownProvider.warnings.length, 1);
assert.match(unknownProvider.warnings[0], /not in KNOWN_TRIGGER_CATALOG/);
assert.equal(unknownProvider.fixture.type, 'notaprovider.something');

const unknownEvent = scaffoldFixture('github.not_a_real_event');
assert.equal(unknownEvent.warnings.length, 1);
assert.match(unknownEvent.warnings[0], /not a known github trigger/);
});

test('runInvoke --scaffold writes the skeleton and exits clean', async () => {
const io = collectingIO();
const previousExitCode = process.exitCode;
const result = await runInvoke(['--scaffold', 'cron.tick'], io);
const exitCode = process.exitCode;
process.exitCode = previousExitCode;

assert.equal(result, undefined);
assert.notEqual(exitCode, 1);
const skeleton = JSON.parse(io.out.join(''));
assert.equal(skeleton.type, 'cron.tick');
});

test('scaffoldFixture: non-tick cron types are PRESERVED with a warning, never rewritten', () => {
const { fixture, warnings } = scaffoldFixture('cron.daily');
assert.equal(fixture.type, 'cron.daily');
assert.equal(warnings.length, 1);
assert.match(warnings[0], /preserving requested type "cron.daily"/);
});
130 changes: 129 additions & 1 deletion packages/cli/src/invoke-command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { randomUUID } from 'node:crypto';
import path from 'node:path';
import { pathToFileURL } from 'node:url';
import { bundleStager, preflightPersona } from '@agentworkforce/deploy';
import { KNOWN_TRIGGER_CATALOG } from '@agentworkforce/persona-kit';
import {
simulateInvocation,
type RawGatewayEnvelope,
Expand All @@ -11,6 +12,7 @@ import {
} from '@agentworkforce/runtime';

export const INVOKE_USAGE = `usage: agentworkforce invoke <persona-path> --fixture <file> [flags]
agentworkforce invoke --scaffold <type> [--output <file>]

Simulate an invocation: execute the persona's handler against fixture event
envelope(s) with every external side effect recorded, NOT executed, and emit
Expand All @@ -35,6 +37,10 @@ Flags:
/slack/channels/_index.json).
--workspace <id> Workspace id for the simulated ctx; defaults to
the first envelope's workspace.
--scaffold <type> Emit a fixture skeleton for an event type instead
of running a persona. Provider payloads are left
as explicit TODO holes; prefer runs export after
a real fire exists.
-h, --help Print this message.

Exit code: 0 when every dispatched envelope succeeded, 1 when any handler
Expand All @@ -51,14 +57,19 @@ export interface InvokeOptions {
workspaceId?: string;
}

export type ParsedInvokeArgs = InvokeOptions | { help: true };
export type ParsedInvokeArgs =
| InvokeOptions
| { help: true }
| { scaffold: string; outputPath?: string };

/** Parse `invoke` args. Throws on usage errors (caller maps to exit 1). */
export function parseInvokeArgs(args: readonly string[]): ParsedInvokeArgs {
let personaPath: string | undefined;
let fixturePath: string | undefined;
let outputPath: string | undefined;
let workspaceId: string | undefined;
let scaffoldType: string | undefined;
let sawFixture = false;
const inputs: Record<string, string> = {};
const seeds: Record<string, string> = {};

Expand All @@ -67,13 +78,19 @@ export function parseInvokeArgs(args: readonly string[]): ParsedInvokeArgs {
if (a === '-h' || a === '--help') {
return { help: true };
} else if (a === '--fixture') {
sawFixture = true;
fixturePath = expectValue('--fixture', args[++i]);
} else if (a.startsWith('--fixture=')) {
sawFixture = true;
fixturePath = expectInline('--fixture', a.slice('--fixture='.length));
} else if (a === '--output') {
outputPath = expectValue('--output', args[++i]);
} else if (a.startsWith('--output=')) {
outputPath = expectInline('--output', a.slice('--output='.length));
} else if (a === '--scaffold') {
scaffoldType = expectValue('--scaffold', args[++i]);
} else if (a.startsWith('--scaffold=')) {
scaffoldType = expectInline('--scaffold', a.slice('--scaffold='.length));
} else if (a === '--workspace') {
workspaceId = expectValue('--workspace', args[++i]);
} else if (a.startsWith('--workspace=')) {
Expand All @@ -95,6 +112,22 @@ export function parseInvokeArgs(args: readonly string[]): ParsedInvokeArgs {
}
}

if (scaffoldType) {
const invalidScaffoldFlags = [
sawFixture ? '--fixture' : '',
workspaceId ? '--workspace' : '',
Object.keys(inputs).length > 0 ? '--input' : '',
Object.keys(seeds).length > 0 ? '--seed' : '',
personaPath ? '<persona-path>' : ''
].filter(Boolean);
if (invalidScaffoldFlags.length > 0) {
throw new Error(
`invoke: --scaffold only accepts --output; remove ${invalidScaffoldFlags.join(', ')}`
);
}
// Scaffold mode authors a fixture skeleton; no persona/fixture needed.
return { scaffold: scaffoldType, ...(outputPath ? { outputPath: path.resolve(outputPath) } : {}) };
}
if (!personaPath) {
throw new Error('invoke: missing persona path. Usage: agentworkforce invoke <persona-path> --fixture <file>');
}
Expand Down Expand Up @@ -261,6 +294,18 @@ export async function runInvoke(
io.stdout(INVOKE_USAGE);
return undefined;
}
if ('scaffold' in opts) {
const { fixture, warnings } = scaffoldFixture(opts.scaffold);
for (const warning of warnings) io.stderr(`warn: ${warning}\n`);
const text = `${JSON.stringify(fixture, null, 2)}\n`;
if (opts.outputPath) {
await writeFile(opts.outputPath, text, 'utf8');
io.stderr(`fixture skeleton written to ${opts.outputPath}\n`);
} else {
io.stdout(text);
}
return undefined;
}
Comment on lines +297 to +308

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle scaffold write failures through the same invoke error path.

--scaffold does async output I/O outside the guarded invoke try/catch, so output write failures can reject runInvoke directly instead of emitting a normalized invoke: ... error and setting exit code 1.

Suggested patch
-  if ('scaffold' in opts) {
-    const { fixture, warnings } = scaffoldFixture(opts.scaffold);
-    for (const warning of warnings) io.stderr(`warn: ${warning}\n`);
-    const text = `${JSON.stringify(fixture, null, 2)}\n`;
-    if (opts.outputPath) {
-      await writeFile(opts.outputPath, text, 'utf8');
-      io.stderr(`fixture skeleton written to ${opts.outputPath}\n`);
-    } else {
-      io.stdout(text);
-    }
-    return undefined;
-  }
+  if ('scaffold' in opts) {
+    try {
+      const { fixture, warnings } = scaffoldFixture(opts.scaffold);
+      for (const warning of warnings) io.stderr(`warn: ${warning}\n`);
+      const text = `${JSON.stringify(fixture, null, 2)}\n`;
+      if (opts.outputPath) {
+        await writeFile(opts.outputPath, text, 'utf8');
+        io.stderr(`fixture skeleton written to ${opts.outputPath}\n`);
+      } else {
+        io.stdout(text);
+      }
+      process.exitCode = 0;
+    } catch (err) {
+      io.stderr(`invoke: ${err instanceof Error ? err.message : String(err)}\n`);
+      process.exitCode = 1;
+    }
+    return undefined;
+  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if ('scaffold' in opts) {
const { fixture, warnings } = scaffoldFixture(opts.scaffold);
for (const warning of warnings) io.stderr(`warn: ${warning}\n`);
const text = `${JSON.stringify(fixture, null, 2)}\n`;
if (opts.outputPath) {
await writeFile(opts.outputPath, text, 'utf8');
io.stderr(`fixture skeleton written to ${opts.outputPath}\n`);
} else {
io.stdout(text);
}
return undefined;
}
if ('scaffold' in opts) {
try {
const { fixture, warnings } = scaffoldFixture(opts.scaffold);
for (const warning of warnings) io.stderr(`warn: ${warning}\n`);
const text = `${JSON.stringify(fixture, null, 2)}\n`;
if (opts.outputPath) {
await writeFile(opts.outputPath, text, 'utf8');
io.stderr(`fixture skeleton written to ${opts.outputPath}\n`);
} else {
io.stdout(text);
}
process.exitCode = 0;
} catch (err) {
io.stderr(`invoke: ${err instanceof Error ? err.message : String(err)}\n`);
process.exitCode = 1;
}
return undefined;
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/invoke-command.ts` around lines 277 - 288, The scaffold
branch performs async I/O (scaffoldFixture, writeFile, io.stdout/stderr) outside
the guarded invoke error path, so failures bypass the normalized invoke error
handling; fix this by ensuring the scaffold branch is executed inside the same
try/catch used by runInvoke (or wrap the scaffold I/O in a try/catch that
rethrows the caught error) so that writeFile or io.* rejections propagate to the
existing invoke error handler and produce the normalized "invoke: ..." error and
exit code; locate the scaffold branch referencing opts.scaffold,
scaffoldFixture, writeFile, io.stdout/io.stderr and either move it into the main
invoke try block or add a small try { ... await writeFile(...) / io.stdout(...)
} catch (err) { throw err } to let the outer handler process it.


try {
return await runInvokeWithOptions(opts, io);
Expand Down Expand Up @@ -369,3 +414,86 @@ export function renderHumanSummary(result: SimulationResult): string {
}
return `${lines.join('\n')}\n`;
}

/**
* Cold-start fixture authoring (workforce#189): emit a RawGatewayEnvelope
* skeleton for an event type before any real fire exists. The frame is
* filled in; for provider events the `resource` payload shape is decided by
* adapter normalization + cloud's buildEnvelope and CANNOT be guessed here,
* so it is left as an explicit TODO hole — prefer
* `agentworkforce runs export <runId> --fixture …` once a real fire exists.
*
* `<type>` is validated against KNOWN_TRIGGER_CATALOG with the same
* warn-don't-block stance as lintTriggers.
*/
export function scaffoldFixture(type: string): {
fixture: Record<string, unknown>;
warnings: string[];
} {
const warnings: string[] = [];
const occurredAt = new Date().toISOString();

if (type === 'cron.tick' || type.startsWith('cron.')) {
if (type !== 'cron.tick') {
// Preserve what was asked for (a silent rewrite would scaffold a
// DIFFERENT event type than requested) but warn: the gateway
// delivers schedule fires as `cron.tick`, and the runner shim
// treats any `cron.*` as a cron event.
warnings.push(
`the gateway delivers schedule fires as "cron.tick"; preserving requested type "${type}" (the runner shim still dispatches any cron.* as a cron event)`
);
}
return {
fixture: {
id: 'evt_local_1',
workspace: 'ws-local',
type,
occurredAt,
name: 'TODO: your schedule name (persona schedules[].name)',
cron: '0 9 * * 1'
},
warnings
};
}

const firstDot = type.indexOf('.');
const provider = firstDot > 0 ? type.slice(0, firstDot) : '';
Comment on lines +448 to +460

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Any cron.* scaffold request is silently rewritten to cron.tick, so the generated fixture can represent a different event type than the user asked for. Preserve the requested type (or emit a warning/error) instead of coercing it. [incorrect variable usage]

Severity Level: Major ⚠️
- ⚠️ `invoke --scaffold` cron fixtures use incorrect type value.
- ⚠️ Mismatch between requested type and written fixture JSON.
Steps of Reproduction ✅
1. Invoke the CLI in scaffold mode, e.g. run `agentworkforce invoke --scaffold
cron.tick:my-schedule` so that `runInvoke()` in
`packages/cli/src/invoke-command.ts:261-289` is called with `args = ['--scaffold',
'cron.tick:my-schedule']`.

2. Inside `parseInvokeArgs()` at `packages/cli/src/invoke-command.ts:61-125`, the loop
sets `scaffoldType = 'cron.tick:my-schedule'` and, because `scaffoldType` is truthy,
returns `{ scaffold: scaffoldType, outputPath?: ... }` at lines 107-110 without requiring
a persona or fixture.

3. Back in `runInvoke()` at `packages/cli/src/invoke-command.ts:273-288`, the `'scaffold'
in opts` branch calls `scaffoldFixture(opts.scaffold)` with the original type string
`'cron.tick:my-schedule'` and then writes the returned `fixture` JSON to stdout or
`--output`.

4. In `scaffoldFixture()` at `packages/cli/src/invoke-command.ts:12-31` (diff lines
419-431), the condition `if (type === 'cron.tick' || type.startsWith('cron.'))` matches
`'cron.tick:my-schedule'`, but the constructed fixture hardcodes `"type": "cron.tick"`
instead of the requested `type` value, so the generated envelope's `type` field no longer
matches the string the user passed on the command line.

Fix in Cursor | Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** packages/cli/src/invoke-command.ts
**Line:** 419:431
**Comment:**
	*Incorrect Variable Usage: Any `cron.*` scaffold request is silently rewritten to `cron.tick`, so the generated fixture can represent a different event type than the user asked for. Preserve the requested type (or emit a warning/error) instead of coercing it.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
👍 | 👎

const eventName = firstDot > 0 ? type.slice(firstDot + 1) : '';
if (!provider || !eventName) {
warnings.push(
`"${type}" is not a recognized envelope type shape (expected "cron.tick" or "<provider>.<event>"); emitting a provider-style skeleton anyway`
);
} else {
const catalog = KNOWN_TRIGGER_CATALOG as Record<string, unknown>;
const events = catalog[provider];
if (events === undefined) {
warnings.push(
`provider "${provider}" is not in KNOWN_TRIGGER_CATALOG (known: ${Object.keys(catalog).join(', ')}); scaffolding anyway`
);
} else {
const known = Array.isArray(events)
? events.includes(eventName)
: typeof events === 'object' && events !== null && eventName in (events as Record<string, unknown>);
if (!known) {
warnings.push(
`event "${eventName}" is not a known ${provider} trigger in KNOWN_TRIGGER_CATALOG; scaffolding anyway`
);
}
}
}

return {
fixture: {
id: 'evt_local_1',
workspace: 'ws-local',
type,
occurredAt,
resource: {
TODO:
'the provider payload shape is decided by adapter normalization + cloud buildEnvelope; ' +
'export a real fire with `agentworkforce runs export <runId> --fixture event.json` to get the exact shape'
}
},
warnings
};
}
Loading
Loading