Skip to content

feat: structured NDJSON wire-event consumption, --workspace forwarding, prepare script#48

Merged
manojp99 merged 8 commits into
mainfrom
feat/structured-wire-events-and-workspace-plumbing
Jun 11, 2026
Merged

feat: structured NDJSON wire-event consumption, --workspace forwarding, prepare script#48
manojp99 merged 8 commits into
mainfrom
feat/structured-wire-events-and-workspace-plumbing

Conversation

@manojp99

@manojp99 manojp99 commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Four-bug chain fix: Completes end-to-end plumbing for structured wire-event consumption from engine → wrapper → host (e.g. paperclip). Fixes missing JSON display mode, missing wrapper-ts flag forwarding, iterator queue notification delivery, and improves CLI human-readable display.
  • New --display ndjson flag: Opt-in structured NDJSON wire-event emission from the engine. Hosts can now receive usage events with enriched fields (cost, model, provider, cache metrics, LLM duration). Backward-compatible; defaults to text.
  • Engine --workspace plumbing to wrapper-ts: Wrapper now forwards workspace?: string to engine --workspace <slug> flag, allowing hosts managing multiple agents to isolate engine state per agent.
  • Prepare script for git-dep installs: Enables consuming wrapper-ts via git ref (for early testing) — prepare script runs at install time to build dist/.

Context

From #45: Engine now emits enriched usage notifications with cost, model, provider, cache metrics, and session cost totals. However, these were reaching the engine only — hosts using the async iterator pattern (for await (const ev of handle.submit())) weren't receiving them. The gap required plumbing at four layers:

  1. Engine — ✅ already emits JSON via --display ndjson (from feat(streaming-hook): enrich wire events with cost, LLM timing, model, thinking, and sub-agent attribution #45) and pushes to callback.
  2. Wrapper push to iterator queue — ❌ fixed by 35084da.
  3. Wrapper --display forwarding — ❌ fixed by 82605c7 + d1dcb67.
  4. Host integration — paperclip side (separate PR).

This PR closes the first three gaps. Hosts using the async iterator pattern will now receive full usage notifications with cost and metadata.

Changes

Commit What
a8e89e3 Prepare script for git-dep installs
2e164ef CLI text display now includes cost/cache/model (independent of NDJSON path)
00d535f New JsonDisplaySystem in CLI; --display ndjson flag added
82605c7 Wrapper-ts SpawnAgentParams adds displayMode?: "text" | "ndjson" → assembleArgv emits --display <mode>
d1dcb67 Regenerate wrapper-ts dist/ for displayMode support
35084da Fix: Push NDJSON notifications onto the async iterator queue (not just the callback)
c19ed74 Wrapper-ts workspace?: string → assembleArgv emits --workspace <slug>

Verification

  • ✅ Typechecks: wrapper-ts and CLI both pass strict checks.
  • ✅ Prepare script works: tested locally via git-dep install pattern.
  • ✅ Backward-compatible: defaults to --display text, existing hosts unaffected.
  • ✅ New fields reach iterator: confirmed via wrapper integration tests.

Known follow-up

  • Missing contract test: No E2E integration test yet asserting "engine emits cost → wrapper forwards → host receives event.params.cost". Recommend opening a follow-up issue for this.
  • Wrapper-ts version bump: This adds new public fields to the DisplayMode union and SpawnAgentParams. Recommend a semver-minor bump (e.g., 0.7.0) when publishing.

Generated with Amplifier

Manoj Prabhakar Paidiparthy and others added 8 commits June 10, 2026 11:28
@microsoft-amplifier
chore(wrapper-ts): release 0.6.2 — timeout opt-in
34c603f 
Bumps amplifier-agent-ts from 0.6.1 to 0.6.2 to ship the timeout
opt-in fix landed in #41. Tag wrapper-v0.6.2 will trigger the
publish-wrapper.yml workflow to release this version to npm.

Headline change: SessionHandle.submit() no longer silently imposes
a 10-minute wall-clock cap when timeoutMs is undefined. The timer
is now opt-in (timeoutMs > 0 to arm). DEFAULT_TIMEOUT_MS is now
exported for callers that want the legacy cap.

Refs: #41

Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
chore(wrapper-ts): add prepare script for git-dep installs
a8e89e3 
Enables consumers to install the wrapper directly from a git ref by
auto-running 'npm run build' (which chains prebuild: gen-types -> tsc)
during install. This produces dist/ that the published package would
have shipped.

Standard 'pnpm install' / 'npm install' from the published tarball is
unaffected -- prepare only runs for git refs and 'npm publish'.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
fix(cli): surface enriched usage fields in CLI display (#45 follow-up)
2e164ef 
The streaming hook (fa3b237) enriched the wire `usage` notification
with cost, model, provider, llmDurationMs, cacheReadTokens,
cacheWriteTokens, sessionCostTotal, and (for delegated sub-agents)
agentName. The CLI display formatter (defaults_cli.py:_summarize)
only read inputTokens / outputTokens, so the stderr human-readable
log line stayed minimal:

  [usage] in=4202 out=467

After this patch the line includes every enriched field the engine
supplies (each guarded individually so terse usage events still
render cleanly for older engines or providers that don't enrich):

  [usage] in=4202 out=467 cost=$0.067644 cache_read=9339     cache_write=9339 dur=6411ms model=claude-opus-4-5     provider=anthropic

Downstream impact: hosts that capture amplifier-agent stderr (e.g.
paperclip's amplifier-local adapter, which persists raw stderr to
heartbeat-run NDJSON logs and renders it as the run transcript) will
now see the enriched fields in their transcript views without any
host-side changes.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
feat(cli): add JsonDisplaySystem + --display ndjson for structured …
00d535f 
…host consumption

Closes the contract gap between the streaming-hook enrichment (#45)
and host integrations that consume the wire-event stream.  Today,
CliDisplaySystem is hardcoded as the only DisplaySystem, writing
`[type] summary` human-readable lines to stderr.  Hosts using
amplifier-agent-ts (e.g. paperclip's amplifier-local adapter) wire
`parseNdjsonStream` onto child stderr, expecting one JSON-RPC
notification per line -- but never get any, because no display ever
emits JSON.  The wrapper's `onJson` callback never fires; structured
fields (cost, model, provider, llm duration, cache token counts,
session cost total, delegated agentName) added by #45 sit in Python
dicts and never reach disk.

This adds a second DisplaySystem implementation alongside CliDisplay
and a CLI flag to choose between them:

  amplifier-agent run --display text|ndjson  (default: text)

`text` keeps the existing human-readable behavior verbatim.  `ndjson`
swaps in JsonDisplaySystem, which emits one JSON object per event
shaped as:

    {"method": "<event-type>", "params": <rest of event>}

This matches the JSON-RPC notification shape the wrapper-ts session
parser expects (session.ts:380-395), so host adapters can switch on
`event.method` and receive the enriched fields directly on
`event.params`.

Backward-compatible by design:
  - Default is `text`; existing wrappers that don't pass `--display`
    continue to receive the human-text format.
  - The new flag is additive; no breaking change to argv contract.
  - JsonDisplaySystem ignores verbosity flags -- hosts filter their
    own consumption (the structured stream is the canonical contract).

Contract notes for future maintainers:
  - The NDJSON stream is now part of the engine's external interface.
    Fields are additive-only; never rename a field without a versioning
    plan.  Hosts should ignore unknown `params` keys.
  - Stdout discipline preserved: JsonDisplaySystem writes only to
    the injected stream (typically sys.stderr).  The §4.1 envelope
    on stdout is unchanged.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
feat(wrapper-ts): forward displayMode opt to engine via --display
82605c7 
Pairs with the engine-side `JsonDisplaySystem` + `--display` flag added
in commit 00d535f.  Hosts can now request structured stderr emission
(one JSON-RPC notification per line) by passing
`displayMode: "ndjson"` to `spawnAgent()`.

Without this, the wrapper's `parseNdjsonStream` consumer on
`child.stderr` sees only human-readable text from `CliDisplaySystem`
and never invokes `display.onEvent` for the engine's wire-event stream
-- so hosts wanting cost/model/cache/duration enrichment from #45
never see it.

Wiring:
  - `SpawnAgentParams.displayMode?: "text" | "ndjson"` (new public field).
  - Forwarded through `SessionHandleParams.displayMode` to
    `assembleArgv()`, which emits `--display <mode>` when set.
  - When omitted, the wrapper emits no `--display` flag, so older
    engines (pre-#45-followup) keep working with this wrapper.

Backward-compatible by design:
  - The new field is optional everywhere on the path.
  - Existing callers (no displayMode) keep their current behavior.
  - Engine defaults to `text` if no `--display` flag is emitted.

Engine compatibility note added to the public docstring: setting
`displayMode` requires an engine that accepts `--display` (older engines
fail with click "no such option"). Hosts using link: or paired releases
will be in sync; hosts mixing wrapper@new + engine@old should omit
`displayMode`.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
chore(wrapper-ts): regenerate dist/ for displayMode plumbing
d1dcb67 
Built from the src/ changes in 82605c7. dist/ is tracked in this repo
so consumers installing via git refs get the built artifacts without a
separate build step.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
fix(wrapper-ts): also push NDJSON notifications onto the iterator queue
35084da 
Previously the parseNdjsonStream onJson handler in SessionHandle.submit()
dispatched parsed notifications ONLY to params.display?.onEvent (the
push callback). Iterator consumers (`for await (const ev of
handle.submit(...))`) never saw them -- the iterator queue only received
init, activity ticks, and the final terminal event.

This is the third bug in the chain that prevented paperclip's
amplifier-local adapter from recording cost data:

  1. engine had no JSON display mode (fixed 00d535f: JsonDisplaySystem)
  2. wrapper didn't forward --display flag (fixed 82605c7: displayMode)
  3. wrapper delivered notifications to callback but not iterator (this fix)

Paperclip's execute.ts iterates handle.submit() and switches on
event.method for usage/result/tool/* events. With this fix, the
existing `case "notification":` branch finally receives data and the
adapter populates AdapterExecutionResult.{costUsd, usage, model,
provider}. cost_events table starts getting rows.

Hosts that subscribe via both display.onEvent AND the iterator will
receive each notification twice -- acceptable trade-off; subscribe to
one or the other. Documented inline.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@microsoft-amplifier
feat(wrapper-ts): forward workspace opt to engine via --workspace
c19ed74 
Hosts that manage multiple agents per process (paperclip's
amplifier-local adapter being the immediate case — runs CEO + CTO +
sub-agents per company) need each agent's session state to land in
its own engine workspace directory. Without this, every spawn shares
a cwd-derived slug (e.g. "default-9e80f0e7") and all transcripts
mingle under one workspaces/.../sessions/ tree, making debugging
and history navigation painful.

The engine already accepts `--workspace <name>` (validated against
`[a-z0-9][a-z0-9-]{0,63}`). This plumbs it through:

  SpawnAgentParams.workspace
    → SessionHandleParams.workspace
      → AssembleArgvInput.workspace
        → argv: --workspace <slug>

When omitted, the wrapper emits no `--workspace` flag — the engine
falls back to cwd-derived auto-slug (existing behavior preserved).

Backward-compatible by design:
  - Field is optional throughout.
  - Existing hosts (no workspace) keep their auto-derived slug.
  - Older engines that accept --workspace just receive what was
    already valid argv.

Engine compatibility note: `--workspace` has been a click option
on `amplifier-agent run` for a while (single_turn.py), so this
doesn't gate behind a new engine version.

🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier)

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@manojp99 manojp99 merged commit 9f650db into main Jun 11, 2026
2 of 3 checks passed
@manojp99 manojp99 mentioned this pull request Jun 12, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@manojp99