[pull] main from CopilotKit:main - #1
Merged
Merged
Conversation
`proxyAndRecord` collected every upstream chunk into an array and replayed the entire body via a single `res.end()` at the bottom of the function. For `text/event-stream` responses this collapses all frames into one client-visible write, defeating progressive rendering in downstream consumers. When upstream declares `content-type: text/event-stream`, tee each incoming chunk to the client while still accumulating into the recording buffer. Non-SSE responses continue to use the existing buffered relay path unchanged. Latent since v1.6.0 (`900399b`); surfaced in production once showcase traffic began exercising `--proxy-only` more heavily under tightened v1.14.0+ fixture handling.
) ## Summary - `proxyAndRecord` buffered upstream SSE bodies and replayed in a single `res.end()`, collapsing chunks into one client-visible write. - Showcase's `--proxy-only` mode hit this path in production, breaking progressive streaming downstream. - Surgical split: when upstream `content-type` is `text/event-stream`, relay chunks to the client as they arrive while still tee-ing into the recording buffer. Non-SSE behavior unchanged. ## Context Latent since v1.6.0 (`900399b`). Exposed in production after v1.14.0+ fixture handling tightened and more showcase traffic began hitting the proxy path. Live curl against deployed `showcase-aimock-production` confirmed: 11 SSE frames in 35µs after a 515ms TTFB — classic buffer-and-replay signature. ## Test plan - [x] Full aimock suite green (2436 passing) - [x] Build green - [x] New `recorder SSE progressive streaming` test — red on pre-fix recorder (single data event, `arrivalTimes.length === 1`), green after fix (multi-frame span >=100ms) - [ ] Post-release: showcase `:latest` auto-picks up; Jordan verifies via Playwright probe
- Matching Semantics section covering last-message-only matching, first-wins file order, substring vs exact, shadowing warnings - toolCallId matching correctly described (last role:tool message per router.ts, not last-message-is-tool) - Provider Support Matrix: Vertex AI column added - Ollama Reasoning corrected (shipped in 1.8.0, was '—') - toolName scope, endpoint enum scope, finishReason full Responses-API mapping, onTranscription asymmetry, onToolResult cross-ref, shadowing format, Azure overrides footnote
- Recording Multi-Turn Conversations section - CLI Flags table expanded: -f/--fixtures, --journal-max, --fixture-counts-max, --agui-*, --chaos-*, --watch, -p, -h, --log-level, --validate-on-load - Auth Header Forwarding documents strip-list behavior (v1.6.1+) - requestTransform example: trailing-space fixture key removed - Docker examples use absolute $(pwd) paths, drop npx prefix, include -h 0.0.0.0 for port-mapping - Rust async-openai example corrected to Client::with_config form - proxyOnly disambiguated (recording vs proxy-only modes) - pseudocode annotated as simplified, mock.stop() cleanup added - Stale 2025 timestamp replaced with generic placeholder - class=cmt → class=cm (broken CSS class)
…ionally Previously TOC id-assignment was gated to pages with 4+ headings, silently breaking cross-page anchor links to short pages.
…ages - Docker examples: -h 0.0.0.0 host-bind, absolute $(pwd) paths, strip redundant npx prefix that became a positional arg to image entrypoint, replace --config (unsupported by published image CLI) with flag-driven or npx-local alternatives - class=cmt → class=cm (21 occurrences, .cmt undefined in style.css)
- Kotlin and Python fail-fast on health-check loops (previously swallowed exceptions silently) - Go SDK rewrite (sashabaranov/go-openai), Rust SDK rewrite (async-openai) - Python on_message signature corrected - Invalid JSON cleanup in chaos-testing and piyook examples - MSW v2 WebSocket claim corrected, Weaviate claim removed, MSW import statement added - Helm values adds livenessProbe + readinessProbe stanzas - Docker teardown hardening (--rm, if: always()) - /health vs /__aimock/health standardized - Provider counts restored to 11+ with accurate variant enumeration (OpenAI Chat/Responses/Realtime, Claude, Gemini REST/Live, Azure, Bedrock, Vertex AI, Ollama, Cohere); competitor-equivalent counting (VidaiMock 11+, Mokksy 11 vs 5, mock-llm 11 vs 1) - Stripped redundant npx prefixes from Docker examples - Replaced --config Docker examples with flag-driven equivalents - Added -h 0.0.0.0 to every Docker example for port-mapping - Synchronized provider-count phrasing across all migration pages
npx @copilotkit/aimock -f … resolves to the config-driven aimock bin which rejects -f; rewrote remaining occurrences across integration, docker, metrics, and multimedia pages to npx -p @copilotkit/aimock llmock …
…r count - Multi-target feature bullets (multi-turn, sequential conversations) - Multimedia and Protocol-Mock feature bullets now link to individual feature pages - LLM Providers count restored to 11+ with full variant enumeration (OpenAI Chat / Responses / Realtime, Claude, Gemini REST / Live, Azure, Bedrock, Vertex AI, Ollama, Cohere); earlier collapse to 8 undersold aimock's coverage (competitors count variants separately) - aimock Suite table LLMock row updated to match - Quick Start: sets OPENAI_BASE_URL + OPENAI_API_KEY before SDK construction with an inline ordering warning - Docker one-liner: absolute $(pwd)/fixtures:/fixtures path - LLMock class-name asymmetry after v1.7.0 package rename explained inline - llmock CLI bin explicitly noted as a legacy alias without --config or convert support
….14.6 - v1.14.1: Railway-specific language scrubbed - v1.14.2: --journal-max=-1 rejection + createServer() default flip annotated with BREAKING / BEHAVIOR CHANGE markers - All 15 historical version entries standardized on Keep-a-Changelog categories (Added / Changed / Fixed / Removed) instead of mixed Changesets-style - v1.14.4 entry: multi-turn docs, matching semantics, Docker examples, npx bin sweep, CSS class fix, provider-count restoration, migration page quality sweep - v1.14.5 entry: SSE streaming fix (absorbed from main) - v1.14.6 entry: DevX cleanup (sidebar TOC, README Quick Start, fixtures DevX, record-replay DevX, package.json hardening, package-lock.json removal)
- engines.node raised to >=24.0.0 (matches OIDC publish requirement) - preinstall: only-allow pnpm guard - Deprecated @google/generative-ai swapped for @google/genai - files includes CHANGELOG.md - repository.url canonicalized to git+https://… form - typesVersions gains .d.cts entries - Optional peerDependencies for vitest / jest - prepare: husky || true tightened to husky - release script gains pnpm test && pnpm lint pre-check - Version bumped to 1.14.6
Final cleanup of claude-sonnet-4-20250514 → claude-sonnet-4-6 migration (v1.14.3) for two remaining test fixtures that still referenced the retiring model id.
Under the hero headline, show a single rounded card with both a
"npx -p @copilotkit/aimock llmock -f ./fixtures" line and a
"docker run -v $(pwd)/fixtures:/fixtures -p 4010:4010 ghcr.io/copilotkit/aimock"
line, separated by a muted centered "or". Mirrors pathfinder.copilotkit.dev's
post-hero treatment so the landing page answers "how do I try this right
now?" in one paste, instead of just pointing at `npm install`.
`npm install @copilotkit/aimock` is demoted to a smaller secondary
"Or install as a library:" line directly below for the programmatic API /
vitest integration path.
The copy-button helper is generalized to read the adjacent <code>
textContent via `btn.closest('.run-cmd')`, so each command copies its
own line and future variants work for free.
## Summary Multi-turn conversation documentation + v1.14.4/5/6 release lineage + three docs-correctness sweeps + the Bucket-C DevX cleanup — all in one PR per the "related work, one PR" policy. ### Multi-turn documentation (original scope) - New `/multi-turn` page with the tool-round idiom walkthrough, matching-semantics table, recording guidance - "Matching Semantics" section on `/fixtures` - "Recording Multi-Turn Conversations" section on `/record-replay` - Bumps through v1.14.6 (v1.14.4 feature bump, v1.14.5 hotfix absorbed from main, v1.14.6 adds the DevX sweep) ### Docs-wide correctness sweeps - **Docker examples**: ENTRYPOINT mismatch, host-bind `-h 0.0.0.0`, absolute `$(pwd)` paths, `--config` scope replaced with flag-driven or npx-local alternatives - **npx bin resolution**: `npx @copilotkit/aimock -f …` resolves to the config-driven `aimock` bin which rejects `-f`; rewrote 22+ invocations across migration + integration + feature pages to `npx -p @copilotkit/aimock llmock …` - **CSS class bug**: `class="cmt"` (21 occurrences across 6 pages) → `class="cm"` - **Provider count**: restored to 11+ with proper enumeration (OpenAI Chat/Responses/Realtime, Claude, Gemini REST/Live, Azure, Bedrock, Vertex AI, Ollama, Cohere); Vertex AI column added to Provider Support Matrix ### Migration-page quality sweep (folded from follow-up PRs #123 + #124) - Kotlin + Python fail-fast on health-check loops (previously swallowed exceptions silently) - Go SDK rewrite (`sashabaranov/go-openai`) and Rust SDK rewrite (`async-openai`) - Python `on_message` signature corrected - Invalid JSON cleanup in chaos-testing and piyook - MSW v2 WebSocket claim corrected - Weaviate claim removed (not supported) - MSW import statement added - Helm values adds `livenessProbe` + `readinessProbe` stanzas - Docker teardown hardening (`--rm`, `if: always()`) - `/health` vs `/__aimock/health` standardized on migration pages ### Bucket-C DevX cleanup (6-agent blitz) 36 items across 6 files, prioritized by user-breakage impact per [the Notion plan](https://www.notion.so/34a3aa38185281668c24e1c559b14972): - `docs/sidebar.js`: TOC id-assignment unconditional (cross-page anchors to short pages now resolve) - `README.md`: Quick Start env ordering warning, `OPENAI_API_KEY` requirement, Docker absolute path, `LLMock` class/package naming note, multi-target feature bullets - `docs/fixtures/index.html`: Ollama Reasoning corrected (shipped in 1.8.0, was marked unsupported), `toolName` scope, `endpoint` enum scope, `onTranscription` asymmetry, `onToolResult` table cross-ref, shadowing warning format, Azure overrides footnote, full `finishReason` Responses-API mapping - `docs/record-replay/index.html`: Docker `$(pwd)`, Rust `async-openai` correct form, proxyOnly disambiguation, stale timestamp placeholder, pseudocode null-guard note, `mock.stop()` cleanup - `CHANGELOG.md`: v1.14.1 Railway leak scrubbed, v1.14.2 BREAKING/BEHAVIOR-CHANGE markers, all 15 historical entries standardized on Keep-a-Changelog - `package.json`: `engines.node` raised to 24 (matches OIDC publish), `preinstall: only-allow pnpm` guard, deprecated `@google/generative-ai` swapped for `@google/genai`, `files` includes CHANGELOG, `typesVersions` gains `.d.cts` entries, optional `peerDependencies` for vitest/jest, husky tightened, `release` gains test+lint pre-check, `repository.url` canonical form ### What's NOT in this PR 11 P3 speculative / internal-rendering items per the Notion plan — deferred until visibly broken. Tracked in the [aimock docs/packaging backlog](https://www.notion.so/34a3aa381852811fbbfdfad59453f9d9). ## Test plan - [x] `pnpm run format:check` clean - [x] `pnpm run lint` clean - [x] `npx tsc --noEmit` clean - [x] `pnpm test` 2438 pass, 26 skipped - [x] `pnpm run build` clean - [ ] CI green on merge-retarget to main ## CR discipline - 9-round 7-agent unbiased CR on the original multi-turn scope (landed as PR #121's original 9 commits + 3 follow-up PRs) - Tight single-agent scoped verification on the Bucket-C blitz (36/36 CONFIRMED per [the Notion DevX plan](https://www.notion.so/34a3aa38185281668c24e1c559b14972)) - Classification-logic encoded in `CopilotKit/internal-skills` PR #30 (cr-loop v2.5.0: Procedure 3 bucket-(c) audit, subject-scope partition, out-of-subject bucket (d), convergence filter)
Teach --fixtures to accept https:// or http:// URLs pointing at JSON fixture files in addition to filesystem paths. Remote values are fetched once at boot, parsed, and cached to disk so downstream code treats them identically to a local file. - New src/fixtures-remote.ts helper: resolveFixturesValue(value, opts) returns a local filesystem path regardless of input scheme. - Cache at $XDG_CACHE_HOME/aimock/fixtures/<sha256-of-url>/fixtures.json (default ~/.cache/aimock/fixtures/<sha256>/fixtures.json). - Under --validate-on-load, a fetch failure with a usable cached copy logs a warning and continues; no cache -> process exits non-zero. - Without --validate-on-load, a fetch failure with no cache logs a warning and skips the source (matching the local-path branch). - HTTP fetch has a 10s timeout and a 50 MB size cap, both enforced incrementally so a lying Content-Length can't bypass the limit. - Only https:// and http:// schemes are accepted. file://, ftp://, etc. are rejected with a clear error. - --fixtures is now repeatable; multiple sources are loaded and concatenated. Record / AG-UI record modes still require a local filesystem path (use the first --fixtures value). Tarball (.tar.gz) and zip URL support is intentionally deferred to a later release to keep this change small and dependency-free.
…tures Harden --fixtures URL fetching against two SSRF classes that v1.14.7 left open: 1. Private / reserved destinations. Fetching http://169.254.169.254/ (AWS/GCP/Azure instance metadata), RFC1918, CGNAT, loopback, link- local, ULA, multicast, and other reserved ranges now fails loud with a clear error. Set AIMOCK_ALLOW_PRIVATE_URLS=1 to opt out — required for local dev and tests that target 127.0.0.1. Hostname resolution checks every returned address; any blocked entry in the set rejects the fetch. 2. Redirect follow. The fetch previously used the default redirect: "follow" behavior, which would silently chase a 3xx Location into a different scheme (file://, javascript:) or host — bypassing the scheme gate and the new SSRF denylist. The fetch now uses redirect: "manual" and rejects any 3xx with a fail-loud error telling the user to configure the upstream to serve the final URL directly (GitHub raw content URLs already do this; a CDN that insists on redirecting is incompatible with a fixture loader). Implementation: - New helpers in src/fixtures-remote.ts: isPrivateAddress(addr) and assertAllowedHost(hostname). Both are exported for direct testing. - Uses net.BlockList (Node >= 15) for CIDR membership checks rather than hand-rolled integer math. v4-mapped IPv6 (::ffff:a.b.c.d) is unwrapped and re-checked against the v4 ranges. - SSRF check runs before any network I/O; the assertion throws a clear error before DNS is even consulted for literal-IP hostnames.
Pins behavior across the remote-fixture feature and its security
hardening layer with real-socket tests (no mocks):
src/__tests__/fixtures-remote.test.ts (unit + integration)
- looksLikeUrl scheme recognition and local-path passthrough
- defaultCacheRoot honors XDG_CACHE_HOME
- http(s) success: fetch, cache to disk, return cached path
- http(s) failure modes: 500 fail-loud, cache fallback, timeout,
incremental body-cap enforcement, Content-Length early reject,
invalid-JSON fail-loud
- exported default constants: REMOTE_FETCH_TIMEOUT_MS == 10_000,
REMOTE_MAX_BYTES == 50 * 1024 * 1024
- lying Content-Length safety: chunked-encoding body past cap,
truncation-safety fail-loud contract
- SSRF denylist: table-driven isPrivateAddress across 20 reserved
ranges + 4 public IPs; assertAllowedHost env opt-out +
per-address rejection; integration: 169.254.169.254 and
127.0.0.1 rejected without the opt-out, permitted with it
- redirect rejection: 302 -> file:// and 302 -> https://other
both fail loud under redirect: "manual"
- file-wide beforeAll sets AIMOCK_ALLOW_PRIVATE_URLS=1 so the
existing tests that fetch from 127.0.0.1 still pass
src/__tests__/cli.test.ts (CLI subprocess)
- Remote --fixtures URL success, 500 + no-cache fail-loud,
500 + cache fallback, file:// rejection, filesystem-path
regression guard
- Repeatable --fixtures flag with two local http servers:
asserts both fixtures load (Loaded 2 fixture(s)) and argv
order is preserved in the startup log line
- beforeEach in the remote-URL describe block sets
AIMOCK_ALLOW_PRIVATE_URLS=1 so subprocess fetches against
127.0.0.1 aren't rejected by the new SSRF denylist
…-out Adds a short example pair to the CLI block showing a repeatable HTTPS --fixtures flag alongside a local path, plus a new 'Remote fixture URLs' subsection covering the on-disk cache at ~/.cache/aimock/fixtures/, --validate-on-load fallback behavior, the 10s timeout / 50 MB body cap / redirect rejection, and the private-address denylist with the AIMOCK_ALLOW_PRIVATE_URLS=1 opt-out for local dev.
Ships the remote-fixture feature plus its security hardening layer: https/http URL values for --fixtures (fetch + on-disk cache + fail- loud recovery, 10s timeout, 50 MB incremental body cap), an SSRF denylist rejecting private / reserved addresses by default, and disabled HTTP redirects to prevent scheme-bypass. Opt out of the SSRF denylist with AIMOCK_ALLOW_PRIVATE_URLS=1 for local dev.
## Summary - Remote `https://` / `http://` URLs for `--fixtures` with on-disk cache + fail-loud recovery - SSRF hardening: private-address denylist (`AIMOCK_ALLOW_PRIVATE_URLS=1` opt-out for local dev) - Redirects rejected fail-loud (configure upstream to serve the final URL directly) - 10s hard timeout + 50MB incremental body cap (lying `Content-Length` cannot bypass) - `--fixtures` is now repeatable; argv order preserved ## Why Phase 1 of showcase-aimock wrapper elimination — Railway can pull fixtures from stable GitHub raw URLs instead of rebuilding an image per fixture update. ## Test plan - 30+ new tests across unit + integration + CLI subprocess layers; 2520/2520 suite-wide pass - Real sockets (no mocks) for every fetch path - Coverage: timeout, incremental cap (chunked + lying CL), cache fallback, scheme rejection, redirect rejection, SSRF denylist (table-driven across 20 reserved ranges + 4 public IPs), env opt-out, invalid JSON fail-loud, non-zero exit, repeatable `--fixtures` load order
The recordBase URL check was previously guarded by `if (values.record || values["proxy-only"])` and rejected any first --fixtures value starting with http(s):// as an invalid record destination. For --record mode this is correct — recording writes JSON files to disk and needs a writable base path. For --proxy-only mode it is overbroad: proxy-only forwards unmatched requests without saving, and recorder.ts/agui-recorder.ts already skip all disk writes when proxyOnly is set. All-URL --fixtures invocations should therefore be valid. The URL-rejection now fires only for --record (and --agui-record). In --proxy-only mode with a URL first fixture, fixturePath is left undefined on the RecordConfig — the recorder code path that would consume it is skipped by the existing proxyOnly guard, so no dead filesystem writes are attempted. This unblocks the showcase-aimock Railway service which needs to run aimock in proxy-only mode against two remote GitHub raw fixture URLs with no local fallback. The previous workaround was a shell-wrapped startCommand that mkdirs a dummy /tmp/empty-fixtures before exec-ing aimock.
Four new cases under a new describe block "CLI: --proxy-only with URL-only --fixtures": - --proxy-only + URL-only --fixtures starts successfully, loads the remote fixture, and does not emit the recordBase rejection error. - --record + URL-only --fixtures still errors (regression guard — record writes to disk, URLs are genuinely unsupported as write targets). - --proxy-only + mixed local and URL --fixtures loads both (2 fixtures), preserving argv order (baseline coverage for the combined path). - --agui-proxy-only + URL-only --fixtures starts successfully (parallel AG-UI path gets the same fix). Tests were verified red against the pre-fix cli.ts (the two URL-only cases timed out waiting for "listening on" because the CLI exited with the recordBase error) and green against the fixed cli.ts.
Bumps @copilotkit/aimock to 1.14.8 with CHANGELOG entry for the --proxy-only URL-only --fixtures fix.
## Summary - `--proxy-only` mode no longer rejects URL-only `--fixtures` invocations - The `recordBase` URL check now fires only for `--record` mode (where a writable destination is actually required) - Same fix applied to the parallel `--agui-proxy-only` CLI path - 4 new tests pin the green-path (URL-only `--proxy-only` + AG-UI variant + mixed local/URL) and guard the `--record` regression - Release: bumps `@copilotkit/aimock` to 1.14.8 with CHANGELOG entry ## Why The showcase-aimock Railway service (14+ demo services route through it) hit this blocker when switching from the baked-in wrapper image to `ghcr.io/copilotkit/aimock:1.14.7` with remote fixtures from GitHub raw. The workaround was a shell-wrapped startCommand that `mkdir`s a dummy local path before exec-ing aimock. This fix removes the workaround. The `recordBase` check was guarded by `if (values.record || values["proxy-only"])` and rejected any first `--fixtures` value starting with `http(s)://`. For `--record` that's correct — recording writes files to disk and needs a writable base path. For `--proxy-only` it's overbroad: proxy-only forwards without saving, and both `recorder.ts` and `agui-recorder.ts` already guard disk writes behind `if (!proxyOnly)`. The fix narrows the URL-rejection to `--record`/`--agui-record` and leaves `fixturePath` undefined on the `RecordConfig` when the first fixture is a URL in proxy-only mode. ## Test plan - [ ] CI green - [ ] Manual: `npx -p @copilotkit/aimock aimock --proxy-only --provider-openai https://api.openai.com --fixtures https://... --validate-on-load --host 0.0.0.0 --port 4010` starts cleanly - [ ] showcase-aimock Railway startCommand reverts to the unwrapped form (follow-up in CopilotKit PR #4190 reference)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )