feat(cli): support https URL values for --fixtures - #128
Merged
Conversation
commit: |
5 tasks
jpr5
changed the title
jpr5
force-pushed
the
feat/fixtures-remote-url
branch
from
April 23, 2026 00:21
5c09085 to
8d45701
Compare
Teach --fixtures to accept https:// or http:// URLs pointing at JSON fixture files in addition to filesystem paths. Remote values are fetched once at boot, parsed, and cached to disk so downstream code treats them identically to a local file. - New src/fixtures-remote.ts helper: resolveFixturesValue(value, opts) returns a local filesystem path regardless of input scheme. - Cache at $XDG_CACHE_HOME/aimock/fixtures/<sha256-of-url>/fixtures.json (default ~/.cache/aimock/fixtures/<sha256>/fixtures.json). - Under --validate-on-load, a fetch failure with a usable cached copy logs a warning and continues; no cache -> process exits non-zero. - Without --validate-on-load, a fetch failure with no cache logs a warning and skips the source (matching the local-path branch). - HTTP fetch has a 10s timeout and a 50 MB size cap, both enforced incrementally so a lying Content-Length can't bypass the limit. - Only https:// and http:// schemes are accepted. file://, ftp://, etc. are rejected with a clear error. - --fixtures is now repeatable; multiple sources are loaded and concatenated. Record / AG-UI record modes still require a local filesystem path (use the first --fixtures value). Tarball (.tar.gz) and zip URL support is intentionally deferred to a later release to keep this change small and dependency-free.
…tures Harden --fixtures URL fetching against two SSRF classes that v1.14.7 left open: 1. Private / reserved destinations. Fetching http://169.254.169.254/ (AWS/GCP/Azure instance metadata), RFC1918, CGNAT, loopback, link- local, ULA, multicast, and other reserved ranges now fails loud with a clear error. Set AIMOCK_ALLOW_PRIVATE_URLS=1 to opt out — required for local dev and tests that target 127.0.0.1. Hostname resolution checks every returned address; any blocked entry in the set rejects the fetch. 2. Redirect follow. The fetch previously used the default redirect: "follow" behavior, which would silently chase a 3xx Location into a different scheme (file://, javascript:) or host — bypassing the scheme gate and the new SSRF denylist. The fetch now uses redirect: "manual" and rejects any 3xx with a fail-loud error telling the user to configure the upstream to serve the final URL directly (GitHub raw content URLs already do this; a CDN that insists on redirecting is incompatible with a fixture loader). Implementation: - New helpers in src/fixtures-remote.ts: isPrivateAddress(addr) and assertAllowedHost(hostname). Both are exported for direct testing. - Uses net.BlockList (Node >= 15) for CIDR membership checks rather than hand-rolled integer math. v4-mapped IPv6 (::ffff:a.b.c.d) is unwrapped and re-checked against the v4 ranges. - SSRF check runs before any network I/O; the assertion throws a clear error before DNS is even consulted for literal-IP hostnames.
Pins behavior across the remote-fixture feature and its security
hardening layer with real-socket tests (no mocks):
src/__tests__/fixtures-remote.test.ts (unit + integration)
- looksLikeUrl scheme recognition and local-path passthrough
- defaultCacheRoot honors XDG_CACHE_HOME
- http(s) success: fetch, cache to disk, return cached path
- http(s) failure modes: 500 fail-loud, cache fallback, timeout,
incremental body-cap enforcement, Content-Length early reject,
invalid-JSON fail-loud
- exported default constants: REMOTE_FETCH_TIMEOUT_MS == 10_000,
REMOTE_MAX_BYTES == 50 * 1024 * 1024
- lying Content-Length safety: chunked-encoding body past cap,
truncation-safety fail-loud contract
- SSRF denylist: table-driven isPrivateAddress across 20 reserved
ranges + 4 public IPs; assertAllowedHost env opt-out +
per-address rejection; integration: 169.254.169.254 and
127.0.0.1 rejected without the opt-out, permitted with it
- redirect rejection: 302 -> file:// and 302 -> https://other
both fail loud under redirect: "manual"
- file-wide beforeAll sets AIMOCK_ALLOW_PRIVATE_URLS=1 so the
existing tests that fetch from 127.0.0.1 still pass
src/__tests__/cli.test.ts (CLI subprocess)
- Remote --fixtures URL success, 500 + no-cache fail-loud,
500 + cache fallback, file:// rejection, filesystem-path
regression guard
- Repeatable --fixtures flag with two local http servers:
asserts both fixtures load (Loaded 2 fixture(s)) and argv
order is preserved in the startup log line
- beforeEach in the remote-URL describe block sets
AIMOCK_ALLOW_PRIVATE_URLS=1 so subprocess fetches against
127.0.0.1 aren't rejected by the new SSRF denylist
jpr5
force-pushed
the
feat/fixtures-remote-url
branch
from
April 23, 2026 00:47
8d45701 to
a668f3a
Compare
…-out Adds a short example pair to the CLI block showing a repeatable HTTPS --fixtures flag alongside a local path, plus a new 'Remote fixture URLs' subsection covering the on-disk cache at ~/.cache/aimock/fixtures/, --validate-on-load fallback behavior, the 10s timeout / 50 MB body cap / redirect rejection, and the private-address denylist with the AIMOCK_ALLOW_PRIVATE_URLS=1 opt-out for local dev.
Ships the remote-fixture feature plus its security hardening layer: https/http URL values for --fixtures (fetch + on-disk cache + fail- loud recovery, 10s timeout, 50 MB incremental body cap), an SSRF denylist rejecting private / reserved addresses by default, and disabled HTTP redirects to prevent scheme-bypass. Opt out of the SSRF denylist with AIMOCK_ALLOW_PRIVATE_URLS=1 for local dev.
jpr5
force-pushed
the
feat/fixtures-remote-url
branch
from
April 23, 2026 00:53
a668f3a to
385b6d4
Compare
jpr5
added a commit
that referenced
this pull request
Jul 15, 2026
## Summary - Remote `https://` / `http://` URLs for `--fixtures` with on-disk cache + fail-loud recovery - SSRF hardening: private-address denylist (`AIMOCK_ALLOW_PRIVATE_URLS=1` opt-out for local dev) - Redirects rejected fail-loud (configure upstream to serve the final URL directly) - 10s hard timeout + 50MB incremental body cap (lying `Content-Length` cannot bypass) - `--fixtures` is now repeatable; argv order preserved ## Why Phase 1 of showcase-aimock wrapper elimination — Railway can pull fixtures from stable GitHub raw URLs instead of rebuilding an image per fixture update. ## Test plan - 30+ new tests across unit + integration + CLI subprocess layers; 2520/2520 suite-wide pass - Real sockets (no mocks) for every fetch path - Coverage: timeout, incremental cap (chunked + lying CL), cache fallback, scheme rejection, redirect rejection, SSRF denylist (table-driven across 20 reserved ranges + 4 public IPs), env opt-out, invalid JSON fail-loud, non-zero exit, repeatable `--fixtures` load order
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
https:///http://URLs for--fixtureswith on-disk cache + fail-loud recoveryAIMOCK_ALLOW_PRIVATE_URLS=1opt-out for local dev)Content-Lengthcannot bypass)--fixturesis now repeatable; argv order preservedWhy
Phase 1 of showcase-aimock wrapper elimination — Railway can pull fixtures from stable GitHub raw
URLs instead of rebuilding an image per fixture update.
Test plan
redirect rejection, SSRF denylist (table-driven across 20 reserved ranges + 4 public IPs),
env opt-out, invalid JSON fail-loud, non-zero exit, repeatable
--fixturesload order