From f83452d80d55d476484617ca56cc9e78bc14ab54 Mon Sep 17 00:00:00 2001 From: Landon Cox Date: Wed, 29 Jul 2026 12:58:04 -0700 Subject: [PATCH 1/3] feat: add sealed probe information budgets Add repository sensitivity categories, finite response schemas, and per-run bit ledgers. Canonically validate results and charge response timing against each repository's run budget. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ca8d9d74-46ab-48db-b05f-640cbc6d47be --- containers/agent/sealed-probe-wrapper.sh | 105 +- containers/sealed-probe/broker/broker.js | 233 +++-- containers/sealed-probe/broker/config.js | 41 +- containers/sealed-probe/broker/framing.js | 79 +- containers/sealed-probe/broker/ledger.js | 57 ++ containers/sealed-probe/broker/protocol.js | 631 +++++++++--- containers/sealed-probe/broker/scheduler.js | 76 ++ containers/sealed-probe/broker/sensitivity.js | 27 + containers/sealed-probe/broker/server.js | 25 +- docs/awf-config-spec.md | 395 ++++++-- docs/awf-config.schema.json | 44 +- src/awf-config-schema.json | 44 +- src/commands/build-config.test.ts | 4 +- ...nfig-file-sealed-probes-validation.test.ts | 56 +- src/config-file.ts | 7 +- src/parsers/sealed-probe-parser.test.ts | 68 +- src/parsers/sealed-probe-parser.ts | 63 +- src/sealed-probe/broker.test.ts | 403 ++++++-- src/sealed-probe/end-to-end.test.ts | 79 +- src/sealed-probe/ledger.test.ts | 112 +++ src/sealed-probe/manager.test.ts | 8 +- src/sealed-probe/manager.ts | 6 +- src/sealed-probe/preflight.test.ts | 23 +- src/sealed-probe/preflight.ts | 18 +- src/sealed-probe/protocol-parity.test.ts | 377 +++++-- src/sealed-probe/protocol.test.ts | 731 ++++++++++---- src/sealed-probe/protocol.ts | 929 +++++++++++++----- src/sealed-probe/scheduler.test.ts | 163 +++ src/sealed-probe/skill.test.ts | 41 +- src/sealed-probe/skill.ts | 150 ++- src/sealed-probe/staging.test.ts | 20 +- src/sealed-probe/staging.ts | 17 +- src/sealed-probe/types.ts | 24 +- src/sealed-probe/workflow-integration.test.ts | 2 +- src/sealed-probe/wrapper.test.ts | 101 +- .../agent-environment/excluded-vars.test.ts | 2 +- src/services/sealed-probe-compose.test.ts | 2 +- src/services/sealed-probe-service.test.ts | 5 +- src/services/sealed-probe-service.ts | 2 +- src/types/index.ts | 4 + src/types/sealed-probe-options.ts | 71 +- 41 files changed, 4057 insertions(+), 1188 deletions(-) create mode 100644 containers/sealed-probe/broker/ledger.js create mode 100644 containers/sealed-probe/broker/scheduler.js create mode 100644 containers/sealed-probe/broker/sensitivity.js create mode 100644 src/sealed-probe/ledger.test.ts create mode 100644 src/sealed-probe/scheduler.test.ts diff --git a/containers/agent/sealed-probe-wrapper.sh b/containers/agent/sealed-probe-wrapper.sh index 72992bbd7..dd5f05057 100755 --- a/containers/agent/sealed-probe-wrapper.sh +++ b/containers/agent/sealed-probe-wrapper.sh @@ -1,7 +1,7 @@ #!/bin/sh # /usr/local/bin/sealed-probe # -# Agent-facing sealed-probe CLI. +# Agent-facing sealed-probe CLI (protocol v2). # # Forwards a *narrow* request to the trusted sealed-probe broker over a # dedicated Unix socket. It is analogous to gh-cli-proxy-wrapper.sh, but the @@ -10,59 +10,55 @@ # or a credential. It accepts exactly: # # --repo owner/repo (exactly once) -# --outcome LABEL (exactly three times) +# --schema '' (exactly once; a finite response schema, see +# src/sealed-probe/protocol.ts) # the probe script on stdin # # Output contract: exactly one line of canonical JSON on stdout, nothing on # stderr, and exit status 0 — for every outcome and for every failure. # Transport, framing, and validation failures all produce the same local -# {"result":"ERROR"} so the agent cannot distinguish them. +# {"status":"error"} so the agent cannot distinguish them by exit status. # -# Dependencies: curl (also required by the existing gh wrapper). +# The wrapper does not (and cannot, in POSIX sh) validate the schema's +# structure, cardinality, or information-budget charge — that is the trusted +# broker's job, enforced *before* it copies a seed or launches Python. The +# wrapper's only responsibilities are: enforce the fixed CLI shape, transport +# the request unmodified, and pass the broker's response through unmodified. +# +# Dependencies: curl, base64 (both already required/available in the agent +# image). -CANONICAL_ERROR='{"result":"ERROR"}' +CANONICAL_ERROR='{"status":"error"}' SOCKET="${AWF_SEALED_PROBE_SOCKET:-/run/awf-sealed-probe/broker.sock}" -PROTOCOL_VERSION=1 -MAX_OUTCOME_BYTES=64 +PROTOCOL_VERSION=2 +# Keep in sync with MAX_SCHEMA_BYTES in src/sealed-probe/protocol.ts and +# containers/sealed-probe/broker/protocol.js. +MAX_SCHEMA_BYTES=4096 emit_error() { printf '%s\n' "$CANONICAL_ERROR" exit 0 } -# Rejects anything that is not a bounded ASCII enum identifier. -# Mirrors (and is re-enforced by) the broker's protocol validation. -valid_outcome() { - [ -n "$1" ] || return 1 - [ "$1" != "ERROR" ] || return 1 - [ "$(printf '%s' "$1" | wc -c)" -le "$MAX_OUTCOME_BYTES" ] || return 1 - printf '%s' "$1" | LC_ALL=C grep -Eq '^[A-Za-z][A-Za-z0-9_-]{0,63}$' || return 1 - return 0 -} - REPO="" -OUTCOME_1="" -OUTCOME_2="" -OUTCOME_3="" -OUTCOME_COUNT=0 +SCHEMA="" +HAVE_REPO=0 +HAVE_SCHEMA=0 while [ $# -gt 0 ]; do case "$1" in --repo) [ $# -ge 2 ] || emit_error - [ -z "$REPO" ] || emit_error + [ "$HAVE_REPO" -eq 0 ] || emit_error REPO="$2" + HAVE_REPO=1 shift 2 ;; - --outcome) + --schema) [ $# -ge 2 ] || emit_error - OUTCOME_COUNT=$((OUTCOME_COUNT + 1)) - case "$OUTCOME_COUNT" in - 1) OUTCOME_1="$2" ;; - 2) OUTCOME_2="$2" ;; - 3) OUTCOME_3="$2" ;; - *) emit_error ;; - esac + [ "$HAVE_SCHEMA" -eq 0 ] || emit_error + SCHEMA="$2" + HAVE_SCHEMA=1 shift 2 ;; *) @@ -73,21 +69,20 @@ while [ $# -gt 0 ]; do esac done -[ -n "$REPO" ] || emit_error -[ "$OUTCOME_COUNT" -eq 3 ] || emit_error +[ "$HAVE_REPO" -eq 1 ] || emit_error +[ "$HAVE_SCHEMA" -eq 1 ] || emit_error printf '%s' "$REPO" | LC_ALL=C grep -Eq '^[A-Za-z0-9][A-Za-z0-9-]{0,38}/[A-Za-z0-9._-]{1,100}$' || emit_error case "$REPO" in *..* ) emit_error ;; esac -valid_outcome "$OUTCOME_1" || emit_error -valid_outcome "$OUTCOME_2" || emit_error -valid_outcome "$OUTCOME_3" || emit_error +[ -n "$SCHEMA" ] || emit_error +[ "$(printf '%s' "$SCHEMA" | wc -c)" -le "$MAX_SCHEMA_BYTES" ] || emit_error -[ "$OUTCOME_1" != "$OUTCOME_2" ] || emit_error -[ "$OUTCOME_1" != "$OUTCOME_3" ] || emit_error -[ "$OUTCOME_2" != "$OUTCOME_3" ] || emit_error +# base64url, no padding: standard base64 with `+/` -> `-_`, `=` stripped, and +# newlines removed (wrapping width varies across base64 implementations). +SCHEMA_B64=$(printf '%s' "$SCHEMA" | base64 | tr -d '\n' | tr '+/' '-_' | tr -d '=') || emit_error # The script must arrive on stdin; an interactive terminal means no script. [ ! -t 0 ] || emit_error @@ -95,32 +90,38 @@ valid_outcome "$OUTCOME_3" || emit_error [ -S "$SOCKET" ] || emit_error # --noproxy '*' keeps HTTP(S)_PROXY from redirecting a Unix-socket request. -# --max-time bounds the wait at the schema's maximum probe timeout plus slack; -# the broker always answers, so this only guards a dead socket. +# --max-time bounds the wait comfortably above the largest timing bucket (10 +# minutes); the broker always answers at a fixed bucket boundary, so this +# only guards a dead socket. RESPONSE=$( curl --silent --show-error \ --noproxy '*' \ --unix-socket "$SOCKET" \ - --max-time 3900 \ + --max-time 660 \ -X POST \ -H "Expect:" \ -H "Content-Type: application/octet-stream" \ -H "X-AWF-Probe-Version: ${PROTOCOL_VERSION}" \ -H "X-AWF-Repo: ${REPO}" \ - -H "X-AWF-Outcome-1: ${OUTCOME_1}" \ - -H "X-AWF-Outcome-2: ${OUTCOME_2}" \ - -H "X-AWF-Outcome-3: ${OUTCOME_3}" \ + -H "X-AWF-Schema-B64: ${SCHEMA_B64}" \ --data-binary @- \ "http://localhost/probe" 2>/dev/null ) || emit_error -# Only the canonical serialization of a declared outcome (or the reserved -# ERROR sentinel) is ever printed. Anything else is treated as a failure. -for expected in "$OUTCOME_1" "$OUTCOME_2" "$OUTCOME_3" "ERROR"; do - if [ "$RESPONSE" = "{\"result\":\"${expected}\"}" ]; then +# Pass the broker's canonical response through unmodified, but only if it has +# one of the two shapes the protocol ever produces. Anything else (a dead or +# misbehaving broker, a transport-level fragment) is treated as a failure +# rather than forwarded verbatim. +case "$RESPONSE" in + '{"status":"error"}') printf '%s\n' "$RESPONSE" exit 0 - fi -done - -emit_error + ;; + '{"status":"ok","result":'*'}') + printf '%s\n' "$RESPONSE" + exit 0 + ;; + *) + emit_error + ;; +esac diff --git a/containers/sealed-probe/broker/broker.js b/containers/sealed-probe/broker/broker.js index d8452df98..e521f248b 100644 --- a/containers/sealed-probe/broker/broker.js +++ b/containers/sealed-probe/broker/broker.js @@ -2,123 +2,190 @@ const crypto = require('crypto'); const { - CANONICAL_ERROR_RESULT_JSON, - canonicalizeSealedProbeResult, - parseSealedProbeResult, + CANONICAL_ERROR_JSON, + canonicalOkJson, + parseAndValidateProbeOutput, + queryBitsForSchema, validateSealedProbeRequest, } = require('./protocol'); +const { createLedger } = require('./ledger'); +const { createRealClock, waitForBucket } = require('./scheduler'); const defaultWorkspace = require('./workspace'); const defaultRunner = require('./probe-runner'); /** - * The trusted sealed-probe broker. + * The trusted sealed-probe broker (protocol v2). * * Responsibilities, in order, for every request: * - * 1. consume one unit of the per-run invocation budget (atomically — Node's - * single-threaded event loop makes the check-and-increment indivisible - * because there is no `await` between them); - * 2. validate the request against the fixed protocol *before* any repository - * is copied or any container is launched; - * 3. map the normalized repo id through AWF's static seed map to an opaque + * 1. consume one unit of the per-run *invocation* budget (`maxInvocations`, + * an operational cap independent of the bits below) — atomically, since + * Node's single-threaded event loop makes the check-and-increment + * indivisible because there is no `await` between them; + * 2. validate the request — including the finite response schema — against + * the fixed protocol *before* any repository is copied or any container + * is launched; + * 3. compute that schema's maximum complete-transcript information charge + * and atomically debit it from the repository's per-run bit ledger + * (see `./ledger`); an invocation proceeds iff the charge fits the + * remaining balance — there is no separate per-query cap; + * 4. map the normalized repo id through AWF's static seed map to an opaque * seed directory the caller never sees or names; - * 4. build a fresh private writable copy and launch the probe with a fixed - * argument vector; - * 5. strictly validate and canonically re-serialize the result; - * 6. destroy the copy. + * 5. build a fresh private writable copy and launch the probe with a fixed + * argument vector, using a monotonic clock for every timing decision; + * 6. strictly validate the result against the approved schema and + * canonically re-serialize it — raw probe bytes/stdout/stderr/exit + * status never reach the caller; + * 7. destroy the private copy, then respond at the first timing bucket + * boundary at or after all secret-dependent processing completed (see + * `./scheduler`). * * Every failure at every step produces the identical canonical - * `{"result":"ERROR"}`. The reason is recorded in the protected audit log, + * `{"status":"error"}`. The reason is recorded in the protected audit log, * which is never mounted into the agent or a probe. * * Invocations are serialized. That bounds concurrent resource use and removes - * any cross-invocation race in workspace creation/teardown. + * any cross-invocation race in workspace creation/teardown/ledger access. */ function createBroker(params) { const { config, seedMap, runId, audit } = params; const workspace = params.workspace || defaultWorkspace; const runner = params.runner || defaultRunner; + const clock = params.clock || createRealClock(); + const ledger = params.ledger || createLedger(seedMap); let invocationsUsed = 0; let tail = Promise.resolve(); - async function execute(request) { + /** + * Executes one request and reports its canonical result through + * `respond` (called exactly once). The invocations run only through + * validation, ledger debit, workspace creation, probe launch, and result + * validation actually reach the point where the response must be + * time-bucketed; everything rejected before that responds immediately. + */ + async function execute(request, respond) { const invocationId = crypto.randomBytes(12).toString('hex'); + let responded = false; + const safeRespond = (json) => { + if (responded) return; + responded = true; + respond(json); + }; const validation = validateSealedProbeRequest(request); if (!validation.valid) { audit.failure(invocationId, 'invalid-request', validation.errors.join('; ')); - return CANONICAL_ERROR_RESULT_JSON; + safeRespond(CANONICAL_ERROR_JSON); + return; + } + const { privateRepo, schema, script } = validation.request; + const repoKey = privateRepo.toLowerCase(); + + const seed = seedMap.get(repoKey); + if (!seed) { + audit.failure(invocationId, 'repo-not-allowed', privateRepo); + safeRespond(CANONICAL_ERROR_JSON); + return; } - const seedId = seedMap.get(request.privateRepo.toLowerCase()); - if (!seedId) { - audit.failure(invocationId, 'repo-not-allowed', request.privateRepo); - return CANONICAL_ERROR_RESULT_JSON; + // Compute and debit the charge for THIS invocation's schema *before* + // copying a seed or launching Python. Every invocation may declare a + // different schema; there is no separate per-query cap — only whether + // this charge fits the repository's remaining run balance. + const charge = queryBitsForSchema(schema); + if (!ledger.tryDebit(repoKey, charge)) { + audit.failure(invocationId, 'bit-budget-exhausted', `repo=${privateRepo} charge=${charge}`); + safeRespond(CANONICAL_ERROR_JSON); + return; } - // Start the per-invocation deadline now, before workspace creation. - // createInvocationWorkspace is synchronous (fs.cpSync) and can block for - // the full seed copy duration — the deadline must cover that too. - const deadlineMs = config.timeoutSeconds * 1000; - const deadlineStart = Date.now(); + // From here on the charge is committed (never refunded) and every + // response must be time-bucketed: workspace creation and probe + // execution both run against secret repository content, so their + // latency alone is a signal. + const startMs = clock.nowMs(); let layout; + let failureReason; + let canonicalResult; + try { layout = workspace.createInvocationWorkspace({ config, invocationId, - seedId, - script: request.script, + seedId: seed.seedId, + script, }); } catch (error) { - audit.failure(invocationId, 'workspace-create-failed', error.message); - safeDestroy(invocationId); - return CANONICAL_ERROR_RESULT_JSON; - } - - const elapsedMs = Date.now() - deadlineStart; - if (elapsedMs >= deadlineMs) { - audit.failure(invocationId, 'timeout', 'workspace-creation-overran-deadline'); - safeDestroy(invocationId); - return CANONICAL_ERROR_RESULT_JSON; + failureReason = ['workspace-create-failed', error.message]; } - const remainingMs = deadlineMs - elapsedMs; - let result = CANONICAL_ERROR_RESULT_JSON; - try { - const run = await runner.runProbeContainer({ config, runId, invocationId, timeoutMs: remainingMs }); - - if (run.timedOut) { - audit.failure(invocationId, 'timeout'); - } else if (run.exitCode !== 0) { - audit.failure(invocationId, 'non-zero-exit', `exit=${run.exitCode}`); + if (layout) { + const remainingMs = config.timeoutSeconds * 1000 - (clock.nowMs() - startMs); + if (remainingMs <= 0) { + failureReason = ['timeout', 'workspace-creation-overran-deadline']; } else { - const raw = workspace.readProbeOutput(layout.outPath); - if (raw === undefined) { - // Covers a missing file, an oversized file, invalid UTF-8, and any - // non-regular replacement (symlink/FIFO/device/socket). - audit.failure(invocationId, 'unreadable-output'); - } else { - const parsed = parseSealedProbeResult(raw, request.outcomes); - if (parsed.result === 'ERROR') { - audit.failure(invocationId, 'nonconformant-output'); + try { + const run = await runner.runProbeContainer({ config, runId, invocationId, timeoutMs: remainingMs }); + if (run.timedOut) { + failureReason = ['timeout']; + } else if (run.exitCode !== 0) { + failureReason = ['non-zero-exit', `exit=${run.exitCode}`]; } else { - audit.invocation({ invocationId, repo: request.privateRepo }); - // Never echo probe bytes: re-serialize the validated symbol ourselves. - result = canonicalizeSealedProbeResult(parsed.result); + const raw = workspace.readProbeOutput(layout.outPath); + if (raw === undefined) { + // Covers a missing file, an oversized file, invalid UTF-8, and + // any non-regular replacement (symlink/FIFO/device/socket). + failureReason = ['unreadable-output']; + } else { + const parsed = parseAndValidateProbeOutput(raw, schema); + if (!parsed.ok) { + failureReason = ['nonconformant-output']; + } else { + canonicalResult = parsed.canonical; + } + } } + } catch (error) { + failureReason = ['launch-failed', error.message]; } } - } catch (error) { - audit.failure(invocationId, 'launch-failed', error.message); } - if (!safeDestroy(invocationId)) { - return CANONICAL_ERROR_RESULT_JSON; + // Teardown is part of the observable operation: repository size and tree + // shape can affect deletion time, and queued requests must not expose that + // duration outside the charged timing bucket. + if (layout && !safeDestroy(invocationId)) { + failureReason = ['cleanup-failed']; + canonicalResult = undefined; } - return result; + + const elapsedMs = clock.nowMs() - startMs; + const { bucketMs, overflowed } = await waitForBucket(startMs, elapsedMs, clock); + + if (overflowed) { + // Fail closed: processing (not the script itself, which is bounded by + // `sealedProbes.timeout <= largest bucket`) overran every configured + // bucket — pathological infrastructure latency. Never emit a + // successful result at unbucketed timing. + audit.failure(invocationId, 'timing-bucket-overflow', failureReason ? failureReason.join(':') : undefined); + safeRespond(CANONICAL_ERROR_JSON); + } else if (canonicalResult !== undefined) { + audit.invocation({ + invocationId, + repo: privateRepo, + sensitivity: seed.sensitivity, + bits: charge, + bucketMs, + }); + safeRespond(canonicalOkJson(canonicalResult)); + } else { + audit.failure(invocationId, failureReason ? failureReason[0] : 'unknown', failureReason ? failureReason[1] : undefined); + safeRespond(CANONICAL_ERROR_JSON); + } + } function safeDestroy(invocationId) { @@ -133,23 +200,38 @@ function createBroker(params) { return { /** - * Handles one request and resolves to the canonical result JSON. + * Handles one request. `respond` is called exactly once with the + * canonical result JSON, as soon as it is ready to send (which, for any + * invocation that reached workspace creation, is exactly at a timing + * bucket boundary — never earlier). The returned promise resolves once + * all broker-side bookkeeping for the invocation (including workspace + * cleanup) is complete; it carries no value and exists only to let the + * caller serialize/await broker shutdown. * * Requests are queued so at most one probe runs at a time. */ - handle(request) { - // Budget is consumed per *response*, not per launch: every response the - // agent observes is one of the four symbols, so every response — including - // a rejection — is what the budget bounds. + handle(request, respond) { + let responded = false; + const safeRespond = (json) => { + if (responded) return; + responded = true; + respond(json); + }; + + // The invocation-count cap is operational and independent of the bit + // ledger: it is consumed per *response*, not per launch, so every + // response the agent observes — including a rejection — counts + // against it. if (invocationsUsed >= config.maxInvocations) { - audit.failure('budget', 'budget-exhausted', `max=${config.maxInvocations}`); - return Promise.resolve(CANONICAL_ERROR_RESULT_JSON); + audit.failure('budget', 'invocation-count-exhausted', `max=${config.maxInvocations}`); + safeRespond(CANONICAL_ERROR_JSON); + return Promise.resolve(); } invocationsUsed += 1; - const queued = tail.then(() => execute(request)).catch((error) => { + const queued = tail.then(() => execute(request, safeRespond)).catch((error) => { audit.failure('queue', 'unexpected-error', error && error.message); - return CANONICAL_ERROR_RESULT_JSON; + safeRespond(CANONICAL_ERROR_JSON); }); tail = queued.then( () => undefined, @@ -162,6 +244,9 @@ function createBroker(params) { get invocationsUsed() { return invocationsUsed; }, + + /** @internal Exposed for tests. */ + ledger, }; } diff --git a/containers/sealed-probe/broker/config.js b/containers/sealed-probe/broker/config.js index 1581201ef..352b62f14 100644 --- a/containers/sealed-probe/broker/config.js +++ b/containers/sealed-probe/broker/config.js @@ -2,6 +2,8 @@ const fs = require('fs'); const path = require('path'); +const { TIMING_BUCKETS_MS } = require('./protocol'); +const { SEALED_PROBE_SENSITIVITY_RUN_BITS } = require('./sensitivity'); /** * Broker configuration. @@ -48,6 +50,24 @@ function parsePositiveInt(name, fallback) { return parsed; } +/** + * Parses the per-invocation timeout, additionally re-enforcing (defense in + * depth; AWF's host-side preflight already rejects an out-of-range value + * before this container ever starts) that it cannot exceed the largest + * response-timing bucket. See `./scheduler` for why that ceiling matters. + */ +function parseTimeoutSeconds() { + const maxSeconds = TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] / 1000; + const parsed = parsePositiveInt('AWF_SEALED_PROBE_TIMEOUT', 30); + if (parsed > maxSeconds) { + throw new Error( + `Environment variable AWF_SEALED_PROBE_TIMEOUT must be at most ${maxSeconds} ` + + '(the largest response-timing bucket, in seconds)', + ); + } + return parsed; +} + function loadConfig() { const memoryLimit = process.env.AWF_SEALED_PROBE_MEMORY || '512m'; if (!/^[1-9][0-9]*[bkmgBKMG]$/.test(memoryLimit)) { @@ -77,7 +97,7 @@ function loadConfig() { // which is not necessarily the broker's (ARC/DinD split filesystems). hostWorkDir: requireEnv('AWF_SEALED_PROBE_HOST_WORK_DIR'), dockerRuntime, - timeoutSeconds: parsePositiveInt('AWF_SEALED_PROBE_TIMEOUT', 30), + timeoutSeconds: parseTimeoutSeconds(), maxInvocations: parsePositiveInt('AWF_SEALED_PROBE_MAX_INVOCATIONS', 32), memoryLimit, socketUid: parsePositiveInt('AWF_SEALED_PROBE_SOCKET_UID', 0), @@ -86,16 +106,18 @@ function loadConfig() { } /** - * Loads the AWF-generated repo → opaque seed id map. + * Loads the AWF-generated repo → { opaque seed id, sensitivity } map. * * The map is the *only* way a repository can be selected: a request supplies * a normalized `owner/repo` id, which is looked up here. Callers never supply - * a path, and an unknown id is simply absent from the map. + * a path, and an unknown id is simply absent from the map. Sensitivity is + * carried in the (AWF-trusted, host-written) map itself, never accepted from + * a request — a request cannot choose or override its repository's budget. */ function loadSeedMap(seedMapPath) { const parsed = JSON.parse(fs.readFileSync(seedMapPath, 'utf8')); - if (!parsed || parsed.version !== 1 || !Array.isArray(parsed.seeds)) { - throw new Error('Seed map is malformed'); + if (!parsed || parsed.version !== 2 || !Array.isArray(parsed.seeds)) { + throw new Error('Seed map is malformed or is an unsupported version'); } if (typeof parsed.runId !== 'string' || !/^[0-9a-f]{8,}$/.test(parsed.runId)) { throw new Error('Seed map has no usable runId'); @@ -103,7 +125,12 @@ function loadSeedMap(seedMapPath) { const seeds = new Map(); for (const entry of parsed.seeds) { - if (!entry || typeof entry.repo !== 'string' || typeof entry.seedId !== 'string') { + if ( + !entry + || typeof entry.repo !== 'string' + || typeof entry.seedId !== 'string' + || !Object.prototype.hasOwnProperty.call(SEALED_PROBE_SENSITIVITY_RUN_BITS, entry.sensitivity) + ) { throw new Error('Seed map entry is malformed'); } // Seed ids are AWF-generated opaque hex names. Re-validating here means a @@ -111,7 +138,7 @@ function loadSeedMap(seedMapPath) { if (!/^[0-9a-f]{16,64}$/.test(entry.seedId)) { throw new Error('Seed map entry has an unexpected seed id'); } - seeds.set(entry.repo.toLowerCase(), entry.seedId); + seeds.set(entry.repo.toLowerCase(), { seedId: entry.seedId, sensitivity: entry.sensitivity }); } return { runId: parsed.runId, seeds }; diff --git a/containers/sealed-probe/broker/framing.js b/containers/sealed-probe/broker/framing.js index 539cab1dd..c2835ea6a 100644 --- a/containers/sealed-probe/broker/framing.js +++ b/containers/sealed-probe/broker/framing.js @@ -1,36 +1,48 @@ 'use strict'; -const { MAX_SCRIPT_BYTES, OUTCOME_COUNT } = require('./protocol'); +const { MAX_SCHEMA_BYTES, MAX_SCRIPT_BYTES, strictParseJson } = require('./protocol'); /** - * Wire framing for the agent → broker request. + * Wire framing for the agent → broker request (protocol v2). * - * The request is deliberately *not* caller-supplied JSON: the agent-facing - * wrapper is a POSIX shell script, and asking it to emit correct JSON for - * arbitrary script bytes would be both fragile and an unnecessary parser on - * the untrusted path. Instead the three scalar fields travel as fixed headers - * and the script travels as the raw body, and the broker assembles the - * canonical request object itself. + * The request is deliberately *not* caller-supplied JSON at the transport + * level: the agent-facing wrapper is a POSIX shell script, and asking it to + * emit correct JSON for arbitrary script bytes would be both fragile and an + * unnecessary parser on the untrusted path. Instead the scalar/JSON fields + * travel as fixed headers and the script travels as the raw body, and the + * broker assembles the canonical `{privateRepo, schema, script}` request + * object itself. * - * The assembled object is then validated by the shared protocol rules, so the - * framing adds no new degrees of freedom. + * The schema travels base64url-encoded in a header (not the body) because + * HTTP header values are restricted to a printable-ASCII-ish subset, while a + * `const`/`enum` schema literal may contain arbitrary non-control UTF-8. The + * assembled object is then validated by the shared protocol rules + * (`validateSealedProbeRequest`), so this framing layer adds no new degrees + * of freedom — it only assembles the object and enforces cheap size/shape + * bounds before that shared validation runs. */ /** Supported request framing version. */ -const PROBE_PROTOCOL_VERSION = '1'; +const PROBE_PROTOCOL_VERSION = '2'; const VERSION_HEADER = 'x-awf-probe-version'; const REPO_HEADER = 'x-awf-repo'; -const OUTCOME_HEADERS = ['x-awf-outcome-1', 'x-awf-outcome-2', 'x-awf-outcome-3']; +const SCHEMA_HEADER = 'x-awf-schema-b64'; /** Every header the broker accepts. Anything else is a rejected control. */ -const ALLOWED_AWF_HEADERS = new Set([VERSION_HEADER, REPO_HEADER, ...OUTCOME_HEADERS]); +const ALLOWED_AWF_HEADERS = new Set([VERSION_HEADER, REPO_HEADER, SCHEMA_HEADER]); + +/** Base64url alphabet only (no padding, no `+`/`/`). */ +const BASE64URL_PATTERN = /^[A-Za-z0-9_-]+$/; + +/** Generous ceiling on the encoded header length for a schema of at most `MAX_SCHEMA_BYTES`. */ +const MAX_SCHEMA_HEADER_LENGTH = Math.ceil((MAX_SCHEMA_BYTES * 4) / 3) + 4; /** * Rejects duplicated or unexpected `x-awf-*` headers. * * Duplicates matter because Node joins repeated headers with `", "`, which - * would silently synthesize a fourth outcome value out of two. + * would silently corrupt a base64url value or a repo slug. */ function validateRawHeaders(rawHeaders) { const seen = new Set(); @@ -48,6 +60,25 @@ function validateRawHeaders(rawHeaders) { return undefined; } +/** Decodes and UTF-8-validates the base64url schema header. */ +function decodeSchemaHeader(value) { + if (typeof value !== 'string' || value.length === 0 || value.length > MAX_SCHEMA_HEADER_LENGTH) { + return undefined; + } + if (!BASE64URL_PATTERN.test(value)) return undefined; + + let decoded; + try { + decoded = Buffer.from(value, 'base64url'); + } catch { + return undefined; + } + const text = decoded.toString('utf8'); + // Reject anything that was not valid UTF-8 to begin with (round-trip check). + if (!Buffer.from(text, 'utf8').equals(decoded)) return undefined; + return text; +} + /** * Assembles the canonical request object from a framed HTTP request. * @@ -66,19 +97,17 @@ function buildRequestFromFrame(headers, rawHeaders, script) { return { error: 'missing repository selector' }; } - const outcomes = []; - for (const header of OUTCOME_HEADERS) { - const value = headers[header]; - if (typeof value !== 'string') { - return { error: `missing outcome header: ${header}` }; - } - outcomes.push(value); + const schemaText = decodeSchemaHeader(headers[SCHEMA_HEADER]); + if (schemaText === undefined) { + return { error: 'missing or malformed schema header' }; } - if (outcomes.length !== OUTCOME_COUNT) { - return { error: 'wrong number of outcomes' }; + + const parsedSchema = strictParseJson(schemaText); + if (!parsedSchema) { + return { error: 'schema header is not valid JSON' }; } - return { request: { privateRepo, outcomes, script } }; + return { request: { privateRepo, schema: parsedSchema.value, script } }; } /** @@ -128,7 +157,7 @@ module.exports = { PROBE_PROTOCOL_VERSION, VERSION_HEADER, REPO_HEADER, - OUTCOME_HEADERS, + SCHEMA_HEADER, buildRequestFromFrame, readBoundedBody, }; diff --git a/containers/sealed-probe/broker/ledger.js b/containers/sealed-probe/broker/ledger.js new file mode 100644 index 000000000..3609de346 --- /dev/null +++ b/containers/sealed-probe/broker/ledger.js @@ -0,0 +1,57 @@ +'use strict'; + +const { SEALED_PROBE_SENSITIVITY_RUN_BITS } = require('./sensitivity'); + +/** + * Per-repository information-budget ledger. + * + * There is no per-query cap: every invocation may use an arbitrarily + * different schema, and its maximum complete-transcript charge (see + * `queryBitsForSchema` in `./protocol`) is computed and debited from the + * repository's shared run balance *before* a seed is copied or Python is + * launched. An invocation is allowed iff its charge fits the remaining + * balance. Charges are never refunded, regardless of the invocation's + * outcome (success, failure, or timeout) — the broker committed to + * revealing up to that many bits of signal the moment it decided to run. + * + * The ledger's scope is one broker process — one AWF run. The broker has no + * durable identity or storage across runs. + */ + +/** + * Builds a ledger from the loaded seed map. + * + * @param seeds `Map` as returned + * by `config.loadSeedMap`. + */ +function createLedger(seeds) { + const remaining = new Map(); + for (const [repoKey, seed] of seeds) { + remaining.set(repoKey, SEALED_PROBE_SENSITIVITY_RUN_BITS[seed.sensitivity]); + } + + return { + /** + * Atomically checks and debits `bits` from `repoKey`'s remaining + * balance. Returns `true` (and debits) iff the charge is affordable; + * returns `false` (and leaves the balance untouched) otherwise. Safe + * to call synchronously with no intervening `await` — Node's + * single-threaded event loop makes this indivisible. + */ + tryDebit(repoKey, bits) { + if (!remaining.has(repoKey)) return false; + const current = remaining.get(repoKey); + if (current === null) return true; // unmetered (public) + if (bits > current) return false; + remaining.set(repoKey, current - bits); + return true; + }, + + /** Returns the remaining balance for a repo, or `undefined` if unknown. */ + remainingBits(repoKey) { + return remaining.get(repoKey); + }, + }; +} + +module.exports = { createLedger }; diff --git a/containers/sealed-probe/broker/protocol.js b/containers/sealed-probe/broker/protocol.js index dcb9ef0a7..af2facc0e 100644 --- a/containers/sealed-probe/broker/protocol.js +++ b/containers/sealed-probe/broker/protocol.js @@ -1,7 +1,7 @@ 'use strict'; /** - * Sealed-probe request/result protocol — broker-side implementation. + * Sealed-probe request/result protocol v2 — broker-side implementation. * * This is a deliberate, behaviour-identical mirror of `src/sealed-probe/ * protocol.ts`. The broker runs inside its own container image and cannot @@ -12,17 +12,34 @@ * Do not "improve" one side without the other. */ -const OUTCOME_COUNT = 3; -const RESERVED_ERROR_OUTCOME = 'ERROR'; -const MAX_OUTCOME_BYTES = 64; +const PROBE_PROTOCOL_VERSION = 2; + +const MAX_SCHEMA_BYTES = 4096; +const MAX_SCHEMA_DEPTH = 6; +const MAX_SCHEMA_NODES = 64; +const MAX_ENUM_VALUES = 4096; +const MAX_LITERAL_STRING_BYTES = 64; +const MAX_OBJECT_FIELDS = 16; +const MAX_TUPLE_ITEMS = 16; +const MAX_ARRAY_LENGTH = 64; +const MAX_UNION_VARIANTS = 16; const MAX_SCRIPT_BYTES = 64 * 1024; -const MAX_REQUEST_BYTES = 256 * 1024; -const MAX_RESULT_BYTES = 1024; +const MAX_REQUEST_BYTES = MAX_SCRIPT_BYTES + MAX_SCHEMA_BYTES + 1024; +const MAX_RESULT_BYTES = 8 * 1024; const MAX_PRIVATE_REPO_LENGTH = 140; +const TIMING_BUCKETS_MS = [10, 100, 1_000, 10_000, 60_000, 600_000]; + +function ceilLog2(n) { + return ceilLog2BigInt(BigInt(n)); +} + +const TIMING_BUCKET_BITS = ceilLog2(TIMING_BUCKETS_MS.length); +const RESULT_STATUS_BIT_COST = 1; + const SEALED_PROBE_REPO_PATTERN = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})\/(?!\.\.?$)(?!.*\.\.)[A-Za-z0-9._-]{1,100}$/; -const OUTCOME_PATTERN = /^[A-Za-z][A-Za-z0-9_-]{0,63}$/; +const IDENTIFIER_PATTERN = /^[A-Za-z][A-Za-z0-9_]{0,63}$/; function utf8ByteLength(value) { return Buffer.byteLength(value, 'utf8'); @@ -36,113 +53,335 @@ function hasControlCharacters(value) { return false; } -/** Builds the closed result schema the broker enforces for a request. */ -function buildSealedProbeResultSchema(outcomes) { - return { - type: 'object', - additionalProperties: false, - required: ['result'], - properties: { - result: { - type: 'string', - enum: [...outcomes, RESERVED_ERROR_OUTCOME], - }, - }, - }; +function isValidLiteral(value) { + if (value === null) return true; + if (typeof value === 'boolean') return true; + if (typeof value === 'number') return Number.isInteger(value) && Number.isSafeInteger(value); + if (typeof value === 'string') { + return !hasControlCharacters(value) && utf8ByteLength(value) <= MAX_LITERAL_STRING_BYTES; + } + return false; } -function validateOutcome(outcome) { - if (typeof outcome !== 'string') return 'outcome must be a string'; - if (outcome.length === 0) return 'outcome must not be empty'; - if (outcome === RESERVED_ERROR_OUTCOME) { - return `outcome must not use the reserved value "${RESERVED_ERROR_OUTCOME}"`; - } - if (hasControlCharacters(outcome)) return 'outcome must not contain control characters'; - if (utf8ByteLength(outcome) > MAX_OUTCOME_BYTES) { - return `outcome must be at most ${MAX_OUTCOME_BYTES} UTF-8 bytes`; - } - if (!OUTCOME_PATTERN.test(outcome)) { - return 'outcome must be an ASCII identifier starting with a letter and containing only letters, digits, "_" or "-"'; - } +function literalTypeTag(value) { + return value === null ? 'null' : typeof value; +} + +function failSchema(ctx, message) { + if (ctx.errors.length === 0) ctx.errors.push(message); return undefined; } -function validateOutcomes(outcomes) { - if (!Array.isArray(outcomes)) { - return [`outcomes must be an array of exactly ${OUTCOME_COUNT} strings`]; +function buildSchemaNode(raw, ctx, depth) { + if (ctx.errors.length > 0) return undefined; + if (depth > MAX_SCHEMA_DEPTH) { + return failSchema(ctx, `schema exceeds maximum depth of ${MAX_SCHEMA_DEPTH}`); } - - const errors = []; - if (outcomes.length !== OUTCOME_COUNT) { - errors.push(`outcomes must contain exactly ${OUTCOME_COUNT} entries`); + ctx.nodeCount += 1; + if (ctx.nodeCount > MAX_SCHEMA_NODES) { + return failSchema(ctx, `schema exceeds maximum node count of ${MAX_SCHEMA_NODES}`); } - - outcomes.forEach((outcome, index) => { - const error = validateOutcome(outcome); - if (error) errors.push(`outcomes[${index}]: ${error}`); - }); - - const stringOutcomes = outcomes.filter((o) => typeof o === 'string'); - if (new Set(stringOutcomes).size !== stringOutcomes.length) { - errors.push('outcomes must be unique'); + if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { + return failSchema(ctx, 'schema node must be a JSON object'); } - return errors; + const node = raw; + switch (node.type) { + case 'const': { + if (Object.keys(node).length !== 2 || !('value' in node)) { + return failSchema(ctx, 'const schema must have exactly "type" and "value"'); + } + if (!isValidLiteral(node.value)) { + return failSchema(ctx, 'const value must be a bounded string, a safe integer, a boolean, or null'); + } + return { type: 'const', value: node.value }; + } + case 'boolean': { + if (Object.keys(node).length !== 1) { + return failSchema(ctx, 'boolean schema must have only "type"'); + } + return { type: 'boolean' }; + } + case 'enum': { + if (Object.keys(node).length !== 2 || !('values' in node)) { + return failSchema(ctx, 'enum schema must have exactly "type" and "values"'); + } + const values = node.values; + if (!Array.isArray(values) || values.length === 0) { + return failSchema(ctx, 'enum values must be a non-empty array'); + } + if (values.length > MAX_ENUM_VALUES) { + return failSchema(ctx, `enum values must contain at most ${MAX_ENUM_VALUES} entries`); + } + for (const value of values) { + if (!isValidLiteral(value)) { + return failSchema(ctx, 'enum values must be bounded strings, safe integers, booleans, or null'); + } + } + const firstTag = literalTypeTag(values[0]); + if (!values.every((value) => literalTypeTag(value) === firstTag)) { + return failSchema(ctx, 'enum values must all be the same JSON type'); + } + const uniqueCount = new Set(values.map((value) => JSON.stringify(value))).size; + if (uniqueCount !== values.length) { + return failSchema(ctx, 'enum values must be unique'); + } + return { type: 'enum', values }; + } + case 'integer': { + if (Object.keys(node).length !== 3 || !('minimum' in node) || !('maximum' in node)) { + return failSchema(ctx, 'integer schema must have exactly "type", "minimum", and "maximum"'); + } + const { minimum, maximum } = node; + if (typeof minimum !== 'number' || !Number.isSafeInteger(minimum)) { + return failSchema(ctx, 'integer minimum must be a safe integer'); + } + if (typeof maximum !== 'number' || !Number.isSafeInteger(maximum)) { + return failSchema(ctx, 'integer maximum must be a safe integer'); + } + if (maximum < minimum) { + return failSchema(ctx, 'integer maximum must be >= minimum'); + } + return { type: 'integer', minimum, maximum }; + } + case 'object': { + if (Object.keys(node).length !== 2 || !('fields' in node)) { + return failSchema(ctx, 'object schema must have exactly "type" and "fields"'); + } + const fieldsRaw = node.fields; + if (typeof fieldsRaw !== 'object' || fieldsRaw === null || Array.isArray(fieldsRaw)) { + return failSchema(ctx, 'object "fields" must be a JSON object mapping field name to schema'); + } + const fieldNames = Object.keys(fieldsRaw); + if (fieldNames.length === 0) { + return failSchema(ctx, 'object schema must declare at least one field'); + } + if (fieldNames.length > MAX_OBJECT_FIELDS) { + return failSchema(ctx, `object schema must declare at most ${MAX_OBJECT_FIELDS} fields`); + } + for (const name of fieldNames) { + if (!IDENTIFIER_PATTERN.test(name)) { + return failSchema(ctx, `object field name "${name}" is not a bounded ASCII identifier`); + } + } + const fields = []; + for (const name of fieldNames) { + const child = buildSchemaNode(fieldsRaw[name], ctx, depth + 1); + if (!child) return undefined; + fields.push({ name, schema: child }); + } + return { type: 'object', fields }; + } + case 'tuple': { + if (Object.keys(node).length !== 2 || !('items' in node)) { + return failSchema(ctx, 'tuple schema must have exactly "type" and "items"'); + } + const itemsRaw = node.items; + if (!Array.isArray(itemsRaw) || itemsRaw.length === 0) { + return failSchema(ctx, 'tuple "items" must be a non-empty array'); + } + if (itemsRaw.length > MAX_TUPLE_ITEMS) { + return failSchema(ctx, `tuple schema must declare at most ${MAX_TUPLE_ITEMS} items`); + } + const items = []; + for (const itemRaw of itemsRaw) { + const child = buildSchemaNode(itemRaw, ctx, depth + 1); + if (!child) return undefined; + items.push(child); + } + return { type: 'tuple', items }; + } + case 'array': { + if (Object.keys(node).length !== 3 || !('items' in node) || !('length' in node)) { + return failSchema(ctx, 'array schema must have exactly "type", "items", and "length"'); + } + const { length } = node; + if (typeof length !== 'number' || !Number.isInteger(length) || length < 0 || length > MAX_ARRAY_LENGTH) { + return failSchema(ctx, `array "length" must be an integer between 0 and ${MAX_ARRAY_LENGTH}`); + } + const child = buildSchemaNode(node.items, ctx, depth + 1); + if (!child) return undefined; + return { type: 'array', items: child, length }; + } + case 'union': { + if (Object.keys(node).length !== 2 || !('variants' in node)) { + return failSchema(ctx, 'union schema must have exactly "type" and "variants"'); + } + const variantsRaw = node.variants; + if (typeof variantsRaw !== 'object' || variantsRaw === null || Array.isArray(variantsRaw)) { + return failSchema(ctx, 'union "variants" must be a JSON object mapping tag to schema'); + } + const tags = Object.keys(variantsRaw); + if (tags.length === 0) { + return failSchema(ctx, 'union schema must declare at least one variant'); + } + if (tags.length > MAX_UNION_VARIANTS) { + return failSchema(ctx, `union schema must declare at most ${MAX_UNION_VARIANTS} variants`); + } + for (const tag of tags) { + if (!IDENTIFIER_PATTERN.test(tag)) { + return failSchema(ctx, `union tag "${tag}" is not a bounded ASCII identifier`); + } + } + const variants = []; + for (const tag of tags) { + const child = buildSchemaNode(variantsRaw[tag], ctx, depth + 1); + if (!child) return undefined; + variants.push({ tag, schema: child }); + } + return { type: 'union', variants }; + } + default: + return failSchema( + ctx, + 'schema node "type" must be one of: const, boolean, enum, integer, object, tuple, array, union', + ); + } } -function validateSealedProbeRequest(request) { - if (typeof request !== 'object' || request === null || Array.isArray(request)) { - return { valid: false, errors: ['request must be a JSON object'] }; +function validateSchema(raw) { + let serialized; + try { + serialized = JSON.stringify(raw) ?? ''; + } catch { + return { valid: false, errors: ['schema must be JSON-serializable'] }; } - - const errors = []; - const { privateRepo, outcomes, script } = request; - const allowedKeys = new Set(['privateRepo', 'outcomes', 'script']); - for (const key of Object.keys(request)) { - if (!allowedKeys.has(key)) { - errors.push(`request.${key} is not supported`); - } + if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { + return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; } - if (typeof privateRepo !== 'string' || privateRepo.length === 0) { - errors.push('privateRepo must be a non-empty string'); - } else if ( - privateRepo.length > MAX_PRIVATE_REPO_LENGTH - || !SEALED_PROBE_REPO_PATTERN.test(privateRepo) - ) { - errors.push( - 'privateRepo must be an "owner/repo" slug (no scheme, host, path traversal, query, fragment, or wildcard)', - ); + const ctx = { errors: [], nodeCount: 0 }; + const schema = buildSchemaNode(raw, ctx, 0); + if (!schema || ctx.errors.length > 0) { + return { valid: false, errors: ctx.errors.length > 0 ? ctx.errors : ['invalid schema'] }; } + return { valid: true, schema }; +} - errors.push(...validateOutcomes(outcomes)); - - if (typeof script !== 'string' || script.length === 0) { - errors.push('script must be a non-empty string'); - } else if (utf8ByteLength(script) > MAX_SCRIPT_BYTES) { - errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); +function ceilLog2BigInt(n) { + if (n <= 1n) return 0; + let bits = 0; + let remainder = n - 1n; + while (remainder > 0n) { + remainder >>= 1n; + bits += 1; } + return bits; +} - let serialized; - try { - serialized = JSON.stringify(request); - } catch { - errors.push('request must be JSON-serializable'); - } - if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { - errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); +function schemaCardinality(schema) { + switch (schema.type) { + case 'const': + return 1n; + case 'boolean': + return 2n; + case 'enum': + return BigInt(schema.values.length); + case 'integer': + return BigInt(schema.maximum) - BigInt(schema.minimum) + 1n; + case 'object': + return schema.fields.reduce((acc, field) => acc * schemaCardinality(field.schema), 1n); + case 'tuple': + return schema.items.reduce((acc, item) => acc * schemaCardinality(item), 1n); + case 'array': + return schemaCardinality(schema.items) ** BigInt(schema.length); + case 'union': + return schema.variants.reduce((acc, variant) => acc + schemaCardinality(variant.schema), 0n); + default: + throw new Error(`unreachable schema type: ${schema.type}`); } +} - if (errors.length > 0) return { valid: false, errors }; - return { valid: true }; +function queryBitsForSchema(schema) { + return RESULT_STATUS_BIT_COST + ceilLog2BigInt(schemaCardinality(schema)) + TIMING_BUCKET_BITS; } -function canonicalizeSealedProbeResult(result) { - return JSON.stringify({ result }); +function jsonLiteralEquals(value, literal) { + if (literal === null) return value === null; + if (typeof literal === 'number') return typeof value === 'number' && Number.isInteger(value) && value === literal; + return value === literal; } -const CANONICAL_ERROR_RESULT_JSON = canonicalizeSealedProbeResult(RESERVED_ERROR_OUTCOME); +function validateValueAgainstSchema(schema, value) { + switch (schema.type) { + case 'const': + return jsonLiteralEquals(value, schema.value); + case 'boolean': + return typeof value === 'boolean'; + case 'enum': + return schema.values.some((candidate) => jsonLiteralEquals(value, candidate)); + case 'integer': + return ( + typeof value === 'number' + && Number.isInteger(value) + && value >= schema.minimum + && value <= schema.maximum + ); + case 'object': { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + if (Object.keys(value).length !== schema.fields.length) return false; + return schema.fields.every( + (field) => + Object.prototype.hasOwnProperty.call(value, field.name) + && validateValueAgainstSchema(field.schema, value[field.name]), + ); + } + case 'tuple': + return ( + Array.isArray(value) + && value.length === schema.items.length + && schema.items.every((itemSchema, index) => validateValueAgainstSchema(itemSchema, value[index])) + ); + case 'array': + return ( + Array.isArray(value) + && value.length === schema.length + && value.every((item) => validateValueAgainstSchema(schema.items, item)) + ); + case 'union': { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + if (Object.keys(value).length !== 2 || !('tag' in value) || !('value' in value) || typeof value.tag !== 'string') { + return false; + } + const variant = schema.variants.find((candidate) => candidate.tag === value.tag); + return variant !== undefined && validateValueAgainstSchema(variant.schema, value.value); + } + default: + return false; + } +} +function canonicalizeSchemaValue(schema, value) { + switch (schema.type) { + case 'const': + return JSON.stringify(schema.value); + case 'boolean': + case 'enum': + case 'integer': + return JSON.stringify(value); + case 'object': { + const parts = schema.fields.map( + (field) => `${JSON.stringify(field.name)}:${canonicalizeSchemaValue(field.schema, value[field.name])}`, + ); + return `{${parts.join(',')}}`; + } + case 'tuple': + return `[${schema.items.map((itemSchema, index) => canonicalizeSchemaValue(itemSchema, value[index])).join(',')}]`; + case 'array': + return `[${value.map((item) => canonicalizeSchemaValue(schema.items, item)).join(',')}]`; + case 'union': { + const variant = schema.variants.find((candidate) => candidate.tag === value.tag); + if (!variant) return 'null'; + return `{"tag":${JSON.stringify(value.tag)},"value":${canonicalizeSchemaValue(variant.schema, value.value)}}`; + } + default: + throw new Error(`unreachable schema type: ${schema.type}`); + } +} + +// ── Strict JSON parsing (no `JSON.parse`) ──────────────────────────────────── + +const MAX_JSON_PARSE_DEPTH = 32; const JSON_WHITESPACE = new Set([' ', '\t', '\n', '\r']); function skipJsonWhitespace(text, index) { @@ -159,9 +398,7 @@ function parseJsonStringLiteral(text, start) { while (i < text.length) { const ch = text[i]; - if (ch === '"') { - return { value, endIndex: i + 1 }; - } + if (ch === '"') return { value, endIndex: i + 1 }; if (ch === '\\') { const escape = text[i + 1]; @@ -187,7 +424,6 @@ function parseJsonStringLiteral(text, start) { } if (ch.charCodeAt(0) < 0x20) return undefined; - value += ch; i++; } @@ -195,67 +431,196 @@ function parseJsonStringLiteral(text, start) { return undefined; } -function tryExtractStrictResultValue(raw) { - let i = skipJsonWhitespace(raw, 0); +function parseJsonNumber(text, start) { + let i = start; + if (text[i] === '-') i++; + if (text[i] === '0') { + i++; + } else if (text[i] >= '1' && text[i] <= '9') { + while (text[i] >= '0' && text[i] <= '9') i++; + } else { + return undefined; + } + if (text[i] === '.') { + i++; + if (!(text[i] >= '0' && text[i] <= '9')) return undefined; + while (text[i] >= '0' && text[i] <= '9') i++; + } + if (text[i] === 'e' || text[i] === 'E') { + i++; + if (text[i] === '+' || text[i] === '-') i++; + if (!(text[i] >= '0' && text[i] <= '9')) return undefined; + while (text[i] >= '0' && text[i] <= '9') i++; + } + const raw = text.slice(start, i); + const value = Number(raw); + if (!Number.isFinite(value)) return undefined; + return { value, endIndex: i }; +} - if (raw[i] !== '{') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +function parseJsonValue(text, index, depth) { + if (depth > MAX_JSON_PARSE_DEPTH) return undefined; + const ch = text[index]; - if (raw.slice(i, i + 8) !== '"result"') return undefined; - i += 8; - i = skipJsonWhitespace(raw, i); + if (ch === '{') return parseJsonObject(text, index, depth); + if (ch === '[') return parseJsonArray(text, index, depth); + if (ch === '"') { + const literal = parseJsonStringLiteral(text, index); + return literal && { value: literal.value, endIndex: literal.endIndex }; + } + if (text.startsWith('true', index)) return { value: true, endIndex: index + 4 }; + if (text.startsWith('false', index)) return { value: false, endIndex: index + 5 }; + if (text.startsWith('null', index)) return { value: null, endIndex: index + 4 }; + if (ch === '-' || (ch >= '0' && ch <= '9')) return parseJsonNumber(text, index); + return undefined; +} - if (raw[i] !== ':') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +function parseJsonObject(text, index, depth) { + let i = skipJsonWhitespace(text, index + 1); + const obj = {}; + if (text[i] === '}') return { value: obj, endIndex: i + 1 }; + + for (;;) { + i = skipJsonWhitespace(text, i); + const key = parseJsonStringLiteral(text, i); + if (!key) return undefined; + i = skipJsonWhitespace(text, key.endIndex); + if (text[i] !== ':') return undefined; + i = skipJsonWhitespace(text, i + 1); + const value = parseJsonValue(text, i, depth + 1); + if (!value) return undefined; + if (Object.prototype.hasOwnProperty.call(obj, key.value)) return undefined; + obj[key.value] = value.value; + i = skipJsonWhitespace(text, value.endIndex); + if (text[i] === ',') { i += 1; continue; } + if (text[i] === '}') return { value: obj, endIndex: i + 1 }; + return undefined; + } +} - const parsed = parseJsonStringLiteral(raw, i); - if (!parsed) return undefined; - i = skipJsonWhitespace(raw, parsed.endIndex); +function parseJsonArray(text, index, depth) { + let i = skipJsonWhitespace(text, index + 1); + const arr = []; + if (text[i] === ']') return { value: arr, endIndex: i + 1 }; + + for (;;) { + i = skipJsonWhitespace(text, i); + const value = parseJsonValue(text, i, depth + 1); + if (!value) return undefined; + arr.push(value.value); + i = skipJsonWhitespace(text, value.endIndex); + if (text[i] === ',') { i += 1; continue; } + if (text[i] === ']') return { value: arr, endIndex: i + 1 }; + return undefined; + } +} - if (raw[i] !== '}') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +function strictParseJson(text) { + const start = skipJsonWhitespace(text, 0); + const result = parseJsonValue(text, start, 0); + if (!result) return undefined; + const end = skipJsonWhitespace(text, result.endIndex); + if (end !== text.length) return undefined; + return { value: result.value }; +} - if (i !== raw.length) return undefined; +// ── Request/result validation and canonical envelopes ─────────────────────── - return parsed.value; -} +function validateSealedProbeRequest(raw) { + if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { + return { valid: false, errors: ['request must be a JSON object'] }; + } + + const errors = []; + const { privateRepo, schema: schemaRaw, script } = raw; + const allowedKeys = new Set(['privateRepo', 'schema', 'script']); + for (const key of Object.keys(raw)) { + if (!allowedKeys.has(key)) errors.push(`request.${key} is not supported`); + } + + if (typeof privateRepo !== 'string' || privateRepo.length === 0) { + errors.push('privateRepo must be a non-empty string'); + } else if (privateRepo.length > MAX_PRIVATE_REPO_LENGTH || !SEALED_PROBE_REPO_PATTERN.test(privateRepo)) { + errors.push( + 'privateRepo must be an "owner/repo" slug (no scheme, host, path traversal, query, fragment, or wildcard)', + ); + } + + const schemaValidation = validateSchema(schemaRaw); + if (!schemaValidation.valid) { + errors.push(...schemaValidation.errors.map((error) => `schema: ${error}`)); + } -function parseSealedProbeResult(raw, outcomes) { - if (utf8ByteLength(raw) > MAX_RESULT_BYTES) { - return { result: RESERVED_ERROR_OUTCOME }; + if (typeof script !== 'string' || script.length === 0) { + errors.push('script must be a non-empty string'); + } else if (utf8ByteLength(script) > MAX_SCRIPT_BYTES) { + errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); + } + + let serialized; + try { + serialized = JSON.stringify(raw); + } catch { + errors.push('request must be JSON-serializable'); } - const value = tryExtractStrictResultValue(raw); + if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { + errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); + } + if ( - value !== undefined - && (value === RESERVED_ERROR_OUTCOME || outcomes.includes(value)) + errors.length > 0 + || !schemaValidation.valid + || typeof privateRepo !== 'string' + || typeof script !== 'string' ) { - return { result: value }; + return { valid: false, errors }; } - return { result: RESERVED_ERROR_OUTCOME }; + + return { valid: true, request: { privateRepo, schema: schemaValidation.schema, script } }; +} + +const CANONICAL_ERROR_JSON = '{"status":"error"}'; + +function canonicalOkJson(canonicalResultJson) { + return `{"status":"ok","result":${canonicalResultJson}}`; } -function parseSealedProbeResultJson(raw, outcomes) { - return canonicalizeSealedProbeResult(parseSealedProbeResult(raw, outcomes).result); +function parseAndValidateProbeOutput(raw, schema) { + if (utf8ByteLength(raw) > MAX_RESULT_BYTES) return { ok: false }; + const parsed = strictParseJson(raw); + if (!parsed) return { ok: false }; + if (!validateValueAgainstSchema(schema, parsed.value)) return { ok: false }; + return { ok: true, canonical: canonicalizeSchemaValue(schema, parsed.value) }; } module.exports = { - OUTCOME_COUNT, - RESERVED_ERROR_OUTCOME, - MAX_OUTCOME_BYTES, + PROBE_PROTOCOL_VERSION, + MAX_SCHEMA_BYTES, + MAX_SCHEMA_DEPTH, + MAX_SCHEMA_NODES, + MAX_ENUM_VALUES, + MAX_LITERAL_STRING_BYTES, + MAX_OBJECT_FIELDS, + MAX_TUPLE_ITEMS, + MAX_ARRAY_LENGTH, + MAX_UNION_VARIANTS, MAX_SCRIPT_BYTES, MAX_REQUEST_BYTES, MAX_RESULT_BYTES, MAX_PRIVATE_REPO_LENGTH, + TIMING_BUCKETS_MS, + TIMING_BUCKET_BITS, + RESULT_STATUS_BIT_COST, SEALED_PROBE_REPO_PATTERN, - CANONICAL_ERROR_RESULT_JSON, - buildSealedProbeResultSchema, - validateOutcome, - validateOutcomes, + CANONICAL_ERROR_JSON, + validateSchema, + ceilLog2BigInt, + schemaCardinality, + queryBitsForSchema, + validateValueAgainstSchema, + canonicalizeSchemaValue, + strictParseJson, validateSealedProbeRequest, - canonicalizeSealedProbeResult, - parseSealedProbeResult, - parseSealedProbeResultJson, + canonicalOkJson, + parseAndValidateProbeOutput, }; diff --git a/containers/sealed-probe/broker/scheduler.js b/containers/sealed-probe/broker/scheduler.js new file mode 100644 index 000000000..809978d07 --- /dev/null +++ b/containers/sealed-probe/broker/scheduler.js @@ -0,0 +1,76 @@ +'use strict'; + +const { TIMING_BUCKETS_MS } = require('./protocol'); + +/** + * Response-timing bucketing. + * + * A probe's actual completion latency is itself a secret-dependent signal + * (a script that raises early on one branch and runs to completion on + * another leaks information purely through wall-clock time, with no + * dependence on the declared response schema at all). This module makes + * every *launched* invocation's observable response time fall on one of a + * small, fixed set of boundaries (`TIMING_BUCKETS_MS`), regardless of how + * long the actual work took within that bucket. + * + * Design notes (see `docs/awf-config-spec.md` §14 for the full writeup): + * + * - Time is measured with a monotonic clock (`process.hrtime.bigint()` by + * default, injectable for tests), never `Date.now()`, so system clock + * adjustments cannot shift a response across a bucket boundary. + * - `waitForBucket` resolves the bucket only after probe execution, result + * validation, Docker removal, and host workspace teardown complete. + * Repository size and tree shape can affect cleanup latency, so cleanup + * must be included before choosing the charged timing bucket. Invocations + * remain serialized, preventing queued requests from observing an + * unaccounted cleanup delay from the preceding invocation. + * - If processing latency already exceeds the *last* bucket boundary + * (only possible if infrastructure overhead — not the script itself, + * which is bounded by `sealedProbes.timeout <= 600s`, see preflight.ts — + * pushes total processing past 10 minutes), the broker fails closed: it + * treats the invocation as a canonical error and responds immediately + * rather than waiting indefinitely for a nonexistent next boundary. This + * is a deliberately safe fail-closed fallback for a pathological + * infrastructure-latency edge case, not a normal code path. + */ + +/** Resolves the smallest configured bucket at or after `elapsedMs`. */ +function resolveTimingBucket(elapsedMs) { + for (const bucketMs of TIMING_BUCKETS_MS) { + if (elapsedMs <= bucketMs) return { bucketMs, overflowed: false }; + } + return { bucketMs: TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1], overflowed: true }; +} + +/** Real monotonic clock. Milliseconds, sub-millisecond precision preserved as a float. */ +function createRealClock() { + return { + nowMs: () => Number(process.hrtime.bigint()) / 1e6, + sleep: (ms) => new Promise((resolve) => setTimeout(resolve, ms)), + }; +} + +/** + * Waits (if necessary) until `startMs + bucket` on `clock`, where `bucket` + * is the smallest configured boundary at or after `elapsedMs`. + * + * `elapsedMs` is measured by the caller as `clock.nowMs() - startMs` at the + * moment all processing (including cleanup) completed. This function only + * performs the remaining wait to the selected fixed boundary. + * + * @returns `{ bucketMs, overflowed }`. When `overflowed` is `true`, the + * caller must fail closed (canonical error) rather than waiting further. + */ +async function waitForBucket(startMs, elapsedMs, clock) { + const { bucketMs, overflowed } = resolveTimingBucket(elapsedMs); + if (overflowed) return { bucketMs, overflowed }; + + const targetMs = startMs + bucketMs; + const remainingMs = targetMs - clock.nowMs(); + if (remainingMs > 0) { + await clock.sleep(remainingMs); + } + return { bucketMs, overflowed }; +} + +module.exports = { TIMING_BUCKETS_MS, resolveTimingBucket, createRealClock, waitForBucket }; diff --git a/containers/sealed-probe/broker/sensitivity.js b/containers/sealed-probe/broker/sensitivity.js new file mode 100644 index 000000000..4fbb203be --- /dev/null +++ b/containers/sealed-probe/broker/sensitivity.js @@ -0,0 +1,27 @@ +'use strict'; + +/** + * Repository sensitivity categories and their fixed per-run information + * budgets — broker-side mirror of `SEALED_PROBE_SENSITIVITY_RUN_BITS` in + * `src/types/sealed-probe-options.ts`. Kept in a tiny standalone module (not + * `protocol.js`) because it is config/ledger data, not wire protocol. + * + * `null` means "unmetered": `public` still runs through the same finite + * schema/result validation and operational limits (`maxInvocations`, + * timeouts, sandboxing) as every other category, but its responses are not + * debited against a confidentiality ledger. `sealed` is `0`, which — + * because every accepted query's minimum charge is 4 bits (1 status bit + + * 3 timing bits) — always exceeds the remaining balance, so a `sealed` + * repository can never fund a single query and therefore never copies a + * seed or launches Python. + */ +const SEALED_PROBE_SENSITIVITIES = ['public', 'internal', 'confidential', 'sealed']; + +const SEALED_PROBE_SENSITIVITY_RUN_BITS = { + public: null, + internal: 64, + confidential: 8, + sealed: 0, +}; + +module.exports = { SEALED_PROBE_SENSITIVITIES, SEALED_PROBE_SENSITIVITY_RUN_BITS }; diff --git a/containers/sealed-probe/broker/server.js b/containers/sealed-probe/broker/server.js index 04a80bf92..e7f7ebdc7 100644 --- a/containers/sealed-probe/broker/server.js +++ b/containers/sealed-probe/broker/server.js @@ -6,7 +6,7 @@ const { createAuditLog } = require('./audit'); const { createBroker } = require('./broker'); const { loadConfig, loadSeedMap } = require('./config'); const { buildRequestFromFrame, readBoundedBody } = require('./framing'); -const { CANONICAL_ERROR_RESULT_JSON } = require('./protocol'); +const { CANONICAL_ERROR_JSON } = require('./protocol'); const { assertProbeImageAvailable } = require('./probe-runner'); /** @@ -22,12 +22,15 @@ const { assertProbeImageAvailable } = require('./probe-runner'); * The agent-visible socket has no `/health` route. The compose healthcheck * instead polls for a broker-internal ready file written by `main()` after * the socket starts accepting connections. This removes a distinguishable - * fifth response (the health status body) from the agent-observable surface. + * extra response (the health status body) from the agent-observable surface. * - * `/probe` always answers `200` with a canonical result body. Status codes, - * headers, and bodies are identical for success and for every failure class, - * so the response carries exactly one of the four permitted symbols and - * nothing else. + * `/probe` always answers `200` with a canonical result body: `{"status": + * "ok","result":}` or `{"status":"error"}` — status code and headers + * are identical either way, and every failure class collapses to the same + * error body. For any invocation that reached workspace creation, the + * response is additionally held until a fixed timing-bucket boundary (see + * `./scheduler`) before being sent, so response latency does not leak + * unbucketed secret-dependent signal either. */ const RESULT_HEADERS = { @@ -47,7 +50,7 @@ function createServer(deps) { if (req.method !== 'POST' || req.url !== '/probe') { // Not part of the API. Answer with the canonical error rather than a // distinguishable 404/405 so probing the surface yields no extra signal. - sendResult(res, CANONICAL_ERROR_RESULT_JSON); + sendResult(res, CANONICAL_ERROR_JSON); req.resume(); return; } @@ -56,22 +59,22 @@ function createServer(deps) { .then((body) => { if (body.error !== undefined) { audit.failure('framing', 'body-rejected', body.error); - sendResult(res, CANONICAL_ERROR_RESULT_JSON); + sendResult(res, CANONICAL_ERROR_JSON); return undefined; } const framed = buildRequestFromFrame(req.headers, req.rawHeaders, body.script); if (framed.error !== undefined) { audit.failure('framing', 'frame-rejected', framed.error); - sendResult(res, CANONICAL_ERROR_RESULT_JSON); + sendResult(res, CANONICAL_ERROR_JSON); return undefined; } - return broker.handle(framed.request).then((result) => sendResult(res, result)); + return broker.handle(framed.request, (result) => sendResult(res, result)); }) .catch((error) => { audit.failure('server', 'unhandled-error', error && error.message); - if (!res.headersSent) sendResult(res, CANONICAL_ERROR_RESULT_JSON); + if (!res.headersSent) sendResult(res, CANONICAL_ERROR_JSON); }); }); } diff --git a/docs/awf-config-spec.md b/docs/awf-config-spec.md index 019e429fe..f0adecc39 100644 --- a/docs/awf-config-spec.md +++ b/docs/awf-config-spec.md @@ -1541,15 +1541,33 @@ Each record follows the `blocked-request-diag/v` schema: ### 14.1 Purpose A *sealed probe* lets an agent ask a trusted broker to run a short, -agent-authored Python 3 script against a private repository and report back -one of a small, closed set of outcomes — without the agent ever gaining -network or filesystem access to that repository. - -The agent observes exactly one of four symbols per invocation: one of the -three outcomes it declared, or the reserved value `"ERROR"`. That bounds the -content a single invocation can convey to at most 2 bits. -`sealedProbes.maxInvocations` bounds the cumulative disclosure for the run. -Completion timing remains an acknowledged residual side channel (§14.9). +agent-authored Python 3 script against a private repository and get back a +value conforming to a finite response schema the agent declares up front — +without the agent ever gaining network or filesystem access to that +repository. + +Every private repository configured for sealed probes carries one of four +fixed **sensitivity categories**, each with an immutable maximum number of +bits the broker may reveal about that repository across an entire AWF run +(not per query): + +| Sensitivity | Run budget | Notes | +|-------------|-----------:|-------| +| `public` | unmetered | Still schema/operationally bounded, but responses are never debited against a ledger. | +| `internal` | 64 bits/run | Default for legacy bare-string entries (§14.2). | +| `confidential` | 8 bits/run | | +| `sealed` | 0 bits/run | Can never fund even the cheapest possible query — never copies a seed or launches Python. | + +There is **no per-query cap**. Every invocation may declare an arbitrarily +different response schema; the broker computes that invocation's maximum +complete-transcript information charge (§14.3) and debits it from the +repository's shared run balance *before* copying a seed or launching Python. +An invocation is allowed iff its charge fits the remaining balance — a cheap +boolean question and an expensive high-cardinality question both draw from +the same budget, just at different rates. Charges are never refunded, +regardless of outcome (success, failure, or timeout). +`sealedProbes.maxInvocations` is a separate, independent operational limit +(§14.2) unrelated to the bit ledger. ### 14.2 Configuration @@ -1559,7 +1577,10 @@ The root object MAY contain a `sealedProbes` section: { "sealedProbes": { "enabled": true, - "privateRepos": ["my-org/my-private-repo"], + "privateRepos": [ + { "repo": "my-org/my-private-repo", "sensitivity": "internal" }, + { "repo": "my-org/public-docs", "sensitivity": "public" } + ], "runtime": "docker", "timeout": 30, "memoryLimit": "512m", @@ -1572,23 +1593,32 @@ The root object MAY contain a `sealedProbes` section: | Field | Type | Constraints | Default | |-------|------|-------------|---------| | `enabled` | boolean | — | `false` | -| `privateRepos` | string[] | Non-empty and unique when `enabled` is `true`. Each entry MUST be a bare `owner/repo` slug — no scheme/host (`://`), path traversal (`..`), query string (`?`), fragment (`#`), wildcard (`*`), or extra path segments. | `[]` | -| `runtime` | string | One of `"docker"`, `"gvisor"`, `"sbx"` | `"docker"` | -| `timeout` | integer | `1`–`3600` seconds | `30` | +| `privateRepos` | array | Non-empty and unique (by repo slug, case-insensitively) when `enabled` is `true`. Each entry is either an object `{ "repo": "owner/repo", "sensitivity": "public" \| "internal" \| "confidential" \| "sealed" }`, or (one-release legacy compatibility) a bare `owner/repo` string, normalized to `{ repo, sensitivity: "internal" }` with a warning. Each `repo` MUST be a bare `owner/repo` slug — no scheme/host (`://`), path traversal (`..`), query string (`?`), fragment (`#`), wildcard (`*`), or extra path segments. | `[]` | +| `runtime` | string | One of `"docker"`, `"gvisor"` | `"docker"` | +| `timeout` | integer | `1`–`600` seconds (bounded by the largest response-timing bucket, §14.3) | `30` | | `memoryLimit` | string | Docker-style memory limit, e.g. `"512m"`, `"1g"` | `"512m"` | | `interpreter` | string | Only `"python3"` is currently supported | `"python3"` | -| `maxInvocations` | integer | `1`–`10000` | `32` | +| `maxInvocations` | integer | `1`–`10000`; an independent operational cap, unrelated to the per-repository bit ledger | `32` | Property-level constraints are defined normatively by the `sealedProbes` subschema in `docs/awf-config.schema.json`. +**Legacy `privateRepos` string entries.** A bare `owner/repo` string is +accepted for one release for backward compatibility and is normalized to +`{ repo, sensitivity: "internal" }`, emitting a warning +(`sealedProbes.privateRepos entry "..." is a legacy bare string...`) through +the same warning channel other config normalization uses. New configuration +SHOULD use the explicit object form so the intended sensitivity is never +left implicit. + **Mapping:** every `sealedProbes.*` field is *(config-only; no CLI equivalent)*. There is no `--sealed-probes-*` CLI flag family. The config-file value is passed through `config-mapper.ts` and normalized (defaults applied -via `src/types/sealed-probe-options.ts`'s `SEALED_PROBE_DEFAULTS`, in -`src/parsers/sealed-probe-parser.ts`) into `WrapperConfig.sealedProbes`. Only -an explicit `enabled: true` normalizes to an enabled config; omission or any -other value normalizes to `enabled: false`. +via `src/types/sealed-probe-options.ts`'s `SEALED_PROBE_DEFAULTS`, legacy +string entries normalized in `src/parsers/sealed-probe-parser.ts`) into +`WrapperConfig.sealedProbes`. Only an explicit `enabled: true` normalizes to +an enabled config; omission or any other value normalizes to +`enabled: false`. When `enabled` is `false` or the section is absent, AWF stages nothing, starts no broker, mounts no socket, sets no environment variable, installs no CLI, @@ -1597,22 +1627,29 @@ section. **Preflight (fail-closed).** With `enabled: true`, AWF aborts before the primary agent starts when: `privateRepos` is empty or contains an unsafe or -duplicated slug; `runtime` is `"sbx"` (AWF has no no-network per-invocation -sealed-probe launcher for it and never downgrades — see §14.8); `runtime` is -`"gvisor"` and the `runsc` OCI runtime is not registered with the Docker -daemon; `container.containerRuntime` is a microVM backend, which cannot -receive the broker socket; the resolved Docker host is not a `unix://` socket, -which a `network_mode: none` broker cannot reach; the interpreter or a limit is -unsupported; no staging credential is present in `GH_TOKEN`/`GITHUB_TOKEN`; or -any seed cannot be materialized and verified. - -### 14.3 Request/Result Protocol +duplicated slug; `runtime` is `"gvisor"` and the `runsc` OCI runtime is not +registered with the Docker daemon; `container.containerRuntime` is a +microVM backend, which cannot receive the broker socket; the resolved Docker +host is not a `unix://` socket, which a `network_mode: none` broker cannot +reach; the interpreter or a limit is unsupported; `timeout` exceeds 600 +seconds — the largest response-timing bucket (§14.3) — because a longer +timeout could let an invocation's completion time itself leak unbucketed +secret-dependent information; no staging credential is present in +`GH_TOKEN`/`GITHUB_TOKEN`; or any seed cannot be materialized and verified. + +The seed map the broker reads carries each repository's trusted +`sensitivity` alongside its opaque seed id — the map is built entirely from +AWF configuration, so a request can never choose or override its own +repository's sensitivity or run budget. + +### 14.3 Request/Result Protocol (v2) `src/sealed-probe/protocol.ts` defines the wire protocol. The broker restates it in `containers/sealed-probe/broker/protocol.js` because it runs from its own container image and cannot import AWF's TypeScript sources; the two -implementations are pinned together by `src/sealed-probe/protocol-parity.test.ts`, -which runs one shared vector table through both. +implementations are pinned together by +`src/sealed-probe/protocol-parity.test.ts`, which runs one large shared +vector table (schemas, values, requests, and probe results) through both. **Request.** A sealed-probe request is a JSON object with exactly three fields: @@ -1620,56 +1657,177 @@ fields: ```json { "privateRepo": "my-org/my-private-repo", - "outcomes": ["found", "not_found", "rate_limited"], + "schema": { "type": "boolean" }, "script": "" } ``` - `privateRepo` MUST match the same `owner/repo` slug rule as `sealedProbes.privateRepos` entries (§14.2). -- `outcomes` MUST be an array of exactly three unique ASCII identifiers. Each - identifier MUST start with a letter, contain only letters, digits, `_`, or - `-`, be at most 64 bytes, and MUST NOT equal the reserved value `"ERROR"` - (§14.4). +- `schema` MUST be a valid document in the finite schema DSL below. - `script` MUST be non-empty and at most 64 KiB (`MAX_SCRIPT_BYTES`). The - overall serialized request MUST be at most 256 KiB (`MAX_REQUEST_BYTES`), - as a defense-in-depth cap independent of the per-field limits. - -**Result.** A probe reports its outcome as the closed JSON object -`{"result": ""}` — no other keys are -permitted. - -### 14.4 The Reserved `"ERROR"` Outcome + overall serialized request MUST be at most `MAX_SCRIPT_BYTES + + MAX_SCHEMA_BYTES + 1024` bytes (`MAX_REQUEST_BYTES`), as a defense-in-depth + cap independent of the per-field limits. + +**Result.** A successful probe result is the canonical envelope +`{"status":"ok","result":}`, where `` conforms exactly to the +request's declared `schema`. Every failure mode — invalid request, +disallowed repository, exhausted bit budget, launch failure, timeout, crash, +non-conformant probe output, or internal error — collapses to the single +canonical `{"status":"error"}`, indistinguishable from one another by +design. + +#### The finite schema DSL + +The response schema is a deliberately finite, agent-authored algebra — **not** +general JSON Schema. Every invocation may use a different schema. Supported +node types: + +| type | shape | notes | +|------|-------|-------| +| `const` | `{"type":"const","value":}` | exactly one fixed value | +| `boolean` | `{"type":"boolean"}` | `true` or `false` | +| `enum` | `{"type":"enum","values":[,...]}` | unique literals, all the same JSON type | +| `integer` | `{"type":"integer","minimum":N,"maximum":M}` | inclusive bounded range, safe-integer bounds only | +| `object` | `{"type":"object","fields":{"name":,...}}` | every declared field required; no extra properties | +| `tuple` | `{"type":"tuple","items":[,...]}` | fixed-length, independently-typed positions | +| `array` | `{"type":"array","items":,"length":N}` | fixed length, single uniform item schema | +| `union` | `{"type":"union","variants":{"tag":,...}}` | value is `{"tag":"","value":<...>}`; variants are disjoint by tag | + +A literal (in `const`/`enum`) is a JSON string (at most `MAX_LITERAL_STRING_BYTES` += 64 bytes UTF-8, no control characters), a safe integer, a boolean, or +`null`. There is no way to express an unbounded string, a float, a regex, +recursion, `$ref`, an optional field, `additionalProperties`, or an +untagged/overlapping union — these are structurally impossible to write, not +merely disallowed by a validator. + +This is a deliberately safe *subset* of what a general schema language could +express, chosen so every schema has a computable, bounded cardinality and a +linear-time validator with no backtracking. If a future requirement needs a +richer construct, it must justify a new bounded primitive rather than +weakening these bounds. Every schema is additionally bounded structurally: + +| Bound | Constant | Value | +|-------|----------|------:| +| Max nesting depth | `MAX_SCHEMA_DEPTH` | 6 | +| Max total schema nodes | `MAX_SCHEMA_NODES` | 64 | +| Max serialized schema size | `MAX_SCHEMA_BYTES` | 4096 bytes | +| Max `enum` values | `MAX_ENUM_VALUES` | 4096 | +| Max `object` fields | `MAX_OBJECT_FIELDS` | 16 | +| Max `tuple` items | `MAX_TUPLE_ITEMS` | 16 | +| Max `array` length | `MAX_ARRAY_LENGTH` | 64 | +| Max `union` variants | `MAX_UNION_VARIANTS` | 16 | +| Max literal string length | `MAX_LITERAL_STRING_BYTES` | 64 bytes | + +In practice, `MAX_SCHEMA_BYTES` is often the binding constraint for wide +`enum`/`object`/`tuple` schemas well before the corresponding count bound is +reached (e.g. a numeric `enum` of exactly `MAX_ENUM_VALUES` values already +exceeds `MAX_SCHEMA_BYTES` once serialized). + +#### Budget: cardinality and bit charge + +"Schema cardinality" is the number of distinguishable values a schema +admits — 2 for `boolean`, `N` for an `N`-member `enum`, the product of field +cardinalities for `object`/`tuple`/`array`, the sum of variant cardinalities +for `union`, and 1 for `const`. Cardinality is computed with unbounded +(`BigInt`) arithmetic so no schema can overflow it into an incorrect small +number. + +Every accepted invocation's information charge is: + +```text +charge = RESULT_STATUS_BIT_COST (1 — ok/error is itself observable) + + ceil(log2(schema cardinality)) (the declared response schema) + + TIMING_BUCKET_BITS (3 — six timing buckets, §14.3.1) +``` -`"ERROR"` is reserved by the protocol and MUST NOT appear in a request's -`outcomes` array. It is the sentinel value produced by the result parser -itself (never by a well-behaved script) whenever a result cannot be trusted. +`RESULT_STATUS_BIT_COST` is `1`; `TIMING_BUCKET_BITS` is +`ceil(log2(TIMING_BUCKETS_MS.length))` = `ceil(log2(6))` = `3`. The cheapest +possible schema (`const`, cardinality 1) still charges `1 + 0 + 3 = 4` bits — +this is the practical floor a repository's remaining balance is checked +against to decide whether it can fund *any* further invocation at all. + +The charge is computed and the ledger is debited **before** a seed is +copied or Python is launched (§14.2, §14.7); it is never refunded regardless +of the invocation's outcome, because the broker committed to revealing up to +that many bits of signal the moment it decided to run. + +#### 14.3.1 Response-timing buckets + +A probe's raw completion latency is itself a secret-dependent signal — a +script that raises early on one code path and runs to completion on another +leaks information purely through wall-clock time, independent of the +declared schema. The broker makes every *launched* invocation's observable +response time land on one of six fixed boundaries, using a monotonic clock +(`process.hrtime.bigint()`, never `Date.now()`, so system clock adjustments +cannot shift a response across a boundary): + +```text +TIMING_BUCKETS_MS = [10ms, 100ms, 1s, 10s, 60s, 600s] +``` -### 14.5 Strict, Non-Schema Result Parsing +The broker returns at the first bucket boundary at or after the invocation's +processing (execution + output validation + container removal + workspace +teardown) actually completes. This is +included in the information budget as `TIMING_BUCKET_BITS` (3 bits — for six +buckets) whether or not the script's own answer would otherwise convey any +signal, because latency alone is observable and must be paid for like any +other channel. + +**Cleanup is included in the bucketed measurement.** Repository size and +tree shape can affect container and workspace teardown, so the broker +completes cleanup before measuring elapsed time and selecting the response +bucket. Invocations are serialized; consequently a queued request cannot +observe a preceding invocation's unaccounted cleanup duration. Cleanup +failure maps to canonical error and is recorded only in the protected audit +log. + +**Fail-closed timing overflow.** `sealedProbes.timeout` is capped at 600 +seconds (the largest bucket) at preflight for exactly this reason: the +script itself can never make processing exceed 600 seconds. If +infrastructure overhead (not the script) ever pushed total processing past +600 seconds, the broker treats the invocation as the canonical error and +responds immediately, discarding even an otherwise-valid successful result, +rather than waiting for a nonexistent next boundary or leaking an +unbucketed excess duration. This is a deliberate, tested (`broker.test.ts`) +safe fallback for a pathological latency edge case, not a normal code path. + +### 14.4 Canonical Failure Closure + +Every failure mode — an invalid request, a disallowed repository, an +exhausted bit budget, an exhausted `maxInvocations` count, a launch failure, +a timeout, a script crash, non-conformant probe output, a timing-bucket +overflow, or an internal broker error — collapses to the single canonical +`{"status":"error"}`. Failures are indistinguishable from each other by +design: the agent cannot infer which failure mode occurred from the +response alone. + +### 14.5 Strict, Non-Schema Result Parsing and Post-Execution Validation Result parsing intentionally does **not** execute a general-purpose JSON -Schema validator against the (potentially attacker-influenced) result text. -Because a valid result has exactly one fixed shape — a single `"result"` key -whose value is one of at most three known strings — `parseSealedProbeResult` -enforces that shape with a small, linear-time, non-backtracking hand-written -grammar instead. This rejects, and canonicalizes to `{"result":"ERROR"}` -rather than throwing: +Schema validator against the (potentially attacker-influenced) raw probe +output text. `strictParseJson` enforces well-formedness with a small, +linear-time, non-backtracking hand-written grammar — rejecting, rather than +throwing, on: - malformed JSON of any kind; -- duplicate `"result"` keys (the grammar only accepts a single key/value - pair, so a second key is trailing data); -- any extra fields; -- any leading or trailing content outside the single JSON object; and -- a value that is not a string, or is a string outside the declared - `outcomes` enum (including the literal string `"ERROR"`, which is never a - legitimately-declared outcome per §14.4). - -`buildSealedProbeResultSchema()` constructs a plain data representation of -the closed schema (`{type: "object", additionalProperties: false, required: -["result"], properties: {result: {type: "string", enum: [...outcomes]}}}`). -It is the broker-constructed schema — callers can never supply one — and is -used for documentation and introspection; enforcement is performed by the -hand-written parser above, never by a schema engine. +- duplicate object keys; +- any leading or trailing content outside the single JSON value; and +- invalid UTF-8 or invalid JSON string escapes. + +The parsed value is then validated against the **exact** schema the request +declared (`validateValueAgainstSchema`) — wrong type, out-of-range integer, +an undeclared enum member, extra or missing object fields, the wrong +tuple/array length, or an unrecognized union tag are all rejected. A value +that passes validation is canonically re-serialized +(`canonicalizeSchemaValue`) before being wrapped in the `{"status":"ok",...}` +envelope, so the exact byte layout the probe wrote (whitespace, key order, +duplicate-safe encoding) never reaches the agent — only a canonical +re-encoding of the validated value does. + +Raw probe bytes, stdout, stderr, and exit status never reach the agent under +any circumstance, success or failure. ### 14.6 Offline Staging @@ -1694,8 +1852,16 @@ runs a trusted host-side staging phase (`src/sealed-probe/staging.ts`): 7. deletes the askpass helper and the isolated staging `HOME`, so no staging artifact survives into the broker/agent phase. +The generated seed map (`{ repo, seedId, sensitivity }` per entry) carries +each repository's trusted `sensitivity` alongside its opaque seed id; this +map is the broker's *only* source of sensitivity information — a request +field can never supply or override it. + Staging failure aborts the run. There is no fallback clone or fetch anywhere -else in the system: neither the broker nor a probe has a network path. +else in the system: neither the broker nor a probe has a network path. A +`sealed` (0-bit) repository is still staged like any other (so its +configuration is validated the same way), but its run budget structurally +guarantees the broker never copies that seed or launches Python for it. ### 14.7 Trusted Broker and Probe Sandbox @@ -1708,9 +1874,12 @@ resolved Docker socket so it can launch probes; that path is never placed in the agent's environment or volumes. The broker maps a normalized `owner/repo` id through the AWF-generated seed -map to an opaque seed directory. Callers never supply a path, URL, ref, mount, -image, command, environment, runtime, or limit. For each valid request the -broker creates a fresh, full, private writable copy of exactly one seed and +map to an opaque seed directory and its trusted sensitivity. Callers never +supply a path, URL, ref, mount, image, command, environment, runtime, limit, +or sensitivity. For each request that passes schema validation and clears +its repository's remaining bit ledger (in that order — an invalid schema or +an unaffordable charge is rejected before any seed is touched), the broker +creates a fresh, full, private writable copy of exactly one seed and launches one probe container with a fixed argument vector: - `--network none`, `--read-only`, `--user 65534:65534`, `--cap-drop ALL`, @@ -1724,14 +1893,21 @@ launches one probe container with a fixed argument vector: - no Docker socket, no seed parent, no other repository, no workspace, no credentials, and no prior invocation's data. -The copy is destroyed after the result is validated, so repository mutations -are ephemeral and are never returned or persisted. The result file is opened -with `O_NOFOLLOW` and must be a regular file within the size cap, so replacing -`/probe/out` with a symlink, FIFO, device, or socket cannot make the broker -read anything else. - -Probe stdout/stderr is capped and discarded. Failure reasons are written only -to `/sealed-probes/audit/`, which is mounted into the broker alone. +The invocation's workspace is torn down before the fixed timing bucket is +selected (§14.3.1), so cleanup duration remains inside the charged timing +channel. A cleanup failure produces canonical error and is recorded in the +protected audit log (`reason: 'cleanup-failed'`). Repository mutations are +ephemeral and are never returned or persisted. The result file is opened +with `O_NOFOLLOW` and must +be a regular file within the size cap, so replacing `/probe/out` with a +symlink, FIFO, device, or socket cannot make the broker read anything else. + +Probe stdout/stderr is capped and discarded — never parsed, never returned, +never logged in a form reachable by the agent. Failure reasons (with +protected detail, e.g. `repo-not-allowed`, `bit-budget-exhausted`, +`invalid-request`, `probe-launch-failed`, `timing-bucket-overflow`, +`cleanup-failed`) are written only to `/sealed-probes/audit/`, +which is mounted into the broker alone. ### 14.8 Agent Interface @@ -1739,20 +1915,31 @@ When sealed probes are enabled, the agent receives exactly two bind mounts — the broker socket directory (read-write) and a generated skill directory (read-only) — plus three environment variables (`AWF_SEALED_PROBE_SOCKET`, `AWF_SEALED_PROBE_SKILL`, -`AWF_SEALED_PROBE_REPOS`). GitHub tokens are removed from the agent -environment whenever sealed probes are enabled, independently of the API and -DIFC proxies. +`AWF_SEALED_PROBE_REPOS`, the last a comma-separated list of configured repo +slugs only — never sensitivities or budgets). GitHub tokens are removed from +the agent environment whenever sealed probes are enabled, independently of +the API and DIFC proxies. `containers/agent/sealed-probe-wrapper.sh` is installed on the agent's `PATH` -as `sealed-probe`. It accepts only `--repo` once, `--outcome` exactly three -times, and the script on stdin; every other option, the `--flag=value` form, -and positional arguments are rejected. It always prints exactly one canonical -JSON line, writes nothing to stderr, and exits `0` — for outcomes and for -every failure, including transport failures, which produce -`{"result":"ERROR"}` locally. +as `sealed-probe` (protocol v2). It accepts only `--repo` once, `--schema` +once (a JSON document, at most `MAX_SCHEMA_BYTES` bytes), and the script on +stdin; every other option, the `--flag=value` form, and positional arguments +are rejected without contacting the broker. It always prints exactly one +canonical JSON line, writes nothing to stderr, and exits `0` — for both +outcomes and for every failure, including transport failures, which produce +`{"status":"error"}` locally. The wrapper cannot itself validate the +schema's structure, cardinality, or bit charge (that is the trusted +broker's job, enforced before it copies a seed or launches Python); its only +responsibilities are enforcing the fixed CLI shape, base64url-encoding the +schema into a request header, transporting the script body unmodified, and +passing the broker's response through unmodified. The generated `SKILL.md` is written under `/sealed-probes/agent/` and -mounted read-only at `/run/awf-sealed-probe-skill/SKILL.md`. AWF deliberately +mounted read-only at `/run/awf-sealed-probe-skill/SKILL.md`. It documents, +per configured repository, its sensitivity and run budget (e.g. `` `octo/alpha` +— 64 bits/run (`internal`) ``), the finite schema DSL, the bit-charge +formula, the timing buckets, and the operational `maxInvocations` limit — +so an agent can design informed, low-cardinality questions. AWF deliberately does **not** mount it into `$HOME/.copilot/skills` or the workspace's `.github/skills`: Docker would create the mount point inside host user state or inside the checked-out workspace. Agents therefore discover it through @@ -1763,12 +1950,30 @@ For the same reason, a microVM primary agent runtime (`sbx`) is rejected at preflight: it does not receive Compose bind mounts, so the socket and skill could not be exposed. Sealed probes are never partially enabled. -### 14.9 Residual Channels and Limits - -- Each invocation reveals one of four symbols — at most 2 bits. -- `maxInvocations` counts every response, including rejections, because each - response is itself one of the four symbols. -- Completion timing is not mitigated in v1 and remains observable. +### 14.9 Protocol v1 Compatibility + +Protocol v1 (three fixed outcomes plus the reserved `"ERROR"` sentinel, no +schema, no sensitivity, no bit ledger) is superseded by v2. There is no +runtime v1/v2 auto-negotiation in the current wrapper or broker — both are +deployed together as part of the same AWF release, and the wrapper always +sends `X-AWF-Probe-Version: 2`. A safe compatibility translation for legacy +v1 three-outcome calls (mapping a fixed three-value `enum` schema to the old +`outcomes` shape) is a natural extension point if a future release needs to +accept both wire versions from mismatched wrapper/broker builds, but is not +implemented today because AWF always deploys the wrapper and broker as a +matched pair. + +### 14.10 Residual Channels and Limits + +- Every launched invocation's disclosure is bounded by its own declared + schema's charge (§14.3) — not a fixed per-invocation cap — debited from its + repository's run budget; `public` repositories are schema/operationally + bounded but not bit-metered. +- `maxInvocations` counts every response, including rejections, as an + independent operational limit unrelated to the bit ledger. +- Response timing is bucketed to one of six fixed boundaries and charged as + part of the budget (§14.3.1); container and workspace cleanup complete + before the bucket is selected (§14.3.1, §14.7). - Per-invocation aggregate disk usage is bounded by the wall-clock timeout and a per-file size limit rather than a hard filesystem quota. - The probe rootfs is the broker image, so it also contains a Node runtime and diff --git a/docs/awf-config.schema.json b/docs/awf-config.schema.json index 618bf6e56..eb3ad6f98 100644 --- a/docs/awf-config.schema.json +++ b/docs/awf-config.schema.json @@ -848,14 +848,42 @@ }, "privateRepos": { "type": "array", - "description": "Private repositories the sealed-probe broker may run probes against. Each entry must be a bare `owner/repo` slug — no scheme, host, credentials, path traversal, query string, fragment, or wildcard. AWF stages one immutable seed per entry before the primary agent starts.", + "description": "Private repositories the sealed-probe broker may run probes against, each with a trusted confidentiality category that fixes its per-run information budget (`public` unmetered, `internal` 64 bits/run, `confidential` 8 bits/run, `sealed` 0 bits/run — never launches a probe). A legacy bare `owner/repo` string is accepted for one release only and normalized to `{ repo, sensitivity: \"internal\" }` with a warning; update it to the object form. Each `repo` must be a bare `owner/repo` slug — no scheme, host, credentials, path traversal, query string, fragment, or wildcard. Repository names must be unique case-insensitively. AWF stages one immutable seed per entry before the primary agent starts.", "items": { - "type": "string", - "maxLength": 140, - "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + "oneOf": [ + { + "type": "string", + "maxLength": 140, + "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "repo", + "sensitivity" + ], + "properties": { + "repo": { + "type": "string", + "maxLength": 140, + "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + }, + "sensitivity": { + "type": "string", + "enum": [ + "public", + "internal", + "confidential", + "sealed" + ], + "description": "Confidentiality category, which fixes this repository's immutable per-run information budget. Cannot be increased by configuration." + } + } + } + ] }, - "minItems": 1, - "uniqueItems": true + "minItems": 1 }, "runtime": { "type": "string", @@ -869,8 +897,8 @@ "timeout": { "type": "integer", "minimum": 1, - "maximum": 3600, - "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical ERROR. Default: 30.", + "maximum": 600, + "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 600 (the largest response-timing bucket) so every invocation's completion always lands inside a bucket. Default: 30.", "default": 30 }, "memoryLimit": { diff --git a/src/awf-config-schema.json b/src/awf-config-schema.json index 618bf6e56..eb3ad6f98 100644 --- a/src/awf-config-schema.json +++ b/src/awf-config-schema.json @@ -848,14 +848,42 @@ }, "privateRepos": { "type": "array", - "description": "Private repositories the sealed-probe broker may run probes against. Each entry must be a bare `owner/repo` slug — no scheme, host, credentials, path traversal, query string, fragment, or wildcard. AWF stages one immutable seed per entry before the primary agent starts.", + "description": "Private repositories the sealed-probe broker may run probes against, each with a trusted confidentiality category that fixes its per-run information budget (`public` unmetered, `internal` 64 bits/run, `confidential` 8 bits/run, `sealed` 0 bits/run — never launches a probe). A legacy bare `owner/repo` string is accepted for one release only and normalized to `{ repo, sensitivity: \"internal\" }` with a warning; update it to the object form. Each `repo` must be a bare `owner/repo` slug — no scheme, host, credentials, path traversal, query string, fragment, or wildcard. Repository names must be unique case-insensitively. AWF stages one immutable seed per entry before the primary agent starts.", "items": { - "type": "string", - "maxLength": 140, - "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + "oneOf": [ + { + "type": "string", + "maxLength": 140, + "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "repo", + "sensitivity" + ], + "properties": { + "repo": { + "type": "string", + "maxLength": 140, + "pattern": "^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/(?!\\.\\.?$)(?!.*\\.\\.)[A-Za-z0-9._-]{1,100}$" + }, + "sensitivity": { + "type": "string", + "enum": [ + "public", + "internal", + "confidential", + "sealed" + ], + "description": "Confidentiality category, which fixes this repository's immutable per-run information budget. Cannot be increased by configuration." + } + } + } + ] }, - "minItems": 1, - "uniqueItems": true + "minItems": 1 }, "runtime": { "type": "string", @@ -869,8 +897,8 @@ "timeout": { "type": "integer", "minimum": 1, - "maximum": 3600, - "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical ERROR. Default: 30.", + "maximum": 600, + "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 600 (the largest response-timing bucket) so every invocation's completion always lands inside a bucket. Default: 30.", "default": 30 }, "memoryLimit": { diff --git a/src/commands/build-config.test.ts b/src/commands/build-config.test.ts index 4e3451cde..556bea183 100644 --- a/src/commands/build-config.test.ts +++ b/src/commands/build-config.test.ts @@ -456,7 +456,7 @@ describe('buildConfig', () => { ...makeInputs().options, sealedProbes: { enabled: true, - privateRepos: ['octo/repo'], + privateRepos: [{ repo: 'octo/repo', sensitivity: 'confidential' }], runtime: 'sbx', timeout: 90, memoryLimit: '2g', @@ -467,7 +467,7 @@ describe('buildConfig', () => { })); expect(config.sealedProbes).toEqual({ enabled: true, - privateRepos: ['octo/repo'], + privateRepos: [{ repo: 'octo/repo', sensitivity: 'confidential' }], runtime: 'sbx', timeout: 90, memoryLimit: '2g', diff --git a/src/config-file-sealed-probes-validation.test.ts b/src/config-file-sealed-probes-validation.test.ts index d67e863b6..f78d2e26d 100644 --- a/src/config-file-sealed-probes-validation.test.ts +++ b/src/config-file-sealed-probes-validation.test.ts @@ -5,11 +5,14 @@ describe('validateAwfFileConfig — sealedProbes', () => { expect(validateAwfFileConfig({ sealedProbes: {} })).toEqual([]); }); - it('accepts a fully-specified valid sealedProbes section', () => { + it('accepts a fully-specified valid sealedProbes section using object-form privateRepos', () => { const errors = validateAwfFileConfig({ sealedProbes: { enabled: true, - privateRepos: ['octo-org/octo-repo', 'octo-org/other.repo'], + privateRepos: [ + { repo: 'octo-org/octo-repo', sensitivity: 'internal' }, + { repo: 'octo-org/other.repo', sensitivity: 'confidential' }, + ], runtime: 'gvisor', timeout: 60, memoryLimit: '1g', @@ -21,6 +24,40 @@ describe('validateAwfFileConfig — sealedProbes', () => { expect(errors).toEqual([]); }); + it('accepts a legacy bare-string privateRepos entry (one-release compatibility)', () => { + expect(validateAwfFileConfig({ sealedProbes: { privateRepos: ['octo-org/octo-repo'] } })).toEqual([]); + }); + + it('accepts a mix of legacy string and object-form privateRepos entries', () => { + const errors = validateAwfFileConfig({ + sealedProbes: { + privateRepos: ['octo/legacy', { repo: 'octo/object-form', sensitivity: 'sealed' }], + }, + }); + expect(errors).toEqual([]); + }); + + it('rejects an object-form privateRepos entry with an invalid sensitivity value', () => { + const errors = validateAwfFileConfig({ + sealedProbes: { privateRepos: [{ repo: 'octo/repo', sensitivity: 'top-secret' }] }, + }); + expect(errors.length).toBeGreaterThan(0); + }); + + it('rejects an object-form privateRepos entry missing sensitivity', () => { + const errors = validateAwfFileConfig({ + sealedProbes: { privateRepos: [{ repo: 'octo/repo' }] }, + }); + expect(errors.length).toBeGreaterThan(0); + }); + + it('rejects an object-form privateRepos entry with unsupported extra properties', () => { + const errors = validateAwfFileConfig({ + sealedProbes: { privateRepos: [{ repo: 'octo/repo', sensitivity: 'internal', extra: true }] }, + }); + expect(errors.length).toBeGreaterThan(0); + }); + it('accepts privateRepos without enabled (not required unless enabled)', () => { expect(validateAwfFileConfig({ sealedProbes: { privateRepos: ['octo/repo'] } })).toEqual([]); }); @@ -30,12 +67,10 @@ describe('validateAwfFileConfig — sealedProbes', () => { expect(validateAwfFileConfig({ sealedProbes: { enabled: true, privateRepos: [] } }).length).toBeGreaterThan(0); }); - it('rejects duplicate privateRepos entries', () => { - const errors = validateAwfFileConfig({ - sealedProbes: { privateRepos: ['octo/repo', 'octo/repo'] }, - }); - expect(errors.length).toBeGreaterThan(0); - }); + // Duplicate-entry rejection depends on comparing normalized repo keys + // across entries (which may mix legacy strings and objects), so it lives + // in `src/sealed-probe/preflight.ts` (see preflight.test.ts) rather than + // in the raw JSON Schema, which validates one array item at a time. it.each([ ['a URL', 'https://github.com/octo/repo'], @@ -79,6 +114,11 @@ describe('validateAwfFileConfig — sealedProbes', () => { expect(validateAwfFileConfig({ sealedProbes: { timeout: 30 } })).toEqual([]); }); + it('accepts the maximum timeout of 600 seconds (the largest timing bucket) and rejects one second above it', () => { + expect(validateAwfFileConfig({ sealedProbes: { timeout: 600 } })).toEqual([]); + expect(validateAwfFileConfig({ sealedProbes: { timeout: 601 } }).length).toBeGreaterThan(0); + }); + it('rejects an invalid memoryLimit format', () => { expect(validateAwfFileConfig({ sealedProbes: { memoryLimit: '512' } }).length).toBeGreaterThan(0); expect(validateAwfFileConfig({ sealedProbes: { memoryLimit: '0m' } }).length).toBeGreaterThan(0); diff --git a/src/config-file.ts b/src/config-file.ts index 228fb172d..aec8512ba 100644 --- a/src/config-file.ts +++ b/src/config-file.ts @@ -171,7 +171,12 @@ export interface AwfFileConfig { */ sealedProbes?: { enabled?: boolean; - privateRepos?: string[]; + /** + * Each entry is either a trusted repository descriptor, or (for one + * release only) a legacy bare `owner/repo` string, normalized to + * `{ repo, sensitivity: 'internal' }` with a warning. + */ + privateRepos?: Array; runtime?: 'docker' | 'gvisor'; timeout?: number; memoryLimit?: string; diff --git a/src/parsers/sealed-probe-parser.test.ts b/src/parsers/sealed-probe-parser.test.ts index 67168e645..b3367a8ad 100644 --- a/src/parsers/sealed-probe-parser.test.ts +++ b/src/parsers/sealed-probe-parser.test.ts @@ -25,22 +25,74 @@ describe('normalizeSealedProbesConfig', () => { }); it('preserves an explicit enabled: true', () => { - expect(normalizeSealedProbesConfig({ enabled: true, privateRepos: ['octo/repo'] })?.enabled).toBe(true); - }); - - it('deduplicates privateRepos', () => { - const config = normalizeSealedProbesConfig({ privateRepos: ['octo/repo', 'octo/repo', 'octo/other'] }); - expect(config?.privateRepos).toEqual(['octo/repo', 'octo/other']); + expect( + normalizeSealedProbesConfig({ enabled: true, privateRepos: [{ repo: 'octo/repo', sensitivity: 'internal' }] }) + ?.enabled, + ).toBe(true); }); it('defaults privateRepos to an empty array when omitted', () => { expect(normalizeSealedProbesConfig({})?.privateRepos).toEqual([]); }); + it('passes through object-form privateRepos entries unchanged, in order, without deduplicating', () => { + const warn = jest.fn(); + const config = normalizeSealedProbesConfig( + { + privateRepos: [ + { repo: 'octo/repo', sensitivity: 'internal' }, + { repo: 'octo/repo', sensitivity: 'internal' }, + { repo: 'octo/other', sensitivity: 'confidential' }, + ], + }, + { warn }, + ); + expect(config?.privateRepos).toEqual([ + { repo: 'octo/repo', sensitivity: 'internal' }, + { repo: 'octo/repo', sensitivity: 'internal' }, + { repo: 'octo/other', sensitivity: 'confidential' }, + ]); + // Duplicate rejection is `src/sealed-probe/preflight.ts`'s job, not the + // normalizer's — the normalizer only fills defaults and migrates legacy + // strings, so it must not silently drop or warn about anything here. + expect(warn).not.toHaveBeenCalled(); + }); + + it('normalizes a legacy bare-string entry to {repo, sensitivity: "internal"} and warns once', () => { + const warn = jest.fn(); + const config = normalizeSealedProbesConfig({ privateRepos: ['octo/legacy'] }, { warn }); + + expect(config?.privateRepos).toEqual([{ repo: 'octo/legacy', sensitivity: 'internal' }]); + expect(warn).toHaveBeenCalledTimes(1); + expect(warn.mock.calls[0][0]).toContain('octo/legacy'); + expect(warn.mock.calls[0][0]).toContain('legacy bare string'); + }); + + it('warns once per legacy string entry, independently, in a mixed list', () => { + const warn = jest.fn(); + const config = normalizeSealedProbesConfig( + { + privateRepos: ['octo/legacy-one', { repo: 'octo/object-form', sensitivity: 'sealed' }, 'octo/legacy-two'], + }, + { warn }, + ); + + expect(config?.privateRepos).toEqual([ + { repo: 'octo/legacy-one', sensitivity: 'internal' }, + { repo: 'octo/object-form', sensitivity: 'sealed' }, + { repo: 'octo/legacy-two', sensitivity: 'internal' }, + ]); + expect(warn).toHaveBeenCalledTimes(2); + }); + + it('defaults to logger.warn when no warn override is supplied (does not throw)', () => { + expect(() => normalizeSealedProbesConfig({ privateRepos: ['octo/legacy'] })).not.toThrow(); + }); + it('preserves explicitly-set fields over defaults', () => { const config = normalizeSealedProbesConfig({ enabled: true, - privateRepos: ['octo/repo'], + privateRepos: [{ repo: 'octo/repo', sensitivity: 'confidential' }], runtime: 'gvisor', timeout: 120, memoryLimit: '2g', @@ -50,7 +102,7 @@ describe('normalizeSealedProbesConfig', () => { expect(config).toEqual({ enabled: true, - privateRepos: ['octo/repo'], + privateRepos: [{ repo: 'octo/repo', sensitivity: 'confidential' }], runtime: 'gvisor', timeout: 120, memoryLimit: '2g', diff --git a/src/parsers/sealed-probe-parser.ts b/src/parsers/sealed-probe-parser.ts index c6aa2f6db..b334a47f4 100644 --- a/src/parsers/sealed-probe-parser.ts +++ b/src/parsers/sealed-probe-parser.ts @@ -1,5 +1,49 @@ import type { AwfFileConfig } from '../config-file'; -import { SEALED_PROBE_DEFAULTS, type SealedProbesConfig } from '../types/sealed-probe-options'; +import { logger } from '../logger'; +import { + SEALED_PROBE_DEFAULTS, + type SealedProbeRepository, + type SealedProbesConfig, +} from '../types/sealed-probe-options'; + +/** Sensitivity legacy bare-string `privateRepos` entries are normalized to. */ +const LEGACY_REPO_DEFAULT_SENSITIVITY = 'internal'; + +type RawPrivateRepoEntry = NonNullable['privateRepos'] extends + | Array + | undefined + ? T + : never; + +/** + * Normalizes one `privateRepos` entry to a {@link SealedProbeRepository}. + * + * A bare string is a one-release compatibility path: it is accepted and + * normalized to `{ repo, sensitivity: 'internal' }`, with a warning, so + * existing configs keep working for one release while they migrate to the + * explicit object form. `warn` is injectable so callers (and tests) can + * observe/suppress it without depending on the process-wide logger. + */ +function normalizePrivateRepoEntry( + entry: RawPrivateRepoEntry, + warn: (message: string) => void, +): SealedProbeRepository { + if (typeof entry === 'string') { + warn( + `sealedProbes.privateRepos entry "${entry}" is a legacy bare string. It is being normalized to ` + + `{ repo: "${entry}", sensitivity: "${LEGACY_REPO_DEFAULT_SENSITIVITY}" } for this release only. ` + + 'Update your AWF configuration to the explicit object form before the next release, when this ' + + 'compatibility path is removed.', + ); + return { repo: entry, sensitivity: LEGACY_REPO_DEFAULT_SENSITIVITY }; + } + return { repo: entry.repo, sensitivity: entry.sensitivity }; +} + +export interface NormalizeSealedProbesConfigOptions { + /** Overrides how legacy-string-entry warnings are emitted. Defaults to `logger.warn`. */ + warn?: (message: string) => void; +} /** * Normalizes the raw `sealedProbes` section of an AWF config file into a @@ -8,9 +52,12 @@ import { SEALED_PROBE_DEFAULTS, type SealedProbesConfig } from '../types/sealed- * * By the time this runs, `raw` has already passed schema validation * (`validateAwfFileConfig` / docs/awf-config.schema.json), so bounds, - * enums, and the "enabled requires non-empty unique privateRepos" rule are - * assumed to already hold. This function only fills in defaults — it does - * not re-validate. + * enums, and the "enabled requires non-empty privateRepos" rule are assumed + * to already hold. This function only fills in defaults and normalizes the + * legacy-string `privateRepos` compatibility path — it does not re-validate + * repository shape, uniqueness, or sensitivity (see + * `src/sealed-probe/preflight.ts` for the fail-closed checks that require + * comparing multiple entries at once). * * Returns `undefined` when `raw` is `undefined`, i.e. the config file did * not include a `sealedProbes` section at all. When the section is present @@ -18,14 +65,20 @@ import { SEALED_PROBE_DEFAULTS, type SealedProbesConfig } from '../types/sealed- */ export function normalizeSealedProbesConfig( raw: AwfFileConfig['sealedProbes'] | undefined, + options: NormalizeSealedProbesConfigOptions = {}, ): SealedProbesConfig | undefined { if (!raw) return undefined; + const warn = options.warn ?? ((message: string) => logger.warn(message)); + const privateRepos: SealedProbeRepository[] = (raw.privateRepos ?? []).map((entry) => + normalizePrivateRepoEntry(entry, warn), + ); + return { // Only an explicit `true` enables sealed probes; anything else (including // omission) normalizes to disabled. enabled: raw.enabled === true, - privateRepos: raw.privateRepos ? [...new Set(raw.privateRepos)] : [], + privateRepos, runtime: raw.runtime ?? SEALED_PROBE_DEFAULTS.runtime, timeout: raw.timeout ?? SEALED_PROBE_DEFAULTS.timeout, memoryLimit: raw.memoryLimit ?? SEALED_PROBE_DEFAULTS.memoryLimit, diff --git a/src/sealed-probe/broker.test.ts b/src/sealed-probe/broker.test.ts index 218b59166..ef7755129 100644 --- a/src/sealed-probe/broker.test.ts +++ b/src/sealed-probe/broker.test.ts @@ -1,26 +1,36 @@ import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; +import { EventEmitter } from 'events'; /** - * Behavioural tests for the trusted broker, exercised through its real - * filesystem workspace code with a mocked Docker runner. + * Behavioural tests for the trusted broker (protocol v2), exercised through + * its real filesystem workspace code with a mocked Docker runner and an + * injectable clock. * - * These stand in for a full end-to-end probe run: they prove the writable-copy - * semantics, the seed's immutability, repository isolation, the invocation - * budget, workspace teardown, and — most importantly — that every failure path - * produces the byte-identical canonical `ERROR` with no extra signal. + * These stand in for a full end-to-end probe run: they prove the + * writable-copy semantics, the seed's immutability, repository isolation, + * the operational invocation budget, the per-repository *bit* ledger (no + * per-query cap — every invocation's schema-derived charge is computed and + * debited before launch), the timing-bucket response discipline (via a fake + * monotonic clock, never real time), workspace teardown, and — most + * importantly — that every failure path produces the byte-identical + * canonical `{"status":"error"}` with no extra signal. */ /* eslint-disable @typescript-eslint/no-require-imports */ const brokerDir = path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker'); const { createBroker } = require(path.join(brokerDir, 'broker.js')); const workspace = require(path.join(brokerDir, 'workspace.js')); const { buildProbeArgs } = require(path.join(brokerDir, 'probe-runner.js')); -const { buildRequestFromFrame } = require(path.join(brokerDir, 'framing.js')); +const { buildRequestFromFrame, readBoundedBody } = require(path.join(brokerDir, 'framing.js')); +const { TIMING_BUCKETS_MS } = require(path.join(brokerDir, 'scheduler.js')); /* eslint-enable @typescript-eslint/no-require-imports */ -const CANONICAL_ERROR = '{"result":"ERROR"}'; -const OUTCOMES = ['YES', 'NO', 'UNKNOWN']; +const CANONICAL_ERROR = '{"status":"error"}'; +// A fixed-shape object schema (one enum-valued field) keeps most vectors +// structurally identical to the old three-outcome protocol while exercising +// the new schema-carrying request and ok/error envelope. +const OUTCOME_SCHEMA = { type: 'object', fields: { result: { type: 'enum', values: ['YES', 'NO', 'UNKNOWN'] } } }; interface AuditRecord { kind: string; @@ -41,10 +51,42 @@ function createAudit(): { records: AuditRecord[]; log: Record value, + sleep: (ms: number) => { + sleeps.push(ms); + value += ms; + return Promise.resolve(); + }, + }, + advance(ms: number): void { + value += ms; + }, + sleeps, + }; +} + +/** Awaits `broker.handle`, capturing the single callback response. */ +async function invoke( + broker: { handle: (request: unknown, respond: (json: string) => void) => Promise }, + request: unknown, +): Promise { + let response = ''; + await broker.handle(request, (json: string) => { + response = json; + }); + return response; +} + describe('sealed-probe broker', () => { let root: string; let config: Record; - let seedMap: Map; + let seedMap: Map; const seedIdA = 'a'.repeat(32); const seedIdB = 'b'.repeat(32); @@ -105,8 +147,8 @@ describe('sealed-probe broker', () => { createSeed(seedIdA, { 'README.md': 'repo A secret\n' }); createSeed(seedIdB, { 'README.md': 'repo B secret\n' }); seedMap = new Map([ - ['octo/alpha', seedIdA], - ['octo/beta', seedIdB], + ['octo/alpha', { seedId: seedIdA, sensitivity: 'internal' }], + ['octo/beta', { seedId: seedIdB, sensitivity: 'confidential' }], ]); }); @@ -117,16 +159,21 @@ describe('sealed-probe broker', () => { function build( runner: { runProbeContainer: (params: never) => Promise }, - workspaceOverride = workspace, + opts: { + workspace?: typeof workspace; + clock?: { nowMs: () => number; sleep: (ms: number) => Promise }; + seeds?: Map; + } = {}, ) { const audit = createAudit(); const broker = createBroker({ config, - seedMap, + seedMap: opts.seeds || seedMap, runId: 'run-1234abcd', audit: audit.log, - workspace: workspaceOverride, + workspace: opts.workspace || workspace, runner, + clock: opts.clock, }); return { broker, audit }; } @@ -150,17 +197,17 @@ describe('sealed-probe broker', () => { const validRequest = (repo = 'octo/alpha') => ({ privateRepo: repo, - outcomes: [...OUTCOMES], + schema: OUTCOME_SCHEMA, script: 'probe', }); - it('returns the canonically re-serialized declared outcome', async () => { + it('returns the canonically re-serialized declared outcome inside the ok envelope', async () => { const runner = probeRunner((invocationDir) => { fs.writeFileSync(path.join(invocationDir, 'out'), ' {"result": "YES"} '); }); const { broker } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe('{"result":"YES"}'); + expect(await invoke(broker, validRequest())).toBe('{"status":"ok","result":{"result":"YES"}}'); }); it('gives the probe a read-only copy of the repo and leaves the seed unchanged', async () => { @@ -173,7 +220,7 @@ describe('sealed-probe broker', () => { }); const { broker } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe('{"result":"NO"}'); + expect(await invoke(broker, validRequest())).toBe('{"status":"ok","result":{"result":"NO"}}'); expect(observed).toBe('repo A secret\n'); // The seed itself is untouched. expect(fs.readFileSync(path.join(seedPath(seedIdA), 'README.md'), 'utf8')).toBe('repo A secret\n'); @@ -186,7 +233,7 @@ describe('sealed-probe broker', () => { }); const { broker } = build(runner); - await broker.handle(validRequest()); + await invoke(broker, validRequest()); expect(fs.readdirSync(String(config.workDir))).toEqual([]); }); @@ -197,19 +244,14 @@ describe('sealed-probe broker', () => { const runner = probeRunner((invocationDir) => { repoContents = fs.readFileSync(path.join(invocationDir, 'repo', 'README.md'), 'utf8'); fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); - // In Docker the probe sees /probe/ containing only repo/ and out (tmpfs). - // On the host the invocation root also holds script.py (mounted at - // probeScriptPath, outside /probe in Docker). Verify no private data leaks. siblings = fs.readdirSync(invocationDir).sort(); }); const { broker } = build(runner); - await broker.handle(validRequest('octo/beta')); + await invoke(broker, validRequest('octo/beta')); expect(repoContents).toBe('repo B secret\n'); - // script.py is the submitted (non-secret) script; out and repo are expected. expect(siblings).toEqual(['out', 'repo', 'script.py']); - // No trace of the other seed. expect(repoContents).not.toContain('repo A'); }); @@ -219,24 +261,23 @@ describe('sealed-probe broker', () => { }); const { broker, audit } = build(runner); - await expect(broker.handle(validRequest('octo/not-configured'))).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest('octo/not-configured'))).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ kind: 'failure', reason: 'repo-not-allowed' }); expect(fs.readdirSync(String(config.workDir))).toEqual([]); }); it.each([ - ['extra launch control', { ...{ privateRepo: 'octo/alpha', outcomes: [...OUTCOMES], script: 'x' }, image: 'evil' }], - ['caller-supplied schema', { privateRepo: 'octo/alpha', outcomes: [...OUTCOMES], script: 'x', schema: {} }], - ['four outcomes', { privateRepo: 'octo/alpha', outcomes: ['A', 'B', 'C', 'D'], script: 'x' }], - ['reserved outcome', { privateRepo: 'octo/alpha', outcomes: ['A', 'B', 'ERROR'], script: 'x' }], - ['path selector', { privateRepo: '../../seeds', outcomes: [...OUTCOMES], script: 'x' }], + ['extra launch control field', { ...validRequest(), image: 'evil' }], + ['a smuggled sensitivity override (requests cannot choose sensitivity)', { ...validRequest(), sensitivity: 'public' }], + ['invalid schema construct', { ...validRequest(), schema: { type: 'nope' } }], + ['path traversal repo selector', { privateRepo: '../../seeds', schema: OUTCOME_SCHEMA, script: 'x' }], ])('rejects %s before copying or launching', async (_name, request) => { const runner = probeRunner(() => { throw new Error('probe must not launch'); }); const { broker, audit } = build(runner); - await expect(broker.handle(request)).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, request)).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'invalid-request' }); expect(fs.readdirSync(String(config.workDir))).toEqual([]); }); @@ -245,7 +286,6 @@ describe('sealed-probe broker', () => { [ 'no output file', (invocationDir: string): void => { - // Remove the pre-created output file to simulate a probe that never wrote. fs.unlinkSync(path.join(invocationDir, 'out')); }, 'unreadable-output', @@ -253,22 +293,20 @@ describe('sealed-probe broker', () => { [ 'oversized output', (invocationDir: string): void => { - fs.writeFileSync(path.join(invocationDir, 'out'), 'x'.repeat(4096)); + fs.writeFileSync(path.join(invocationDir, 'out'), 'x'.repeat(8193)); }, 'unreadable-output', ], [ 'symlinked output', (invocationDir: string): void => { - // Replace the pre-created output file with a symlink to test that - // readProbeOutput rejects symlinks (O_NOFOLLOW defence). fs.unlinkSync(path.join(invocationDir, 'out')); fs.symlinkSync('/etc/hosts', path.join(invocationDir, 'out')); }, 'unreadable-output', ], [ - 'undeclared outcome', + 'undeclared enum value', (invocationDir: string): void => { fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"MAYBE"}'); }, @@ -302,18 +340,11 @@ describe('sealed-probe broker', () => { }, 'unreadable-output', ], - [ - 'script-written ERROR', - (invocationDir: string): void => { - fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"ERROR"}'); - }, - 'nonconformant-output', - ], ])('maps %s to the canonical error', async (_name, behaviour, reason) => { const runner = probeRunner(behaviour as (invocationDir: string) => void); const { broker, audit } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason }); expect(fs.readdirSync(String(config.workDir))).toEqual([]); }); @@ -324,7 +355,7 @@ describe('sealed-probe broker', () => { }, { timedOut: true, exitCode: 137 }); const { broker, audit } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'timeout' }); }); @@ -334,11 +365,11 @@ describe('sealed-probe broker', () => { }, { exitCode: 2 }); const { broker, audit } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'non-zero-exit' }); }); - it('maps cleanup failure to the canonical error instead of returning a valid outcome', async () => { + it('includes cleanup before the response and maps cleanup failure to canonical error', async () => { const runner = probeRunner((invocationDir) => { fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); }); @@ -348,9 +379,9 @@ describe('sealed-probe broker', () => { throw new Error('cleanup failed'); }, }; - const { broker, audit } = build(runner, cleanupFailingWorkspace); + const { broker, audit } = build(runner, { workspace: cleanupFailingWorkspace }); - await expect(broker.handle(validRequest())).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'cleanup-failed' }); }); @@ -362,22 +393,18 @@ describe('sealed-probe broker', () => { } as unknown as { runProbeContainer: (params: never) => Promise }; const { broker, audit } = build(runner); - await expect(broker.handle(validRequest())).resolves.toBe(CANONICAL_ERROR); + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'launch-failed' }); expect(fs.readdirSync(String(config.workDir))).toEqual([]); }); - it('produces byte-identical responses for success-shaped and every failure-shaped answer', async () => { + it('produces byte-identical responses for every failure-shaped answer', async () => { const failures = await Promise.all([ - build(probeRunner(() => {})).broker.handle(validRequest()), - build(probeRunner(() => {})).broker.handle(validRequest('octo/nope')), - build(probeRunner(() => {})).broker.handle({ privateRepo: 'octo/alpha', outcomes: ['A'], script: 'x' }), + invoke(build(probeRunner(() => {})).broker, validRequest('octo/nope')), + invoke(build(probeRunner(() => {})).broker, { privateRepo: 'octo/alpha', schema: { type: 'nope' }, script: 'x' }), ]); expect(new Set(failures)).toEqual(new Set([CANONICAL_ERROR])); - for (const failure of failures) { - expect(failure).toBe(CANONICAL_ERROR); - } }); it('enforces the per-run invocation budget atomically and without launching', async () => { @@ -392,24 +419,21 @@ describe('sealed-probe broker', () => { } as unknown as { runProbeContainer: (params: never) => Promise }; const { broker, audit } = build(runner); - const results = await Promise.all( - Array.from({ length: 5 }, () => broker.handle(validRequest())), - ); + const results = await Promise.all(Array.from({ length: 5 }, () => invoke(broker, validRequest()))); - expect(results.filter((r) => r === '{"result":"YES"}')).toHaveLength(3); + expect(results.filter((r) => r === '{"status":"ok","result":{"result":"YES"}}')).toHaveLength(3); expect(results.filter((r) => r === CANONICAL_ERROR)).toHaveLength(2); expect(launches).toHaveLength(3); - expect(audit.records.filter((r) => r.reason === 'budget-exhausted')).toHaveLength(2); + expect(audit.records.filter((r) => r.reason === 'invocation-count-exhausted')).toHaveLength(2); }); it('records failure reasons only in the protected audit log, never in the response', async () => { - // Probe deletes the output file so the broker cannot read it. const runner = probeRunner((invocationDir) => { fs.unlinkSync(path.join(invocationDir, 'out')); }); const { broker, audit } = build(runner); - const response = await broker.handle(validRequest()); + const response = await invoke(broker, validRequest()); expect(response).toBe(CANONICAL_ERROR); expect(JSON.stringify(audit.records)).toContain('unreadable-output'); @@ -430,6 +454,176 @@ describe('sealed-probe broker', () => { expect(fs.readlinkSync(path.join(layout.repoDir, 'README-link'))).toBe('README.md'); }); + + describe('per-repository bit ledger (no per-query cap)', () => { + it("debits an invocation's exact schema charge before copying a seed or launching Python", async () => { + const runner = probeRunner((invocationDir) => { + fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); + }); + const { broker } = build(runner); + + // 1 (status) + 2 (ceil(log2(3)) for the 3-valued enum) + 3 (timing) = 6 bits. + const before = broker.ledger.remainingBits('octo/alpha'); + await invoke(broker, validRequest()); + expect(broker.ledger.remainingBits('octo/alpha')).toBe(before - 6); + }); + + it('never debits the ledger for a request rejected before validation succeeds', async () => { + const runner = probeRunner(() => { + throw new Error('probe must not launch'); + }); + const { broker } = build(runner); + + const before = broker.ledger.remainingBits('octo/alpha'); + await invoke(broker, { ...validRequest(), schema: { type: 'nope' } }); + expect(broker.ledger.remainingBits('octo/alpha')).toBe(before); + }); + + it('denies (without launching) an invocation whose schema charge exceeds the remaining balance', async () => { + const runner = probeRunner(() => { + throw new Error('probe must not launch: charge exceeds confidential (8-bit) budget'); + }); + const { broker, audit } = build(runner); + + // A 256-value enum costs 1 + 8 + 3 = 12 bits — more than octo/beta's + // 8-bit "confidential" run budget. + const expensiveSchema = { type: 'enum', values: Array.from({ length: 256 }, (_, i) => i) }; + const response = await invoke(broker, { privateRepo: 'octo/beta', schema: expensiveSchema, script: 'x' }); + + expect(response).toBe(CANONICAL_ERROR); + expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'bit-budget-exhausted' }); + expect(fs.readdirSync(String(config.workDir))).toEqual([]); + }); + + it('a sealed-sensitivity repository (0-bit run budget) can never afford even the cheapest schema', async () => { + const sealedSeedMap = new Map([['octo/sealed', { seedId: seedIdA, sensitivity: 'sealed' }]]); + const runner = probeRunner(() => { + throw new Error('a sealed repo must never launch a probe'); + }); + const { broker, audit } = build(runner, { seeds: sealedSeedMap }); + + // The cheapest possible schema (const) still costs 1 + 0 + 3 = 4 bits > 0. + const response = await invoke(broker, { + privateRepo: 'octo/sealed', + schema: { type: 'const', value: 'x' }, + script: 'x', + }); + + expect(response).toBe(CANONICAL_ERROR); + expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'bit-budget-exhausted' }); + expect(fs.readdirSync(String(config.workDir))).toEqual([]); + }); + + it('a public-sensitivity repository is unmetered and never runs out of budget', async () => { + const publicSeedMap = new Map([['octo/public', { seedId: seedIdA, sensitivity: 'public' }]]); + config.maxInvocations = 20; + const runner = probeRunner((invocationDir) => { + fs.writeFileSync(path.join(invocationDir, 'out'), '[0,0,0,0,0,0,0,0]'); + }); + const { broker } = build(runner, { seeds: publicSeedMap }); + + // A tuple of eight 16-bit integers costs 1 + 128 + 3 = 132 bits — far + // beyond even "internal"'s 64-bit run budget, many times over. Only + // "public" (unmetered, `null` in the ledger) could ever afford it more + // than zero times. + const bigSchema = { + type: 'tuple', + items: Array.from({ length: 8 }, () => ({ type: 'integer', minimum: 0, maximum: 65535 })), + }; + for (let i = 0; i < 10; i++) { + // eslint-disable-next-line no-await-in-loop + expect(await invoke(broker, { privateRepo: 'octo/public', schema: bigSchema, script: 'x' })).not.toBe( + CANONICAL_ERROR, + ); + } + expect(broker.ledger.remainingBits('octo/public')).toBeNull(); + }); + }); + + describe('response-timing bucketing (fake monotonic clock — no real time elapses)', () => { + it('buckets a fast-completing invocation to the smallest boundary at or after elapsed processing time', async () => { + const { clock, advance, sleeps } = createFakeClock(); + const runner = probeRunner((invocationDir) => { + advance(50); // Simulate 50ms of processing — falls in the 100ms bucket. + fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); + }); + const { broker, audit } = build(runner, { clock }); + + await invoke(broker, validRequest()); + + const invocationRecord = audit.records.find((r) => r.kind === 'invocation'); + expect(invocationRecord).toMatchObject({ bucketMs: 100 }); + // Waited the remaining 50ms to reach the 100ms boundary. + expect(sleeps).toEqual([50]); + }); + + it('does not wait at all when processing already lands exactly on a bucket boundary', async () => { + const { clock, advance, sleeps } = createFakeClock(); + const runner = probeRunner((invocationDir) => { + advance(10); // Exactly the smallest bucket. + fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); + }); + const { broker, audit } = build(runner, { clock }); + + await invoke(broker, validRequest()); + + expect(audit.records.find((r) => r.kind === 'invocation')).toMatchObject({ bucketMs: 10 }); + expect(sleeps).toEqual([]); + }); + + it('buckets a failure response exactly like a success response', async () => { + const { clock, advance } = createFakeClock(); + const runner = probeRunner((invocationDir) => { + advance(500); // Falls in the 1000ms bucket. + fs.writeFileSync(path.join(invocationDir, 'out'), 'not valid json'); + }); + const { broker, audit } = build(runner, { clock }); + + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); + const failureRecord = [...audit.records].reverse().find((r) => r.kind === 'failure'); + // Failure records don't currently carry bucketMs (only invocation + // records do), but the wait itself must still have occurred — this is + // implicitly proven by the overflow test below reaching a different + // code path only when elapsed exceeds every bucket. + expect(failureRecord).toMatchObject({ reason: 'nonconformant-output' }); + }); + + it('includes workspace cleanup latency when selecting the timing bucket', async () => { + const { clock, advance, sleeps } = createFakeClock(); + const runner = probeRunner((invocationDir) => { + advance(5); + fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); + }); + const cleanupWorkspace = { + ...workspace, + destroyInvocationWorkspace: (workDir: string, invocationId: string) => { + advance(50); + workspace.destroyInvocationWorkspace(workDir, invocationId); + }, + }; + const { broker, audit } = build(runner, { clock, workspace: cleanupWorkspace }); + + await invoke(broker, validRequest()); + + expect(audit.records.find((r) => r.kind === 'invocation')).toMatchObject({ bucketMs: 100 }); + expect(sleeps).toEqual([45]); + }); + + it('fails closed with the canonical error when processing overruns every configured bucket, even for an otherwise-valid result', async () => { + const { clock, advance } = createFakeClock(); + const runner = probeRunner((invocationDir) => { + // Pathological infrastructure latency far beyond the largest bucket + // (600_000ms) — never possible from the script itself, which is + // capped at sealedProbes.timeout <= 600s by preflight.ts. + advance(TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] + 1); + fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); + }); + const { broker, audit } = build(runner, { clock }); + + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); + expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'timing-bucket-overflow' }); + }); + }); }); describe('probe container arguments', () => { @@ -490,7 +684,6 @@ describe('probe container arguments', () => { it('backs /probe with a size-limited tmpfs for aggregate storage enforcement', () => { const joined = args().join(' '); expect(joined).toMatch(/--tmpfs \/probe:rw,nosuid,nodev,size=\d+,uid=65534,gid=65534,mode=0700/); - // No writable bind mount for the full /probe dir: a probe cannot fill the host FS expect(joined).not.toContain(':/probe:rw'); expect(joined).not.toContain(':/probe/repo:rw'); }); @@ -521,24 +714,27 @@ describe('probe container arguments', () => { }); }); -describe('request framing', () => { +describe('request framing (protocol v2)', () => { + function base64url(text: string): string { + return Buffer.from(text, 'utf8').toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, ''); + } + + const schema = { type: 'boolean' }; const headers = { - 'x-awf-probe-version': '1', + 'x-awf-probe-version': '2', 'x-awf-repo': 'octo/alpha', - 'x-awf-outcome-1': 'YES', - 'x-awf-outcome-2': 'NO', - 'x-awf-outcome-3': 'UNKNOWN', + 'x-awf-schema-b64': base64url(JSON.stringify(schema)), }; const rawHeaders = Object.entries(headers).flat(); - it('assembles the canonical request object', () => { + it('assembles the canonical request object, decoding the schema header', () => { expect(buildRequestFromFrame(headers, rawHeaders, 'print(1)')).toEqual({ - request: { privateRepo: 'octo/alpha', outcomes: ['YES', 'NO', 'UNKNOWN'], script: 'print(1)' }, + request: { privateRepo: 'octo/alpha', schema, script: 'print(1)' }, }); }); it('rejects an unsupported protocol version', () => { - expect(buildRequestFromFrame({ ...headers, 'x-awf-probe-version': '2' }, rawHeaders, 'x').error) + expect(buildRequestFromFrame({ ...headers, 'x-awf-probe-version': '1' }, rawHeaders, 'x').error) .toMatch(/protocol version/); }); @@ -547,8 +743,8 @@ describe('request framing', () => { expect(buildRequestFromFrame(headers, withExtra, 'x').error).toMatch(/unsupported request control header/); }); - it('rejects duplicated headers so outcomes cannot be smuggled', () => { - const duplicated = [...rawHeaders, 'X-AWF-Outcome-1', 'SNEAKY']; + it('rejects duplicated headers so the repo or schema cannot be smuggled', () => { + const duplicated = [...rawHeaders, 'X-AWF-Repo', 'octo/sneaky']; expect(buildRequestFromFrame(headers, duplicated, 'x').error).toMatch(/duplicate request header/); }); @@ -556,13 +752,60 @@ describe('request framing', () => { return Object.fromEntries(Object.entries(headers).filter(([key]) => key !== name)); } - it('rejects a missing outcome', () => { - expect(buildRequestFromFrame(omit('x-awf-outcome-3'), rawHeaders, 'x').error) - .toMatch(/missing outcome header/); + it('rejects a missing schema header', () => { + expect(buildRequestFromFrame(omit('x-awf-schema-b64'), rawHeaders, 'x').error) + .toMatch(/missing or malformed schema header/); }); it('rejects a missing repository selector', () => { expect(buildRequestFromFrame(omit('x-awf-repo'), rawHeaders, 'x').error) .toMatch(/missing repository selector/); }); + + it('rejects a schema header that is not valid base64url', () => { + expect(buildRequestFromFrame({ ...headers, 'x-awf-schema-b64': 'not base64url!!' }, rawHeaders, 'x').error) + .toMatch(/missing or malformed schema header/); + }); + + it('rejects a schema header that decodes to invalid JSON', () => { + const badSchema = base64url('not json at all'); + expect(buildRequestFromFrame({ ...headers, 'x-awf-schema-b64': badSchema }, rawHeaders, 'x').error) + .toMatch(/not valid JSON/); + }); + + it('rejects a schema header that decodes to invalid UTF-8', () => { + const invalidUtf8 = Buffer.from([0xff, 0xfe]).toString('base64url'); + expect(buildRequestFromFrame({ ...headers, 'x-awf-schema-b64': invalidUtf8 }, rawHeaders, 'x').error) + .toMatch(/missing or malformed schema header/); + }); +}); + +describe('bounded request body reading', () => { + function fakeRequest(chunks: (Buffer | string)[]): EventEmitter & { pause: () => void } { + const emitter = new EventEmitter() as EventEmitter & { pause: () => void }; + emitter.pause = jest.fn(); + process.nextTick(() => { + for (const chunk of chunks) emitter.emit('data', Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk)); + emitter.emit('end'); + }); + return emitter; + } + + it('reads a well-formed script body', async () => { + const req = fakeRequest(['print', '(1)']); + await expect(readBoundedBody(req)).resolves.toEqual({ script: 'print(1)' }); + }); + + it('rejects a body exceeding the script size cap while streaming', async () => { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const { MAX_SCRIPT_BYTES } = require(path.join(brokerDir, 'protocol.js')); + const req = fakeRequest(['x'.repeat(MAX_SCRIPT_BYTES + 1)]); + const result = await readBoundedBody(req); + expect(result).toEqual({ error: 'script exceeds maximum size' }); + }); + + it('rejects a body that is not valid UTF-8', async () => { + const req = fakeRequest([Buffer.from([0xff, 0xfe])]); + await expect(readBoundedBody(req)).resolves.toEqual({ error: 'script is not valid UTF-8' }); + }); }); diff --git a/src/sealed-probe/end-to-end.test.ts b/src/sealed-probe/end-to-end.test.ts index 00f123953..0826057f2 100644 --- a/src/sealed-probe/end-to-end.test.ts +++ b/src/sealed-probe/end-to-end.test.ts @@ -10,9 +10,11 @@ import type { Server } from 'http'; * real `sealed-probe` wrapper → real Unix socket → real broker server → * real workspace/seed handling → (mocked) probe container. * - * Only the Docker launch is mocked, so this covers the framing, the protocol, - * the writable-copy semantics, repository isolation, the invocation budget, - * and the uniform failure closure without needing a Docker daemon or a real + * Only the Docker launch is mocked, so this covers the v2 framing (repo + + * base64url schema header), the finite schema DSL, the writable-copy + * semantics, repository isolation, the per-repository sensitivity/bit + * ledger (no per-query cap), the operational invocation budget, and the + * uniform failure closure — without needing a Docker daemon or a real * private repository. */ /* eslint-disable @typescript-eslint/no-require-imports */ @@ -23,7 +25,8 @@ const workspace = require(path.join(brokerDir, 'workspace.js')); /* eslint-enable @typescript-eslint/no-require-imports */ const WRAPPER = path.join(__dirname, '..', '..', 'containers', 'agent', 'sealed-probe-wrapper.sh'); -const CANONICAL_ERROR = '{"result":"ERROR"}'; +const CANONICAL_ERROR = '{"status":"error"}'; +const OUTCOME_SCHEMA = JSON.stringify({ type: 'enum', values: ['YES', 'NO'] }); interface WrapperResult { stdout: string; @@ -63,11 +66,8 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { runProbeContainer: async ({ invocationId }: { invocationId: string }) => { const invocationDir = path.join(String(config.workDir), invocationId); const readme = fs.readFileSync(path.join(invocationDir, 'repo', 'README.md'), 'utf8'); - // Write the answer to the pre-created output file. - fs.writeFileSync( - path.join(invocationDir, 'out'), - JSON.stringify({ result: readme.includes('alpha') ? 'YES' : 'NO' }), - ); + // Write the answer to the pre-created output file, conforming to the enum schema above. + fs.writeFileSync(path.join(invocationDir, 'out'), JSON.stringify(readme.includes('alpha') ? 'YES' : 'NO')); return { exitCode: 0, timedOut: false }; }, }; @@ -129,7 +129,10 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { const broker = createBroker({ config, - seedMap: new Map([['octo/alpha', seedIdA], ['octo/beta', seedIdB]]), + seedMap: new Map([ + ['octo/alpha', { seedId: seedIdA, sensitivity: 'internal' }], + ['octo/beta', { seedId: seedIdB, sensitivity: 'confidential' }], + ]), runId: 'e2e-run', audit: auditLog, workspace, @@ -146,18 +149,18 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { fs.rmSync(root, { recursive: true, force: true }); }); - const args = (repo: string) => ['--repo', repo, '--outcome', 'YES', '--outcome', 'NO', '--outcome', 'UNKNOWN']; + const args = (repo: string, schema = OUTCOME_SCHEMA) => ['--repo', repo, '--schema', schema]; it('returns the outcome the probe computed from its own repository copy', async () => { const result = await runWrapper(socketPath, args('octo/alpha')); - expect(result.stdout).toBe('{"result":"YES"}\n'); + expect(result.stdout).toBe('{"status":"ok","result":"YES"}\n'); expect(result.stderr).toBe(''); expect(result.status).toBe(0); }); it('gives each repository its own contents and never the other one', async () => { - expect((await runWrapper(socketPath, args('octo/beta'))).stdout).toBe('{"result":"NO"}\n'); + expect((await runWrapper(socketPath, args('octo/beta'))).stdout).toBe('{"status":"ok","result":"NO"}\n'); }); it('leaves the immutable seed untouched after the probe mutates its copy', async () => { @@ -177,18 +180,31 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { expect(audit.some((record) => record.reason === 'repo-not-allowed')).toBe(true); }); - it('enforces the invocation budget across the socket', async () => { + it('enforces the operational invocation budget across the socket, independent of the bit budget', async () => { const first = await runWrapper(socketPath, args('octo/alpha')); const second = await runWrapper(socketPath, args('octo/alpha')); const third = await runWrapper(socketPath, args('octo/alpha')); - expect(first.stdout).toBe('{"result":"YES"}\n'); - expect(second.stdout).toBe('{"result":"YES"}\n'); + expect(first.stdout).toBe('{"status":"ok","result":"YES"}\n'); + expect(second.stdout).toBe('{"status":"ok","result":"YES"}\n'); expect(third.stdout).toBe(`${CANONICAL_ERROR}\n`); expect(third.stderr).toBe(''); expect(third.status).toBe(0); }); + it('enforces the confidential (8-bit) per-repository run budget across the socket', async () => { + // octo/beta is "confidential" (8 bits/run). A 2-value enum costs + // 1 + 1 + 3 = 5 bits, so two invocations (10 bits) exceed the budget — + // the second must be denied even though maxInvocations (2) alone would + // still allow it. + const first = await runWrapper(socketPath, args('octo/beta')); + const second = await runWrapper(socketPath, args('octo/beta')); + + expect(first.stdout).toBe('{"status":"ok","result":"NO"}\n'); + expect(second.stdout).toBe(`${CANONICAL_ERROR}\n`); + expect(audit.some((record) => record.reason === 'bit-budget-exhausted')).toBe(true); + }); + it('rejects an oversized script with the canonical error', async () => { const result = await runWrapper(socketPath, args('octo/alpha'), 'x'.repeat(64 * 1024 + 10)); @@ -196,4 +212,35 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { expect(result.stderr).toBe(''); expect(result.status).toBe(0); }); + + it('rejects a request whose probe output does not conform to its own declared schema', async () => { + const nonConformingRunner = { + runProbeContainer: async ({ invocationId }: { invocationId: string }) => { + const invocationDir = path.join(String(config.workDir), invocationId); + fs.writeFileSync(path.join(invocationDir, 'out'), '"MAYBE"'); // not in the declared enum + return { exitCode: 0, timedOut: false }; + }, + }; + const auditLog = { + invocation: () => { /* not asserted */ }, + failure: (invocationId: string, reason: string) => audit.push({ kind: 'failure', invocationId, reason }), + lifecycle: () => { /* not asserted */ }, + }; + const broker = createBroker({ + config, + seedMap: new Map([['octo/alpha', { seedId: seedIdA, sensitivity: 'internal' }]]), + runId: 'e2e-run-2', + audit: auditLog, + workspace, + runner: nonConformingRunner, + }); + await new Promise((resolve) => server.close(() => resolve())); + server = createServer({ broker, audit: auditLog }); + await listenOnSocket(server, config, auditLog); + + const result = await runWrapper(socketPath, args('octo/alpha')); + + expect(result.stdout).toBe(`${CANONICAL_ERROR}\n`); + expect(result.status).toBe(0); + }); }); diff --git a/src/sealed-probe/ledger.test.ts b/src/sealed-probe/ledger.test.ts new file mode 100644 index 000000000..d92616d15 --- /dev/null +++ b/src/sealed-probe/ledger.test.ts @@ -0,0 +1,112 @@ +import * as path from 'path'; + +/** + * Unit tests for the per-repository information-budget ledger. + * + * There is no per-query cap: every invocation's schema-derived charge (see + * `queryBitsForSchema` in `./protocol`) is atomically checked against and + * debited from the repository's shared run balance. These tests exercise + * the ledger in isolation, independent of the broker's orchestration. + */ +/* eslint-disable @typescript-eslint/no-require-imports */ +const brokerDir = path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker'); +const { createLedger } = require(path.join(brokerDir, 'ledger.js')); +/* eslint-enable @typescript-eslint/no-require-imports */ + +interface Ledger { + tryDebit(repoKey: string, bits: number): boolean; + remainingBits(repoKey: string): number | null | undefined; +} + +function buildLedger(seeds: Array<[string, string]>): Ledger { + return createLedger(new Map(seeds.map(([repo, sensitivity]) => [repo, { seedId: 'seed', sensitivity }]))); +} + +describe('createLedger', () => { + it('starts each repository at its sensitivity-derived run budget', () => { + const ledger = buildLedger([ + ['octo/pub', 'public'], + ['octo/int', 'internal'], + ['octo/conf', 'confidential'], + ['octo/sealed', 'sealed'], + ]); + + expect(ledger.remainingBits('octo/pub')).toBeNull(); + expect(ledger.remainingBits('octo/int')).toBe(64); + expect(ledger.remainingBits('octo/conf')).toBe(8); + expect(ledger.remainingBits('octo/sealed')).toBe(0); + }); + + it('returns undefined for a repository outside the ledger', () => { + const ledger = buildLedger([['octo/int', 'internal']]); + expect(ledger.remainingBits('octo/unknown')).toBeUndefined(); + }); + + it('debits exactly the requested charge on success', () => { + const ledger = buildLedger([['octo/int', 'internal']]); + expect(ledger.tryDebit('octo/int', 10)).toBe(true); + expect(ledger.remainingBits('octo/int')).toBe(54); + expect(ledger.tryDebit('octo/int', 54)).toBe(true); + expect(ledger.remainingBits('octo/int')).toBe(0); + }); + + it('denies (without debiting) a charge exceeding the remaining balance', () => { + const ledger = buildLedger([['octo/conf', 'confidential']]); + expect(ledger.tryDebit('octo/conf', 9)).toBe(false); + expect(ledger.remainingBits('octo/conf')).toBe(8); + }); + + it('allows a charge exactly equal to the remaining balance (exhausting it)', () => { + const ledger = buildLedger([['octo/conf', 'confidential']]); + expect(ledger.tryDebit('octo/conf', 8)).toBe(true); + expect(ledger.remainingBits('octo/conf')).toBe(0); + // Even the cheapest possible charge (4 bits: 1 status + 0 const + 3 timing) is now unaffordable. + expect(ledger.tryDebit('octo/conf', 4)).toBe(false); + }); + + it('a sealed (0-bit) repository can never afford any positive charge', () => { + const ledger = buildLedger([['octo/sealed', 'sealed']]); + expect(ledger.tryDebit('octo/sealed', 1)).toBe(false); + expect(ledger.tryDebit('octo/sealed', 0)).toBe(true); // A zero-bit charge is not physically possible in practice (min charge is 4), but is not itself unaffordable. + expect(ledger.remainingBits('octo/sealed')).toBe(0); + }); + + it('a public (unmetered) repository can never be exhausted regardless of charge size', () => { + const ledger = buildLedger([['octo/pub', 'public']]); + expect(ledger.tryDebit('octo/pub', 1_000_000)).toBe(true); + expect(ledger.tryDebit('octo/pub', Number.MAX_SAFE_INTEGER)).toBe(true); + expect(ledger.remainingBits('octo/pub')).toBeNull(); + }); + + it('denies a debit against an unknown repository', () => { + const ledger = buildLedger([['octo/int', 'internal']]); + expect(ledger.tryDebit('octo/unknown', 1)).toBe(false); + }); + + it('tracks balances independently per repository', () => { + const ledger = buildLedger([ + ['octo/a', 'internal'], + ['octo/b', 'internal'], + ]); + expect(ledger.tryDebit('octo/a', 60)).toBe(true); + expect(ledger.remainingBits('octo/a')).toBe(4); + expect(ledger.remainingBits('octo/b')).toBe(64); + }); + + it('never refunds a charge, regardless of the invocation outcome', () => { + // The ledger API has no refund/credit operation at all — modeling the + // "never refunded" guarantee structurally rather than behaviorally. + const ledger = buildLedger([['octo/int', 'internal']]); + expect(Object.keys(ledger)).not.toContain('refund'); + expect(Object.keys(ledger)).not.toContain('credit'); + }); + + it('accumulates many small debits down to exactly zero remaining', () => { + const ledger = buildLedger([['octo/int', 'internal']]); + for (let i = 0; i < 16; i++) { + expect(ledger.tryDebit('octo/int', 4)).toBe(true); + } + expect(ledger.remainingBits('octo/int')).toBe(0); + expect(ledger.tryDebit('octo/int', 1)).toBe(false); + }); +}); diff --git a/src/sealed-probe/manager.test.ts b/src/sealed-probe/manager.test.ts index e6cce46d2..606d9f6aa 100644 --- a/src/sealed-probe/manager.test.ts +++ b/src/sealed-probe/manager.test.ts @@ -26,7 +26,7 @@ const mockReleaseSeedPermissions = releaseSeedPermissions as jest.MockedFunction const sealedProbes: SealedProbesConfig = { enabled: true, - privateRepos: ['octo/private'], + privateRepos: [{ repo: 'octo/private', sensitivity: 'internal' }], runtime: 'docker', timeout: 30, memoryLimit: '512m', @@ -91,9 +91,11 @@ describe('prepareSealedProbes', () => { expect(fs.existsSync(paths.skillPath)).toBe(true); const seedMap = JSON.parse(fs.readFileSync(paths.seedMapPath, 'utf8')); - expect(seedMap.version).toBe(1); + expect(seedMap.version).toBe(2); expect(seedMap.runId).toMatch(/^[0-9a-f]{32}$/); - expect(seedMap.seeds).toEqual([{ repo: 'octo/private', seedId: expect.stringMatching(/^[0-9a-f]{32}$/) }]); + expect(seedMap.seeds).toEqual([ + { repo: 'octo/private', seedId: expect.stringMatching(/^[0-9a-f]{32}$/), sensitivity: 'internal' }, + ]); expect(fs.statSync(paths.seedMapPath).mode & 0o777).toBe(0o600); }); diff --git a/src/sealed-probe/manager.ts b/src/sealed-probe/manager.ts index c4ae5f84a..9ceb2933b 100644 --- a/src/sealed-probe/manager.ts +++ b/src/sealed-probe/manager.ts @@ -161,7 +161,11 @@ export async function prepareSealedProbes( writeSeedMap(paths, { version: SEALED_PROBE_SEED_MAP_VERSION, runId: staging.runId, - seeds: staging.seeds.map((seed) => ({ repo: seed.repoKey, seedId: seed.seedId })), + seeds: staging.seeds.map((seed) => ({ + repo: seed.repoKey, + seedId: seed.seedId, + sensitivity: seed.sensitivity, + })), }); writeSealedProbeSkill(paths, { diff --git a/src/sealed-probe/preflight.test.ts b/src/sealed-probe/preflight.test.ts index 48174ca9a..6b484842e 100644 --- a/src/sealed-probe/preflight.test.ts +++ b/src/sealed-probe/preflight.test.ts @@ -2,13 +2,18 @@ import type { WrapperConfig } from '../types'; import execa from 'execa'; import { assertProbeRuntimeAvailable, preflightTestHelpers, validateSealedProbeConfig } from './preflight'; import type { SealedProbesConfig } from '../types'; +import type { SealedProbeRepository } from '../types/sealed-probe-options'; jest.mock('execa', () => ({ __esModule: true, default: jest.fn() })); const mockExeca = execa as unknown as jest.Mock; +function repo(name: string, sensitivity: SealedProbeRepository['sensitivity'] = 'internal'): SealedProbeRepository { + return { repo: name, sensitivity }; +} + const baseSealedProbes: SealedProbesConfig = { enabled: true, - privateRepos: ['octo/private'], + privateRepos: [repo('octo/private')], runtime: 'docker', timeout: 30, memoryLimit: '512m', @@ -49,14 +54,14 @@ describe('validateSealedProbeConfig', () => { ['octo/../etc', 'traversal'], ['user:token@octo/private', 'credentials'], ['octo/private/extra', 'extra path segment'], - ])('rejects unsafe repository slug %s (%s)', (repo) => { - const errors = validateSealedProbeConfig(buildConfig({ privateRepos: [repo] }), envWithToken); + ])('rejects unsafe repository slug %s (%s)', (repoSlug) => { + const errors = validateSealedProbeConfig(buildConfig({ privateRepos: [repo(repoSlug)] }), envWithToken); expect(errors.join('\n')).toContain('is not a bare owner/repo slug'); }); it('rejects case-insensitive duplicates', () => { const errors = validateSealedProbeConfig( - buildConfig({ privateRepos: ['octo/private', 'Octo/Private'] }), + buildConfig({ privateRepos: [repo('octo/private'), repo('Octo/Private')] }), envWithToken, ); expect(errors.join('\n')).toContain('duplicate entry'); @@ -120,6 +125,16 @@ describe('validateSealedProbeConfig', () => { expect(errors.join('\n')).toContain('is not a Docker memory limit'); }); + it('accepts a timeout at exactly the largest timing bucket (600s)', () => { + expect(validateSealedProbeConfig(buildConfig({ timeout: 600 }), envWithToken)).toEqual([]); + }); + + it('rejects a timeout beyond the largest timing bucket, which could leak unbucketed timing', () => { + const errors = validateSealedProbeConfig(buildConfig({ timeout: 601 }), envWithToken); + expect(errors.join('\n')).toContain('timeout must be at most 600 seconds'); + expect(errors.join('\n')).toContain('unbucketed secret-dependent information'); + }); + it('rejects an unsupported interpreter', () => { const errors = validateSealedProbeConfig( buildConfig({ interpreter: 'ruby' as unknown as SealedProbesConfig['interpreter'] }), diff --git a/src/sealed-probe/preflight.ts b/src/sealed-probe/preflight.ts index ebb7d9829..a11ba3a98 100644 --- a/src/sealed-probe/preflight.ts +++ b/src/sealed-probe/preflight.ts @@ -3,7 +3,7 @@ import { getLocalDockerEnv } from '../host-env'; import { runtimeUsesComposeAgent } from '../container-runtime'; import type { SealedProbesConfig, WrapperConfig } from '../types'; import { normalizeRepoKey } from './paths'; -import { SEALED_PROBE_REPO_PATTERN } from './protocol'; +import { SEALED_PROBE_REPO_PATTERN, TIMING_BUCKETS_MS } from './protocol'; import { resolveStagingToken } from './staging'; /** @@ -63,7 +63,8 @@ export function validateSealedProbeConfig( } const seenKeys = new Set(); - for (const repo of sealedProbes.privateRepos) { + for (const entry of sealedProbes.privateRepos) { + const repo = entry.repo; if (!SEALED_PROBE_REPO_PATTERN.test(repo)) { errors.push( `sealedProbes.privateRepos entry "${repo}" is not a bare owner/repo slug ` + @@ -90,8 +91,21 @@ export function validateSealedProbeConfig( errors.push(`sealedProbes.interpreter "${sealedProbes.interpreter}" is not supported`); } + // The largest observable timing bucket bounds how long the broker can ever + // wait before answering (see `TIMING_BUCKETS_MS` in ./protocol). Capping the + // configured timeout at that same ceiling guarantees every completed + // invocation — success, failure, or timeout — always lands inside a + // bucket, so response latency alone can never distinguish a timeout from a + // merely slow-but-successful script. + const maxTimeoutSeconds = TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] / 1000; if (!Number.isInteger(sealedProbes.timeout) || sealedProbes.timeout < 1) { errors.push('sealedProbes.timeout must be a positive integer number of seconds'); + } else if (sealedProbes.timeout > maxTimeoutSeconds) { + errors.push( + `sealedProbes.timeout must be at most ${maxTimeoutSeconds} seconds ` + + `(the largest response-timing bucket); a longer timeout could let an invocation's ` + + 'completion time itself leak unbucketed secret-dependent information', + ); } if (!Number.isInteger(sealedProbes.maxInvocations) || sealedProbes.maxInvocations < 1) { diff --git a/src/sealed-probe/protocol-parity.test.ts b/src/sealed-probe/protocol-parity.test.ts index 2d78af48e..2bd8bf26e 100644 --- a/src/sealed-probe/protocol-parity.test.ts +++ b/src/sealed-probe/protocol-parity.test.ts @@ -1,94 +1,310 @@ import * as path from 'path'; import { - buildSealedProbeResultSchema, - canonicalizeSealedProbeResult, - parseSealedProbeResult, - validateSealedProbeRequest, - MAX_OUTCOME_BYTES, + CANONICAL_ERROR_JSON, + MAX_ARRAY_LENGTH, + MAX_ENUM_VALUES, + MAX_OBJECT_FIELDS, + MAX_PRIVATE_REPO_LENGTH, MAX_REQUEST_BYTES, MAX_RESULT_BYTES, + MAX_SCHEMA_BYTES, + MAX_SCHEMA_DEPTH, + MAX_SCHEMA_NODES, MAX_SCRIPT_BYTES, - OUTCOME_COUNT, - RESERVED_ERROR_OUTCOME, + MAX_TUPLE_ITEMS, + MAX_UNION_VARIANTS, + PROBE_PROTOCOL_VERSION, + RESULT_STATUS_BIT_COST, SEALED_PROBE_REPO_PATTERN, - type SealedProbeOutcomes, + TIMING_BUCKETS_MS, + TIMING_BUCKET_BITS, + canonicalOkJson, + canonicalizeSchemaValue, + ceilLog2BigInt, + parseAndValidateProbeOutput, + queryBitsForSchema, + schemaCardinality, + strictParseJson, + validateSchema, + validateSealedProbeRequest, + validateValueAgainstSchema, + type SealedProbeSchemaNode, } from './protocol'; /** * The broker runs in its own container image and cannot import AWF's - * TypeScript sources, so `containers/sealed-probe/broker/protocol.js` restates - * the protocol. This suite runs one shared vector table through *both* - * implementations and fails the moment they disagree, which is what makes the - * duplication safe. + * TypeScript sources, so `containers/sealed-probe/broker/protocol.js` + * restates the entire v2 protocol (finite schema algebra, cardinality/bit + * charge, strict JSON parsing, request/result validation, canonicalization). + * This suite runs one shared vector table through *both* implementations and + * fails the moment they disagree, which is what makes the duplication safe. */ // eslint-disable-next-line @typescript-eslint/no-require-imports const brokerProtocol = require( path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker', 'protocol.js'), ); -const OUTCOMES: SealedProbeOutcomes = ['YES', 'NO', 'UNKNOWN']; +const SCHEMA_VECTORS: Array<{ name: string; schema: unknown }> = [ + { name: 'const string', schema: { type: 'const', value: 'ok' } }, + { name: 'const number', schema: { type: 'const', value: 42 } }, + { name: 'const boolean', schema: { type: 'const', value: false } }, + { name: 'const null', schema: { type: 'const', value: null } }, + { name: 'const extra property', schema: { type: 'const', value: 1, extra: true } }, + { name: 'boolean', schema: { type: 'boolean' } }, + { name: 'boolean extra property', schema: { type: 'boolean', extra: true } }, + { name: 'string enum', schema: { type: 'enum', values: ['a', 'b', 'c'] } }, + { name: 'integer enum', schema: { type: 'enum', values: [1, 2, 3] } }, + { name: 'enum duplicate values', schema: { type: 'enum', values: ['a', 'a'] } }, + { name: 'enum mixed types', schema: { type: 'enum', values: ['a', 1] } }, + { name: 'enum empty', schema: { type: 'enum', values: [] } }, + { name: `enum oversized (${MAX_ENUM_VALUES + 1})`, schema: { type: 'enum', values: Array.from({ length: MAX_ENUM_VALUES + 1 }, (_, i) => i) } }, + { name: 'integer bounded', schema: { type: 'integer', minimum: 0, maximum: 255 } }, + { name: 'integer maximum below minimum', schema: { type: 'integer', minimum: 10, maximum: 0 } }, + { name: 'integer non-integer bound', schema: { type: 'integer', minimum: 0.5, maximum: 10 } }, + { + name: 'object fixed fields', + schema: { + type: 'object', + fields: { ok: { type: 'boolean' }, count: { type: 'integer', minimum: 0, maximum: 3 } }, + }, + }, + { name: 'object empty fields', schema: { type: 'object', fields: {} } }, + { + name: `object oversized (${MAX_OBJECT_FIELDS + 1} fields)`, + schema: { + type: 'object', + fields: Object.fromEntries(Array.from({ length: MAX_OBJECT_FIELDS + 1 }, (_, i) => [`f${i}`, { type: 'boolean' }])), + }, + }, + { name: 'object invalid field name', schema: { type: 'object', fields: { 'bad name': { type: 'boolean' } } } }, + { name: 'tuple', schema: { type: 'tuple', items: [{ type: 'boolean' }, { type: 'boolean' }] } }, + { name: 'tuple empty', schema: { type: 'tuple', items: [] } }, + { + name: `tuple oversized (${MAX_TUPLE_ITEMS + 1} items)`, + schema: { type: 'tuple', items: Array.from({ length: MAX_TUPLE_ITEMS + 1 }, () => ({ type: 'boolean' })) }, + }, + { name: 'array fixed length', schema: { type: 'array', items: { type: 'boolean' }, length: 5 } }, + { name: 'array zero length', schema: { type: 'array', items: { type: 'boolean' }, length: 0 } }, + { name: 'array negative length', schema: { type: 'array', items: { type: 'boolean' }, length: -1 } }, + { name: `array oversized length (${MAX_ARRAY_LENGTH + 1})`, schema: { type: 'array', items: { type: 'boolean' }, length: MAX_ARRAY_LENGTH + 1 } }, + { + name: 'union tagged disjoint', + schema: { + type: 'union', + variants: { a: { type: 'boolean' }, b: { type: 'integer', minimum: 0, maximum: 9 } }, + }, + }, + { name: 'union empty variants', schema: { type: 'union', variants: {} } }, + { + name: `union oversized (${MAX_UNION_VARIANTS + 1} variants)`, + schema: { + type: 'union', + variants: Object.fromEntries(Array.from({ length: MAX_UNION_VARIANTS + 1 }, (_, i) => [`v${i}`, { type: 'boolean' }])), + }, + }, + { name: 'union invalid tag', schema: { type: 'union', variants: { '1bad': { type: 'boolean' } } } }, + { name: 'unknown node type', schema: { type: 'string' } }, + { name: 'not an object', schema: 'nope' }, + { name: 'null', schema: null }, + { name: 'array instead of object', schema: [1, 2] }, + { name: 'nested composite', schema: { + type: 'object', + fields: { + status: { type: 'enum', values: ['ok', 'error'] }, + items: { type: 'array', items: { type: 'integer', minimum: 0, maximum: 9 }, length: 3 }, + pair: { type: 'tuple', items: [{ type: 'boolean' }, { type: 'const', value: 'x' }] }, + choice: { type: 'union', variants: { a: { type: 'boolean' }, b: { type: 'boolean' } } }, + }, + } }, + { + name: `depth exceeded (${MAX_SCHEMA_DEPTH + 1} levels)`, + schema: (() => { + let deep: unknown = { type: 'boolean' }; + for (let i = 0; i <= MAX_SCHEMA_DEPTH; i++) deep = { type: 'array', items: deep, length: 1 }; + return deep; + })(), + }, + { + name: 'depth at exact limit', + schema: (() => { + let atLimit: unknown = { type: 'boolean' }; + for (let i = 0; i < MAX_SCHEMA_DEPTH; i++) atLimit = { type: 'array', items: atLimit, length: 1 }; + return atLimit; + })(), + }, + { + name: `node count exceeded (${MAX_SCHEMA_NODES} leaves)`, + schema: { type: 'tuple', items: Array.from({ length: MAX_SCHEMA_NODES }, () => ({ type: 'boolean' })) }, + }, + { name: 'undefined', schema: undefined }, +]; + +const VALID_SCHEMAS_FOR_VALUE_TESTS: Array<{ + name: string; + schema: SealedProbeSchemaNode; + values: unknown[]; +}> = [ + { name: 'const', schema: { type: 'const', value: 'ok' }, values: ['ok', 'not-ok', 1, null] }, + { name: 'boolean', schema: { type: 'boolean' }, values: [true, false, 1, 'true', null] }, + { name: 'enum', schema: { type: 'enum', values: ['a', 'b'] }, values: ['a', 'b', 'c', 1] }, + { + name: 'integer', + schema: { type: 'integer', minimum: 0, maximum: 10 }, + values: [0, 5, 10, 11, -1, 5.5, '5'], + }, + { + name: 'object', + schema: { type: 'object', fields: [{ name: 'ok', schema: { type: 'boolean' } }] }, + values: [{ ok: true }, {}, { ok: true, extra: 1 }, { ok: 'no' }, null, [true]], + }, + { + name: 'tuple', + schema: { type: 'tuple', items: [{ type: 'boolean' }, { type: 'boolean' }] }, + values: [[true, false], [true], [true, false, true], 'not-an-array'], + }, + { + name: 'array', + schema: { type: 'array', items: { type: 'boolean' }, length: 2 }, + values: [[true, false], [true], [true, false, true]], + }, + { + name: 'union', + schema: { + type: 'union', + variants: [ + { tag: 'a', schema: { type: 'boolean' } }, + { tag: 'b', schema: { type: 'integer', minimum: 0, maximum: 9 } }, + ], + }, + values: [ + { tag: 'a', value: true }, + { tag: 'b', value: 5 }, + { tag: 'b', value: true }, + { tag: 'c', value: true }, + { tag: 'a', value: true, extra: 1 }, + true, + ], + }, +]; const REQUEST_VECTORS: Array<{ name: string; request: unknown }> = [ - { name: 'valid request', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 'print(1)' } }, + { name: 'valid request', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 'print(1)' } }, { name: 'not an object', request: 'nope' }, { name: 'null', request: null }, { name: 'array', request: [] }, - { name: 'extra control field', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 'x', image: 'evil' } }, - { name: 'timeout control field', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 'x', timeout: 9999 } }, - { name: 'schema control field', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 'x', schema: {} } }, - { name: 'missing repo', request: { outcomes: [...OUTCOMES], script: 'x' } }, - { name: 'url repo', request: { privateRepo: 'https://github.com/octo/private', outcomes: [...OUTCOMES], script: 'x' } }, - { name: 'traversal repo', request: { privateRepo: 'octo/../../etc', outcomes: [...OUTCOMES], script: 'x' } }, - { name: 'wildcard repo', request: { privateRepo: 'octo/*', outcomes: [...OUTCOMES], script: 'x' } }, - { name: 'query repo', request: { privateRepo: 'octo/private?x=1', outcomes: [...OUTCOMES], script: 'x' } }, - { name: 'two outcomes', request: { privateRepo: 'octo/private', outcomes: ['A', 'B'], script: 'x' } }, - { name: 'four outcomes', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', 'C', 'D'], script: 'x' } }, - { name: 'duplicate outcomes', request: { privateRepo: 'octo/private', outcomes: ['A', 'A', 'B'], script: 'x' } }, - { name: 'reserved outcome', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', 'ERROR'], script: 'x' } }, - { name: 'empty outcome', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', ''], script: 'x' } }, - { name: 'control character outcome', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', 'C\n'], script: 'x' } }, - { name: 'oversized outcome', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', 'x'.repeat(MAX_OUTCOME_BYTES + 1)], script: 'x' } }, - { name: 'non-string outcome', request: { privateRepo: 'octo/private', outcomes: ['A', 'B', 3], script: 'x' } }, - { name: 'outcomes not an array', request: { privateRepo: 'octo/private', outcomes: 'A,B,C', script: 'x' } }, - { name: 'empty script', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: '' } }, - { name: 'non-string script', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 42 } }, - { name: 'oversized script', request: { privateRepo: 'octo/private', outcomes: [...OUTCOMES], script: 'x'.repeat(MAX_SCRIPT_BYTES + 1) } }, + { name: 'extra control field', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 'x', image: 'evil' } }, + { name: 'timeout control field', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 'x', timeout: 9999 } }, + { name: 'missing repo', request: { schema: { type: 'boolean' }, script: 'x' } }, + { name: 'url repo', request: { privateRepo: 'https://github.com/octo/private', schema: { type: 'boolean' }, script: 'x' } }, + { name: 'traversal repo', request: { privateRepo: 'octo/../../etc', schema: { type: 'boolean' }, script: 'x' } }, + { name: 'wildcard repo', request: { privateRepo: 'octo/*', schema: { type: 'boolean' }, script: 'x' } }, + { name: 'query repo', request: { privateRepo: 'octo/private?x=1', schema: { type: 'boolean' }, script: 'x' } }, + { name: `oversized repo (> ${MAX_PRIVATE_REPO_LENGTH})`, request: { privateRepo: `octo/${'r'.repeat(MAX_PRIVATE_REPO_LENGTH)}`, schema: { type: 'boolean' }, script: 'x' } }, + { name: 'missing schema', request: { privateRepo: 'octo/private', script: 'x' } }, + { name: 'invalid schema', request: { privateRepo: 'octo/private', schema: { type: 'nope' }, script: 'x' } }, + { name: 'empty script', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: '' } }, + { name: 'non-string script', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 42 } }, + { name: 'oversized script', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 'x'.repeat(MAX_SCRIPT_BYTES + 1) } }, + { name: 'script at exact size cap', request: { privateRepo: 'octo/private', schema: { type: 'boolean' }, script: 'x'.repeat(MAX_SCRIPT_BYTES) } }, ]; -const RESULT_VECTORS: string[] = [ - '{"result":"YES"}', - '{"result": "NO"}', - ' {"result":"UNKNOWN"} ', - '{"result":"ERROR"}', - '{"result":"MAYBE"}', - '{"result":"yes"}', - '{"result":"YES"} trailing', - '{"result":"YES"}{"result":"NO"}', - '{"result":"YES","extra":1}', - '{"result":"YES","result":"NO"}', - '{"answer":"YES"}', - '{"result":1}', - '{"result":null}', - '{"result":["YES"]}', - '["YES"]', - '', - 'YES', - '{', - '{"result":"YE\\u0053"}', - '{"result":"YES\\n"}', - '{"result":"Y\u0000ES"}', - `{"result":"${'x'.repeat(MAX_RESULT_BYTES)}"}`, +const RESULT_VECTORS: Array<{ name: string; schema: SealedProbeSchemaNode; raw: string }> = [ + { name: 'valid enum result', schema: { type: 'enum', values: ['YES', 'NO', 'UNKNOWN'] }, raw: '{"result":"YES"}' }, + { + name: 'whitespace tolerant', + schema: { type: 'object', fields: [{ name: 'result', schema: { type: 'enum', values: ['NO'] } }] }, + raw: ' { "result" : "NO" } ', + }, + { name: 'malformed JSON', schema: { type: 'boolean' }, raw: 'not json at all' }, + { name: 'duplicate keys', schema: { type: 'object', fields: [{ name: 'result', schema: { type: 'boolean' } }] }, raw: '{"result":true,"result":false}' }, + { name: 'trailing data', schema: { type: 'boolean' }, raw: 'true extra' }, + { name: 'two values concatenated', schema: { type: 'boolean' }, raw: 'true false' }, + { name: 'extra fields', schema: { type: 'object', fields: [{ name: 'ok', schema: { type: 'boolean' } }] }, raw: '{"ok":true,"extra":1}' }, + { name: 'value outside enum', schema: { type: 'enum', values: ['a', 'b'] }, raw: '"c"' }, + { name: 'wrong type', schema: { type: 'boolean' }, raw: '1' }, + { name: 'null value against boolean', schema: { type: 'boolean' }, raw: 'null' }, + { name: 'array instead of object', schema: { type: 'object', fields: [{ name: 'a', schema: { type: 'boolean' } }] }, raw: '["a"]' }, + { name: 'empty string', schema: { type: 'boolean' }, raw: '' }, + { name: 'single-quoted string', schema: { type: 'boolean' }, raw: "'true'" }, + { name: 'unterminated string', schema: { type: 'enum', values: ['x'] }, raw: '"x' }, + { name: 'raw control character', schema: { type: 'enum', values: ['line\nbreak'] }, raw: '"line\nbreak"' }, + { name: 'unicode escape', schema: { type: 'enum', values: ['s'] }, raw: '"\\u0073"' }, + { name: 'invalid hex escape', schema: { type: 'boolean' }, raw: '"\\uZZZZ"' }, + { name: 'invalid escape letter', schema: { type: 'boolean' }, raw: '"\\x41"' }, + { name: 'oversized result', schema: { type: 'enum', values: ['x'.repeat(MAX_RESULT_BYTES)] }, raw: `"${'x'.repeat(MAX_RESULT_BYTES)}"` }, + { + name: 'nested object matches regardless of key order', + schema: { + type: 'object', + fields: [ + { name: 'a', schema: { type: 'boolean' } }, + { name: 'b', schema: { type: 'boolean' } }, + ], + }, + raw: '{"b":true,"a":false}', + }, ]; describe('sealed-probe protocol parity (TypeScript vs broker JavaScript)', () => { it('exposes identical protocol constants', () => { - expect(brokerProtocol.OUTCOME_COUNT).toBe(OUTCOME_COUNT); - expect(brokerProtocol.RESERVED_ERROR_OUTCOME).toBe(RESERVED_ERROR_OUTCOME); - expect(brokerProtocol.MAX_OUTCOME_BYTES).toBe(MAX_OUTCOME_BYTES); + expect(brokerProtocol.PROBE_PROTOCOL_VERSION).toBe(PROBE_PROTOCOL_VERSION); + expect(brokerProtocol.MAX_SCHEMA_BYTES).toBe(MAX_SCHEMA_BYTES); + expect(brokerProtocol.MAX_SCHEMA_DEPTH).toBe(MAX_SCHEMA_DEPTH); + expect(brokerProtocol.MAX_SCHEMA_NODES).toBe(MAX_SCHEMA_NODES); + expect(brokerProtocol.MAX_ENUM_VALUES).toBe(MAX_ENUM_VALUES); + expect(brokerProtocol.MAX_OBJECT_FIELDS).toBe(MAX_OBJECT_FIELDS); + expect(brokerProtocol.MAX_TUPLE_ITEMS).toBe(MAX_TUPLE_ITEMS); + expect(brokerProtocol.MAX_ARRAY_LENGTH).toBe(MAX_ARRAY_LENGTH); + expect(brokerProtocol.MAX_UNION_VARIANTS).toBe(MAX_UNION_VARIANTS); expect(brokerProtocol.MAX_SCRIPT_BYTES).toBe(MAX_SCRIPT_BYTES); expect(brokerProtocol.MAX_REQUEST_BYTES).toBe(MAX_REQUEST_BYTES); expect(brokerProtocol.MAX_RESULT_BYTES).toBe(MAX_RESULT_BYTES); + expect(brokerProtocol.MAX_PRIVATE_REPO_LENGTH).toBe(MAX_PRIVATE_REPO_LENGTH); + expect(brokerProtocol.TIMING_BUCKETS_MS).toEqual(TIMING_BUCKETS_MS); + expect(brokerProtocol.TIMING_BUCKET_BITS).toBe(TIMING_BUCKET_BITS); + expect(brokerProtocol.RESULT_STATUS_BIT_COST).toBe(RESULT_STATUS_BIT_COST); expect(brokerProtocol.SEALED_PROBE_REPO_PATTERN.source).toBe(SEALED_PROBE_REPO_PATTERN.source); + expect(brokerProtocol.CANONICAL_ERROR_JSON).toBe(CANONICAL_ERROR_JSON); + }); + + it.each(SCHEMA_VECTORS)('agrees on schema validity: $name', ({ schema }) => { + const ts = validateSchema(schema); + const js = brokerProtocol.validateSchema(schema); + expect(js.valid).toBe(ts.valid); + if (ts.valid && js.valid) { + expect(js.schema).toEqual(ts.schema); + } + }); + + it.each(SCHEMA_VECTORS.filter((v) => validateSchema(v.schema).valid))( + 'agrees on cardinality and query-bit charge for valid schema: $name', + ({ schema }) => { + const tsValidation = validateSchema(schema); + const jsValidation = brokerProtocol.validateSchema(schema); + if (!tsValidation.valid || !jsValidation.valid) throw new Error('unreachable: filtered to valid schemas'); + + const tsCardinality = schemaCardinality(tsValidation.schema); + const jsCardinality = brokerProtocol.schemaCardinality(jsValidation.schema); + expect(jsCardinality).toBe(tsCardinality); + + const tsBits = queryBitsForSchema(tsValidation.schema); + const jsBits = brokerProtocol.queryBitsForSchema(jsValidation.schema); + expect(jsBits).toBe(tsBits); + }, + ); + + it.each( + VALID_SCHEMAS_FOR_VALUE_TESTS.flatMap(({ name, schema, values }) => + values.map((value, index) => ({ name: `${name}[${index}]`, schema, value })), + ), + )('agrees on value validation and canonicalization: $name', ({ schema, value }) => { + const tsValid = validateValueAgainstSchema(schema, value); + const jsValid = brokerProtocol.validateValueAgainstSchema(schema, value); + expect(jsValid).toBe(tsValid); + + if (tsValid && jsValid) { + expect(brokerProtocol.canonicalizeSchemaValue(schema, value)).toBe(canonicalizeSchemaValue(schema, value)); + } }); it.each(REQUEST_VECTORS)('agrees on request validity: $name', ({ request }) => { @@ -96,30 +312,33 @@ describe('sealed-probe protocol parity (TypeScript vs broker JavaScript)', () => const js = brokerProtocol.validateSealedProbeRequest(request); expect(js.valid).toBe(ts.valid); - expect(js.errors ?? []).toEqual(ts.valid ? [] : ts.errors); + if (!ts.valid && !js.valid) { + expect(js.errors).toEqual(ts.errors); + } }); - it.each(RESULT_VECTORS.map((raw, index) => ({ index, raw })))( - 'agrees on result parsing for vector $index', - ({ raw }) => { - expect(brokerProtocol.parseSealedProbeResult(raw, [...OUTCOMES])).toEqual( - parseSealedProbeResult(raw, OUTCOMES), - ); - }, - ); + it.each(RESULT_VECTORS)('agrees on probe output parsing/validation: $name', ({ schema, raw }) => { + const ts = parseAndValidateProbeOutput(raw, schema); + const js = brokerProtocol.parseAndValidateProbeOutput(raw, schema); + expect(js).toEqual(ts); + }); - it('agrees on canonical serialization', () => { - for (const outcome of [...OUTCOMES, RESERVED_ERROR_OUTCOME]) { - expect(brokerProtocol.canonicalizeSealedProbeResult(outcome)).toBe( - canonicalizeSealedProbeResult(outcome), - ); + it('agrees on strict JSON parsing', () => { + const vectors = ['{"a":1}', '{"a":1,"a":2}', '{"a":1} extra', 'not json', '', '"\\u0073"', '"\\uZZZZ"']; + for (const raw of vectors) { + expect(brokerProtocol.strictParseJson(raw)).toEqual(strictParseJson(raw)); } - expect(brokerProtocol.CANONICAL_ERROR_RESULT_JSON).toBe('{"result":"ERROR"}'); }); - it('agrees on the closed result schema', () => { - expect(brokerProtocol.buildSealedProbeResultSchema([...OUTCOMES])).toEqual( - buildSealedProbeResultSchema(OUTCOMES), - ); + it('agrees on ceilLog2BigInt across boundary values', () => { + for (const n of [0n, 1n, 2n, 3n, 4n, 5n, 8n, 9n, 1024n, 1025n, 2n ** 64n]) { + expect(brokerProtocol.ceilLog2BigInt(n)).toBe(ceilLog2BigInt(n)); + } + }); + + it('agrees on the canonical ok envelope wrapper', () => { + for (const canonical of ['true', '"ok"', '{"a":1}']) { + expect(brokerProtocol.canonicalOkJson(canonical)).toBe(canonicalOkJson(canonical)); + } }); }); diff --git a/src/sealed-probe/protocol.test.ts b/src/sealed-probe/protocol.test.ts index ed4b93cd1..5df439c2b 100644 --- a/src/sealed-probe/protocol.test.ts +++ b/src/sealed-probe/protocol.test.ts @@ -1,35 +1,59 @@ import { - OUTCOME_COUNT, - RESERVED_ERROR_OUTCOME, - MAX_OUTCOME_BYTES, - MAX_SCRIPT_BYTES, + CANONICAL_ERROR_JSON, + MAX_ARRAY_LENGTH, + MAX_ENUM_VALUES, + MAX_LITERAL_STRING_BYTES, + MAX_OBJECT_FIELDS, + MAX_PRIVATE_REPO_LENGTH, MAX_REQUEST_BYTES, MAX_RESULT_BYTES, - OUTCOME_PATTERN, + MAX_SCHEMA_BYTES, + MAX_SCHEMA_DEPTH, + MAX_SCHEMA_NODES, + MAX_SCRIPT_BYTES, + MAX_TUPLE_ITEMS, + MAX_UNION_VARIANTS, + PROBE_PROTOCOL_VERSION, + RESULT_STATUS_BIT_COST, SEALED_PROBE_REPO_PATTERN, - validateOutcome, - validateOutcomes, + TIMING_BUCKETS_MS, + TIMING_BUCKET_BITS, + canonicalOkJson, + canonicalizeSchemaValue, + ceilLog2BigInt, + parseAndValidateProbeOutput, + queryBitsForSchema, + schemaCardinality, + strictParseJson, + validateSchema, validateSealedProbeRequest, - buildSealedProbeResultSchema, - canonicalizeSealedProbeResult, - CANONICAL_ERROR_RESULT_JSON, - parseSealedProbeResult, - parseSealedProbeResultJson, - type SealedProbeOutcomes, + validateValueAgainstSchema, + type SealedProbeSchemaNode, } from './protocol'; -const OUTCOMES: SealedProbeOutcomes = ['success', 'timeout', 'blocked']; +describe('protocol constants', () => { + it('fixes the wire protocol version at 2', () => { + expect(PROBE_PROTOCOL_VERSION).toBe(2); + }); -describe('SEALED_PROBE_REPO_PATTERN', () => { - it.each([ - 'octo/repo', - 'octo-org/octo-repo', - 'my-org/my.repo-name_2', - 'a/b', - ])('accepts a valid owner/repo slug: %s', (slug) => { - expect(SEALED_PROBE_REPO_PATTERN.test(slug)).toBe(true); + it('has exactly six timing buckets and 3 timing bits', () => { + expect(TIMING_BUCKETS_MS).toEqual([10, 100, 1_000, 10_000, 60_000, 600_000]); + expect(TIMING_BUCKET_BITS).toBe(3); }); + it('charges 1 bit for the ok/error distinction', () => { + expect(RESULT_STATUS_BIT_COST).toBe(1); + }); +}); + +describe('SEALED_PROBE_REPO_PATTERN', () => { + it.each(['octo/repo', 'octo-org/octo-repo', 'my-org/my.repo-name_2', 'a/b'])( + 'accepts a valid owner/repo slug: %s', + (slug) => { + expect(SEALED_PROBE_REPO_PATTERN.test(slug)).toBe(true); + }, + ); + it.each([ ['a full URL', 'https://github.com/octo/repo'], ['a scheme-relative URL', '//github.com/octo/repo'], @@ -49,96 +73,456 @@ describe('SEALED_PROBE_REPO_PATTERN', () => { }); }); -describe('validateOutcome', () => { - it('accepts a normal short label', () => { - expect(validateOutcome('success')).toBeUndefined(); +describe('ceilLog2BigInt', () => { + it.each([ + [0n, 0], + [1n, 0], + [2n, 1], + [3n, 2], + [4n, 2], + [5n, 3], + [8n, 3], + [9n, 4], + [1024n, 10], + [1025n, 11], + ])('ceilLog2BigInt(%s) === %s', (n, expected) => { + expect(ceilLog2BigInt(n)).toBe(expected); + }); + + it('handles very large cardinalities without floating-point overflow', () => { + // 2^100, computed without ever going through a floating-point log. + const huge = 2n ** 100n; + expect(ceilLog2BigInt(huge)).toBe(100); + expect(ceilLog2BigInt(huge + 1n)).toBe(101); + }); +}); + +describe('validateSchema', () => { + it('accepts a const schema', () => { + const result = validateSchema({ type: 'const', value: 'ok' }); + expect(result).toEqual({ valid: true, schema: { type: 'const', value: 'ok' } }); + }); + + it('rejects a const schema with extra properties', () => { + expect(validateSchema({ type: 'const', value: 'ok', extra: 1 }).valid).toBe(false); + }); + + it('accepts a boolean schema and rejects extra properties', () => { + expect(validateSchema({ type: 'boolean' })).toEqual({ valid: true, schema: { type: 'boolean' } }); + expect(validateSchema({ type: 'boolean', extra: 1 }).valid).toBe(false); + }); + + it('accepts a unique enum schema of a single JSON type', () => { + const result = validateSchema({ type: 'enum', values: ['a', 'b', 'c'] }); + expect(result).toEqual({ valid: true, schema: { type: 'enum', values: ['a', 'b', 'c'] } }); + }); + + it('rejects an enum with duplicate values', () => { + expect(validateSchema({ type: 'enum', values: ['a', 'a'] }).valid).toBe(false); + }); + + it('rejects an enum mixing JSON types', () => { + expect(validateSchema({ type: 'enum', values: ['a', 1] }).valid).toBe(false); + }); + + it('rejects an empty enum', () => { + expect(validateSchema({ type: 'enum', values: [] }).valid).toBe(false); + }); + + it(`rejects an enum exceeding ${MAX_ENUM_VALUES} values`, () => { + const values = Array.from({ length: MAX_ENUM_VALUES + 1 }, (_, i) => i); + const result = validateSchema({ type: 'enum', values }); + expect(result.valid).toBe(false); + if (!result.valid) { + expect(result.errors.join(' ')).toMatch(/at most 4096 entries|4096 bytes/); + } + }); + + it('accepts a moderately sized enum comfortably under both the count and byte caps', () => { + const values = Array.from({ length: 200 }, (_, i) => i); + expect(validateSchema({ type: 'enum', values }).valid).toBe(true); + }); + + it('accepts a bounded integer schema and rejects maximum < minimum', () => { + expect(validateSchema({ type: 'integer', minimum: 0, maximum: 10 }).valid).toBe(true); + expect(validateSchema({ type: 'integer', minimum: 10, maximum: 0 }).valid).toBe(false); + }); + + it('rejects a non-integer or unsafe integer bound', () => { + expect(validateSchema({ type: 'integer', minimum: 0.5, maximum: 10 }).valid).toBe(false); + expect(validateSchema({ type: 'integer', minimum: 0, maximum: Number.MAX_SAFE_INTEGER + 1 }).valid).toBe(false); + }); + + it('accepts a required fixed object schema', () => { + const result = validateSchema({ + type: 'object', + fields: { ok: { type: 'boolean' }, count: { type: 'integer', minimum: 0, maximum: 3 } }, + }); + expect(result.valid).toBe(true); }); - it('rejects non-strings', () => { - expect(validateOutcome(42)).toMatch(/must be a string/); - expect(validateOutcome(null)).toMatch(/must be a string/); - expect(validateOutcome(undefined)).toMatch(/must be a string/); + it('rejects an object schema with zero fields or too many fields', () => { + expect(validateSchema({ type: 'object', fields: {} }).valid).toBe(false); + const tooMany: Record = {}; + for (let i = 0; i < MAX_OBJECT_FIELDS + 1; i++) tooMany[`f${i}`] = { type: 'boolean' }; + expect(validateSchema({ type: 'object', fields: tooMany }).valid).toBe(false); }); - it('rejects the empty string', () => { - expect(validateOutcome('')).toMatch(/must not be empty/); + it('rejects an object field name that is not a bounded ASCII identifier', () => { + expect(validateSchema({ type: 'object', fields: { 'bad name': { type: 'boolean' } } }).valid).toBe(false); + expect(validateSchema({ type: 'object', fields: { '1bad': { type: 'boolean' } } }).valid).toBe(false); }); - it('rejects the reserved ERROR sentinel', () => { - expect(validateOutcome('ERROR')).toMatch(/reserved/); + it('accepts a tuple schema and rejects an empty or oversized one', () => { + expect(validateSchema({ type: 'tuple', items: [{ type: 'boolean' }, { type: 'boolean' }] }).valid).toBe(true); + expect(validateSchema({ type: 'tuple', items: [] }).valid).toBe(false); + const tooMany = Array.from({ length: MAX_TUPLE_ITEMS + 1 }, () => ({ type: 'boolean' })); + expect(validateSchema({ type: 'tuple', items: tooMany }).valid).toBe(false); }); - it('rejects strings containing control characters', () => { - expect(validateOutcome('bad\nvalue')).toMatch(/control characters/); - expect(validateOutcome('bad\tvalue')).toMatch(/control characters/); - expect(validateOutcome('bad\x00value')).toMatch(/control characters/); + it('accepts a fixed-length array schema and rejects an out-of-range length', () => { + expect(validateSchema({ type: 'array', items: { type: 'boolean' }, length: 3 }).valid).toBe(true); + expect(validateSchema({ type: 'array', items: { type: 'boolean' }, length: 0 }).valid).toBe(true); + expect(validateSchema({ type: 'array', items: { type: 'boolean' }, length: -1 }).valid).toBe(false); + expect(validateSchema({ type: 'array', items: { type: 'boolean' }, length: MAX_ARRAY_LENGTH + 1 }).valid).toBe( + false, + ); }); - it('accepts an identifier at exactly the byte cap', () => { - const label = 'a'.repeat(MAX_OUTCOME_BYTES); - expect(validateOutcome(label)).toBeUndefined(); + it('accepts a tagged disjoint union schema and rejects an empty or oversized one', () => { + expect( + validateSchema({ + type: 'union', + variants: { a: { type: 'boolean' }, b: { type: 'integer', minimum: 0, maximum: 1 } }, + }).valid, + ).toBe(true); + expect(validateSchema({ type: 'union', variants: {} }).valid).toBe(false); + const tooMany: Record = {}; + for (let i = 0; i < MAX_UNION_VARIANTS + 1; i++) tooMany[`v${i}`] = { type: 'boolean' }; + expect(validateSchema({ type: 'union', variants: tooMany }).valid).toBe(false); }); - it('rejects a label exceeding the UTF-8 byte cap', () => { - const label = 'a'.repeat(MAX_OUTCOME_BYTES + 1); - expect(validateOutcome(label)).toMatch(/64 UTF-8 bytes/); + it('rejects a union tag that is not a bounded ASCII identifier', () => { + expect(validateSchema({ type: 'union', variants: { '1bad': { type: 'boolean' } } }).valid).toBe(false); }); - it('rejects values that cannot be transported as safe enum identifiers', () => { - expect(validateOutcome('has space')).toMatch(/ASCII identifier/); - expect(validateOutcome('💥')).toMatch(/ASCII identifier/); - expect(validateOutcome('1STARTS_WITH_DIGIT')).toMatch(/ASCII identifier/); - expect(validateOutcome('HAS.DOT')).toMatch(/ASCII identifier/); - expect(OUTCOME_PATTERN.test('YES_1')).toBe(true); + it('rejects an unknown schema node type', () => { + expect(validateSchema({ type: 'string' }).valid).toBe(false); + expect(validateSchema({}).valid).toBe(false); + expect(validateSchema(null).valid).toBe(false); + expect(validateSchema('not an object').valid).toBe(false); + expect(validateSchema([1, 2]).valid).toBe(false); + }); + + it(`rejects a schema exceeding maximum depth of ${MAX_SCHEMA_DEPTH}`, () => { + let deep: unknown = { type: 'boolean' }; + for (let i = 0; i <= MAX_SCHEMA_DEPTH; i++) { + deep = { type: 'array', items: deep, length: 1 }; + } + expect(validateSchema(deep).valid).toBe(false); + }); + + it('accepts a schema at exactly the maximum depth', () => { + let atLimit: unknown = { type: 'boolean' }; + for (let i = 0; i < MAX_SCHEMA_DEPTH; i++) { + atLimit = { type: 'array', items: atLimit, length: 1 }; + } + expect(validateSchema(atLimit).valid).toBe(true); + }); + + it(`rejects a schema exceeding ${MAX_SCHEMA_NODES} total nodes`, () => { + // A tuple of many boolean leaves quickly exceeds the node-count bound + // (root + N leaves) independent of depth. + const items = Array.from({ length: MAX_SCHEMA_NODES }, () => ({ type: 'boolean' })); + expect(validateSchema({ type: 'tuple', items }).valid).toBe(false); + }); + + it(`rejects a const literal string exceeding ${MAX_LITERAL_STRING_BYTES} bytes`, () => { + expect(validateSchema({ type: 'const', value: 'a'.repeat(MAX_LITERAL_STRING_BYTES) }).valid).toBe(true); + expect(validateSchema({ type: 'const', value: 'a'.repeat(MAX_LITERAL_STRING_BYTES + 1) }).valid).toBe(false); + }); + + it(`rejects a schema serialization exceeding ${MAX_SCHEMA_BYTES} bytes`, () => { + // An enum of many small distinct strings is a compact way to blow the + // byte cap without hitting node/field/tuple-count bounds first. + const values = Array.from({ length: 2000 }, (_, i) => `v${i}`); + expect(validateSchema({ type: 'enum', values }).valid).toBe(false); + }); + + it('rejects a schema that is not JSON-serializable', () => { + const cyclic: Record = { type: 'boolean' }; + cyclic.self = cyclic; + expect(validateSchema(cyclic).valid).toBe(false); + }); + + it('rejects undefined', () => { + expect(validateSchema(undefined).valid).toBe(false); }); }); -describe('validateOutcomes', () => { - it('accepts exactly three unique valid outcomes', () => { - expect(validateOutcomes(['a', 'b', 'c'])).toEqual([]); +describe('schemaCardinality and queryBitsForSchema', () => { + it('computes cardinality 1 for const (0 bits)', () => { + const schema: SealedProbeSchemaNode = { type: 'const', value: 'ok' }; + expect(schemaCardinality(schema)).toBe(1n); + expect(queryBitsForSchema(schema)).toBe(RESULT_STATUS_BIT_COST + 0 + TIMING_BUCKET_BITS); }); - it('rejects a non-array', () => { - expect(validateOutcomes('not-an-array').length).toBeGreaterThan(0); + it('computes cardinality 2 for boolean (1 bit)', () => { + const schema: SealedProbeSchemaNode = { type: 'boolean' }; + expect(schemaCardinality(schema)).toBe(2n); + expect(queryBitsForSchema(schema)).toBe(RESULT_STATUS_BIT_COST + 1 + TIMING_BUCKET_BITS); }); - it(`rejects fewer or more than ${OUTCOME_COUNT} entries`, () => { - expect(validateOutcomes(['a', 'b']).length).toBeGreaterThan(0); - expect(validateOutcomes(['a', 'b', 'c', 'd']).length).toBeGreaterThan(0); + it('computes cardinality equal to the enum length', () => { + const schema: SealedProbeSchemaNode = { type: 'enum', values: ['a', 'b', 'c', 'd'] }; + expect(schemaCardinality(schema)).toBe(4n); + expect(queryBitsForSchema(schema)).toBe(RESULT_STATUS_BIT_COST + 2 + TIMING_BUCKET_BITS); }); - it('rejects duplicate outcomes', () => { - const errors = validateOutcomes(['a', 'a', 'b']); - expect(errors).toContain('outcomes must be unique'); + it('computes cardinality as the inclusive integer range size', () => { + const schema: SealedProbeSchemaNode = { type: 'integer', minimum: 0, maximum: 255 }; + expect(schemaCardinality(schema)).toBe(256n); + expect(queryBitsForSchema(schema)).toBe(RESULT_STATUS_BIT_COST + 8 + TIMING_BUCKET_BITS); }); - it('rejects a reserved ERROR outcome anywhere in the tuple', () => { - const errors = validateOutcomes(['a', RESERVED_ERROR_OUTCOME, 'b']); - expect(errors.some((e) => e.includes('reserved'))).toBe(true); + it('multiplies cardinality across object fields', () => { + const schema: SealedProbeSchemaNode = { + type: 'object', + fields: [ + { name: 'a', schema: { type: 'boolean' } }, + { name: 'b', schema: { type: 'integer', minimum: 0, maximum: 3 } }, + ], + }; + // 2 * 4 = 8 + expect(schemaCardinality(schema)).toBe(8n); }); - it('aggregates multiple per-item errors', () => { - const errors = validateOutcomes(['', 'ok', 42]); - expect(errors.some((e) => e.includes('outcomes[0]'))).toBe(true); - expect(errors.some((e) => e.includes('outcomes[2]'))).toBe(true); + it('multiplies cardinality across tuple items', () => { + const schema: SealedProbeSchemaNode = { + type: 'tuple', + items: [{ type: 'boolean' }, { type: 'boolean' }, { type: 'boolean' }], + }; + expect(schemaCardinality(schema)).toBe(8n); + }); + + it('raises item cardinality to the fixed array length', () => { + const schema: SealedProbeSchemaNode = { type: 'array', items: { type: 'boolean' }, length: 10 }; + expect(schemaCardinality(schema)).toBe(1024n); + }); + + it('handles a zero-length array as cardinality 1', () => { + const schema: SealedProbeSchemaNode = { type: 'array', items: { type: 'boolean' }, length: 0 }; + expect(schemaCardinality(schema)).toBe(1n); + }); + + it('sums cardinality across disjoint union variants', () => { + const schema: SealedProbeSchemaNode = { + type: 'union', + variants: [ + { tag: 'a', schema: { type: 'boolean' } }, + { tag: 'b', schema: { type: 'integer', minimum: 0, maximum: 9 } }, + ], + }; + // 2 + 10 = 12 + expect(schemaCardinality(schema)).toBe(12n); + }); + + it('never overflows even for a schema near the configured bounds', () => { + // Cardinality far beyond Number.MAX_SAFE_INTEGER — must stay exact as a BigInt. + const schema: SealedProbeSchemaNode = { type: 'array', items: { type: 'integer', minimum: 0, maximum: 65535 }, length: 8 }; + const expected = 65536n ** 8n; + expect(schemaCardinality(schema)).toBe(expected); + expect(queryBitsForSchema(schema)).toBe( + RESULT_STATUS_BIT_COST + ceilLog2BigInt(expected) + TIMING_BUCKET_BITS, + ); + }); + + it('charges exactly 4 bits for the cheapest possible schema (const)', () => { + // 1 (status) + 0 (const) + 3 (timing) = 4 — the floor for every invocation. + expect(queryBitsForSchema({ type: 'const', value: 1 })).toBe(4); + }); +}); + +describe('validateValueAgainstSchema', () => { + it('validates const by exact value equality', () => { + expect(validateValueAgainstSchema({ type: 'const', value: 'ok' }, 'ok')).toBe(true); + expect(validateValueAgainstSchema({ type: 'const', value: 'ok' }, 'not-ok')).toBe(false); + expect(validateValueAgainstSchema({ type: 'const', value: null }, null)).toBe(true); + expect(validateValueAgainstSchema({ type: 'const', value: 1 }, 1)).toBe(true); + expect(validateValueAgainstSchema({ type: 'const', value: 1 }, '1')).toBe(false); + }); + + it('validates boolean by strict type', () => { + const schema: SealedProbeSchemaNode = { type: 'boolean' }; + expect(validateValueAgainstSchema(schema, true)).toBe(true); + expect(validateValueAgainstSchema(schema, false)).toBe(true); + expect(validateValueAgainstSchema(schema, 1)).toBe(false); + expect(validateValueAgainstSchema(schema, 'true')).toBe(false); + }); + + it('validates enum membership only, rejecting unknown members', () => { + const schema: SealedProbeSchemaNode = { type: 'enum', values: ['a', 'b'] }; + expect(validateValueAgainstSchema(schema, 'a')).toBe(true); + expect(validateValueAgainstSchema(schema, 'c')).toBe(false); + }); + + it('validates integer range and rejects non-integers', () => { + const schema: SealedProbeSchemaNode = { type: 'integer', minimum: 0, maximum: 10 }; + expect(validateValueAgainstSchema(schema, 5)).toBe(true); + expect(validateValueAgainstSchema(schema, 0)).toBe(true); + expect(validateValueAgainstSchema(schema, 10)).toBe(true); + expect(validateValueAgainstSchema(schema, 11)).toBe(false); + expect(validateValueAgainstSchema(schema, -1)).toBe(false); + expect(validateValueAgainstSchema(schema, 5.5)).toBe(false); + }); + + it('validates fixed object shape: no missing, no extra fields', () => { + const schema: SealedProbeSchemaNode = { + type: 'object', + fields: [{ name: 'ok', schema: { type: 'boolean' } }], + }; + expect(validateValueAgainstSchema(schema, { ok: true })).toBe(true); + expect(validateValueAgainstSchema(schema, {})).toBe(false); + expect(validateValueAgainstSchema(schema, { ok: true, extra: 1 })).toBe(false); + expect(validateValueAgainstSchema(schema, { ok: 'not-a-bool' })).toBe(false); + expect(validateValueAgainstSchema(schema, null)).toBe(false); + expect(validateValueAgainstSchema(schema, [true])).toBe(false); + }); + + it('validates fixed-length tuples exactly', () => { + const schema: SealedProbeSchemaNode = { type: 'tuple', items: [{ type: 'boolean' }, { type: 'boolean' }] }; + expect(validateValueAgainstSchema(schema, [true, false])).toBe(true); + expect(validateValueAgainstSchema(schema, [true])).toBe(false); + expect(validateValueAgainstSchema(schema, [true, false, true])).toBe(false); + }); + + it('validates fixed-length arrays exactly', () => { + const schema: SealedProbeSchemaNode = { type: 'array', items: { type: 'boolean' }, length: 2 }; + expect(validateValueAgainstSchema(schema, [true, false])).toBe(true); + expect(validateValueAgainstSchema(schema, [true])).toBe(false); + expect(validateValueAgainstSchema(schema, [true, false, true])).toBe(false); + }); + + it('validates a tagged union: exact tag/value shape, no untagged escape', () => { + const schema: SealedProbeSchemaNode = { + type: 'union', + variants: [ + { tag: 'a', schema: { type: 'boolean' } }, + { tag: 'b', schema: { type: 'integer', minimum: 0, maximum: 9 } }, + ], + }; + expect(validateValueAgainstSchema(schema, { tag: 'a', value: true })).toBe(true); + expect(validateValueAgainstSchema(schema, { tag: 'b', value: 5 })).toBe(true); + expect(validateValueAgainstSchema(schema, { tag: 'b', value: true })).toBe(false); + expect(validateValueAgainstSchema(schema, { tag: 'c', value: true })).toBe(false); + expect(validateValueAgainstSchema(schema, { tag: 'a', value: true, extra: 1 })).toBe(false); + expect(validateValueAgainstSchema(schema, true)).toBe(false); }); }); +describe('canonicalizeSchemaValue', () => { + it('re-serializes const to its declared literal, ignoring the input value', () => { + expect(canonicalizeSchemaValue({ type: 'const', value: 'ok' }, 'ok')).toBe('"ok"'); + }); + + it('re-serializes boolean/enum/integer values directly', () => { + expect(canonicalizeSchemaValue({ type: 'boolean' }, true)).toBe('true'); + expect(canonicalizeSchemaValue({ type: 'enum', values: ['a', 'b'] }, 'b')).toBe('"b"'); + expect(canonicalizeSchemaValue({ type: 'integer', minimum: 0, maximum: 10 }, 7)).toBe('7'); + }); + + it('re-serializes an object in declared field order regardless of input key order', () => { + const schema: SealedProbeSchemaNode = { + type: 'object', + fields: [ + { name: 'b', schema: { type: 'boolean' } }, + { name: 'a', schema: { type: 'boolean' } }, + ], + }; + expect(canonicalizeSchemaValue(schema, { a: false, b: true })).toBe('{"b":true,"a":false}'); + }); + + it('re-serializes tuples and arrays positionally', () => { + const tuple: SealedProbeSchemaNode = { type: 'tuple', items: [{ type: 'boolean' }, { type: 'boolean' }] }; + expect(canonicalizeSchemaValue(tuple, [true, false])).toBe('[true,false]'); + + const array: SealedProbeSchemaNode = { type: 'array', items: { type: 'boolean' }, length: 2 }; + expect(canonicalizeSchemaValue(array, [false, true])).toBe('[false,true]'); + }); + + it('re-serializes a tagged union as {"tag":...,"value":...}', () => { + const schema: SealedProbeSchemaNode = { + type: 'union', + variants: [{ tag: 'a', schema: { type: 'boolean' } }], + }; + expect(canonicalizeSchemaValue(schema, { tag: 'a', value: true })).toBe('{"tag":"a","value":true}'); + }); +}); + +describe('strictParseJson', () => { + it('parses valid JSON values', () => { + expect(strictParseJson('{"a":1}')).toEqual({ value: { a: 1 } }); + expect(strictParseJson('[1,2,3]')).toEqual({ value: [1, 2, 3] }); + expect(strictParseJson('true')).toEqual({ value: true }); + expect(strictParseJson('null')).toEqual({ value: null }); + expect(strictParseJson(' "spaced" ')).toEqual({ value: 'spaced' }); + }); + + it('rejects duplicate object keys instead of silently keeping the last', () => { + expect(strictParseJson('{"a":1,"a":2}')).toBeUndefined(); + }); + + it('rejects trailing data after the value', () => { + expect(strictParseJson('{"a":1} extra')).toBeUndefined(); + expect(strictParseJson('{"a":1}{}')).toBeUndefined(); + }); + + it('rejects malformed JSON', () => { + expect(strictParseJson('not json')).toBeUndefined(); + expect(strictParseJson("{'a':1}")).toBeUndefined(); + expect(strictParseJson('{"a":1')).toBeUndefined(); + expect(strictParseJson('')).toBeUndefined(); + }); + + it('rejects raw control characters embedded in a string', () => { + expect(strictParseJson('{"a":"line\nbreak"}')).toBeUndefined(); + }); + + it.each([ + ['{"a":"s\\"uccess"}', { a: 's"uccess' }], + ['{"a":"s\\\\uccess"}', { a: 's\\uccess' }], + ['{"a":"\\u0073"}', { a: 's' }], + ])('parses standard JSON escapes: %s', (raw, expected) => { + expect(strictParseJson(raw)).toEqual({ value: expected }); + }); + + it.each(['{"a":"\\x41"}', '{"a":"\\uZZZZ"}', '{"a":"trailing\\\\'])( + 'rejects invalid escapes: %s', + (raw) => { + expect(strictParseJson(raw)).toBeUndefined(); + }, + ); +}); + describe('validateSealedProbeRequest', () => { const validRequest = { privateRepo: 'octo/repo', - outcomes: OUTCOMES, + schema: { type: 'boolean' }, script: 'print("hello")', }; it('accepts a well-formed request', () => { - expect(validateSealedProbeRequest(validRequest)).toEqual({ valid: true }); + const result = validateSealedProbeRequest(validRequest); + expect(result).toEqual({ + valid: true, + request: { privateRepo: 'octo/repo', schema: { type: 'boolean' }, script: 'print("hello")' }, + }); }); it('rejects non-object requests', () => { - expect(validateSealedProbeRequest(null)).toEqual({ valid: false, errors: expect.any(Array) }); - expect(validateSealedProbeRequest('string')).toEqual({ valid: false, errors: expect.any(Array) }); - expect(validateSealedProbeRequest([1, 2, 3])).toEqual({ valid: false, errors: expect.any(Array) }); + expect(validateSealedProbeRequest(null).valid).toBe(false); + expect(validateSealedProbeRequest('string').valid).toBe(false); + expect(validateSealedProbeRequest([1, 2, 3]).valid).toBe(false); }); it('rejects a privateRepo that looks like a URL', () => { @@ -146,16 +530,28 @@ describe('validateSealedProbeRequest', () => { expect(result.valid).toBe(false); }); + it(`rejects a privateRepo exceeding ${MAX_PRIVATE_REPO_LENGTH} characters`, () => { + const long = `octo/${'r'.repeat(MAX_PRIVATE_REPO_LENGTH)}`; + const result = validateSealedProbeRequest({ ...validRequest, privateRepo: long }); + expect(result.valid).toBe(false); + }); + it('rejects a missing privateRepo', () => { const rest: Record = { ...validRequest }; delete rest.privateRepo; - const result = validateSealedProbeRequest(rest); + expect(validateSealedProbeRequest(rest).valid).toBe(false); + }); + + it('rejects an invalid schema', () => { + const result = validateSealedProbeRequest({ ...validRequest, schema: { type: 'nope' } }); expect(result.valid).toBe(false); + if (!result.valid) { + expect(result.errors.some((e) => e.startsWith('schema:'))).toBe(true); + } }); it('rejects an empty script', () => { - const result = validateSealedProbeRequest({ ...validRequest, script: '' }); - expect(result.valid).toBe(false); + expect(validateSealedProbeRequest({ ...validRequest, script: '' }).valid).toBe(false); }); it('rejects a script exceeding the size cap', () => { @@ -167,30 +563,23 @@ describe('validateSealedProbeRequest', () => { }); it('accepts a script at exactly the size cap', () => { - const result = validateSealedProbeRequest({ ...validRequest, script: 'x'.repeat(MAX_SCRIPT_BYTES) }); - expect(result.valid).toBe(true); + expect(validateSealedProbeRequest({ ...validRequest, script: 'x'.repeat(MAX_SCRIPT_BYTES) }).valid).toBe(true); }); - it('rejects a request whose overall serialized size exceeds the request cap even though every declared field is within its own cap', () => { - // Attach an oversized extra property so the whole-request size guard - // (independent of the per-field script/outcome/privateRepo caps) triggers. + it('rejects a request whose overall serialized size exceeds the request cap', () => { const result = validateSealedProbeRequest({ ...validRequest, extraPadding: 'x'.repeat(MAX_REQUEST_BYTES), }); - expect(result.valid).toBe(false); if (!result.valid) { expect(result.errors.some((e) => e.includes('request must be at most'))).toBe(true); + expect(result.errors.some((e) => e.includes('extraPadding is not supported'))).toBe(true); } }); it('rejects unsupported request fields before launch', () => { - const result = validateSealedProbeRequest({ - ...validRequest, - runtime: 'docker', - }); - + const result = validateSealedProbeRequest({ ...validRequest, runtime: 'docker' }); expect(result).toEqual({ valid: false, errors: expect.arrayContaining(['request.runtime is not supported']), @@ -201,19 +590,16 @@ describe('validateSealedProbeRequest', () => { const cyclic: Record = { ...validRequest }; cyclic.self = cyclic; const result = validateSealedProbeRequest(cyclic); - expect(result).toEqual({ - valid: false, - errors: expect.arrayContaining(['request.self is not supported', 'request must be JSON-serializable']), - }); - }); - - it('rejects invalid outcomes on the request', () => { - const result = validateSealedProbeRequest({ ...validRequest, outcomes: ['a', 'a', 'b'] }); expect(result.valid).toBe(false); + if (!result.valid) { + expect(result.errors).toEqual( + expect.arrayContaining(['request.self is not supported', 'request must be JSON-serializable']), + ); + } }); it('aggregates errors across multiple invalid fields', () => { - const result = validateSealedProbeRequest({ privateRepo: '', outcomes: ['a'], script: '' }); + const result = validateSealedProbeRequest({ privateRepo: '', schema: { type: 'nope' }, script: '' }); expect(result.valid).toBe(false); if (!result.valid) { expect(result.errors.length).toBeGreaterThan(1); @@ -221,140 +607,65 @@ describe('validateSealedProbeRequest', () => { }); }); -describe('buildSealedProbeResultSchema', () => { - it('builds the exact closed-schema representation', () => { - expect(buildSealedProbeResultSchema(OUTCOMES)).toEqual({ - type: 'object', - additionalProperties: false, - required: ['result'], - properties: { - result: { - type: 'string', - enum: ['success', 'timeout', 'blocked', 'ERROR'], - }, - }, - }); +describe('canonical envelopes', () => { + it('exposes the exact canonical error JSON', () => { + expect(CANONICAL_ERROR_JSON).toBe('{"status":"error"}'); }); - it('includes the reserved ERROR sentinel as the fourth enum value', () => { - const schema = buildSealedProbeResultSchema(OUTCOMES); - expect(schema.properties.result.enum).toEqual([ - 'success', - 'timeout', - 'blocked', - RESERVED_ERROR_OUTCOME, - ]); + it('wraps an already-canonicalized result value into the ok envelope', () => { + expect(canonicalOkJson('true')).toBe('{"status":"ok","result":true}'); + expect(canonicalOkJson('"ok"')).toBe('{"status":"ok","result":"ok"}'); }); }); -describe('canonicalizeSealedProbeResult', () => { - it('produces the exact canonical JSON shape', () => { - expect(canonicalizeSealedProbeResult('success')).toBe('{"result":"success"}'); - }); - - it('exposes a precomputed canonical error result constant', () => { - expect(CANONICAL_ERROR_RESULT_JSON).toBe('{"result":"ERROR"}'); - }); -}); - -describe('parseSealedProbeResult', () => { - it('accepts an exact match to a declared outcome', () => { - expect(parseSealedProbeResult('{"result":"success"}', OUTCOMES)).toEqual({ result: 'success' }); - }); - - it('tolerates surrounding whitespace', () => { - expect(parseSealedProbeResult(' \n{ "result" : "success" }\t\n', OUTCOMES)).toEqual({ result: 'success' }); - }); - - it('maps malformed JSON to the reserved ERROR result', () => { - expect(parseSealedProbeResult('not json at all', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('maps duplicate "result" keys to the reserved ERROR result', () => { - expect(parseSealedProbeResult('{"result":"success","result":"timeout"}', OUTCOMES)) - .toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); +describe('parseAndValidateProbeOutput', () => { + const schema: SealedProbeSchemaNode = { type: 'enum', values: ['success', 'timeout', 'blocked'] }; - it('maps trailing data after the object to the reserved ERROR result', () => { - expect(parseSealedProbeResult('{"result":"success"} extra', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - expect(parseSealedProbeResult('{"result":"success"}{}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); + it('accepts and canonicalizes a valid result', () => { + expect(parseAndValidateProbeOutput('{"result":"success"}', { type: 'object', fields: [{ name: 'result', schema }] })) + .toEqual({ ok: true, canonical: '{"result":"success"}' }); }); - it('maps extra fields to the reserved ERROR result', () => { - expect(parseSealedProbeResult('{"result":"success","extra":1}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); + it('accepts a bare schema value directly (no envelope object required by the schema itself)', () => { + expect(parseAndValidateProbeOutput('"success"', schema)).toEqual({ ok: true, canonical: '"success"' }); }); - it('maps a non-enum result value to the reserved ERROR result', () => { - expect(parseSealedProbeResult('{"result":"not-a-declared-outcome"}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); + it('rejects malformed JSON', () => { + expect(parseAndValidateProbeOutput('not json', schema)).toEqual({ ok: false }); }); - it('maps a literal ERROR value to the (already reserved) ERROR result', () => { - expect(parseSealedProbeResult('{"result":"ERROR"}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); + it('rejects a value outside the enum', () => { + expect(parseAndValidateProbeOutput('"not-a-declared-outcome"', schema)).toEqual({ ok: false }); }); - it('maps a non-string result value to the reserved ERROR result', () => { - expect(parseSealedProbeResult('{"result":42}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - expect(parseSealedProbeResult('{"result":null}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - expect(parseSealedProbeResult('{"result":true}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('maps an empty string to the reserved ERROR result', () => { - expect(parseSealedProbeResult('', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('maps an oversized result to the reserved ERROR result', () => { - const oversized = `{"result":"${'x'.repeat(MAX_RESULT_BYTES)}"}`; - expect(parseSealedProbeResult(oversized, OUTCOMES)) - .toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('maps an array instead of an object to the reserved ERROR result', () => { - expect(parseSealedProbeResult('["success"]', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('rejects single-quoted strings as malformed JSON', () => { - expect(parseSealedProbeResult("{'result':'success'}", OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('rejects unterminated strings', () => { - expect(parseSealedProbeResult('{"result":"success', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it('rejects raw control characters embedded in the string', () => { - expect(parseSealedProbeResult('{"result":"line\nbreak"}', OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); - }); - - it.each([ - '{"result":"s\\"uccess"}', - '{"result":"s\\\\uccess"}', - '{"result":"s\\/uccess"}', - '{"result":"s\\buccess"}', - '{"result":"s\\fuccess"}', - '{"result":"s\\nuccess"}', - '{"result":"s\\ruccess"}', - '{"result":"s\\tuccess"}', - '{"result":"\\u0073uccess"}', - ])('parses standard JSON escapes before enforcing the outcome enum: %s', (raw) => { - const expected = raw.includes('\\u0073') ? 'success' : RESERVED_ERROR_OUTCOME; - expect(parseSealedProbeResult(raw, OUTCOMES)).toEqual({ result: expected }); + it(`rejects output exceeding ${MAX_RESULT_BYTES} bytes`, () => { + const oversized = `"${'x'.repeat(MAX_RESULT_BYTES)}"`; + expect(parseAndValidateProbeOutput(oversized, { type: 'enum', values: [oversized.slice(1, -1)] })).toEqual({ + ok: false, + }); }); - it.each([ - '{"result":"\\x73uccess"}', - '{"result":"\\uZZZZ"}', - '{"result":"trailing\\\\', - ])('rejects invalid JSON string escapes: %s', (raw) => { - expect(parseSealedProbeResult(raw, OUTCOMES)).toEqual({ result: RESERVED_ERROR_OUTCOME }); + it('rejects duplicate-key JSON', () => { + expect( + parseAndValidateProbeOutput('{"a":1,"a":2}', { type: 'object', fields: [{ name: 'a', schema: { type: 'boolean' } }] }), + ).toEqual({ ok: false }); }); -}); -describe('parseSealedProbeResultJson', () => { - it('returns canonical JSON for a valid result', () => { - expect(parseSealedProbeResultJson('{"result":"success"}', OUTCOMES)).toBe('{"result":"success"}'); + it('rejects an empty string', () => { + expect(parseAndValidateProbeOutput('', schema)).toEqual({ ok: false }); }); - it('returns the canonical error JSON for any invalid input', () => { - expect(parseSealedProbeResultJson('garbage', OUTCOMES)).toBe(CANONICAL_ERROR_RESULT_JSON); - expect(parseSealedProbeResultJson('{"result":"success","result":"timeout"}', OUTCOMES)).toBe(CANONICAL_ERROR_RESULT_JSON); + it('normalizes canonical output regardless of source whitespace/key order', () => { + const objSchema: SealedProbeSchemaNode = { + type: 'object', + fields: [ + { name: 'a', schema: { type: 'boolean' } }, + { name: 'b', schema: { type: 'boolean' } }, + ], + }; + expect(parseAndValidateProbeOutput('{ "b" : true , "a" : false }', objSchema)).toEqual({ + ok: true, + canonical: '{"a":false,"b":true}', + }); }); }); diff --git a/src/sealed-probe/protocol.ts b/src/sealed-probe/protocol.ts index 3f9ebf8fe..2fc8026a8 100644 --- a/src/sealed-probe/protocol.ts +++ b/src/sealed-probe/protocol.ts @@ -1,243 +1,584 @@ /** - * Sealed-probe request/result protocol: validation and canonicalization. + * Sealed-probe request/result protocol v2: a deliberately finite, + * agent-authored response-schema algebra plus request/result validation and + * canonicalization. * * This module defines the wire protocol for sealed probes independently of - * any broker or sandbox runtime (neither of which exist yet — this is the - * configuration/protocol foundation only, see docs/awf-config-spec.md §14). + * any broker or sandbox runtime. * * Protocol summary: - * - A **request** asks the (future) broker to run a script against a - * private repository and report which of exactly three declared - * `outcomes` occurred. - * - A **result** is the script's report, expressed as the closed JSON - * object `{"result": ""}`. - * - `"ERROR"` is a reserved sentinel: it can never be one of the three - * declared outcomes, and every parsing/validation failure canonicalizes - * to `{"result":"ERROR"}` rather than throwing or passing through - * untrusted data. + * - A **request** asks the trusted broker to run an agent-authored Python + * script against a private repository and report a value conforming to + * an agent-authored, but AWF-bounded, finite response **schema**. + * - The schema is drawn from a small, closed algebra (`const`, `boolean`, + * unique `enum`, bounded `integer`, fixed `object`, `tuple`, fixed-length + * `array`, and tagged `union`) — general JSON Schema is not accepted. + * Every construct has a computable, finite cardinality (number of + * distinguishable values), calculated with `BigInt` so it can never + * silently overflow. + * - A **result** is always exactly one of two canonical envelopes: + * `{"status":"ok","result":}` or `{"status":"error"}`. The error + * state lives *outside* the declared schema — the schema only describes + * the shape of a successful `result` value — so, unlike protocol v1, + * schemas never need a reserved sentinel member. + * - Every accepted invocation reserves a fixed number of information-budget + * bits: one for the ok/error distinction, `ceil(log2(cardinality))` for + * the success payload, and {@link TIMING_BUCKET_BITS} for the observable + * response-timing bucket (see `docs/awf-config-spec.md` §14). Budget + * accounting itself lives in the broker's per-repository ledger; this + * module only computes the charge for a given schema. * - * Result parsing deliberately does NOT use a general-purpose JSON Schema - * validator. The accepted result shape is a single fixed, closed schema (one - * required key, string enum value), so it is parsed with a small - * hand-written, linear-time (no backtracking) grammar below. This avoids - * pulling arbitrary/attacker-influenced JSON Schema documents into an - * execution path. + * Schema parsing and result parsing deliberately do NOT use a general-purpose + * JSON Schema validator, nor `JSON.parse`. Both use small, hand-written, + * linear-time (no backtracking) recursive-descent parsers below, bounded by + * fixed depth/node/size limits, so nothing attacker-influenced (schema text + * or probe output) can grow an unbounded parse tree, and duplicate object + * keys — which `JSON.parse` would silently collapse — are rejected outright. + * + * `containers/sealed-probe/broker/protocol.js` is a deliberate, + * behaviour-identical mirror of this module for the broker's container + * image, which cannot import AWF's TypeScript sources. Keep both in sync; + * `src/sealed-probe/protocol-parity.test.ts` runs shared vectors through + * both and fails the moment they disagree. */ -/** Number of outcomes a sealed-probe request must declare. */ -export const OUTCOME_COUNT = 3; +/** Wire protocol version. Only this exact value is accepted. */ +export const PROBE_PROTOCOL_VERSION = 2; + +/** Maximum size, in UTF-8 bytes, of a serialized agent-authored schema. */ +export const MAX_SCHEMA_BYTES = 4096; + +/** Maximum nesting depth of a schema (object/tuple/array/union children). */ +export const MAX_SCHEMA_DEPTH = 6; + +/** Maximum total number of schema nodes (bounds parse/cardinality work). */ +export const MAX_SCHEMA_NODES = 64; + +/** Maximum number of members in one `enum` schema. */ +export const MAX_ENUM_VALUES = 4096; + +/** Maximum size, in UTF-8 bytes, of one `const`/`enum` string literal. */ +export const MAX_LITERAL_STRING_BYTES = 64; + +/** Maximum number of fields in one `object` schema. */ +export const MAX_OBJECT_FIELDS = 16; -/** Reserved outcome value. Cannot be used as a declared outcome. */ -export const RESERVED_ERROR_OUTCOME = 'ERROR'; +/** Maximum number of items in one `tuple` schema. */ +export const MAX_TUPLE_ITEMS = 16; -/** Maximum size, in UTF-8 bytes, of a single outcome string. */ -export const MAX_OUTCOME_BYTES = 64; +/** Maximum fixed length of one `array` schema. */ +export const MAX_ARRAY_LENGTH = 64; -/** Safe enum identifier accepted by every transport and canonical serializer. */ -export const OUTCOME_PATTERN = /^[A-Za-z][A-Za-z0-9_-]{0,63}$/; +/** Maximum number of variants in one `union` schema. */ +export const MAX_UNION_VARIANTS = 16; /** Maximum size, in UTF-8 bytes, of a probe script. */ export const MAX_SCRIPT_BYTES = 64 * 1024; -/** Maximum size, in UTF-8 bytes, of a serialized sealed-probe request. */ -export const MAX_REQUEST_BYTES = 256 * 1024; +/** + * Maximum size, in UTF-8 bytes, of the assembled `{privateRepo, schema, + * script}` request object considered as a whole (sanity bound; the schema + * and script are already independently bounded above). + */ +export const MAX_REQUEST_BYTES = MAX_SCRIPT_BYTES + MAX_SCHEMA_BYTES + 1024; -/** Maximum size, in UTF-8 bytes, of the probe result file. */ -export const MAX_RESULT_BYTES = 1024; +/** Maximum size, in UTF-8 bytes, of the probe's raw output file. */ +export const MAX_RESULT_BYTES = 8 * 1024; /** Maximum length of a `privateRepo` "owner/repo" slug. */ export const MAX_PRIVATE_REPO_LENGTH = 140; +/** Number of observable response-timing buckets (see `docs/awf-config-spec.md` §14). */ +export const TIMING_BUCKETS_MS: readonly number[] = [10, 100, 1_000, 10_000, 60_000, 600_000]; + +/** + * Bits reserved for the timing side channel: `ceil(log2(TIMING_BUCKETS_MS.length))`. + * Fixed at 3 for the current six-bucket design; recomputed defensively below + * so the constant can never silently drift out of sync with the bucket list. + */ +export const TIMING_BUCKET_BITS = ceilLog2(TIMING_BUCKETS_MS.length); + +/** Bits reserved for the canonical ok/error distinction. */ +export const RESULT_STATUS_BIT_COST = 1; + /** * Matches a bare `owner/repo` slug only: no scheme/host (`://`), no path * traversal (`..`), no query string or fragment (`?`/`#`), no wildcard * (`*`), and no extra path segments (only one `/` is allowed). * - * Keep in sync with `sealedProbes.privateRepos.items.pattern` in + * Keep in sync with `sealedProbes.privateRepos.items` in * `docs/awf-config.schema.json` (JSON Schema cannot share a regex constant * with TypeScript source). */ export const SEALED_PROBE_REPO_PATTERN = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})\/(?!\.\.?$)(?!.*\.\.)[A-Za-z0-9._-]{1,100}$/; -/** A sealed-probe request's declared outcomes: exactly three distinct strings. */ -export type SealedProbeOutcomes = readonly [string, string, string]; +/** Bounded ASCII identifier accepted for object field names and union tags. */ +const IDENTIFIER_PATTERN = /^[A-Za-z][A-Za-z0-9_]{0,63}$/; -/** A sealed-probe execution request. */ -export interface SealedProbeRequest { - /** Private repository (`owner/repo`) the probe script runs against. */ - privateRepo: string; - /** Exactly three distinct, non-reserved outcome labels the script may report. */ - outcomes: SealedProbeOutcomes; - /** The probe script source. */ - script: string; +function utf8ByteLength(value: string): number { + return Buffer.byteLength(value, 'utf8'); } -/** A sealed-probe result: always exactly one of the declared outcomes, or the reserved `"ERROR"` sentinel. */ -export interface SealedProbeResult { - result: string; +function hasControlCharacters(value: string): boolean { + for (let i = 0; i < value.length; i++) { + const code = value.charCodeAt(i); + if (code < 0x20 || code === 0x7f) return true; + } + return false; } -export type SealedProbeValidation = - | { valid: true } - | { valid: false; errors: string[] }; +/** Non-negative integer ceiling of `log2(n)`, for plain (non-BigInt) `n >= 1`. */ +function ceilLog2(n: number): number { + return ceilLog2BigInt(BigInt(n)); +} + +// ── Finite schema algebra ──────────────────────────────────────────────────── + +/** A JSON scalar literal usable in `const`/`enum` schema nodes. */ +export type JsonLiteral = string | number | boolean | null; + +export interface ConstSchemaNode { + readonly type: 'const'; + readonly value: JsonLiteral; +} +export interface BooleanSchemaNode { + readonly type: 'boolean'; +} +export interface EnumSchemaNode { + readonly type: 'enum'; + readonly values: readonly JsonLiteral[]; +} +export interface IntegerSchemaNode { + readonly type: 'integer'; + readonly minimum: number; + readonly maximum: number; +} +export interface ObjectSchemaNode { + readonly type: 'object'; + readonly fields: readonly { name: string; schema: SealedProbeSchemaNode }[]; +} +export interface TupleSchemaNode { + readonly type: 'tuple'; + readonly items: readonly SealedProbeSchemaNode[]; +} +export interface ArraySchemaNode { + readonly type: 'array'; + readonly items: SealedProbeSchemaNode; + readonly length: number; +} +export interface UnionSchemaNode { + readonly type: 'union'; + readonly variants: readonly { tag: string; schema: SealedProbeSchemaNode }[]; +} /** - * The exact closed-schema representation of a valid result object for a - * given set of declared outcomes. + * A validated, finite response schema. * - * This is a plain data representation for documentation/introspection use - * by future broker code — it is never executed against a JSON Schema - * engine here. Actual enforcement is done by {@link parseSealedProbeResult}. + * This is the *parsed* representation — every instance has already passed + * {@link validateSchema}'s bounds (depth, node count, enum/field/item counts, + * literal sizes). Cardinality, value validation, and canonical serialization + * below all assume that. */ -export interface SealedProbeResultSchema { - readonly type: 'object'; - readonly additionalProperties: false; - readonly required: readonly ['result']; - readonly properties: { - readonly result: { - readonly type: 'string'; - readonly enum: readonly string[]; - }; - }; -} - -/** Builds the closed-schema representation of a valid result for the given outcomes. */ -export function buildSealedProbeResultSchema(outcomes: SealedProbeOutcomes): SealedProbeResultSchema { - return { - type: 'object', - additionalProperties: false, - required: ['result'], - properties: { - result: { - type: 'string', - enum: [...outcomes, RESERVED_ERROR_OUTCOME], - }, - }, - }; +export type SealedProbeSchemaNode = + | ConstSchemaNode + | BooleanSchemaNode + | EnumSchemaNode + | IntegerSchemaNode + | ObjectSchemaNode + | TupleSchemaNode + | ArraySchemaNode + | UnionSchemaNode; + +export type SealedProbeSchemaValidation = + | { valid: true; schema: SealedProbeSchemaNode } + | { valid: false; errors: string[] }; + +function isValidLiteral(value: unknown): value is JsonLiteral { + if (value === null) return true; + if (typeof value === 'boolean') return true; + if (typeof value === 'number') return Number.isInteger(value) && Number.isSafeInteger(value); + if (typeof value === 'string') { + return !hasControlCharacters(value) && utf8ByteLength(value) <= MAX_LITERAL_STRING_BYTES; + } + return false; } -function utf8ByteLength(value: string): number { - return Buffer.byteLength(value, 'utf8'); +function literalTypeTag(value: JsonLiteral): string { + return value === null ? 'null' : typeof value; } -function hasControlCharacters(value: string): boolean { - for (let i = 0; i < value.length; i++) { - const code = value.charCodeAt(i); - if (code < 0x20 || code === 0x7f) return true; - } - return false; +interface SchemaParseContext { + errors: string[]; + nodeCount: number; +} + +function failSchema(ctx: SchemaParseContext, message: string): undefined { + if (ctx.errors.length === 0) ctx.errors.push(message); + return undefined; } /** - * Validates a single declared outcome label. - * Returns an error message, or `undefined` when valid. + * Builds one validated {@link SealedProbeSchemaNode}, enforcing every finite + * bound as it recurses. Stops at the first violation (`ctx.errors` becomes + * non-empty) rather than continuing to build a tree that will be discarded. */ -export function validateOutcome(outcome: unknown): string | undefined { - if (typeof outcome !== 'string') return 'outcome must be a string'; - if (outcome.length === 0) return 'outcome must not be empty'; - if (outcome === RESERVED_ERROR_OUTCOME) { - return `outcome must not use the reserved value "${RESERVED_ERROR_OUTCOME}"`; +function buildSchemaNode(raw: unknown, ctx: SchemaParseContext, depth: number): SealedProbeSchemaNode | undefined { + if (ctx.errors.length > 0) return undefined; + if (depth > MAX_SCHEMA_DEPTH) { + return failSchema(ctx, `schema exceeds maximum depth of ${MAX_SCHEMA_DEPTH}`); } - if (hasControlCharacters(outcome)) return 'outcome must not contain control characters'; - if (utf8ByteLength(outcome) > MAX_OUTCOME_BYTES) { - return `outcome must be at most ${MAX_OUTCOME_BYTES} UTF-8 bytes`; + ctx.nodeCount += 1; + if (ctx.nodeCount > MAX_SCHEMA_NODES) { + return failSchema(ctx, `schema exceeds maximum node count of ${MAX_SCHEMA_NODES}`); } - if (!OUTCOME_PATTERN.test(outcome)) { - return 'outcome must be an ASCII identifier starting with a letter and containing only letters, digits, "_" or "-"'; + if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { + return failSchema(ctx, 'schema node must be a JSON object'); + } + + const node = raw as Record; + switch (node.type) { + case 'const': { + if (Object.keys(node).length !== 2 || !('value' in node)) { + return failSchema(ctx, 'const schema must have exactly "type" and "value"'); + } + if (!isValidLiteral(node.value)) { + return failSchema(ctx, 'const value must be a bounded string, a safe integer, a boolean, or null'); + } + return { type: 'const', value: node.value as JsonLiteral }; + } + case 'boolean': { + if (Object.keys(node).length !== 1) { + return failSchema(ctx, 'boolean schema must have only "type"'); + } + return { type: 'boolean' }; + } + case 'enum': { + if (Object.keys(node).length !== 2 || !('values' in node)) { + return failSchema(ctx, 'enum schema must have exactly "type" and "values"'); + } + const values = node.values; + if (!Array.isArray(values) || values.length === 0) { + return failSchema(ctx, 'enum values must be a non-empty array'); + } + if (values.length > MAX_ENUM_VALUES) { + return failSchema(ctx, `enum values must contain at most ${MAX_ENUM_VALUES} entries`); + } + for (const value of values) { + if (!isValidLiteral(value)) { + return failSchema(ctx, 'enum values must be bounded strings, safe integers, booleans, or null'); + } + } + const literals = values as JsonLiteral[]; + const firstTag = literalTypeTag(literals[0]); + if (!literals.every((value) => literalTypeTag(value) === firstTag)) { + return failSchema(ctx, 'enum values must all be the same JSON type'); + } + const uniqueCount = new Set(literals.map((value) => JSON.stringify(value))).size; + if (uniqueCount !== literals.length) { + return failSchema(ctx, 'enum values must be unique'); + } + return { type: 'enum', values: literals }; + } + case 'integer': { + if (Object.keys(node).length !== 3 || !('minimum' in node) || !('maximum' in node)) { + return failSchema(ctx, 'integer schema must have exactly "type", "minimum", and "maximum"'); + } + const { minimum, maximum } = node; + if (typeof minimum !== 'number' || !Number.isSafeInteger(minimum)) { + return failSchema(ctx, 'integer minimum must be a safe integer'); + } + if (typeof maximum !== 'number' || !Number.isSafeInteger(maximum)) { + return failSchema(ctx, 'integer maximum must be a safe integer'); + } + if (maximum < minimum) { + return failSchema(ctx, 'integer maximum must be >= minimum'); + } + return { type: 'integer', minimum, maximum }; + } + case 'object': { + if (Object.keys(node).length !== 2 || !('fields' in node)) { + return failSchema(ctx, 'object schema must have exactly "type" and "fields"'); + } + const fieldsRaw = node.fields; + if (typeof fieldsRaw !== 'object' || fieldsRaw === null || Array.isArray(fieldsRaw)) { + return failSchema(ctx, 'object "fields" must be a JSON object mapping field name to schema'); + } + const fieldNames = Object.keys(fieldsRaw); + if (fieldNames.length === 0) { + return failSchema(ctx, 'object schema must declare at least one field'); + } + if (fieldNames.length > MAX_OBJECT_FIELDS) { + return failSchema(ctx, `object schema must declare at most ${MAX_OBJECT_FIELDS} fields`); + } + for (const name of fieldNames) { + if (!IDENTIFIER_PATTERN.test(name)) { + return failSchema(ctx, `object field name "${name}" is not a bounded ASCII identifier`); + } + } + const fields: { name: string; schema: SealedProbeSchemaNode }[] = []; + for (const name of fieldNames) { + const child = buildSchemaNode((fieldsRaw as Record)[name], ctx, depth + 1); + if (!child) return undefined; + fields.push({ name, schema: child }); + } + return { type: 'object', fields }; + } + case 'tuple': { + if (Object.keys(node).length !== 2 || !('items' in node)) { + return failSchema(ctx, 'tuple schema must have exactly "type" and "items"'); + } + const itemsRaw = node.items; + if (!Array.isArray(itemsRaw) || itemsRaw.length === 0) { + return failSchema(ctx, 'tuple "items" must be a non-empty array'); + } + if (itemsRaw.length > MAX_TUPLE_ITEMS) { + return failSchema(ctx, `tuple schema must declare at most ${MAX_TUPLE_ITEMS} items`); + } + const items: SealedProbeSchemaNode[] = []; + for (const itemRaw of itemsRaw) { + const child = buildSchemaNode(itemRaw, ctx, depth + 1); + if (!child) return undefined; + items.push(child); + } + return { type: 'tuple', items }; + } + case 'array': { + if (Object.keys(node).length !== 3 || !('items' in node) || !('length' in node)) { + return failSchema(ctx, 'array schema must have exactly "type", "items", and "length"'); + } + const { length } = node; + if (typeof length !== 'number' || !Number.isInteger(length) || length < 0 || length > MAX_ARRAY_LENGTH) { + return failSchema(ctx, `array "length" must be an integer between 0 and ${MAX_ARRAY_LENGTH}`); + } + const child = buildSchemaNode(node.items, ctx, depth + 1); + if (!child) return undefined; + return { type: 'array', items: child, length }; + } + case 'union': { + if (Object.keys(node).length !== 2 || !('variants' in node)) { + return failSchema(ctx, 'union schema must have exactly "type" and "variants"'); + } + const variantsRaw = node.variants; + if (typeof variantsRaw !== 'object' || variantsRaw === null || Array.isArray(variantsRaw)) { + return failSchema(ctx, 'union "variants" must be a JSON object mapping tag to schema'); + } + const tags = Object.keys(variantsRaw); + if (tags.length === 0) { + return failSchema(ctx, 'union schema must declare at least one variant'); + } + if (tags.length > MAX_UNION_VARIANTS) { + return failSchema(ctx, `union schema must declare at most ${MAX_UNION_VARIANTS} variants`); + } + for (const tag of tags) { + if (!IDENTIFIER_PATTERN.test(tag)) { + return failSchema(ctx, `union tag "${tag}" is not a bounded ASCII identifier`); + } + } + const variants: { tag: string; schema: SealedProbeSchemaNode }[] = []; + for (const tag of tags) { + const child = buildSchemaNode((variantsRaw as Record)[tag], ctx, depth + 1); + if (!child) return undefined; + variants.push({ tag, schema: child }); + } + return { type: 'union', variants }; + } + default: + return failSchema( + ctx, + 'schema node "type" must be one of: const, boolean, enum, integer, object, tuple, array, union', + ); } - return undefined; } /** - * Validates a full `outcomes` value: must be an array of exactly - * {@link OUTCOME_COUNT} unique, individually-valid outcome labels. - * Returns an array of human-readable errors (empty = valid). + * Validates and parses an agent-authored schema. + * + * Rejects anything outside the finite algebra above: unbounded strings, + * floats, regex domains, recursion/`$ref` (there is no such construct to + * begin with), optional properties, `additionalProperties`, and overlapping + * untagged unions are all structurally impossible to express, so they are + * rejected by construction rather than by a separate deny-list. */ -export function validateOutcomes(outcomes: unknown): string[] { - if (!Array.isArray(outcomes)) { - return [`outcomes must be an array of exactly ${OUTCOME_COUNT} strings`]; +export function validateSchema(raw: unknown): SealedProbeSchemaValidation { + let serialized: string; + try { + serialized = JSON.stringify(raw) ?? ''; + } catch { + return { valid: false, errors: ['schema must be JSON-serializable'] }; } - - const errors: string[] = []; - if (outcomes.length !== OUTCOME_COUNT) { - errors.push(`outcomes must contain exactly ${OUTCOME_COUNT} entries`); + if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { + return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; } - outcomes.forEach((outcome, index) => { - const error = validateOutcome(outcome); - if (error) errors.push(`outcomes[${index}]: ${error}`); - }); - - const stringOutcomes = outcomes.filter((o): o is string => typeof o === 'string'); - if (new Set(stringOutcomes).size !== stringOutcomes.length) { - errors.push('outcomes must be unique'); + const ctx: SchemaParseContext = { errors: [], nodeCount: 0 }; + const schema = buildSchemaNode(raw, ctx, 0); + if (!schema || ctx.errors.length > 0) { + return { valid: false, errors: ctx.errors.length > 0 ? ctx.errors : ['invalid schema'] }; } + return { valid: true, schema }; +} - return errors; +/** Ceiling of `log2(n)` for a non-negative `BigInt`, without floating point. */ +export function ceilLog2BigInt(n: bigint): number { + if (n <= 1n) return 0; + let bits = 0; + let remainder = n - 1n; + while (remainder > 0n) { + remainder >>= 1n; + bits += 1; + } + return bits; } /** - * Validates an unknown value as a {@link SealedProbeRequest}. - * Enforces field shape, the `privateRepo` slug pattern, outcome rules, and - * script/request size caps. + * Computes a schema's successful-outcome cardinality (number of + * distinguishable valid values) as a `BigInt`, so it can never silently + * overflow even for schemas near the configured bounds. */ -export function validateSealedProbeRequest(request: unknown): SealedProbeValidation { - if (typeof request !== 'object' || request === null || Array.isArray(request)) { - return { valid: false, errors: ['request must be a JSON object'] }; - } - - const errors: string[] = []; - const requestRecord = request as Record; - const { privateRepo, outcomes, script } = requestRecord; - const allowedKeys = new Set(['privateRepo', 'outcomes', 'script']); - for (const key of Object.keys(requestRecord)) { - if (!allowedKeys.has(key)) { - errors.push(`request.${key} is not supported`); - } +export function schemaCardinality(schema: SealedProbeSchemaNode): bigint { + switch (schema.type) { + case 'const': + return 1n; + case 'boolean': + return 2n; + case 'enum': + return BigInt(schema.values.length); + case 'integer': + return BigInt(schema.maximum) - BigInt(schema.minimum) + 1n; + case 'object': + return schema.fields.reduce((acc, field) => acc * schemaCardinality(field.schema), 1n); + case 'tuple': + return schema.items.reduce((acc, item) => acc * schemaCardinality(item), 1n); + case 'array': + return schemaCardinality(schema.items) ** BigInt(schema.length); + case 'union': + return schema.variants.reduce((acc, variant) => acc + schemaCardinality(variant.schema), 0n); } +} - if (typeof privateRepo !== 'string' || privateRepo.length === 0) { - errors.push('privateRepo must be a non-empty string'); - } else if ( - privateRepo.length > MAX_PRIVATE_REPO_LENGTH - || !SEALED_PROBE_REPO_PATTERN.test(privateRepo) - ) { - errors.push( - 'privateRepo must be an "owner/repo" slug (no scheme, host, path traversal, query, fragment, or wildcard)', - ); - } +/** + * The maximum complete-transcript information charge, in bits, for one + * invocation using this schema: + * + * ```text + * queryBits = 1 (ok/error) + ceil(log2(successCardinality)) + 3 (timing) + * ``` + * + * This is the value the broker's per-repository ledger debits *before* + * copying a seed or launching Python — never refunded, regardless of the + * actual result or completion bucket. + */ +export function queryBitsForSchema(schema: SealedProbeSchemaNode): number { + return RESULT_STATUS_BIT_COST + ceilLog2BigInt(schemaCardinality(schema)) + TIMING_BUCKET_BITS; +} - errors.push(...validateOutcomes(outcomes)); +function jsonLiteralEquals(value: unknown, literal: JsonLiteral): boolean { + if (literal === null) return value === null; + if (typeof literal === 'number') return typeof value === 'number' && Number.isInteger(value) && value === literal; + return value === literal; +} - if (typeof script !== 'string' || script.length === 0) { - errors.push('script must be a non-empty string'); - } else if (utf8ByteLength(script) > MAX_SCRIPT_BYTES) { - errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); +/** + * Strictly validates a parsed JSON value against an already-approved schema: + * exact JSON type, enum membership, integer range, exact required + * object/tuple/array shape (no extras, no missing fields, exact length), and + * an explicit tagged-union variant. Never coerces. + */ +export function validateValueAgainstSchema(schema: SealedProbeSchemaNode, value: unknown): boolean { + switch (schema.type) { + case 'const': + return jsonLiteralEquals(value, schema.value); + case 'boolean': + return typeof value === 'boolean'; + case 'enum': + return schema.values.some((candidate) => jsonLiteralEquals(value, candidate)); + case 'integer': + return ( + typeof value === 'number' + && Number.isInteger(value) + && value >= schema.minimum + && value <= schema.maximum + ); + case 'object': { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const obj = value as Record; + if (Object.keys(obj).length !== schema.fields.length) return false; + return schema.fields.every( + (field) => + Object.prototype.hasOwnProperty.call(obj, field.name) + && validateValueAgainstSchema(field.schema, obj[field.name]), + ); + } + case 'tuple': + return ( + Array.isArray(value) + && value.length === schema.items.length + && schema.items.every((itemSchema, index) => validateValueAgainstSchema(itemSchema, value[index])) + ); + case 'array': + return ( + Array.isArray(value) + && value.length === schema.length + && value.every((item) => validateValueAgainstSchema(schema.items, item)) + ); + case 'union': { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const obj = value as Record; + if (Object.keys(obj).length !== 2 || !('tag' in obj) || !('value' in obj) || typeof obj.tag !== 'string') { + return false; + } + const variant = schema.variants.find((candidate) => candidate.tag === obj.tag); + return variant !== undefined && validateValueAgainstSchema(variant.schema, obj.value); + } } +} - let serialized: string | undefined; - try { - serialized = JSON.stringify(request); - } catch { - errors.push('request must be JSON-serializable'); - } - if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { - errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); +/** + * Canonically re-serializes an already-validated value. + * + * The broker calls this on its own parsed representation — never on the raw + * bytes a probe wrote — so two different serializations of the same + * semantic value (whitespace, key order, numeric formatting) collapse to the + * identical observable transcript. + */ +export function canonicalizeSchemaValue(schema: SealedProbeSchemaNode, value: unknown): string { + switch (schema.type) { + case 'const': + return JSON.stringify(schema.value); + case 'boolean': + case 'enum': + case 'integer': + return JSON.stringify(value); + case 'object': { + const obj = value as Record; + const parts = schema.fields.map( + (field) => `${JSON.stringify(field.name)}:${canonicalizeSchemaValue(field.schema, obj[field.name])}`, + ); + return `{${parts.join(',')}}`; + } + case 'tuple': { + const arr = value as unknown[]; + return `[${schema.items.map((itemSchema, index) => canonicalizeSchemaValue(itemSchema, arr[index])).join(',')}]`; + } + case 'array': { + const arr = value as unknown[]; + return `[${arr.map((item) => canonicalizeSchemaValue(schema.items, item)).join(',')}]`; + } + case 'union': { + const obj = value as { tag: string; value: unknown }; + const variant = schema.variants.find((candidate) => candidate.tag === obj.tag); + // Unreachable when `value` already passed validateValueAgainstSchema. + if (!variant) return 'null'; + return `{"tag":${JSON.stringify(obj.tag)},"value":${canonicalizeSchemaValue(variant.schema, obj.value)}}`; + } } - - if (errors.length > 0) return { valid: false, errors }; - return { valid: true }; } -/** Produces the canonical JSON representation of a sealed-probe result. */ -export function canonicalizeSealedProbeResult(result: string): string { - const canonical: SealedProbeResult = { result }; - return JSON.stringify(canonical); -} +// ── Strict JSON parsing (no `JSON.parse`) ──────────────────────────────────── -/** The canonical JSON text for the reserved error result: `{"result":"ERROR"}`. */ -export const CANONICAL_ERROR_RESULT_JSON = canonicalizeSealedProbeResult(RESERVED_ERROR_OUTCOME); +/** Hard cap on parser recursion, independent of any schema's own depth bound. */ +const MAX_JSON_PARSE_DEPTH = 32; const JSON_WHITESPACE = new Set([' ', '\t', '\n', '\r']); @@ -247,16 +588,16 @@ function skipJsonWhitespace(text: string, index: number): number { return i; } +interface ParsedNode { + value: unknown; + endIndex: number; +} + /** * Parses a JSON string literal starting at `text[start]` (`text[start]` must - * be `"`). Handles standard JSON escapes. Returns `undefined` for anything - * that is not a well-formed, terminated JSON string (including raw control - * characters, which JSON requires to be escaped). + * be `"`). Rejects raw control characters and invalid/unterminated escapes. */ -function parseJsonStringLiteral( - text: string, - start: number, -): { value: string; endIndex: number } | undefined { +function parseJsonStringLiteral(text: string, start: number): { value: string; endIndex: number } | undefined { if (text[start] !== '"') return undefined; let i = start + 1; @@ -264,9 +605,7 @@ function parseJsonStringLiteral( while (i < text.length) { const ch = text[i]; - if (ch === '"') { - return { value, endIndex: i + 1 }; - } + if (ch === '"') return { value, endIndex: i + 1 }; if (ch === '\\') { const escape = text[i + 1]; @@ -287,86 +626,218 @@ function parseJsonStringLiteral( continue; } default: - return undefined; // invalid escape sequence + return undefined; } } - // Raw control characters are not permitted inside a JSON string literal. if (ch.charCodeAt(0) < 0x20) return undefined; - value += ch; i++; } - return undefined; // unterminated string + return undefined; } -/** - * Strictly parses `raw` against the exact closed grammar - * `{"result": }` — nothing more, nothing less. - * - * Rejects (returns `undefined` for): malformed JSON, any leading/trailing - * content beyond the single object, duplicate `"result"` keys (the grammar - * only permits one `key: value` pair, so a second key is trailing data), - * extra fields, and non-string values. - */ -function tryExtractStrictResultValue(raw: string): string | undefined { - let i = skipJsonWhitespace(raw, 0); +function parseJsonNumber(text: string, start: number): ParsedNode | undefined { + let i = start; + if (text[i] === '-') i++; + if (text[i] === '0') { + i++; + } else if (text[i] >= '1' && text[i] <= '9') { + while (text[i] >= '0' && text[i] <= '9') i++; + } else { + return undefined; + } + if (text[i] === '.') { + i++; + if (!(text[i] >= '0' && text[i] <= '9')) return undefined; + while (text[i] >= '0' && text[i] <= '9') i++; + } + if (text[i] === 'e' || text[i] === 'E') { + i++; + if (text[i] === '+' || text[i] === '-') i++; + if (!(text[i] >= '0' && text[i] <= '9')) return undefined; + while (text[i] >= '0' && text[i] <= '9') i++; + } + const raw = text.slice(start, i); + const value = Number(raw); + if (!Number.isFinite(value)) return undefined; + return { value, endIndex: i }; +} - if (raw[i] !== '{') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +function parseJsonValue(text: string, index: number, depth: number): ParsedNode | undefined { + if (depth > MAX_JSON_PARSE_DEPTH) return undefined; + const ch = text[index]; - if (raw.slice(i, i + 8) !== '"result"') return undefined; - i += 8; - i = skipJsonWhitespace(raw, i); + if (ch === '{') return parseJsonObject(text, index, depth); + if (ch === '[') return parseJsonArray(text, index, depth); + if (ch === '"') { + const literal = parseJsonStringLiteral(text, index); + return literal && { value: literal.value, endIndex: literal.endIndex }; + } + if (text.startsWith('true', index)) return { value: true, endIndex: index + 4 }; + if (text.startsWith('false', index)) return { value: false, endIndex: index + 5 }; + if (text.startsWith('null', index)) return { value: null, endIndex: index + 4 }; + if (ch === '-' || (ch >= '0' && ch <= '9')) return parseJsonNumber(text, index); + return undefined; +} - if (raw[i] !== ':') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +function parseJsonObject(text: string, index: number, depth: number): ParsedNode | undefined { + let i = skipJsonWhitespace(text, index + 1); + const obj: Record = {}; + if (text[i] === '}') return { value: obj, endIndex: i + 1 }; + + for (;;) { + i = skipJsonWhitespace(text, i); + const key = parseJsonStringLiteral(text, i); + if (!key) return undefined; + i = skipJsonWhitespace(text, key.endIndex); + if (text[i] !== ':') return undefined; + i = skipJsonWhitespace(text, i + 1); + const value = parseJsonValue(text, i, depth + 1); + if (!value) return undefined; + // Reject duplicate keys outright rather than silently keeping the last + // occurrence (which is what `JSON.parse` does) — a dedicated strict + // parser, not a more permissive result encoding, is the safer choice. + if (Object.prototype.hasOwnProperty.call(obj, key.value)) return undefined; + obj[key.value] = value.value; + i = skipJsonWhitespace(text, value.endIndex); + if (text[i] === ',') { i += 1; continue; } + if (text[i] === '}') return { value: obj, endIndex: i + 1 }; + return undefined; + } +} - const parsed = parseJsonStringLiteral(raw, i); - if (!parsed) return undefined; - i = skipJsonWhitespace(raw, parsed.endIndex); +function parseJsonArray(text: string, index: number, depth: number): ParsedNode | undefined { + let i = skipJsonWhitespace(text, index + 1); + const arr: unknown[] = []; + if (text[i] === ']') return { value: arr, endIndex: i + 1 }; + + for (;;) { + i = skipJsonWhitespace(text, i); + const value = parseJsonValue(text, i, depth + 1); + if (!value) return undefined; + arr.push(value.value); + i = skipJsonWhitespace(text, value.endIndex); + if (text[i] === ',') { i += 1; continue; } + if (text[i] === ']') return { value: arr, endIndex: i + 1 }; + return undefined; + } +} - if (raw[i] !== '}') return undefined; - i++; - i = skipJsonWhitespace(raw, i); +/** + * Strictly parses exactly one JSON value from `text` — no trailing data, + * no duplicate object keys. + */ +export function strictParseJson(text: string): { value: unknown } | undefined { + const start = skipJsonWhitespace(text, 0); + const result = parseJsonValue(text, start, 0); + if (!result) return undefined; + const end = skipJsonWhitespace(text, result.endIndex); + if (end !== text.length) return undefined; + return { value: result.value }; +} - if (i !== raw.length) return undefined; // trailing data +// ── Request/result validation and canonical envelopes ─────────────────────── - return parsed.value; +/** A sealed-probe execution request, already assembled from wire framing. */ +export interface SealedProbeRequest { + /** Private repository (`owner/repo`) the probe script runs against. */ + privateRepo: string; + /** The agent-authored, AWF-bounded finite response schema. */ + schema: SealedProbeSchemaNode; + /** The probe script source. */ + script: string; } +export type SealedProbeValidation = + | { valid: true; request: SealedProbeRequest } + | { valid: false; errors: string[] }; + /** - * Parses a probe's raw stdout/result text into a {@link SealedProbeResult}. - * - * Always succeeds: malformed JSON, duplicate keys, trailing data, extra - * fields, or a value outside the declared `outcomes` enum all canonicalize - * to `{ result: "ERROR" }` rather than throwing. + * Validates an unknown value as a {@link SealedProbeRequest}: field shape, + * the `privateRepo` slug pattern, the finite response schema, and the + * script size cap. */ -export function parseSealedProbeResult(raw: string, outcomes: SealedProbeOutcomes): SealedProbeResult { - if (utf8ByteLength(raw) > MAX_RESULT_BYTES) { - return { result: RESERVED_ERROR_OUTCOME }; +export function validateSealedProbeRequest(raw: unknown): SealedProbeValidation { + if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { + return { valid: false, errors: ['request must be a JSON object'] }; + } + + const errors: string[] = []; + const record = raw as Record; + const { privateRepo, schema: schemaRaw, script } = record; + const allowedKeys = new Set(['privateRepo', 'schema', 'script']); + for (const key of Object.keys(record)) { + if (!allowedKeys.has(key)) errors.push(`request.${key} is not supported`); + } + + if (typeof privateRepo !== 'string' || privateRepo.length === 0) { + errors.push('privateRepo must be a non-empty string'); + } else if (privateRepo.length > MAX_PRIVATE_REPO_LENGTH || !SEALED_PROBE_REPO_PATTERN.test(privateRepo)) { + errors.push( + 'privateRepo must be an "owner/repo" slug (no scheme, host, path traversal, query, fragment, or wildcard)', + ); + } + + const schemaValidation = validateSchema(schemaRaw); + if (!schemaValidation.valid) { + errors.push(...schemaValidation.errors.map((error) => `schema: ${error}`)); + } + + if (typeof script !== 'string' || script.length === 0) { + errors.push('script must be a non-empty string'); + } else if (utf8ByteLength(script) > MAX_SCRIPT_BYTES) { + errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); + } + + let serialized: string | undefined; + try { + serialized = JSON.stringify(raw); + } catch { + errors.push('request must be JSON-serializable'); + } + if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { + errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); } - const value = tryExtractStrictResultValue(raw); + if ( - value !== undefined - && ( - value === RESERVED_ERROR_OUTCOME - || (outcomes as readonly string[]).includes(value) - ) + errors.length > 0 + || !schemaValidation.valid + || typeof privateRepo !== 'string' + || typeof script !== 'string' ) { - return { result: value }; + return { valid: false, errors }; } - return { result: RESERVED_ERROR_OUTCOME }; + + return { valid: true, request: { privateRepo, schema: schemaValidation.schema, script } }; +} + +/** The canonical JSON text for every failure: `{"status":"error"}`. */ +export const CANONICAL_ERROR_JSON = '{"status":"error"}'; + +/** Wraps an already-canonicalized result value into the canonical success envelope. */ +export function canonicalOkJson(canonicalResultJson: string): string { + return `{"status":"ok","result":${canonicalResultJson}}`; } /** - * Parses a probe's raw result text and returns its canonical JSON - * representation directly (equivalent to - * `canonicalizeSealedProbeResult(parseSealedProbeResult(raw, outcomes).result)`). + * Parses and validates a probe's raw output file contents against the + * request's approved schema, returning the broker's own canonical + * re-serialization of the value on success. + * + * Every failure mode — oversized output, malformed JSON, duplicate keys, + * wrong type, out-of-range value, unknown enum member, missing/extra + * fields, wrong tuple/array length, unknown union tag — maps to the same + * `{ ok: false }`, which callers turn into {@link CANONICAL_ERROR_JSON}. */ -export function parseSealedProbeResultJson(raw: string, outcomes: SealedProbeOutcomes): string { - return canonicalizeSealedProbeResult(parseSealedProbeResult(raw, outcomes).result); +export function parseAndValidateProbeOutput( + raw: string, + schema: SealedProbeSchemaNode, +): { ok: true; canonical: string } | { ok: false } { + if (utf8ByteLength(raw) > MAX_RESULT_BYTES) return { ok: false }; + const parsed = strictParseJson(raw); + if (!parsed) return { ok: false }; + if (!validateValueAgainstSchema(schema, parsed.value)) return { ok: false }; + return { ok: true, canonical: canonicalizeSchemaValue(schema, parsed.value) }; } diff --git a/src/sealed-probe/scheduler.test.ts b/src/sealed-probe/scheduler.test.ts new file mode 100644 index 000000000..45308255c --- /dev/null +++ b/src/sealed-probe/scheduler.test.ts @@ -0,0 +1,163 @@ +import * as path from 'path'; + +/** + * Unit tests for response-timing bucketing. + * + * A probe's raw completion latency is itself a secret-dependent signal; the + * broker must make every launched invocation's *observable* response time + * land on one of six fixed boundaries (10ms, 100ms, 1s, 10s, 1m, 10m), + * using a monotonic clock, regardless of how long the underlying work took. + * These tests use a fully deterministic fake clock (no real elapsed time) + * to avoid any flakiness from real-time assertions. + */ +/* eslint-disable @typescript-eslint/no-require-imports */ +const brokerDir = path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker'); +const { TIMING_BUCKETS_MS, resolveTimingBucket, createRealClock, waitForBucket } = require( + path.join(brokerDir, 'scheduler.js'), +); +/* eslint-enable @typescript-eslint/no-require-imports */ + +interface FakeClock { + nowMs(): number; + sleep(ms: number): Promise; + advance(ms: number): void; + sleepCalls: number[]; +} + +/** A fake monotonic clock: `nowMs()` only moves via explicit `advance()`, and `sleep()` advances time itself (as a real clock would while paused). */ +function createFakeClock(startMs = 0): FakeClock { + let now = startMs; + const sleepCalls: number[] = []; + return { + nowMs: () => now, + sleep: (ms: number) => { + sleepCalls.push(ms); + now += ms; + return Promise.resolve(); + }, + advance: (ms: number) => { + now += ms; + }, + sleepCalls, + }; +} + +describe('TIMING_BUCKETS_MS', () => { + it('is exactly the six documented boundaries, ascending', () => { + expect(TIMING_BUCKETS_MS).toEqual([10, 100, 1000, 10000, 60000, 600000]); + }); +}); + +describe('resolveTimingBucket', () => { + it('resolves 0ms elapsed to the smallest (10ms) bucket', () => { + expect(resolveTimingBucket(0)).toEqual({ bucketMs: 10, overflowed: false }); + }); + + it.each([ + [10, 10], + [11, 100], + [100, 100], + [101, 1000], + [1000, 1000], + [1001, 10000], + [10000, 10000], + [10001, 60000], + [60000, 60000], + [60001, 600000], + [600000, 600000], + ])('elapsed=%ims resolves to bucket=%ims (inclusive boundaries)', (elapsedMs, expectedBucketMs) => { + expect(resolveTimingBucket(elapsedMs)).toEqual({ bucketMs: expectedBucketMs, overflowed: false }); + }); + + it('marks anything past the last (600000ms) bucket as overflowed', () => { + expect(resolveTimingBucket(600001)).toEqual({ bucketMs: 600000, overflowed: true }); + expect(resolveTimingBucket(10_000_000)).toEqual({ bucketMs: 600000, overflowed: true }); + }); +}); + +describe('createRealClock', () => { + it('exposes a monotonic nowMs derived from process.hrtime.bigint, and a setTimeout-based sleep', async () => { + const clock = createRealClock(); + const before = clock.nowMs(); + expect(typeof before).toBe('number'); + await clock.sleep(1); + const after = clock.nowMs(); + expect(after).toBeGreaterThanOrEqual(before); + }); +}); + +describe('waitForBucket (fake clock — fully deterministic, no real time elapsed)', () => { + it('waits the remainder of the bucket when processing finished early', async () => { + const clock = createFakeClock(1000); + clock.advance(3); // 3ms of "processing" already elapsed on the clock + const result = await waitForBucket(1000, 3, clock); // elapsed=3ms -> bucket=10ms + expect(result).toEqual({ bucketMs: 10, overflowed: false }); + expect(clock.sleepCalls).toEqual([7]); // 10 - 3 + expect(clock.nowMs()).toBe(1010); + }); + + it('does not sleep at all when processing lands exactly on the bucket boundary', async () => { + const clock = createFakeClock(5000); + clock.advance(100); // processing consumed exactly the 100ms bucket + const result = await waitForBucket(5000, 100, clock); // elapsed exactly at 100ms bucket + expect(result).toEqual({ bucketMs: 100, overflowed: false }); + expect(clock.sleepCalls).toEqual([]); + expect(clock.nowMs()).toBe(5100); + }); + + it('does not sleep (and does not go negative) when clock drift makes remaining time negative', async () => { + // elapsedMs=3 resolves to the 10ms bucket, but if `clock.nowMs()` has + // somehow already advanced past startMs+bucketMs by the time + // waitForBucket computes the remainder, it must not call sleep with a + // negative duration. + const clock = createFakeClock(1000); + clock.advance(50); // now = 1050, past startMs(1000) + bucket(10) = 1010 + const result = await waitForBucket(1000, 3, clock); + expect(result).toEqual({ bucketMs: 10, overflowed: false }); + expect(clock.sleepCalls).toEqual([]); + }); + + it('selects successively larger buckets as elapsed time grows', async () => { + const cases: Array<[number, number]> = [ + [5, 10], + [50, 100], + [500, 1000], + [5000, 10000], + [50000, 60000], + [500000, 600000], + ]; + for (const [elapsedMs, expectedBucketMs] of cases) { + const clock = createFakeClock(0); + clock.advance(elapsedMs); + // eslint-disable-next-line no-await-in-loop + const result = await waitForBucket(0, elapsedMs, clock); + expect(result.bucketMs).toBe(expectedBucketMs); + expect(result.overflowed).toBe(false); + expect(clock.sleepCalls).toEqual([expectedBucketMs - elapsedMs]); + } + }); + + it('reports overflow (and never sleeps) once elapsed processing exceeds the last bucket', async () => { + const clock = createFakeClock(0); + const result = await waitForBucket(0, 600001, clock); + expect(result).toEqual({ bucketMs: 600000, overflowed: true }); + expect(clock.sleepCalls).toEqual([]); + }); + + it('never lets sleep duration itself vary with the exact sub-bucket elapsed time beyond the bucket granularity', async () => { + // Two invocations with different elapsed processing times, both inside + // the same bucket window, must each independently reach exactly the + // bucket boundary from their own start point — i.e. the *absolute* + // response time (startMs + bucketMs) is what's fixed, not a constant + // sleep duration. + const clockA = createFakeClock(0); + clockA.advance(1); + await waitForBucket(0, 1, clockA); + expect(clockA.nowMs()).toBe(10); + + const clockB = createFakeClock(0); + clockB.advance(9); + await waitForBucket(0, 9, clockB); + expect(clockB.nowMs()).toBe(10); + }); +}); diff --git a/src/sealed-probe/skill.test.ts b/src/sealed-probe/skill.test.ts index 4df66ae8d..909ca59cc 100644 --- a/src/sealed-probe/skill.test.ts +++ b/src/sealed-probe/skill.test.ts @@ -6,7 +6,12 @@ import { generateSealedProbeSkill, writeSealedProbeSkill } from './skill'; describe('generateSealedProbeSkill', () => { const skill = generateSealedProbeSkill({ - repos: ['octo/alpha', 'octo/beta'], + repos: [ + { repo: 'octo/alpha', sensitivity: 'internal' }, + { repo: 'octo/beta', sensitivity: 'confidential' }, + { repo: 'octo/gamma', sensitivity: 'public' }, + { repo: 'octo/delta', sensitivity: 'sealed' }, + ], timeoutSeconds: 45, maxInvocations: 9, }); @@ -17,17 +22,27 @@ describe('generateSealedProbeSkill', () => { expect(skill).toContain('description:'); }); - it('lists exactly the configured repositories', () => { - expect(skill).toContain('- `octo/alpha`'); - expect(skill).toContain('- `octo/beta`'); + it('lists exactly the configured repositories with their sensitivity and run budget', () => { + expect(skill).toContain('- `octo/alpha` — 64 bits/run (`internal`)'); + expect(skill).toContain('- `octo/beta` — 8 bits/run (`confidential`)'); + expect(skill).toContain('- `octo/gamma` — unmetered (`public`)'); + expect(skill).toContain('- `octo/delta` — 0 bits/run (`sealed` — never runs a script)'); expect(skill).toContain('Any other repository is rejected.'); }); - it('documents the fixed CLI contract and its refusals', () => { + it('documents the fixed v2 CLI contract and its refusals', () => { expect(skill).toContain('--repo owner/repo'); - expect(skill).toContain('--outcome'); + expect(skill).toContain('--schema'); expect(skill).toContain('exactly one `--repo`'); - expect(skill).toContain('You cannot choose the image, command, interpreter,'); + expect(skill).toContain('exactly one `--schema`'); + expect(skill).toMatch(/You cannot choose the image, command,\s+interpreter,/); + }); + + it('documents every finite schema construct', () => { + for (const kind of ['const', 'boolean', 'enum', 'integer', 'object', 'tuple', 'array', 'union']) { + expect(skill).toContain(`\`${kind}\``); + } + expect(skill).toContain('not** general\nJSON Schema'); }); it('documents the script contract against /probe/repo and /probe/out', () => { @@ -36,15 +51,15 @@ describe('generateSealedProbeSkill', () => { expect(skill).toContain('standard library only'); }); - it('states the configured budget and the two-bit capacity', () => { + it('states the configured operational budget and the per-invocation bit charge formula', () => { expect(skill).toContain('at most 45 second(s)'); expect(skill).toContain('At most 9 invocation(s)'); - expect(skill).toContain('at most 2 bits'); + expect(skill).toContain('1 (ok/error) + ceil(log2(schema cardinality)) + 3 (timing)'); }); - it('warns that all failures are indistinguishable', () => { - expect(skill).toContain('{"result":"ERROR"}'); - expect(skill).toContain('indistinguishable from each other by design'); + it('documents the canonical error and that failures are indistinguishable', () => { + expect(skill).toContain('{"status":"error"}'); + expect(skill).toMatch(/indistinguishable from\s+each other by design/); }); it('never suggests the agent can read the repository directly', () => { @@ -66,7 +81,7 @@ describe('writeSealedProbeSkill', () => { it('writes the skill into the AWF-owned agent artifact directory only', () => { const paths = resolveSealedProbePaths(workDir); const containerPath = writeSealedProbeSkill(paths, { - repos: ['octo/alpha'], + repos: [{ repo: 'octo/alpha', sensitivity: 'internal' }], timeoutSeconds: 30, maxInvocations: 32, }); diff --git a/src/sealed-probe/skill.ts b/src/sealed-probe/skill.ts index efffa3391..797cd62b0 100644 --- a/src/sealed-probe/skill.ts +++ b/src/sealed-probe/skill.ts @@ -1,41 +1,53 @@ import * as fs from 'fs'; +import type { SealedProbeRepository } from '../types/sealed-probe-options'; +import { SEALED_PROBE_SENSITIVITY_RUN_BITS } from '../types/sealed-probe-options'; import { AGENT_SKILL_PATH, PROBE_MOUNT_DIR, type SealedProbePaths, } from './paths'; -import { OUTCOME_COUNT, RESERVED_ERROR_OUTCOME } from './protocol'; +import { CANONICAL_ERROR_JSON, MAX_SCRIPT_BYTES, RESULT_STATUS_BIT_COST, TIMING_BUCKETS_MS, TIMING_BUCKET_BITS } from './protocol'; /** * Generates the sealed-probe skill document handed to the primary agent. * * The document is *guidance*, not a security boundary: every rule it states is * independently enforced by the `sealed-probe` wrapper and by the trusted - * broker. Its job is to tell the agent which repositories exist, what the CLI - * contract is, and what the (deliberately tiny) observable output is. + * broker. Its job is to tell the agent which repositories exist (and at what + * confidentiality budget), the v2 request contract (agent-authored finite + * schema plus script), and the observable canonical result envelope. */ interface SealedProbeSkillParams { - /** Configured repository slugs, in configuration order. */ - repos: string[]; + /** Configured repositories, in configuration order. */ + repos: SealedProbeRepository[]; /** Per-invocation wall-clock limit, in seconds. */ timeoutSeconds: number; - /** Per-run invocation budget. */ + /** Per-run invocation budget (an independent operational cap; see "Budget" below). */ maxInvocations: number; } +function formatRunBudget(repo: SealedProbeRepository): string { + const bits = SEALED_PROBE_SENSITIVITY_RUN_BITS[repo.sensitivity]; + if (bits === null) return `unmetered (\`${repo.sensitivity}\`)`; + if (bits === 0) return `0 bits/run (\`${repo.sensitivity}\` — never runs a script)`; + return `${bits} bits/run (\`${repo.sensitivity}\`)`; +} + export function generateSealedProbeSkill(params: SealedProbeSkillParams): string { const { repos, timeoutSeconds, maxInvocations } = params; - const repoList = repos.map((repo) => `- \`${repo}\``).join('\n'); + const repoList = repos.map((repo) => `- \`${repo.repo}\` — ${formatRunBudget(repo)}`).join('\n'); + const bucketList = TIMING_BUCKETS_MS.map((ms) => (ms >= 1000 ? `${ms / 1000}s` : `${ms}ms`)).join(', '); return `--- name: sealed-probe description: >- Run a short Python 3 script against one pre-approved private repository - inside a sealed, offline sandbox and learn only which of exactly - ${OUTCOME_COUNT} declared outcomes occurred. Use when you must answer a - bounded question about private repository contents that you are not allowed - to read. + inside a sealed, offline sandbox and get back a value conforming to a + finite response schema you declare up front. Use when you must answer a + bounded question about private repository contents that you are not + allowed to read, and only when your remaining per-repository information + budget can afford the answer's schema. --- # Sealed probe @@ -46,46 +58,80 @@ The sandbox has no network, no credentials, no host access, and no access to this workspace. You never see the repository contents, the script's stdout/stderr, its files, -its diffs, its exit status, or any diagnostics. **The only thing you observe is -one of ${OUTCOME_COUNT + 1} symbols**: one of the ${OUTCOME_COUNT} outcomes you -declared, or the fixed value \`${RESERVED_ERROR_OUTCOME}\`. +its diffs, its exit status, or any diagnostics. The only thing you observe is +one canonical JSON result: + +- \`{"status":"ok","result":}\` where \`\` conforms to the exact + response schema you declared, or +- \`${CANONICAL_ERROR_JSON}\` for **every** failure mode (invalid request, + disallowed repository, exhausted budget, launch failure, timeout, crash, + non-conformant output, internal error). Failures are indistinguishable from + each other by design — do not try to infer which one occurred. ## Available repositories ${repoList} -Any other repository is rejected. +Any other repository is rejected. The sensitivity and run budget shown above +are fixed by AWF configuration; a request cannot choose or override them. ## Invoking \`\`\`bash sealed-probe \\ --repo owner/repo \\ - --outcome YES \\ - --outcome NO \\ - --outcome UNKNOWN \\ + --schema '{"type":"boolean"}' \\ < probe.py \`\`\` Rules enforced by the CLI: - exactly one \`--repo\`, and it must be one of the repositories listed above; -- exactly ${OUTCOME_COUNT} \`--outcome\` values: unique ASCII identifiers that - start with a letter, contain only letters, digits, \`_\`, or \`-\`, are at - most 64 bytes, and do not equal \`${RESERVED_ERROR_OUTCOME}\`; -- the script is read from stdin; -- there are no other options. You cannot choose the image, command, interpreter, - runtime, timeout, mount, path, ref, URL, environment, or credentials. +- exactly one \`--schema\`: a JSON document (see "Response schema" below); +- the script is read from stdin and must be at most ${MAX_SCRIPT_BYTES} bytes; +- there are no other options. You cannot choose the image, command, + interpreter, runtime, timeout, mount, path, ref, URL, environment, or + credentials. The CLI always prints exactly one line of JSON and always exits \`0\`. +## Response schema + +The schema is a deliberately finite, agent-authored algebra — **not** general +JSON Schema. Every invocation may use a different schema. Supported node +types: + +| type | shape | notes | +| --- | --- | --- | +| \`const\` | \`{"type":"const","value":}\` | one fixed value | +| \`boolean\` | \`{"type":"boolean"}\` | \`true\` or \`false\` | +| \`enum\` | \`{"type":"enum","values":[,...]}\` | unique literals, same JSON type | +| \`integer\` | \`{"type":"integer","minimum":N,"maximum":M}\` | inclusive bounded range | +| \`object\` | \`{"type":"object","fields":{"name":,...}}\` | every field required, no extras | +| \`tuple\` | \`{"type":"tuple","items":[,...]}\` | fixed-length, per-position schema | +| \`array\` | \`{"type":"array","items":,"length":N}\` | fixed length, uniform item schema | +| \`union\` | \`{"type":"union","variants":{"tag":,...}}\` | value is \`{"tag":"...","value":...}\` | + +A literal (in \`const\`/\`enum\`) is a JSON string (at most 64 bytes, no control +characters), a safe integer, a boolean, or \`null\`. There is no way to express +an unbounded string, a float, a regex, recursion, \`$ref\`, an optional field, +\`additionalProperties\`, or an untagged/overlapping union — these are +structurally impossible, not merely disallowed. + +Example — a bounded integer count: + +\`\`\`json +{"type": "integer", "minimum": 0, "maximum": 100} +\`\`\` + ## Script contract The script runs as \`python3\` with the **standard library only** (no third-party packages, no package installation, no network). It may read and freely modify \`${PROBE_MOUNT_DIR}/repo\`; every mutation is discarded when the probe ends. -It must write its answer to \`${PROBE_MOUNT_DIR}/out\` as a single JSON object: +It must write its answer to \`${PROBE_MOUNT_DIR}/out\` as a single JSON value +conforming exactly to your declared schema: \`\`\`python import json @@ -93,34 +139,48 @@ from pathlib import Path repo = Path("${PROBE_MOUNT_DIR}/repo") found = any(repo.rglob("Dockerfile")) -Path("${PROBE_MOUNT_DIR}/out").write_text(json.dumps({"result": "YES" if found else "NO"})) +Path("${PROBE_MOUNT_DIR}/out").write_text(json.dumps(found)) \`\`\` -Anything else in the output — extra keys, trailing data, a value outside your -declared outcomes, malformed JSON, an oversized file, no file at all — is -reported to you as \`${RESERVED_ERROR_OUTCOME}\`. +Anything else in the output — wrong type, out-of-range value, unknown enum +member, extra/missing fields, wrong tuple/array length, malformed or +duplicate-key JSON, an oversized file, no file at all — is reported to you as +\`${CANONICAL_ERROR_JSON}\`. -## Result +## Budget -\`\`\`json -{"result":"YES"} -\`\`\` +Every invocation reserves a fixed information charge from its repository's +run budget, computed **before** anything runs: -\`{"result":"${RESERVED_ERROR_OUTCOME}"}\` is returned for **every** failure: -invalid request, disallowed repository, exhausted invocation budget, launch -failure, timeout, crash, out-of-memory, non-conformant output, or internal -error. The failures are indistinguishable from each other by design — do not -try to infer which one occurred. +\`\`\`text +charge = ${RESULT_STATUS_BIT_COST} (ok/error) + ceil(log2(schema cardinality)) + ${TIMING_BUCKET_BITS} (timing) +\`\`\` -## Budget +"Schema cardinality" is the number of distinguishable values your declared +schema admits (2 for \`boolean\`, N for an N-member \`enum\`, the product of +field cardinalities for \`object\`/\`tuple\`, and so on). The charge is debited +from the repository's remaining run balance whether the script succeeds, +fails, or times out, and is never refunded. An invocation is only allowed if +its charge fits the remaining balance — there is no separate per-query cap, +so a cheap boolean question and an expensive high-cardinality question both +draw from the same shared budget, just at different rates. + +Timing is charged because it is observable: the broker always returns at the +first bucket boundary at or after your script actually finishes (bucket +boundaries: ${bucketList}), so a fast script and a slow script are +distinguishable through response latency alone, and that must be paid for +like any other signal. - Each invocation may run for at most ${timeoutSeconds} second(s). -- At most ${maxInvocations} invocation(s) are permitted for this entire run. - Further calls return \`${RESERVED_ERROR_OUTCOME}\` without running anything. - -Because each answer is one of ${OUTCOME_COUNT + 1} symbols, a probe conveys -at most 2 bits about the repository. Design one high-value question per -invocation. +- At most ${maxInvocations} invocation(s) are permitted for this entire run, + independent of the bit budget above. Further calls return + \`${CANONICAL_ERROR_JSON}\` without running anything. +- A repository whose remaining budget cannot afford even the cheapest + possible schema (a \`const\` schema, minimum charge + ${RESULT_STATUS_BIT_COST + TIMING_BUCKET_BITS} bits) can no longer be probed + at all for the rest of this run. + +Design one high-value, low-cardinality question per invocation. `; } diff --git a/src/sealed-probe/staging.test.ts b/src/sealed-probe/staging.test.ts index 10f1d057f..2c42cb417 100644 --- a/src/sealed-probe/staging.test.ts +++ b/src/sealed-probe/staging.test.ts @@ -2,6 +2,7 @@ import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; import execa from 'execa'; +import type { SealedProbeRepository } from '../types/sealed-probe-options'; import { resolveSealedProbePaths, type SealedProbePaths } from './paths'; import { buildCloneUrl, @@ -79,7 +80,7 @@ function createFakeGit(options: { onClone?: (dest: string) => void } = {}): { async function stage( workDir: string, runner: GitRunner, - repos: string[] = ['octo/private'], + repos: SealedProbeRepository[] = [{ repo: 'octo/private', sensitivity: 'internal' }], ): Promise<{ paths: SealedProbePaths; result: Awaited> }> { const paths = resolveSealedProbePaths(workDir); fs.mkdirSync(paths.root, { recursive: true, mode: 0o700 }); @@ -302,6 +303,7 @@ describe('stageSealedProbeSeeds', () => { expect(result.seeds[0].commit).toBe('0123456789abcdef0123456789abcdef01234567'); expect(result.seeds[0].seedId).toMatch(/^[0-9a-f]{32}$/); expect(result.seeds[0].repoKey).toBe('octo/private'); + expect(result.seeds[0].sensitivity).toBe('internal'); }); it('makes every staged path read-only', async () => { @@ -344,16 +346,24 @@ describe('stageSealedProbeSeeds', () => { it('stages every configured repository into its own seed', async () => { const { runner } = createFakeGit(); - const { result } = await stage(workDir, runner, ['octo/one', 'octo/two']); + const { result } = await stage(workDir, runner, [ + { repo: 'octo/one', sensitivity: 'internal' }, + { repo: 'octo/two', sensitivity: 'confidential' }, + ]); expect(result.seeds).toHaveLength(2); expect(new Set(result.seeds.map((seed) => seed.seedId)).size).toBe(2); + expect(result.seeds.map((seed) => seed.sensitivity)).toEqual(['internal', 'confidential']); }); it('rejects duplicate repositories before overwriting an existing seed', async () => { const { runner } = createFakeGit(); - await expect(stage(workDir, runner, ['octo/private', 'octo/private'])) - .rejects.toThrow(/already exists/); + await expect( + stage(workDir, runner, [ + { repo: 'octo/private', sensitivity: 'internal' }, + { repo: 'octo/private', sensitivity: 'internal' }, + ]), + ).rejects.toThrow(/already exists/); }); it('uses the production git runner when no test runner is supplied', async () => { @@ -372,7 +382,7 @@ describe('stageSealedProbeSeeds', () => { fs.mkdirSync(paths.root, { recursive: true, mode: 0o700 }); const result = await stageSealedProbeSeeds({ - repos: ['octo/private'], + repos: [{ repo: 'octo/private', sensitivity: 'internal' }], paths, runId: 'f'.repeat(32), token: TOKEN, diff --git a/src/sealed-probe/staging.ts b/src/sealed-probe/staging.ts index f74c99510..6bdf45d20 100644 --- a/src/sealed-probe/staging.ts +++ b/src/sealed-probe/staging.ts @@ -2,6 +2,7 @@ import * as fs from 'fs'; import * as path from 'path'; import execa from 'execa'; import { logger } from '../logger'; +import type { SealedProbeRepository } from '../types/sealed-probe-options'; import { deriveSeedId, normalizeRepoKey, type SealedProbePaths } from './paths'; import { SEALED_PROBE_REPO_PATTERN } from './protocol'; import type { SealedProbeSeed, SealedProbeStagingResult } from './types'; @@ -82,8 +83,8 @@ const defaultGitRunner: GitRunner = async (args, options) => { }; export interface StageSealedProbeSeedsParams { - /** Repository slugs exactly as configured (already schema-validated). */ - repos: string[]; + /** Trusted repository descriptors exactly as configured (already schema-validated). */ + repos: SealedProbeRepository[]; /** Resolved sealed-probe filesystem layout. */ paths: SealedProbePaths; /** Run-unique id used to derive opaque seed directory names. */ @@ -331,13 +332,14 @@ export function scrubSeed(seedPath: string): void { } async function stageOneSeed( - repo: string, + repository: SealedProbeRepository, params: Required> & { gitRunner: GitRunner; gitEnv: NodeJS.ProcessEnv; }, ): Promise { const { paths, runId, gitRunner, gitEnv } = params; + const { repo, sensitivity } = repository; const seedId = deriveSeedId(runId, repo); const seedPath = path.join(paths.seedsDir, seedId); @@ -376,6 +378,9 @@ async function stageOneSeed( seedId, seedPath, commit: commit.trim(), + // Trusted AWF configuration state, carried unmodified — staging never + // derives sensitivity from anything the clone/checkout produced. + sensitivity, }; } @@ -403,9 +408,9 @@ export async function stageSealedProbeSeeds( const seeds: SealedProbeSeed[] = []; try { - for (const repo of repos) { - logger.info(`Sealed probes: staging seed for ${repo}...`); - seeds.push(await stageOneSeed(repo, { paths, runId, gitRunner, gitEnv })); + for (const repository of repos) { + logger.info(`Sealed probes: staging seed for ${repository.repo} (sensitivity: ${repository.sensitivity})...`); + seeds.push(await stageOneSeed(repository, { paths, runId, gitRunner, gitEnv })); } } catch (error) { releaseSeedPermissions(paths.seedsDir); diff --git a/src/sealed-probe/types.ts b/src/sealed-probe/types.ts index 74fc59a20..9ad146a81 100644 --- a/src/sealed-probe/types.ts +++ b/src/sealed-probe/types.ts @@ -6,8 +6,16 @@ * consumes. */ -/** Version of the on-disk seed-map document. */ -export const SEALED_PROBE_SEED_MAP_VERSION = 1; +import type { SealedProbeSensitivity } from '../types/sealed-probe-options'; + +/** + * Version of the on-disk seed-map document. + * + * v2 adds trusted `sensitivity` metadata to every entry (see + * {@link SealedProbeSeedMap}) so the broker can derive each repository's + * per-run information budget without trusting anything the agent sends. + */ +export const SEALED_PROBE_SEED_MAP_VERSION = 2; /** One staged, immutable repository seed. */ export interface SealedProbeSeed { @@ -21,6 +29,8 @@ export interface SealedProbeSeed { seedPath: string; /** Commit the seed was materialized at, recorded for protected audit state. */ commit: string; + /** Trusted confidentiality category, carried unmodified into the seed map. */ + sensitivity: SealedProbeSensitivity; } /** @@ -28,14 +38,16 @@ export interface SealedProbeSeed { * read-only into the broker. * * It intentionally contains only what the broker needs: the mapping from a - * normalized repo id to an AWF-chosen opaque seed directory name, plus the - * run id used for container labelling/orphan cleanup. No credentials, no - * absolute host paths, and no caller-controllable fields. + * normalized repo id to an AWF-chosen opaque seed directory name plus its + * trusted sensitivity, and the run id used for container labelling/orphan + * cleanup. No credentials, no absolute host paths, and no caller-controllable + * fields — in particular, `sensitivity` is trusted AWF configuration state + * that a probe request can never choose or override. */ export interface SealedProbeSeedMap { version: typeof SEALED_PROBE_SEED_MAP_VERSION; runId: string; - seeds: Array<{ repo: string; seedId: string }>; + seeds: Array<{ repo: string; seedId: string; sensitivity: SealedProbeSensitivity }>; } /** Result of the trusted host staging phase. */ diff --git a/src/sealed-probe/workflow-integration.test.ts b/src/sealed-probe/workflow-integration.test.ts index 521df184e..9492d1e83 100644 --- a/src/sealed-probe/workflow-integration.test.ts +++ b/src/sealed-probe/workflow-integration.test.ts @@ -24,7 +24,7 @@ jest.mock('../container-runtime', () => ({ const sealedProbes: SealedProbesConfig = { enabled: true, - privateRepos: ['octo/private'], + privateRepos: [{ repo: 'octo/private', sensitivity: 'internal' }], runtime: 'docker', timeout: 30, memoryLimit: '512m', diff --git a/src/sealed-probe/wrapper.test.ts b/src/sealed-probe/wrapper.test.ts index 58a534237..f5caa4986 100644 --- a/src/sealed-probe/wrapper.test.ts +++ b/src/sealed-probe/wrapper.test.ts @@ -6,16 +6,23 @@ import * as path from 'path'; /** * Behavioural tests for `containers/agent/sealed-probe-wrapper.sh`, the only - * sealed-probe capability the agent receives. + * sealed-probe capability the agent receives (protocol v2). * * The wrapper is executed for real against a stub broker on a Unix socket, so - * these assertions cover the actual shell semantics: accepted options, the - * exact request framing, and — critically — that every failure produces the - * identical canonical result on stdout, nothing on stderr, and exit status 0. + * these assertions cover the actual shell semantics: accepted options + * (`--repo`, `--schema`), the exact request framing (version header, repo + * header, base64url-encoded schema header, raw script body), and — + * critically — that every failure produces the identical canonical + * `{"status":"error"}` on stdout, nothing on stderr, and exit status 0. */ const WRAPPER = path.join(__dirname, '..', '..', 'containers', 'agent', 'sealed-probe-wrapper.sh'); -const CANONICAL_ERROR = '{"result":"ERROR"}'; +const CANONICAL_ERROR = '{"status":"error"}'; +const BOOLEAN_SCHEMA = '{"type":"boolean"}'; + +function base64url(text: string): string { + return Buffer.from(text, 'utf8').toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, ''); +} interface StubRequest { method?: string; @@ -107,15 +114,15 @@ function runWrapper( }); } -const VALID_ARGS = ['--repo', 'octo/private', '--outcome', 'YES', '--outcome', 'NO', '--outcome', 'UNKNOWN']; +const VALID_ARGS = ['--repo', 'octo/private', '--schema', BOOLEAN_SCHEMA]; describe('sealed-probe wrapper', () => { it('forwards a valid request and prints the broker result verbatim', async () => { - const harness = await startStubBroker(() => '{"result":"YES"}'); + const harness = await startStubBroker(() => '{"status":"ok","result":true}'); try { const result = await runWrapper(VALID_ARGS, { socketPath: harness.socketPath }); - expect(result.stdout).toBe('{"result":"YES"}\n'); + expect(result.stdout).toBe('{"status":"ok","result":true}\n'); expect(result.stderr).toBe(''); expect(result.status).toBe(0); } finally { @@ -123,8 +130,8 @@ describe('sealed-probe wrapper', () => { } }); - it('sends only the fixed framing: version, repo, three outcomes, raw script body', async () => { - const harness = await startStubBroker(() => '{"result":"NO"}'); + it('sends only the fixed v2 framing: version, repo, base64url schema, raw script body', async () => { + const harness = await startStubBroker(() => '{"status":"ok","result":false}'); try { await runWrapper(VALID_ARGS, { socketPath: harness.socketPath, script: 'import json\n' }); @@ -133,26 +140,35 @@ describe('sealed-probe wrapper', () => { expect(request.method).toBe('POST'); expect(request.url).toBe('/probe'); expect(request.body).toBe('import json\n'); - expect(request.headers['x-awf-probe-version']).toBe('1'); + expect(request.headers['x-awf-probe-version']).toBe('2'); expect(request.headers['x-awf-repo']).toBe('octo/private'); - expect(request.headers['x-awf-outcome-1']).toBe('YES'); - expect(request.headers['x-awf-outcome-2']).toBe('NO'); - expect(request.headers['x-awf-outcome-3']).toBe('UNKNOWN'); + expect(request.headers['x-awf-schema-b64']).toBe(base64url(BOOLEAN_SCHEMA)); const awfHeaders = Object.keys(request.headers).filter((name) => name.startsWith('x-awf-')); - expect(awfHeaders.sort()).toEqual([ - 'x-awf-outcome-1', - 'x-awf-outcome-2', - 'x-awf-outcome-3', - 'x-awf-probe-version', - 'x-awf-repo', - ]); + expect(awfHeaders.sort()).toEqual(['x-awf-probe-version', 'x-awf-repo', 'x-awf-schema-b64']); + } finally { + await harness.close(); + } + }); + + it('base64url-encodes a schema containing +, /, and padding-triggering lengths without leaking raw JSON in headers', async () => { + const harness = await startStubBroker(() => '{"status":"ok","result":"A"}'); + try { + const schema = JSON.stringify({ type: 'enum', values: ['A', 'B', 'C'] }); + await runWrapper(['--repo', 'octo/private', '--schema', schema], { socketPath: harness.socketPath }); + + const request = harness.requests[0]; + const headerValue = String(request.headers['x-awf-schema-b64']); + expect(headerValue).toBe(base64url(schema)); + // base64url must never contain the standard-base64 `+`, `/`, or `=` characters. + expect(headerValue).not.toMatch(/[+/=]/); + expect(Buffer.from(headerValue, 'base64').toString('utf8')).toBe(schema); } finally { await harness.close(); } }); - it('passes through the reserved ERROR result', async () => { + it('passes through the canonical error result unmodified', async () => { const harness = await startStubBroker(() => CANONICAL_ERROR); try { const result = await runWrapper(VALID_ARGS, { socketPath: harness.socketPath }); @@ -165,12 +181,12 @@ describe('sealed-probe wrapper', () => { }); it.each([ - ['an undeclared outcome', '{"result":"MAYBE"}'], - ['extra fields', '{"result":"YES","leak":"secret"}'], - ['a non-canonical encoding', '{ "result" : "YES" }'], - ['malformed JSON', '{"result":'], + ['a result missing the ok/error envelope', '{"result":true}'], + ['a non-canonical encoding', '{ "status" : "ok", "result" : true }'], + ['malformed JSON', '{"status":'], ['an empty body', ''], ['unexpected prose', 'boom'], + ['an unknown status value', '{"status":"maybe","result":true}'], ])('replaces %s from the broker with the canonical error', async (_name, body) => { const harness = await startStubBroker(() => body); try { @@ -191,22 +207,21 @@ describe('sealed-probe wrapper', () => { }); describe('rejects unsupported input without contacting the broker', () => { + const oversizedSchema = JSON.stringify({ type: 'enum', values: Array.from({ length: 700 }, (_, i) => `v${i}`) }); + const cases: Array<[string, string[]]> = [ ['no arguments', []], - ['missing repo', ['--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], - ['two outcomes', ['--repo', 'octo/private', '--outcome', 'A', '--outcome', 'B']], - ['four outcomes', ['--repo', 'octo/private', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C', '--outcome', 'D']], - ['duplicate outcomes', ['--repo', 'octo/private', '--outcome', 'A', '--outcome', 'A', '--outcome', 'B']], - ['reserved outcome', ['--repo', 'octo/private', '--outcome', 'A', '--outcome', 'B', '--outcome', 'ERROR']], - ['empty outcome', ['--repo', 'octo/private', '--outcome', '', '--outcome', 'B', '--outcome', 'C']], - ['oversized outcome', ['--repo', 'octo/private', '--outcome', 'x'.repeat(65), '--outcome', 'B', '--outcome', 'C']], - ['outcome with a control character', ['--repo', 'octo/private', '--outcome', 'A\nB', '--outcome', 'B', '--outcome', 'C']], - ['outcome with a quote', ['--repo', 'octo/private', '--outcome', 'A"B', '--outcome', 'B', '--outcome', 'C']], - ['repeated repo', ['--repo', 'octo/a', '--repo', 'octo/b', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], - ['url repo', ['--repo', 'https://github.com/octo/private', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], - ['traversal repo', ['--repo', 'octo/../etc', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], - ['wildcard repo', ['--repo', 'octo/*', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], - ['equals-form option', ['--repo=octo/private', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']], + ['missing repo', ['--schema', BOOLEAN_SCHEMA]], + ['missing schema', ['--repo', 'octo/private']], + ['empty schema', ['--repo', 'octo/private', '--schema', '']], + ['oversized schema (over MAX_SCHEMA_BYTES)', ['--repo', 'octo/private', '--schema', oversizedSchema]], + ['repeated repo', ['--repo', 'octo/a', '--repo', 'octo/b', '--schema', BOOLEAN_SCHEMA]], + ['repeated schema', ['--repo', 'octo/private', '--schema', BOOLEAN_SCHEMA, '--schema', BOOLEAN_SCHEMA]], + ['url repo', ['--repo', 'https://github.com/octo/private', '--schema', BOOLEAN_SCHEMA]], + ['traversal repo', ['--repo', 'octo/../etc', '--schema', BOOLEAN_SCHEMA]], + ['wildcard repo', ['--repo', 'octo/*', '--schema', BOOLEAN_SCHEMA]], + ['equals-form repo option', ['--repo=octo/private', '--schema', BOOLEAN_SCHEMA]], + ['equals-form schema option', ['--repo', 'octo/private', `--schema=${BOOLEAN_SCHEMA}`]], ['positional argument', [...VALID_ARGS, 'extra']], ['unsupported --image', [...VALID_ARGS, '--image', 'evil']], ['unsupported --timeout', [...VALID_ARGS, '--timeout', '9999']], @@ -215,11 +230,11 @@ describe('sealed-probe wrapper', () => { ['unsupported --env', [...VALID_ARGS, '--env', 'GH_TOKEN=x']], ['unsupported --ref', [...VALID_ARGS, '--ref', 'main']], ['dangling --repo', ['--repo']], - ['dangling --outcome', ['--repo', 'octo/private', '--outcome']], + ['dangling --schema', ['--repo', 'octo/private', '--schema']], ]; it.each(cases)('%s', async (_name, args) => { - const harness = await startStubBroker(() => '{"result":"YES"}'); + const harness = await startStubBroker(() => '{"status":"ok","result":true}'); try { const result = await runWrapper(args, { socketPath: harness.socketPath }); @@ -239,7 +254,7 @@ describe('sealed-probe wrapper', () => { const outputs = await Promise.all([ runWrapper(VALID_ARGS, { socketPath: harness.socketPath }), runWrapper([...VALID_ARGS, '--image', 'evil'], { socketPath: harness.socketPath }), - runWrapper(['--repo', 'octo/*', '--outcome', 'A', '--outcome', 'B', '--outcome', 'C']), + runWrapper(['--repo', 'octo/*', '--schema', BOOLEAN_SCHEMA]), runWrapper(VALID_ARGS), ]); diff --git a/src/services/agent-environment/excluded-vars.test.ts b/src/services/agent-environment/excluded-vars.test.ts index 0109cdc22..6c0e7f83a 100644 --- a/src/services/agent-environment/excluded-vars.test.ts +++ b/src/services/agent-environment/excluded-vars.test.ts @@ -202,7 +202,7 @@ describe('buildExclusionSet', () => { describe('when sealed probes are enabled (repository credential isolation)', () => { const sealedProbes = { enabled: true, - privateRepos: ['octo/private'], + privateRepos: [{ repo: 'octo/private', sensitivity: 'internal' as const }], runtime: 'docker' as const, timeout: 30, memoryLimit: '512m', diff --git a/src/services/sealed-probe-compose.test.ts b/src/services/sealed-probe-compose.test.ts index fa97cafa5..cbbb7618d 100644 --- a/src/services/sealed-probe-compose.test.ts +++ b/src/services/sealed-probe-compose.test.ts @@ -9,7 +9,7 @@ let mockConfig: WrapperConfig; const sealedProbes: SealedProbesConfig = { enabled: true, - privateRepos: ['octo/alpha'], + privateRepos: [{ repo: 'octo/alpha', sensitivity: 'internal' }], runtime: 'docker', timeout: 30, memoryLimit: '512m', diff --git a/src/services/sealed-probe-service.test.ts b/src/services/sealed-probe-service.test.ts index fbfe989da..36324ed5e 100644 --- a/src/services/sealed-probe-service.test.ts +++ b/src/services/sealed-probe-service.test.ts @@ -13,7 +13,10 @@ const WORK_DIR = '/tmp/awf-1700000000'; const sealedProbes: SealedProbesConfig = { enabled: true, - privateRepos: ['octo/alpha', 'octo/beta'], + privateRepos: [ + { repo: 'octo/alpha', sensitivity: 'internal' }, + { repo: 'octo/beta', sensitivity: 'confidential' }, + ], runtime: 'docker', timeout: 45, memoryLimit: '256m', diff --git a/src/services/sealed-probe-service.ts b/src/services/sealed-probe-service.ts index 93a499c09..450297a8e 100644 --- a/src/services/sealed-probe-service.ts +++ b/src/services/sealed-probe-service.ts @@ -230,7 +230,7 @@ export function buildSealedProbeService(params: SealedProbeServiceParams): Seale const agentEnvAdditions: Record = { AWF_SEALED_PROBE_SOCKET: AGENT_SOCKET_PATH, AWF_SEALED_PROBE_SKILL: AGENT_SKILL_PATH, - AWF_SEALED_PROBE_REPOS: sealedProbes.privateRepos.join(','), + AWF_SEALED_PROBE_REPOS: sealedProbes.privateRepos.map((repository) => repository.repo).join(','), }; // The agent receives four sealed-probe mounts: diff --git a/src/types/index.ts b/src/types/index.ts index 90640726a..c2e2c42b4 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -44,7 +44,11 @@ export { export { type SealedProbeRuntime, type SealedProbeInterpreter, + type SealedProbeSensitivity, + type SealedProbeRepository, type SealedProbesConfig, type SealedProbeOptions, SEALED_PROBE_DEFAULTS, + SEALED_PROBE_SENSITIVITIES, + SEALED_PROBE_SENSITIVITY_RUN_BITS, } from './sealed-probe-options'; diff --git a/src/types/sealed-probe-options.ts b/src/types/sealed-probe-options.ts index 8479400ae..72c905a2b 100644 --- a/src/types/sealed-probe-options.ts +++ b/src/types/sealed-probe-options.ts @@ -14,6 +14,63 @@ export type SealedProbeRuntime = 'docker' | 'gvisor'; /** Script interpreters supported for sealed-probe execution. */ export type SealedProbeInterpreter = 'python3'; +/** + * Repository confidentiality categories. + * + * Each category has a fixed, immutable maximum number of bits the broker may + * reveal about that repository across an entire AWF run (not per query — see + * {@link SEALED_PROBE_SENSITIVITY_RUN_BITS}). Users select a category; they + * cannot raise its numeric limit. A future release may add a *reducing* + * numeric override, but no category may ever be granted more than its listed + * maximum. + */ +export type SealedProbeSensitivity = 'public' | 'internal' | 'confidential' | 'sealed'; + +/** Every supported sensitivity value, for schema/validation enumeration. */ +export const SEALED_PROBE_SENSITIVITIES: readonly SealedProbeSensitivity[] = [ + 'public', + 'internal', + 'confidential', + 'sealed', +]; + +/** + * Immutable per-repository run-budget table. + * + * `null` means "unmetered": `public` still runs through the same finite + * schema/result validation and operational limits (`maxInvocations`, + * timeouts, sandboxing) as every other category, but its responses are not + * debited against a confidentiality ledger. `sealed` is `0`, which — because + * every accepted query's minimum charge is 4 bits (1 error bit + 3 timing + * bits) — always exceeds the remaining balance, so a `sealed` repository can + * never fund a single query and therefore never copies a seed or launches + * Python. + * + * The scope of this budget is one AWF run: the broker has no durable + * identity or storage across runs, so this is deliberately not a + * "lifetime" budget. + */ +export const SEALED_PROBE_SENSITIVITY_RUN_BITS: Readonly> = { + public: null, + internal: 64, + confidential: 8, + sealed: 0, +}; + +/** + * A trusted, per-repository descriptor. + * + * `sensitivity` is supplied only in AWF configuration (never in an agent + * request) and flows unmodified into the seed map the broker reads — the + * agent cannot choose or override it. + */ +export interface SealedProbeRepository { + /** Repository slug in `owner/repo` form, exactly as configured. */ + repo: string; + /** Confidentiality category, which fixes this repository's run budget. */ + sensitivity: SealedProbeSensitivity; +} + /** * Fully-normalized sealed-probe configuration, with every field resolved to * an explicit value ({@link SEALED_PROBE_DEFAULTS} applied where the AWF @@ -31,13 +88,19 @@ export interface SealedProbesConfig { enabled: boolean; /** - * Private repositories (in `owner/repo` form) the sealed-probe broker is - * permitted to fetch source from. Required to be non-empty and unique - * (enforced by the schema) whenever `enabled` is `true`. + * Private repositories the sealed-probe broker is permitted to fetch + * source from, each with its trusted confidentiality category. Required to + * be non-empty and unique (case-insensitively, enforced by preflight) + * whenever `enabled` is `true`. + * + * Legacy bare-string entries in the AWF config file are normalized to + * `{ repo, sensitivity: 'internal' }` with a warning (one-release + * compatibility) — by the time a {@link SealedProbesConfig} exists, every + * entry is already an object. * * @default [] */ - privateRepos: string[]; + privateRepos: SealedProbeRepository[]; /** * Sandbox runtime backend used to execute the probe script. From e0ca1670d87395d5b7362451e4b7119b509032ab Mon Sep 17 00:00:00 2001 From: Landon Cox Date: Wed, 29 Jul 2026 15:34:09 -0700 Subject: [PATCH 2/3] fix: close sealed probe information channels Reserve final-bucket cleanup time and rebucket late responses. Count malformed requests and clean partial workspaces. Align broker policy mirrors and restore coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ca8d9d74-46ab-48db-b05f-640cbc6d47be --- containers/sealed-probe/broker/broker.js | 11 +-- containers/sealed-probe/broker/config.js | 13 ++- containers/sealed-probe/broker/protocol.js | 23 ++--- containers/sealed-probe/broker/scheduler.js | 46 ++++++++-- containers/sealed-probe/broker/server.js | 6 +- docs/awf-config-spec.md | 38 ++++---- docs/awf-config.schema.json | 4 +- src/awf-config-schema.json | 4 +- ...nfig-file-sealed-probes-validation.test.ts | 6 +- src/sealed-probe/broker.test.ts | 21 ++++- src/sealed-probe/end-to-end.test.ts | 6 +- src/sealed-probe/preflight.test.ts | 12 +-- src/sealed-probe/preflight.ts | 19 ++-- src/sealed-probe/protocol-parity.test.ts | 22 ++++- src/sealed-probe/protocol.test.ts | 90 +++++++++++++++---- src/sealed-probe/protocol.ts | 34 +++---- src/sealed-probe/scheduler.test.ts | 38 ++++++-- 17 files changed, 258 insertions(+), 135 deletions(-) diff --git a/containers/sealed-probe/broker/broker.js b/containers/sealed-probe/broker/broker.js index e521f248b..a6da55644 100644 --- a/containers/sealed-probe/broker/broker.js +++ b/containers/sealed-probe/broker/broker.js @@ -156,8 +156,9 @@ function createBroker(params) { // Teardown is part of the observable operation: repository size and tree // shape can affect deletion time, and queued requests must not expose that - // duration outside the charged timing bucket. - if (layout && !safeDestroy(invocationId)) { + // duration outside the charged timing bucket. Destroy by invocation id + // even when creation threw after materializing only part of the workspace. + if (!safeDestroy(invocationId)) { failureReason = ['cleanup-failed']; canonicalResult = undefined; } @@ -166,9 +167,9 @@ function createBroker(params) { const { bucketMs, overflowed } = await waitForBucket(startMs, elapsedMs, clock); if (overflowed) { - // Fail closed: processing (not the script itself, which is bounded by - // `sealedProbes.timeout <= largest bucket`) overran every configured - // bucket — pathological infrastructure latency. Never emit a + // Fail closed: processing (not the script itself, whose timeout + // preserves a final-bucket post-processing margin) overran every + // configured bucket — pathological infrastructure latency. Never emit a // successful result at unbucketed timing. audit.failure(invocationId, 'timing-bucket-overflow', failureReason ? failureReason.join(':') : undefined); safeRespond(CANONICAL_ERROR_JSON); diff --git a/containers/sealed-probe/broker/config.js b/containers/sealed-probe/broker/config.js index 352b62f14..7a0f9d11c 100644 --- a/containers/sealed-probe/broker/config.js +++ b/containers/sealed-probe/broker/config.js @@ -2,7 +2,7 @@ const fs = require('fs'); const path = require('path'); -const { TIMING_BUCKETS_MS } = require('./protocol'); +const { MAX_PROBE_TIMEOUT_SECONDS } = require('./protocol'); const { SEALED_PROBE_SENSITIVITY_RUN_BITS } = require('./sensitivity'); /** @@ -53,16 +53,15 @@ function parsePositiveInt(name, fallback) { /** * Parses the per-invocation timeout, additionally re-enforcing (defense in * depth; AWF's host-side preflight already rejects an out-of-range value - * before this container ever starts) that it cannot exceed the largest - * response-timing bucket. See `./scheduler` for why that ceiling matters. + * before this container ever starts) that it preserves the final response + * bucket's post-processing margin. */ function parseTimeoutSeconds() { - const maxSeconds = TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] / 1000; const parsed = parsePositiveInt('AWF_SEALED_PROBE_TIMEOUT', 30); - if (parsed > maxSeconds) { + if (parsed > MAX_PROBE_TIMEOUT_SECONDS) { throw new Error( - `Environment variable AWF_SEALED_PROBE_TIMEOUT must be at most ${maxSeconds} ` + - '(the largest response-timing bucket, in seconds)', + `Environment variable AWF_SEALED_PROBE_TIMEOUT must be at most ${MAX_PROBE_TIMEOUT_SECONDS} seconds ` + + '(the final response bucket reserves one minute for termination, validation, and cleanup)', ); } return parsed; diff --git a/containers/sealed-probe/broker/protocol.js b/containers/sealed-probe/broker/protocol.js index af2facc0e..48d184152 100644 --- a/containers/sealed-probe/broker/protocol.js +++ b/containers/sealed-probe/broker/protocol.js @@ -24,11 +24,13 @@ const MAX_TUPLE_ITEMS = 16; const MAX_ARRAY_LENGTH = 64; const MAX_UNION_VARIANTS = 16; const MAX_SCRIPT_BYTES = 64 * 1024; -const MAX_REQUEST_BYTES = MAX_SCRIPT_BYTES + MAX_SCHEMA_BYTES + 1024; const MAX_RESULT_BYTES = 8 * 1024; const MAX_PRIVATE_REPO_LENGTH = 140; const TIMING_BUCKETS_MS = [10, 100, 1_000, 10_000, 60_000, 600_000]; +const FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS = 60_000; +const MAX_PROBE_TIMEOUT_SECONDS = + (TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] - FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS) / 1000; function ceilLog2(n) { return ceilLog2BigInt(BigInt(n)); @@ -246,15 +248,15 @@ function validateSchema(raw) { } catch { return { valid: false, errors: ['schema must be JSON-serializable'] }; } - if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { - return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; - } const ctx = { errors: [], nodeCount: 0 }; const schema = buildSchemaNode(raw, ctx, 0); if (!schema || ctx.errors.length > 0) { return { valid: false, errors: ctx.errors.length > 0 ? ctx.errors : ['invalid schema'] }; } + if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { + return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; + } return { valid: true, schema }; } @@ -557,16 +559,6 @@ function validateSealedProbeRequest(raw) { errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); } - let serialized; - try { - serialized = JSON.stringify(raw); - } catch { - errors.push('request must be JSON-serializable'); - } - if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { - errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); - } - if ( errors.length > 0 || !schemaValidation.valid @@ -605,10 +597,11 @@ module.exports = { MAX_ARRAY_LENGTH, MAX_UNION_VARIANTS, MAX_SCRIPT_BYTES, - MAX_REQUEST_BYTES, MAX_RESULT_BYTES, MAX_PRIVATE_REPO_LENGTH, TIMING_BUCKETS_MS, + FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS, + MAX_PROBE_TIMEOUT_SECONDS, TIMING_BUCKET_BITS, RESULT_STATUS_BIT_COST, SEALED_PROBE_REPO_PATTERN, diff --git a/containers/sealed-probe/broker/scheduler.js b/containers/sealed-probe/broker/scheduler.js index 809978d07..73245b18f 100644 --- a/containers/sealed-probe/broker/scheduler.js +++ b/containers/sealed-probe/broker/scheduler.js @@ -26,14 +26,21 @@ const { TIMING_BUCKETS_MS } = require('./protocol'); * unaccounted cleanup delay from the preceding invocation. * - If processing latency already exceeds the *last* bucket boundary * (only possible if infrastructure overhead — not the script itself, - * which is bounded by `sealedProbes.timeout <= 600s`, see preflight.ts — - * pushes total processing past 10 minutes), the broker fails closed: it + * whose timeout preserves a final-minute processing margin — pushes total + * processing past 10 minutes), the broker fails closed: it * treats the invocation as a canonical error and responds immediately * rather than waiting indefinitely for a nonexistent next boundary. This * is a deliberately safe fail-closed fallback for a pathological * infrastructure-latency edge case, not a normal code path. */ +/** + * Public host-scheduler tolerance after a requested wake-up. Delays beyond + * this bound are padded to the next fixed boundary instead of being returned + * at a continuously varying late time. + */ +const TIMER_WAKE_TOLERANCE_MS = 5; + /** Resolves the smallest configured bucket at or after `elapsedMs`. */ function resolveTimingBucket(elapsedMs) { for (const bucketMs of TIMING_BUCKETS_MS) { @@ -62,15 +69,36 @@ function createRealClock() { * caller must fail closed (canonical error) rather than waiting further. */ async function waitForBucket(startMs, elapsedMs, clock) { - const { bucketMs, overflowed } = resolveTimingBucket(elapsedMs); - if (overflowed) return { bucketMs, overflowed }; + let observedElapsedMs = Math.max(elapsedMs, clock.nowMs() - startMs); + + while (true) { + const { bucketMs, overflowed } = resolveTimingBucket(observedElapsedMs); + if (overflowed) return { bucketMs, overflowed }; + + const targetMs = startMs + bucketMs; + const remainingMs = targetMs - clock.nowMs(); + if (remainingMs === 0) { + return { bucketMs, overflowed: false }; + } + if (remainingMs < 0) { + observedElapsedMs = clock.nowMs() - startMs; + continue; + } - const targetMs = startMs + bucketMs; - const remainingMs = targetMs - clock.nowMs(); - if (remainingMs > 0) { await clock.sleep(remainingMs); + const wakeMs = clock.nowMs(); + if (wakeMs <= targetMs + TIMER_WAKE_TOLERANCE_MS) { + return { bucketMs, overflowed: false }; + } + + observedElapsedMs = wakeMs - startMs; } - return { bucketMs, overflowed }; } -module.exports = { TIMING_BUCKETS_MS, resolveTimingBucket, createRealClock, waitForBucket }; +module.exports = { + TIMING_BUCKETS_MS, + TIMER_WAKE_TOLERANCE_MS, + resolveTimingBucket, + createRealClock, + waitForBucket, +}; diff --git a/containers/sealed-probe/broker/server.js b/containers/sealed-probe/broker/server.js index e7f7ebdc7..490117a47 100644 --- a/containers/sealed-probe/broker/server.js +++ b/containers/sealed-probe/broker/server.js @@ -59,15 +59,13 @@ function createServer(deps) { .then((body) => { if (body.error !== undefined) { audit.failure('framing', 'body-rejected', body.error); - sendResult(res, CANONICAL_ERROR_JSON); - return undefined; + return broker.handle(undefined, (result) => sendResult(res, result)); } const framed = buildRequestFromFrame(req.headers, req.rawHeaders, body.script); if (framed.error !== undefined) { audit.failure('framing', 'frame-rejected', framed.error); - sendResult(res, CANONICAL_ERROR_JSON); - return undefined; + return broker.handle(undefined, (result) => sendResult(res, result)); } return broker.handle(framed.request, (result) => sendResult(res, result)); diff --git a/docs/awf-config-spec.md b/docs/awf-config-spec.md index f0adecc39..9e69bf57a 100644 --- a/docs/awf-config-spec.md +++ b/docs/awf-config-spec.md @@ -1595,7 +1595,7 @@ The root object MAY contain a `sealedProbes` section: | `enabled` | boolean | — | `false` | | `privateRepos` | array | Non-empty and unique (by repo slug, case-insensitively) when `enabled` is `true`. Each entry is either an object `{ "repo": "owner/repo", "sensitivity": "public" \| "internal" \| "confidential" \| "sealed" }`, or (one-release legacy compatibility) a bare `owner/repo` string, normalized to `{ repo, sensitivity: "internal" }` with a warning. Each `repo` MUST be a bare `owner/repo` slug — no scheme/host (`://`), path traversal (`..`), query string (`?`), fragment (`#`), wildcard (`*`), or extra path segments. | `[]` | | `runtime` | string | One of `"docker"`, `"gvisor"` | `"docker"` | -| `timeout` | integer | `1`–`600` seconds (bounded by the largest response-timing bucket, §14.3) | `30` | +| `timeout` | integer | `1`–`540` seconds (the final minute of the 10-minute response bucket is reserved for termination, validation, and cleanup; §14.3) | `30` | | `memoryLimit` | string | Docker-style memory limit, e.g. `"512m"`, `"1g"` | `"512m"` | | `interpreter` | string | Only `"python3"` is currently supported | `"python3"` | | `maxInvocations` | integer | `1`–`10000`; an independent operational cap, unrelated to the per-repository bit ledger | `32` | @@ -1631,10 +1631,10 @@ duplicated slug; `runtime` is `"gvisor"` and the `runsc` OCI runtime is not registered with the Docker daemon; `container.containerRuntime` is a microVM backend, which cannot receive the broker socket; the resolved Docker host is not a `unix://` socket, which a `network_mode: none` broker cannot -reach; the interpreter or a limit is unsupported; `timeout` exceeds 600 -seconds — the largest response-timing bucket (§14.3) — because a longer -timeout could let an invocation's completion time itself leak unbucketed -secret-dependent information; no staging credential is present in +reach; the interpreter or a limit is unsupported; `timeout` exceeds 540 +seconds — the 10-minute response bucket reserves its final minute for Docker +termination, result validation, container removal, and workspace cleanup; no +staging credential is present in `GH_TOKEN`/`GITHUB_TOKEN`; or any seed cannot be materialized and verified. The seed map the broker reads carries each repository's trusted @@ -1665,10 +1665,9 @@ fields: - `privateRepo` MUST match the same `owner/repo` slug rule as `sealedProbes.privateRepos` entries (§14.2). - `schema` MUST be a valid document in the finite schema DSL below. -- `script` MUST be non-empty and at most 64 KiB (`MAX_SCRIPT_BYTES`). The - overall serialized request MUST be at most `MAX_SCRIPT_BYTES + - MAX_SCHEMA_BYTES + 1024` bytes (`MAX_REQUEST_BYTES`), as a defense-in-depth - cap independent of the per-field limits. +- `script` MUST be non-empty and at most 64 KiB (`MAX_SCRIPT_BYTES`). Script + and schema sizes are enforced independently on their raw UTF-8 bytes; JSON + escaping does not reduce either allowance. **Result.** A successful probe result is the canonical envelope `{"status":"ok","result":}`, where `` conforms exactly to the @@ -1769,7 +1768,10 @@ TIMING_BUCKETS_MS = [10ms, 100ms, 1s, 10s, 60s, 600s] The broker returns at the first bucket boundary at or after the invocation's processing (execution + output validation + container removal + workspace -teardown) actually completes. This is +teardown) actually completes. A public 5ms host-scheduler tolerance covers +ordinary timer jitter. If a selected boundary has already passed, or a timer +wakes more than 5ms late, the broker re-resolves and pads to the next fixed +boundary rather than responding at the late, continuously varying time. This is included in the information budget as `TIMING_BUCKET_BITS` (3 bits — for six buckets) whether or not the script's own answer would otherwise convey any signal, because latency alone is observable and must be paid for like any @@ -1783,15 +1785,13 @@ observe a preceding invocation's unaccounted cleanup duration. Cleanup failure maps to canonical error and is recorded only in the protected audit log. -**Fail-closed timing overflow.** `sealedProbes.timeout` is capped at 600 -seconds (the largest bucket) at preflight for exactly this reason: the -script itself can never make processing exceed 600 seconds. If -infrastructure overhead (not the script) ever pushed total processing past -600 seconds, the broker treats the invocation as the canonical error and -responds immediately, discarding even an otherwise-valid successful result, -rather than waiting for a nonexistent next boundary or leaking an -unbucketed excess duration. This is a deliberate, tested (`broker.test.ts`) -safe fallback for a pathological latency edge case, not a normal code path. +**Fail-closed timing overflow.** `sealedProbes.timeout` is capped at 540 +seconds, reserving the final minute before the 600-second boundary for Docker +termination, result validation, container removal, and workspace cleanup. If +pathological infrastructure overhead nevertheless pushes total processing or +a late scheduler wake past the last boundary, the broker discards even an +otherwise-valid successful result and returns the canonical error. This is a +deliberate, tested (`broker.test.ts`) fallback, not a normal code path. ### 14.4 Canonical Failure Closure diff --git a/docs/awf-config.schema.json b/docs/awf-config.schema.json index eb3ad6f98..9f2c6181e 100644 --- a/docs/awf-config.schema.json +++ b/docs/awf-config.schema.json @@ -897,8 +897,8 @@ "timeout": { "type": "integer", "minimum": 1, - "maximum": 600, - "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 600 (the largest response-timing bucket) so every invocation's completion always lands inside a bucket. Default: 30.", + "maximum": 540, + "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 540 so the 10-minute response bucket reserves its final minute for termination, validation, container removal, and workspace cleanup. Default: 30.", "default": 30 }, "memoryLimit": { diff --git a/src/awf-config-schema.json b/src/awf-config-schema.json index eb3ad6f98..9f2c6181e 100644 --- a/src/awf-config-schema.json +++ b/src/awf-config-schema.json @@ -897,8 +897,8 @@ "timeout": { "type": "integer", "minimum": 1, - "maximum": 600, - "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 600 (the largest response-timing bucket) so every invocation's completion always lands inside a bucket. Default: 30.", + "maximum": 540, + "description": "Maximum wall-clock time in seconds allowed for a single probe invocation. Exceeding it returns the canonical error. Capped at 540 so the 10-minute response bucket reserves its final minute for termination, validation, container removal, and workspace cleanup. Default: 30.", "default": 30 }, "memoryLimit": { diff --git a/src/config-file-sealed-probes-validation.test.ts b/src/config-file-sealed-probes-validation.test.ts index f78d2e26d..54250aa6a 100644 --- a/src/config-file-sealed-probes-validation.test.ts +++ b/src/config-file-sealed-probes-validation.test.ts @@ -114,9 +114,9 @@ describe('validateAwfFileConfig — sealedProbes', () => { expect(validateAwfFileConfig({ sealedProbes: { timeout: 30 } })).toEqual([]); }); - it('accepts the maximum timeout of 600 seconds (the largest timing bucket) and rejects one second above it', () => { - expect(validateAwfFileConfig({ sealedProbes: { timeout: 600 } })).toEqual([]); - expect(validateAwfFileConfig({ sealedProbes: { timeout: 601 } }).length).toBeGreaterThan(0); + it('accepts the maximum timeout of 540 seconds and rejects one second above it', () => { + expect(validateAwfFileConfig({ sealedProbes: { timeout: 540 } })).toEqual([]); + expect(validateAwfFileConfig({ sealedProbes: { timeout: 541 } }).length).toBeGreaterThan(0); }); it('rejects an invalid memoryLimit format', () => { diff --git a/src/sealed-probe/broker.test.ts b/src/sealed-probe/broker.test.ts index ef7755129..2549f4dfa 100644 --- a/src/sealed-probe/broker.test.ts +++ b/src/sealed-probe/broker.test.ts @@ -385,6 +385,25 @@ describe('sealed-probe broker', () => { expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'cleanup-failed' }); }); + it('destroys a partial workspace when workspace creation throws', async () => { + const partialWorkspace = { + ...workspace, + createInvocationWorkspace: (params: { config: { workDir: string }; invocationId: string }) => { + fs.mkdirSync(path.join(params.config.workDir, params.invocationId), { recursive: true }); + fs.writeFileSync(path.join(params.config.workDir, params.invocationId, 'partial'), 'data'); + throw new Error('copy failed'); + }, + }; + const runner = probeRunner(() => { + throw new Error('probe must not launch'); + }); + const { broker, audit } = build(runner, { workspace: partialWorkspace }); + + expect(await invoke(broker, validRequest())).toBe(CANONICAL_ERROR); + expect(audit.records[audit.records.length - 1]).toMatchObject({ reason: 'workspace-create-failed' }); + expect(fs.readdirSync(String(config.workDir))).toEqual([]); + }); + it('maps a launch failure to the canonical error', async () => { const runner = { runProbeContainer: async () => { @@ -614,7 +633,7 @@ describe('sealed-probe broker', () => { const runner = probeRunner((invocationDir) => { // Pathological infrastructure latency far beyond the largest bucket // (600_000ms) — never possible from the script itself, which is - // capped at sealedProbes.timeout <= 600s by preflight.ts. + // capped at sealedProbes.timeout <= 540s by preflight.ts. advance(TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] + 1); fs.writeFileSync(path.join(invocationDir, 'out'), '{"result":"YES"}'); }); diff --git a/src/sealed-probe/end-to-end.test.ts b/src/sealed-probe/end-to-end.test.ts index 0826057f2..cfac00eea 100644 --- a/src/sealed-probe/end-to-end.test.ts +++ b/src/sealed-probe/end-to-end.test.ts @@ -205,12 +205,16 @@ describe('sealed probe end-to-end (wrapper → socket → broker)', () => { expect(audit.some((record) => record.reason === 'bit-budget-exhausted')).toBe(true); }); - it('rejects an oversized script with the canonical error', async () => { + it('rejects an oversized script and charges it against maxInvocations', async () => { const result = await runWrapper(socketPath, args('octo/alpha'), 'x'.repeat(64 * 1024 + 10)); + const admitted = await runWrapper(socketPath, args('octo/alpha')); + const exhausted = await runWrapper(socketPath, args('octo/alpha')); expect(result.stdout).toBe(`${CANONICAL_ERROR}\n`); expect(result.stderr).toBe(''); expect(result.status).toBe(0); + expect(admitted.stdout).toBe('{"status":"ok","result":"YES"}\n'); + expect(exhausted.stdout).toBe(`${CANONICAL_ERROR}\n`); }); it('rejects a request whose probe output does not conform to its own declared schema', async () => { diff --git a/src/sealed-probe/preflight.test.ts b/src/sealed-probe/preflight.test.ts index 6b484842e..a5a1ab200 100644 --- a/src/sealed-probe/preflight.test.ts +++ b/src/sealed-probe/preflight.test.ts @@ -125,14 +125,14 @@ describe('validateSealedProbeConfig', () => { expect(errors.join('\n')).toContain('is not a Docker memory limit'); }); - it('accepts a timeout at exactly the largest timing bucket (600s)', () => { - expect(validateSealedProbeConfig(buildConfig({ timeout: 600 }), envWithToken)).toEqual([]); + it('accepts a timeout that preserves the final one-minute processing margin (540s)', () => { + expect(validateSealedProbeConfig(buildConfig({ timeout: 540 }), envWithToken)).toEqual([]); }); - it('rejects a timeout beyond the largest timing bucket, which could leak unbucketed timing', () => { - const errors = validateSealedProbeConfig(buildConfig({ timeout: 601 }), envWithToken); - expect(errors.join('\n')).toContain('timeout must be at most 600 seconds'); - expect(errors.join('\n')).toContain('unbucketed secret-dependent information'); + it('rejects a timeout that consumes the final timing bucket processing margin', () => { + const errors = validateSealedProbeConfig(buildConfig({ timeout: 541 }), envWithToken); + expect(errors.join('\n')).toContain('timeout must be at most 540 seconds'); + expect(errors.join('\n')).toContain('reserves its final minute'); }); it('rejects an unsupported interpreter', () => { diff --git a/src/sealed-probe/preflight.ts b/src/sealed-probe/preflight.ts index a11ba3a98..bfcad0093 100644 --- a/src/sealed-probe/preflight.ts +++ b/src/sealed-probe/preflight.ts @@ -3,7 +3,7 @@ import { getLocalDockerEnv } from '../host-env'; import { runtimeUsesComposeAgent } from '../container-runtime'; import type { SealedProbesConfig, WrapperConfig } from '../types'; import { normalizeRepoKey } from './paths'; -import { SEALED_PROBE_REPO_PATTERN, TIMING_BUCKETS_MS } from './protocol'; +import { MAX_PROBE_TIMEOUT_SECONDS, SEALED_PROBE_REPO_PATTERN } from './protocol'; import { resolveStagingToken } from './staging'; /** @@ -91,20 +91,15 @@ export function validateSealedProbeConfig( errors.push(`sealedProbes.interpreter "${sealedProbes.interpreter}" is not supported`); } - // The largest observable timing bucket bounds how long the broker can ever - // wait before answering (see `TIMING_BUCKETS_MS` in ./protocol). Capping the - // configured timeout at that same ceiling guarantees every completed - // invocation — success, failure, or timeout — always lands inside a - // bucket, so response latency alone can never distinguish a timeout from a - // merely slow-but-successful script. - const maxTimeoutSeconds = TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] / 1000; + // Reserve the final minute of the 10-minute response bucket for Docker + // termination, result validation, container removal, and workspace cleanup. + // The script timeout cannot consume the entire observable boundary. if (!Number.isInteger(sealedProbes.timeout) || sealedProbes.timeout < 1) { errors.push('sealedProbes.timeout must be a positive integer number of seconds'); - } else if (sealedProbes.timeout > maxTimeoutSeconds) { + } else if (sealedProbes.timeout > MAX_PROBE_TIMEOUT_SECONDS) { errors.push( - `sealedProbes.timeout must be at most ${maxTimeoutSeconds} seconds ` + - `(the largest response-timing bucket); a longer timeout could let an invocation's ` + - 'completion time itself leak unbucketed secret-dependent information', + `sealedProbes.timeout must be at most ${MAX_PROBE_TIMEOUT_SECONDS} seconds ` + + '(the 10-minute response bucket reserves its final minute for termination, validation, and cleanup)', ); } diff --git a/src/sealed-probe/protocol-parity.test.ts b/src/sealed-probe/protocol-parity.test.ts index 2bd8bf26e..d0eb0f16e 100644 --- a/src/sealed-probe/protocol-parity.test.ts +++ b/src/sealed-probe/protocol-parity.test.ts @@ -5,7 +5,7 @@ import { MAX_ENUM_VALUES, MAX_OBJECT_FIELDS, MAX_PRIVATE_REPO_LENGTH, - MAX_REQUEST_BYTES, + MAX_PROBE_TIMEOUT_SECONDS, MAX_RESULT_BYTES, MAX_SCHEMA_BYTES, MAX_SCHEMA_DEPTH, @@ -15,6 +15,7 @@ import { MAX_UNION_VARIANTS, PROBE_PROTOCOL_VERSION, RESULT_STATUS_BIT_COST, + FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS, SEALED_PROBE_REPO_PATTERN, TIMING_BUCKETS_MS, TIMING_BUCKET_BITS, @@ -30,6 +31,10 @@ import { validateValueAgainstSchema, type SealedProbeSchemaNode, } from './protocol'; +import { + SEALED_PROBE_SENSITIVITIES, + SEALED_PROBE_SENSITIVITY_RUN_BITS, +} from '../types/sealed-probe-options'; /** * The broker runs in its own container image and cannot import AWF's @@ -43,6 +48,10 @@ import { const brokerProtocol = require( path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker', 'protocol.js'), ); +// eslint-disable-next-line @typescript-eslint/no-require-imports +const brokerSensitivity = require( + path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker', 'sensitivity.js'), +); const SCHEMA_VECTORS: Array<{ name: string; schema: unknown }> = [ { name: 'const string', schema: { type: 'const', value: 'ok' } }, @@ -257,16 +266,25 @@ describe('sealed-probe protocol parity (TypeScript vs broker JavaScript)', () => expect(brokerProtocol.MAX_ARRAY_LENGTH).toBe(MAX_ARRAY_LENGTH); expect(brokerProtocol.MAX_UNION_VARIANTS).toBe(MAX_UNION_VARIANTS); expect(brokerProtocol.MAX_SCRIPT_BYTES).toBe(MAX_SCRIPT_BYTES); - expect(brokerProtocol.MAX_REQUEST_BYTES).toBe(MAX_REQUEST_BYTES); expect(brokerProtocol.MAX_RESULT_BYTES).toBe(MAX_RESULT_BYTES); expect(brokerProtocol.MAX_PRIVATE_REPO_LENGTH).toBe(MAX_PRIVATE_REPO_LENGTH); expect(brokerProtocol.TIMING_BUCKETS_MS).toEqual(TIMING_BUCKETS_MS); + expect(brokerProtocol.FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS) + .toBe(FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS); + expect(brokerProtocol.MAX_PROBE_TIMEOUT_SECONDS).toBe(MAX_PROBE_TIMEOUT_SECONDS); expect(brokerProtocol.TIMING_BUCKET_BITS).toBe(TIMING_BUCKET_BITS); expect(brokerProtocol.RESULT_STATUS_BIT_COST).toBe(RESULT_STATUS_BIT_COST); expect(brokerProtocol.SEALED_PROBE_REPO_PATTERN.source).toBe(SEALED_PROBE_REPO_PATTERN.source); expect(brokerProtocol.CANONICAL_ERROR_JSON).toBe(CANONICAL_ERROR_JSON); }); + it('keeps broker sensitivity categories and run budgets aligned with host policy', () => { + expect(brokerSensitivity.SEALED_PROBE_SENSITIVITIES).toEqual(SEALED_PROBE_SENSITIVITIES); + expect(brokerSensitivity.SEALED_PROBE_SENSITIVITY_RUN_BITS).toEqual( + SEALED_PROBE_SENSITIVITY_RUN_BITS, + ); + }); + it.each(SCHEMA_VECTORS)('agrees on schema validity: $name', ({ schema }) => { const ts = validateSchema(schema); const js = brokerProtocol.validateSchema(schema); diff --git a/src/sealed-probe/protocol.test.ts b/src/sealed-probe/protocol.test.ts index 5df439c2b..4b769a9e0 100644 --- a/src/sealed-probe/protocol.test.ts +++ b/src/sealed-probe/protocol.test.ts @@ -5,7 +5,6 @@ import { MAX_LITERAL_STRING_BYTES, MAX_OBJECT_FIELDS, MAX_PRIVATE_REPO_LENGTH, - MAX_REQUEST_BYTES, MAX_RESULT_BYTES, MAX_SCHEMA_BYTES, MAX_SCHEMA_DEPTH, @@ -30,6 +29,11 @@ import { validateValueAgainstSchema, type SealedProbeSchemaNode, } from './protocol'; +import { + SEALED_PROBE_DEFAULTS as EXPORTED_DEFAULTS, + SEALED_PROBE_SENSITIVITIES as EXPORTED_SENSITIVITIES, + SEALED_PROBE_SENSITIVITY_RUN_BITS as EXPORTED_RUN_BITS, +} from '../types'; describe('protocol constants', () => { it('fixes the wire protocol version at 2', () => { @@ -44,6 +48,12 @@ describe('protocol constants', () => { it('charges 1 bit for the ok/error distinction', () => { expect(RESULT_STATUS_BIT_COST).toBe(1); }); + + it('exposes sealed-probe policy constants through the public types barrel', () => { + expect(EXPORTED_DEFAULTS.timeout).toBe(30); + expect(EXPORTED_SENSITIVITIES).toEqual(['public', 'internal', 'confidential', 'sealed']); + expect(EXPORTED_RUN_BITS).toEqual({ public: null, internal: 64, confidential: 8, sealed: 0 }); + }); }); describe('SEALED_PROBE_REPO_PATTERN', () => { @@ -107,6 +117,23 @@ describe('validateSchema', () => { expect(validateSchema({ type: 'const', value: 'ok', extra: 1 }).valid).toBe(false); }); + it('rejects malformed literal and schema-node shapes', () => { + expect(validateSchema({ type: 'const' }).valid).toBe(false); + expect(validateSchema({ type: 'const', value: { arbitrary: 'object' } }).valid).toBe(false); + expect(validateSchema({ type: 'const', value: 'line\nbreak' }).valid).toBe(false); + expect(validateSchema({ type: 'enum' }).valid).toBe(false); + expect(validateSchema({ type: 'enum', values: [undefined] }).valid).toBe(false); + expect(validateSchema({ type: 'enum', values: [null] }).valid).toBe(true); + expect(validateSchema({ type: 'integer', minimum: 0 }).valid).toBe(false); + expect(validateSchema({ type: 'object' }).valid).toBe(false); + expect(validateSchema({ type: 'object', fields: [] }).valid).toBe(false); + expect(validateSchema({ type: 'tuple' }).valid).toBe(false); + expect(validateSchema({ type: 'array', items: { type: 'boolean' } }).valid).toBe(false); + expect(validateSchema({ type: 'union' }).valid).toBe(false); + expect(validateSchema({ type: 'union', variants: [] }).valid).toBe(false); + expect(validateSchema({ type: 'union', variants: { bad: { type: 'unknown' } } }).valid).toBe(false); + }); + it('accepts a boolean schema and rejects extra properties', () => { expect(validateSchema({ type: 'boolean' })).toEqual({ valid: true, schema: { type: 'boolean' } }); expect(validateSchema({ type: 'boolean', extra: 1 }).valid).toBe(false); @@ -231,10 +258,15 @@ describe('validateSchema', () => { }); it(`rejects a schema exceeding ${MAX_SCHEMA_NODES} total nodes`, () => { - // A tuple of many boolean leaves quickly exceeds the node-count bound - // (root + N leaves) independent of depth. - const items = Array.from({ length: MAX_SCHEMA_NODES }, () => ({ type: 'boolean' })); - expect(validateSchema({ type: 'tuple', items }).valid).toBe(false); + const fields = Object.fromEntries( + Array.from({ length: 16 }, (_, i) => [ + `f${i}`, + { type: 'tuple', items: Array.from({ length: 4 }, () => ({ type: 'boolean' })) }, + ]), + ); + const result = validateSchema({ type: 'object', fields }); + expect(result.valid).toBe(false); + if (!result.valid) expect(result.errors.join(' ')).toContain('maximum node count'); }); it(`rejects a const literal string exceeding ${MAX_LITERAL_STRING_BYTES} bytes`, () => { @@ -456,6 +488,7 @@ describe('canonicalizeSchemaValue', () => { variants: [{ tag: 'a', schema: { type: 'boolean' } }], }; expect(canonicalizeSchemaValue(schema, { tag: 'a', value: true })).toBe('{"tag":"a","value":true}'); + expect(canonicalizeSchemaValue(schema, { tag: 'missing', value: true })).toBe('null'); }); }); @@ -491,11 +524,38 @@ describe('strictParseJson', () => { it.each([ ['{"a":"s\\"uccess"}', { a: 's"uccess' }], ['{"a":"s\\\\uccess"}', { a: 's\\uccess' }], + ['{"a":"s\\/uccess"}', { a: 's/uccess' }], + ['{"a":"\\b\\f\\n\\r\\t"}', { a: '\b\f\n\r\t' }], ['{"a":"\\u0073"}', { a: 's' }], ])('parses standard JSON escapes: %s', (raw, expected) => { expect(strictParseJson(raw)).toEqual({ value: expected }); }); + it.each([ + ['0', 0], + ['-1', -1], + ['12.5', 12.5], + ['1e3', 1000], + ['1E+3', 1000], + ['1e-3', 0.001], + ['{}', {}], + ['[]', []], + ['false', false], + ])('parses JSON number and empty-container form %s', (raw, expected) => { + expect(strictParseJson(raw)).toEqual({ value: expected }); + }); + + it.each(['01', '-', '1.', '1e', '1e+', '1e999', '{"a" 1}', '{"a":}', '[1', '[1,]', '{"a":1,}'])( + 'rejects malformed number or container syntax: %s', + (raw) => { + expect(strictParseJson(raw)).toBeUndefined(); + }, + ); + + it('rejects JSON nesting beyond the parser depth bound', () => { + expect(strictParseJson(`${'['.repeat(40)}0${']'.repeat(40)}`)).toBeUndefined(); + }); + it.each(['{"a":"\\x41"}', '{"a":"\\uZZZZ"}', '{"a":"trailing\\\\'])( 'rejects invalid escapes: %s', (raw) => { @@ -566,16 +626,10 @@ describe('validateSealedProbeRequest', () => { expect(validateSealedProbeRequest({ ...validRequest, script: 'x'.repeat(MAX_SCRIPT_BYTES) }).valid).toBe(true); }); - it('rejects a request whose overall serialized size exceeds the request cap', () => { - const result = validateSealedProbeRequest({ - ...validRequest, - extraPadding: 'x'.repeat(MAX_REQUEST_BYTES), - }); - expect(result.valid).toBe(false); - if (!result.valid) { - expect(result.errors.some((e) => e.includes('request must be at most'))).toBe(true); - expect(result.errors.some((e) => e.includes('extraPadding is not supported'))).toBe(true); - } + it('accepts an escape-heavy script at the raw script cap', () => { + expect( + validateSealedProbeRequest({ ...validRequest, script: '\n'.repeat(MAX_SCRIPT_BYTES) }).valid, + ).toBe(true); }); it('rejects unsupported request fields before launch', () => { @@ -586,15 +640,13 @@ describe('validateSealedProbeRequest', () => { }); }); - it('rejects a request that cannot be serialized', () => { + it('rejects a cyclic request through its unsupported field', () => { const cyclic: Record = { ...validRequest }; cyclic.self = cyclic; const result = validateSealedProbeRequest(cyclic); expect(result.valid).toBe(false); if (!result.valid) { - expect(result.errors).toEqual( - expect.arrayContaining(['request.self is not supported', 'request must be JSON-serializable']), - ); + expect(result.errors).toEqual(expect.arrayContaining(['request.self is not supported'])); } }); diff --git a/src/sealed-probe/protocol.ts b/src/sealed-probe/protocol.ts index 2fc8026a8..0973a7878 100644 --- a/src/sealed-probe/protocol.ts +++ b/src/sealed-probe/protocol.ts @@ -75,13 +75,6 @@ export const MAX_UNION_VARIANTS = 16; /** Maximum size, in UTF-8 bytes, of a probe script. */ export const MAX_SCRIPT_BYTES = 64 * 1024; -/** - * Maximum size, in UTF-8 bytes, of the assembled `{privateRepo, schema, - * script}` request object considered as a whole (sanity bound; the schema - * and script are already independently bounded above). - */ -export const MAX_REQUEST_BYTES = MAX_SCRIPT_BYTES + MAX_SCHEMA_BYTES + 1024; - /** Maximum size, in UTF-8 bytes, of the probe's raw output file. */ export const MAX_RESULT_BYTES = 8 * 1024; @@ -91,6 +84,17 @@ export const MAX_PRIVATE_REPO_LENGTH = 140; /** Number of observable response-timing buckets (see `docs/awf-config-spec.md` §14). */ export const TIMING_BUCKETS_MS: readonly number[] = [10, 100, 1_000, 10_000, 60_000, 600_000]; +/** + * Time reserved inside the final bucket for Docker termination, result + * validation, container removal, and workspace cleanup after the script's + * configured wall-clock budget expires. + */ +export const FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS = 60_000; + +/** Largest configurable script timeout while preserving the final-bucket margin. */ +export const MAX_PROBE_TIMEOUT_SECONDS = + (TIMING_BUCKETS_MS[TIMING_BUCKETS_MS.length - 1] - FINAL_TIMING_BUCKET_PROCESSING_MARGIN_MS) / 1000; + /** * Bits reserved for the timing side channel: `ceil(log2(TIMING_BUCKETS_MS.length))`. * Fixed at 3 for the current six-bucket design; recomputed defensively below @@ -407,15 +411,15 @@ export function validateSchema(raw: unknown): SealedProbeSchemaValidation { } catch { return { valid: false, errors: ['schema must be JSON-serializable'] }; } - if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { - return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; - } const ctx: SchemaParseContext = { errors: [], nodeCount: 0 }; const schema = buildSchemaNode(raw, ctx, 0); if (!schema || ctx.errors.length > 0) { return { valid: false, errors: ctx.errors.length > 0 ? ctx.errors : ['invalid schema'] }; } + if (raw === undefined || utf8ByteLength(serialized) > MAX_SCHEMA_BYTES) { + return { valid: false, errors: [`schema must be a JSON value of at most ${MAX_SCHEMA_BYTES} bytes`] }; + } return { valid: true, schema }; } @@ -791,16 +795,6 @@ export function validateSealedProbeRequest(raw: unknown): SealedProbeValidation errors.push(`script must be at most ${MAX_SCRIPT_BYTES} bytes`); } - let serialized: string | undefined; - try { - serialized = JSON.stringify(raw); - } catch { - errors.push('request must be JSON-serializable'); - } - if (serialized !== undefined && utf8ByteLength(serialized) > MAX_REQUEST_BYTES) { - errors.push(`request must be at most ${MAX_REQUEST_BYTES} bytes`); - } - if ( errors.length > 0 || !schemaValidation.valid diff --git a/src/sealed-probe/scheduler.test.ts b/src/sealed-probe/scheduler.test.ts index 45308255c..25dcbba97 100644 --- a/src/sealed-probe/scheduler.test.ts +++ b/src/sealed-probe/scheduler.test.ts @@ -12,7 +12,13 @@ import * as path from 'path'; */ /* eslint-disable @typescript-eslint/no-require-imports */ const brokerDir = path.join(__dirname, '..', '..', 'containers', 'sealed-probe', 'broker'); -const { TIMING_BUCKETS_MS, resolveTimingBucket, createRealClock, waitForBucket } = require( +const { + TIMING_BUCKETS_MS, + TIMER_WAKE_TOLERANCE_MS, + resolveTimingBucket, + createRealClock, + waitForBucket, +} = require( path.join(brokerDir, 'scheduler.js'), ); /* eslint-enable @typescript-eslint/no-require-imports */ @@ -105,16 +111,32 @@ describe('waitForBucket (fake clock — fully deterministic, no real time elapse expect(clock.nowMs()).toBe(5100); }); - it('does not sleep (and does not go negative) when clock drift makes remaining time negative', async () => { - // elapsedMs=3 resolves to the 10ms bucket, but if `clock.nowMs()` has - // somehow already advanced past startMs+bucketMs by the time - // waitForBucket computes the remainder, it must not call sleep with a - // negative duration. + it('re-resolves to a later fixed boundary when the initially selected boundary has passed', async () => { const clock = createFakeClock(1000); clock.advance(50); // now = 1050, past startMs(1000) + bucket(10) = 1010 const result = await waitForBucket(1000, 3, clock); - expect(result).toEqual({ bucketMs: 10, overflowed: false }); - expect(clock.sleepCalls).toEqual([]); + expect(result).toEqual({ bucketMs: 100, overflowed: false }); + expect(clock.sleepCalls).toEqual([50]); + expect(clock.nowMs()).toBe(1100); + }); + + it('re-buckets a timer wake-up later than the public scheduler tolerance', async () => { + let now = 3; + const sleepCalls: number[] = []; + const clock = { + nowMs: () => now, + sleep: (ms: number) => { + sleepCalls.push(ms); + now += ms + (sleepCalls.length === 1 ? TIMER_WAKE_TOLERANCE_MS + 1 : 0); + return Promise.resolve(); + }, + }; + + const result = await waitForBucket(0, 3, clock); + + expect(result).toEqual({ bucketMs: 100, overflowed: false }); + expect(sleepCalls).toEqual([7, 84]); + expect(now).toBe(100); }); it('selects successively larger buckets as elapsed time grows', async () => { From ea6f4fe7f300f7c8b71efa34c91e8ab0b078dd02 Mon Sep 17 00:00:00 2001 From: Landon Cox Date: Wed, 29 Jul 2026 15:50:28 -0700 Subject: [PATCH 3/3] docs: add sealed probes guide (#6734) Document sealed probe configuration, budgets, schemas, timing, and failures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ca8d9d74-46ab-48db-b05f-640cbc6d47be --- .../src/content/docs/guides/sealed-probes.md | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) create mode 100644 docs-site/src/content/docs/guides/sealed-probes.md diff --git a/docs-site/src/content/docs/guides/sealed-probes.md b/docs-site/src/content/docs/guides/sealed-probes.md new file mode 100644 index 000000000..4f8dd19f3 --- /dev/null +++ b/docs-site/src/content/docs/guides/sealed-probes.md @@ -0,0 +1,282 @@ +--- +title: Sealed Probes +description: Run narrow, brokered Python scripts against private repositories without exposing repository contents to the primary agent. +--- + +A **sealed probe** lets an agent ask a trusted broker to run a short, agent-authored Python 3 script against a private repository and get back a single value conforming to a finite schema the agent declares up front -- without the agent ever seeing repository contents, receiving diagnostic output, or gaining network access to the repository. + +The feature is config-only: there are no `--sealed-probes-*` CLI flags. Everything is expressed in the AWF JSON configuration file. + +## Use cases + +Sealed probes are designed for **bounded, answerable questions** about a private repository where the question and its full range of answers can be expressed as a finite schema. + +**Good uses** + +- "Does this repository contain a `SECURITY.md` at the root?" -- boolean, 1 bit of payload +- "How many Python files are in `src/`?" -- bounded integer with a known upper limit +- "Which license identifier is declared: MIT, Apache-2.0, GPL-3.0, or something else?" -- small enum +- "Is the `requires-python` minimum in `pyproject.toml` at least 3.10?" -- boolean +- "Do both repositories declare the same major API version in their manifest?" -- each queried separately; answers compared by the agent after two probes + +**Not suited for** + +- Extracting source code, documentation, or any variable-length text -- unbounded strings are structurally impossible in the schema DSL +- Arbitrary repository exploration or browsing +- Tasks where the answer space cannot be described by a finite schema before the probe runs +- Repositories marked `sealed` (0-bit budget) -- these can never fund even the cheapest query + +:::note +Sealed probes bound *quantity* of information revealed, not *semantics*. Classifying a repository's sensitivity level is an operator responsibility; the feature enforces the declared limit but cannot validate that the classification is correct. +::: + +## Architecture + +The trust boundary operates in four stages: + +1. **Trusted host staging.** Before any container starts, AWF clones each configured repository using `GH_TOKEN`/`GITHUB_TOKEN`, strips all credentials, remotes, hooks, and write bits from the resulting seed, and records the resolved commit in trusted staging metadata. Submodules and gitdir pointers are rejected. The staging credential is scrubbed after this phase and never reaches the broker or agent. + +2. **Trusted broker over Unix socket.** A dedicated `awf-sealed-probe-broker` container with `network_mode: none` serves requests over a Unix socket mounted into the agent. It receives no network, no Squid proxy, and no external bridge. Its only connections are the Unix socket and the Docker socket (agent-invisible), used to launch probes. The broker holds the seed map -- including each repository's trusted sensitivity -- which the agent can never read or modify. + +3. **Fresh, no-network probe sandbox.** For each accepted request the broker creates a private writable copy of exactly one seed, then launches a single-use container with no network, a read-only root filesystem with bounded writable tmpfs mounts at `/tmp` and `/probe`, no capabilities, a restrictive seccomp profile, and fixed memory, CPU, PID, and timeout limits. The agent-authored script runs at `/awf/probe-script.py` and must write its result to `/probe/out`. Stdout, stderr, and exit status are discarded. + +4. **Canonical finite result and cleanup.** After the script exits, the broker validates the result file against the declared schema using a non-backtracking hand-written parser, re-serializes the canonical form, tears down the workspace, then -- only after cleanup completes -- selects the timing bucket and responds. The agent receives exactly `{"status":"ok","result":}` or `{"status":"error"}` with nothing else. + +## Configuration + +Add a `sealedProbes` section to your AWF JSON config file: + +```json +{ + "sealedProbes": { + "enabled": true, + "privateRepos": [ + { "repo": "my-org/private-service", "sensitivity": "internal" }, + { "repo": "my-org/public-docs", "sensitivity": "public" } + ], + "runtime": "docker", + "timeout": 30, + "memoryLimit": "512m", + "interpreter": "python3", + "maxInvocations": 32 + } +} +``` + +### Field reference + +| Field | Type | Constraints | Default | +|---|---|---|---| +| `enabled` | boolean | Only explicit `true` enables the feature; omission normalizes to `false` | `false` | +| `privateRepos` | array | Required non-empty when `enabled: true`; entries must be unique by slug (case-insensitive) | `[]` | +| `runtime` | string | `"docker"` or `"gvisor"` (gvisor requires `runsc` registered with the Docker daemon) | `"docker"` | +| `timeout` | integer | `1`-`540` seconds; the final 60 seconds before the 600-second bucket boundary are reserved for termination, validation, and cleanup | `30` | +| `memoryLimit` | string | Docker memory format, e.g. `"512m"`, `"1g"` | `"512m"` | +| `interpreter` | string | Only `"python3"` is currently supported | `"python3"` | +| `maxInvocations` | integer | `1`-`10000`; an independent operational cap unrelated to per-repository bit budgets | `32` | + +**`privateRepos` entry format.** Each entry must be an object: + +```json +{ "repo": "owner/repo", "sensitivity": "internal" } +``` + +The `sensitivity` value must be `public`, `internal`, `confidential`, or `sealed`. The `repo` value must be a bare `owner/repo` slug with no scheme, host, path traversal, query string, fragment, wildcard, or extra path segments. + +**Legacy bare strings.** For one release, a bare `"owner/repo"` string is accepted and normalized to `{ "repo": "...", "sensitivity": "internal" }` with a warning. New configuration should always use the object form so the intended sensitivity is explicit. + +**Disabled behavior.** When `enabled` is `false` or the section is absent, AWF stages nothing, starts no broker, mounts no socket, sets no environment variable, installs no CLI, and generates no skill. + +**Preflight failures** (all fail before the primary agent starts): `privateRepos` is empty, contains an invalid slug, or has duplicates; `runtime` is `"gvisor"` and `runsc` is not registered; the container runtime is a microVM backend (which cannot receive Compose bind mounts); the Docker host is not a `unix://` socket; `timeout` exceeds 540; no staging credential is present; or any seed cannot be materialized and verified. + +## Sensitivity categories + +Every repository carries a fixed sensitivity that sets an immutable maximum number of bits the broker may reveal about that repository across the entire AWF run. The budget is per-run only; the broker has no durable state across runs. + +| Sensitivity | Run budget | Notes | +|---|---|---| +| `public` | unmetered | Responses are never debited against a ledger, but are still schema- and operationally bounded | +| `internal` | 64 bits/run | Default for legacy bare-string entries | +| `confidential` | 8 bits/run | | +| `sealed` | 0 bits/run | Can never fund even the cheapest query; seed is staged and validated but Python is never launched | + +The minimum charge for any single invocation is 4 bits (see [information charge](#information-charge)), so a `confidential` repository can fund at most two questions before its budget is exhausted, and a `sealed` repository can never be queried. + +Sensitivity is set in AWF configuration only. The generated skill advertises each configured repository's sensitivity and initial run budget so the agent can design an affordable schema. A request cannot supply or override sensitivity, and the broker never exposes the remaining ledger balance. + +## Information charge + +Every accepted invocation is charged from its repository's run budget **before** any seed is copied or Python is launched. The charge is never refunded regardless of outcome. + +``` +charge = 1 (ok/error distinction is itself observable) + + ceil(log2(cardinality)) (the declared response schema) + + 3 (six timing buckets; ceil(log2(6)) = 3) +``` + +**Cardinality** is the number of distinguishable values the schema admits: 1 for `const`, 2 for `boolean`, N for an N-member `enum`, `max - min + 1` for `integer`, the product of field cardinalities for `object`/`tuple`/`array`, the sum of variant cardinalities for `union`. Cardinality is computed with `BigInt` arithmetic so it cannot overflow. + +**Cost examples** + +| Schema | Cardinality | charge | +|---|---|---| +| `{"type":"const","value":42}` | 1 | 1 + 0 + 3 = **4 bits** | +| `{"type":"boolean"}` | 2 | 1 + 1 + 3 = **5 bits** | +| `{"type":"enum","values":["MIT","Apache-2.0","GPL-3.0","unknown"]}` | 4 | 1 + 2 + 3 = **6 bits** | +| `{"type":"integer","minimum":0,"maximum":255}` | 256 | 1 + 8 + 3 = **12 bits** | + +An `internal` repository with a 64-bit budget can fund 12 consecutive boolean questions (60 bits), leaving 4 bits for one `const` question. If every query uses a `const` schema, it can fund 16 questions. + +`maxInvocations` is a separate, independent operational limit. It counts every response -- including those rejected by schema validation, budget exhaustion, or malformed requests -- and is unrelated to the bit ledger. + +## Timing buckets + +Probe response latency is itself a side channel: a script that exits early on one code path and runs longer on another leaks information through wall-clock time. The broker makes every launched invocation's observable response time land on one of six fixed boundaries: + +| Bucket | Boundary | +|---|---| +| 1 | 10 ms | +| 2 | 100 ms | +| 3 | 1 s | +| 4 | 10 s | +| 5 | 60 s | +| 6 | 600 s | + +The broker returns at the first bucket boundary at or after processing (execution + validation + container removal + workspace teardown) actually completes. Container and workspace cleanup are included in the measurement, so cleanup duration cannot be observed as a separate residual channel. + +**Scheduler tolerance.** A public 5 ms tolerance covers ordinary timer jitter. If a timer wakes more than 5 ms late or the selected boundary has already passed, the broker re-resolves to the next fixed boundary rather than responding at the late, continuously varying time. + +**Timing overflow.** If pathological infrastructure pushes total processing past the last bucket (600 s), the broker discards the result -- even a successful one -- and returns the canonical error. The 540-second timeout cap exists to preserve the final 60 seconds of the last bucket for cleanup. + +The three timing bits are charged as part of every accepted invocation's budget because latency alone is observable. + +## Agent interface + +When sealed probes are enabled the agent container receives: + +- A Unix socket directory (read-write) mounted at `$AWF_SEALED_PROBE_SOCKET` +- A generated skill file (read-only) at `$AWF_SEALED_PROBE_SKILL` +- `AWF_SEALED_PROBE_REPOS` -- a comma-separated list of configured repo slugs + +The generated skill lists each repository's configured sensitivity and initial run budget. It does not expose the broker's remaining ledger balance. + +GitHub tokens are removed from the agent environment whenever sealed probes are enabled, independently of the API and CLI proxies. + +The `sealed-probe` command is installed on the agent's `PATH` and is the only supported way to invoke a probe. + +### Invoking the `sealed-probe` command + +``` +sealed-probe --repo --schema '' < script.py +``` + +- `--repo` must appear exactly once. The value must be a valid `owner/repo` slug matching a configured repository. +- `--schema` must appear exactly once. The value is a JSON document (at most 4096 bytes) conforming to the finite schema DSL. +- The probe script arrives on **stdin**. Interactive terminals are rejected. +- Any other flag, the `--flag=value` form, and positional arguments are rejected without contacting the broker. + +The command always prints exactly one canonical JSON line to stdout, writes nothing to stderr, and exits with status 0 -- for both outcomes and for every failure, including transport failures. + +### Practical example + +Ask whether a repository contains a `SECURITY.md` at its root. Schema cardinality is 2, charge is 5 bits from the repository's run budget. + +```bash +sealed-probe \ + --repo my-org/private-service \ + --schema '{"type":"boolean"}' \ + <<'EOF' +import json, os + +result = os.path.isfile('/probe/repo/SECURITY.md') +with open('/probe/out', 'w') as f: + json.dump(result, f) +EOF +``` + +On success: + +```json +{"status":"ok","result":true} +``` + +On any failure (invalid repo, exhausted budget, script crash, timeout, non-conformant output, etc.): + +```json +{"status":"error"} +``` + +**Probe environment.** The script runs as an unprivileged user (uid 65534) with no network and a read-only filesystem, except for `/probe`. The repository tree is at `/probe/repo/`. The script must write exactly one JSON value conforming to the declared schema to `/probe/out`. Stdout and stderr are discarded and never reach the agent. + +## Finite response schema DSL + +The schema the agent declares is a closed algebra -- not general JSON Schema. Supported node types: + +| Type | Shape | Cardinality | +|---|---|---| +| `const` | `{"type":"const","value":}` | 1 | +| `boolean` | `{"type":"boolean"}` | 2 | +| `enum` | `{"type":"enum","values":[,...]}` | number of members | +| `integer` | `{"type":"integer","minimum":N,"maximum":M}` | M - N + 1 | +| `object` | `{"type":"object","fields":{"name":,...}}` | product of field cardinalities | +| `tuple` | `{"type":"tuple","items":[,...]}` | product of item cardinalities | +| `array` | `{"type":"array","items":,"length":N}` | item cardinality to the power N | +| `union` | `{"type":"union","variants":{"tag":,...}}` | sum of variant cardinalities; value is `{"tag":"","value":<...>}` | + +A literal (used in `const` and `enum`) must be a string (at most 64 bytes UTF-8, no control characters), a safe integer, a boolean, or `null`. All `enum` values must share the same JSON type and must be unique. + +There is no way to express an unbounded string, a float, a regex, recursion, `$ref`, an optional field, `additionalProperties`, or an untagged/overlapping union. These are structurally impossible to write in the DSL, not merely rejected by a validator. + +### Schema size limits + +| Bound | Value | +|---|---| +| Max serialized schema size | 4096 bytes | +| Max nesting depth | 6 | +| Max total schema nodes | 64 | +| Max `enum` values | 4096 | +| Max `object` fields | 16 | +| Max `tuple` items | 16 | +| Max fixed `array` length | 64 | +| Max `union` variants | 16 | +| Max literal string length | 64 bytes | + +In practice the 4096-byte size limit is the binding constraint for wide `enum` or `object` schemas well before the count limits are reached. + +### Validation and canonicalization + +The schema is validated **before** the broker copies a seed or launches Python. If the schema is structurally invalid the request is rejected immediately (canonical error) without touching the repository. + +After the script exits, the result file is parsed with a non-backtracking hand-written parser that rejects malformed JSON, duplicate object keys, leading or trailing content, and invalid UTF-8. The parsed value is then validated against the exact declared schema. A value that passes is canonically re-serialized before being wrapped in the response envelope -- the exact byte layout written by the probe (whitespace, key order) never reaches the agent. + +## Failure semantics + +All failure modes collapse to a single canonical response: + +```json +{"status":"error"} +``` + +Failures that map to this response include: invalid request format, schema validation failure, repo not in `privateRepos`, exhausted bit budget, exhausted `maxInvocations`, probe launch failure, timeout, script crash, non-conformant output, timing-bucket overflow, and internal broker errors. + +Failures are indistinguishable from each other by design: the agent cannot infer which failure mode occurred from the response alone. + +`maxInvocations` counts **every** response, including rejected requests. It is a separate operational limit unrelated to per-repository bit budgets. Once exhausted, all further requests return `{"status":"error"}` without consulting the bit ledger. + +Failure details (with protected labels such as `repo-not-allowed`, `bit-budget-exhausted`, `invalid-request`, `launch-failed`, `timing-bucket-overflow`, and `cleanup-failed`) are written only to the protected audit log at `/sealed-probes/audit/`. They are never returned to the agent. + +## Security limitations + +Sealed probes provide a **quantitative information bound**, not a semantic approval of disclosed content: + +- The bit budget limits how many bits of signal the broker may reveal, not whether any particular fact is sensitive. +- Timing is included in the charge and bucketed, but six bucket outcomes are still observable (3 bits). Repeated probes can reveal additional bucket outcomes, and each accepted invocation pays that timing charge. +- Agent-authored code is arbitrary Python within the sandbox. The sandbox enforces isolation, but a probe can compute and express any value that fits the declared schema. +- `public` repositories are unmetered. The schema and operational limits (`maxInvocations`, timeouts, sandboxing) still apply, but there is no bit ledger to exhaust. +- Budgets reset each AWF run. The broker has no durable identity or storage across runs. +- Classifying a repository's sensitivity level is an operator responsibility. Selecting a less restrictive category with a larger budget than warranted undermines the bound the feature provides. + +## See also + +- [Security Architecture](/gh-aw-firewall/reference/security-architecture) - Firewall trust model and isolation layers +- [AWF config spec section 14](https://github.com/github/gh-aw-firewall/blob/main/docs/awf-config-spec.md#14-sealed-probes) - Normative specification with full field constraints, protocol details, and staging implementation notes