feat: add confidentiality-aware sealed probe budgets - #6728
Conversation
Add repository sensitivity categories, finite response schemas, and per-run bit ledgers. Canonically validate results and charge response timing against each repository's run budget. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ca8d9d74-46ab-48db-b05f-640cbc6d47be
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 99.13% | 98.80% | 📉 -0.33% |
| Statements | 99.03% | 98.35% | 📉 -0.68% |
| Functions | 99.14% | 98.97% | 📉 -0.17% |
| Branches | 95.69% | 94.73% | 📉 -0.96% |
📁 Per-file Coverage Changes (2 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/sealed-probe/protocol.ts |
100.0% → 92.0% (-8.03%) | 99.4% → 86.6% (-12.78%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
There was a problem hiding this comment.
Pull request overview
Adds confidentiality-aware disclosure budgets and protocol v2 to the sealed-probe subsystem.
Changes:
- Adds repository sensitivity profiles, trusted seed metadata, and per-run ledgers.
- Introduces finite response schemas, cardinality accounting, and canonical result validation.
- Adds response-time bucketing with expanded tests and documentation.
Show a summary per file
| File | Description |
|---|---|
src/types/sealed-probe-options.ts |
Defines sensitivities, budgets, and repository descriptors. |
src/types/index.ts |
Exports new sealed-probe types and constants. |
src/services/sealed-probe-service.ts |
Adapts agent repository environment generation. |
src/services/sealed-probe-service.test.ts |
Updates service tests for descriptors. |
src/services/sealed-probe-compose.test.ts |
Updates Compose fixtures. |
src/services/agent-environment/excluded-vars.test.ts |
Updates credential-isolation fixtures. |
src/sealed-probe/wrapper.test.ts |
Tests protocol-v2 wrapper framing. |
src/sealed-probe/workflow-integration.test.ts |
Updates workflow integration configuration. |
src/sealed-probe/types.ts |
Adds sensitivity to seed-map v2. |
src/sealed-probe/staging.ts |
Carries sensitivity through staging. |
src/sealed-probe/staging.test.ts |
Tests sensitivity preservation. |
src/sealed-probe/skill.ts |
Documents schemas, budgets, and timing. |
src/sealed-probe/skill.test.ts |
Tests generated skill guidance. |
src/sealed-probe/scheduler.test.ts |
Tests timing-bucket scheduling. |
src/sealed-probe/protocol-parity.test.ts |
Expands protocol implementation parity tests. |
src/sealed-probe/preflight.ts |
Adds timing-related timeout validation. |
src/sealed-probe/preflight.test.ts |
Tests repository and timeout preflight. |
src/sealed-probe/manager.ts |
Writes sensitivity into seed maps. |
src/sealed-probe/manager.test.ts |
Tests seed-map v2 generation. |
src/sealed-probe/ledger.test.ts |
Tests per-repository budget accounting. |
src/sealed-probe/end-to-end.test.ts |
Exercises protocol v2 end to end. |
src/sealed-probe/broker.test.ts |
Expands broker security and timing tests. |
src/parsers/sealed-probe-parser.ts |
Normalizes descriptor and legacy configurations. |
src/parsers/sealed-probe-parser.test.ts |
Tests normalization and warnings. |
src/config-file.ts |
Updates raw configuration types. |
src/config-file-sealed-probes-validation.test.ts |
Tests descriptor schema validation. |
src/commands/build-config.test.ts |
Updates normalized configuration expectations. |
src/awf-config-schema.json |
Adds sensitivity descriptors and timeout bounds. |
docs/awf-config.schema.json |
Updates published configuration schema. |
docs/awf-config-spec.md |
Documents protocol v2 and budgets. |
containers/sealed-probe/broker/server.js |
Integrates canonical protocol-v2 responses. |
containers/sealed-probe/broker/sensitivity.js |
Defines broker-side sensitivity budgets. |
containers/sealed-probe/broker/scheduler.js |
Implements response-time bucketing. |
containers/sealed-probe/broker/protocol.js |
Implements finite schemas and validation. |
containers/sealed-probe/broker/ledger.js |
Implements run-budget accounting. |
containers/sealed-probe/broker/framing.js |
Implements protocol-v2 request framing. |
containers/sealed-probe/broker/config.js |
Loads seed-map v2 and timeout constraints. |
containers/sealed-probe/broker/broker.js |
Orchestrates validation, charging, and scheduling. |
containers/agent/sealed-probe-wrapper.sh |
Updates the agent CLI to protocol v2. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (5)
containers/sealed-probe/broker/broker.js:239
- Queue time is outside the timing bucket because
startMsis recorded only whenexecutebegins aftertail. With concurrent calls, the second caller observes the first probe's secret-dependent execution and cleanup duration before its own bucket starts (and can hit the wrapper's 660-second timeout while queued), creating more than the six charged timing outcomes. Concurrent requests must be rejected without later execution or their entire client-visible queue time must be included in a bounded, charged schedule.
const queued = tail.then(() => execute(request, safeRespond)).catch((error) => {
audit.failure('queue', 'unexpected-error', error && error.message);
safeRespond(CANONICAL_ERROR_JSON);
});
tail = queued.then(
() => undefined,
() => undefined,
);
containers/sealed-probe/broker/protocol.js:287
- An accepted depth-6 schema can make this exponentiation materialize an enormous
BigIntbefore any budget comparison. For example, six nested length-64 arrays overbooleanhave cardinality2^(64^6), requiring roughly 16 GiB just for the integer; an untrusted agent can therefore block or OOM the broker with a tiny schema. Compute the bit width symbolically (and cap as soon as it exceeds every finite ledger) rather than materializing full cardinalities.
case 'array':
return schemaCardinality(schema.items) ** BigInt(schema.length);
containers/sealed-probe/broker/protocol.js:493
- Using a normal object makes
__proto__bypass both duplicate detection and extra-key validation: assigning that key changes the object's prototype without creating an own property. For example, repeated__proto__keys are accepted, and a result can carry a hidden__proto__object whileObject.keysstill matches the schema. Build parsed objects with a null prototype (in both protocol mirrors) so every JSON key is represented and checked.
function parseJsonObject(text, index, depth) {
let i = skipJsonWhitespace(text, index + 1);
const obj = {};
if (text[i] === '}') return { value: obj, endIndex: i + 1 };
for (;;) {
i = skipJsonWhitespace(text, i);
const key = parseJsonStringLiteral(text, i);
if (!key) return undefined;
i = skipJsonWhitespace(text, key.endIndex);
if (text[i] !== ':') return undefined;
i = skipJsonWhitespace(text, i + 1);
const value = parseJsonValue(text, i, depth + 1);
if (!value) return undefined;
if (Object.prototype.hasOwnProperty.call(obj, key.value)) return undefined;
obj[key.value] = value.value;
containers/sealed-probe/broker/protocol.js:458
- Converting the numeric lexeme to
Numberloses whether it was actually an integer before schema validation. Inputs such as1.0000000000000001round to1, soNumber.isIntegerlater accepts a floating result (and the same parser can normalize floating schema bounds into integers), contrary to the finite integer contract. Preserve/validate the exact numeric lexeme before rounding in both protocol implementations.
const raw = text.slice(start, i);
const value = Number(raw);
if (!Number.isFinite(value)) return undefined;
return { value, endIndex: i };
containers/sealed-probe/broker/broker.js:186
- The bucket wait completes before the protected audit record is written, so the response is sent after additional synchronous, outcome-dependent filesystem work. Audit serialization/write latency (including different success/failure records and filesystem stalls) therefore shifts the observable response off its charged boundary. Complete all audit work before selecting/waiting for the final boundary, or perform another bounded scheduling step afterward.
if (overflowed) {
// Fail closed: processing (not the script itself, which is bounded by
// `sealedProbes.timeout <= largest bucket`) overran every configured
// bucket — pathological infrastructure latency. Never emit a
// successful result at unbucketed timing.
audit.failure(invocationId, 'timing-bucket-overflow', failureReason ? failureReason.join(':') : undefined);
safeRespond(CANONICAL_ERROR_JSON);
} else if (canonicalResult !== undefined) {
audit.invocation({
invocationId,
repo: privateRepo,
sensitivity: seed.sensitivity,
bits: charge,
bucketMs,
});
safeRespond(canonicalOkJson(canonicalResult));
} else {
audit.failure(invocationId, failureReason ? failureReason[0] : 'unknown', failureReason ? failureReason[1] : undefined);
safeRespond(CANONICAL_ERROR_JSON);
- Files reviewed: 41/41 changed files
- Comments generated: 6
- Review effort level: Medium
lpcox
changed the title
Reserve final-bucket cleanup time and rebucket late responses. Count malformed requests and clean partial workspaces. Align broker policy mirrors and restore coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ca8d9d74-46ab-48db-b05f-640cbc6d47be
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (2 files)
Coverage comparison generated by |
|
✅ Smoke Claude passed |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
🔌 Smoke Services — All services reachable! ✅ |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
|
📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤 |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed... |
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ |
|
✅ Build Test Suite completed successfully! |
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed... |
|
❌ Security Guard failed. Please review the logs for details. |
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
❌ Contribution Check failed. Please review the logs for details. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤 |
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
🔌 Smoke Services — All services reachable! ✅ |
|
✅ Smoke Claude passed |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed... |
|
❌ Security Guard failed. Please review the logs for details. |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed... |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
@lpcox Network isolation egress smoke test results: EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (github.com) reachable — Overall status: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
Smoke Test Results
Recent merged PRs:
Overall: PASS cc @lpcox
|
Smoke Test: Claude Engine Validation
Overall result: PASS
|
Smoke Test: GitHub Actions Services Connectivity
Overall: FAIL —
|
|
Smoke Test: Copilot BYOK (Direct) Mode
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com Overall: PASS cc @lpcox
|
|
Smoke Test: API Proxy OpenTelemetry Tracing — all scenarios passed ✅
Summary: 5/5 scenarios pass. No issues found.
|
Chroot Version Comparison Results
Overall: FAILED — Node.js version mismatch between host and chroot environments.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS Note: Java tests required overriding Maven local repo path (
|
|
Smoke Test: Docker Sbx Validation
Overall: PASS cc @lpcox
|
|
Smoke test:
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
Smoke Test Results
Overall Status: FAIL
|
Summary
public,internal,confidential, orsealedBigIntcardinality accounting, strict result validation, and broker-side canonical serializationmaxInvocationsas an independent operational limit and provide one-release compatibility for legacy repository stringsValidation
npm run type-checknpm run lint— no errorsnpm run test:unit -- --runInBand— 275 suites, 4,783 testsCloses #6725