Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The system is orchestrated by `src/cli.ts` and managed by `src/docker-manager.ts
**4. Bounded-Query Broker (optional)** — `containers/bounded-query/`, no network
- Enabled via `boundedQueries.enabled` in the AWF config file (config-only; there is no CLI flag family)
- The only AWF service with `network_mode: none`: no `awf-net`, no external bridge, no DNS, no Squid, no host gateway
- Reachable only through one Unix socket in `<workDir>/bounded-queries/run/`, bind-mounted into the agent at `/run/awf-bounded-query/broker.sock`
- Reachable only through one Unix socket in a run-specific `/var/tmp` ingress root, bind-mounted into the agent at `/run/awf-bounded-query/broker.sock`; all seeds, workspaces, maps, control state, and audits live in a disjoint broker-private `/var/tmp` root
- Receives the resolved Docker socket so it can launch per-invocation query containers; that path never enters the agent's env or volumes
- The broker (`bounded-query-broker`) and query sandbox (`bounded-query`) are separate published images; a one-shot networkless Compose service pulls the sandbox image before broker startup so the broker (which has no network) can launch query containers
- Queries run `python3` with `--network none`, `--read-only`, non-root, `--cap-drop ALL`, `no-new-privileges`, a seccomp profile, and time/memory/CPU/PID/file-size bounds
Expand Down
2 changes: 1 addition & 1 deletion containers/bounded-query/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ RUN chmod -R a-w /opt/awf \
&& node --check /opt/awf/broker/healthcheck.js

# Fixed broker-only mount points.
RUN mkdir -p /srv/awf/seeds /srv/awf/work /run/awf-bounded-query /var/log/awf-bounded-query
RUN mkdir -p /srv/awf/seeds /srv/awf/work /run/awf-bounded-query /run/awf-bounded-query-control /var/log/awf-bounded-query

# The broker is root only to copy host-owned read-only seeds into private
# workspaces and hand those workspaces to the unprivileged query uid.
Expand Down
6 changes: 4 additions & 2 deletions containers/bounded-query/broker/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ const WORK_DIR = '/srv/awf/work';
const SEED_MAP_PATH = '/srv/awf/seed-map.json';
const SOCKET_DIR = '/run/awf-bounded-query';
const SOCKET_PATH = path.join(SOCKET_DIR, 'broker.sock');
const CONTROL_DIR = '/run/awf-bounded-query-control';
const AUDIT_DIR = '/var/log/awf-bounded-query';
/** Broker-private readiness marker; the audit directory is never agent-mounted. */
const READY_PATH = path.join(AUDIT_DIR, 'broker.ready');
/** Broker-private readiness marker; the control directory is never agent-mounted. */
const READY_PATH = path.join(CONTROL_DIR, 'broker.ready');
const QUERY_SECCOMP_PATH = '/opt/awf/query-seccomp.json';

/** Mount points inside the query container. Fixed, never caller-supplied. */
Expand Down Expand Up @@ -84,6 +85,7 @@ function loadConfig() {
seedMapPath: SEED_MAP_PATH,
socketDir: SOCKET_DIR,
socketPath: SOCKET_PATH,
controlDir: CONTROL_DIR,
readyPath: READY_PATH,
auditDir: AUDIT_DIR,
querySeccompPath: QUERY_SECCOMP_PATH,
Expand Down
3 changes: 2 additions & 1 deletion containers/bounded-query/broker/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ async function main() {
await listenOnSocket(server, config, audit);

// Write the ready file AFTER the socket is accepting connections. The
// compose healthcheck polls this file in the broker-only audit mount.
// compose healthcheck polls this file in the broker-only control mount.
fs.mkdirSync(config.controlDir, { recursive: true, mode: 0o700 });
fs.writeFileSync(config.readyPath, '', { mode: 0o644 });

audit.lifecycle('listening', {
Expand Down
20 changes: 17 additions & 3 deletions docs/awf-config-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -1837,7 +1837,9 @@ runs a trusted host-side staging phase (`src/bounded-query/staging.ts`):
1. resolves the staging credential from `GH_TOKEN` or `GITHUB_TOKEN`;
2. clones each configured repository from an AWF-constructed
`https://github.com/<owner>/<repo>.git` URL into a run-unique, opaque seed
directory under `<workDir>/bounded-queries/seeds/`. The credential is passed
directory under a dedicated per-run private root outside `/tmp`, the
workspace, mounted home/tool directories, and configured agent mounts. The
credential is passed
only through a `GIT_ASKPASS` helper reading it from the child process
environment — never in argv, never in the URL, never in a log line, and
never in the generated compose file;
Expand Down Expand Up @@ -1906,7 +1908,7 @@ Query stdout/stderr is capped and discarded — never parsed, never returned,
never logged in a form reachable by the agent. Failure reasons (with
protected detail, e.g. `repo-not-allowed`, `bit-budget-exhausted`,
`invalid-request`, `query-launch-failed`, `timing-bucket-overflow`,
`cleanup-failed`) are written only to `<workDir>/bounded-queries/audit/`,
`cleanup-failed`) are written only below the dedicated broker-private root,
which is mounted into the broker alone.

### 14.8 Agent Interface
Expand Down Expand Up @@ -1934,7 +1936,7 @@ responsibilities are enforcing the fixed CLI shape, base64url-encoding the
schema into a request header, transporting the script body unmodified, and
passing the broker's response through unmodified.

The generated `SKILL.md` is written under `<workDir>/bounded-queries/agent/` and
The generated `SKILL.md` is written under the run-specific ingress root and
mounted read-only at `/run/awf-bounded-query-skill/SKILL.md`. It documents,
per configured repository, its sensitivity and run budget (e.g. `` `octo/alpha`
— 64 bits/run (`internal`) ``), the finite schema DSL, the bit-charge
Expand All @@ -1946,6 +1948,18 @@ or inside the checked-out workspace. Agents therefore discover it through
`AWF_BOUNDED_QUERY_SKILL` rather than through automatic skill discovery. This
is a documented limitation, not an oversight.

All seeds, invocation workspaces, the seed map, broker control state, and
protected audit data live below
`/var/tmp/awf-bounded-query-private-<uid>-<workDir digest>/`. Only the disjoint
`/var/tmp/awf-bounded-query-ingress-<uid>-<workDir digest>/run/` and generated
skill directory are agent-visible through explicit bind mounts. Before
credential-bearing staging, AWF resolves each path through
its longest existing ancestor (following symlinks) and rejects any private-root
overlap with the union of Docker, gVisor, and sbx agent-visible mounts,
including `/tmp`, the workspace, custom volumes, and whitelisted home tool
directories. Docker-in-Docker host-path translation is checked and applied to
the private broker mounts and ingress mounts symmetrically.

For the same reason, a microVM primary agent runtime (`sbx`) is rejected at
preflight: it does not receive Compose bind mounts, so the socket and skill
could not be exposed. Bounded queries are never partially enabled.
Expand Down
2 changes: 1 addition & 1 deletion docs/bounded-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ Failures are indistinguishable from each other by design: the agent cannot infer

`maxInvocations` counts **every** response, including rejected requests. It is a separate operational limit unrelated to per-repository bit budgets. Once exhausted, all further requests return `{"status":"error"}` without consulting the bit ledger.

Failure details (with protected labels such as `repo-not-allowed`, `bit-budget-exhausted`, `invalid-request`, `launch-failed`, `timing-bucket-overflow`, and `cleanup-failed`) are written only to the protected audit log at `<workDir>/bounded-queries/audit/`. They are never returned to the agent.
Failure details (with protected labels such as `repo-not-allowed`, `bit-budget-exhausted`, `invalid-request`, `launch-failed`, `timing-bucket-overflow`, and `cleanup-failed`) are written only below the dedicated broker-private root (`/var/tmp/awf-bounded-query-private-<uid>-<digest>/audit/`). The root is rejected before staging if realpath-aware preflight finds any overlap with a Compose, gVisor, or sbx agent mount. They are never returned to the agent.

## Security limitations

Expand Down
3 changes: 2 additions & 1 deletion src/artifact-preservation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import { logger } from './logger';
import { fixArtifactPermissionsForRootless } from './artifact-permissions';
import { getLocalDockerEnv } from './host-env';
import { resolveBoundedQueryPaths } from './bounded-query/paths';

const BOUNDED_QUERY_AUDIT_CONTAINER_PATH =
'awf-bounded-query-broker:/var/log/awf-bounded-query/bounded-query.jsonl';
Expand All @@ -16,21 +17,21 @@
*/
export function preserveIptablesAudit(workDir: string, auditDir?: string): void {
const iptablesAuditSrc = path.join(workDir, 'init-signal', 'iptables-audit.txt');
const boundedQueryRoot = path.join(workDir, 'bounded-queries');
const boundedQueryRoot = resolveBoundedQueryPaths(workDir).root;
const targetAuditDir = auditDir || path.join(workDir, 'audit');
if (!fs.existsSync(targetAuditDir)) return;

Check warning on line 22 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 22 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 22 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found existsSync from package "fs" with non literal argument at index 0

if (fs.existsSync(iptablesAuditSrc)) {

Check warning on line 24 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 24 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 24 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found existsSync from package "fs" with non literal argument at index 0
try {
fs.copyFileSync(iptablesAuditSrc, path.join(targetAuditDir, 'iptables-audit.txt'));
fs.chmodSync(path.join(targetAuditDir, 'iptables-audit.txt'), 0o644);

Check warning on line 27 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found chmodSync from package "fs" with non literal argument at index 0

Check warning on line 27 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found chmodSync from package "fs" with non literal argument at index 0

Check warning on line 27 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found chmodSync from package "fs" with non literal argument at index 0
logger.debug('Copied iptables audit state to audit directory');
} catch (error) {
logger.debug('Could not copy iptables audit file:', error);
}
}

if (fs.existsSync(boundedQueryRoot)) {

Check warning on line 34 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 34 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 34 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found existsSync from package "fs" with non literal argument at index 0
try {
const destination = path.join(targetAuditDir, 'bounded-query.jsonl');
const result = execa.sync(
Expand Down Expand Up @@ -78,7 +79,7 @@
}: PreserveDirectoryOptions): void {
if (runtimeDir) {
const targetDir = runtimeSubdir ? path.join(runtimeDir, runtimeSubdir) : runtimeDir;
if (fs.existsSync(targetDir)) {

Check warning on line 82 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 82 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 82 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found existsSync from package "fs" with non literal argument at index 0
try {
execa.sync('chmod', ['-R', 'a+rX', targetDir]);
logger.info(`${availableLabel} available at: ${targetDir}`);
Expand All @@ -91,7 +92,7 @@

const sourceDir = path.join(workDir, workSubdir);
const destinationDir = path.join(os.tmpdir(), `${destinationBaseName}-${timestamp}`);
if (fs.existsSync(sourceDir) && fs.readdirSync(sourceDir).length > 0) {

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found readdirSync from package "fs" with non literal argument at index 0

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / ESLint

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found readdirSync from package "fs" with non literal argument at index 0

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 22)

Found existsSync from package "fs" with non literal argument at index 0

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found readdirSync from package "fs" with non literal argument at index 0

Check warning on line 95 in src/artifact-preservation.ts

View workflow job for this annotation

GitHub Actions / Build and Lint (Node 20)

Found existsSync from package "fs" with non literal argument at index 0
try {
fs.renameSync(sourceDir, destinationDir);
if (chmodPreservedDir) {
Expand Down
99 changes: 92 additions & 7 deletions src/bounded-query/manager.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,9 @@ describe('prepareBoundedQueries', () => {
});

afterEach(() => {
releaseSeedPermissions(resolveBoundedQueryPaths(workDir).seedsDir);
const paths = resolveBoundedQueryPaths(workDir);
releaseSeedPermissions(paths.seedsDir);
fs.rmSync(paths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
});

Expand All @@ -88,7 +90,9 @@ describe('prepareBoundedQueries', () => {
expect(fs.existsSync(paths.workDir)).toBe(true);
expect(fs.existsSync(paths.runDir)).toBe(true);
expect(fs.existsSync(paths.auditDir)).toBe(true);
expect(fs.existsSync(paths.controlDir)).toBe(true);
expect(fs.existsSync(paths.skillPath)).toBe(true);
expect(paths.root.startsWith(workDir)).toBe(false);

const seedMap = JSON.parse(fs.readFileSync(paths.seedMapPath, 'utf8'));
expect(seedMap.version).toBe(2);
Expand Down Expand Up @@ -155,6 +159,26 @@ describe('prepareBoundedQueries', () => {
}
});

it('rejects a pre-existing private root instead of reusing attacker-controlled state', async () => {
const paths = resolveBoundedQueryPaths(workDir);
fs.mkdirSync(paths.root);
await expect(prepareBoundedQueries(buildConfig(workDir), { env: { GH_TOKEN: 't' }, gitRunner }))
.rejects.toThrow(/EEXIST|file already exists/);
});

it('rejects a pre-existing ingress root instead of following a planted symlink', async () => {
const paths = resolveBoundedQueryPaths(workDir);
const target = fs.mkdtempSync(path.join('/var/tmp', 'awf-bounded-query-ingress-target-'));
fs.symlinkSync(target, paths.ingressRoot);
try {
await expect(prepareBoundedQueries(buildConfig(workDir), { env: { GH_TOKEN: 't' }, gitRunner }))
.rejects.toThrow(/EEXIST|file already exists/);
} finally {
fs.rmSync(paths.ingressRoot, { force: true });
fs.rmSync(target, { recursive: true, force: true });
}
});

it('aborts when a seed cannot be staged', async () => {
const failing: GitRunner = async () => {
throw new Error('fatal: repository not found');
Expand All @@ -178,19 +202,21 @@ describe('teardownBoundedQueries', () => {

it('restores seed write permissions so generic cleanup can remove them', async () => {
const workDir = fs.mkdtempSync(path.join(os.tmpdir(), 'awf-bounded-query-teardown-'));
const paths = resolveBoundedQueryPaths(workDir);
try {
await prepareBoundedQueries(buildConfig(workDir), { env: { GH_TOKEN: 't' }, gitRunner });
const paths = resolveBoundedQueryPaths(workDir);

expect(() => fs.rmSync(paths.seedsDir, { recursive: true })).toThrow();

// No query containers exist for this run, so the docker lookup is a
// no-op; the permission restore is what must happen.
await teardownBoundedQueries(buildConfig(workDir));

expect(() => fs.rmSync(paths.seedsDir, { recursive: true })).not.toThrow();
expect(fs.existsSync(paths.root)).toBe(false);
expect(fs.existsSync(paths.ingressRoot)).toBe(false);
} finally {
releaseSeedPermissions(resolveBoundedQueryPaths(workDir).seedsDir);
releaseSeedPermissions(paths.seedsDir);
fs.rmSync(paths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
}
});
Expand All @@ -205,7 +231,9 @@ describe('teardownBoundedQueries', () => {

expect(() => fs.rmSync(paths.seedsDir, { recursive: true })).toThrow();
} finally {
releaseSeedPermissions(resolveBoundedQueryPaths(workDir).seedsDir);
const cleanupPaths = resolveBoundedQueryPaths(workDir);
releaseSeedPermissions(cleanupPaths.seedsDir);
fs.rmSync(cleanupPaths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
}
});
Expand Down Expand Up @@ -253,6 +281,7 @@ describe('teardownBoundedQueries', () => {
await teardownBoundedQueries(buildConfig(workDir));
expect(mockExeca).not.toHaveBeenCalled();
} finally {
fs.rmSync(paths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
}
});
Expand All @@ -264,9 +293,11 @@ describe('teardownBoundedQueries', () => {
mockExeca.mockRejectedValueOnce(new Error('docker unavailable'));

await expect(teardownBoundedQueries(buildConfig(workDir))).resolves.toBeUndefined();
expect(() => fs.rmSync(resolveBoundedQueryPaths(workDir).seedsDir, { recursive: true })).not.toThrow();
expect(fs.existsSync(resolveBoundedQueryPaths(workDir).root)).toBe(false);
} finally {
releaseSeedPermissions(resolveBoundedQueryPaths(workDir).seedsDir);
const cleanupPaths = resolveBoundedQueryPaths(workDir);
releaseSeedPermissions(cleanupPaths.seedsDir);
fs.rmSync(cleanupPaths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
}
});
Expand All @@ -285,7 +316,61 @@ describe('teardownBoundedQueries', () => {
await expect(teardownBoundedQueries(buildConfig(workDir))).resolves.toBeUndefined();
expect(mockReleaseSeedPermissions).toHaveBeenCalledWith(paths.seedsDir);
} finally {
fs.rmSync(paths.root, { recursive: true, force: true });
fs.rmSync(workDir, { recursive: true, force: true });
}
});

it('repairs rootless private-state permissions and retries cleanup', () => {
const paths = resolveBoundedQueryPaths('/tmp/rootless-cleanup');
const permissionError = Object.assign(new Error('permission denied'), { code: 'EACCES' });
const removeTree = jest.fn()
.mockImplementationOnce(() => { throw permissionError; })
.mockImplementation(() => undefined);
const repairPermissions = jest.fn();

managerTestHelpers.removePrivateState(
buildConfig('/tmp/rootless-cleanup'),
paths,
{ removeTree, repairPermissions },
);

expect(repairPermissions).toHaveBeenCalledWith(
[paths.root, paths.ingressRoot],
undefined,
undefined,
undefined,
undefined,
);
expect(removeTree).toHaveBeenCalledTimes(3);
});

it('surfaces cleanup failures after rootless permission repair', () => {
const paths = resolveBoundedQueryPaths('/tmp/rootless-retry-failure');
const permissionError = Object.assign(new Error('permission denied'), { code: 'EACCES' });
const removeTree = jest.fn()
.mockImplementationOnce(() => { throw permissionError; })
.mockImplementationOnce(() => { throw new Error('still denied'); });

expect(() => managerTestHelpers.removePrivateState(
buildConfig('/tmp/rootless-retry-failure'),
paths,
{ removeTree, repairPermissions: jest.fn() },
)).not.toThrow();
});

it('surfaces non-permission cleanup failures without attempting repair', () => {
const paths = resolveBoundedQueryPaths('/tmp/private-cleanup-failure');
const repairPermissions = jest.fn();

expect(() => managerTestHelpers.removePrivateState(
buildConfig('/tmp/private-cleanup-failure'),
paths,
{
removeTree: () => { throw new Error('I/O failure'); },
repairPermissions,
},
)).not.toThrow();
expect(repairPermissions).not.toHaveBeenCalled();
});
});
Loading
Loading