UiPath · CarlesUIPath · Aug 5, 2026
@@ -107,3 +107,4 @@ Deferred lint/test guardrails surfaced during reviews. Promote to a `CExxx` rule
   caught them. The cleanup plan explicitly deferred this as YAGNI for the
   one-time purge, but any future doc rename/deletion re-opens the same blind
   spot — caught in the 2026-07-03 open-source-docs-cleanup implementation run.
+- [ ] CE033 form (c): a raw `.model_dump(` feeding the AGENT-READABLE `task.yaml` write in docker_runner must route through `agent_safe_dump` instead — no lint check today because distinguishing the stripped `task.yaml` write from the legitimate root-only `task_full.json` `model_dump` in the same function needs data-flow analysis, not a single-node AST match. Codebase currently compliant (task.yaml uses agent_safe_dump). — caught in docker-isolation-user-separation Phase 5.
@@ -137,7 +137,7 @@ jobs:
       - name: Run test suite with coverage
         run: |
           .venv/bin/pytest tests/ -v \
-            -m "not live and not lint" \
+            -m "not live and not lint and not docker_root" \
             --cov=coder_eval \
             --cov-report=term-missing \
             --cov-report=xml \
@@ -277,7 +277,7 @@ jobs:
         run: .venv/Scripts/pyright
 
       - name: Run test suite
-        run: .venv/Scripts/pytest tests/ -v -m "not live and not lint"
+        run: .venv/Scripts/pytest tests/ -v -m "not live and not lint and not docker_root"
 
       - name: Set up Node.js 20
         uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020  # v4.4.0
@@ -324,6 +324,34 @@ jobs:
           path: runs/win-smoke/
           retention-days: 7
 
+  docker-isolation:
+    name: Docker user/permission isolation (root-in-container EACCES)
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd  # v6.0.2
+
+      - name: Set up Python 3.13
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405  # v6.2.0
+        with:
+          python-version: "3.13"
+
+      - name: Install uv
+        run: |
+          python -m pip install --upgrade pip
+          pip install uv
+
+      - name: Build coder-eval-agent base Docker image
+        run: make docker-image
+
+      # Authoritative environment for the isolation acceptance proof: native Linux
+      # overlayfs + real root inside the container (no Docker-Desktop uid remap).
+      # Runs pytest -m docker_root as root inside the built image; the six-surface
+      # EACCES-as-agent-uid test executes here (it hard-skips off-root elsewhere).
+      - name: Run docker isolation EACCES suite (root-in-container)
+        run: make test-docker-isolation
+
   e2e-smoke:
     name: E2E Smoke Tests (Real API)
     runs-on: ubuntu-latest

@@ -1,4 +1,4 @@
-.PHONY: help install format check typecheck test test-live test-smoke verify verify-noextra clean run lint docs-indexes docker-image docker-image-full coder-eval-runtime docker-images
+.PHONY: help install format check typecheck test test-live test-smoke verify verify-noextra clean run lint docs-indexes docker-image docker-image-full coder-eval-runtime docker-images test-docker-isolation
 
 # Single source of the installed coder-eval version (used to tag the docker
 # images). Referenced lazily inside the docker recipes, so it doesn't run on
@@ -37,7 +37,7 @@ typecheck:  ## Run type checking with pyright
 	uv run pyright
 
 test:  ## Run test suite (excludes live + lint tests; run `make lint` for those)
-	uv run pytest -n auto -m "not live and not lint" tests/
+	uv run pytest -n auto -m "not live and not lint and not docker_root" tests/
 
 test-live:  ## Run live-only tests (real Anthropic API + claude CLI; requires ANTHROPIC_API_KEY)
 	uv run pytest -m live tests/ -v
@@ -54,7 +54,7 @@ verify:  ## Run all verification steps (CI equivalent)
 	uv run pytest tests/test_custom_lint.py -v --tb=short --no-header -p no:warnings
 	# uv run pip-audit --desc --skip-editable
 	# uv run bandit -r src/ -ll --format json -o bandit-report.json
-	uv run pytest tests/ -n auto -m "not live and not lint" --cov=coder_eval --cov-report=term-missing --cov-report=xml --cov-fail-under=80
+	uv run pytest tests/ -n auto -m "not live and not lint and not docker_root" --cov=coder_eval --cov-report=term-missing --cov-report=xml --cov-fail-under=80
 
 verify-noextra:  ## Verify the framework works without the optional [uipath] extra
 	# Build a throwaway venv that has ONLY the [dev] extra (no [uipath]); confirms
@@ -100,6 +100,13 @@ coder-eval-runtime:  ## Build the relocatable runtime kit image (COPY --from sou
 docker-images: docker-image coder-eval-runtime  ## Build BOTH base images (agent for rebase + runtime kit for inject); no creds
 	@echo "Built coder-eval-agent + coder-eval-runtime — ready for both rebase and inject tasks."
 
+test-docker-isolation:  ## Run the root-in-container EACCES isolation suite inside the built agent image
+	@VERSION=$$($(VERSION_CMD)); \
+	echo "Running docker user/permission isolation tests as root inside coder-eval-agent:$$VERSION"; \
+	docker run --rm --entrypoint "" -v $(PWD):/src -w /src coder-eval-agent:$$VERSION \
+		sh -c "uv pip install --system -q pytest pytest-asyncio pytest-mock pytest-cov >/dev/null 2>&1 || true; \
+			python -m pytest tests/test_docker_user_separation.py -m docker_root -p no:cacheprovider -o addopts='' -v"
+
 docker-image-full:  ## Build with the UiPath extra (opt-in; uipath resolves from public PyPI, no credentials needed). Codex is always baked in.
 	@VERSION=$$($(VERSION_CMD)); \
 	echo "Building coder-eval-agent:$$VERSION (full: + uipath extra)"; \

@@ -17,14 +17,37 @@ ENV DEBIAN_FRONTEND=noninteractive \
     PIP_NO_CACHE_DIR=1
 
 # System deps: git for repo-source templates; curl/ca-certs for HTTPS;
-# build-essential because some Python deps (pylint plugins) compile.
+# build-essential because some Python deps (pylint plugins) compile;
+# util-linux for `setpriv` (the user/permission isolation drop-privilege shim).
 RUN apt-get update && apt-get install -y --no-install-recommends \
         ca-certificates \
         curl \
         git \
         build-essential \
+        util-linux \
     && rm -rf /var/lib/apt/lists/*
 
+# Unprivileged agent user for the docker user/permission isolation barrier. The
+# in-container entrypoint stays root (grading needs it) and drops only the
+# agent-under-test's CLI subprocess to this uid via the setpriv shim below. The
+# uid/gid are the SSOT literals in src/coder_eval/models/container_paths.py
+# (AGENT_UID/AGENT_GID = 2000); a drift-guard test asserts they match. No `USER`
+# directive is set — the container process must remain root.
+ARG AGENT_UID=2000
+ARG AGENT_GID=2000
+# -m -d /home/agent bakes an agent-owned 0755 HOME so the dropped CLI's HOME can
+# point somewhere agent-writable (~/.claude etc.) instead of root's 0700 /root.
+# The /home/agent literal is the SSOT AGENT_HOME in container_paths.py; a
+# drift-guard test asserts they match.
+RUN groupadd -g ${AGENT_GID} agent \
+    && useradd -u ${AGENT_UID} -g ${AGENT_GID} -m -d /home/agent -s /usr/sbin/nologin agent \
+    && chmod 0755 /home/agent
+
+# Drop-privilege shim (mirrors CONTAINER_DROP_SHIM in container_paths.py). Baked
+# so the codex + antigravity spawn wiring can route their CLI binary through it.
+COPY docker/coder_eval_drop_privilege.sh /usr/local/bin/coder_eval_drop_privilege.sh
+RUN chmod +x /usr/local/bin/coder_eval_drop_privilege.sh
+
 # Node LTS + the Claude Code CLI, pinned. The agent binary is a dominant
 # non-model driver of eval results, so it travels with the coder_eval release
 # tag and is bumped deliberately -- mirrors the codex CLI pin

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+# Drop-privilege shim (SSOT for the docker user/permission isolation barrier).
+#
+# Runs its argv as the unprivileged `agent` uid baked into the image. The
+# container entrypoint stays root (grading needs it); only the agent-under-test's
+# CLI subprocess is routed through this shim so it (and every tool it spawns)
+# executes as `agent:agent` and gets EACCES on the root-0700 grading material.
+#
+# Reused by the codex (launch_args_override) and antigravity (PATH-shadow) spawn
+# wiring. The `agent` user is defined in docker/Dockerfile (useradd -u 2000);
+# the path here is mirrored by CONTAINER_DROP_SHIM in
+# src/coder_eval/models/container_paths.py.
+set -euo pipefail
+exec setpriv --reuid=agent --regid=agent --clear-groups -- "$@"
@@ -10,6 +10,13 @@
 # Forwards any args through to `coder-eval _run-task-internal` (the host appends
 # `--output`/`--task-dir`). For manual debugging, pass the same flag:
 #   docker run --rm --entrypoint /usr/local/bin/coder_eval_entrypoint.sh <image> --input /tmp/foo
+#
+# User/permission isolation barrier: this entrypoint (and _run-task-internal) runs
+# as ROOT — grading needs it. All lock/chown of grading material and the per-agent
+# uid drop live in Python (cli/run_task_internal_command.py), NOT here. SSOT for
+# the barrier constants (mirrored from src/coder_eval/models/container_paths.py):
+#   AGENT_UID = AGENT_GID = 2000, AGENT_USERNAME = "agent"
+#   CONTAINER_DROP_SHIM = /usr/local/bin/coder_eval_drop_privilege.sh
 set -euo pipefail
 
 exec coder-eval _run-task-internal "$@"
@@ -307,3 +307,134 @@ The host's `DockerRunner` (`coder_eval/isolation/docker_runner.py`) renders the
 Inside the container, the entrypoint invokes `coder-eval _run-task-internal` (hidden subcommand), which loads the staged YAML + context, runs the standard in-process Orchestrator (driver auto-coerced back to `tempdir`), and writes `task.json` to the output mount. Host reads it and feeds the existing aggregation pipeline.
 
 A `result_kind` discriminator on `CriterionResult` ensures `ClassificationCriterionResult` subclasses survive the JSON round-trip — without it, host-side aggregation would silently lose `observed_label`/`expected_label`.
+
+## Grading-material isolation (user/permission barrier)
+
+Under `driver: docker`, a scavenging agent can otherwise read its own grading
+rubric off the mounted inputs (`/work/input/task.yaml`, the skills-repo `tests/`
+tree, `check_*.py` graders, `RESOLUTION.md`). Coder Eval closes this with a
+**Unix DAC permission barrier**: all grading material is root-owned mode-0700 and
+the agent-under-test's CLI subprocess runs as a **dedicated unprivileged uid**
+(`agent`, uid/gid 2000, baked into the image). Grading (`SuccessChecker`,
+`run_command`, judges) stays in the container's **root** process, which ignores
+DAC and reads the locked material via `$TASK_DIR`/`$SKILLS_REPO_PATH`.
+
+### What the agent uid can and cannot read
+
+| Path | Owner/mode | Agent uid |
+|---|---|---|
+| `/work/input` (`task.yaml`, `task_full.json`, `context.json`) | root 0700 | **EACCES** |
+| skills-repo / plugin mount (`tests/`, `check_*.py`, `reference_agents/`, `RESOLUTION.md`) | root 0700 | **EACCES** |
+| per-task-dir mount | root 0700 | **EACCES** |
+| reference-solution mount (absolute/escaping `reference.file`/`reference.directory`) | root 0700 | **EACCES** |
+| `/proc/1/environ` (root PID1) | kernel-restricted | **EACCES** |
+| `/work/skills` (world-readable skill-DOCS copy: docs/commands/skills only) | agent | readable |
+| `/work/output/artifacts/<task>` (its workspace) | agent | read/write |
+
+**`/work/output` is NOT locked during the turn.** It is a bind mount *shared with
+the host*, which writes the liveness heartbeat there as a non-root uid; a
+root-0700 lock would make the heartbeat unwritable and self-reap the container. So
+`/work/output` is world-traversable and the agent's own `artifacts/<task>`
+subdir is agent-owned. The real mitigation for the grading artifact is temporal,
+not permission-based: **`task.json` is written only AFTER the agent turn ends** (it
+is not a live read surface during the turn), and its `source_yaml` is **nulled** in
+the agent-visible context (the raw YAML rides on the root-only `task_full.json`
+instead). A root-0700 file *placed* under `/work/output` IS EACCES to the agent
+(the lock mechanism works there) — `/work/output` is simply not blanket-locked.
+
+The `/work/input`, per-task-dir, and skills-repo/plugin mounts are bind-mounted
+**read-write** (not `:ro`) precisely so the in-container root entrypoint's
+root-0700 `chmod`/`chown` lock applies: an `os.chmod` on a `:ro` bind mount fails
+with `EROFS` and would silently leave the material agent-readable. The lock denies
+the dropped agent uid; the agent still cannot write (0700-root), and grading runs
+as root.
+
+The agent-readable `task.yaml` is additionally **criteria-stripped**
+(`TaskDefinition.agent_safe_dump()` — defence in depth); the full criteria travel
+in a root-only `task_full.json` sibling the entrypoint merges back before grading.
+`agent_safe_dump` strips **only** `success_criteria` and `reference` — every other
+field (`initial_prompt`, `system_prompt`, pre/post commands, `metadata`) survives
+into the agent-readable `task.yaml`, so grading material must never be authored
+into those fields. The skill-DOCS copy carries only the plugin-discovery subtrees
+(`PLUGIN_AGENT_ALLOWED_SUBDIRS`), never grader/reference/fixture trees. The RAW
+skills-repo/plugin mount (which does carry the grader trees) is locked separately:
+the host rewrites the staged task's plugin paths to `/work/skills`, so it forwards
+the ORIGINAL host mount paths via `context.json` (`plugin_host_paths`) and the
+entrypoint locks those real in-container mount paths root-0700. The reference
+solution is handled the same way: `agent_safe_dump` strips the `reference` field
+from `task.yaml`, but an absolute (or `..`-escaping) `reference.file`/`reference.directory`
+is still bind-mounted for the in-container grader — so the host forwards its resolved
+mount targets via `context.json` (`reference_host_paths`) and the entrypoint locks
+those root-0700 too (mounted rw, like the plugin mounts, so the `chmod` isn't `EROFS`'d).
+
+### Per-harness drop seam (agent-agnostic)
+
+Every built-in harness spawns a controllable CLI-binary subprocess, so the drop
+is at that spawn seam — no orchestrator fork, no two-container split:
+
+- **claude-code** — `ClaudeAgentOptions.user = "agent"` (SDK forwards to
+  `subprocess.Popen(user=)`, a POSIX setuid).
+- **codex** — `CodexConfig.launch_args_override` routes the bundled codex binary
+  through the drop shim (`coder_eval_drop_privilege.sh` → `setpriv --reuid=agent`).
+- **antigravity** — the existing PATH-shadow (`_harness_spawn_guard`) injects a
+  `localharness` wrapper that execs the real binary through the same shim.
+
+If the drop is requested but the container is not root, the entrypoint **fails
+loud** — it never silently runs the agent as the container owner.
+
+### Scope, portability, and the Docker-Desktop caveat
+
+- **Docker + Linux only.** Unix uid/permissions exist only inside the Linux
+  container. The **Windows nightly slice runs `--driver tempdir`** (agent on host,
+  no container, no uid boundary): it is immune (criteria live in memory, never
+  written to an agent-readable file) but is **not covered by this barrier** — a
+  separate host-side isolation follow-up tracks it.
+- **Docker-Desktop bind-mount uid remap.** On macOS/Windows Docker Desktop, a
+  bind-mount's owner can be remapped, which can defeat an in-container `chown` on
+  a *bind-mounted* path. The barrier applies `chmod 0700` (owner root), which
+  denies the agent uid regardless of any remap; a materialized grader root can
+  live in the container rootfs (not a mount) where its `chown root:root` is always
+  authoritative. **The Linux CI/nightly host (native overlayfs, real root) is the
+  authoritative environment**; do not rely on the barrier on Docker-Desktop dev
+  machines.
+
+### Surface #6 — baked image content (authoring invariant)
+
+The barrier locks *runtime-mounted* material, but image **content** is not a
+permission surface. **Authoring invariant: mocks and tooling baked into the image
+must not encode task-specific expected values.** `tests/test_docker_image_no_answer_leak.py`
+is the deterministic sensor — it scans the Dockerfile + its `COPY` sources for
+answer sentinels and for `check_*.py` / `RESOLUTION.md` / `tests/tasks` grader
+material, and fails the build if any are baked.
+
+### Live check + nightly re-audit
+
+- **Live check.** Run `tasks/.../template_aware_create_adversarial.yaml` (the
+  "read /work" adversary) under a real `--driver docker` run and confirm the
+  agent's `found_criteria.txt` contains no grading material.
+- **Acceptance proof (CI).** `make test-docker-isolation` runs the six-surface
+  EACCES-as-agent-uid suite **as root inside the built image** (the `docker-isolation`
+  CI job); it is the authoritative check that the barrier holds.
+- **Per-harness uid probe (manual/nightly).** For each harness, run a minimal task
+  whose prompt writes `id -u` to a workspace file and confirm it equals `2000`
+  (== the baked agent uid) — the objective proof the agent-agnostic drop takes
+  effect. (Not wired as a live CI test to avoid model spend; the deterministic
+  EACCES proof + the per-harness wiring unit tests establish the mechanism.)
+- **Nightly re-audit.** Re-run the trajectory scan that produced the original leak
+  audit (reads of `check_*.py` / task-YAML / `RESOLUTION.md` / `$SKILLS_REPO_PATH/tests`
+  / `/work/input`) and confirm a per-run leak rate of 0. The reusable
+  `scan_for_leak_techniques` detector is the CI-cheap proxy.
+
+### Rollout notes
+
+- **Aggregate pass rates shift down ~2.4% (honest correction).** Before the
+  barrier, ~2.4% of nightly replicates passed by reading the suite (claude-code
+  highest, ~5–6%). Those tasks must now succeed on merit, so pass rates drop by
+  roughly that margin. **Annotate the first post-fix nightly** in the evalboard
+  ("leak-barrier landed") so the step-down is not read as a regression.
+- **Re-run contaminated carried passes.** The maturity feature carries forward
+  passes; any task that previously passed via a leak has a contaminated carried
+  pass. Invalidate + re-run the carried passes for every task the audit flagged
+  as `answer`/`oracle`/`recon`.
+- **~0 wall-clock cost.** The drop is a chmod/chown + a setuid at spawn inside the
+  single existing container — no extra container, no second pass.
@@ -182,6 +182,7 @@ external = [
     "CE012",
     "CE013",
     "CE018",
+    "CE033",
 ]  # custom architectural lint rules (tests/lint/)
 
 [tool.ruff.lint.pylint]
@@ -291,6 +292,10 @@ addopts = [
     "-ra",              # Show summary of all test outcomes
     "--showlocals",     # Show local variables in tracebacks
     "--color=yes",      # Force colored output
+    # Root-in-container docker-isolation tests never pass on a non-root host;
+    # they run via `make test-docker-isolation` (pytest -m docker_root inside
+    # the built image). Exclude them from the default host suite.
+    "-m", "not docker_root",
 ]
 
 # Asyncio configuration
@@ -307,6 +312,7 @@ markers = [
     "live: marks tests that hit real external services (Anthropic API, AWS Bedrock, etc.)",
     "lint: marks tests that enforce custom architectural lint rules",
     "divergence: pins a layer-4-vs-layer-5 merge divergence (or crash) that the declarative-merge refactor intentionally flips; see tests/test_merge_characterization.py",
+    "docker_root: marks tests that require root inside the built agent image (run via `make test-docker-isolation`, auto-excluded from host `make test`)",
 ]
 
 # Test discovery patterns