Skip to content

refactor: adopt canonical Remote Dev data layout - #72

Merged
eXPerience83 merged 41 commits into
mainfrom
refactor/canonical-data-layout
Aug 2, 2026
Merged

refactor: adopt canonical Remote Dev data layout#72
eXPerience83 merged 41 commits into
mainfrom
refactor/canonical-data-layout

Conversation

@eXPerience83

@eXPerience83 eXPerience83 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Closes #70

Summary

  • replace the experimental Codex-specific data root with the canonical REMOTE_DEV_DATA_ROOT contract
  • split persistence into workspaces, private per-role state, and secrets
  • mount only the exact Codex child paths while keeping the base launcher mount-free
  • derive the file-backed Codex password from the same canonical root instead of introducing a second path variable
  • add an authoritative host-side preflight that rejects missing, symlinked or malformed paths and unsafe password-file permissions before deployment
  • retain long bind syntax with create_host_path: false as defense-in-depth rather than relying on inconsistent Compose runtime enforcement
  • update generic Compose, the TrueNAS reference YAML, English/Spanish documentation, architecture, security, release guidance and project status
  • add regressions for exact mount boundaries, canonical path selection, preflight behavior and removal of earlier experimental data-root names

Manual validation

Deferred until the related implementation slices are ready, following the maintainer decision to consolidate real TrueNAS validation. Do not create the new TrueNAS datasets or deploy this YAML before the PR is merged and its edge image is published.

Out of scope

Validation plan

  • repository, Compose and preflight regression tests in CI
  • image build and existing runtime smoke tests
  • SBOM, Trivy and fixable-critical vulnerability gate
  • final consolidated TrueNAS validation for preflight, persistence, sessions, credentials, isolation and recreation after the relevant slices are merged

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@eXPerience83, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 4 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 447d8208-77c5-462f-99d2-ce2bdafff243

📥 Commits

Reviewing files that changed from the base of the PR and between ca55f59 and 38a3908.

📒 Files selected for processing (15)
  • .env.example
  • .github/workflows/build-amd64.yml
  • CHANGELOG.md
  • Makefile
  • PROJECT_STATUS.md
  • README.es.md
  • README.md
  • compose/docker-compose.yml
  • compose/truenas.yml
  • docs/architecture.md
  • docs/releases.md
  • docs/security.md
  • scripts/preflight-data-layout.py
  • scripts/test_canonical_data_layout.py
  • scripts/test_preflight_data_layout.py

Summary by CodeRabbit

  • New Features

    • Added a canonical Remote Dev image and persistent-data layout, with compatibility support for existing image names.
    • Improved Compose and TrueNAS deployments with explicit bind mounts, isolated credentials, and safer secret handling.
    • Added validation for storage paths, authentication boundaries, mount restrictions, and deployment configuration.
  • Documentation

    • Updated README files, architecture, security, release, changelog, and project-status documentation.
    • Documented runtime roles, approval modes, authentication, licensing, deployment, rollback, and security requirements.
  • Bug Fixes

    • Missing bind-mount directories now fail validation instead of being created automatically.

Walkthrough

The change adopts a role-neutral REMOTE_DEV_DATA_ROOT layout, converts Compose and TrueNAS mounts to explicit bind mounts, strengthens launcher and credential validation, and updates documentation, release guidance, and CI validation.

Changes

Canonical runtime layout

Layer / File(s) Summary
Runtime contract and mount wiring
.env.example, compose/*, docs/architecture.md, README*, PROJECT_STATUS.md
The stack now uses role-scoped paths under REMOTE_DEV_DATA_ROOT. Compose and TrueNAS use explicit bind mounts with host-path creation disabled. The launcher remains mount-free and isolated from agent credentials and state.
Canonical layout validation
scripts/test_canonical_data_layout.py
A new validation script checks rendered Compose services, mount targets, secret paths, forbidden mounts, required source configuration, and legacy data-root references.
Compose and authentication validation
scripts/test_single_stack_compose.py, Makefile, .github/workflows/build-amd64.yml
Validation now checks shared images, service roles, authentication separation, credential boundaries, network restrictions, and TrueNAS launcher isolation. CI and make validate run the new canonical layout test.
Security and release documentation
docs/security.md, docs/releases.md, CHANGELOG.md
Security, promotion, rollback, and changelog documentation now describe canonical persistence, authentication boundaries, mount restrictions, digest-based images, and missing-path deployment behavior.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

  • eXPerience83/remote-dev-containers issue 31 — The changes implement its single-image, isolated-service Remote Dev architecture.
  • eXPerience83/remote-dev-containers issue 71 — The canonical workspace layout established here is a prerequisite for the planned SMB integration.
  • eXPerience83/remote-dev-containers issue 25 — The changes implement its role-neutral persistence and launcher isolation requirements.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement the canonical data-root variable, narrow mounts, launcher isolation, missing-path failures, documentation updates, and static validation required by issue #70.
Out of Scope Changes check ✅ Passed The changes remain within the linked issue scope and support the requested persistence, isolation, validation, documentation, and deployment updates.
Title check ✅ Passed The title clearly and concisely identifies the primary change: adopting the canonical Remote Dev data layout.
Description check ✅ Passed The description explains the changes, scope, validation plan, deferred work, security boundaries, and deployment impact in sufficient detail.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/canonical-data-layout

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@eXPerience83
eXPerience83 marked this pull request as ready for review August 2, 2026 14:56

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca55f59513

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/test_canonical_data_layout.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/releases.md`:
- Around line 69-76: Align all three documentation statements with the
implemented authentication behavior: in docs/releases.md lines 69-76, qualify
the no-mount check to allow the optional authentication-secret mount; in
CHANGELOG.md lines 15-16, scope the authentication guard to agent terminals; and
in CHANGELOG.md lines 48-50, state that the launcher receives no agent password
by default while permitting the documented launcher-auth exception.

In `@scripts/test_canonical_data_layout.py`:
- Around line 154-168: Update validate_repository_has_no_legacy_data_root to
scan only files tracked by Git instead of every path under ROOT.rglob("*"),
while preserving the existing binary-file filtering and legacy-variable/path
assertions for each readable tracked file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 27b4a576-d750-426f-9c63-cf5742d2a972

📥 Commits

Reviewing files that changed from the base of the PR and between 3377935 and ca55f59.

📒 Files selected for processing (14)
  • .env.example
  • .github/workflows/build-amd64.yml
  • CHANGELOG.md
  • Makefile
  • PROJECT_STATUS.md
  • README.es.md
  • README.md
  • compose/docker-compose.yml
  • compose/truenas.yml
  • docs/architecture.md
  • docs/releases.md
  • docs/security.md
  • scripts/test_canonical_data_layout.py
  • scripts/test_single_stack_compose.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (10)
.github/workflows/**

⚙️ CodeRabbit configuration file

.github/workflows/**: Review GitHub Actions for least-privilege permissions, safe event triggers, untrusted input handling,
accidental publication, tag-channel correctness, secret exposure, reproducible builds, SBOM/provenance
generation and appropriate validation before pushing images.

Files:

  • .github/workflows/build-amd64.yml
**/*.{yml,yaml,Dockerfile,sh,bash,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{yml,yaml,Dockerfile,sh,bash,js,ts,tsx}: Never use privileged containers, Docker or Podman sockets, SYS_ADMIN, host-root mounts, or broad persistence mounts such as /root, /home, /opt, /usr/local, or a parent data root.
Preserve mandatory ttyd authentication for agent terminals, origin checking for all web endpoints, tmux reconnect behavior, image identity checks, and existing Codex login, start, and resume behavior.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.{yml,yaml,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{yml,yaml,js,ts,tsx}: The launcher role is navigation only: it may link only to fixed, validated stack services and must not execute agents or relay/proxy agent terminal HTTP or WebSocket traffic without an explicit threat-model review.
Keep the unauthenticated private-network default, optional launcher authentication, origin checks, CSP, method restrictions, and secret-free health behavior covered by tests.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.{yml,yaml,html,js,ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Never embed credentials in launcher URLs, HTML, JavaScript, logs, or diagnostics.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.{yml,yaml}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{yml,yaml}: By default, the launcher receives only web/routing configuration and no password secret; it must never receive agent workspaces, agent state, GitHub/Git/SSH mounts, agent secrets, or the Docker socket.
Launcher and agent services must use the same final image reference or digest while retaining separate container roles and state boundaries.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.{yml,yaml,sh,bash}

📄 CodeRabbit inference engine (AGENTS.md)

Implement issue #25 in separately reviewed slices: role-neutral commands and validation, configurable Codex approval modes, canonical image and variable naming with time-bounded aliases, shared image digest, Compose and state migration, and outer hardening with cross-service canaries.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.{sh,bash,js,ts,tsx,yml,yaml}

📄 CodeRabbit inference engine (AGENTS.md)

Run the narrowest relevant tests during development and complete required CI before merge; preserve or extend coverage for role/start-mode validation, wrappers, launcher security, mounts, image identity, health checks, Codex flows, credential hardening, ttyd, tmux, provenance, notices, SBOM, Trivy, and the critical-vulnerability gate.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
compose/**

⚙️ CodeRabbit configuration file

compose/**: Review Docker Compose files for TrueNAS and generic Docker compatibility.
Flag privileged mode, host networking, Docker socket mounts, overly broad host mounts,
missing authentication, unsafe public bindings and persistence paths that could lose credentials or workspaces.
Require launcher and agent services to reference one image while keeping agent state out of the launcher.

Files:

  • compose/truenas.yml
  • compose/docker-compose.yml
**/*.md

⚙️ CodeRabbit configuration file

**/*.md: Check that documentation matches the implemented behavior and clearly distinguishes experimental edge builds
from stable releases. Flag outdated versions, unsafe deployment guidance and claims not proven by CI or testing.

Files:

  • README.md
  • docs/security.md
  • PROJECT_STATUS.md
  • CHANGELOG.md
  • docs/architecture.md
  • docs/releases.md
  • README.es.md
scripts/test_single_stack_compose.py

⚙️ CodeRabbit configuration file

scripts/test_single_stack_compose.py: Check that topology assertions are deterministic and really prove one image reference, fixed roles,
no launcher access to agent state, no Docker socket/privilege escalation and preserved Codex compatibility.

Files:

  • scripts/test_single_stack_compose.py
🪛 ast-grep (0.45.0)
scripts/test_canonical_data_layout.py

[error] 41-59: Command coming from incoming request
Context: subprocess.run(
[
"docker",
"compose",
"--env-file",
empty_env.name,
"-f",
str(path),
"config",
"--format",
"json",
],
cwd=ROOT,
env=compose_environment(),
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(subprocess-from-request)

scripts/test_single_stack_compose.py

[info] 176-176: use jsonify instead of json.dumps for JSON output
Context: json.dumps(launcher, sort_keys=True)
Note: [CWE-116] Improper Encoding or Escaping of Output.

(use-jsonify)

🪛 dotenv-linter (4.0.0)
.env.example

[warning] 5-5: [UnorderedKey] The REMOTE_DEV_DATA_ROOT key should go before the REMOTE_DEV_IMAGE key

(UnorderedKey)

🪛 LanguageTool
docs/security.md

[style] ~69-~69: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...tests or rendered environment output. - Never mount agent data or credentials into th...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[style] ~70-~70: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ata or credentials into the launcher. - Never mount Docker or Podman sockets. - Never...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[style] ~71-~71: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...Never mount Docker or Podman sockets. - Never use privileged: true, host PID, host ...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[style] ~72-~72: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ost networking or added capabilities. - Never mount the parent data root, host root, ...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)

CHANGELOG.md

[style] ~39-~39: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ... around role-neutral implementations. - Changed the normal TrueNAS portal from the Code...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)


[style] ~40-~40: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ing independent Codex authentication. - Changed the stateless launcher to require no pa...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)

🪛 Ruff (0.16.0)
scripts/test_canonical_data_layout.py

[warning] 27-27: Boolean-typed positional argument in function definition

(FBT001)


[warning] 42-60: Prefer capture_output over sending stdout and stderr to PIPE

Replace with capture_output keyword argument

(UP022)


[error] 42-42: subprocess call: check for execution of untrusted input

(S603)


[error] 43-53: Starting a process with a partial executable path

(S607)


[warning] 62-62: Avoid specifying long messages outside the exception class

(TRY003)

scripts/test_single_stack_compose.py

[error] 260-260: Possible hardcoded password assigned to: "WEB_PASSWORD_FILE"

(S105)

🔇 Additional comments (12)
docs/security.md (1)

3-82: LGTM!

Also applies to: 91-106

docs/releases.md (1)

7-41: LGTM!

Also applies to: 54-56, 58-68, 77-81, 83-93

.env.example (1)

4-6: LGTM!

compose/docker-compose.yml (2)

121-121: LGTM!


84-108: 🗄️ Data Integrity & Integration | 🏗️ Heavy lift

Confirm create_host_path: false is actually enforced by the Compose version in use; the whole "fail instead of silently create" contract depends on it.

Docker Compose has an open, currently-unresolved issue (docker/compose#13602) where bind.create_host_path: false is kept in the rendered config for the affected mount, so the value appears to be parsed but not enforced at runtime. The Compose spec itself documents that the option "Creates a directory at the source path on host if there is nothing present" and "Defaults to true", confirming that explicit false is required to opt out of auto-creation — exactly what this PR relies on. If the Compose binary used in production or CI has this bug, every mount in both Compose files will silently auto-create a missing host directory instead of failing, undermining the core safety claim repeated across the docs.

  • compose/docker-compose.yml#L84-L108: verify the Docker Compose version used to run this file honors create_host_path: false; if not guaranteed, add a preflight check (e.g., a small script or Makefile target) that asserts each ${REMOTE_DEV_DATA_ROOT} child path exists before docker compose up.
  • compose/truenas.yml#L84-L114: apply the same verification and compensating control for the TrueNAS bind mounts, including the password file bind.
  • scripts/test_canonical_data_layout.py#L77-L89: tighten validate_bind_mount to assert bind.get("create_host_path") is False instead of is not True, and update or remove the comment claiming Compose "omits" the explicit false value, since current evidence contradicts it.

Once confirmed, documentation claims that restate this guarantee (docs/architecture.md line 143, PROJECT_STATUS.md line 25, README.md line 89, README.es.md line 89) should be revisited if the underlying Compose behavior cannot be trusted without a compensating check.

docs/architecture.md (1)

1-193: LGTM!

PROJECT_STATUS.md (1)

21-25: LGTM!

Also applies to: 35-45

README.es.md (1)

3-170: LGTM!

README.md (1)

3-6: LGTM!

Also applies to: 16-129, 138-170

scripts/test_single_stack_compose.py (1)

16-17: LGTM!

Also applies to: 27-27, 84-88, 119-139, 158-206, 208-274, 284-297

Makefile (1)

14-19: LGTM!

.github/workflows/build-amd64.yml (1)

34-34: LGTM!

Also applies to: 59-59

Comment thread docs/releases.md Outdated
Comment thread scripts/test_canonical_data_layout.py

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5340e5a3a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread compose/docker-compose.yml Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 846115b401

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/preflight-data-layout.py Outdated
Comment thread scripts/preflight-data-layout.py Outdated

Copy link
Copy Markdown
Owner Author

Addressed the remaining Codex review findings on the final branch:

  • the Codex file-password source now derives exclusively from REMOTE_DEV_DATA_ROOT; WEB_PASSWORD_PATH was removed;
  • the authoritative preflight applies the same trailing-LF handling as the shell runtime and rejects effectively empty, multiline, CR-containing or NUL-containing passwords;
  • the preflight inspects every path component from the configured root downward without following symlinks and stops before reading through a symlinked tree;
  • regression tests cover newline-only/multiline passwords, final-component symlinks and intermediate-directory symlinks.

The complete CI run for the final head is in progress.

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 38a3908f92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@eXPerience83
eXPerience83 merged commit 951f70d into main Aug 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: adopt canonical Remote Dev data layout

1 participant