Skip to content

Commit c3405fb

Browse files
jmelahmanclaude
andauthored
chore(devtools): improve devcontainer usability w/ rootless docker (onyx-dot-app#10054)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 3e96293 commit c3405fb

6 files changed

Lines changed: 124 additions & 100 deletions

File tree

.devcontainer/Dockerfile

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
FROM ubuntu:26.04@sha256:cc925e589b7543b910fea57a240468940003fbfc0515245a495dd0ad8fe7cef1
22

33
RUN apt-get update && apt-get install -y --no-install-recommends \
4-
acl \
54
curl \
65
fd-find \
76
fzf \

.devcontainer/README.md

Lines changed: 10 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,6 @@ A containerized development environment for working on Onyx.
1414

1515
## Usage
1616

17-
### VS Code
18-
19-
1. Install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
20-
2. Open this repo in VS Code
21-
3. "Reopen in Container" when prompted
22-
2317
### CLI (`ods dev`)
2418

2519
The [`ods` devtools CLI](../tools/ods/README.md) provides workspace-aware wrappers
@@ -39,25 +33,8 @@ ods dev exec npm test
3933
ods dev stop
4034
```
4135

42-
If you don't have `ods` installed, use the `devcontainer` CLI directly:
43-
44-
```bash
45-
npm install -g @devcontainers/cli
46-
47-
devcontainer up --workspace-folder .
48-
devcontainer exec --workspace-folder . zsh
49-
```
50-
5136
## Restarting the container
5237

53-
### VS Code
54-
55-
Open the Command Palette (`Ctrl+Shift+P` / `Cmd+Shift+P`) and run:
56-
57-
- **Dev Containers: Reopen in Container** — restarts the container without rebuilding
58-
59-
### CLI
60-
6138
```bash
6239
# Restart the container
6340
ods dev restart
@@ -66,12 +43,6 @@ ods dev restart
6643
ods dev rebuild
6744
```
6845

69-
Or without `ods`:
70-
71-
```bash
72-
devcontainer up --workspace-folder . --remove-existing-container
73-
```
74-
7546
## Image
7647

7748
The devcontainer uses a prebuilt image published to `onyxdotapp/onyx-devcontainer`.
@@ -88,15 +59,19 @@ The `devcontainer` target is defined in `docker-bake.hcl` at the repo root.
8859
## User & permissions
8960

9061
The container runs as the `dev` user by default (`remoteUser` in devcontainer.json).
91-
An init script (`init-dev-user.sh`) runs at container start to ensure `dev` has
92-
read/write access to the bind-mounted workspace:
62+
An init script (`init-dev-user.sh`) runs at container start to ensure the active
63+
user has read/write access to the bind-mounted workspace:
9364

9465
- **Standard Docker**`dev`'s UID/GID is remapped to match the workspace owner,
9566
so file permissions work seamlessly.
9667
- **Rootless Docker** — The workspace appears as root-owned (UID 0) inside the
97-
container due to user-namespace mapping. The init script grants `dev` access via
98-
POSIX ACLs (`setfacl`), which adds a few seconds to the first container start on
99-
large repos.
68+
container due to user-namespace mapping. `ods dev up` auto-detects rootless Docker
69+
and sets `DEVCONTAINER_REMOTE_USER=root` so the container runs as root — which
70+
maps back to your host user via the user namespace. New files are owned by your
71+
host UID and no ACL workarounds are needed.
72+
73+
To override the auto-detection, set `DEVCONTAINER_REMOTE_USER` before running
74+
`ods dev up`.
10075

10176
## Docker socket
10277

@@ -109,9 +84,7 @@ from inside. `ods dev` auto-detects the socket path and sets `DOCKER_SOCK`:
10984
| macOS (Docker Desktop) | `~/.docker/run/docker.sock` |
11085
| Linux (standard Docker) | `/var/run/docker.sock` |
11186

112-
To override, set `DOCKER_SOCK` before running `ods dev up`. When using the
113-
VS Code extension or `devcontainer` CLI directly (without `ods`), you must set
114-
`DOCKER_SOCK` yourself.
87+
To override, set `DOCKER_SOCK` before running `ods dev up`.
11588

11689
## Firewall
11790

.devcontainer/devcontainer.json

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,15 @@
77
"source=${localEnv:HOME}/.claude,target=/home/dev/.claude,type=bind",
88
"source=${localEnv:HOME}/.claude.json,target=/home/dev/.claude.json,type=bind",
99
"source=${localEnv:HOME}/.zshrc,target=/home/dev/.zshrc.host,type=bind,readonly",
10-
"source=${localEnv:HOME}/.gitconfig,target=/home/dev/.gitconfig.host,type=bind,readonly",
11-
"source=${localEnv:HOME}/.ssh,target=/home/dev/.ssh.host,type=bind,readonly",
12-
"source=${localEnv:HOME}/.config/nvim,target=/home/dev/.config/nvim.host,type=bind,readonly",
10+
"source=${localEnv:HOME}/.gitconfig,target=/home/dev/.gitconfig,type=bind,readonly",
11+
"source=${localEnv:HOME}/.config/nvim,target=/home/dev/.config/nvim,type=bind,readonly",
1312
"source=onyx-devcontainer-cache,target=/home/dev/.cache,type=volume",
1413
"source=onyx-devcontainer-local,target=/home/dev/.local,type=volume"
1514
],
16-
"remoteUser": "dev",
15+
"containerEnv": {
16+
"SSH_AUTH_SOCK": "/tmp/ssh-agent.sock"
17+
},
18+
"remoteUser": "${localEnv:DEVCONTAINER_REMOTE_USER:dev}",
1719
"updateRemoteUserUID": false,
1820
"workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind,consistency=delegated",
1921
"workspaceFolder": "/workspace",

.devcontainer/init-dev-user.sh

Lines changed: 60 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -8,38 +8,68 @@ set -euo pipefail
88
# We remap dev to that UID -- fast and seamless.
99
#
1010
# Rootless Docker: Workspace appears as root-owned (UID 0) inside the
11-
# container due to user-namespace mapping. We can't remap
12-
# dev to UID 0 (that's root), so we grant access with
13-
# POSIX ACLs instead.
11+
# container due to user-namespace mapping. Requires
12+
# DEVCONTAINER_REMOTE_USER=root (set automatically by
13+
# ods dev up). Container root IS the host user, so
14+
# bind-mounts and named volumes are symlinked into /root.
1415

1516
WORKSPACE=/workspace
1617
TARGET_USER=dev
18+
REMOTE_USER="${SUDO_USER:-$TARGET_USER}"
1719

1820
WS_UID=$(stat -c '%u' "$WORKSPACE")
1921
WS_GID=$(stat -c '%g' "$WORKSPACE")
2022
DEV_UID=$(id -u "$TARGET_USER")
2123
DEV_GID=$(id -g "$TARGET_USER")
2224

23-
DEV_HOME=/home/"$TARGET_USER"
25+
# devcontainer.json bind-mounts and named volumes target /home/dev regardless
26+
# of remoteUser. When running as root ($HOME=/root), Phase 1 bridges the gap
27+
# with symlinks from ACTIVE_HOME → MOUNT_HOME.
28+
MOUNT_HOME=/home/"$TARGET_USER"
2429

25-
# Ensure directories that tools expect exist under ~dev.
26-
# ~/.local and ~/.cache are named Docker volumes -- ensure they are owned by dev.
27-
mkdir -p "$DEV_HOME"/.local/state "$DEV_HOME"/.local/share
28-
chown -R "$TARGET_USER":"$TARGET_USER" "$DEV_HOME"/.local
29-
chown -R "$TARGET_USER":"$TARGET_USER" "$DEV_HOME"/.cache
30+
if [ "$REMOTE_USER" = "root" ]; then
31+
ACTIVE_HOME="/root"
32+
else
33+
ACTIVE_HOME="$MOUNT_HOME"
34+
fi
35+
36+
# ── Phase 1: home directory setup ───────────────────────────────────
37+
38+
# ~/.local and ~/.cache are named Docker volumes mounted under MOUNT_HOME.
39+
mkdir -p "$MOUNT_HOME"/.local/state "$MOUNT_HOME"/.local/share
3040

31-
# Copy host configs mounted as *.host into their real locations.
32-
# This gives the dev user owned copies without touching host originals.
33-
if [ -d "$DEV_HOME/.ssh.host" ]; then
34-
cp -a "$DEV_HOME/.ssh.host" "$DEV_HOME/.ssh"
35-
chmod 700 "$DEV_HOME/.ssh"
36-
chmod 600 "$DEV_HOME"/.ssh/id_* 2>/dev/null || true
37-
chown -R "$TARGET_USER":"$TARGET_USER" "$DEV_HOME/.ssh"
41+
# When running as root, symlink bind-mounts and named volumes into /root
42+
# so that $HOME-relative tools (Claude Code, git, etc.) find them.
43+
if [ "$ACTIVE_HOME" != "$MOUNT_HOME" ]; then
44+
for item in .claude .cache .local; do
45+
[ -d "$MOUNT_HOME/$item" ] || continue
46+
if [ -e "$ACTIVE_HOME/$item" ] && [ ! -L "$ACTIVE_HOME/$item" ]; then
47+
echo "warning: replacing $ACTIVE_HOME/$item with symlink to $MOUNT_HOME/$item" >&2
48+
rm -rf "$ACTIVE_HOME/$item"
49+
fi
50+
ln -sfn "$MOUNT_HOME/$item" "$ACTIVE_HOME/$item"
51+
done
52+
# Symlink files (not directories).
53+
for file in .claude.json .gitconfig .zshrc.host; do
54+
[ -f "$MOUNT_HOME/$file" ] && ln -sf "$MOUNT_HOME/$file" "$ACTIVE_HOME/$file"
55+
done
56+
57+
# Nested mount: .config/nvim
58+
if [ -d "$MOUNT_HOME/.config/nvim" ]; then
59+
mkdir -p "$ACTIVE_HOME/.config"
60+
if [ -e "$ACTIVE_HOME/.config/nvim" ] && [ ! -L "$ACTIVE_HOME/.config/nvim" ]; then
61+
echo "warning: replacing $ACTIVE_HOME/.config/nvim with symlink" >&2
62+
rm -rf "$ACTIVE_HOME/.config/nvim"
63+
fi
64+
ln -sfn "$MOUNT_HOME/.config/nvim" "$ACTIVE_HOME/.config/nvim"
65+
fi
3866
fi
39-
if [ -d "$DEV_HOME/.config/nvim.host" ]; then
40-
mkdir -p "$DEV_HOME/.config"
41-
cp -a "$DEV_HOME/.config/nvim.host" "$DEV_HOME/.config/nvim"
42-
chown -R "$TARGET_USER":"$TARGET_USER" "$DEV_HOME/.config/nvim"
67+
68+
# ── Phase 2: workspace access ───────────────────────────────────────
69+
70+
# Root always has workspace access; Phase 1 handled home setup.
71+
if [ "$REMOTE_USER" = "root" ]; then
72+
exit 0
4373
fi
4474

4575
# Already matching -- nothing to do.
@@ -61,45 +91,17 @@ if [ "$WS_UID" != "0" ]; then
6191
echo "warning: failed to remap $TARGET_USER UID to $WS_UID" >&2
6292
fi
6393
fi
64-
if ! chown -R "$TARGET_USER":"$TARGET_USER" /home/"$TARGET_USER" 2>&1; then
65-
echo "warning: failed to chown /home/$TARGET_USER" >&2
94+
if ! chown -R "$TARGET_USER":"$TARGET_USER" "$MOUNT_HOME" 2>&1; then
95+
echo "warning: failed to chown $MOUNT_HOME" >&2
6696
fi
6797
else
6898
# ── Rootless Docker ──────────────────────────────────────────────
69-
# Workspace is root-owned inside the container. Grant dev access
70-
# via POSIX ACLs (preserves ownership, works across the namespace
71-
# boundary).
72-
if command -v setfacl &>/dev/null; then
73-
setfacl -Rm "u:${TARGET_USER}:rwX" "$WORKSPACE"
74-
setfacl -Rdm "u:${TARGET_USER}:rwX" "$WORKSPACE" # default ACL for new files
75-
76-
# Git refuses to operate in repos owned by a different UID.
77-
# Host gitconfig is mounted readonly as ~/.gitconfig.host.
78-
# Create a real ~/.gitconfig that includes it plus container overrides.
79-
printf '[include]\n\tpath = %s/.gitconfig.host\n[safe]\n\tdirectory = %s\n' \
80-
"$DEV_HOME" "$WORKSPACE" > "$DEV_HOME/.gitconfig"
81-
chown "$TARGET_USER":"$TARGET_USER" "$DEV_HOME/.gitconfig"
82-
83-
# If this is a worktree, the main .git dir is bind-mounted at its
84-
# host absolute path. Grant dev access so git operations work.
85-
GIT_COMMON_DIR=$(git -C "$WORKSPACE" rev-parse --git-common-dir 2>/dev/null || true)
86-
if [ -n "$GIT_COMMON_DIR" ] && [ "$GIT_COMMON_DIR" != "$WORKSPACE/.git" ]; then
87-
[ ! -d "$GIT_COMMON_DIR" ] && GIT_COMMON_DIR="$WORKSPACE/$GIT_COMMON_DIR"
88-
if [ -d "$GIT_COMMON_DIR" ]; then
89-
setfacl -Rm "u:${TARGET_USER}:rwX" "$GIT_COMMON_DIR"
90-
setfacl -Rdm "u:${TARGET_USER}:rwX" "$GIT_COMMON_DIR"
91-
git config -f "$DEV_HOME/.gitconfig" --add safe.directory "$(dirname "$GIT_COMMON_DIR")"
92-
fi
93-
fi
94-
95-
# Also fix bind-mounted dirs under ~dev that appear root-owned.
96-
for dir in /home/"$TARGET_USER"/.claude; do
97-
[ -d "$dir" ] && setfacl -Rm "u:${TARGET_USER}:rwX" "$dir" && setfacl -Rdm "u:${TARGET_USER}:rwX" "$dir"
98-
done
99-
[ -f /home/"$TARGET_USER"/.claude.json ] && \
100-
setfacl -m "u:${TARGET_USER}:rw" /home/"$TARGET_USER"/.claude.json
101-
else
102-
echo "warning: setfacl not found; dev user may not have write access to workspace" >&2
103-
echo " install the 'acl' package or set remoteUser to root" >&2
104-
fi
99+
# Workspace is root-owned (UID 0) due to user-namespace mapping.
100+
# The supported path is remoteUser=root (set DEVCONTAINER_REMOTE_USER=root),
101+
# which is handled above. If we reach here, the user is running as dev
102+
# under rootless Docker without the override.
103+
echo "error: rootless Docker detected but remoteUser is not root." >&2
104+
echo " Set DEVCONTAINER_REMOTE_USER=root before starting the container," >&2
105+
echo " or use 'ods dev up' which sets it automatically." >&2
106+
exit 1
105107
fi

tools/ods/cmd/dev_into.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ Examples:
2929
// runDevExec executes "devcontainer exec --workspace-folder <root> <command...>".
3030
func runDevExec(command []string) {
3131
checkDevcontainerCLI()
32+
ensureDockerSock()
33+
ensureRemoteUser()
3234

3335
root, err := paths.GitRoot()
3436
if err != nil {

tools/ods/cmd/dev_up.go

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,10 +148,53 @@ func worktreeGitMount(root string) (string, bool) {
148148
return mount, true
149149
}
150150

151+
// sshAgentMount returns a --mount flag value that forwards the host's SSH agent
152+
// socket into the container. Returns ("", false) when SSH_AUTH_SOCK is unset or
153+
// the socket is not accessible.
154+
func sshAgentMount() (string, bool) {
155+
sock := os.Getenv("SSH_AUTH_SOCK")
156+
if sock == "" {
157+
log.Debug("SSH_AUTH_SOCK not set — skipping SSH agent forwarding")
158+
return "", false
159+
}
160+
if _, err := os.Stat(sock); err != nil {
161+
log.Debugf("SSH_AUTH_SOCK=%s not accessible: %v", sock, err)
162+
return "", false
163+
}
164+
mount := fmt.Sprintf("type=bind,source=%s,target=/tmp/ssh-agent.sock", sock)
165+
log.Debugf("Forwarding SSH agent: %s", sock)
166+
return mount, true
167+
}
168+
169+
// ensureRemoteUser sets DEVCONTAINER_REMOTE_USER when rootless Docker is
170+
// detected. Container root maps to the host user in rootless mode, so running
171+
// as root inside the container avoids the UID mismatch on new files.
172+
// Must be called after ensureDockerSock.
173+
func ensureRemoteUser() {
174+
if os.Getenv("DEVCONTAINER_REMOTE_USER") != "" {
175+
return
176+
}
177+
178+
if runtime.GOOS == "linux" {
179+
sock := os.Getenv("DOCKER_SOCK")
180+
xdg := os.Getenv("XDG_RUNTIME_DIR")
181+
// Heuristic: rootless Docker on Linux typically places its socket
182+
// under $XDG_RUNTIME_DIR. If DOCKER_SOCK was set to a custom path
183+
// outside XDG_RUNTIME_DIR, set DEVCONTAINER_REMOTE_USER=root manually.
184+
if xdg != "" && strings.HasPrefix(sock, xdg) {
185+
log.Debug("Rootless Docker detected — setting DEVCONTAINER_REMOTE_USER=root")
186+
if err := os.Setenv("DEVCONTAINER_REMOTE_USER", "root"); err != nil {
187+
log.Warnf("Failed to set DEVCONTAINER_REMOTE_USER: %v", err)
188+
}
189+
}
190+
}
191+
}
192+
151193
// runDevcontainer executes "devcontainer <action> --workspace-folder <root> [extraArgs...]".
152194
func runDevcontainer(action string, extraArgs []string) {
153195
checkDevcontainerCLI()
154196
ensureDockerSock()
197+
ensureRemoteUser()
155198

156199
root, err := paths.GitRoot()
157200
if err != nil {
@@ -162,6 +205,9 @@ func runDevcontainer(action string, extraArgs []string) {
162205
if mount, ok := worktreeGitMount(root); ok {
163206
args = append(args, "--mount", mount)
164207
}
208+
if mount, ok := sshAgentMount(); ok {
209+
args = append(args, "--mount", mount)
210+
}
165211
args = append(args, extraArgs...)
166212

167213
log.Debugf("Running: devcontainer %v", args)

0 commit comments

Comments
 (0)