First run of `dockerized node:16.14 --version` results in an error

[halostatue]

❯ bin/dockerized node:16.14 --version
[+] Running 6/0
 ⠿ Network dockerized_default          Created                                                                         0.0s
 ⠿ Volume "dockerized_pip_cache"       Created                                                                         0.0s
 ⠿ Volume "dockerized_go_cache"        Created                                                                         0.0s
 ⠿ Volume "dockerized_home"            Created                                                                         0.0s
 ⠿ Volume "dockerized_node_modules"    Created                                                                         0.0s
 ⠿ Volume "dockerized_python_modules"  Created                                                                         0.0s
[+] Running 10/10
 ⠿ node Pulled                                                                                                        69.2s
   ⠿ c732c99090fe Pull complete                                                                                       42.9s
   ⠿ 26cef8e19215 Pull complete                                                                                       43.1s
   ⠿ 741a7b3bab3c Pull complete                                                                                       43.3s
   ⠿ 66bff08cf973 Pull complete                                                                                       44.9s
   ⠿ 3d35ac8f6f1a Pull complete                                                                                       66.9s
   ⠿ a25eb52b374a Pull complete                                                                                       67.0s
   ⠿ bc98bafd2f98 Pull complete                                                                                       67.9s
   ⠿ 82c51e978386 Pull complete                                                                                       68.0s
   ⠿ 24dba3274ff0 Pull complete                                                                                       68.0s
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/node': chown /Users/austin/.dockerized/apps/node: permission denied

Second run outputs the expected version (v16.14.2).

Trying with dockerized python:3 --version results in a similar error on the first run:

❯ bin/dockerized python:3 --version
[+] Running 10/10
 ⠿ python Pulled                                                                                                       7.5s
   ⠿ fa223d8c149d Already exists                                                                                       0.0s
   ⠿ 17ffbd2ab159 Already exists                                                                                       0.0s
   ⠿ a3953e76cee2 Already exists                                                                                       0.0s
   ⠿ 6f906570592f Already exists                                                                                       0.0s
   ⠿ c478f6a18a39 Already exists                                                                                       0.0s
   ⠿ 965f10264bda Pull complete                                                                                        3.1s
   ⠿ fae4e8c73329 Pull complete                                                                                        6.1s
   ⠿ 2ac2661d4813 Pull complete                                                                                        6.2s
   ⠿ 993092bb086d Pull complete                                                                                        6.3s
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/python': chown /Users/austin/.dockerized/apps/python: permission denied

[boukeversteegh]

Unfortunately I cannot reproduce this error on Windows or Ubuntu.

Any hints as to what this might cause would be very welcome.

Some things I'm curious about:

• What are the permissions+owners on ~/.dockerized/apps before the first run?
• After the first run, what folders are there, and what are their permissions+owners?
• After the second run, what folders are there, and what are their permissions+owners?
• Do you also have this problem when mounting a local folder with docker compose, or docker run?
  • e.g. docker run --rm -it -v "$PWD/foobar:/foobar" golang ls -lah /foobar

[halostatue]

Let me try this again with the new version just released.

• ~/.dockerized does not exist.

First run:

❯ dockerized/bin/dockerized node:16.14 --version
[+] Running 5/0
 ⠿ Volume "dockerized_pip_cache"       Created                                                                   0.0s
 ⠿ Volume "dockerized_go_cache"        Created                                                                   0.0s
 ⠿ Volume "dockerized_home"            Created                                                                   0.0s
 ⠿ Volume "dockerized_node_modules"    Created                                                                   0.0s
 ⠿ Volume "dockerized_python_modules"  Created                                                                   0.0s
[+] Running 10/10
 ⠿ node Pulled                                                                                                  73.0s
   ⠿ b9f330b3a7e2 Pull complete                                                                                 44.6s
   ⠿ 8a80651adb7e Pull complete                                                                                 44.8s
   ⠿ a7b38ee622cb Pull complete                                                                                 45.0s
   ⠿ fc937997baec Pull complete                                                                                 50.8s
   ⠿ d799cbd56049 Pull complete                                                                                 69.8s
   ⠿ ebce7ea98d19 Pull complete                                                                                 69.9s
   ⠿ 49fceb34a8de Pull complete                                                                                 70.9s
   ⠿ bbde1338a04a Pull complete                                                                                 71.0s
   ⠿ fe3a6810ee35 Pull complete                                                                                 71.0s
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/node': chown /Users/austin/.dockerized/apps/node: permission denied

❯ find ~/.dockerized -ls
36790113        0 drwxr-xr-x    3 austin           staff                  96 22 Apr 13:19 /Users/austin/.dockerized
36790114        0 drwxr-xr-x    3 austin           staff                  96 22 Apr 13:19 /Users/austin/.dockerized/apps
36790115        0 drwxr-xr-x    2 austin           staff                  64 22 Apr 13:19 /Users/austin/.dockerized/apps/node

Second run:

❯ dockerized/bin/dockerized node:16.14 --version
v16.14.2

❯ find ~/.dockerized -ls
36790113        0 drwxr-xr-x    3 austin           staff                  96 22 Apr 13:19 /Users/austin/.dockerized
36790114        0 drwxr-xr-x    3 austin           staff                  96 22 Apr 13:19 /Users/austin/.dockerized/apps
36790115        0 drwxr-xr-x    2 austin           staff                  64 22 Apr 13:19 /Users/austin/.dockerized/apps/node

I have several other applications that I run through docker images (mostly those I have made myself) and have no problems mounting local folders. On the other hand, they aren’t trying to create the folder structures at this point.

Trying a variant of your suggestion:

❯ docker run --rm -it -v "$HOME:/work" golang sh -c "mkdir -p /work/.dockerized/foo && find /work/.dockerized"
/work/.dockerized
/work/.dockerized/foo

[halostatue]

Test variants I am now trying:

> rm -rf ~/.dockerized
> dockerized node:16.14 --version
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/node': mkdir /Users/austin/.dockerized/apps: no such file or directory
> dockerized node:16.14 --version
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/node': chown /Users/austin/.dockerized/apps/node: permission denied
❯ dockerized/bin/dockerized node:16.14 --version
v16.14.2
❯ dockerized/bin/dockerized python:3 --version
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/python': chown /Users/austin/.dockerized/apps/python: permission denied
❯ dockerized/bin/dockerized python:3 --version
Python 3.10.4

After this, I did the following:

❯ rm -rf ~/.dockerized
❯ mkdir -p ~/.dockerized/apps
❯ dockerized/bin/dockerized node:16.14 --version
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/node': chown /Users/austin/.dockerized/apps/node: permission denied
❯ dockerized/bin/dockerized node:16.14 --version
v16.14.2
❯ dockerized/bin/dockerized python:3 --version
Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/apps/python': chown /Users/austin/.dockerized/apps/python: permission denied
❯ dockerized/bin/dockerized python:3 --version
Python 3.10.4

Is this related to the ownership model we identified during the build process?

[boukeversteegh]

Thanks for running these tests. You're probably right that it's related to the path-creation, done by docker.

I have a few more questions if you don't mind.

Trying a variant of your suggestion:

❯ docker run --rm -it -v "$HOME:/work" golang sh -c "mkdir -p /work/.dockerized/foo && find /work/.dockerized"
/work/.dockerized
/work/.dockerized/foo

• Am I correct to assume these files are owned by root?

• What happens if you mount a non-existing nested path to a container directly? E.g.

  docker run --rm -it -v "$HOME/.dockerized/tmp/a/b:/work" golang find /work -ls
  

Really wondering if this might be a docker issue.

• Which version of docker are you running?

• And was it installed from homebrew or the website?

• Any idea if docker on MacOS runs as root, as austin or as something else, e.g. docker?

  It's somewhat strange that docker can create the folders, owned by your user account, but then cannot chown them anymore. I suspect that docker is running under a non-root dedicated user.

[halostatue]

I’m not using Docker Desktop, but Colima.

I ran into some similar issues off macOS recently where I needed to pass --user $(id -u):$(id -g) as a parameter to docker run in order to be able to create the directories.

I can’t quite figure out why these are misbehaving on macOS.

[boukeversteegh]

I see. Well, Colima or Docker, this should work.

I read that Colima still relies on the docker-cli, so could you try the following command? The point is testing if docker can create the nested paths on the host system without running into chmod issues.

docker run --rm -it -v "$HOME/.dockerized/tmp/a/b:/work" golang find /work -ls

If this command fails with the same error, we know it's a docker issue, and we need to work around it (for example by pre-writing the directories). If it's not failing then there are the following few possibilities:

• See if it's docker-compose related. Then we should compare it with a docker-compose file which creates a new nested host path.
  • If that command succeeds, then dockerized must mess something up, by not completely following the docker-composed procedures.
• It could be something in Colima. Then some diagnostics logs could help clarify the process.

[halostatue]

❯ docker run --rm -it -v "$HOME/.dockerized/tmp/a/b:/work" golang find /work -ls
docker: Error response from daemon: error while creating mount source path '/Users/austin/.dockerized/tmp/a/b': chown /Users/austin/.dockerized/tmp/a/b: permission denied.

[halostatue]

This might be a lima issue (colima runs under lima): lima-vm/lima#505, lima-vm/lima#231.