[BUG] Random failure when mixing multiple platforms of the same image

[sysblade]

Description

When using docker-compose to up multiple containers using multiple platform, the up command would randomly fail, it would work fine when rerun it less than 1s after, but would otherwise rethrow the same error:

Error response from daemon: image with reference debian was found but its platform (linux/arm64/v8) does not match the specified platform (linux/amd64)

Steps To Reproduce

docker-compose.yml

services:
  hello-amd64:
    image: debian
    platform: linux/amd64
    entrypoint: "echo hello"

  hello-arm64:
    image: debian
    platform: linux/arm64
    entrypoint: "echo hello"

Run docker compose up multiple time, quickly, without doing any other change

> docker compose up
[+] Running 2/2
 ✔ hello-amd64 Pulled                                                                                                                                                                    1.2s
 ✔ hello-arm64 Pulled                                                                                                                                                                    1.2s
[+] Running 1/3
 ⠋ Container docker-bug-hello-arm64-1               Creating                                                                                                                             0.0s
 ⠋ Container 8c92a008ec7b_docker-bug-hello-amd64-1  Recreate                                                                                                                             0.0s
 ✔ Container docker-bug-hello-amd64-1               Removed                                                                                                                              0.0s
Error response from daemon: image with reference debian was found but its platform (linux/arm64/v8) does not match the specified platform (linux/amd64)


> docker compose up
[+] Running 2/2
 ✔ hello-amd64 Pulled                                                                                                                                                                    1.2s
 ✔ hello-arm64 Pulled                                                                                                                                                                    1.2s
[+] Running 1/1
 ✔ Container docker-bug-hello-arm64-1  Recreated                                                                                                                                         0.0s
Attaching to 8c92a008ec7b_docker-bug-hello-amd64-1, hello-arm64-1
hello-arm64-1                          | hello
8c92a008ec7b_docker-bug-hello-amd64-1  | hello
hello-arm64-1 exited with code 0
8c92a008ec7b_docker-bug-hello-amd64-1 exited with code 0

It would then work randomly work or throw error in a non consistent way

Compose Version

Docker Compose version 2.36.0

Docker Environment

Client: Docker Engine - Community
 Version:    28.1.1
 Context:    colima
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.23.0
    Path:     /opt/homebrew/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.36.0
    Path:     /opt/homebrew/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 7
  Running: 1
  Paused: 0
  Stopped: 6
 Images: 6
 Server Version: 28.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-50-generic
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 5.773GiB
 Name: colima
 ID: 228bb634-46b0-4d85-ab51-e55cc1f7fda8
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[sysblade]

Looks related to orbstack/orbstack#1848 ?

[ndeloof]

AFAICT this issue only affects docker engine with Containerd integration disabled: Without containerd image store there will be a race for the tag; first container will see the image, but wrong platform. Second container will either see the tag, or will trigger a pull (which overwrites the other image)

See https://docs.docker.com/engine/storage/containerd/ to enable this feature

[sysblade]

Should there be some kind of warning showing up in docker compose ? That should be quite easy to detect at runtime.

[ndeloof]

For sure, we could search for docker info's driver-type: io.containerd.snapshotter.v1 as documented,
but IMHO this is a corner case to run the same image with distinct architectures, I wonder this deserves a dedicated diagnostic mechanism.