[BUG] `docker compose up --watch` crashing on file changes

[antoniodcorrea]

Description

Current behavior:

With a minimal node service setup, the command docker compose watch behaves as expected, while docker compose up --watch crashes.

Expected behavior:

Using docker compose up --watch should reload service on file changes.

Steps To Reproduce

Given this tree:

.
├── docker-compose.yml
└── service
    ├── Dockerfile
    ├── package.json
    └── src
        └── index.js

docker-compose.yml

services:
  service:
    build:
      context: ./service
    develop:
      watch:
        - action: sync+restart
          path: ./service
          target: /node_workdir
    command: npm run test

Dockerfile

FROM node:latest

# Use a specific user
RUN useradd -ms /bin/sh -u 1001 node_user
USER node_user

# Set workdir and install dependencies
WORKDIR /node_workdir
COPY --chown=node_user:node_user . .

package.json

{
  "scripts": {
    "test": "node ./src/index.js"
  }
}

index.js

setInterval(() => console.log(new Date().toUTCString()), 500);

Running this with docker compose watch behaves as expected, even with file changes:

 ✔ Container docker-compose-watch-example-service-1  Started                                                                                     0.7s 
Watch enabled
Syncing "service" after changes were detected
[+] Restarting 1/1
 ✔ Container docker-compose-watch-example-service-1  Started                                                                                     0.8s 
Syncing "service" after changes were detected
[+] Restarting 1/1
 ✔ Container docker-compose-watch-example-service-1  Started         

Now running the same image with docker compose up --watch will success with this output to console:

[+] Running 2/0
 ✔ Network docker-compose-watch-example_default      Created                                                                                     0.0s 
 ✔ Container docker-compose-watch-example-service-1  Created                                                                                     0.0s 
Attaching to service-1
           ⦿ Watch enabled
service-1  | 
service-1  | > test
service-1  | > node ./src/index.js
service-1  | 
service-1  | Sat, 31 Aug 2024 10:09:19 GMT
service-1  | Sat, 31 Aug 2024 10:09:20 GMT

But when I modify the contents of index.js, the container crashes with this output:

service-1  | Sat, 31 Aug 2024 10:10:00 GMT
service-1  | Sat, 31 Aug 2024 10:10:01 GMT
           ⦿ Syncing "service" after changes were detected
[+] Restarting 0/1
[+] Restarting 0/1r-compose-watch-example-service-1  Restarting                                                                                  0.2s 
 ⠼ Container docker-compose-watch-example-service-1  Restarting                                                                                  0.5s 
service-1  | npm error path /node_workdir
service-1  | npm error command failed
service-1  | npm error signal SIGTERM
service-1  | npm error command sh -c node ./src/index.js
service-1  | npm notice
service-1  | npm notice New patch version of npm available! 10.8.2 -> 10.8.3
service-1  | npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.3
service-1  | npm notice To update run: npm install -g npm@10.8.3
[+] Restarting 1/1otice
 ✔ Container docker-compose-watch-example-service-1  Started                                                                                     0.8s 
service-1 exited with code 0
           ⦿ Watch disabled
~/Desktop/docker-compose-watch-example ➤ a02f451|master
9184 ± :                                                    

Compose Version

Docker Compose version v2.29.1-desktop.1

Docker Environment

Client: Docker Engine - Community
 Version:    26.0.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     /Users/USER_NAME/.docker/cli-plugins/docker-scout

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.0-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 7.655GiB
 Name: docker-desktop
 ID: fdc04834-89c4-4be5-80c0-663c9fd5361e
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/USER_NAME/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

[ndeloof]

This error in logs is how npm reacts to SIGTERM signal send by restart command
You can get the same using docker kill --signal SIGTERM <container> as your application is running. Same happens as you stop the compose application, just logs are already detached and you don't get those in your console, but you can retrieve them using docker logs xxx-service-1.

This has no other impact but pollute your logs: container is actually restarted with updated content.

[antoniodcorrea]

This error in logs is how npm reacts to SIGTERM signal send by restart command You can get the same using docker kill --signal SIGTERM <container> as your application is running. Same happens as you stop the compose application, just logs are already detached and you don't get those in your console, but you can retrieve them using docker logs xxx-service-1.

This has no other impact but pollute your logs: container is actually restarted with updated content.

Thanks for the update. The issue is not about the logs themselves, but about the container crashing and not being able to restart.
If this crash is intended behavior, I would be curious about the use case for docker compose up --watch on node applications using npm.

[ndeloof]

The container is not "crashing", but is restarted : SIGTERM signal is sent to stop process, then (re)start. Also, AFAICT the container restarts successfully (at least, it does for me with your sample configuration)

[antoniodcorrea]

Thanks a lot. This is interesting, for me the posted example is crashing —docker desktop 26.0.0 on OSX 14.0—:

service-1 exited with code 0
           ⦿ Watch disabled
~/Desktop/docker-compose-watch-example ➤ a02f451|master
9184 ± :                   

[antoniodcorrea]

Actually, you are right, it is not crashing, but detaching the console.
After reload I can still access to its logs in docker, but it does not react anymore to changes.

[ndeloof]

that's unexpected. Compose should detect container has restarted and (re)attach. Unfortunately I can't reproduce

[jangita]

Actually, you are right, it is not crashing, but detaching the console. After reload I can still access to its logs in docker, but it does not react anymore to changes.

Similar when I'm doing python/django - as a temporary fix I use a separate file monitoring tool then I remove the restart option on my docker-compose.yml watch section. In your case maybe nodemon for now?

[manzanit0]

I'm having the same issue: the console detaches. The sync+restart action works fine and the new container is started, but without trailing the logs of the new container.

Some details:

$ docker compose version
Docker Compose version v2.27.1-desktop.1

Compose sub-snippet:

  golang:
    container_name: golang-foo
    build:
      context: ./
      dockerfile: dev.Dockerfile
    command: go run ./cmd/api
    ports:
      - 8080:8080
    develop:
      watch:
        - action: sync+restart
          path: ./
          target: /usr/src/app
    environment:
      - PGUSER=root
      - PGPASSWORD=password
      - PGDATABASE=mcduck_db
      - PGHOST=postgresql
      - PGPORT=5432
    networks:
      - mcduck
    depends_on:
      - jaeger
      - postgresql

[jhrotko]

@manzanit0 I believe that issue was fixed in version v2.29.2

[antoniodcorrea]

As @jhrotko pointed out, this issue has been fixed by #12014 and released in v2.29.2.
But it was closed as Closed as not planned, which is confusing for anyone coming here, and probably should be closed as Closed as it is not a feature, but a bug.