docker-compose-migrate
_{Zero-downtime Docker Compose migration toolkit}

Export entire Docker Compose stacks — project files, named volumes, and database dumps — from one Linux host and import them on another. Fully interactive, single-archive, no registry required.

---

How It Works

 SOURCE HOST                          TARGET HOST
┌──────────────────────┐             ┌──────────────────────┐
│                      │             │                      │
│  migrate-export.sh   │  ── SCP ──▶ │  migrate-import.sh   │
│                      │  .tar.gz    │  (auto-generated)    │
│  1. Discover projects│             │                      │
│  2. Dump databases   │             │  1. Restore volumes  │
│  3. Export volumes   │             │  2. Start services   │
│  4. Package archive  │             │  3. Restore DB dumps │
│                      │             │                      │
└──────────────────────┘             └──────────────────────┘

The export script discovers all Docker Compose projects on the source, performs application-level database dumps while services are still running, then safely stops containers to export volume data. Everything is packaged into a single .tar.gz archive — including an auto-generated import script that handles the reverse process on the target.

---

Supported Databases

Engine	Dump Method	Restore Method
PostgreSQL / TimescaleDB	pg_dumpall → gzip	psql pipe
MySQL	mysqldump --all-databases → gzip	mysql pipe
MariaDB	mariadb-dump / mysqldump → gzip	mysql pipe
MongoDB	mongodump --archive --gzip	mongorestore --archive
Redis	BGSAVE → docker cp RDB	stop → cp → start

---

Quick Start

Prerequisites

• Source host: Docker Engine + Compose (v2 plugin or standalone), jq, rsync, root access
• Target host: Docker Engine + Compose

1 — Upload the export script

scp migrate-export.sh user@source-host:~

2 — Run the export

ssh user@source-host
sudo ./migrate-export.sh

Follow the interactive prompts to select projects, confirm DB dumps, and stop/restart services.

3 — Transfer the archive

# From the source (as root, archive is mode 600)
sudo scp /tmp/compose-migration-*.tar.gz user@target-host:~

4 — Run the import

ssh user@target-host
tar xzf compose-migration-*.tar.gz
cd compose-migration-*/
chmod +x migrate-import.sh
sudo ./migrate-import.sh

The import script will restore volumes, pull images, start services, then auto-detect and restore database dumps with interactive confirmation.

---

Repository Contents

.
├── migrate-export.sh      # Run on source — export everything
├── .gitignore
├── .gitattributes         # Enforces LF line endings for .sh files
└── README.md

migrate-import.sh is not a standalone file — it is embedded inside migrate-export.sh and generated into the archive at export time.

---

What Gets Exported

Data	How
Compose project files	rsync (excludes node_modules, .git, __pycache__, .venv)
Resolved compose config	docker compose config snapshot
Named Docker volumes	4-method discovery: compose config → full-name construction → label filter → prefix scan
Database dumps	Application-level dumps while DB is running (see table above)
Volume map	volume-map.txt — lossless safe_filename ↔ original_name mapping
Dump manifest	dump-manifest.txt — `filename

---

Safety & Security

🔒	Secure temp files	mktemp -d with random suffix — immune to symlink attacks in /tmp
🔒	Archive permissions	Mode 600 (root-only) — secrets in .env files are not world-readable
🔒	Concurrent-run lock	flock prevents two simultaneous exports from racing
🔒	DB passwords hidden	MySQL/MariaDB passwords passed via -e MYSQL_PWD=... — not visible in process list
🛡️	Signal handling	Ctrl+C / crash auto-restarts stopped services via trap handler
🛡️	Disk space check	Warns if /tmp has less than 1 GB before starting
🛡️	SHA-256 checksum	Generated for the archive — verify integrity with sha256sum -c
🛡️	Dump validation	Detects trivially empty dumps (gzip header only) and removes them
⚠️	Bind mount warnings	Detects host-path mounts outside the project directory
⚠️	Unexported volume warnings	Flags volumes attached to containers that weren't exported
⚠️	Build service warnings	Flags services needing docker compose build on the target
📋	Restore log	All DB restore operations logged to restore.log with OK/FAIL/SKIP status

---

Manual DB Restore (Fallback)

If auto-restore is skipped or fails, restore dumps manually:

# PostgreSQL
gunzip -c <container>-dbdump.sql.gz | docker exec -i <container> psql -U postgres

# MySQL / MariaDB
gunzip -c <container>-dbdump.sql.gz | docker exec -i <container> mysql -u root -p

# MongoDB
docker exec -i <container> mongorestore --archive --gzip < <container>-dbdump.archive.gz

# Redis — use stop/cp/start, NOT restart (prevents RDB overwrite on shutdown)
docker stop <container>
docker cp <container>-dbdump.rdb <container>:/data/dump.rdb
docker start <container>

---

Common Issues & Tips

SSH key permissions

SSH refuses keys that are readable by other users. Before using a .pem key:

Linux / macOS:

chmod 600 key.pem

Windows (PowerShell):

icacls key.pem /inheritance:r /grant:r "$($env:USERNAME):R"

Using a PEM key with SCP and SSH

If your server uses key-based authentication (e.g. AWS Lightsail), pass the key with -i:

# Upload
scp -i key.pem migrate-export.sh user@source-host:~

# Connect
ssh -i key.pem user@source-host

Archive owned by root — can't download

The export script runs as root, so the archive in /tmp is root-owned. If you need to SCP it as a regular user, fix permissions first:

# On the source host
sudo chmod 644 /tmp/compose-migration-*.tar.gz

Then download normally:

scp -i key.pem user@source-host:/tmp/compose-migration-*.tar.gz .

Alternatively, download directly via sudo scp from the source or use SSH piping:

ssh -i key.pem user@source-host "sudo cat /tmp/compose-migration-*.tar.gz" > archive.tar.gz

Script not executable after upload

chmod +x migrate-export.sh

Windows line endings breaking the script

If you edited a .sh file on Windows, it may have \r\n line endings that bash cannot parse. Fix with:

sed -i 's/\r$//' migrate-export.sh

---

Notes

• Non-destructive — each run creates a timestamped archive; safe to run multiple times
• Images are not bundled — they are pulled fresh on the target via docker compose pull
• Post-migration checklist:
  1. Verify services: docker ps
  2. Update DNS / firewall rules to the new host
  3. Test thoroughly before decommissioning the source
  4. Delete archives from both hosts
  5. Rotate credentials on the new host

---

Tested On

• Source: Amazon Lightsail (Ubuntu) running ThingsBoard and Node-RED Docker Compose stacks
• Target: Ubuntu Server (Proxmox VM)

---

License

MIT