feat: add Docker Compose deployment driver and image management by j4n · Pull Request #6 · chatmail/cmlxc

j4n · 2026-04-22T15:30:22Z

This adds a docker deploy driver that builds a Docker image in the builder,
transfers it to an LXC relay container (with nesting enabled), and
starts it via docker compose. DNS zones are extracted from the
running container and loaded into PowerDNS.
Specifically:

Add docker build, docker list, and docker prune subcommands for
managing the Docker image cache in the builder independently of any
relay deployment.
Add docker logs, docker ps, docker shell, and docker pull for
inspecting running deployments and pulling pre-built GHCR images.
Support --image PATH on docker deploy to load a pre-exported
tarball, skipping the build step entirely. zstd compression is
used when available, plain tar otherwise.
Update README with Docker usage examples, driver documentation, and
image management commands.
Already used in CI: the chatmail/docker repo's workflow calls
cmlxc docker deploy --source ghcr:TAG to integration-test each
image build (pending trigger PR in chatmail/relay).

Try it:

# Initialize the environment (if not already done)
cmlxc init

# Deploy a relay via Docker
cmlxc docker deploy --source @main dk0

# Or pull a pre-built image from GHCR
cmlxc docker deploy --source ghcr:main dk0

# Or pre-build an image and deploy from tarball
cmlxc docker build --source @main --output ./chatmail.tar.zst
cmlxc docker deploy --image ./chatmail.tar.zst dk1

# Inspect running services
cmlxc docker ps dk0
cmlxc docker logs dk0 -f

# Manage the image cache
cmlxc docker list
cmlxc docker prune --dry-run
cmlxc docker prune --all

# Run tests against the Docker relay
cmlxc test-mini dk0
cmlxc test-cmdeploy dk0

Containers with Docker or other networking can expose IPs on multiple interfaces. _extract_ip() now accepts an optional subnet filter so wait_ready() and list_managed() only pick addresses on incusbr0.

Move the initialization check (DNS container running + base image present) from cli._check_init() into Incus.check_init() so that drivers can call it without depending on the CLI module.

Allows drivers to pass additional Incus config keys (e.g. security.nesting=true for Docker-in-LXC) when launching containers. Threaded through Container and RelayContainer.

Fresh containers from cached images have stale package lists, causing dnsutils install to fail with unmet dependencies.

Move the initenv.sh hook from CmdeployDriver.on_init_relay() into the Driver base class as the default implementation -- both cmdeploy and docker drivers used identical bodies. Extract run_cmdeploy_pytest() as a standalone function so that any driver sharing the cmdeploy test suite (currently CmdeployDriver and DockerDriver) can call it without duplicating the env_exports / pytest command construction.

initenv.sh already uses uv when available. This ensures it's installed on the builder container so all drivers benefit. Installs to /usr/local/bin so it's on PATH for non-interactive shells.

When the source ref is a full 40-char SHA (e.g. from CI dispatch), the shallow git-main clone won't have it. Detect this case and fetch just that commit with --depth 1 before checkout.

Query running Docker Compose service names from a container via incus exec. Used by the docker driver's ps subcommand and SSH config generation.

When no explicit -v flags are passed and RUNNER_DEBUG=1 is set (GitHub Actions "Enable debug logging" rerun), auto-bump to -vvv.

- Register DockerDriver in DRIVER_BY_NAME - test-cmdeploy: dispatch to driver class from container metadata, add --relay-ref option - Fix _print_builder_repos to use driver REPO_NAME (avoids dupes)

- Add cmlxc_ref input to test feature branches - Disable AppArmor for Docker-in-LXC systemd support - Cache localchat-docker image (strip Docker images before export) - Split cache into restore/save for better failure handling - Per-service failure diagnostics (dovecot, postfix, failed units) - install incus-base instead of full incus package

Add DockerDriver for deploying chatmail relays via Docker Compose inside LXC containers (Docker-in-LXC with security.nesting). Driver capabilities: - Build images from relay source, tag by git SHA - Transfer between builder and relay via piped docker save/load - Pull pre-built images from GHCR (--source ghcr:TAG) - Load local tarballs (--image, zstd if available) - Healthcheck polling with --since log streaming at -vv - SSH forwarding into Docker containers for test compatibility - DNS zone extraction and PowerDNS loading - Tiered image pruning (default/deep/all) - security.privileged gated behind CI=true CLI subcommands: deploy, build, list, pull, logs, ps, shell, prune

j4n · 2026-04-23T09:14:51Z

+          # Disable AppArmor restrictions so Docker-in-LXC containers
+          # can run systemd (needs cgroup notification socket access).
+          sudo systemctl stop apparmor || true
+          sudo apparmor_parser -R /etc/apparmor.d/* 2>/dev/null || true


not entirely sure if this is needed or if we can just run lxc with unbound profile

j4n · 2026-04-23T09:16:08Z

+            incus exec "$c" -- journalctl --no-pager -n 200 || true
+            # Dump Docker container logs if present
+            svc=chatmail
+            if incus exec "$c" -- docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q "$svc"; then


we could consider making this a helper script in the dockerized build to facilitate log extraction for debugging.

j4n · 2026-04-23T09:18:42Z

+          # Publish the builder LXC container as a cached image (the Docker
+          # container inside gets recreated on compose up, so the LXC is clean).
+          # Only skip localchat-cmdeploy on failure -- it bakes deploy state
+          # directly into the LXC and would carry broken config into the next run.


clarify that we can cache the docker image container as well because its not containing state for this case - maybe clean up /opt completely

j4n · 2026-04-23T09:19:53Z


 Standard workflow:
-init -> deploy-cmdeploy/deploy-madmail -> test-cmdeploy/test-madmail/test-mini.
+init -> deploy-cmdeploy/deploy-madmail/docker deploy -> test-*/test-mini.


that does not make sense

j4n · 2026-04-23T09:26:16Z

-        for name in DRIVER_BY_NAME:
-            path = f"/root/{name}-git-main"
+        seen = set()
+        for name, drv_cls in DRIVER_BY_NAME.items():


clarify change

j4n · 2026-04-23T09:27:15Z

        cfg = []
        cfg += ("-c", f"{LABEL_KEY}=true")
        cfg += ("-c", f"{LABEL_DOMAIN}={self.domain}")
+        if extra_config:


purge, was added for --slow thats now gone; I did drop commit bc9f0 but it seems its still in here

j4n · 2026-04-23T09:29:25Z

            out.print(f"  Fetching {cls.REPO_NAME}-git-main from upstream ...")
            bld_ct.bash(f"cd {tmp_dest} && git fetch origin")

+        # Install uv for faster venv/pip operations (used by initenv.sh)


this did speed things up a bunch, but with cached images shouldnt might not give us much and depends on a relay pr thats not there yet.

j4n · 2026-04-23T09:30:53Z

+                    f"cd {repo_path} && "
+                    f"git fetch --depth 1 origin {source.ref}"
+                )
+            elif source.ref != "main":


need to double check this catches all variants we need.

j4n · 2026-04-23T09:32:23Z

 CMDEPLOY = "cmdeploy"


+def run_cmdeploy_pytest(driver, second_domain=None):


should be called run test_cmdeploy

j4n · 2026-04-23T09:34:02Z

+        self._bridge_subnet = NotImplemented
+
+    @property
+    def bridge_subnet(self):


could clarify why we need it

j4n · 2026-04-23T09:35:47Z

+    cmlxc docker ps dk0
+    cmlxc docker logs dk0
+    cmlxc docker logs dk0 -f
+


maybe detail image cleanup as well

j4n · 2026-04-23T09:38:35Z

+            "config",
+            "set",
+            ct.name,
+            "security.nesting=true",


double check - we're setting this in the ci again? necessary?

j4n · 2026-04-23T09:42:40Z

+    """Display docker disk usage summary from builder."""
+    raw = bld_ct.bash("docker system df", check=False)
+    if raw:
+        for line in raw.strip().splitlines():


this printer wraparound is ripe for a wrapper

j4n · 2026-04-23T09:43:35Z

+        )
+
+
+_PRUNE_COMMANDS = {


this probably should come first, check if the autoprune is mentioned in all the right places

j4n · 2026-04-23T09:46:47Z

+    )
+
+
+def build_docker_cmd(args, out):


this image building has a substantial linecount with export etc, maybe we can take it out, estimate how much it would save. build then would require either local docker or a pushed image from the registry, which might just be enough.

j4n · 2026-04-23T09:47:40Z

+        "service",
+        nargs="?",
+        default=DOCKER_COMPOSE_SERVICE,
+        metavar="SERVICE",


unnecessary metavars

j4n · 2026-04-23T09:48:27Z

+    )
+    group.add_argument(
+        "--all",
+        dest="prune_all",


could reduce LOC by making this a subcommand maybe

j4n · 2026-04-23T09:51:49Z

+
+        self.ct.write_deploy_state(DOCKER, source=source)
+
+    def _load_local_image(self):


thats nice to have but not really needed probably, along with export. either we deal with docker and have it, or we dont I suppose

j4n · 2026-04-23T09:53:03Z

+            ("healthcheck state",
+             f"docker inspect {svc} --format '{{{{json .State.Health}}}}' 2>/dev/null"),
+            ("dovecot journal",
+             f"docker exec {svc} journalctl -u dovecot --no-pager -n 30 2>&1"),


this section is duplicated I think

j4n · 2026-04-23T09:53:46Z

+                    self.out.print(f"  {line}")
+
+    def _patch_container_ini(self):
+        """Apply test rate-limit overrides inside the Docker container.


we added this but cmdeploy already has this we should leverage this probably

j4n · 2026-04-23T09:54:10Z

+        )
+
+    def _load_dns(self, dns_ct):
+        """Extract DNS zone from Docker container and load into PowerDNS."""


must be done in every driver, consolidate?

j4n · 2026-04-23T09:55:38Z

+            mode="600",
+        )
+
+    def _get_image_relay_sha(self):


I smell duplication

The `git reset --hard origin/{ref}` is only useful for branch refs (fast-forward to latest remote). For SHA refs it always fails silently since there's no remote tracking branch. Only run it for branch refs.

so we can reuse it for docker

- Extract get_image_label_sha() helper, dedup docker inspect label extraction in pull_image and _get_image_relay_sha - Extract _print_indented() helper for show_docker_df and _dump_docker_logs - Remove redundant metavar="RELAY" and metavar="SERVICE" - Simplify prune: positional level arg instead of --deep/--all flags - Clarify ensure_docker() docstring (why nesting is set here)

j4n marked this pull request as draft April 22, 2026 15:30

j4n added 12 commits April 22, 2026 17:31

fix: filter container IPs by Incus bridge subnet

9bffa89

Containers with Docker or other networking can expose IPs on multiple interfaces. _extract_ip() now accepts an optional subnet filter so wait_ready() and list_managed() only pick addresses on incusbr0.

refactor: extract check_init() to Incus class

225461e

Move the initialization check (DNS container running + base image present) from cli._check_init() into Incus.check_init() so that drivers can call it without depending on the CLI module.

feat: add extra_config parameter to container launch()/ensure()

bc9f098

Allows drivers to pass additional Incus config keys (e.g. security.nesting=true for Docker-in-LXC) when launching containers. Threaded through Container and RelayContainer.

fix: apt-get update before installing dnsutils

52309ab

Fresh containers from cached images have stale package lists, causing dnsutils install to fail with unmet dependencies.

feat(builder): install uv in prep_builder for faster venv setup

c35df2f

initenv.sh already uses uv when available. This ensures it's installed on the builder container so all drivers benefit. Installs to /usr/local/bin so it's on PATH for non-interactive shells.

feat(driver_base): support SHA-based source refs in init_builder

0e90f91

When the source ref is a full 40-char SHA (e.g. from CI dispatch), the shallow git-main clone won't have it. Detect this case and fetch just that commit with --depth 1 before checkout.

feat(incus): add _get_docker_services helper

98f5127

Query running Docker Compose service names from a container via incus exec. Used by the docker driver's ps subcommand and SSH config generation.

feat(cli): auto-detect RUNNER_DEBUG for verbose output

2c0d4f1

When no explicit -v flags are passed and RUNNER_DEBUG=1 is set (GitHub Actions "Enable debug logging" rerun), auto-bump to -vvv.

feat(cli): integrate Docker driver, --relay-ref

e3bb9c0

- Register DockerDriver in DRIVER_BY_NAME - test-cmdeploy: dispatch to driver class from container metadata, add --relay-ref option - Fix _print_builder_repos to use driver REPO_NAME (avoids dupes)

j4n force-pushed the j4n/docker-support branch from 59363ea to 20a246d Compare April 22, 2026 15:35

j4n commented Apr 23, 2026

View reviewed changes

Comment thread src/cmlxc/driver_docker.py

mode="600",

)

def _get_image_relay_sha(self):

Copy link
Copy Markdown

Author

j4n Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I smell duplication

j4n added 8 commits April 28, 2026 08:56

docker: always rebuild test venv

589f990

docker: set env CHATMAIL_IMAGE for docker compose, doc

5c932ed

docker: replace --relay-ref with $RELAY_REF

c45c656

refactor(init_builder): skip git reset --hard for SHA refs

a105e70

The `git reset --hard origin/{ref}` is only useful for branch refs (fast-forward to latest remote). For SHA refs it always fails silently since there's no remote tracking branch. Only run it for branch refs.

refactor(cmdeploy): extract TEST_INI_OVERRIDES

a999e91

so we can reuse it for docker

refactor(cmlxc,docker): rename run_cmdeploy_pytest run_test_cmdeploy

81f80d6

refactor(docker): use TEST_INI_OVERRIDES from driver_cmdeploy

6efa309

j4n force-pushed the j4n/docker-support branch from 0e2b4d0 to d5f6b04 Compare April 28, 2026 08:37

		CMDEPLOY = "cmdeploy"


		def run_cmdeploy_pytest(driver, second_domain=None):


		self.ct.write_deploy_state(DOCKER, source=source)

		def _load_local_image(self):

		)


		_PRUNE_COMMANDS = {

Conversation

j4n commented Apr 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

j4n Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

j4n Apr 23, 2026 •

edited

Loading