Optimize Docker dev workflow: faster builds, startup, and dependency iteration#585
Merged
Merged
Conversation
marcvergees
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR overhauls the Docker-based development workflow to make the everyday loop dramatically faster. Switching branches, restarting the stack, and changing dependencies all used to cost minutes; they now cost seconds. The changes touch how dependencies are installed, how the image is built, how the container starts up, and how the
maketargets behave.Motivation
The local development experience had three recurring pain points:
make updid not reflect dependency changes. Adding a package torequirements.txtand runningmake upwould silently keep running the old environment, so contributors had to fall back to the heavier full-setup command, which was slow.These costs added up across a day and discouraged testing each other's branches, which directly slows down collaboration.
What changed
Faster dependency installation
Dependency installation now uses a modern, much faster installer instead of plain
pip, backed by a persistent cache that survives across builds. The result is that even a cold install completes in a fraction of the previous time, and repeat installs are effectively instant.We also explicitly request CPU-only builds of the heavy machine-learning wheels. Previously the image could pull in large GPU/CUDA components that are useless in this environment, bloating both download and build time.
Smarter build strategy
The development image is optimized for build speed (it is a local image, so size matters less), while the production image is optimized for small size via a multi-stage build. This split removes redundant work from the dev build path while keeping production images lean.
Pick up dependency changes without a full rebuild
The container now records which dependency set it was built with and, on startup, only reinstalls when the live
requirements.txtactually differs. When nothing changed, startup skips the install entirely.For the common "I just added one package" case, a dedicated fast path installs only the new dependency into the already-running container in a second or two no image rebuild and no waiting.
Faster container startup
Service health checks were tuned so that databases and supporting services report healthy as soon as they are actually ready, rather than idling until the next scheduled check. This alone removed the bulk of the fixed startup delay.
Better
makeworkflowmake upno longer rebuilds the image, so routine starts are near-instant; application code is already live via the bind mount.CI alignment
Continuous integration installs dependencies with the same fast installer, making CI runs quicker and consistent with local development.
Performance results
Measured on the same machine, before vs. after these changes:
The previous "dependency changed" path also required using the heavier
full-setup command because
make upwould not pick up the change at all;that is no longer necessary.
Why this matters
The dev loop is now fast enough that switching branches to test a PR, restarting after a change, or adding a dependency are no longer disruptions. Lowering this friction makes it much more practical for contributors to run and review each other's work, which is the main goal.
Notes and follow-ups
maketargets and startup script), and the CI workflow updates.