Skip to content

Optimize Docker dev workflow: faster builds, startup, and dependency iteration#585

Merged
marcvergees merged 6 commits into
fireform-core:developmentfrom
chetanr25:optimisations
Jun 23, 2026
Merged

Optimize Docker dev workflow: faster builds, startup, and dependency iteration#585
marcvergees merged 6 commits into
fireform-core:developmentfrom
chetanr25:optimisations

Conversation

@chetanr25

Copy link
Copy Markdown
Collaborator

This PR overhauls the Docker-based development workflow to make the everyday loop dramatically faster. Switching branches, restarting the stack, and changing dependencies all used to cost minutes; they now cost seconds. The changes touch how dependencies are installed, how the image is built, how the container starts up, and how the make targets behave.

Motivation

The local development experience had three recurring pain points:

  1. Starting the stack was slow. Bringing services up routinely took well over a minute, and most of that time was spent waiting on container health checks and on reinstalling dependencies that hadn't actually changed.
  2. make up did not reflect dependency changes. Adding a package to requirements.txt and running make up would silently keep running the old environment, so contributors had to fall back to the heavier full-setup command, which was slow.
  3. Switching branches was painful. Reviewing other contributors' work or hopping between your own branches — frequently triggered a near-complete reinstall, making quick context switches expensive.

These costs added up across a day and discouraged testing each other's branches, which directly slows down collaboration.

What changed

Faster dependency installation

Dependency installation now uses a modern, much faster installer instead of plain pip, backed by a persistent cache that survives across builds. The result is that even a cold install completes in a fraction of the previous time, and repeat installs are effectively instant.

We also explicitly request CPU-only builds of the heavy machine-learning wheels. Previously the image could pull in large GPU/CUDA components that are useless in this environment, bloating both download and build time.

Smarter build strategy

The development image is optimized for build speed (it is a local image, so size matters less), while the production image is optimized for small size via a multi-stage build. This split removes redundant work from the dev build path while keeping production images lean.

Pick up dependency changes without a full rebuild

The container now records which dependency set it was built with and, on startup, only reinstalls when the live requirements.txt actually differs. When nothing changed, startup skips the install entirely.

For the common "I just added one package" case, a dedicated fast path installs only the new dependency into the already-running container in a second or two no image rebuild and no waiting.

Faster container startup

Service health checks were tuned so that databases and supporting services report healthy as soon as they are actually ready, rather than idling until the next scheduled check. This alone removed the bulk of the fixed startup delay.

Better make workflow

  • make up no longer rebuilds the image, so routine starts are near-instant; application code is already live via the bind mount.
  • A new target installs newly added dependencies into the running container without a rebuild.
  • A new status target prints a compact, one-line-per-container health summary, and both startup commands end with a clear "ready" banner.

CI alignment

Continuous integration installs dependencies with the same fast installer, making CI runs quicker and consistent with local development.

Performance results

Measured on the same machine, before vs. after these changes:

Scenario Before After
Switch branches, then start the stack ~2 min 2 s ~23 s
Start again with nothing changed ~14-21 s ~2 s
Change a dependency and bring the stack up ~1 min 48 s ~3 s

The previous "dependency changed" path also required using the heavier
full-setup command because make up would not pick up the change at all;
that is no longer necessary.

Why this matters

The dev loop is now fast enough that switching branches to test a PR, restarting after a change, or adding a dependency are no longer disruptions. Lowering this friction makes it much more practical for contributors to run and review each other's work, which is the main goal.

Notes and follow-ups

  • A large share of the remaining image size and rebuild cost comes from a heavy machine-learning dependency chain pulled in transitively by the form-detection library. A full image rebuild is still proportional to that size. Investigating a lighter installation path for that dependency is a worthwhile follow-up that would shrink the image and speed up every build step further.
  • These changes are split across focused commits: the image and compose changes, the developer workflow (make targets and startup script), and the CI workflow updates.

@chetanr25 chetanr25 marked this pull request as ready for review June 23, 2026 12:47
@marcvergees marcvergees merged commit d74efbc into fireform-core:development Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants