`db reset` against an already-running stack leaves Kong's outbound template-serving listener (8088) permanently unbound

[AlbinoGeek]

Affected area

Database

Supabase CLI version

2.111.1-beta.8

Operating system

Linux (Fedora 44, kernel 7.1.5), Docker

Installation method

brew

Command

supabase db reset

Run against an already-running local stack (not a fresh stop/start).

Actual output

Kong's nginx no longer listens on 8088 at all. netstat -tlnp inside the Kong container shows only 8000 (proxy), 8001 (admin, loopback), 8443, 8444 (loopback) — no 8088. GoTrue's logs show continuous failures:

{"error":"templatemailer: template type \"magic_link\": Get \"http://<kong-container>:8088/email/magic_link.html\": dial tcp <kong-ip>:8088: connect: connection refused","event":"templatemailer_template_body_http_error","level":"error","mail_type":"magic_link", ...}

Repeating on GoTrue's own template-reload interval (GOTRUE_MAILER_TEMPLATE_RELOADING_ENABLED=true, ~10-12s) indefinitely — this does not self-heal.

The resulting email sent to a real user uses GoTrue's compiled-in default template (From: Admin <admin@email.com>, subject "Your sign-in link"), not the project's configured one. For a project like this one, whose custom template is written to not use {{ .ConfirmationURL }} (specifically because a GET-based confirmation URL is spent by mail-gateway link-prefetchers before the recipient ever clicks), the fallback default template silently reintroduces that exact GET-spends-the-token failure mode with no error surfaced anywhere in the request/response path a caller sees.

Expected behavior

Kong continues listening on 8088 (or is restarted with the rest of the
stack such that it starts listening again), and GoTrue's next email uses
the configured custom template.

Steps to reproduce

1. supabase start (fresh local project, config.toml has [auth.email.template.magic_link] with a content_path set).
2. Confirm Kong is listening on 8088 (from inside the Kong container: netstat -tlnp | grep 8088, or check GoTrue can fetch the template — trigger a magic-link email and confirm it uses the custom template).
3. Run supabase db reset (stack still up from step 1 — not a fresh stop/start).
4. Immediately check Kong's listeners again from inside the container.

Crash report ID

No response

Docker and service versions

Relevant images: `kong:2.8.1`, `gotrue:v2.194.0`

Additional context

Summary

Running supabase db reset against an already-running local stack leaves Kong's nginx process no longer listening on port 8088 — the internal-only port GoTrue uses to fetch custom email templates configured via [auth.email.template.*] in config.toml. Kong's own container is never restarted (docker inspect's State.StartedAt does not change across repeated resets), which rules out a container-recreation race. GoTrue then permanently falls back to its built-in default email templates — not a transient few seconds, but indefinitely (observed 13+ minutes continuously in one run) until Kong's container is manually docker restarted.

This is a stronger/different variant of #4668 ("Auth email templates fall back to defaults if Kong template endpoint is unavailable during startup"), which describes a brief startup race that self-heals once Kong becomes healthy. What's reported here does not self-heal — Kong's 8088 listener stays down indefinitely after db reset, and no restart is needed to observe it (unlike the startup case, repeated resets are not required either — a single db reset against a running stack reproduced it every time in a 3/3 test).

Root cause (as far as I was able to determine without CLI source access)

• Verified via docker inspect supabase_kong_<project> --format '{{.State.StartedAt}}' that Kong's container is not restarted or recreated by db reset when the stack is already running — the timestamp is identical before and after multiple consecutive resets.
• Verified via docker exec supabase_kong_<project> netstat -tlnp that the 8088 listener specifically disappears after a reset, while 8000/8001/8443/8444 remain bound.
• Verified via docker restart supabase_auth_<project> (the workaround documented in Auth email templates fall back to defaults if Kong template endpoint is unavailable during startup #4668) alone: does not fix it — GoTrue's own template-reload loop keeps retrying and keeps getting connection refused, because the target (Kong) is still missing the listener.
• Verified via docker restart supabase_kong_<project> alone: fixes it immediately — nginx rebinds all its configured listeners (including 8088) on a genuine process restart, and GoTrue's next reload cycle (within ~10s) successfully fetches the real template.
• A genuinely fresh supabase stop --no-backup && supabase start (full container recreation) does not reproduce the persistent failure — Kong binds 8088 within ~1s of container start in 3/3 trials, and any single failed fetch attempt from GoTrue (a real race, but sub-second) self-heals on GoTrue's own next reload cycle 10s later.

So the fault appears specific to whatever db reset's "Restarting containers..." step does to an already-running Kong container — it's neither a full container restart (ruled out by StartedAt) nor apparently a full "do nothing" (the listener does go away) — possibly an nginx worker-reload path (I did not see a SIGHUP logged in Kong's own output at the time of the reset, for what that's worth) that drops one listen socket without logging an error.

Suggested workaround for others hitting this

docker restart supabase_kong_<project-id> after any db reset run against an already-running stack reliably fixes it. We've now baked this into our own project's db:reset wrapper script as an unconditional step, since no HTTP-level healthcheck can distinguish "Kong is fine" from "Kong's outbound template port specifically is gone" without direct Docker access.

Relationship to #4668

Filing as a new issue rather than a comment on #4668 since the failure shape is meaningfully different: #4668 is a brief, self-healing startup race; this is a persistent, non-self-healing failure triggered by db reset against an already-running stack, where the documented #4668 workaround (restart auth) does not help — only restarting Kong does. Happy to fold into #4668 or open as a duplicate/related issue if maintainers prefer.

[AlbinoGeek]

I'm going to find every race in this CLI lol, from daily driving. Not even looking for them :P.