Skip to content

[Bug]: OpenCode leaks orphaned /opencode serve processes that survive backend crash/restart #5241

Description

@HI0890

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Start T3 Code client + T3 Remote (i have the remote done via npx t3@nightly serve --host "$(tailscale ip -4)")
  2. Start a new session with a OpenCode provider and send a message
  3. Confirm the server is live and find its parent ps -eo pid,ppid,pgid,args | grep "[o]pencode serve"
  4. Abruptly kill the backend - Simulating a crash / kill -9 / force-quit "C^
  5. Observe the orphan - same command as step 2 -> the opencode serve is still alive but now with PPID = 1 (reparent ed to init)
  6. Restart the T3 backend and wait for the session-reaper threshold -> ps -eo pid,ppid,pgid,etime,rss,args | awk '$2==1 && /[o]pencode serve/' -> the orphaned opencode serve would remain untouched as noting reclaims it only a manual kill or reboot of the machine would clear it.

Expected behavior

A provider server process that T3 spawns should have its lifetime strictly bounded by the T3 backend that owns it, regradless of how that backend terminates. Every one of these should tear the opencode serve down:

  • Thread Stop button / Esc -> serve exits
  • Graceful backend shutdown (normal quit, SIGTERM that completes teardown) -> all serves exit
  • Abrupt backend death (crash, SIGKILL, OOM-kill, force-quit) -> all serves exit
  • Backend restart -> no leftover opencode serve from prior runs; clean slate
  • Prolonged inactivity -> idle serves reaped (the 30-min reaper)

Actual behavior

  • Thread Stop / Esc -> serve exits
  • Graceful quit /SIGTERM -> serves exit
  • SIGKILL / crash / OOM / Force-quit -> serve survices - Orphaned to PID 1, never exits
  • Backend restart -> prior serves still running, invisible to the new backend (still visible in the ui and actionable but not owned by the Backend Process) i think
  • Inactivity reaper -> cannot reach orphans - it only closes session owned by a live backend

Damage — resources held by 28 orphaned opencode serve (measured live)

Resource Measured Notes
Orphaned processes 28 all PPID == 1, PGID == <own pid>, ages 34 min – 10 days
RAM — PSS (realistic) ~8.8 GB unique + shared-fraction; ~93% of OpenCode footprint is waste
RAM — Private (USS) ~8.8 GB non-shareable, non-reclaimable until killed
RAM — RSS (nominal) ~9.6 GB double-counts shared libs
Swap pinned ~1.6 GB system actively swapping due to zombies
Worst single orphan 752 MB RSS / 705 MB USS fresh repro hit ~476 MB within minutes
CPU already consumed ~6.5 days cumulative 449 idle threads still get scheduled
Open file descriptors 709 (max 31/proc) toward per-user ulimit -n
Listening TCP ports 28 127.0.0.1:33817–46541, ephemeral range
opencode.db on disk 13 GB shared, cannot be checkpointed (see below)
opencode.db-wal 23 MB cannot checkpoint — 56 zombie handles hold it open
opencode.log 128 MB held open (append) by all 28

Top orphans by RSS:

PID Age RSS Port
1295328 4d 21h 752 MB 38007
2414521 3d 22h 486 MB 44449
436765 34 min (fresh repro) 476 MB 40381
2839687 9d 21h 415 MB 41025
2439553 10d 3h 372 MB 41351

Impact

Major degradation or frequent failure

Version or commit

main @ e60821f or t3: 0.0.32-nightly.20260802.980

Environment

Linux 7.0.0-27-generic x86_64, Node v24.18.0. Opecode 1.18.3

Logs or stack traces

No stack trace or log line exists for this: abrupt backend death precludes logging, and the orphaned opencode serve then runs silently. The evidence is the process table — 28 orphaned serves (PPID=1,PGID==PID, ages up to 10 days) holding ~9 GB RAM, 28 loopback ports, and 56 handles on a shared 13 GB opencode.db whose WAL can no longer checkpoint.

Screenshots, recordings, or supporting files

No response

Workaround

A potential option on how to fix this:

  1. Record the spawn i.e small JSON file or something,
  2. Reap on startup i.e for each record check if its pid is alive AND its PPID is 1 (i.e its been orphaned) -> kill it. if its pid is dead -> drop the entry. then rewrite the file with whatever is still legitimely alive.

but this depends on how often you "restart" T3 backend (i.e right now with lots of updates so would this be fine but a process which does not depend on the "restart" rate would be good later)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.needs-triageIssue needs maintainer review and initial categorization.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions