Skip to content

[Bug]: Web server started from tmux can silently disappear after sibling pane OOM kill #1591

Description

@Rhiz3K

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Start the web server from a tmux pane:
    bun run --cwd apps/server start -- --mode web --host 0.0.0.0 --port 3773 --no-browser
  2. Keep that server running for a while in tmux.
  3. In the same user session / tmux child scope, run a memory-hungry process until the machine hits global OOM.
  4. Watch what happens to the T3 web session.

Expected behavior

If some unrelated process in the same user/tmux scope gets OOM-killed, the T3 web server should either:

  • survive, or
  • restart automatically, or
  • at minimum leave a clear app-level signal that it died.

Actual behavior

The web UI just disappears because the tmux child scope gets marked oom-kill and the T3 process is gone. From the T3 side there is no obvious crash log or graceful shutdown signal in server.log, so it looks like the app randomly vanished.

What made this especially confusing is that the server data/state was fine; only the long-running process/session disappeared.

Impact

This makes long-running remote web sessions feel flaky. If you are hosting T3 in tmux on a low-memory box, one unrelated OOM event can nuke the web server and leave you thinking T3 crashed.

Version or commit

main @ a60daa1c764a97e88f2eef63bbac960b79274ed6 (v0.0.15-6-ga60daa1c, package version 0.0.15)

Environment

  • OS: Ubuntu 24.04.4 LTS
  • Kernel: 6.8.0-106-generic
  • Runtime: Bun 1.3.3, Node v24.12.0
  • Mode: web
  • Start command: bun run --cwd apps/server start -- --mode web --host 0.0.0.0 --port 3773 --no-browser
  • Access pattern: remote over Tailscale
  • CPU: 16 vCPU, Intel Xeon Processor (Skylake, IBRS, no TSX) on KVM
  • RAM: 30 GiB
  • Swap: 8 GiB

Logs or stack traces

Kernel / systemd around the failure:

Mar 30 14:26:03 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=fail2ban.service,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service/tmux-spawn-081f04ee-3589-4c98-b8f8-4f71d236845e.scope,task=python3,pid=1472448,uid=1000
Mar 30 14:26:03 kernel: Out of memory: Killed process 1472448 (python3) total-vm:17173720kB, anon-rss:16904704kB, file-rss:3072kB, shmem-rss:0kB, UID:1000 pgtables:33696kB oom_score_adj:0
Mar 30 14:26:03 systemd[1]: user@1000.service: A process of this unit has been killed by the OOM killer.
Mar 30 14:26:03 systemd[1733]: tmux-spawn-081f04ee-3589-4c98-b8f8-4f71d236845e.scope: A process of this unit has been killed by the OOM killer.
Mar 30 14:26:07 systemd[1733]: tmux-spawn-081f04ee-3589-4c98-b8f8-4f71d236845e.scope: Failed with result 'oom-kill'.

The killed process itself was:

[1472448] 1000 1472448 4293430 4226944 4226176 768 0 34504704 15584 0 python3

And at the same time, T3's own server.log had no app crash stacktrace or shutdown message. It only showed the previous healthy startup.

Screenshots, recordings, or supporting files

No response

Workaround

Run T3 under systemd --user with Restart=always instead of relying on a tmux session for persistence. That immediately solved it for me.

Extra context

This may not be a pure T3 bug so much as a rough edge in the recommended "just run it in tmux" style of deployment. But from the user perspective the failure mode is brutal: the app vanishes, there is no obvious T3-level clue why, and the fix ends up being "don't host it in tmux if you care about uptime".

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions