perf(build): stop unpacking node_modules wholesale from the Windows asar

[tsouth89]

Fixes the install-time and cold-start half of #5876.

What Changed

WINDOWS_ASAR_UNPACK was ["apps/server/dist/**", "**/node_modules/**"]. This inverts the CLI bundler's dependency rule — bundle everything except the packages that genuinely cannot be inlined — and narrows asarUnpack to exactly that set.

A package earns an exemption for one of two reasons:

• Native addons. A .node binary cannot be inlined into JS and must sit on disk for both the Windows primary and the Linux Node inside WSL. The JS wrappers that dlopen them count too (ffi-rs, @ff-labs/fff-node, msgpackr-extract, node-gyp-build), since they resolve their binary by real filesystem path at runtime.
• Bun-only entry points. @effect/platform-bun and @effect/sql-sqlite-bun are reached through a runtime-conditional dynamic import and resolve bun:sqlite, which does not exist when bundling for Node.

Both consumers now derive from one list in scripts/lib/cli-external-packages.ts, so they cannot drift.

Why

The Windows installer writes 14,687 files, 13,875 of them loose node_modules files, to support 20 native binaries. The entire Electron runtime is 22 files because it stays inside the archive.

That count costs twice: NSIS install time tracks file count, not bytes; and each file is a separate open/stat/scan the first time the server runs after an install, when the file cache is cold and the on-access scanner is not.

Measured on this repo, win/nsis x64:

	before	after
files written at install	14,687	1,192 (−92%)
loose node_modules files	13,875	370
native .node binaries	20	20
installer size	145.0 MiB	138.9 MiB

Cold start, extracting each build's payload to a fresh directory so the files had never been read, alternating run order between builds:

	before	after
server boot to Listening on	9,044ms / 10,160ms	3,667ms / 3,779ms
module load only (--version)	6,521 / 6,238 / 6,208ms	761 / 659 / 654ms

The main window is not created until the backend answers HTTP, so that ~6s comes off a cold launch.

Why one shared list

A package that is external but not unpacked still resolves on the Windows primary, which runs under ELECTRON_RUN_AS_NODE and reads app.asar transparently. It fails only under WSL. That asymmetry makes the drift invisible on the platform you are most likely to test on.

node-gyp-build-optional-packages hit exactly this while I was writing the patch — matched as external by the node-gyp-build prefix, missed by a glob without a trailing wildcard. There are tests for the invariant.

Verification

Extracted app.asar.unpacked into a directory with no node_modules ancestor — what plain node sees under WSL — and booted the server there. Migrations ran, it listened on 127.0.0.1, and no module failed to resolve. node-pty, ffi-rs, msgpackr-extract and @ff-labs/fff-node all load from that isolated tree.

scripts/build-desktop-artifact.test.ts (30) and the new scripts/lib/cli-external-packages.test.ts (7) pass. vp lint and @t3tools/server typecheck are clean.

One caveat on my verification: the build warned No WSL node-pty prebuild provided, so I exercised the WSL module resolution path with Linux-shaped constraints rather than a real WSL launch. Happy to rerun with a Linux pty.node prebuild if you want that closed before merging.

UI Changes

None.

Checklist

• This PR is small and focused
• I explained what changed and why
• I included before/after screenshots for any UI changes (n/a — no UI change)
• I included a video for animation/interaction changes (n/a — no motion change)

---

Note

Medium Risk
Desktop packaging and bundling policy changes can break WSL if a native or dlopen-dependent package is omitted from the shared external list; tests guard the external dependency closure but regressions would mainly surface on WSL, not the Windows primary backend.

Overview
Narrows Windows desktop packaging by inverting the server CLI bundler: JS runtime deps are inlined into apps/server/dist, and only packages that cannot be bundled (native addons, dlopen wrappers, and their on-disk JS closure, plus Bun-only entry points) stay external.

Adds scripts/lib/cli-external-packages.ts as the single source of truth for shouldBundleCliDependency (used by apps/server/vite.config.ts) and CLI_EXTERNAL_PACKAGE_UNPACK_GLOBS (used by WINDOWS_ASAR_UNPACK in scripts/build-desktop-artifact.ts), replacing wholesale **/node_modules/** asar unpack. New tests assert unpack globs match external prefixes and that runtime dependencies of external packages are not bundled away (a failure mode that only shows under WSL).

Updates WSL preflight in DesktopWslEnvironment.ts to resolve node-pty/package.json instead of effect when validating unpacked server deps, with matching user-facing error text for exit code 3.

^{Reviewed by Cursor Bugbot for commit bfd6089. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Stop unpacking all of node_modules from the Windows asar by bundling most CLI dependencies

• Introduces cli-external-packages.ts as a single source of truth for which packages stay external at runtime (native addons like node-pty, ffi-rs, FFI helpers) vs. build-only externals (bun-only entry points).
• Changes WINDOWS_ASAR_UNPACK in build-desktop-artifact.ts from unpacking **/node_modules/** wholesale to unpacking only apps/server/dist/** and globs derived from the external package list.
• Shares shouldBundleCliDependency between the desktop build script and vite.config.ts, replacing the local implementation in vite config.
• Updates the WSL health probe in DesktopWslEnvironment.ts to resolve node-pty/package.json instead of effect, targeting native packages that are still unpacked on disk.
• Behavioral Change: the asar now contains bundled JS dependencies instead of raw node_modules; only native addons and their wrappers are unpacked on the host filesystem.

^{Macroscope summarized bfd6089.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a5b6143-368f-4823-950d-ec575d1da4ed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR changes the packaging strategy from unpacking all node_modules to only specific native packages. While well-tested with dependency closure verification, it affects runtime file availability with platform-specific failure modes (works on Windows, can fail on WSL) that warrant verification.

^{You can customize Macroscope's approvability policy. Learn more.}

[tsouth89]

Fair verdict, and the concern was the right one — so I went and tested it under real WSL. It found a bug. Pushed a fix in 0aacacd.

What broke. node-gyp-build-optional-packages is external, so it loads from the real filesystem, so its own require resolves from the real filesystem too. It requires detect-libc, which was not on my list and therefore got bundled — present only inside app.asar. The Windows primary reads that transparently under ELECTRON_RUN_AS_NODE and resolves it fine. Plain node under WSL cannot. msgpackr-extract failed through the same chain.

Measured on Ubuntu 24.04 with Linux node v24.18.0, against the packaged tree copied out of the NSIS payload:

before:
  MISSING (cjs) msgpackr-extract [MODULE_NOT_FOUND] Cannot find module 'detect-libc'
  MISSING (cjs) node-gyp-build-optional-packages [MODULE_NOT_FOUND] Cannot find module 'detect-libc'
  PROBE-RESULT: 2 unresolved

after:
  OK   (cjs) msgpackr-extract
  OK   (cjs) ffi-rs
  OK   (cjs) node-gyp-build-optional-packages
  OK   (esm) @ff-labs/fff-node
  PROBE-RESULT: no resolution failures

t3 --version runs from that tree under Linux node in both cases.

Why the list alone was never going to be enough. The real invariant is that an external package's entire runtime dependency closure must be external. detect-libc isn't native and doesn't look special; no amount of reading the list surfaces it. So it's now a test, not a convention: it walks each runtime-external package's declared dependencies transitively and fails if any would be bundled away. It reproduces this exact failure on the old list.

Writing that test surfaced a distinction the single list had flattened, which I've now made explicit:

• Runtime-external (native addons, their dlopen wrappers, and their closure) — Node loads these from disk, so the closure must be external.
• Build-only external (@effect/platform-bun, @effect/sql-sqlite-bun) — external purely so the bundler never resolves bun:*. Node never loads them, so their closure genuinely doesn't need to be external. The closure test skips them deliberately.

On node-pty. It still fails to load under WSL, but that is pre-existing and unrelated: this build was produced without --wsl-prebuild, which the build itself warns about, and node_modules/node-pty/prebuilds/ contains only darwin-arm64, darwin-x64, win32-arm64, win32-x64 in both the baseline and the patched build. Identical either way, so nothing here changes it. Resolution succeeds; it's the native binary for the platform that's absent.

Cost of the fix: 6 files (1,192 → 1,198). Native binaries and installer size unchanged.

Happy to squash the two commits if you'd prefer a single one.

[tsouth89]

Confirmed, both points. Good catch — the guard was worse than useless, because it reported success. Fixed in 45b4517.

Point 1 is worse than "can pass". I probed it rather than reasoning about it. From scripts/lib, every seed failed:

FAILS  node-pty MODULE_NOT_FOUND
FAILS  msgpackr-extract MODULE_NOT_FOUND
FAILS  ffi-rs MODULE_NOT_FOUND
FAILS  node-gyp-build MODULE_NOT_FOUND
FAILS  detect-libc MODULE_NOT_FOUND
FAILS  node-addon-api MODULE_NOT_FOUND
FAILS  @ff-labs/fff-node ERR_PACKAGE_PATH_NOT_EXPORTED

From apps/server only its direct dependencies resolve; the transitive packages that actually caused the WSL breakage still don't. And ERR_PACKAGE_PATH_NOT_EXPORTED is a second hole the same catch swallowed — an exports map can refuse the /package.json subpath even when the package is right there.

The tell I missed at the time: when the guard first ran it reported violations only from the @effect/platform-bun chain, and never node-gyp-build-optional-packages -> detect-libc — the bug it was written for. It never read that package.

Point 2 confirmed. !prefix.endsWith("/") dropped @yuuang/, @ff-labs/ and @msgpackr-extract/ entirely. Seeding a queue with prefix strings was wrong anyway — a prefix isn't a package name.

Fix. Manifests are now read off disk from the pnpm store, which is the same tree asarUnpack globs target: it reaches transitive packages and isn't subject to resolution or exports semantics. Seeds come from what's actually installed and matches a prefix, so scoped prefixes are covered.

Plus a guard test that fails unless node-pty, node-gyp-build-optional-packages and detect-libc are actually found, so a check that reads nothing can't report success again.

Verified by mutation, not assertion. Removing detect-libc from the list now fails with exactly the real bug:

these dependencies of external packages would be bundled away and fail to
resolve under WSL: node-gyp-build-optional-packages -> detect-libc

The previous version of the test passed with detect-libc removed. That's the difference.

39 tests pass across this file and build-desktop-artifact.test.ts; vp lint clean; @t3tools/server typecheck exits 0.

[ikifar2012]

Hey @tsouth89,

Was having trouble launching this in WSL only mode ran it though an agent and found this

line 243 in apps/desktop/src/wsl/DesktopWslEnvironment.ts

needs to be updated from:

try { require.resolve("effect"); } catch (_e) { process.exit(3); }

to this

try { require.resolve("node-pty/package.json"); } catch (_e) { process.exit(3); }

Seems like the rest of the file is fine and after modification of that one line everything seems to be working as expected

[tsouth89]

Nice find, thanks. You're right about the cause: this branch inlines the server's JS deps into the bundle, so effect isn't on disk anymore and that probe was checking for something that no longer exists. Verified against the packaged tree, require.resolve("effect") fails while require.resolve("node-pty/package.json") resolves.

Pushed your fix in 2134d96. Used node-pty since it's external precisely because it can't be inlined, so it stays a valid sentinel either way. Also updated the two comments that still described the old behaviour, and the exit-3 message downstream.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2134d96. Configure here.}

[tsouth89]

Good catch, fixed in the latest push. Reworded it to name node-pty and the native packages that actually have to be unpacked, rather than effect.

Checked the rest of the file and the tests for other references to the old sentinel while I was in there. The only remaining mentions of effect are in the comment explaining why the sentinel changed.