Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/server/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ with metadata, Suspense, streaming) for HTML, or `api.js` /
| `stream.js` | Server-side stream-action builders (#248). `stream.append/prepend/before/after/replace/update/remove(target, content?)` compose the `<webjs-stream action target>` HTML (one `<template>` per insert action) the client `renderStream` / `<webjs-stream>` element applies surgically; `streamResponse(...parts)` wraps them in a `Response` carrying `STREAM_MIME` (`text/vnd.webjs-stream.html`); `acceptsStream(req)` reports whether the request negotiated the stream path (its `Accept` carries the MIME), the seam an app branches on so a JS-off form returns a normal render. The target id is attribute-escaped. Pure + dependency-free; the content is NOT escaped (server-authored HTML). A page `action` returning `streamResponse` is honored verbatim by `page-action.js` (`runPageAction` returns a returned `Response` as-is). Exported from `index.js` |
| `check.js` | Convention validator backing `webjs check`. Correctness-only; rules include `use-server-exports-callable` (#464: a `.server.{js,ts}` file declaring `'use server'` must export at least one CALLABLE; the registrar registers only function exports, so a file exporting zero functions, or only a non-function `const` / a type / only verb config, registers nothing and the call 404s silently. Asserts "exports a callable", not "returns a value" (a void or `redirect()`-throwing action is fine); conservative on a re-export or a factory-produced const, which might be a function. The complement of `use-server-needs-extension` and `one-action-per-configured-file`), `no-browser-globals-in-render`, `no-non-erasable-typescript`, `no-server-import-in-browser-module` (a page / layout / component that SHIPS to the browser, i.e. the build does NOT elide it, must not transitively import a server-only `.server.{ts,js}` UTILITY: that import becomes a throw-at-load stub in the browser and crashes the page at runtime while passing typecheck. Reuses the build's own elision verdict (`analyzeElision` over the module graph, scanned components, and route table) so it fires ONLY on modules that genuinely ship; a display-only page the framework elides is never flagged. Also covers shipping components and the always-shipped `error` / `loading` / `not-found` route modules (never elided). Skips `'use server'` ACTIONS, which resolve to a working RPC stub and are the legitimate way to call the server from a shipping module, and skips imports written inside code-example strings (the module-graph scanner masks string-embedded `import`s, so a docs `<pre>` sample never becomes an edge). Scope note for dynamic imports (#751): a string-literal `import('./x.ts')` is now a GATE edge (servable), but this rule's server-import detection still runs over STATIC edges, so a dynamic `import('./x.server.ts')` of a no-`'use server'` utility is not flagged (its throw-at-load is deferred to call time). A computed `import(expr)` is not a check rule at all (it would false-positive on a valid computed npm / reachable-app import, failing the check-is-correctness-only line); the dev server surfaces it with a 404 hint when the target 404s.) |
| `vendor.js` | Resolve bare-specifier npm deps. `resolveVendorImports(appDir, getBareImports)` reads `.webjs/vendor/importmap.json` if present (committed pin file) and short-circuits BEFORE running the bare-import scan; only when there is no pin file does it invoke the `getBareImports` thunk (the whole-app `scanBareImports` walk) and call `api.jspm.io/generate`. The scan keeps framework-internal packages off the jspm path BEFORE the generate call: `@webjsdev/core` is served locally (the `BUILTIN` set), and the server-only `@webjsdev/cli` / `@webjsdev/server` / `@webjsdev/mcp` (the `FRAMEWORK_SERVER_ONLY` set) are excluded (#713) so a server-only package never reaches jspm, leaving the 401 fallback below for genuine third-party private deps. `jspmGenerate` resolves the WHOLE install set in ONE generate call (a single `install[]` array) so jspm computes one mutually-consistent graph (#446): a direct dep and a transitive that needs a newer version of the same package share one URL instead of skewing (direct pinned local, transitive floating to jspm-latest -> missing-export crash). Per-package isolation survives as a FALLBACK ONLY: on a permanent 401 (an unresolvable private/server-only install) it probes each install alone, drops the unresolvable one(s), and re-runs the unified call over the resolvable subset so survivors stay coherent; on a transient 5xx/network failure it serves merged per-install fragments and flags the resolve for retry (`lastLiveResolveFailed`). `pinAll` builds the same `install[]` and calls the same `jspmGenerate`, and now ALSO persists the flattened transitive entries the unified resolve returns (`derivePinParts` recovers their pkg/version/subpath from the resolved URL), so a `webjs vendor pin` snapshot and the live runtime importmap agree on the same specifier->URL set for a given dep set (the vendor-runtime parity invariant). So a pinned app does no vendor static analysis at boot (runtime-first); the elision-aware prune of a pinned map (`prunePinToReachable`) runs lazily in `ensureReady`, not at boot. **SRI integrity (sha384, keyed by the FINAL URL) is returned on BOTH paths.** The pin path returns the committed `integrity` verbatim; the LIVE path computes it after resolving via `computeLiveIntegrity`, which fetches each cross-origin (`https://`) target and hashes the raw bytes (`fetchLiveIntegrity` -> `sha384Integrity`), skipping same-origin `/__webjs/...` targets (#235). Bounded (parallel with a small concurrency cap + a per-fetch timeout) and FAIL-OPEN: a bundle fetch failure skips that one URL's integrity and emits a single count-based `console.warn`, never breaking the resolve, so a CDN hiccup cannot take the app down. Hashes are cached per process by URL (`liveIntegrityCache`, cleared by `clearVendorCache`) so a re-resolve does not re-fetch an immutable bundle; this is NOT a persistent cache (that is the pin file's job). The returned `integrity` map keys on the same FINAL URL that `vendorIntegrityFor(url)` looks up, so ssr.js's `integrityAttr` / importmap emission fires for free on the live path too. Backs the `webjs vendor pin / unpin / list / audit / outdated / update` CLI surface plus the `--from <provider>` (jspm, jsdelivr, unpkg, skypack) and `--download` modes. `--download` mode also serves cached bundle files from `.webjs/vendor/`. After a `pin`, the CLI calls `ensureVendorCommittable(appDir)`: vendoring is opt-in, so the pins are meant for source control, and a `.gitignore` that excludes `.webjs/` would swallow them. The helper probes `git check-ignore`; if the pin output is ignored it heals the app's own `.gitignore` (rewrites a bare `.webjs` directory exclusion to the contents-glob form, then appends the `!.webjs/vendor/` negation, then re-probes and reverts if a broader rule still wins), and otherwise is a clean no-op. It NEVER fabricates a `.gitignore` (a parent-repo or `.git/info/exclude` source yields a printed notice instead), so the no-vendor default is untouched. **Importmap-coherence validation (#450):** `checkImportmapCoherence(imports, { getManifest })` inspects a PRODUCED importmap (it does NOT re-resolve, that is #446's job) and for each resolved package checks that the version pinned for every OTHER resolved package it declares a dependency / peer range on satisfies that range, returning conflicts naming both packages, the range, and the pinned version (defense-in-depth for a hand-edited pin, a partial vendor pin, or the #446 skew). Pure in `(imports, getManifest)` and built on the dependency-free `satisfiesSemverRange` + `extractPinnedVersions` (version parsed from the importmap URL), so the SAME pinned dep set yields the SAME verdict over a live importmap and a vendored `.webjs/vendor/importmap.json` (the runtime-vs-vendored parity invariant). Surfaced via `webjs doctor` (warn-only), degrades to `unverified` when a manifest is unavailable, and never warns on a semver range shape it cannot statically evaluate. |
| `module-graph.js` | Dependency graph for transitive preload hints. Both walks (`transitiveDeps` for preloads, `reachableFromEntries` for the auth gate) stop at `.server.*` boundaries, so a preload set is always a subset of the servable set. The import scanner runs its regexes over the FULLY-BLANKED mask (`redactStringsAndTemplates(src, true)`, which blanks every comment / string / template / regex body to spaces, delimiters kept) and reads each specifier back from raw `src` via the match's group indices (the `d` flag). Scanning over the blanked mask means a stray `import`/`export` word inside ANY literal or comment (example code in an `html\`\`` template, a `// comment`, a `/regex/`) can never anchor a regex match whose lazy body spans across the literal's closing delimiter into the NEXT real `from '<spec>'` and consumes it, silently dropping that real edge (a differential test against a real AST caught this class across the comment / template / regex delimiters, #753). A differential test (`packages/server/test/scanner-fuzz/`) proves the lexer's import-edge, `register()` / `customElements.define()`, and WebComponent class-body extraction agrees with a real TypeScript parse over the repo corpus plus adversarial fixtures (the register/class corpus tolerates only verdict-safe over-matches, never a MISS). **Dynamic-import edges (#751):** a string-literal `import('./widget.ts')` (matched by `DYNAMIC_IMPORT_RE`, the same redaction-mask + `#`-alias rules as the static scan) is tracked as a SEPARATE edge class kept in a `WeakMap` keyed by the graph (read via `dynamicEdges(graph)`). `reachableFromEntries` (the gate) unions these in so a lazily-imported app module is servable instead of 404ing, and a dynamically-imported module's own static subtree is walked too; but `transitiveDeps` (preload) and the elision analysis stay on the STATIC graph only, so a dynamic import is admitted-but-not-preloaded (lazy by author intent) and never flips an elision verdict. The `.server.*` boundary holds for dynamic edges (a dynamic `import('./x.server.ts')` is admitted as a stub, not traversed into). A computed `import(expr)` cannot be captured and stays out (a `webjs check` warning surfaces it). **`#` path-alias expansion (#555):** `appImportsMap(appDir)` reads + caches the app's `package.json "imports"` map, and `expandImportAlias(spec, appDir)` expands a matching `#`-prefixed specifier (e.g. `#lib/db.server.ts` under the scaffold's catch-all `"#*": "./*"`) to its real app-relative target. `resolveImport` calls it BEFORE the relative branch and `parseFile` lets alias specs through, so the graph / auth gate / elision / `no-server-import-in-browser-module` all see the REAL path (an alias cannot launder a `.server.ts` past the boundary). Key-shape-agnostic (wildcard + exact, any base); `IMPORTS_CACHE` is cleared per appDir on each `buildModuleGraph`. **Bare (npm vendor) edges (#754):** a bare specifier (`dayjs`, `@scope/pkg/sub`) is NOT a static graph edge (the gate / elision are unchanged), but the exact specifier is recorded per file in a SEPARATE `WeakMap` keyed by the graph (read via `bareImports(graph)`) so `ssr.js` can map it to a vendor importmap URL and emit a `modulepreload` (flattening the CDN waterfall one level). `node:` builtins + protocol specifiers are excluded; because the scan runs over the fully-blanked mask (#753), a `from '<spec>'` written inside example code cannot become a phantom vendor edge (the earlier #754 quote-position guard is subsumed by the blanked-mask scan). |
| `module-graph.js` | Dependency graph for transitive preload hints. Both walks (`transitiveDeps` for preloads, `reachableFromEntries` for the auth gate) stop at `.server.*` boundaries, so a preload set is always a subset of the servable set. The import scanner runs its regexes over the FULLY-BLANKED mask (`redactStringsAndTemplates(src, true)`, which blanks every comment / string / template / regex body to spaces, delimiters kept) and reads each specifier back from raw `src` via the match's group indices (the `d` flag). Scanning over the blanked mask means a stray `import`/`export` word inside ANY literal or comment (example code in an `html\`\`` template, a `// comment`, a `/regex/`) can never anchor a regex match whose lazy body spans across the literal's closing delimiter into the NEXT real `from '<spec>'` and consumes it, silently dropping that real edge (a differential test against a real AST caught this class across the comment / template / regex delimiters, #753). A differential test (`packages/server/test/scanner-fuzz/`) proves the lexer's import-edge, `register()` / `customElements.define()`, and WebComponent class-body extraction agrees with a real TypeScript parse over the repo corpus plus adversarial fixtures (the register/class corpus tolerates only verdict-safe over-matches, never a MISS). **Dynamic-import edges (#751):** a string-literal `import('./widget.ts')` (matched by `DYNAMIC_IMPORT_RE`, same `#`-alias rules as the static scan) is tracked as a SEPARATE edge class kept in a `WeakMap` keyed by the graph (read via `dynamicEdges(graph)`). The dynamic scan does NOT run over the fully-blanked mask the static scan uses (which blanks `${...}` template-hole code too): it runs over `redactToPlaceholders(src)` (#918), which keeps hole code readable so a dynamic import written directly inside a hole (`html\`${import('./x.ts')}\``, real code in an expression position) is captured instead of 404ing, while string / template-TEXT bodies are tokenized to `__STR_<idx>__` (the specifier recovered from the `literals` array) and comment / regex bodies blanked, so a dynamic import written as literal TEXT is still not an edge. The static `IMPORT_RE` / `EXPORT_FROM_RE` stay on the blanked mask (a statement cannot appear in a `${}` hole, so only the dynamic form needs hole-awareness) and `DYNAMIC_IMPORT_RE` is tightly anchored on `import(` (no lazy cross-delimiter span), so the placeholder scan does not reopen the #753 swallow class. `reachableFromEntries` (the gate) unions these in so a lazily-imported app module is servable instead of 404ing, and a dynamically-imported module's own static subtree is walked too; but `transitiveDeps` (preload) and the elision analysis stay on the STATIC graph only, so a dynamic import is admitted-but-not-preloaded (lazy by author intent) and never flips an elision verdict. The `.server.*` boundary holds for dynamic edges (a dynamic `import('./x.server.ts')` is admitted as a stub, not traversed into). A computed `import(expr)` cannot be captured and stays out (a `webjs check` warning surfaces it). **`#` path-alias expansion (#555):** `appImportsMap(appDir)` reads + caches the app's `package.json "imports"` map, and `expandImportAlias(spec, appDir)` expands a matching `#`-prefixed specifier (e.g. `#lib/db.server.ts` under the scaffold's catch-all `"#*": "./*"`) to its real app-relative target. `resolveImport` calls it BEFORE the relative branch and `parseFile` lets alias specs through, so the graph / auth gate / elision / `no-server-import-in-browser-module` all see the REAL path (an alias cannot launder a `.server.ts` past the boundary). Key-shape-agnostic (wildcard + exact, any base); `IMPORTS_CACHE` is cleared per appDir on each `buildModuleGraph`. **Bare (npm vendor) edges (#754):** a bare specifier (`dayjs`, `@scope/pkg/sub`) is NOT a static graph edge (the gate / elision are unchanged), but the exact specifier is recorded per file in a SEPARATE `WeakMap` keyed by the graph (read via `bareImports(graph)`) so `ssr.js` can map it to a vendor importmap URL and emit a `modulepreload` (flattening the CDN waterfall one level). `node:` builtins + protocol specifiers are excluded; because the scan runs over the fully-blanked mask (#753), a `from '<spec>'` written inside example code cannot become a phantom vendor edge (the earlier #754 quote-position guard is subsumed by the blanked-mask scan). |
| `importmap.js` | Browser import-map builder. `setCoreInstall(coreDir, distMode)` binds the importmap to the resolved `@webjsdev/core` install and runs `buildCoreEntries()`, which reads the package's `package.json` and derives one importmap line per exported subpath from its `exports` field, picking the `default` condition in dist mode and the `source` (`src/*.js`) condition otherwise. In dist mode the browser surface is ONE self-contained bundle: the `exports` `default` for the always-load browser subpaths (`/directives`, `/context`, `/task`, `/client-router`) all point at `dist/webjs-core-browser.js`, so those entries plus the bare specifier collapse onto that single file (each import picks its named exports from it) instead of a fan of per-subpath bundles + code-split chunks. `/lazy-loader` keeps its own file (on-demand). In src/dev mode each subpath stays granular (`src/*.js`) since there is no bundle to collapse into. `dev.js` calls `setCoreInstall` at boot based on `existsSync(coreDir/dist/webjs-core.js) && existsSync(coreDir/dist/webjs-core-browser.js)`. The bare `@webjsdev/core` specifier always points at the BROWSER entry (`index-browser.js` or `dist/webjs-core-browser.js`); the slim entry drops `renderToString`, `renderToStream`, and `setCspNonceProvider` so server-only bytes do not ride the wire. Node-side consumers resolve via the package.json exports and still get the full `index.js`. `buildImportMap({ fingerprint })` content-hashes each same-origin target via `asset-hash.js`'s `withAssetHash` when `fingerprint` is true (the served map); the internal `importMapHash()` computation passes `false` so the published build id stays a stable per-deploy fingerprint independent of per-file hashes (#243). `vendorPreconnectOrigins(max?)` derives the cross-origin vendor CDN origins from the resolved vendor map (`_extraEntries`), most-common first + bounded, for the auto vendor preconnect (#243): returns `[]` for a same-origin pinned / empty map. **`#` alias browser scopes (#555):** `importAliasBrowserEntries(importsMap, topLevelDirs)` derives the browser importmap entries for the app's `"imports"` aliases, derived from the SAME map the server resolver reads (lockstep). The scaffold's catch-all `"#*": "./*"` expands into one trailing-slash prefix scope per top-level dir (`#lib/` -> `/lib/`, ...; a bare `#` cannot prefix-match, so dev.js's `appTopLevelDirs` scan supplies the dirs and a new folder is covered on the next boot); a per-dir or exact key maps directly. `setImportAliasEntries` binds them at boot and folds them into `buildImportMap`. **`vendorPreloadTargets(specifiers)` (#754):** maps a set of reached bare specifiers to `[{ href, integrity }]` taken DIRECTLY from `buildImportMap().imports[spec]` (byte-identical to the importmap target, so the browser does not double-fetch) + the matching `integrity`; excludes `@webjsdev/core*` (same-origin, already on the boot path), dedups by href, and DROPS a specifier absent from the importmap (unpinned / unreached / elided, so no over-fetch). `ssr.js` feeds it the reached vendor set and emits a `modulepreload` per target. |
| `component-scanner.js` | Maps every webjs component class to its browser-visible URL |
| `component-elision.js` | Static analyser deciding which display-only component modules can be elided from the browser, plus the serve-time side-effect-import stripper. Conservative denylist of interactivity signals (single source of truth). `analyzeElision` also returns `shippedRouteModules` (#646): for each page/layout that ships whole (neither inert nor import-only), the first client-effecting blocker that pins it (a non-component in its closure, or `null` when the module's own code is the cause) plus a human `reason`. A reporting layer over the existing verdict, consumed by the `webjs doctor` advisory |
Expand Down
Loading
Loading