#1152 Run every test in the default gate, classified by what it reaches - #1155
Merged
Conversation
`nmr test` and `nmr test:coverage` now select the `unit` and `tool` tiers, which name what a test reaches rather than how much of the codebase it covers, and `nmr test:tool` replaces `nmr test:integration`. Files still named `*.int.test.ts` match no tier, so the residual `unit` tier collects them and they run in the default gate until they are renamed.
Nine suites that reach `git` or `tsx` now carry the `tool` infix and run under `nmr test:tool`; the two that reach only the filesystem carry `unit`. All eleven run in the default gate and in CI, where previously none did. The real-library install suite is now `install-real-library.unit.test.ts`, beside the fixture-based `install.unit.test.ts`.
The two suites that install or deploy the whole content catalog now carry a 30-second budget of their own, matching what the tiers above `unit` are given. That work runs long under parallel-worker load, and the `unit` tier's default budget is too tight to absorb it. Every other suite in the package now runs on its tier's own budget rather than a package-wide ceiling, so a hung test fails in seconds instead of minutes.
Each Vitest config file now layers the repo's shared settings in through nmr's own mechanism rather than through wrapper functions of this repo's own, so a package can supplement or override them and can target a single test tier. The shared settings move to `.config/vitest/shared-options.ts`.
Build output no longer carries the `test-utils` directory, so `@codeassembly/kb` ships only modules a consumer can reach through its exports. The `files` entry that previously excluded it by hand is gone with it.
Project guidance now states how tests are grouped -- by what they reach while running, on nmr's four-tier ladder -- which tiers `nmr test` and CI cover, and how to classify a new test. It records that real-library suites no longer sit outside the gate, and that such a suite declares its own timeout instead.
Agent guidance now describes the commands nmr 0.24 actually provides: `nmr ci` as the code-quality gate with `nmr prepush` adding the audit, and the check-result cache that lets a command report a pass without re-running.
Eight suites across `agents`, `kb`, `fleet`, and `mcp` spawn `git` or a Node process and now carry the `tool` infix, so each takes that tier's 30-second budget and `nmr test:tool` selects it. Two of them had been left on Vitest's 5-second default when the package-wide ceiling in `agents` was removed. `fleet`'s git adapter suite and `mcp`'s stdio transport suite no longer declare a timeout of their own; the tier supplies a longer one.
The drift guard locates workspace packages through `fileURLToPath` rather than `URL.pathname`, so it still finds them when the repo is checked out under a path containing a space or a non-ASCII character. Previously it reported every package as missing a `vitest.config.ts`.
Project guidance now says a suite is `tool` when the code it drives spawns something, that neither a grep of the test file nor its import graph settles the question, and that running the suite against a logging shim does.
A reader classifying a test determines its reach by shimming `node:child_process` and running the tier, which surfaces a spawn made inside a dependency alongside a direct one. Shimming a named program instead only confirms a suspicion already held.
The browser-bundle suite runs in the `tool` tier, matching the esbuild binary it spawns through esbuild's JavaScript API. Its timeout rises from five seconds to the thirty the named tiers carry.
Dependency auditProduction dependency audit passed. |
williamthorsen
marked this pull request as ready for review
August 3, 2026 06:50
williamthorsen
added a commit
that referenced
this pull request
Aug 4, 2026
…e-v0.2.2 codeassembly-v0.4.0 factory-v0.2.2 codeassembly-mcp-v0.2.2 codeassembly-v0.4.0 - #1153 feat: Make codeassembly and kb CLI tools publishable (#1164) - #1091 fix: Anchor a project-deployed link where its target deploys (#1159) - #1156 refactor: Rename packages to publishable names (#1157) - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - refactor: Refine typescript-preferences - #1150 fix: State doc-description form and make comment mood opt-in (#1151) - deps: Upgrade first-party linting deps to latest version - #1137 feat: Establish personal rulebooks for code layout and TypeScript preferences (#1144) - #1126 feat: Check a package's own guidance content before it ships (#1138) - refactor: Fix lint - tooling: Remove redundant lint rules - #1133 fix: Reject an anchor link that names no heading (#1135) - #1122 feat: Capture lede decisions as an accumulating corpus (#1132) - #1125 feat: Honor invocation tokens in rulebook bodies (#1129) - #1107 feat: Render rulebook links and path tokens per harness (#1124) - #1115 feat: Sync guidance at build and install so an upgrade cannot leave it stale (#1123) - #1114 feat: Adopt a dependency's guidance by naming the package (#1121) - #1088 feat: Deliver project ambient rulebooks per harness (#1113) - #1087 refactor: Rename the authoring rulebook and mark its enforced rules (#1092) - #1095 tooling: Move compilation out of the install lifecycle into a bootstrap step (#1102) - #1094 refactor: Remove the ambient ripgrep dependency from the test suite (#1097) - deps: Upgrade all deps to latest version - tooling: Upgrade all deps to latest version & modernize configs - refactor: Fix lint - #1077 feat: Show the proposed edit above post-review menu options (#1078) - #1068 feat: Deliver ambient rulebooks mechanically, retiring GLOBAL.md (#1075) - #1069 feat: Add a no-second-person rule to the lede-voice doctrine (#1070) - #1050 feat: Add a redundancy rule to the lede-voice doctrine (#1058) - #1035 internal: Add lifecycle workspace with the canonical envelope, vocabulary & lane fold (#1049) - #1006 internal: Extend lifecycle-event instrumentation to five high-traffic skills (#1034) - #1028 feat: Rule out absence-of-removed-code tests (#1032) - #114 feat: Let reviewers emit gated insights into review artifacts (#1031) - #1027 internal: Retire input.received and redundant skill.progress emits (#1030) - #1026 fix: Name the side effects an approval ask authorizes (#1029) - #1022 fix: Normalize action and question label identifiers across asks blocks (#1025) factory-v0.2.2 - #1156 refactor: Rename packages to publishable names (#1157) - deps: Upgrade all deps to latest version - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - tooling: Remove redundant lint rules - deps: Upgrade deps to latest version - deps: Upgrade all deps to latest version - deps: Upgrade all deps to latest version - tooling: Upgrade all deps to latest version & modernize configs - refactor: Fix lint - deps: Upgrade all deps to latest version kb-v0.3.0 - #1153 feat: Make codeassembly and kb CLI tools publishable (#1164) - #1156 refactor: Rename packages to publishable names (#1157) - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - tooling: Remove redundant lint rules - #1095 tooling: Move compilation out of the install lifecycle into a bootstrap step (#1102) - tooling: Upgrade all deps to latest version & modernize configs - refactor: Fix lint codeassembly-lifecycle-v0.2.0 - #1153 feat: Make codeassembly and kb CLI tools publishable (#1164) - #1156 refactor: Rename packages to publishable names (#1157) - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - tooling: Remove redundant lint rules - #1095 tooling: Move compilation out of the install lifecycle into a bootstrap step (#1102) - tooling: Upgrade all deps to latest version & modernize configs - #1038 feat: Add the read-only git adapter for worktree and base-branch ground truth (#1059) - #1051 feat: Bound fold memory and rescan cost with a retention window (#1057) - #1035 internal: Add lifecycle workspace with the canonical envelope, vocabulary & lane fold (#1049) codeassembly-mcp-v0.2.2 - deps: Upgrade all deps to latest version - #1156 refactor: Rename packages to publishable names (#1157) - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - #1095 tooling: Move compilation out of the install lifecycle into a bootstrap step (#1102) - tooling: Upgrade all deps to latest version & modernize configs - refactor: Fix lint codeassembly-run-core-v0.2.2 - #1156 refactor: Rename packages to publishable names (#1157) - #1152 tooling: Run every test in the default gate, classified by what it reaches (#1155) - #1110 tooling: Migrate Vitest to nmr's centralized model (#1154) - #1095 tooling: Move compilation out of the install lifecycle into a bootstrap step (#1102) - tooling: Upgrade all deps to latest version & modernize configs - refactor: Fix lint
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Upgrades
nmrto 0.24, which changes Vitest configuration so that test suites are selected by a tier ("unit", "tool", "localhost", and "remote") corresponding to the services they use.nmr test:unitandnmr test:tooleach run one of these;nmr test:allruns every suite. All tests are covered by the default run.nmr test:integrationno longer exists, and no tests carry the.int.infix. The upgradednmrincludes a caching feature that skips checks that already succeeded against an identical working tree.Why
Eleven tests sat outside the default run and CI, so a break in the real content-library install surfaced only when someone remembered to run them by hand. The upgrade to nmr 0.24 retired the mechanism holding them there, and without a migration they would have rejoined the default run silently -- green, with nothing reporting that a deliberate separation had been lost.
The old grouping also sorted tests by how faithful their fixtures were, which never matched what actually made a test slow or fragile. Sorting by reach does, which is why the exclusion could be retired rather than reproduced: what protected the suite was always the timeout, not keeping the tests out of the gate.
Details
♻️ Refactoring
.config/vitest/shared-options.tsexports a plain options layer in place of the wrapper factories in the module it replaces, and all ten config files compose it through nmr's native layering.fileURLToPath, so it survives a checkout directory that needs escaping.🧪 Tests
agents,fleet,kb,lifecycle, andmcprenamed to state their group; 18 name the external-program group, which previously held none.agents.packages/fleet/src/adapters/__tests__/git.tool.test.tsandpackages/mcp/src/__tests__/protocol-stdio.tool.test.tsdrop their hand-rolled 20-second timeouts in favour of the group's budget, so the number lives in one place.⚙️ Tooling
test-utils/directories, andkb'sfileslist drops the negation that had excluded one.📦 Dependencies
@williamthorsen/nmr0.23.0 → 0.24.0.🤖 Agentic support
.agents/PROJECT.mddescribes the four groups, what the default gate runs, and how to determine reach when it passes through a dependency.Closes #1152