feat: wire up CspIndex orchestrator + global index cache (#18)

.claude/agent-memory/please-implement-executor/MEMORY.md

@@ -0,0 +1,12 @@
+# Memory Index
+
+- [csp typecheck baseline is red by design](csp-typecheck-baseline-red.md) — TS5097 .ts-import errors are project-wide; gate on "no new type errors", not "green tsc"
+- [csp test files are ahead of impl](csp-test-files-ahead-of-impl.md) — some indexing *.test.ts depend on not-yet-existing APIs; a scoped source fix won't make them green
+- [csp worktree test env gotchas](csp-worktree-test-env-gotchas.md) — full `bun test` is flooded by tmpfs ENOSPC and ESLint can't run (no jiti); trust isolated runs
+- [csp bun mock.module is irreversible](csp-bun-mock-module-irreversible.md) — top-level mock.module leaks process-wide across files; use DI/static reassignment instead
+- [csp dense roundtrip has no float drift](csp-dense-roundtrip-no-drift.md) — SelectableBasicBackend save→load is bit-stable; re-normalizing unit vectors is idempotent; NFR-002 safe, T007 can reuse .load
+- [csp loadFromDisk model dim alignment](csp-loadfromdisk-model-dim-alignment.md) — reloaded stub model is fixed 256-dim; align to persisted backend dim or query() throws dim mismatch
+- [csp upstream has no disk cache](csp-upstream-no-disk-cache.md) — semble has no cache.py; global ~/.csp/index cache is csp-original (#162, unported); cache-key STOP gates won't fire
+- [csp loadOrBuildIndex cache contract](csp-loadorbuild-cache-contract.md) — local reuse gated by source-file hash via save(dir,{contentHash}); git keyed by URL+ref only
+- [csp cli cache DI seam](csp-cli-cache-di-seam.md) — cli auto-cache via injectable loadOrBuild seam, build-branch only; mcp (T012) must mirror same key contract
+- [csp clear index safety guard](csp-clear-index-safety-guard.md) — clear index deletes ONLY ~/.csp/index via clearIndexCache; AC-015 guard (basename==='index' && !=home) before rmSync; clear all = index + clearSavings as two independent calls

.claude/agent-memory/please-implement-executor/csp-bun-mock-module-irreversible.md

@@ -0,0 +1,28 @@
+---
+name: csp-bun-mock-module-irreversible
+description: Bun mock.module is process-global and irreversible; use DI/static-method reassignment for module stubs to avoid cross-file leaks
+metadata:
+  type: feedback
+---
+
+In this repo's Bun (1.3.10) test runner, `mock.module(path, factory)` mutates the
+process-wide module registry **irreversibly**. There is no working restore:
+- `afterAll(() => mock.module(path, () => realModule))` does NOT restore — the next
+  test file still sees the stub.
+- `mock.restore()` does NOT restore module mocks either.
+- Bun evaluates every test file's top-level code before running any tests, so a
+  top-level `mock.module` in one file poisons sibling files regardless of file order.
+
+**Why it matters:** a top-level `mock.module('../indexing/index.ts')` in
+`src/mcp/server.test.ts` leaked a stub CspIndex into `src/indexing/index.test.ts`,
+making it fail only in the full suite (passed in isolation). Diagnosing "passes
+isolated, fails in full suite" should immediately suspect a leaked `mock.module`.
+
+**How to apply:** when a test needs to stub a module export, prefer a DI seam over
+`mock.module`. For a class's static methods (e.g. `CspIndex.fromPath/fromGit`),
+reassign them on the imported class object (same reference the SUT imports) and
+restore in `afterAll` — that IS reversible (plain property mutation). Return real
+instances from the stub when tests assert `instanceof`. See `src/mcp/server.test.ts`
+for the pattern. Verify with `bun test <fileA> <fileB>` in BOTH orders.
+
+Related: [[csp-worktree-test-env-gotchas]] (full-suite vs isolated trust).

.claude/agent-memory/please-implement-executor/csp-clear-index-safety-guard.md

@@ -0,0 +1,16 @@
+---
+name: csp-clear-index-safety-guard
+description: clear index deletes ONLY ~/.csp/index via clearIndexCache; AC-015 guard asserts target ends with `index` and != home before rmSync
+metadata:
+  type: project
+---
+
+`csp clear index` / `clear all` wiring (T014, src/cli.ts + src/indexing/cache.ts).
+
+Fact: deletion of the global on-disk index cache is funneled through `clearIndexCache(options)` in `src/indexing/cache.ts`, which targets **only** `resolveIndexRoot(options)` = `<home>/index` (reuses cacheHome/baseDir rules). It returns `{ path, cleared, entries }`. Before any `rmSync(indexRoot, {recursive, force})` it asserts `basename(indexRoot) === 'index' && normalize(indexRoot) !== normalize(home)`, throwing otherwise.
+
+`clear all` runs index removal **then** `clearSavings()` as two independent calls — savings.jsonl is never collateral of an index clear, and `~/.csp/` root is never rmtree'd.
+
+**Why:** AC-015 / track safety constraint — a misconfigured baseDir must not escalate into a home-wide delete that destroys `~/.csp/savings.jsonl`. The T014 dispatch carried a STOP that fires if the computed delete path could include the `~/.csp` root or savings.jsonl.
+
+**How to apply:** When touching clear/cache-deletion code on this track (e.g. T015 README docs, or any future eviction work), keep the `index`-segment guard and the two-independent-actions split intact. Tests inject a temp `baseDir` (never the real home) and assert savings + home survive an index clear. Related: [[csp-cli-cache-di-seam]], [[csp-upstream-no-disk-cache]].

.claude/agent-memory/please-implement-executor/csp-cli-cache-di-seam.md

@@ -0,0 +1,19 @@
+---
+name: csp-cli-cache-di-seam
+description: cli wires auto-cache via injectable loadOrBuild seam; cache only on build branch so --index (loadFromDisk) is untouched; mcp must mirror the same key
+metadata:
+  type: project
+---
+
+cli.ts (`runCli`) wires the `~/.csp/index/<key>` auto-cache for `search`/`find-related` through an injectable seam `RunOptions.loadOrBuild?(source,{content,ref?})` (default `_defaultLoadOrBuild` → `loadOrBuildIndex`). The cache is applied **only on the build (else) branch** — the `--index` branch keeps loading via `readIndex`/`loadFromDisk` directly.
+
+**Why:**
+- `loadOrBuildIndex` (cache.ts) does **not** accept build-fn injection, so the only place to inject a test seam that avoids touching the real `~/.csp` home is the cli layer. Tests pass `loadOrBuild` to assert routing and stay off disk.
+- Auto-cache on build-only keeps T008's explicit-path guarantee intact: `--index <p>` must always load that exact path (mutually-exclusive if/else; build path never runs when `--index` is set).
+- `_defaultLoadOrBuild` re-narrows `ref` (omit when undefined) because `LoadOrBuildOptions.ref` is `string` under `exactOptionalPropertyTypes` — spreading `ref: undefined` is a type error.
+
+**How to apply:**
+- **DONE (T012, commit 1ac5e7a)**: mcp `IndexCache` now mirrors this exact seam. `IndexCacheOptions.loadOrBuild?` (type `LoadOrBuildSeam`, default `defaultLoadOrBuild`) replaced the in-memory miss's direct `CspIndex.fromGit/fromPath` call. The seam carries `modelPath?` too (mcp forwards its pre-warmed model), but the key-bearing fields (source/content/ref, ref omitted when absent) match cli verbatim → cli↔mcp compute the same `~/.csp/index/<key>`. Watcher stays in-memory-evict only (no disk deletion); content-hash inside `loadOrBuildIndex` owns disk reuse-vs-rebuild → single rebuild.
+- mcp's existing `server.test.ts` IndexCache/getIndex tests assert `fromPathCalls`/`fromGitCalls` (static-reassignment mocks). After routing through the seam they inject a `stubLoadOrBuild` that delegates to those static mocks — keeps counters meaningful while staying off `~/.csp`/network. Apply this pattern if you ever add a layer above an already-static-mocked builder.
+- When testing cache-backed cli/mcp paths, inject the seam; never let the default hit real `homedir()/.csp`.
+- See [[csp-loadorbuild-cache-contract]] for how the key is derived (local=source-file hash via `save(dir,{contentHash})`; git=URL+ref only).

.claude/agent-memory/please-implement-executor/csp-dense-roundtrip-no-drift.md

@@ -0,0 +1,22 @@
+---
+name: csp-dense-roundtrip-no-drift
+description: SelectableBasicBackend save→load is bit-stable (no float drift) — re-normalization of unit vectors is idempotent; NFR-002 roundtrip safe
+metadata:
+  type: project
+---
+
+`SelectableBasicBackend` (src/indexing/dense.ts) save→load roundtrip is **bit-stable, no float drift**.
+
+**Why:** The constructor L2-normalizes vectors in place. `save` writes the already-normalized
+vectors to `vectors.bin` (Float32). `load` reconstructs via the constructor, which **re-normalizes**
+— but re-normalizing a unit-length vector divides by ≈1.0 (idempotent). Measured with an isolated
+probe: `maxDiff(b1.vectors, loaded.vectors) = 0`, second roundtrip also 0, query ranking identical.
+
+**How to apply:** This settles the T006/T007 STOP condition (dense float drift breaking NFR-002
+roundtrip equivalence) — it does NOT trigger. T007 `loadFromDisk` can reuse
+`SelectableBasicBackend.load` directly; no "save unnormalized" or `skipNormalize` workaround needed.
+Related: [[csp-test-files-ahead-of-impl]] (T007 loadFromDisk tests already exist in index.test.ts).
+
+The five persisted index artifacts have mutually distinct names — no collision:
+`manifest.json` + `chunks.json` (CspIndex.save) / `bm25.json` (Bm25Index.save) /
+`vectors.bin` + `args.json` (SelectableBasicBackend.save).

.claude/agent-memory/please-implement-executor/csp-loadfromdisk-model-dim-alignment.md

@@ -0,0 +1,23 @@
+---
+name: csp-loadfromdisk-model-dim-alignment
+description: On index load, the reloaded stub model is fixed 256-dim regardless of modelId — align it to the persisted backend dim or query() throws a dim mismatch
+metadata:
+  type: project
+---
+
+`CspIndex.loadFromDisk` (src/indexing/index.ts) reloads the embedding model via
+`loadModel(manifest.modelId)`, but the **stub** `loadModel` (dense.ts) ignores modelId and always
+returns a fixed **256-dim** model (`_DEFAULT_STUB_DIM=256`).
+
+**Why it bites:** `index.test.ts`'s `buildIndex` fixture pairs `makeStubModel(4)` with hand-made
+4-dim vectors. A naive reload then makes `search` encode a 256-dim query and call
+`SelectableBasicBackend.query` against 4-dim stored vectors → `Query vector dimension mismatch` throw.
+
+**How to apply:** After reloading, if `model.dim !== semanticIndex.dim`, rebuild the query model with
+`makeStubModel(semanticIndex.dim)`. This is a stub-era-only correction: the real Model2Vec model has a
+weight-fixed dim, and `fromPath` always embeds+queries with the same `loadModel` instance so dims
+already agree (the branch never fires in the real pipeline). Future cache work (T009/T010 loadOrBuildIndex)
+that restores indexes from disk must preserve this alignment.
+
+Related: [[csp-dense-roundtrip-no-drift]] (the dense backend itself round-trips bit-stable;
+the dim issue is purely about the *separately reloaded* query model, not the stored vectors).

.claude/agent-memory/please-implement-executor/csp-loadorbuild-cache-contract.md

@@ -0,0 +1,17 @@
+---
+name: csp-loadorbuild-cache-contract
+description: loadOrBuildIndex cache-validity contract — source-file hash (not chunks hash) gates local reuse; git keyed by URL+ref only
+metadata:
+  type: project
+---
+
+`loadOrBuildIndex(source, {content?, ref?, modelPath?, baseDir?})` in `src/indexing/cache.ts` is the auto-cache orchestrator wired into CLI search/find-related (T011) and MCP (T012).
+
+**Cache-validity contract (load-bearing):**
+- Local paths: validity = `computeContentHash(collectSourceFiles(source, content))` (a **source-file** hash, computable *before* a build) must equal the cached `manifest.contentHash`. `collectSourceFiles` mirrors `createIndexFromPath`'s scan: `getExtensions(content)` + `walkFiles` (ignore rules) + `MAX_FILE_BYTES` cutoff, paths relative to root.
+- `CspIndex.save(dir, { contentHash? })` was extended so loadOrBuildIndex injects that source-file hash. **Without injection it defaults to `sha256(chunks JSON)`** (T006 behavior) — which is computable only *after* build, so it can't gate a pre-build cache check. The two hash definitions MUST agree or the cache misses forever.
+- Git URLs (T009 STOP fallback): no live re-hash possible + temp-checkout metadata is non-deterministic → keyed by URL+ref alone via `resolveCacheDir`'s `ref` option. Manifest existence = reuse; build-time hash recorded for transparency only, not validation.
+
+**Why:** plan flagged the contentHash-definition mismatch as the central risk; resolving it (optional `save` arg, backward compatible) is what makes the cache actually invalidate on source change.
+
+**How to apply:** when wiring T011/T012, pass `baseDir` only in tests (real callers omit → `~/.csp`). Tests spy on rebuilds via static reassignment of `CspIndex.fromPath` (see [[csp-bun-mock-module-irreversible]]), not `mock.module`. exactOptionalPropertyTypes forbids passing explicit `undefined` — build option objects conditionally.

.claude/agent-memory/please-implement-executor/csp-test-files-ahead-of-impl.md

@@ -0,0 +1,83 @@
+---
+name: csp-test-files-ahead-of-impl
+description: csp indexing test files were authored against APIs that don't exist yet — fixing a single in-scope source file won't make them green
+metadata:
+  type: project
+---
+
+In the csp (`@pleaseai/csp`) indexing track, some `*.test.ts` files were written
+against a target API surface that the implementation does not yet expose. Fixing one
+in-scope source file (e.g. T002 = the 3 compile errors in `src/indexing/create.ts`)
+does **not** make the matching test green, because the test depends on cross-module
+API gaps that are out of that task's `Files:` scope.
+
+Concrete example (`src/indexing/create.test.ts`, observed 2026-06-18, baseline 3 errors):
+- calls `makeStubModel('name', dim)` / `makeStubModel()` but `makeStubModel` is NOT
+  exported from `dense.ts` and its real signature is `makeStubModel(dim: number)`.
+- accesses `bm25Index.documents` but `Bm25Index` exposes no `documents` property
+  (state is a private `#state` field; only `static build`/`getScores`/`save`/`load`).
+- uses `ContentType.Docs` but the enum is `CODE | DOCS | CONFIG` (uppercase).
+- `create.ts` previously had 4 async/Chunk-type errors (lines 49/67/74) — these were
+  RESOLVED in T002 round 2 (2026-06-18, commit a328727): `for await` over `walkFiles`
+  (AsyncIterable), `await chunkSource(...)` (Promise), `language ?? null`, and unifying
+  `dense.ts`/`sparse.ts` local `Chunk` into `../types.ts` (with `export type { Chunk }`
+  re-export so `dense.test.ts`/`sparse.test.ts` keep importing `Chunk` from those modules).
+  Do NOT redo the Chunk unification — it is done.
+
+T003 (2026-06-18, commit 7aa721a) wired `src/indexing/index.ts` (fromPath + ctor options
+object + stats + DEFAULT_CONTENT export + save/loadFromDisk throwing stubs + loadModel tuple).
+That fixed the `DEFAULT_CONTENT` import blocker in `index.test.ts`, but the test file STILL
+won't load — the blockers that remain are ALL outside `src/indexing/index.ts` (T003's only
+Files-scope), so they were NOT fixed and must be handled by whoever owns dense.ts/sparse.ts:
+- `makeStubModel` not exported from `dense.ts`; test calls `makeStubModel('name', dim)` but
+  real sig is private `makeStubModel(dim: number)` → needs dense.ts export + 2-arg signature.
+- test uses `new SelectableBasicBackend(vectors, dim)` but current ctor is `(vectors, BasicArgs)`
+  → needs dense.ts ctor change.
+- test uses `new Bm25Index([['x']])` (public ctor) but ctor is private (`static build` only)
+  → needs sparse.ts change.
+- test line 197 uses `ContentType.Code` (should be `CODE`) → needs the test file itself fixed.
+The `findRelated`/`search` filter-behavior asserts in `index.test.ts` are T004 (behavioral),
+not T003. So `index.test.ts` goes green only after dense.ts + sparse.ts + the test file are
+fixed AND T004 ranking is wired — not within any single one of those scopes.
+
+**Why:** the plan front-loaded test files for the eventual API; impl lands incrementally
+across tasks/phases, so a test can be red at baseline through no fault of the current task.
+
+**How to apply:** when a task's plan scenario says "make X.test.ts green" but the test
+won't even load, check whether the blockers are in your `Files:` scope. If they require
+editing other modules or the test file itself, do NOT expand scope or weaken the test —
+fix your scoped target, verify "no new errors / suite unchanged at baseline", and record
+the cross-scope blocker in the plan's `## Surprises & Discoveries` for the owning task
+(usually T003, which depends on T002). See [[csp-typecheck-baseline-red]] for the
+typecheck gate ("no new errors", not "green tsc").
+
+**RESOLVED in T004 (2026-06-18, commit ba30228):** `index.test.ts` went green for its
+search/findRelated/stats cases. The planner correctly put the cross-file fixes in T004's
+`Files:` (index.ts, index.test.ts, dense.ts). Key correction to the T003 prediction above:
+- Only `makeStubModel` export (dense.ts) was actually needed. The scaffold test's GUESSES
+  `new Bm25Index([['x']])` and `new SelectableBasicBackend(vecs, 4)` were simply WRONG —
+  the real APIs `Bm25Index.build(docs)` and `new SelectableBasicBackend(vecs)` (dim derived
+  from vectors) already existed. So **sparse.ts needed NO change**; the fix was correcting
+  the TEST setup to the real API, not changing the impl. That is why T004's Files had
+  index.test.ts + dense.ts but not sparse.ts.
+- Wiring pattern that avoided the STOP("search.ts API structurally incompatible"): put the
+  blank-query / `topK<=0` / empty-index / empty-selector guards in the CspIndex.search
+  LAYER, then delegate to `search.ts search(query, model, semanticIndex, bm25Index, chunks,
+  topK, {selector?})`. search.ts already returns `[]` for an empty selector (effectiveK→0),
+  so passing the empty `Uint32Array` through (no unfiltered fallback) satisfies the
+  "filters match nothing → []" regression test. `findRelated` re-embeds the seed content,
+  calls `semanticIndex.query(emb, topK+1)`, drops the seed chunk. Both kept SYNC (mcp/
+  server.ts:370 and cli.ts call without await).
+
+**FULLY RESOLVED in T0A (2026-06-18):** the remaining scaffold-test debt is cleared.
+- `types.test.ts` missing exports → added camelCase round-trip helpers to `types.ts`
+  (`chunkToDict`/`chunkFromDict`/`chunkLocation`/`searchResultToDict`/`ChunkDictInput`),
+  kept SEPARATE from search.ts's snake_case `SearchResult.toDict` (two layers, no reuse).
+- `bm25Index.documents` → added a read-only `documents` getter on `Bm25Index` (sparse.ts)
+  returning per-doc token counts (`#state.docLengths`), so `.length === numDocs`.
+- enum casing in `types.test.ts`/`index.test.ts` → aligned tests to `CODE/DOCS/CONFIG`,
+  `SEARCH/FIND_RELATED` (source enum is the contract per CLAUDE.md).
+- `cli.test.ts` stub lacked `toDict` → added a snake_case `toDict`.
+- the "passes-isolated-fails-full-suite" indexing failures were a leaked `mock.module`,
+  fixed via DI in server.test.ts (see [[csp-bun-mock-module-irreversible]]).
+Full suite now 351 pass / 3 fail / 0 error (3 fails = T006/T007 throwing stubs, expected).

.claude/agent-memory/please-implement-executor/csp-typecheck-baseline-red.md

@@ -0,0 +1,16 @@
+---
+name: csp-typecheck-baseline-red
+description: csp project's `bun run typecheck` is red at baseline — TS5097 fires on every relative .ts import; gate on "no new type errors", not "green tsc"
+metadata:
+  type: project
+---
+
+`bun run typecheck` (`tsc --noEmit`) in `@pleaseai/csp` is **red at baseline** and expected to be.
+
+**Why:** the repo mandates `.ts` import extensions (CLAUDE.md: `verbatimModuleSyntax` + `.ts` imports, `moduleResolution: bundler`) but tsconfig does not set `allowImportingTsExtensions`. So `TS5097` ("An import path can only end with a '.ts' extension when 'allowImportingTsExtensions' is enabled") fires on **every** relative import — ~47 occurrences project-wide. There are also pre-existing errors like local-but-not-exported `Chunk`/`SearchResult` in some test files.
+
+**How to apply:** when an implement task touches typecheck, the gate is **"do not add new *type* errors to the touched files, reduce if possible"** — NOT "whole-suite tsc green". Adding any new `.ts` import unavoidably adds one more `TS5097`; that is the established convention (e.g. `utils.ts` does it too), not a regression. Distinguish config-class errors (TS5097) from genuine type errors (TS2xxx mismatches). To check a file's delta, `git stash` and compare `tsc` output filtered to that file before/after.
+
+Baseline test suite (as of 2026-06-18): **316 pass / 5 fail / 3 errors** across 20 files. The 5 fails include "public barrel > exposes ContentType as a runtime enum" and "csp search (stub-mocked) > formats non-empty results as JSON". Verify your change leaves the failing-test *set* unchanged (diff the `^(fail)` lines before/after) rather than chasing whole-suite green.
+
+`bun run lint` is currently broken project-wide: eslint can't load the TS flat config (`jiti` library missing). Pre-existing tooling gap, unrelated to code changes.