feat: shared memory & drive store - #8
Merged
Merged
Conversation
Adds `memoryDir` and `driveDir` to `ServerDerivedPaths`, created at startup alongside the attachments directory. Both stores are user-level and shared across every project, which is the point -- a note about how the user works is not a per-repo fact. But they are derived from `stateDir` rather than the home directory so the existing dev/userdata split still applies. Consolidation clears the capture buffer, so a home-directory default would let a dev server or a test run destroy real captured notes. The regression test asserts dev and production paths differ. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two optional roots on `ServerSettings`, following the `addProjectBaseDirectory` pattern: a trimmed string that decodes to empty, where empty means "use the stateDir-derived default". Keeping resolution on the server rather than baking an absolute path into persisted state means the default can move later without migrating anyone's settings. Schema only -- resolution lives in apps/server, per this package's schema-only rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root resolution (setting wins, stateDir default otherwise), a containment guard, and stable project segments. The guard is a port of `resolveAttachmentRelativePath` and keeps its posture: a path that escapes its root returns null rather than being clamped back inside, because a caller supplying a traversal is either a bug or an attack and rewriting it hides both. Both roots are user-configurable, which makes that guard more load-bearing here than it is for attachments. Project segments append a short digest of the absolute repository path to the sanitized basename. Without it, two checkouts named `api` under different parents would share one bucket and silently merge their notes. `resolveProjectSegmentForThread` is deliberately not here: it needs a SQL query against projection_threads/projection_projects, and belongs with the code that has the client in scope rather than in a path module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four tables: drive_artifacts, memory_notes, memory_note_sources, and memory_note_links. Deliberately not projection_* tables. Projections derive from the orchestration event log and rebuild by replay; these hold primary, user-owned state that nothing replays. Markdown files stay authoritative for notes -- these rows are an index so recall and backlink queries do not scan the corpus, and the consolidation reindex rebuilds them from frontmatter. Two index choices worth noting. The live-path index is unique but partial (WHERE archived_at IS NULL): re-running the same task is the normal case, so archiving a row must release its path for reuse rather than burning the filename forever. And memory_note_links is indexed on to_note_id because backlinks are the point of a Zettelkasten -- "which notes link here?" has to be an indexed lookup, not a scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two passes: known credential shapes (GitHub tokens and PATs, AWS key ids, Slack tokens, JWTs, PEM private keys, credential-named assignments), then a Shannon-entropy fallback for generated tokens no pattern names. Redaction runs on write rather than read because the memory store is shared across every project on the machine. A token captured while working on one repository would otherwise sit in a file that every other project's sessions read at session start; per-project stores would have contained that, and this design deliberately does not. Over-redaction is the failure mode that actually matters -- a redactor that mangles commit hashes gets switched off, and then it protects nothing -- so the entropy pass carries an allowlist and the tests assert that SHAs, UUIDs, paths, URLs, emails, semver, and dotted identifiers all survive untouched. Two regression tests come from a real bug found while probing: the original path allowlist matched [\w./-]+, which is also the shape of base64, so every generated token passed through. It now keys on the path separator and excludes "+"/"=" so base64 with slashes cannot smuggle past the entropy check either. Redaction is a mitigation, not a guarantee: a secret written in prose still gets through, and the module says so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`daily.md` is the short-term buffer every session appends observations to. Each entry carries a provenance header written by the server -- capture time, project segment, thread -- because consolidation needs to know which project an observation came from to set a promoted note's scope, and a model asked to supply that would eventually omit or invent it. An unresolvable project records "unattributed" rather than failing the call: losing an observation is worse than losing its attribution. Appends are a single O_APPEND write, not read-modify-write. Verified by temporarily reverting to read-modify-write: 18 of 20 concurrent captures were lost. The concurrency test is the regression guard for that. Bodies are redacted before reaching disk, and the raw body is never logged or returned. Adds `rotateDaily` alongside `clearDaily`. Truncating in place leaves a window where an append landing between consolidation's read and its clear is discarded unpromoted; renaming the buffer aside first means such an append starts a fresh file and is picked up next cycle. The rotated file is left on disk so a failed run can retry it. Consolidation should use rotation -- `isReservedMemoryFile` keeps rotated buffers out of the note reindex. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Permanent Zettelkasten notes as markdown files plus a SQL index. The file is the source of truth: notes stay hand-editable and greppable, so the index has to be reconstructible from them at any time. That is what `reindexAll` is for, and why writes go file-first -- a file with no row is repaired by the next reindex, while a row with no file is a dangling reference nothing repairs. A malformed file is reported and skipped rather than aborting the pass, so one bad note cannot block indexing the corpus. Two things carry the long-term value and should survive refactoring. Links record a `rel` and a `context` sentence, because a link that explains itself is still useful a year later and a bare backlink is not. And `backlinksFor` is an indexed lookup on to_note_id -- backlinks are the point of a Zettelkasten, so answering "which notes link here?" must not scan the corpus. Reindex skips the daily buffer, rotated buffers, and the curated index via `isReservedMemoryFile`. Indexing consolidation's own working files as notes would be the cycle consuming its own output. Note ordering puts the current project ahead of global notes, which is what makes one shared store safe to read from inside a specific project. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generated files that should not be committed anywhere, made addressable: a stable id, a content hash, and provenance back to the thread, turn, and checkpoint that produced them. Observations say what happened; artifacts say what was actually done, which is what makes them worth citing. Files are namespaced under the project segment so they stay attributable from the path alone, and every write goes through the containment guard. A rejected path fails before touching disk or the database -- a partial write with no row would leave an unreferenced file behind, tested explicitly. `archiveArtifact` is bookkeeping, not deletion: it releases the path via the partial live-path index so a re-run can reuse a natural filename while the previous output stays on disk. `notesCiting` is the reverse of a note's sources. Provenance has to run both ways or "why does the agent believe this?" has no clickable answer. `artifactsCreatedSince` gives consolidation an explicitly bounded input set rather than "everything in the directory". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Widens `McpCapability` to "preview" | "memory" and grants both when a session credential is issued. `requireMcpCapability` previously failed with `PreviewAutomationUnavailableError`, whose `capability` field is the literal "preview". A memory tool denial surfacing as a preview error would be actively misleading in logs, so contracts gains `McpCapabilityUnavailableError` for capabilities beyond preview. Preview keeps its original error, so clients decoding it are unaffected. The implementation covers every capability, so its inferred failure type is the union of both errors -- which would force preview handlers to declare a memory error they can never receive. Overloads narrow the failure per capability instead. `ThreadId` and `EnvironmentId` are imported from baseSchemas rather than orchestration/environment: sourcing them elsewhere left them undefined at module-init time and the schema failed to build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three tools -- memory_append_daily, memory_read_daily, memory_search -- registered beside the preview toolkit on the server's own MCP endpoint. Every provider that speaks MCP gets them without a per-provider adapter, which is what makes an observation captured in a Codex session consolidatable by a Claude session and recallable in a Cursor one. `memory_append_daily` takes only a body. Project, thread, and capture time come from the MCP invocation scope, which the server issues when it mints the credential, so a model cannot attribute an observation to a project it is not working in -- doing so would quietly poison recall there. The absence of those parameters is the guarantee, so a test asserts it. Adds `resolveProjectForThread`, which was carved out of T3 because it needs a SQL query and a path module should not own one. It keys on projection_projects.workspace_root rather than the thread's worktree_path: a worktree is per-branch, so keying on it would split one repository's notes across every branch ever worked on. Attribution is best-effort. A thread with no resolvable project records "unattributed" rather than failing -- losing an observation is worse than losing its attribution. Infrastructure faults die rather than surfacing as tool errors, keeping the contract to "recorded" or "capability denied". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Promotes captured observations into permanent notes. The ordering is the
design, not incidental: lock, reindex, rotate the buffer aside, promote,
write a summary, and only then release the rotated buffer and advance the
marker. A failure anywhere leaves the rotated buffer on disk and the
marker unmoved, so the next run reconsiders the same entries rather than
losing them.
The lock is an exclusive file create ("wx"), so creation is the atomic
test, and it is released through Effect.ensuring -- a crash mid-run cannot
wedge it permanently. A second concurrent run reports "already-running"
rather than queueing or failing, and a test asserts the observation is
promoted exactly once.
Summaries go to a receipts/ subdirectory that the note reindex never
reads. A cycle that consumes its own output eventually spends its whole
budget reprocessing its exhaust, so a test runs two cycles and asserts the
first summary contributes no notes to the second.
Note ids are the capture timestamp plus a short digest of the body.
Position-based ids -- the first thing I wrote -- collide whenever two
observations share a capture second across different runs, and the
collision is silent: the second note overwrites the first. Hashing the
body also makes re-promoting an identical observation idempotent. The
regression test covers both.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assembles the budgeted grounding digest: 2,000 characters total with per-section caps of identity 600, daily 500, brief 600, themes 300, notes ranked current-project-first then by recency, and empty output when nothing meaningful changed. The constraints are the design, so the tests are adversarial: every section oversized at once must still fit the total, each section must cap independently, and a quiet period must produce an empty string rather than a header with nothing under it. A digest that always fires trains the model to ignore it. Named ContinuityBrief, never "receipt" -- RuntimeReceiptBus already owns that word for async runtime milestones. INJECTION IS DEFERRED, deliberately. The task's first step was to confirm a single prompt-composition point every provider passes through, and there isn't one: customInstructions is Copilot-only and CodexDeveloperInstructions is Codex-only. A brief injected for one provider is worse than none, because behaviour would silently differ by provider with nothing to indicate why. Wiring needs a per-provider decision, so this module builds the text and nothing calls it yet. The module docs say so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A toolkit can be built correctly and still never be mounted, and no unit test over the toolkit object catches that. This lists tools over HTTP through the same registration layer McpHttpServer.layer composes, and asserts all three memory tools appear alongside preview_status. Verified to fail when MemoryToolkitRegistrationLive is removed from the merge, so it is a real regression guard rather than a restatement of the registration code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The brief is prepended to the first user message rather than composed into a system prompt. No single prompt-composition point reaches every provider -- customInstructions is Copilot-only, CodexDeveloperInstructions is Codex-only, Claude sends a preset systemPrompt, and Cursor, Grok and OpenCode surface no instruction hook at all. Wiring only the providers that have a hook would make the same note change behaviour in some sessions and not others with nothing on screen to explain why. Every provider accepts messages, so the message path is the one seam that behaves identically for all six. BriefInjection.ts carries a TODO to revisit this as a provider-agnostic session-preamble seam; that is provider-layer work, not memory work. Recall runs on the opening turn only, is wrapped so any failure degrades to no brief rather than blocking the turn, and is delimited so the model reads it as context rather than as the user's request. Title and branch generation continue to see the raw message text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t holds `receipts/` collided with `RuntimeReceiptBus`, which already owns that word for async runtime milestones. Two unrelated meanings for one term is the confusion the plan called out; `summaries/` says what the directory actually contains. Mechanical: the reindex excludes the directory by filtering top-level `.md` files, not by name, so nothing depended on the old string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T12 and T13. Five endpoints: memory.consolidate plus listNotes, getNote, listArtifacts and getArtifact. Consolidation returns a tagged union so "already running" cannot be mistaken for a failure -- it is the normal outcome of pressing the button twice, and a client that models it as an error shows a red toast for something that went fine. The tag also makes it impossible to read the counts without handling the other cases first. List endpoints carry a bounded limit in the schema rather than relying on a server default, since adding a cap once clients depend on getting everything is a breaking change. getNote returns backlinks alongside the note so opening one is a single round trip instead of a waterfall. Wire shapes are camelCase; the snake_case row shape is a storage detail and leaking it would make every client field name a hostage to a future migration. Absolute server paths (the consolidation summaryPath) are dropped on the way out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T15. Both rows follow the "Add project starts in" pattern and touch all four registration points: the row, changedSettingLabels, that memo's dependency array, and restoreDefaults. Two things this needed beyond the row markup: ServerSettingsPatch is maintained by hand rather than derived from ServerSettings, so both fields had to be added there too. Without that the rows would have committed successfully and silently reverted on reload -- covered now by a contracts test, since the next hand-added setting has the same trap waiting. Empty means "use the derived default", so the placeholder shows the resolved path from a new optional memoryPaths field on the server config snapshot. A blank input with no explanation reads as broken and invites typing a path nobody needs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T16. The outcome-to-toast mapping lives in CommandPalette.logic.ts per the repo's convention, so it is unit-tested without rendering. "Already running" and "nothing to do" render as info, never as errors. Pressing the button twice is the normal way to reach the first, and an error toast for a successful no-op teaches people to distrust the feature. A completed run that promoted nothing is also info rather than success, since claiming success for zero work is just noise. Consolidation goes through useConsolidateMemory so the palette entry and the Memory workspace button (T18) share one in-flight state -- two controls with independent running flags would let one look idle during a run. The command is hidden rather than disabled while in flight: a palette entry that does nothing when selected is worse than one that is not offered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
T17, T18 and T19, landed together so they share one verification pass. The rail renders outside AppSidebarLayout, so Memory brings its own sidebar and Threads keeps the thread sidebar untouched. Switching is routing and nothing else -- no thread, panel, or selection store is touched, which is what lets the active thread survive a round trip. Filtering, sorting and selection live in MemoryView.logic.ts per the repo's convention and are unit-tested without rendering. T19's brief activity records every ContinuityBrief injection with the exact injected text, so nothing reaches a prompt from the memory store without a visible record. A failed activity is logged rather than swallowed: it means an injection went unrecorded. T19's other half needed a prerequisite that was missing from the task list: nothing called writeArtifact in production, so no turn could ever write an artifact, leaving the Drive tab permanently empty. Added drive_write_artifact to the memory toolkit, with the same anti-spoofing shape as capture -- the project bucket comes from the invocation scope, never from a parameter, so a model cannot write into another project's drive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both found by the integrated verification pass, neither by a unit test. The sidebar primitive positions its panel `fixed left-0`, which ignores the rail's flex row and drew straight over it. The rail was present and focusable in the accessibility tree the whole time, so only looking at the page caught it. Offsetting the panel is scoped to the app shell rather than changing the shared primitive for every other consumer. Consolidating refreshed the note and artifact lists but not the open detail pane, so a run left the selected note showing pre-run content -- and the note whose content just changed is exactly the one being read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rail renders flush to the top-left, which is where the desktop shell draws close/minimize/zoom on macOS -- the first icon sat under them and could not be clicked. The rail now offsets its top by the title bar strip when those controls overlay the client area, and not in fullscreen or in the browser, where there is nothing to clear. Investigating that surfaced the same collision from a second source: the sidebar's toggle is anchored to the window's left edge, so it was drawn on top of the rail at x=12 regardless of platform. Window-edge insets now add the rail width, and the one inset consumed as a margin *inside* the sidebar takes it back off, so absolute positions are unchanged where no rail is rendered. That override is declared on the shell element rather than :root because a custom property substitutes var() where it is declared -- on :root it would always read the 0px default and silently do nothing. The macOS detection AppSidebarLayout already had is now a shared hook, so the rail and the sidebar cannot disagree about whether to inset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Threads rail button pointed at a static "/", which is the index route -- the new-thread starter. A round trip through Memory therefore dropped whatever thread was open and landed on an empty composer. The rail now records the last Threads-workspace route and returns there. Memory routes are never recorded, or the button would point back into Memory; neither are /pair and /connect, which render outside the app shell and would drop the user into an auth screen they already cleared. While already in Threads the button stays "/", matching what clicking the active workspace does elsewhere. The remembered path is module-scoped rather than React state because the rail unmounts on those excluded routes, and a value that reset on remount would lose exactly the thread this exists to preserve. Still routing-only: no thread, panel, or selection store is read or written. Verified against a seeded project that in-progress composer text survives the round trip, not just the URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The buffer was invisible in the app: Notes reads promoted notes, Drive reads artifacts, and nothing surfaced where observations actually land first. That is the worst thing to have hidden -- it is where over-eager capture and redaction problems show up, and without it you only find out at consolidation time, once the entries are already notes. Tabs are now Daily, Notes, Drive: left to right is the lifecycle of an observation. The workspace still opens on Notes, because Daily is empty immediately after every consolidation and opening there would routinely greet you with nothing. memory.readDaily returns the raw file alongside server-parsed entries. The raw text keeps redaction markers exactly as written; the parsed entries save the client re-implementing the provenance header format, which is a storage detail and would drift as a second parser. Unattributed captures are counted separately rather than folded into the total: one means thread resolution failed, so consolidation cannot scope the note it produces. Consolidating refreshes the buffer first, since a run clears it and a stale view would still list promoted entries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Memory workspace renders without AppSidebarLayout, and that layout is what normally reserves the title bar strip -- so its tab bar started at y=0 and the macOS traffic lights covered the left third of the Daily tab, making it unclickable. The clearance is applied at the shell, next to the branch that decides whether a workspace gets AppSidebarLayout, rather than inside MemoryView. The workspace should not have to know what the window chrome is doing, and any future workspace rendered without the sidebar layout inherits the fix instead of rediscovering the bug. Measured with the macOS insets applied: rail icons at y=52 and y=92, tabs at y=60, all clear of the y<=52 control strip. Browser mode is unchanged, so no dead space appears where there are no window controls. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
github-actions
Bot
added
vouch:trusted
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the memory & drive plan: a user-level Zettelkasten and an artifact
store that live outside any repo, so notes compound across projects and
generated files that shouldn't be committed have somewhere to go.
All twenty tasks (T0a–T19) are done.
Design
Both roots derive from
stateDir, not the home directory.deriveServerPathsalready splits
stateDirintobaseDir/userdataorbaseDir/dev, and thatsplit is what gives dev servers and the test suite isolated state. A hardcoded
~/.t3codedefault would bypass it — and since consolidation clearsdaily.md, a test run would have destroyed real captured observations.Scoping moves from storage to retrieval: one store, but every note and artifact
records where it came from, and recall filters by the current project. That is
what makes a shared store safe without giving up cross-project learning.
Surface
drive_artifacts,memory_notes,memory_note_sources,memory_note_links) plus indexes. Deliberately notprojection_*: projections replay from the event log, this is primary userstate that nothing replays.
memory_append_daily,memory_read_daily,memory_search,drive_write_artifact, behind a newmemorycapability.memory.consolidate,readDaily,listNotes,getNote,listArtifacts,getArtifact.Notes and Drive tabs, two settings rows, and a consolidate command in the
palette.
Invariants worth protecting
tool takes a
projectSegmentparameter, so a model cannot attribute anobservation to another project or write into its drive.
daily.mduses an O_APPEND write,not read-modify-write.
clearing, and is discarded only after every note is written.
Two decisions to review
ContinuityBrief injection. There is no single prompt-composition point every
provider passes through —
customInstructionsis Copilot-only,CodexDeveloperInstructionsis Codex-only, Claude sends a preset system prompt,and Cursor, Grok and OpenCode surface no hook at all. Injecting only where a
hook exists would make the same note change behaviour in some sessions and not
others with nothing on screen to explain it. So the brief is prepended to the
first user message, which behaves identically for all six.
BriefInjection.tscarries a TODO to revisit this as a provider-agnosticsession-preamble seam; that is provider-layer work.
drive_write_artifactwas not in the task list. Nothing calledwriteArtifactin production, so no turn could ever write an artifact and theDrive tab would have been permanently empty. Added with the same anti-spoofing
shape as capture.
Verification
489 focused tests. Typecheck and lint clean on every changed package; the one
remaining
apps/servererror is pre-existing inHostPowerMonitor.ts,confirmed on a clean tree.
Verified in the browser against isolated state: consolidation promotes entries
into notes with correctly differing project segments, clears the buffer, and is
visible live in the Daily tab; a hand-edited note file is picked up by the
self-healing reindex; provenance round-trips note → artifact → note; settings
persist, reset, and show resolved paths; the thread survives a Memory round trip
including in-progress composer text.
Three defects were found by that pass rather than by unit tests, all fixed here:
the sidebar's
fixed left-0panel drew over the rail, consolidation left theopen note stale, and the Threads button returned to the new-thread starter
instead of the open thread.
Not done
title. No synthesis, no tag derivation, no
Behavioral effect:line.measuring, not in a real Electron build.
statusis in the schema so demotion needs no migration.Backing it out
Everything is additive. To disable without reverting, leave the
memorycapability ungranted — the tools become unreachable and nothing writes. To
remove entirely, revert and drop the four tables; no existing table is altered
and no projection rebuild is affected.
🤖 Generated with Claude Code