cortex: phase2/3 health supervision and detached lifecycle hardening

docs/content/docs/(core)/cortex.mdx

@@ -1,6 +1,6 @@
 ---
 title: Cortex
-description: System-level observer that maintains memory bulletins and monitors agent health.
+description: System-level observer that maintains memory bulletins, runs maintenance, and monitors agent health.
 ---
 
 # The Cortex
@@ -84,26 +84,37 @@ These serve different purposes:
 
 The bulletin doesn't replace recall — it reduces how often recall is needed. A channel that already knows the user's name, their current project, and recent decisions from the bulletin doesn't need to spawn a branch for basic context.
 
-## Future Responsibilities
+## Current Responsibilities
 
-The bulletin is the cortex's first and most impactful responsibility. The following capabilities are designed but not yet implemented:
+The bulletin is still the cortex's most visible responsibility, but it is no longer the only one.
 
 ### System Health Monitoring
 
-The cortex monitors running processes and keeps the system clean:
+The cortex already supervises long-running process state:
+
+- **Worker supervision** — detect hanging workers, apply timeout policies, and clean up detached worker state
+- **Branch supervision** — detect stale branches and enforce timeout/cancellation policy
+- **Circuit breakers** — recurring bulletin refresh and maintenance failures back off automatically instead of retrying forever
+
+### Memory Maintenance
+
+The cortex also runs periodic graph hygiene:
+
+- **Decay** — reduce importance for stale non-identity memories
+- **Prune** — delete memories that have fallen below the configured importance floor and age threshold
+- **Merge** — combine near-duplicate memories and rewire graph associations atomically
+
+## Future Responsibilities
 
-- **Worker supervision** — detect hanging workers, kill error loops, clean up stale state
-- **Branch supervision** — kill stale branches, track latency trends
-- **Channel health** — flag channels approaching context limits faster than compactors can manage
-- **Circuit breakers** — after 3 consecutive failures of the same type, disable the failing component
+The remaining cortex roadmap is about richer cross-system inference, not basic supervision:
 
 ### Memory Coherence
 
-The cortex sees memory activity across all channels and maintains the graph:
+The cortex sees memory activity across all channels and can grow into deeper graph stewardship:
 
-- **Consolidation** — merge overlapping memories, create cross-channel associations
-- **Maintenance** — decay old memories, prune low-importance orphans, recompute centrality
-- **Observations** — generate observation-type memories from cross-channel patterns
+- **Consolidation** — merge overlapping memories across channels and create cross-channel associations
+- **Observations** — generate observation-type memories from recurring patterns
+- **Higher-order scoring** — extend maintenance beyond the current decay/prune/merge pass with richer graph analysis
 
 ### The Signal Bus
 
@@ -138,7 +149,7 @@ The cortex is the only singleton in the system (per agent). There's one cortex p
 
 **Compactors** are per-channel, programmatic monitors. They watch one channel's context size and trigger compaction workers. They're not LLM processes.
 
-**The cortex** is an LLM-assisted process that sees across all channels. It doesn't manage context size (that's the compactor's job). It manages the memory bulletin, and will eventually handle memory coherence and system health.
+**The cortex** is an LLM-assisted process that sees across all channels. It doesn't manage context size (that's the compactor's job). It manages the memory bulletin, supervises background health state, and runs periodic memory maintenance. Deeper memory coherence work still grows from here.
 
 ## Interactive Cortex Chat
 
@@ -155,7 +166,7 @@ This makes cortex chat a practical control-room interface for troubleshooting, v
 
 ```toml
 [defaults.cortex]
-# Base tick interval for health monitoring (future).
+# Base tick interval for health monitoring and maintenance scheduling.
 tick_interval_secs = 30
 
 # How often to regenerate the memory bulletin.
@@ -172,6 +183,21 @@ branch_timeout_secs = 60
 
 # Consecutive failures before circuit breaker trips.
 circuit_breaker_threshold = 3
+
+# Interval between maintenance passes.
+maintenance_interval_secs = 3600
+
+# Per-day decay applied during maintenance.
+maintenance_decay_rate = 0.05
+
+# Importance floor for pruning.
+maintenance_prune_threshold = 0.1
+
+# Minimum age before pruning can delete a memory.
+maintenance_min_age_days = 30
+
+# Similarity threshold for duplicate merges.
+maintenance_merge_similarity_threshold = 0.95
 ```
 
 ## Warmup API

docs/content/docs/(core)/memory.mdx

@@ -150,7 +150,7 @@ Importance is influenced by:
 - **Recency** -- recent memories score higher; old memories decay
 - **Graph centrality** -- memories with many strong connections to other memories are more important
 
-A background maintenance process runs periodically to decay old memories, prune memories that have fallen below a threshold, merge near-duplicates, and recompute graph centrality scores.
+A background maintenance process runs periodically to decay old memories, prune memories that have fallen below a threshold, and merge near-duplicates.
 
 Identity and permanent-tagged memories are exempt from decay and pruning. They always survive.
 
@@ -202,6 +202,5 @@ A periodic background process handles graph hygiene:
 - **Decay** -- reduce importance of old, unaccessed memories
 - **Prune** -- delete memories below an importance floor (identity/permanent exempt)
 - **Merge** -- combine near-duplicate memories (>0.95 similarity)
-- **Reindex** -- recompute graph centrality scores
 
-This is a scheduled job managed by the cortex. It runs in workers, doesn't block anything, and keeps the graph healthy over time.
+This is a scheduled job managed by the cortex. It runs as an internal background task in the cortex loop, doesn't block channels, and keeps the graph healthy over time.

docs/design-docs/cortex-implementation.md

@@ -1,6 +1,6 @@
 # Cortex Implementation Plan
 
-The cortex is designed to be the system's self-awareness — supervising processes, maintaining memory coherence, and generating the memory bulletin. Phase 1 plumbing and Phase 2 health supervision are now live; maintenance and consolidation phases remain.
+The cortex is designed to be the system's self-awareness — supervising processes, maintaining memory coherence, and generating the memory bulletin. Phase 1 plumbing, Phase 2 health supervision, and Phase 3 maintenance are now live; consolidation remains.
 
 This doc covers the path from "bulletin generator" to "full system supervisor."
 
@@ -16,8 +16,8 @@ This doc covers the path from "bulletin generator" to "full system supervisor."
 - `Signal` enum — aligned to current `ProcessEvent` surface (worker/branch/tool/memory/compaction/task/link events).
 - `CortexHook` — all methods return `Continue` with trace logging.
 
-**Implemented but never called:**
-- `memory/maintenance.rs` — `apply_decay()` and `prune_memories()` work. `merge_similar_memories()` is a stub returning `Ok(0)`.
+**Implemented and wired into the cortex loop:**
+- `memory/maintenance.rs` — `apply_decay()`, `prune_memories()`, and `merge_similar_memories()` are implemented and wired into the cortex loop.
 
 **Wired through config:**
 - `tick_interval_secs` and `bulletin_interval_secs` are read by the running cortex loop and hot-reload during runtime.
@@ -124,27 +124,31 @@ Per tick, cortex maintains runtime maps for workers, branches, branch latency, a
 - Success resets the counter/tripped flag for that key.
 - Breaker state is in-memory only in Phase 2 and resets on process restart.
 
-## Phase 3: Memory Maintenance
+## Phase 3: Memory Maintenance (Implemented)
 
-Wire up the maintenance code that already exists in `memory/maintenance.rs`.
+Memory maintenance is now wired into the cortex loop.
 
-### Schedule maintenance in the tick loop
+### Scheduling and control
 
-- Add `maintenance_interval_secs` to `CortexConfig` (default: 3600)
-- On the maintenance tick, call `run_maintenance()` with the current `MaintenanceConfig`
-- Log the `MaintenanceReport` (decayed, pruned, merged counts)
-- Respect hot-reload — `MaintenanceConfig` values should come from `CortexConfig`
+- `maintenance_interval_secs` is part of `CortexConfig` and hot-reloads at runtime
+- The cortex tick loop schedules `run_maintenance_with_cancel()` with the current `MaintenanceConfig`
+- Maintenance reports log decayed, pruned, and merged counts
+- Invalid maintenance config values are rejected at load/update time:
+  - `maintenance_interval_secs >= 1`
+  - `maintenance_decay_rate`, `maintenance_prune_threshold`, `maintenance_merge_similarity_threshold` in `[0.0, 1.0]`
+  - `maintenance_min_age_days >= 0`
+- Maintenance runs with timeout, graceful cancellation, forced abort fallback, and a recurring-failure circuit breaker
 
-### Finish `merge_similar_memories()`
+### Merge behavior
 
-- Query LanceDB for high-similarity memory pairs above `merge_similarity_threshold` (default 0.95)
-- Keep the higher-importance memory as the survivor
-- Merge content from the lower-importance memory into the survivor
-- Create `Updates` association from survivor to merged memory
-- Transfer associations from the merged memory to the survivor
-- Soft-delete the merged memory
+- LanceDB similarity search drives near-duplicate detection above `merge_similarity_threshold` (default 0.95)
+- The higher-importance memory stays the survivor
+- Survivor content is updated, associations are rewired atomically in SQLite, and an `Updates` edge is preserved
+- The merged memory is soft-forgotten and its embedding is removed
+- The survivor embedding is recomputed after merge
+- Per-pass work is bounded (candidate and merge caps) to avoid unbounded scans on large corpora
 
-**End state:** Memories decay over time, low-importance orphans get pruned, near-duplicates get merged. All on autopilot.
+**End state:** Memories decay over time, low-importance orphans get pruned, and near-duplicates get merged on autopilot under cortex supervision.
 
 ## Phase 4: Memory Consolidation (LLM-Assisted)