Activate the curiosity subsystem, gate the Rust workspace in CI, and fix what that surfaced

[danielhertz1999-bit]

Summary

Two threads, plus the defects the second one exposed.

1. The curiosity subsystem never ran. fire_curiosity() had no caller in src/: recall
wrote deferred_curiosity_input events that nothing consumed, so no curiosity_question
events and no curiosity_bridge edges were ever produced. Because _knob_tune keys off the
curiosity_bridge edge ratio, that ratio sat pinned at 0 and the monotropism soft-knob was
rewritten on every sleep cycle.

2. The Rust workspace had zero CI coverage. ~482 #[test] functions across 8 crates —
including the storage-critical lillibrain btree, crash-recovery and proptest suites — and
cargo test was invoked nowhere: not in CI, not in scripts, not in a Makefile. ci.yml
installed clippy and rustfmt without ever running them, and deny.toml encoded a licence
allow-list and a GPL/AGPL/LGPL ban that nothing executed.

Curiosity: consume, fire, resolve

• A fail-soft drain folded into the existing DMN_REFLECTION sleep step replays deferred
  inputs through fire_curiosity, producing the question events and bridge edges the rest of
  the system already reads. Watermark-cursored, so no input fires twice.
• A later unambiguous recall on a cue that previously fired a question now writes
  curiosity_resolved — nothing in the engine ever did, so questions could only age out.
  Session-scoped and idempotent, matched on a normalized cue_key.
• HIPPO_CLEANUP prunes deferred inputs at or before the watermark and superseded cursors.

Correctness fixes activation surfaced

• compute_entropy normalized to [0, 1] (divided by log2(k)). The tier thresholds
  (0.4 / 0.7 / 0.9) were calibrated for a unit scale, but the raw Shannon sum grows with hit
  count — with ≥3 balanced hits every question fired at the top tier and the inline/silent
  tiers were unreachable.
• The cooldown is scoped per session in SQL. It previously scanned the 20 most recent global
  question events and filtered in Python, so on a multi-session engine a session's last
  question fell outside the window and the cooldown silently stopped firing.
• _knob_tune persists monotropism only when the value actually changes.
• The pending-question projection and the curiosity_pending MCP tool are age-bounded.

Ranking determinism

recall_for_response and recall_for_benchmark sorted hits by score alone. Python's sort is
stable, so equal-scored hits inherited whatever order the candidate scan produced, making
served ordering depend on scan order. Both now use the same (-score, str(record_id)) key the
candidate sort already used. literal_preservation — which gates the semantic tier boost —
also logged nothing when it fell back, so a bypassed gate left no trace; that is now visible.

CI: the Rust workspace, and the defects it caught immediately

• A rust job in the non-blocking weekly lane: cargo fmt, clippy, test --no-fail-fast,
  and cargo deny check licenses bans against the existing deny.toml. Every step carries
  if: always() so one red cannot mask the rest — without it a failing fmt skips the tests
  entirely, which is the opposite of the point.
• A crash-recovery test that had never run. It matched StoreError::Integrity while the
  pager raises StoreError::SnapshotFence (pager.rs:345). The behaviour was correct and the
  asserted detail string already matched — only the variant had moved, and no test in the
  crate referenced SnapshotFence at all.
• An unlicensed workspace crate. cargo deny, on its first execution ever, rejected
  iai_mcp_vec_core as error[unlicensed] — the only member without a license field. It now
  inherits MIT OR Apache-2.0 like every sibling.
• A deny-by-default clippy error. clippy::approx_constant on a π literal in a codec test
  fixture — the sole reason the Clippy step was red, since the rest are warnings.
• py.typed shipped. It sat at src/py.typed, outside the importable package, so it never
  reached the wheel and nothing downstream treated iai_mcp as typed despite the package
  advertising inline types.
• pip/Cargo caching, and Dependabot across all five manifests (pip, the root cargo workspace,
  the separate desktop/src-tauri workspace, the npm wrapper, and workflow actions) plus
  pip-audit — the Python dependency surface previously had no vulnerability scanning at all.
• The clippy backlog cleared, 142 warnings + 1 error → 15 warnings, 0 errors. Mostly
  cargo clippy --fix, type aliases, and two loop rewrites. One scoped suppression:
  useless_conversion is allowed because ~86 hits point into #[pyfunction] macro expansion
  with nothing in the repo to rewrite; commented with its removal condition, since pyo3 0.29
  would fix it upstream but pulls time 0.3.54, needing rustc 1.88 against this repo's 1.87
  pin. unexpected_cfgs uses check-cfg to declare gil-refs known rather than silence it,
  so any other unexpected cfg still warns.
• A flaky RSS gate de-flaked. test_rss_plateau_across_batches compared single endpoint
  samples (plateau[0] vs plateau[-1]) and failed the macOS gate at final=126 baseline=107.
  It is not a leak: the same code plateaus perfectly flat on Linux (spread 0), so macOS
  libmalloc is retaining freed pages rather than the store growing. It now compares the median
  of the first half of the steady-state samples against the median of the second half, with the
  16 MB margin unchanged.

Testing

Full macOS gate green. cargo test --workspace --no-fail-fast: 481 passed, 1 failed across
52 binaries, up from a 480/2 baseline. cargo deny check licenses bans: ok. New
tests/test_curiosity_drain.py covers the drain, entropy normalization, per-session cooldown,
resolution (cue normalization, cross-session isolation, idempotency), and the prune.

Known reds, left deliberately

• lilli-hd FHRR golden-parity bundle fails on x86-64. This is not a kernel defect:
  measured on x86-64, the Python reference in src/iai_mcp/lilli/tiers/fhrr.py misses its own
  golden in 328/10000 components while the Rust kernel misses 164. ~95% of the diffs
  are components whose pre-truncation value sits within 1e-6 of an integer boundary (byte
  inputs make angles exact multiples of τ/256, so their means land on quantisation
  knife-edges); the rest have resultant magnitude ~1e-16, i.e. antipodal phasors where
  atan2 is undefined. The golden encodes one machine's libm rounding for a computation that
  is ill-conditioned by construction. Untouched, because golden_parity.rs forbids
  regenerating a golden to make a test pass and relaxing byte-exact parity is a maintainer
  decision.
• rustfmt has never been enforced — mechanical, but far too large to fold in here.
• Two test_recall_rank_fusion.py assertions flaked once on CI and passed on a re-run of the
  identical commit; the cause did not reproduce across five local configurations, so it is
  not claimed as fixed. The tie-break above removes one real source of nondeterminism, and
  those assertions now print the served ordering and gate inputs so a recurrence is
  diagnosable from the log alone.

[CodeAbra]

Context, since you have been into this subsystem twice now.

It went quiet on May 16 when I emptied the recall path of everything that was not serving memory. The deferred inputs are the residue of that, not an oversight.

The eleven weeks went on the rule, not the plumbing. Entropy says the engine is unsure; it does not say the user has a problem. What shipped this morning asks only when the top two candidates carry a live contradicts edge, names both sides verbatim, and closes the question as soon as the topic gains any record. Rich but consistent knowledge stays silent. Anything touching profile knobs also clears the full suite first: 5558 tests, ~1h13m a run.

Your Rust lane is the same cost made visible. First real cargo test and two pre-existing reds appear. That is the argument for it, and I want it.

Reopen from current main with the CI and determinism work? cargo test/clippy/fmt run nowhere, deny.toml has never executed, pipeline.py:1773/:2096 sort on score alone. Drops ~700 lines and the conflict.

Golden: 328 against 164 reads as one machine's rounding, not kernel drift. We only build on macOS ARM, so you are the first to look at x86. Tolerance is my call. rustfmt its own PR.

[CodeAbra]

Heads up before your next push, so you are not working against a moving target.

Three things landed on my side since the last comment:

• Served ordering. You spotted the class. It turned out to be four serve sites plus the supersede-cap window, so it is now one shared sort_served_hits with a guard test keeping score-only sorts out of those files.
• rustfmt. The workspace is swept here; cargo fmt --all --check is green.
• The FHRR golden. It now carries the platform tolerance the algorithm actually has, which your measurement is what settled.

That leaves the part I cannot do without you and still want: the CI lane itself. The rust job, dependabot.yml, pip-audit. Nothing here duplicates it, and it should land as your PR.

So please strip everything the formatter touched before pushing again, or we both spend the evening resolving pure whitespace. Curiosity too. What is left after that is the workflow files plus any real crate defects your cargo test run surfaced that I have not covered, and those I want to see.

[CodeAbra]

The /proc/net/unix find is the good kind. You caught that our own doctor already falls back to ss -lxp when lsof comes back empty on Linux, and the test helpers never got the same treatment. An empty PID set reading as "nothing is bound" instead of "the lookup failed" is exactly the shape of bug that stays invisible on a mac-only gate. Taking it.

The golden now states its tolerance rather than pretending it has none. I reproduced your numbers first and it is worse than you found: at n=2, 4885 of 10000 components land within 1e-6 of a quantisation boundary, so no count threshold can hold. The rule ended up per component. A byte may differ only when the resultant magnitude is ~0, or when the pre-truncation value sits within 1e-6 of a boundary and the pair is exactly adjacent. Everything else stays byte-exact, and it is mutation-tested against seven simulated kernel bugs. When 2.8.1 is out, run x86-64 against it and tell me what you see.

The workspace is also rustfmt-clean here now, so your lane can land blocking instead of weekly.

Which brings me back to the ask: please drop the formatter commits from this PR. We swept the same files independently and every one of them now collides. The parts I want are the CI lane, dependabot, pip-audit, the /proc fix, and any other real defect the run surfaced. Right now those are buried under 113 files of whitespace.