From f33e41dc045e9eeeaea208c99145aae05d7c147f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 21 Apr 2026 05:29:11 +0000 Subject: [PATCH 1/2] D3.2 client driver + starter YAML configs (118/118 tests, +1 spec-drift guard) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Final Phase 3 scaffold deliverable — curl-driven lab iteration against the shipped /v1/shader/sweep endpoint. Files: configs/codec/README.md — inventory + DoS-ceiling note + anti-#219 stub:true flag explanation configs/codec/00_pr220_baseline.yaml - PR #220 baseline regression: 6 subspaces × 256 centroids × identity rotation. Expected ICC ≈ 0.195 mean when D2.2 lands real decode-and-compare. configs/codec/10_wider_codebook.yaml - PR #220 fix (a): centroids ∈ {256, 512, 1024}. Cardinality 3, three distinct kernel signatures → warm cache after one pass. configs/codec/12_hadamard_pre_rotation.yaml - PR #220 fix (c): Hadamard × centroids cross-product (2×2 = 4). Hadamard stays Tier-3 F32x16 per Rule C. scripts/codec_sweep.sh - yq YAML → JSON conversion - POST to ${SHADER_LAB_URL}/v1/shader/sweep (default localhost:3001) - jq-pretty request + response - Stub honesty check: prints results[0].stub flag → verifies Phase 0 returns true (machine-checkable anti-#219) - Requires: yq (mikefarah/yq ≥ v4), curl, jq wire.rs +1 test: sweep_request_yaml_shape_deserializes_via_serde_json - Inline JSON fixture mirroring the canonical YAML → JSON shape - If this test breaks, the YAML configs are stale relative to the Rust DTOs → scripts/codec_sweep.sh would fail at runtime - Caught a real drift during development: PascalCase "Identity" vs the DTO's rename_all="lowercase" (YAMLs correctly use lowercase; test fixture had the typo) Phase state: Phase 0 ✅ complete Phase 1 scaffold ✅ (D1.1 / D1.2 / D1.3 shipped; D1.1b queued) Phase 2 scaffold ✅ (D2.1 harness + D2.3 handler; D2.2 queued) Phase 3 scaffold ✅ — D3.1 batch handler + D3.2 client driver shipped ⏳ D3.1b real Lance append writer queued DoS-ceiling note: sweep handler rejects grids with cardinality > 10_000 before enumeration (PR #238 P1 fix). README documents the ceiling so config authors can budget axis lengths. Rule D honored: adding a new codec candidate = authoring a new YAML file in configs/codec/. Zero Rust changes. Zero rebuilds. Rules F honored at the client boundary: YAML → JSON → HTTP ingress. Single deserialisation at the shader-lab's handler; everything after is in-memory Rust (WireSweepRequest → CodecParams → grid enumerate() → per-candidate WireSweepResult). https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh --- .claude/board/STATUS_BOARD.md | 2 +- configs/codec/00_pr220_baseline.yaml | 22 ++++++++ configs/codec/10_wider_codebook.yaml | 23 ++++++++ configs/codec/12_hadamard_pre_rotation.yaml | 24 ++++++++ configs/codec/README.md | 36 ++++++++++++ crates/cognitive-shader-driver/src/wire.rs | 32 +++++++++++ scripts/codec_sweep.sh | 61 +++++++++++++++++++++ 7 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 configs/codec/00_pr220_baseline.yaml create mode 100644 configs/codec/10_wider_codebook.yaml create mode 100644 configs/codec/12_hadamard_pre_rotation.yaml create mode 100644 configs/codec/README.md create mode 100755 scripts/codec_sweep.sh diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 7501c1b7..c5396108 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -79,7 +79,7 @@ afterwards is a JIT kernel, not a rebuild. Plan path: | D-id | Title | Status | PR / Evidence | |---|---|---|---| | D3.1 | Server-side sweep handler + Lance fragment append | **In PR** | branch — `sweep_handler` batch mode: enumerates `WireSweepGrid::enumerate()`, validates each via TryFrom(CodecParams) at ingress, returns `WireSweepResponse { results: [WireSweepResult { kernel_hash, stub:true }], cardinality, elapsed_ms }`. SSE streaming + real calibrate/token-agreement per point deferred to D3.1b. Route: `POST /v1/shader/sweep`. | -| D3.2 | Client-side driver + config files | **Queued** | target ~20 LOC + YAML configs | +| D3.2 | Client-side driver + config files | **In PR** | branch — 3 starter YAML configs (`configs/codec/{00_pr220_baseline, 10_wider_codebook, 12_hadamard_pre_rotation}.yaml`), `scripts/codec_sweep.sh` curl wrapper, `configs/codec/README.md`, YAML-shape spec-drift guard test. 118/118 tests pass. | ### Phase 4 — Frontier analysis — Queued diff --git a/configs/codec/00_pr220_baseline.yaml b/configs/codec/00_pr220_baseline.yaml new file mode 100644 index 00000000..5f0aa40a --- /dev/null +++ b/configs/codec/00_pr220_baseline.yaml @@ -0,0 +1,22 @@ +# Phase 0 baseline — single candidate with defaults. +# Reproduces PR #220's setup (6 subspaces × 256 centroids × identity rotation) +# for regression testing. Expected: ICC ≈ 0.195 mean when D2.2 lands real +# decode-and-compare; Phase 0 stub returns top1_rate = 0.0 + stub:true. +name: pr220_baseline +tensor_path: models/qwen3-tts-0.6b/q_proj.safetensors +grid: + subspaces: [6] + centroids: [256] + residual_depths: [0] + rotations: + - { kind: identity } + distances: [AdcU8] + lane_widths: [F32x16] + residual_centroids: 256 + calibration_rows: 2048 + measurement_rows: 512 + seed: 42 +measure: + - reconstruction_icc_held_out + - token_agreement_top1 +label: "pr220 baseline regression" diff --git a/configs/codec/10_wider_codebook.yaml b/configs/codec/10_wider_codebook.yaml new file mode 100644 index 00000000..47d0e5d1 --- /dev/null +++ b/configs/codec/10_wider_codebook.yaml @@ -0,0 +1,23 @@ +# PR #220 fix (a) — wider codebook (1024+ centroids per subspace). +# Tests the first remedy from the PR #220 "What's Needed to Fix" list. +# The JIT cache warms once per unique (subspaces, centroids, …) tuple; +# this grid generates 1 × 3 × 1 × 1 × 1 × 1 = 3 distinct signatures. +name: wider_codebook_sweep +tensor_path: models/qwen3-tts-0.6b/gate_proj.safetensors +grid: + subspaces: [6] + centroids: [256, 512, 1024] + residual_depths: [0] + rotations: + - { kind: identity } + distances: [AdcU8] + lane_widths: [F32x16] + residual_centroids: 256 + calibration_rows: 2048 + measurement_rows: 512 + seed: 42 +measure: + - reconstruction_icc_held_out + - token_agreement_top1 + - token_agreement_top5 +label: "PR #220 fix (a): wider codebook" diff --git a/configs/codec/12_hadamard_pre_rotation.yaml b/configs/codec/12_hadamard_pre_rotation.yaml new file mode 100644 index 00000000..5e2b0960 --- /dev/null +++ b/configs/codec/12_hadamard_pre_rotation.yaml @@ -0,0 +1,24 @@ +# PR #220 fix (c) — Hadamard pre-rotation. +# Hadamard stays at Tier-3 F32x16 per Rule C (add/sub butterfly, not matmul +# → no AMX benefit). Sylvester construction requires dim power-of-two. +# Combined with wider codebook to probe the compound effect. +name: hadamard_pre_rotation +tensor_path: models/qwen3-tts-0.6b/o_proj.safetensors +grid: + subspaces: [6] + centroids: [256, 512] + residual_depths: [0] + rotations: + - { kind: identity } + - { kind: hadamard, dim: 4096 } + distances: [AdcU8] + lane_widths: [F32x16] + residual_centroids: 256 + calibration_rows: 2048 + measurement_rows: 512 + seed: 42 +measure: + - reconstruction_icc_held_out + - token_agreement_top1 + - token_agreement_top5 +label: "PR #220 fix (c): Hadamard pre-rotation × centroids sweep" diff --git a/configs/codec/README.md b/configs/codec/README.md new file mode 100644 index 00000000..5de3ba02 --- /dev/null +++ b/configs/codec/README.md @@ -0,0 +1,36 @@ +# Codec Sweep YAML Configs + +Request bodies for the lab REST surface (`/v1/shader/sweep`, +`/v1/shader/calibrate`, `/v1/shader/token-agreement`). Consumed by +`scripts/codec_sweep.sh` — see that script for the curl invocation. + +**Rule D enforcement:** adding a new codec candidate is authoring a new +YAML file in this directory. Zero Rust changes. Zero rebuilds. The +running `shader-lab` binary JIT-compiles each unique `CodecParams` +signature; overlapping signatures across YAMLs hit the kernel cache. + +## Inventory + +| File | Target | What it tests | +|------|--------|---------------| +| `00_pr220_baseline.yaml` | q_proj | PR #220 baseline regression (6×256, identity rotation) | +| `10_wider_codebook.yaml` | gate_proj | PR #220 fix (a) — centroids ∈ {256, 512, 1024} | +| `12_hadamard_pre_rotation.yaml` | o_proj | PR #220 fix (c) — Hadamard × centroids | + +Additional configs from plan Appendix A (residual PQ, OPQ, composite, +CartanCascade 4-tier) land as separate YAML files as Phase 1b Cranelift +wiring makes the real decode paths runnable. + +## Expected results under Phase 0/2 stubs + +Every candidate returns `stub: true` + `backend: "stub"` in the response +until D2.2 (real decode-and-compare) lands. Clients that trust these +rates as real measurements hit the machine-checkable `stub` wall — that's +the anti-#219 defense at the type level (see `EPIPHANIES.md` 2026-04-20 +"D0.2 stub flag is anti-#219 defense at the type level"). + +## DoS ceiling + +The sweep handler rejects grids with cardinality > 10,000 before +enumeration. Multiply axis lengths to budget — e.g., `3 × 3 × 3 × 3 = 81` +is fine; `100 × 100 = 10,000` is the exact ceiling. diff --git a/crates/cognitive-shader-driver/src/wire.rs b/crates/cognitive-shader-driver/src/wire.rs index 46cc0d2a..1805fdfe 100644 --- a/crates/cognitive-shader-driver/src/wire.rs +++ b/crates/cognitive-shader-driver/src/wire.rs @@ -1584,6 +1584,38 @@ mod tests { assert_eq!(decoded.label, "phase1_initial_cross_product"); } + #[test] + fn sweep_request_yaml_shape_deserializes_via_serde_json() { + // Documents the canonical JSON shape (after yq conversion from YAML) + // for `configs/codec/*.yaml`. If this test breaks, the checked-in + // YAML configs are stale relative to the Rust DTOs — the + // scripts/codec_sweep.sh pipeline will fail at runtime. + let body = r#"{ + "tensor_path": "models/qwen3-tts-0.6b/q_proj.safetensors", + "grid": { + "subspaces": [6], + "centroids": [256, 512, 1024], + "residual_depths": [0], + "rotations": [ + { "kind": "identity" }, + { "kind": "hadamard", "dim": 4096 } + ], + "distances": ["AdcU8"], + "lane_widths": ["F32x16"], + "residual_centroids": 256, + "calibration_rows": 2048, + "measurement_rows": 512, + "seed": 42 + }, + "measure": ["reconstruction_icc_held_out", "token_agreement_top1"], + "label": "spec-drift guard" + }"#; + let req: WireSweepRequest = serde_json::from_str(body).expect("YAML→JSON shape parses"); + // 1 × 3 × 1 × 2 × 1 × 1 = 6 candidates. + assert_eq!(req.grid.cardinality(), 6); + assert_eq!(req.measure.len(), 2); + } + #[test] fn sweep_measure_serializes_snake_case() { let m = WireMeasure::ReconstructionIccHeldOut; diff --git a/scripts/codec_sweep.sh b/scripts/codec_sweep.sh new file mode 100755 index 00000000..e47803f5 --- /dev/null +++ b/scripts/codec_sweep.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# +# codec_sweep.sh — curl-driven lab iteration for codec sweeps. +# +# Reads a YAML config under configs/codec/, converts to JSON via yq, POSTs +# to the running shader-lab's /v1/shader/sweep endpoint, pretty-prints the +# WireSweepResponse. +# +# Prereqs: yq (mikefarah/yq ≥ v4), curl, jq, a running shader-lab binary +# on SHADER_LAB_URL (default http://localhost:3001). +# +# Usage: +# scripts/codec_sweep.sh configs/codec/00_pr220_baseline.yaml +# SHADER_LAB_URL=http://10.0.0.5:3001 scripts/codec_sweep.sh configs/codec/10_wider_codebook.yaml +# +# Output: JSON WireSweepResponse with per-grid-point stub results (until +# D2.2 lands real decode-and-compare). Every result row has stub:true. + +set -euo pipefail + +SHADER_LAB_URL="${SHADER_LAB_URL:-http://localhost:3001}" +ENDPOINT="${SHADER_LAB_URL}/v1/shader/sweep" + +if [[ $# -lt 1 ]]; then + echo "usage: $0 " + echo " example: $0 configs/codec/00_pr220_baseline.yaml" + exit 2 +fi + +CONFIG="$1" + +if [[ ! -f "$CONFIG" ]]; then + echo "config not found: $CONFIG" >&2 + exit 2 +fi + +if ! command -v yq >/dev/null 2>&1; then + echo "yq not installed — install mikefarah/yq (https://github.com/mikefarah/yq)" >&2 + exit 2 +fi + +# Convert YAML → JSON; POST to endpoint; pretty-print response. +json_body=$(yq -o=json '.' "$CONFIG") + +echo "=== POST $ENDPOINT ===" +echo "--- request ---" +echo "$json_body" | jq '.' +echo +echo "--- response ---" + +response=$(curl -sS -X POST "$ENDPOINT" \ + -H "Content-Type: application/json" \ + -d "$json_body") + +echo "$response" | jq '.' + +echo +echo "=== Stub honesty check ===" +stub_flag=$(echo "$response" | jq '.results[0].stub // "no results"') +echo "results[0].stub = $stub_flag" +echo "Expected: true (Phase 0 stub; D2.2 flips to false when real decode lands)." From ec05b806f7f2bd5fc8e5d254954ad1971847d70a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 21 Apr 2026 05:45:18 +0000 Subject: [PATCH 2/2] coding-practices: add SoA patterns + mandatory simd::* + 3-way BindSpace merge + adhering-agent review checklist MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four additions to .claude/CODING_PRACTICES.md — extends the existing EmbedAnything-patterns content with the SoA / object-does-the-work / substrate-level patterns that crystallized during the codec-sweep lab-infra session (PRs #225–#239). 1. SoA + Object-Does-The-Work Patterns (~100 lines) - Checklist for new DTOs / kernels / caches: sealed builders, stable signatures excluding drift, typed errors, Cache generic-over-handle, stub flag for Phase-N-before-Phase-N+k, feature matrix tested, serialisation at edges, DoS ceilings at construction not enumeration - Five additional anti-patterns (6-10) surfaced by session corrections: stateless-shader vs stateful-engine misframed, hallucinating ndarray surface, feature-matrix blindness, epiphany-dumping orientation-as-discovery, raw struct literals bypassing builders - 10 shipped-pattern reference entries citing the actual files + test counts - 8 principles: object does the work, SoA over AoS, same- substrate-different-view, Stream/Resonance/Bus lifecycle, weights are seeds, scaffold-before-codegen, feature matrix is part of contract, pin your toolchain - Read order for new sessions 2. MANDATORY: `ndarray::simd::*` canonical import (new section) - Correct/wrong examples per Rule B + invariant I2 - AMX sibling module + tile primitives + simd_caps canonical paths - Polyfill hierarchy (Tier 1 AMX → Tier 4 AVX-2, no consumer scalar tier) - Reviewer trigger for `std::arch::*` or `ndarray::hpc::simd_avxNNN::*` reach 3. 3-Way BindSpace Mutation Scheme (new section) - Table: Xor (single-writer reversible) / Bundle (multi-writer saturating, E-SUBSTRATE-1 guaranteed associative in expectation) / Superposition (preserve ambiguity) - When to use each + explicit DON'T-INTERCHANGE rule - Iron rule citation (CLAUDE.md I-SUBSTRATE-MARKOV): Xor on multi-writer path breaks the Markov guarantee - Reviewer trigger for Xor on concurrent-writer paths 4. Adhering-Agent Review Checklist (new section) - Per-agent table mapping 18 specialist agents + 5 meta-agents to the specific checklist sections they own - Spawn pattern: hand PR scope to 1-2 matching agents with pointer to this doc; they walk their rubric and return PASS/FAIL with specific line citations - Agents READ this doc as their rubric, not their personality The doc is now both the author-side pattern guide AND the reviewer-side checklist. Specialist agents adhere to it; PRs are reviewed against it; new sessions load it as part of the mandatory pre-read set. Cross-ref: CLAUDE.md I-SUBSTRATE-MARKOV + I-NOISE-FLOOR-JIRAK; lab-vs-canonical-surface.md invariants I1-I11 + six rules A-F; cognitive-shader-architecture.md 7-layer stack + SoA column types; ripple-dto-contracts.md Stream/Resonance/Bus/ThoughtStruct lifecycle; this session's PRs #225-#239 shipping the SoA patterns in practice. https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh --- .claude/CODING_PRACTICES.md | 336 ++++++++++++++++++++++++++++++++++++ 1 file changed, 336 insertions(+) diff --git a/.claude/CODING_PRACTICES.md b/.claude/CODING_PRACTICES.md index d379e5e0..064b0132 100644 --- a/.claude/CODING_PRACTICES.md +++ b/.claude/CODING_PRACTICES.md @@ -49,3 +49,339 @@ Feature gates: tokenizer = HuggingFace tokenizers 0.22 calibration = candle 0.9 + hf-hub (forward pass + training) ``` + +--- + +## SoA + Object-Does-The-Work Patterns (this workspace, 2026-04-20 shipped) + +Complement the EmbedAnything patterns above. Extracted from the codec-sweep +lab-infra work (PRs #225–#239); validated against `cognitive-shader-architecture.md` +7-layer stack + `ripple-dto-contracts.md` + the lab-vs-canonical surface invariants. + +### Checklist for new DTOs / kernels / caches + +``` +[ ] Is construction sealed behind a Builder that validates? + (raw struct literals bypass the checks — lock the fields pub(crate)) +[ ] Does the object carry a stable signature()/kernel_hash for cache keying, + and does that signature EXCLUDE runtime drift (seeds, timestamps, RNG state)? +[ ] Are typed errors returned instead of strings / Option::None? + (precision-ladder, overfit guards, dim mismatches all get their own variant) +[ ] Is the cache generic-over-handle-type (Cache) so stub tests + real + kernels share the same cache semantics? +[ ] Is there a stub flag in result DTOs for Phase-N-before-Phase-N+k? + (machine-checkable anti-#219: assert!(!result.stub) trips pipelines + that confuse stub output for real measurements) +[ ] Is the feature matrix tested? + (default / serve / grpc / lab must all `cargo check` + `cargo clippy`; + `--features serve` alone has hidden gaps like grpc-only dep missing) +[ ] Is serialisation at edges only? (Rule F — decode at REST ingress, + encode at response/Lance egress; zero serde between stages) +[ ] Are DoS ceilings enforced AT CONSTRUCTION (method check) not AT + ENUMERATION (after work started)? +``` + +### Anti-patterns (surfaced by session corrections) + +``` +6. Stateless-shader vs stateful-engine MISFRAMED as competing + → they're views of one SoA. p64 topology, bgz17 palette distance, + 4096² scan kernels, 16K fingerprints all live as BindSpace + columns; the shader DRIVES through them. Don't build a new + crate for a "higher layer" — extend by column. + +7. Hallucinating ndarray's surface from docs comments + → jitson_cranelift::JitEngine with 2-phase BUILD/RUN Arc-freeze + was ALREADY shipped. Don't invent a worse RwLock scaffold. + Probe `/home/user/ndarray/src/hpc/jitson_cranelift/` FIRST. + +8. Feature-matrix blindness + → `cargo test --features serve` is NOT the full check. A DTO + added to `wire.rs` fails grpc.rs:LINE with E0063 when the + grpc feature compiles. Run default / serve / grpc / lab + before declaring a DTO change complete. + +9. Epiphany-dumping orientation-as-discovery + → EPIPHANIES.md is for NEW findings from code/measurement. + Architecture explained by the user is ORIENTATION, not + epiphany. Tag ORIENTATION entries separately so future + sessions can distinguish discovery from onboarding. + +10. Raw struct literals bypassing builders + → `WireCodecParams { subspaces: 6, .. }` lets callers skip + precision-ladder checks. Lock fields pub(crate), make + builder the only path. Inherent optimization — the invariant + rides inside the type. +``` + +### Patterns shipped (reference implementations in this repo) + +``` +Pattern File Tests Notes +─────── ────────── ───── ───── +SoA columns (read-only, Arc) contract::cognitive_shader + driver::bindspace 16 FingerprintColumns / QualiaColumn / MetaColumn (u32 packed: thinking 6b + awareness 4b + nars_f 8b + nars_c 8b + free_e 6b) / EdgeColumn / temporal / expert +Builder + typed errors contract::cam::CodecParamsBuilder 14 .build() runs precision-ladder + overfit guard BEFORE any JIT compile +Stable signature cache key contract::cam::CodecParams::kernel_signature() 3 Excludes seed; JIT cache stays hot across seed variations inherently +Generic-over-H cache cognitive_shader_driver::codec_kernel_cache 9 CodecKernelCache; hosts StubKernel for tests, real kernels post-D1.1b +Stub flag anti-#219 wire::WireTokenAgreementResult::stub 3 `stub: bool` + `backend: "stub"` default; `!stub` assertions trip silent-fail pipelines +Three-DTO pipeline lifecycle StreamDto → ResonanceDto → BusDto → ThoughtStruct (n/a) Per ripple-dto-contracts.md — same data, four maturity stages; serialise at edges only (Rule F) +Object-safe trait + Box rotation_kernel::RotationKernel, DecodeKernel 24 Send + Sync + Debug bounds enable `Box` in ResidualComposer for recursive composition +Spec-drift guard test wire::sweep_request_yaml_shape_deserializes 1 Inline JSON fixture mirrors checked-in YAML; breaks if YAMLs drift from DTOs +DoS ceiling at construction WireSweepGrid::cardinality() pre-check 1 Early-return on > MAX before enumerate(); not a try/catch after work started +Feature matrix _lab-dtos cognitive-shader-driver/Cargo.toml — _lab-dtos = [serde + serde_json + base64 + bytemuck]; serve + grpc both pull it +Substrate proof-in-code crates/jc/examples/prove_it.rs 6 3/5 pillars pass in ~5s: E-SUBSTRATE-1 bundle assoc, φ-Weyl discrepancy, Jirak Berry-Esseen +``` + +### Principles (why the patterns cluster) + +``` +1. The object does the work. + Validation rides inside the type — Builder::build() checks precision + ladder, Rotation::new() checks pow2, CodecParamsError has one variant + per impossible combination. Downstream code doesn't re-validate. + +2. SoA over AoS, always. + One allocation per COLUMN TYPE; pack meta into u32s for cache-line + reads. 16D qualia as `Box<[f32]>` length N×18, not `Vec<[f32;18]>`. + Zero-copy SIMD views via `&[u64]` cast as U8x64 / U64x8. + +3. Same substrate, different view. + AGI(topic,angle,style,edge,qualia,temporal,expert,cycle) ≡ + Test(input,codec,invariant,meta,icc,session,operator,sweep_id) ≡ + Think(style,angle,nars,entropy,gestalt,qualia). + Three questions asked of the same SoA; foreground different columns + per view. Don't build a new data structure for a new question. + +4. Stream/Resonance/Bus DTO is the data lifecycle. + StreamDto = ingress (preserves order + ambiguity). + ResonanceDto = superposition (preserves contradiction). + BusDto = explicit compile into p64 + CognitiveShader. + ThoughtStruct = durable revisable object on blackboard. + Serialise ONLY at Stream ingress + BusDto egress/persistence. + +5. Weights are seeds. + GGUF → palette + Fingerprint<256> + holographic residual + CausalEdge64. + Inference = Hamming cascade + table lookup. No matmul. No FP inner + loop. `Weights are parameters to run matmul on` is the wrong mental + model — BindSpace columns are the model. + +6. Scaffold-before-codegen. + Ship the cache/trait/composition layer with Stub implementations + FIRST. Test cache semantics + composition in microseconds without + the real JIT engine. Replace Stubs with real impls behind the same + trait when the heavy work lands. + +7. Feature matrix is part of the contract. + A DTO added under `#[cfg(feature = "serve")]` that's referenced by + `grpc.rs` breaks `--features grpc` alone. Always run: + cargo check # default + cargo check --features serve + cargo check --features grpc + cargo check --features lab # umbrella + cargo clippy --features lab -- -D warnings + before declaring complete. + +8. Pin your toolchain. + rust-toolchain.toml at repo root. 1.94 stable today. 1.95+ has + destabilisations (JSON target specs → unstable; mut ref patterns + feature-gated). Bump deliberately in its own PR, never as part of + feature work. +``` + +### Read order for new sessions + +Before touching anything in this workspace: + +1. `.claude/BOOT.md` + `LATEST_STATE.md` + `PR_ARC_INVENTORY.md` (CLAUDE.md mandatory) +2. `.claude/knowledge/cognitive-shader-architecture.md` (7-layer stack + SoA column types + 4 data patterns) +3. `.claude/knowledge/lab-vs-canonical-surface.md` (invariants I1-I11 + six rules A-F) +4. `.claude/contracts/ripple-dto-contracts.md` (StreamDto / ResonanceDto / BusDto / ThoughtStruct — the actual contract, not re-derivable) +5. `.claude/knowledge/encoding-ecosystem.md` (MANDATORY before codec work) +6. `.claude/CODING_PRACTICES.md` (this file) + `.claude/agents/BOOT.md` (delegation targets) + +Before grepping source: consult the matching `.claude/agents/*.md` card. +Before inventing a type: grep `LATEST_STATE.md` Contract Inventory. +Before naming something an "epiphany": check whether it's ORIENTATION +already in the architecture docs; if so, tag it ORIENTATION, not FINDING. + +--- + +## MANDATORY: `ndarray::simd::*` canonical import + +Per lab-vs-canonical invariant **I2** and Rule B of the six JIT kernel +rules. Every SIMD access across `lance-graph` / `cognitive-shader-driver` +/ `thinking-engine` / `bgz-tensor` / `p64-bridge` goes through one of +three canonical surfaces — **nothing else**. + +```rust +// ─── Canonical SIMD lane types (always prefer these) ───────────────── +use ndarray::simd::{F32x16, U8x64, F16x32, F64x8, BF16x32}; +use ndarray::simd::{Fingerprint, hamming_distance_raw, popcount_raw}; + +// ─── AMX sibling module (Intel AMX on Sapphire Rapids+) ────────────── +use ndarray::simd_amx::{amx_available, vnni_dot_u8_i8, vnni_matvec, matvec_dispatch}; + +// ─── AMX tile primitives (inline-asm stable path) ─────────────────── +use ndarray::hpc::amx_matmul::{ + tile_loadconfig, tile_zero, tile_load, tile_store, tile_release, + tile_dpbusd, tile_dpbf16ps, vnni_pack_bf16, +}; + +// ─── Runtime caps detection ───────────────────────────────────────── +use ndarray::hpc::simd_caps::{simd_caps, SimdCaps}; + +// ─── WRONG (private backend reach — reviewer MUST reject) ────────── +use ndarray::hpc::simd_avx512::F32x16; // private impl; namespaces can refactor +use ndarray::hpc::simd_avx2::F32x8; // same +use std::arch::x86_64::_mm512_loadu_ps; // raw intrinsic, not a kernel +use std::simd::Simd; // stdlib portable_simd, not canonical here +``` + +**Polyfill hierarchy** per Rule C — `ndarray::simd::*` primitives resolve +to the right tier at dispatch; consumers never emit raw intrinsics: + +``` +Tier 1 — Intel AMX tiles (Sapphire Rapids+, OS-enabled XCR0+prctl) + matmul-heavy paths only: tile_dpbusd, tile_dpbf16ps +Tier 2 — AVX-512 VNNI: vnni_dot_u8_i8, vnni_matvec +Tier 3 — AVX-512 baseline: F32x16, U8x64, F64x8 (mandatory floor + per target-cpu=x86-64-v4) +Tier 4 — AVX-2 fallback: F32x8, F64x4 (cfg-gated) + +There is NO consumer-visible scalar tier. If a SoA path scalarises, +the SoA invariant is broken. ndarray::simd handles any non-x86 +scalar fallback INTERNALLY; the consumer never hand-rolls. +``` + +**Reviewer trigger:** any PR with `std::arch::*` or +`ndarray::hpc::simd_avxNNN::*` or a scalar loop on a column-sweep +hot path → reject + cite this section. Exception: the ndarray +crate itself implements backends, not a violation. + +--- + +## The 3-Way BindSpace Mutation Scheme + +BindSpace is **read-only** — `Arc<[u64; 256 * N]>` columns, never mutated +in place. Writers hold owned `Copy` microcopies (CausalEdge64, Band, +TruthValue, ThinkingStyle — all ≤ 16 B, stack-only). All mutations cross +the airgap through `contract::collapse_gate::{GateDecision, MergeMode}`. +Three merge modes, three use cases — **never interchange them**. + +``` +┌─────────────────┬────────────────────┬────────────────────────────────┐ +│ MergeMode │ Semantics │ Use when │ +├─────────────────┼────────────────────┼────────────────────────────────┤ +│ MergeMode::Xor │ target ^= delta │ SINGLE writer. XOR is │ +│ │ self-inverse │ self-inverse → reversible, no │ +│ │ commutative+assoc │ locks needed. Good for: │ +│ │ │ ─ per-expert A2A blackboard │ +│ │ │ posts (one expert at a time) │ +│ │ │ ─ single-row edge flips │ +│ │ │ ─ individual CausalEdge64 │ +│ │ │ updates on an owned cursor │ +├─────────────────┼────────────────────┼────────────────────────────────┤ +│ MergeMode:: │ saturating-add │ MULTIPLE concurrent writers │ +│ Bundle │ into int8 │ to the SAME target. Associative │ +│ │ awareness register │ and commutative IN EXPECTATION │ +│ │ │ (E-SUBSTRATE-1 guarantees │ +│ │ │ Chapman-Kolmogorov by │ +│ │ │ construction in d=10_000; │ +│ │ │ JL concentration suppresses │ +│ │ │ deviations at rate e^(-d)). │ +│ │ │ Good for: │ +│ │ │ ─ codec sweep result aggregation│ +│ │ │ ─ thinking-engine lens │ +│ │ │ composition │ +│ │ │ ─ multi-expert consensus │ +│ │ │ ─ soaking CollapseGate deltas │ +├─────────────────┼────────────────────┼────────────────────────────────┤ +│ MergeMode:: │ keep all variants │ AMBIGUITY worth preserving. No │ +│ Superposition │ present until │ writer has enough confidence to │ +│ │ CollapseGate │ commit; next cycle may │ +│ │ resolves │ disambiguate with new evidence. │ +│ │ │ Good for: │ +│ │ │ ─ WechselAmbiguity coref │ +│ │ │ candidates │ +│ │ │ ─ ResonanceDto family_candidates│ +│ │ │ ─ contradiction as structure, │ +│ │ │ not noise │ +└─────────────────┴────────────────────┴────────────────────────────────┘ +``` + +**Iron rule (CLAUDE.md I-SUBSTRATE-MARKOV):** `MergeMode::Xor` on a +**multi-writer** path breaks the Markov guarantee. Two concurrent +XOR writes cancel — racing writers lose their deltas. Always use +`Bundle` when two or more writers can reach the same target +simultaneously. Reversibility vs associativity is the trade-off; +pick based on writer cardinality, not convenience. + +**Reviewer trigger:** any PR that uses `MergeMode::Xor` on a path +where concurrent writers exist → reject + require justification +that the path is truly single-writer (typically via an owned +cursor or serialized commit). + +--- + +## Adhering-Agent Review Checklist + +This document doubles as the **review checklist each specialist agent +consults** when approving code in its domain. Spawn the right agent, +hand it the PR scope, and it walks these sections looking for the +patterns it owns: + +| Agent | Owns these checks | +|----------------------------|--------------------------------------------------------| +| `family-codec-smith` | SoA over AoS; precision-ladder in builder; stub flag | +| | on Phase-N-before-Phase-N+k DTOs; HEEL/HIP/BRANCH/TWIG/ | +| | LEAF progression; role-aware codebooks; no one-codebook | +| | for all families. | +| `bus-compiler` | StreamDto/ResonanceDto/BusDto/ThoughtStruct lifecycle; | +| | BusDto compilable into `CausalEdge64`; style maps to | +| | explicit bus knobs; no vague summaries. | +| `palette-engineer` | `ndarray::simd::*` canonical import ONLY; 256×256 | +| | palette distance tables for bgz17; no std::arch reach. | +| `certification-officer` | Feature matrix complete (default/serve/grpc/lab); spec- | +| | drift guards present; stub flag honored end-to-end; | +| | 4-decimal metric reporting; SplitMix64 seed 0x9E37… | +| | for deterministic sampling. | +| `truth-architect` | The object does the work; three-view SoA isomorphism | +| | (AGI ≡ Test ≡ Think); no new crate for a new question. | +| `integration-lead` | Rule A–F compliance on every JIT kernel; feature gates | +| | coherent; board-hygiene commits paired with code. | +| `ripple-architect` | Ripple architecture (stream → resonance → bus → thought);| +| | contradiction preserved; no collapse into generic | +| | confidence. | +| `host-glove-designer` | AGI-as-glove doctrine; structured payloads over | +| | prompt-only mediation; minimal useful interface. | +| `resonance-cartographer` | ResonanceDto superposition preserved; coherence + | +| | contradiction + pressure + drift all tracked; no early | +| | collapse. | +| `trajectory-cartographer` | BusDto trajectories across cycles; cycle_fingerprint | +| | as cache key + retrieval key + replay seed + cursor. | +| `container-architect` | Container = [u64; 256] 16 Kbit width; SoA column | +| | layout (one Arc per column, packed u32 meta); no | +| | per-row heap allocations. | +| `contradiction-cartographer` | Contradictions as first-class (phase + magnitude); | +| | Staunen markers fire; epiphany vs error-correction | +| | classification. | +| `mirror-kernel-synthesist` | Mirror/reflection primitives respect the 64×64/256×256/ | +| | 4096×4096/16K resolution ladder. | +| `perspective-weaver` | Topic/angle/style/edge columns independently Hamming- | +| | sweepable; AND-across-cascades query model. | +| `thought-struct-scribe` | ThoughtStruct durable + revisable (not replacement- | +| | only); storable on blackboard; distinct from raw text. | +| `savant-research` | Real measurements over synthetic corpus (Rule 23); | +| | NaN scan at every pipeline stage; two-universes | +| | firewall (Basin 1 continuous ≠ Basin 2 discrete). | +| `adk-coordinator` | Multi-agent orchestration; A2A blackboard via | +| | BindSpace expert column. | +| `adk-behavior-monitor` | Agent behavior against board invariants; handover | +| | protocol discipline. | +| `workspace-primer` | New-session orientation; mandatory pre-reads honored. | + +**Spawn pattern:** when authoring a PR, identify the 1–2 agents whose +domains the code touches, hand them the scope with a pointer to this +doc, and let them return approval (PASS) or a blocking-diff list (FAIL +with specific lines + which section was violated). Agents read this +doc as their rubric, not their personality.