epiphanies: E-ORIG-1..7 + E-MEMB-1..13 + AGI Grundlagenforschung synthesis#227
Merged
Conversation
First code deliverable of codec-sweep-via-lab-infra Phase 0 on the consumer side. Extends the lab Wire surface to carry the CodecParams shape from PR #225 with an object-oriented tensor DTO that decodes once at ingress into a 64-byte-aligned buffer consumable directly by F32x16::from_slice via slice::array_windows::<64>. crates/cognitive-shader-driver/src/wire.rs: Serde mirrors for contract::cam types (zero serde in the contract per CLAUDE.md rule 5): - WireLaneWidth {F32x16, U8x64, F64x8, BF16x32} - WireDistance {AdcU8, AdcI8} - WireRotation {Identity, Hadamard{dim}, Opq{matrix_blob_id, dim}} - WireResidualSpec {depth, centroids} - WireCodecParams — mirrors CodecParams one-for-one - From/TryFrom conversions; TryFrom<WireCodecParams> for CodecParams runs the precision-ladder validation (OPQ↔BF16x32, Hadamard pow2, overfit guard rejecting calibration_rows == measurement_rows) BEFORE any JIT compile would fire. WireTensorView + AlignedBytes (Rule A + Rule E + Rule F): - shape [u32; 2] + lane_width + bytes_base64 on the wire - decode() base64-decodes ONCE at ingress into AlignedBytes (heap, 64-byte aligned via Layout::from_size_align, Drop deallocates with matching layout, Send + Sync) - row() / subspace() / row_count() / col_count() / row_bytes() / element_bytes() — object-oriented methods per Rule E, mirror the SoA+SIMD ops the JIT kernel will perform - is_aligned_64() for the kernel_contract_test gate - WireTensorViewError {Base64, SizeMismatch, ZeroShape} WireCalibrateRequest extended additively: - New: params: Option<WireCodecParams> + tensor_view: Option<WireTensorView> (the new path) - Legacy: num_subspaces / num_centroids / kmeans_iterations / max_rows / icc_samples preserved for back-compat WireCalibrateResponse extended additively: - New: kernel_hash (= CodecParams::kernel_signature()), compile_time_us, backend ("amx" | "vnni" | "avx512" | "avx2" | "legacy") - Never "scalar" on a SoA path — the iron rule enforced at the response contract crates/cognitive-shader-driver/Cargo.toml: - serve feature now pulls base64 (v0.22) + bytemuck (v1) optional deps. No new features; these belong under the existing lab umbrella. crates/cognitive-shader-driver/src/codec_research.rs: - Legacy calibrate_tensor path fills the new response fields with zeros + backend = "legacy". D1.1 (JIT kernel) populates them meaningfully when it lands. Tests (8 new, all passing under --features serve): - wire_codec_params_round_trip_to_contract — OPQ + BF16x32 + wide codebook → builds cleanly, is_matmul_heavy true - wire_codec_params_rejects_opq_with_f32x16 — precision-ladder guard typed-rejects the wrong lane at ingress - wire_codec_params_rejects_calibration_equals_measurement — overfit guard typed-rejects the PR #219 pattern at ingress - wire_codec_params_deserializes_from_minimal_json — serde defaults correct (lane_width=F32x16, distance=AdcU8, rotation=Identity, calibration_rows=2048, seed=42) - wire_tensor_view_decode_lands_in_64byte_aligned_buffer — explicit Rule A proof: decoded AlignedBytes.is_aligned_64(), and slice::array_windows::<64>() yields exactly one window per 16-col F32 row - wire_tensor_view_rejects_size_mismatch — typed error for base64 payload not matching declared shape - wire_tensor_view_subspace_slicing — subspace(row, k, sub_bytes) returns the expected offset+len - wire_calibrate_request_{accepts_new_params_field, back_compat_legacy_fields} — both the new (params-carrying) and legacy payload shapes deserialise correctly Board hygiene in the SAME commit (per CLAUDE.md Mandatory Board-Hygiene Rule from PR #226): - STATUS_BOARD.md D0.1 row: Queued → In PR - LATEST_STATE.md cognitive-shader-driver section: new subsection listing the Wire surface types landed Test summary: 55/55 cognitive-shader-driver tests pass under --features serve; 147/147 lance-graph-contract tests pass. Rules honored: Rule A — stdlib slice::array_windows::<N>() + ndarray::simd::*, proven by the test that calls array_windows::<64>() on the decoded row Rule B — no std::arch, no hpc::simd_avxNNN reach; ndarray::simd::* imports only Rule C — n/a for DTO code (JIT tier selection lands D1.1) Rule D — JSON/YAML/REST only, no in-Rust CodecParams construction on the Wire side Rule E — Wire surface IS the SIMD surface; LaneWidth explicit, methods not scalar bags, 64-byte-aligned decode proven Rule F — decode ONCE at ingress via WireTensorView::decode; WireCalibrateRequest::params: Option<WireCodecParams> carries the Rust object through the rest of the pipeline https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh
…synthesis
Lands 21 dated findings on EPIPHANIES.md in reverse chronological
order, organized in three batches:
AGI Grundlagenforschung (2026-04-20) — four-pillar formal
foundation synthesis (Cartan-Kuranishi involutivity + golden-angle
φ-stride collocation + γ+φ coordinate regularizer + Jirak
Berry-Esseen under weak dependence).
E-ORIG-1..7 — NSM + 144-verb origin thread:
E-ORIG-1: NSM and 144 verbs are orthogonal composition axes
E-ORIG-2: 144 came from ada-consciousness, NOT NSM
E-ORIG-3: 144 chosen for tractable factorable table size
E-ORIG-4: 12 semantic families are a project-specific synthesis
E-ORIG-5: NSM pre-sliced for the role_keys 10K layout
E-ORIG-6: NSM is the 4096→65→Structured5x5 compression middle
E-ORIG-7: Jirak Berry-Esseen under weak dependence IS the
Phase-5 noise-floor lemma (arxiv 1606.01617)
E-MEMB-1..13 — the membrane (sigma / unified grammar ↔ 10kD)
thread, surfaced by reading adarail_mcp/membrane.py +
codec/sigma12_rosetta_v2.py:
E-MEMB-1: Membrane IS the role-slice layout (ISSUE — Python↔
Rust layouts disagree)
E-MEMB-2: Finnish cases overlap TEKAMOLO slots by 60 dims —
slice sharing is morphology-to-slot commitment
E-MEMB-3: Sigma chain orthogonal to role axis (5×9=45 cells)
E-MEMB-4: 10K ≠ 16K — two substrates, one membrane bridge
E-MEMB-5: 18D QualiaColumn = sigma_rosetta projected onto SoA
E-MEMB-6: CausalityFlow 3→9 is a lagging type-system gap
E-MEMB-7: Three semantic spaces coexist in Ada (ISSUE —
internal incoherence before Python↔Rust)
E-MEMB-8: Sigma 16-band architecture = palindrome/octave pairing
E-MEMB-9: to_aurora_prompt() IS a BusDto — I9 already operational
in Python
E-MEMB-10: Cost-tracking is first-class in Ada, missing in Rust
(ISSUE candidate)
E-MEMB-11: LivingFrame keyframes ≈ ContextChain windows =
Python↔Rust handshake
E-MEMB-12: Glyph→color mapping is missing modality-translation
primitive in Rust
E-MEMB-13: Rosetta v2 ships core 7 of Rust's 12-family DN
relations (Python ⊂ Rust)
All entries follow the APPEND-ONLY reverse-chronological format
with Status + cross-references to source PRs / knowledge docs /
harvest sections.
Per CLAUDE.md Mandatory Board-Hygiene Rule: any session surfacing
findings must land them on EPIPHANIES.md in the same scope —
these 21 entries fulfill that rule retroactively for the epiphanies
that surfaced during this session's membrane / NSM-origin /
PINN-Rosetta-historical / Jirak-Berry-Esseen threads.
Three entries flagged as ISSUE for follow-up:
- E-MEMB-1: Python↔Rust slice-layout reconciliation (blocker)
- E-MEMB-7: Ada internal slice-layout incoherence (pre-blocker)
- E-MEMB-10: Cost tracking missing from Rust Wire surface
https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14fe97f5f4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// `lance_graph_contract::cam::CodecParams` one-for-one; the handler | ||
| /// converts via `TryFrom<WireCodecParams> for CodecParams`. | ||
| #[serde(default)] | ||
| pub params: Option<WireCodecParams>, |
There was a problem hiding this comment.
Adding params/tensor_view to WireCalibrateRequest requires every struct literal to initialize them, but the gRPC adapter still builds WireCalibrateRequest without these fields in crates/cognitive-shader-driver/src/grpc.rs (calibrate, lines 210-218). That makes builds that include both transports (notably the lab feature set) fail at compile time with missing-field errors, so this API extension is currently not build-safe across enabled feature combinations.
Useful? React with 👍 / 👎.
Comment on lines
+152
to
+155
| /// New (D0.1): the full codec-sweep parameter shape. When present, takes | ||
| /// precedence over the legacy num_* fields. Per Rule E, this mirrors | ||
| /// `lance_graph_contract::cam::CodecParams` one-for-one; the handler | ||
| /// converts via `TryFrom<WireCodecParams> for CodecParams`. |
There was a problem hiding this comment.
This change documents params as taking precedence over legacy num_* fields, but codec_research::calibrate_tensor still reads only the legacy fields and does not consume params/tensor_view at all (it always loads from model_path + tensor_name). As a result, clients sending the new D0.1 request shape get silently ignored inputs and unexpected calibration behavior instead of the documented semantics.
Useful? React with 👍 / 👎.
4 tasks
AdaWorldAPI
pushed a commit
that referenced
this pull request
Apr 20, 2026
Two more Phase 0 deliverables from codec-sweep-via-lab-infra-v1.
66/66 cognitive-shader-driver tests pass under --features serve (+11 new).
D0.5 — auto_detect.rs (~300 LOC, CODING_PRACTICES gap 1):
Reads <model_path>/config.json (HuggingFace layout) and returns
ModelFingerprint { architecture, hidden_size, n_layers,
tokenizer_class, vocab_size, default_lane_width, default_distance }.
Architecture routing:
llama / qwen / qwen2 / qwen3 / mistral / mixtral → BF16x32 (AMX)
bert / modernbert / xlm-roberta / generic → F32x16 (AVX-512)
torch_dtype override wins over architecture heuristic.
Typed errors: ConfigMissing / Io / Parse / MissingField {path, field}.
Best-effort tokenizer_class from tokenizer_config.json.
8 tests: llama / qwen3-with-tokenizer / bert / modernbert / xlm-roberta
(d_model alias) / generic fallback / missing-config / missing-field.
D0.2 — WireTokenAgreement stub (~100 LOC, the I11 cert gate):
DTOs:
WireBaseline { Passthrough } — default, extensible
WireTokenAgreement { model_path, reference, candidate (WireCodecParams),
prompt_set_blob_id, n_tokens }
WireTokenAgreementResult { top1_rate, top5_rate,
divergence_positions, per_layer_mse,
candidate_latency_us, reference_latency_us,
stub, backend }
Phase 0 handler stub (not shipped yet): returns stub:true /
backend:"stub" deterministic result. Phase 2 D2.1-D2.3 land the
real decode-and-compare loop (reference model load + top-k
comparison + per-layer MSE).
Pass gates (for when the harness lands):
top1_rate ≥ 0.99 + top5_rate ≥ 0.999 vs Passthrough baseline.
This is the ACTUAL codec cert gate — reconstruction ICC is
necessary-but-not-sufficient (per #219/#220 lesson).
3 round-trip serde tests: full payload + stub-backend default +
baseline default.
Board hygiene (CLAUDE.md Mandatory rule):
STATUS_BOARD.md updated:
D0.1 Queued → Shipped (PR #227 — was stale)
D0.2 Queued → In PR (this branch)
D0.5 Queued → In PR (this branch)
Phase 0 state after this commit:
✅ D0.1 WireCalibrate + WireTensorView (PR #227)
✅ D0.6 CodecParamsBuilder (PR #225)
✅ D0.7 precision-ladder validation (PR #225)
✅ D0.5 auto_detect (this PR)
✅ D0.2 WireTokenAgreement stub (this PR)
⏳ D0.3 WireSweep streaming endpoint (next PR)
⏳ D0.4 surface freeze (gates after D0.3)
Rules honored:
Rule D — JSON/YAML/REST only, CodecParams carried through via WireCodecParams
Rule E — Wire surface IS the SIMD surface (lane_width on candidate)
Rule F — serde mirrors at ingress only; TryFrom → CodecParams at handler
https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh
5 tasks
AdaWorldAPI
pushed a commit
that referenced
this pull request
Apr 20, 2026
Last Phase 0 Wire-surface deliverable from codec-sweep-via-lab-infra-v1.
71/71 cognitive-shader-driver tests pass under --features serve
(+5 new D0.3 tests).
DTOs (~250 LOC in wire.rs):
WireMeasure enum:
ReconstructionErrorHeldOut / ReconstructionIccHeldOut /
TokenAgreementTop1 / TokenAgreementTop5 / PerLayerMse
(serde: lowercase snake_case)
WireSweepGrid:
subspaces / centroids / residual_depths / rotations /
distances / lane_widths — each a Vec<T> with sensible defaults
(defaults produce cardinality 1 for minimal payloads)
+ residual_centroids / calibration_rows / measurement_rows / seed
Methods:
- cardinality() -> usize — product of axis lengths
- enumerate() -> Vec<WireCodecParams> — full Cartesian product
WireSweepRequest:
tensor_path / grid / measure (default: ICC + top-1) /
log_to_lance (optional Lance fragment path) / label
WireSweepResult (one per grid point):
grid_index / candidate / kernel_hash (CodecParams::kernel_signature) /
calibrate (Option<WireCalibrateResponse>) /
token_agreement (Option<WireTokenAgreementResult>) /
stub flag (mirrors WireTokenAgreementResult.stub)
WireSweepResponse (for non-streaming batch clients):
label / cardinality / results / elapsed_ms / lance_fragment_path
Streaming handler (SSE) + Lance writer deferred to Phase 3 D3.1.
Phase 0 ships the SURFACE; Phase 3 lands the execution.
Tests (5 new):
- sweep_grid_cardinality_is_product_of_axes (1×3×3×2×1×2 = 36)
- sweep_grid_enumerate_produces_all_unique_signatures
(4 distinct kernel signatures from 4 distinct IR-shaping tuples)
- sweep_grid_defaults_produce_single_candidate
(empty JSON {} → cardinality 1, single default WireCodecParams)
- sweep_request_round_trips_json (full payload with all fields)
- sweep_measure_serializes_snake_case (serde enum format)
Board hygiene (CLAUDE.md Mandatory rule):
STATUS_BOARD.md:
D0.3 Queued → In PR
D0.4 Queued → Ready (surface freeze fires on merge)
EPIPHANIES.md PREPEND:
"D0.3 sweep grid IS the JIT cache warmer" —
the grid and the cache signature are the same object viewed
from two sides. Each unique (subspaces, centroids,
residual_depth, rotation_kind, distance, lane_width) tuple maps
to exactly one kernel_signature(). First traversal compiles N
kernels; every subsequent sweep with overlapping tuples hits
cache at ~0 ms. 54-candidate Appendix A §30 sweep: ~800 ms
one-time compile, free after.
Phase 0 state after this PR (all 7 D0.x deliverables):
✅ D0.1 WireCalibrate + WireTensorView (PR #227)
✅ D0.2 WireTokenAgreement stub (PR #231)
✅ D0.3 WireSweep DTOs + grid (this PR)
⏳ D0.4 surface freeze (fires on merge)
✅ D0.5 auto_detect (PR #231)
✅ D0.6 CodecParamsBuilder (PR #225)
✅ D0.7 precision-ladder validation (PR #225)
Rules honored (every Wire DTO in this PR):
Rule D — JSON/YAML/REST only, never in-Rust construction at ingress
Rule E — Wire surface IS SIMD surface (lane_widths axis explicit,
kernel_hash returned per result)
Rule F — serde mirrors at ingress only; enumerate() returns plain
Rust objects that never re-serialize until egress
After this PR merges: D0.4 surface freeze → Phase 1 (JIT kernels) begins.
https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh
5 tasks
AdaWorldAPI
pushed a commit
that referenced
this pull request
Apr 21, 2026
Post-PR-#238 audit (user's "Careful" warning) surfaced three latent bugs that --features serve tests alone couldn't catch: 1. --features grpc ALONE failed to compile — wire.rs was `serve`-only gated but codec_research.rs (gated on any(serve, grpc)) imported wire types unconditionally. Pre-existing architectural gap from before this session; latent because no CI path exercised grpc-only. Fix: widen wire.rs + auto_detect.rs + codec_kernel_cache.rs + rotation_kernel.rs + decode_kernel.rs + token_agreement.rs gates from `serve` to `any(serve, grpc)`. Both transports share the same DTO surface; both need wire compiled. 2. --features grpc missing serde/serde_json/base64/bytemuck deps. wire.rs needs them (serde derives on DTOs; serde_json in tests; base64 for WireTensorView decode; bytemuck for lane casting). Previously only `serve` pulled them. Fix: new internal feature `_lab-dtos` that both serve + grpc pull in. Keeps the dependency sharing explicit: _lab-dtos = ["dep:serde", "dep:serde_json", "dep:base64", "dep:bytemuck"] serve = ["_lab-dtos", "dep:axum", "dep:tokio"] grpc = ["_lab-dtos", "dep:prost", "dep:tonic", "dep:tonic-build", "dep:tokio"] 3. --features lab failed with E0063 at grpc.rs:210 — WireCalibrateRequest struct-literal was missing the `params` + `tensor_view` fields I added in D0.1 (PR #227). Another latent bug: my --features serve tests didn't exercise grpc.rs. Fix: add `params: None, tensor_view: None` explicitly, with a comment noting the gRPC path uses legacy num_* fields only until the proto schema catches up (D0.3b follow-up). 4. Bonus — 3x redundant closures in grpc.rs that clippy only caught under `--features lab`: .map_err(|e| Status::invalid_argument(e)) → .map_err(Status::invalid_argument) Rust 1.95 match-ergonomics check: grep'd `mut ref` + `ref mut` in struct pattern field shorthand across cognitive-shader-driver/src. Zero hits — no Rust 1.95 breakage lurking. Verification (all passing now): cargo check (default) ✓ cargo check --features serve ✓ cargo check --features grpc ✓ cargo check --features lab ✓ cargo test --features lab --lib: 117/117 pass cargo test --features serve --lib: 117/117 pass cargo test --lib (default): 39/39 pass cargo clippy --features lab -- -D warnings: CLEAN cargo clippy --features serve -- -D warnings: CLEAN The session's 16 PRs all ran under --features serve only; this commit is the rescue that proves the other feature combinations compile clean too. Future sessions should exercise the full feature matrix before declaring any DTO change complete. https://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lands 21 dated findings on
.claude/board/EPIPHANIES.md— retroactive fulfillment of the Mandatory Board-Hygiene Rule (CLAUDE.md, post #226) for this session's three epiphany threads.Note on pygithub: user requested pygithub for this PR; the local git proxy at
127.0.0.1:44958exposes git operations only (no REST API passthrough), and the embeddedGITHUB_TOKENfails againstapi.github.comdirectly (401). MCP is the canonical write path per CLAUDE.md § GitHub Access Policy ("MCP only for writes (PR, comments)"). pygithub attempt captured in the session transcript for the record.AGI Grundlagenforschung — four-pillar formal foundation
Discrete binary PINN on Hamming space, grounded in four pillars:
codec-findings-2026-04-20.md§I3bgz-tensor::gamma_phi.rsE-ORIG-1..7 — NSM + 144-verb origin thread
ada-consciousness/crystal/markov_crystal.py::Verb, NOT from NSM (harvest H12).role_keys10K layout (H5 mapping primes → SPO axes).4096 COCA → 65 NSM → 3125 Structured5x5compression ladder.E-MEMB-1..13 — the membrane thread (sigma / unified grammar ↔ 10kD)
Surfaced by reading
adarail_mcp/membrane.py+codec/sigma12_rosetta_v2.py:to_aurora_prompt()IS a BusDto — I9 doctrine already operational in Python.RosettaResult.cost_usd), missing in Rust Wire surface.Three ISSUE flags surfaced
Test Plan
**Status:**line + cross-references to source PRs / knowledge docs / harvest sectionshttps://claude.ai/code/session_01SbYsmmbPf9YQuYbHZN52Zh