C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning + V8 fixes (SLM-27/28/31/34/35/36) by Tyler-R-Kendrick · Pull Request #275 · Tyler-R-Kendrick/slm-training

Tyler-R-Kendrick · 2026-07-17T00:52:21Z

Summary

Six unique pieces on top of main, each with an iter doc, evidence mirror, MODEL_CARD row, and honest caveats. All fixture-grade — nothing promoted, no ship gate touched.

Scope note: while this branch was in flight, PR #285 ("DSL Diffusion Program Wave 2") independently reimplemented F2 (GraphQL), G1 (autoresearch routing), G5 (meta-traces), and C2 with different designs and merged to main first. To avoid duplicate/divergent code, this branch now defers to main's canonical versions of those pieces and keeps only its genuinely-unique work. C3/C4 were renumbered clear of #285's E277(A2)/E278(C2): C3 → E280 (matrix v16), C4 → E281/E282 (matrix v17).

1. Two latent runtime-symbol-feature defect fixes (V8/C2 code on main)

evals/loss_suites.py clears stale runtime symbol features; models/blocks.py project() is shape-polymorphic and fails closed on multi-request batches.

2. C3 — corpus-mined macro tokens (SLM-27, E280)

Tokenizer v3 <MACRO_i> rows; greedy-MDL miner; lossless expansion; fail-closed. Fixture: corpus −34.3% tokens.

3. C4 — names-disappear matched pair (SLM-28, E281/E282)

symbol_anonymization=False surface arm; exact round trip 108/108 at 1.72× tokens. Verdict OPEN — surface arm ahead on structural similarity 5/5 suites.

4. D3 — faithful Kapur tree-edit diffusion baseline (SLM-31, X22)

All-valid forward process + inverse-edit policy + value-guided search. First nonzero meaningful parse at fixture budget vs 0.0 for matched X9.

5. F1 — DSL pack contract (SLM-34)

dsl/packs/: DSLPack = {grammar, canonicalizer, oracle, generator, scope check, placeholder policy}; openui + toy-layout + arith-sketch.

6. G2 — recipe-evolution harness (SLM-35)

Bounded gene space, gate-locked selection, no RL path. Fixture proved the gate lock is load-bearing.

7. G4 — sketch-of-thought reasoning bench (SLM-36)

First checkable-answer machinery: arith-sketch DSL + deterministic evaluator oracle. Matched sketch vs direct-answer arms.

Deferred to main's #285 (dropped here to avoid duplication)

F2 GraphQL (main's graphql-js bridge), G1 autoresearch routing (program_matrix.py), G5 meta-traces (trace_store.py), C2 pseudo-embeddings — all now on main.

Verification

check-changed: 1015 passed, 9 skipped after merging current main (incl. DSL Diffusion Program — Wave 2: A2, F1/F2, G1/G3/G5, B1/B3, C2 (9 issues) #285 + Dashboard visual redesign: make Mission Control deliver its value #286).
Remaining failures are pre-existing on main, unrelated to this changeset: 2 gpu_multi_farm (missing fastmcp optional dep) and 2 quality_matrix_v10 (main's _v10 emits E277/E278 but DSL Diffusion Program — Wave 2: A2, F1/F2, G1/G3/G5, B1/B3, C2 (9 issues) #285's test still expects E263/E264 — byte-identical to origin/main, fails there too).
repo_policy ok; ruff clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

vercel · 2026-07-17T00:52:26Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
slm-training	Ready	Preview, Comment	Jul 17, 2026 12:21pm

coderabbitai · 2026-07-17T00:52:34Z

📝 Walkthrough

Walkthrough

DenoiserTower.project now supports sliced hidden inputs when runtime symbol features represent one request and rejects ambiguous multi-request inputs. evaluate_loss_suites clears runtime symbol features before evaluation, with tests covering both behaviors.

Changes

Runtime symbol feature handling

Layer / File(s)	Summary
Sliced hidden projection support `src/slm_training/models/blocks.py`, `tests/test_models/test_compiler_decode.py`	`DenoiserTower.project` reshapes non-rank-3 hidden tensors for single-request runtime features, restores output shape, and raises `ValueError` for multi-request features. Tests cover standard and candidate-gathered projections.
Evaluation runtime reset `src/slm_training/evals/loss_suites.py`, `tests/test_evals/test_denoising_nll.py`	`evaluate_loss_suites` clears runtime symbol features when supported, and tests verify stale features are removed while weighted NLL evaluation succeeds.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	⚠️ Warning	The title lists tokenizer, Kapur, packs, evolution, and reasoning work, but the PR actually fixes runtime symbol feature handling in loss evaluation and projection.	Rename it to reflect the actual change, e.g. "Clear stale runtime symbol features during loss eval and support sliced denoiser projection".

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/dsl-diffusion-slm-autoresearch-05w3zn

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Salvaged from the superseded B4/C1/C2 branch (old head 9e86c13; the lever implementations there were superseded by the E255-E257 work merged via #259). Both defects are latent on main because no merged row activates runtime_symbol_features yet; both fire the moment one does: - evaluate_loss_suites now clears request-local runtime symbol features on entry: the teacher-forced suites carry no request context, and stale per-batch features from the last training batch crash the batched NLL forward (batch-dim mismatch) or silently bias it when dimensions happen to line up - DenoiserTower.project accepts [D]/[N,D] hidden slices with features active (compiler/tree scorers project slices): shape-polymorphic for a single active request, fail-closed when multi-request features cannot be attributed to a sliced row Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

…, E259) Tokenizer v3 appends 64 reserved <MACRO_i> rows; a macro binds one id to a fixed span of fixed-vocabulary tokens only (struct/component/builtin/lit/byte - never NL or the dynamic <SYM_i>/<BIND_j>/<STATE_k> pools), so macros are alpha-independent by construction. data/macro_induction.py mines the table offline via greedy iterative MDL (net_gain = freq*(len-1) - len) over the canonicalized corpus; expansion at decode is deterministic and lossless, and the table persists in the tokenizer sidecar so train and decode cannot disagree. Fail-closed: spans with dynamic tokens are rejected, orphaned macro ids decode to nothing. Wiring: macro_tokens flag through TwoTowerConfig / ModelBuildConfig / matrix Experiment (default off); new macro_substitution diffusion policy masks whole macro blocks; grammar-gate surface functions treat macro ids as opaque (pre-empting the raw-sentinel-in-Lark-prefix bug class from E257). Fixture row E259 (v12): 16 macros mined on the 108-record corpus, -34.3% corpus tokens incl. table, seen_target_tokens -34.4% at the matched recipe; all honest gates still fail at this budget (wiring evidence only, no matched in-run control, nothing promoted). Evidence in docs/design/iter-e259-c3-macro-tokens-20260717.md + suffixed results mirror; research-lineage upgrades Stitch/LILO to Adapted (compression objective only). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

…28, E260/E261) "When Names Disappear" (arXiv:2510.03178) shows identifier anonymization degrades code models; C1-C3 all build on the defense that OpenUI identifiers carry no semantics. C4 tests that defense head-on with one lever: symbol_anonymization=False routes binder/state names through the existing byte channel verbatim (placeholders keep <SYM_i> in both arms, isolating exactly the identifier lever). Round trip is exact without a symbol table (108/108 fixture corpus records; 1.72x token cost). Both arms decode unconstrained via a new per-experiment grammar_constrained knob - the NAME gate admits only <BIND_j> ids, so constrained decode would confound representation with decode legality. Fail-closed guards refuse surface mode combined with grammar_constrained, macro_tokens, or relative bind encoding. Fixture pair E260/E261 (v13, 80 CPU steps): syntax/meaningful parse 0.0 on both arms (fixture wall); structural similarity favors the surface arm on 5/5 suites despite 1.72x longer targets. Verdict recorded as OPEN - a small adverse data point for the anonymization defense, not a result; decisive test needs a frontier-scale replicated pair. No gate weakened; nothing promoted. Evidence in docs/design/iter-e260-e261-c4-names-disappear-20260717.md + suffixed results mirror; lineage row updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

"Diffusion On Syntax Trees" (Kapur/Jenner/Russell, NeurIPS 2024) is the program's closest prior art but was only Adapted - the X-series borrowed the typed-topology state idea without the paper's all-valid forward process or search. models/tree_edit_diffusion.py now reproduces the mechanism faithfully: forward noise = chains of validity-preserving component replace/add/remove edits (every training state parses), the policy net is supervised on the inverse edit of the chain, and decode is value-guided beam search from a minimal valid seed so every emitted program is valid by construction. Observation channel is prompt conditioning instead of rendered-image feedback (stated boundary - no target render exists at generation time); lineage tag moves to Faithful (mechanism). Matched 80-step CPU fixture screening (X22 vs X9, seed 0, unchanged gates): X22 posts the program's first nonzero meaningful parse at this budget - 0.2 held_out / 0.25 adversarial / 0.25 ood (those two gate rows pass outright) with syntax parse 1.0 everywhere by construction; matched X9 scores 0.0 on every signal. Full gate battery still fails on both rows (smoke + rico) - wiring evidence only, single seed, nothing promoted. Component inventory derives from the fixed grammar vocabulary; v0.5 builtin statements are never mutated (fail-closed skip). Evidence in docs/design/iter-x22-d3-kapur-tree-edit-20260717.md + suffixed grammar matrix mirror; MODEL_CARD row appended. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

…es (SLM-34) The grammar-backend registry already made parse/decode pluggable, but the oracle, canonicalizer, corpus generator, scope check, and placeholder policy were hard-wired to OpenUI at their call sites. dsl/packs/ lifts the full per-DSL bundle into one registrable contract: DSLPack {grammar backend id, canonicalize, canonical_equal, validity_oracle, corpus_generator, scope_check, placeholders}, with a registry mirroring grammar.backends conventions. grammar and validity_oracle are independent fields on purpose - the F4 ontology packs validate by consistency check, not CFG parse. Builtin instances: openui (codec round-trip canonicalizer, validate_output oracle, progspec coverage-guided generator) and toy-layout (identity canonicalizer + template generator, stated in notes) - the second instance exists so anything OpenUI-shaped creeping into the contract breaks its tests first. Two behavior-preserving seams: fastpath engine_for_dsl resolves any registered backend's grammar_path instead of a hard-coded path table, and data/verify _grammar_backend accepts a backend id (default unchanged; the G0-G12 semantic stack remains OpenUI-only and is documented as such). Contract-invariant tests cover both packs (deterministic generation, oracle validity, scope-clean, idempotent canonical form) plus the end-to-end pack fixture loop: generate -> scratch-train -> decode -> oracle re-validation. Docs: grammar-backends.md packs section; design-patterns-dsl.md F1 mapping updated to point at the landed contract. Unblocks F2/F3/F4/G3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

harnesses/experiments/recipe_evolution.py: AlphaEvolve/ShinkaEvolve-Adapted population search over training recipes - only the population/evaluator pattern transfers (no LLM-guided program mutation; the evaluator is the existing frozen train/eval/gate stack). Genes are a bounded typed space over corruption schedule, decode policy, loss-weight, and LR knobs; model geometry is deliberately not a gene. Seeded one-gene mutation + uniform crossover; each unique recipe is evaluated exactly once (the autoresearch no-repeated-knob-signature contract - elites are cached, never re-trained). Selection cannot be weakened by construction: rank_candidates has no threshold knob, gate-passers under the unaltered DEFAULT_SHIP_GATES strictly outrank every gate-failing candidate, missing suites fail closed, and promotable is only true when the best candidate actually passed the frozen gates. No RL path exists; the module pins any future RL leg behind rl_gate.assert_rl_ready. Campaign state persists as an atomic local JSON tree (campaign.json + per-generation artifacts + population.json) compatible with the optional dry-by-default HF sync_campaign mirror. CLI scripts/run_recipe_evolution.py (with --dry-run wiring mode); 8 pure-Python tests covering gene bounds/determinism, ModelBuildConfig mapping, gate-first ranking, fail-closed gate checks, campaign persistence, and promotable-only-on-gate-pass. research-lineage row upgraded to Adapted for AlphaEvolve/ShinkaEvolve with the boundary stated. Fixture campaign evidence lands in a follow-up commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

Campaign g2_fixture_20260717: 4 unique recipes, 2 generations, 20 CPU steps each on the fixture v1 corpus. The evolve -> scratch-train -> eval -> gate-checked-select loop ran end-to-end with zero candidate errors and reported promotable: false honestly (no candidate passed the frozen gates; partial fitness suites fail closed on missing_suite by construction). Two recorded observations: (1) the gate lock is load-bearing immediately - weighted-NLL fitness monotonically rewards driving fidelity_loss_weight toward zero (16.75 -> 16.65 -> 16.41 across fid 0.76/0.5/0.04), exactly the emptier-but-cheaper failure mode the frozen placeholder-fidelity floors block, so selection must stay gate-first; (2) decode-only genes are NLL-fitness-neutral, so campaigns evolving decode policy must select on scoreboard metrics. Wiring evidence only - no gate weakened, nothing promoted, no checkpoint kept. Evidence: iter doc + campaign JSON mirror; MODEL_CARD history row appended. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

First checkable-answer machinery in the repo: the arith-sketch DSL (dsl/grammars/arith_sketch.lark, fifth grammar backend via the generic LarkFileBackend, third F1 pack) carries straight-line reasoning traces whose final root binding is the answer. The pack's validity oracle IS the deterministic evaluator - the transformer's resolve pass expands bound refs deterministically and evaluate_answer computes the number, so validity and answer scoring share one code path (fail-closed: invalid trace == wrong; undefined refs, cycles, division by zero, missing root all reject). New LarkFileBackend.resolved_data seam exposes the resolved bindings for value-semantics packs whose root is not an element dict. harnesses/reasoning/bench.py + scripts/run_reasoning_bench.py: two matched arms on the same tiny scratch TwoTower (compositional corpus-derived tokenizer, DSL-agnostic), same generated word-problem corpus, same oracle - sketch arm (PAL/PoT-Adapted: reason-in-formal-language + deterministic execution on a trained tiny model, not a prompted frozen LLM) vs a direct-answer control. Sketch-of-Thought positioning and the unconstrained-decode boundary are stated in docs/design/reasoning-sketch-harness.md. Fixture run g4_fixture_20260717 (96 train / 24 held-out, 120 CPU steps per arm): both arms 0.0 accuracy - the loop is proven, the PAL/PoT-analog question stays open at this budget. Recorded failure modes: sketch emits forward refs and missing root bindings (exactly what constrained decode fixes, cf. X22); direct collapses to a constant. Lineage: PAL/PoT upgraded to Adapted with the boundary stated; SoT remains Adjacent. Wiring evidence only; nothing promoted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

… (SLM-37) harnesses/distill/meta_trace.py formalizes trace capture for the future DSL-generating meta-model as schema + retention over artifacts the stack already writes (no new infrastructure, per the issue scope). MetaTraceRecord (strict pydantic, schema_version 1) joins per example: the request (prompt, slot contract), the decode spec (model kind, checkpoint sha, config, seed, deterministic flag), the emitted program, per-example eval metrics, and the run's honest-gate verdicts - with W3C trace ids and source-artifact provenance, plus a first-class dsl_id (the gap in the existing distill TraceStore). harvest_run_dir joins eval_<suite>.json details, matrix_result/scoreboard gates, the training recipe, and the trace.json sidecar, degrading gracefully when artifacts are absent. write_corpus persists an append-only traces.jsonl with sha256-per-line manifest under a campaign.json-gated tree (campaign-store conventions, sync_campaign-mirrorable, local authoritative). replay_trace re-decodes from spec, fail-closed on checkpoint-sha mismatch; bit-exact replay is asserted only for the deterministic tree-edit decoder (MaskGIT paths are replay-from-spec - stated boundary). Fixture evidence: 76 records harvested from four real run dirs (E259/E260/E261/X22-reeval) with trace-id joins intact; the replay test reproduces a stored tree-edit decode bit-exactly through from_checkpoint and rejects tampered shas and non-deterministic kinds. Schema documented in docs/design/telemetry.md; iter doc + MODEL_CARD row appended. Noted follow-ups: grammar matrix emits no trace bundles yet; per-step trajectories only via collect_trajectories. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

runtime_symbol_features='replace' (DyVo-style, Adapted): the per-example delta cancels the learned <SYM_i>/<BIND_j> pool row so the tied input embedding AND output projection become the deterministic byte-compositional vector of the symbol's referent surface. No per-example parameters, weight tying and batching untouched; same surface -> identical vector at every slot/position (exact-substitution and cross-slot identity tests). evals/binding_consistency.py: the SLM-26 metric — same-surface vs cross-surface hidden-state cosine at symbol positions (binding margin). E260 (v13, matched vs E255 on everything but the mode, 200 CPU steps): honest fixture negative — structural similarity 0.19-0.29 vs control 0.28-0.37, gates fail on both as always at fixture scale. Probe on the trained checkpoint: same-surface cos 0.9998 vs cross 0.9679 (margin +0.032). The first run flushed a latent stale-feature leak (features set for a training batch crashing a differently-sized eval forward — the defect class PR #275 diagnosed); fixed at the source: training_loss clears request-local features in a finally block. Docs: iter doc, V13 matrix section, results + probe JSON snapshots, MODEL_CARD history, DyVo lineage Adjacent -> Adapted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132PYCBHb1t1pjhDRzLrPov

Second real pack under the F1 contract and the first schema-native backend. GraphQLQueryBackend implements the GrammarBackend Protocol directly on graphql-core (parse / print_ast / validate) rather than a .lark grammar: component_names() returns the schema's type names and library_schema() the per-type field map, so the introspection schema literally is the symbol table - the strongest C1/C2 scope substrate in the program (OpenUI 0.2.x has almost no binding surface). The pack's validity_oracle runs graphql.validate against a committed SDL fixture (resources/graphql/shop_schema.graphql), so "valid" means schema-correct - querying an unknown field or passing a wrong argument name is rejected, not just malformed syntax. The canonicalizer is a real normal form (print_ast(parse(x)), idempotent) that collapses whitespace variants, unlike the toy/arith identity canonicalizers. The corpus generator emits schema-valid queries over 5 templates, each self-checked through the oracle. graphql-core is an optional dependency (the graphql extra in pyproject); the backend's available() gates all F2 tests the way bridge_available() gates the OpenUI bridge tests, so a venv without the extra skips cleanly. graphql-js byte parity is a stated non-goal. Query subset only (no mutations/fragments/directives); the quality-matrix rows and a trained GraphQL model are follow-ups that ride the F1 interface with no new plumbing. Also fixes a test regression from the D3 commit: the scope-diffusion source manifest test now expects the "Faithful (mechanism)" Kapur status alongside Adapted/Adjacent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

Program experiments run through the existing hypothesizer machinery instead of a parallel ad-hoc loop. autoresearch/program_matrices.py encodes the valid-but-empty-wall attack (Track A) as track_a_matrix(): five authentic ExperimentSpecs with distinct knob signatures - A3 coverage-energy remasking, A4 minimum-content decode contracts (auto + fixed floor), A3+A4 combined, and A5 lattice search - each grounded in a real EvidenceSnapshot over the committed E248/E250/E251/lattice iter docs (real sha256/size). Closes the routing gap the encoding exposed: remask_policy and decode_min_content were real model levers but had no typed ExperimentKnobs field, allowlist entry, compile_commands branch, or evaluate_model CLI flag, so the engine could not route them - the five candidates would only have differed by incidental seed/steps. This adds those two knobs end to end (schema + allowlist + decode-stage compile branch + --remask-policy / --decode-min-content flags). A2 (ASAp) is deliberately NOT fabricated as a routable knob since no model-side lever exists; it stays a future matrix row. The matrix passes validate_hypothesis_matrix, compiles to bounded CPU argv via compile_commands (emptiness knobs verified to reach the eval command), and closes the loop through create_hypothesis_feedback. Full autoresearch suite green (49 tests) including the frozen hypothesizer benchmark, which G1 does not touch. Wiring evidence only - the matrix is a reviewable plan, not an executed campaign; nothing trained or promoted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

Docstring every public builder and helper in program_matrices.py, addressing the CodeRabbit docstring-coverage warning on the new module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

main advanced through B1/B3 (#288/#292) and the counterfactual-probe series, which claimed E259/E260/E261 (expanded/gold-AST probes) and E262 (B1 choice codec, matrix v12). This branch's C3/C4 rows had independently used those same IDs, so they are renumbered to keep the research-program experiment ledger globally unique: C3 macro tokens: E259 -> E265 (matrix v12 -> v13) C4 anon control: E260 -> E266 (matrix v13 -> v14) C4 surface ids: E261 -> E267 Renumber applied consistently across run_quality_matrix.py (both new _v13/_v14 experiment functions kept beside main's _v12 E262), the matrix-doc ledger (my C3/C4 sections now follow main's counterfactual + B1 + E263/E264 narrative), the two iter docs and result JSONs (git mv to E265/E266/E267 and v13/v14 filenames), MODEL_CARD, research-lineage, and the G5 provenance refs. Conflict in data/verify/stack.py resolved to main's typed registry-routing _grammar_backend (subsumes the F1 pack seam). Full check-changed green (846 passed); matrix --list confirms E262 + E265/E266/E267 all dispatch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

main's FTPO preference-optimizer campaign (#294-#305) claimed E265-E276, colliding with the earlier C3/C4 renumber. Move C3/C4 clear of main's range: C3 macro tokens: E265 -> E277 (matrix v13) C4 anon control: E266 -> E278 (matrix v14) C4 surface ids: E267 -> E279 Applied across run_quality_matrix.py, the matrix-doc ledger, both iter docs (git mv), result JSONs, MODEL_CARD, research-lineage, and G5 provenance refs. v13/v14 matrix functions are unchanged (main did not add them). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

…slm-autoresearch-05w3zn # Conflicts: # docs/design/quality-experiment-matrix.md # docs/design/research-lineage.md

…verlap PR #285 ("DSL Diffusion Program Wave 2") independently reimplemented F2 (GraphQL), G1 (autoresearch routing), G5 (meta-traces), and C2 with different designs, and merged to main first. To avoid duplicate/divergent code this branch now defers to main's canonical versions of those pieces and keeps only its genuinely-unique work: C3 macro tokens, C4 names-disappear, D3 Kapur tree-edit, F1 pack contract (dsl/packs/), G2 recipe evolution, G4 sketch-of-thought reasoning, and the two runtime-symbol-feature defect fixes. Reconciliation: - Dropped my duplicate F2/G1/G5 files (graphql_query.py, dsl/packs/graphql.py, program_matrices.py, meta_trace.py + tests + iter docs); took main's graphql_js.py, program_matrix.py, trace_store.py and test_graphql_pack.py. - Took main's canonical schemas.py/engine.py/evaluate_model.py (my only edits there were G1, now superseded) and the F2/G1/G5 design docs. - Removed my orphaned graphql-core extra from pyproject. - Renumbered my C3/C4 clear of #285's E277(A2)/E278(C2): C3 E280 (matrix v16), C4 E281/E282 (matrix v17); re-added both as fresh version functions on main's run_quality_matrix.py. Backend registry keeps my ArithSketch (G4) beside main's GraphQLJs. - Merged the overlapping research-lineage rows (C2+C3, A2+C4). check-changed: 1015 passed, 9 skipped. Remaining failures are pre-existing on main and unrelated to this changeset: 2 gpu_multi_farm (missing fastmcp dep) and 2 quality_matrix_v10 (main's _v10 emits E277/E278 but #285's test still expects E263/E264 — byte-identical to origin/main, fails there too). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014AWAXqSioAB3sYXKxgJ3rG

…slm-autoresearch-05w3zn # Conflicts: # docs/design/research-lineage.md # scripts/run_quality_matrix.py # src/slm_training/dsl/grammar/backends/__init__.py

vercel Bot deployed to Preview July 17, 2026 00:52 View deployment

vercel Bot deployed to Preview July 17, 2026 01:22 View deployment

Tyler-R-Kendrick changed the title ~~B4: AR→diffusion adaptation baseline — denoiser_backend=hf + E260 fixture row (SLM-24)~~ B4 + C1: AR→diffusion adaptation baseline (E260) and relative-index scope references (E261) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 01:45 View deployment

vercel Bot deployed to Preview July 17, 2026 01:48 View deployment

Tyler-R-Kendrick force-pushed the claude/dsl-diffusion-slm-autoresearch-05w3zn branch from 9e86c13 to 9979208 Compare July 17, 2026 02:27

Tyler-R-Kendrick changed the title ~~B4 + C1: AR→diffusion adaptation baseline (E260) and relative-index scope references (E261)~~ Fix two latent runtime-symbol-feature defects (salvaged from superseded B4/C1/C2 branch) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 02:27 View deployment

Tyler-R-Kendrick changed the title ~~Fix two latent runtime-symbol-feature defects (salvaged from superseded B4/C1/C2 branch)~~ Fix latent V8 runtime-symbol defects + C3 corpus-mined macro tokens (SLM-27, E259) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 02:57 View deployment

Tyler-R-Kendrick changed the title ~~Fix latent V8 runtime-symbol defects + C3 corpus-mined macro tokens (SLM-27, E259)~~ V8 runtime-symbol defect fixes + C3 macro tokens (SLM-27) + C4 names-disappear pair (SLM-28) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 03:17 View deployment

Tyler-R-Kendrick changed the title ~~V8 runtime-symbol defect fixes + C3 macro tokens (SLM-27) + C4 names-disappear pair (SLM-28)~~ V8 defect fixes + C3 macro tokens + C4 names-disappear pair + D3 Kapur baseline (SLM-27/28/31) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 03:56 View deployment

Tyler-R-Kendrick changed the title ~~V8 defect fixes + C3 macro tokens + C4 names-disappear pair + D3 Kapur baseline (SLM-27/28/31)~~ V8 defect fixes + C3 macros + C4 names pair + D3 Kapur baseline + F1 pack contract (SLM-27/28/31/34) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 04:24 View deployment

Tyler-R-Kendrick changed the title ~~V8 defect fixes + C3 macros + C4 names pair + D3 Kapur baseline + F1 pack contract (SLM-27/28/31/34)~~ V8 fixes + C3 macros + C4 names pair + D3 Kapur baseline + F1 packs + G2 evolution (SLM-27/28/31/34/35) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 04:26 View deployment

vercel Bot deployed to Preview July 17, 2026 04:42 View deployment

Tyler-R-Kendrick changed the title ~~V8 fixes + C3 macros + C4 names pair + D3 Kapur baseline + F1 packs + G2 evolution (SLM-27/28/31/34/35)~~ V8 fixes + C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning (SLM-27/28/31/34/35/36) Jul 17, 2026

vercel Bot deployed to Preview July 17, 2026 04:53 View deployment

Tyler-R-Kendrick mentioned this pull request Jul 17, 2026

DSL Diffusion Program — Wave 2: A2, F1/F2, G1/G3/G5, B1/B3, C2 (9 issues) #285

Merged

vercel Bot deployed to Preview July 17, 2026 05:20 View deployment

vercel Bot deployed to Preview July 17, 2026 05:36 View deployment

vercel Bot deployed to Preview July 17, 2026 05:46 View deployment

vercel Bot deployed to Preview July 17, 2026 06:05 View deployment

claude added 2 commits July 17, 2026 11:24

Merge remote-tracking branch 'origin/main' into claude/dsl-diffusion-…

46f545f

…slm-autoresearch-05w3zn # Conflicts: # docs/design/quality-experiment-matrix.md # docs/design/research-lineage.md

vercel Bot deployed to Preview July 17, 2026 11:28 View deployment

claude added 2 commits July 17, 2026 12:17

Merge remote-tracking branch 'origin/main' into claude/dsl-diffusion-…

67a517d

…slm-autoresearch-05w3zn # Conflicts: # docs/design/research-lineage.md # scripts/run_quality_matrix.py # src/slm_training/dsl/grammar/backends/__init__.py

vercel Bot deployed to Preview July 17, 2026 12:21 View deployment

Tyler-R-Kendrick changed the title ~~V8 fixes + C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning (SLM-27/28/31/34/35/36)~~ C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning + V8 fixes (SLM-27/28/31/34/35/36) Jul 17, 2026

Tyler-R-Kendrick merged commit 13da79e into main Jul 17, 2026
3 of 4 checks passed

Tyler-R-Kendrick deleted the claude/dsl-diffusion-slm-autoresearch-05w3zn branch July 17, 2026 12:21

This was referenced Jul 18, 2026

Integrate exact closure into compiler decode behind a flag (SLM-63) #340

Merged

Persist and replay solver deductions, decisions, and metrics (SLM-64) #341

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning + V8 fixes (SLM-27/28/31/34/35/36)#275

C3/C4 tokenizer levers + D3 Kapur + F1 packs + G2 evolution + G4 reasoning + V8 fixes (SLM-27/28/31/34/35/36)#275
Tyler-R-Kendrick merged 17 commits into
mainfrom
claude/dsl-diffusion-slm-autoresearch-05w3zn

Tyler-R-Kendrick commented Jul 17, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jul 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (2 warnings)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Tyler-R-Kendrick commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

1. Two latent runtime-symbol-feature defect fixes (V8/C2 code on main)

2. C3 — corpus-mined macro tokens (SLM-27, E280)

3. C4 — names-disappear matched pair (SLM-28, E281/E282)

4. D3 — faithful Kapur tree-edit diffusion baseline (SLM-31, X22)

5. F1 — DSL pack contract (SLM-34)

6. G2 — recipe-evolution harness (SLM-35)

7. G4 — sketch-of-thought reasoning bench (SLM-36)

Deferred to main's #285 (dropped here to avoid duplication)

Verification

Uh oh!

vercel Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (2 warnings)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Tyler-R-Kendrick commented Jul 17, 2026 •

edited

Loading

vercel Bot commented Jul 17, 2026 •

edited

Loading

coderabbitai Bot commented Jul 17, 2026 •

edited

Loading