From 86fec0be76a4e5aefc6a1a603ccf8ce134aad64e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 17 Jul 2026 23:14:09 +0000 Subject: [PATCH 1/2] Commit LDI local-decision architecture contract and expand source inventory (SLM-114) Make the local-decision-intervention (LDI) research synthesis and architectural boundaries canonical so future agents cannot recreate a parallel harness, repeat the falsified E249-E284 chain, or treat a local-metric gain as promotion evidence. - local-decision-interventions.md: add the LDI0-01 architecture contract (evidence/objective/actuator/experiment/promotion separation with named owners, six invariants, forbids a second stack, DecisionEventV2 as the next contract) and a measured-chain / current-blocker summary (blocker: stable state support does not imply objective/action-partition support) - local-decision-sources.json: add 9 arXiv works (MGDA, PCGrad, LoRA, PICARD, AdaLoRA, DeepSeekMath/GRPO, GAD/ASAp, min-p, TAB-PO) with metadata verified against arXiv; 33->42 rows, 25->34 papers, unique ids/URIs - research-lineage.md: extend the exact-state local-decision entry with the new sources under the existing Faithful/Adapted/Surrogate/Adjacent vocabulary - quality-experiment-matrix.md: add the LDI campaign index (namespace pointer; claims no unrun row, mints no E ID, records next-free >= E292) - test_harness.py: update+extend the manifest test (counts, source-id uniqueness, required metadata, unique canonical URIs, implementation-status vocabulary) - add iter-ldi0-01-source-expansion-20260717.md evidence memo Documentation/inventory only: no code, model, checkpoint, adapter, or model-card quality change; no ship gate weakened. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01KVT9Qe5YUhgZqvaHLCAYSS --- .../iter-ldi0-01-source-expansion-20260717.md | 74 +++++++++++++++ docs/design/local-decision-interventions.md | 94 +++++++++++++++++-- docs/design/quality-experiment-matrix.md | 31 +++++- docs/design/research-lineage.md | 25 +++-- .../autoresearch/local-decision-sources.json | 83 +++++++++++++++- tests/test_autoresearch/test_harness.py | 39 ++++++-- 6 files changed, 321 insertions(+), 25 deletions(-) create mode 100644 docs/design/iter-ldi0-01-source-expansion-20260717.md diff --git a/docs/design/iter-ldi0-01-source-expansion-20260717.md b/docs/design/iter-ldi0-01-source-expansion-20260717.md new file mode 100644 index 000000000..d79f01235 --- /dev/null +++ b/docs/design/iter-ldi0-01-source-expansion-20260717.md @@ -0,0 +1,74 @@ +# Iter LDI0-01 — local-decision architecture contract + source expansion (2026-07-17) + +**Issue:** SLM-114 (LDI0-01). **Type:** documentation / research-inventory / test +contract. **No code, model, checkpoint, adapter, or model-card quality change.** + +**Reviewed base commit:** `c7cb099` (`origin/main`, "Cache exact choice completion +states (#315)"). + +## What changed + +Made the local-decision-intervention (LDI) research synthesis and architectural +boundaries canonical in the repository so future agents cannot recreate a parallel +harness, repeat the falsified E249-E284 chain, or treat a local-metric gain as +promotion evidence. + +- [`local-decision-interventions.md`](local-decision-interventions.md) — added the + **Architecture contract (LDI0-01)** (evidence / objective / actuator / experiment / + promotion separation with named owners; six invariants; forbids a second stack; + `DecisionEventV2` as the next contract) and a **Measured chain and current + blocker** section (E248-E286 negative chain; blocker = *stable state support does + not imply objective/action-partition support*). +- [`local-decision-sources.json`](../../src/slm_training/resources/autoresearch/local-decision-sources.json) + — added nine academic works (see table). +- [`research-lineage.md`](research-lineage.md) — extended the *Exact-state local + decision preference* entry with the new sources under the existing + Faithful/Adapted/Surrogate/Adjacent vocabulary; no duplicate entries. +- [`quality-experiment-matrix.md`](quality-experiment-matrix.md) — added the **LDI + campaign index** (namespace pointer only; claims no unrun row and mints no E ID; + records the E-ID allocation rule with next-free `≥ E292`). +- `tests/test_autoresearch/test_harness.py` — updated and extended + `test_local_decision_source_manifest_is_complete` (counts, source-ID uniqueness, + required metadata, unique canonical URIs, implementation-status vocabulary, the + nine additions). + +## Source count before / after + +| | Rows | Academic (arXiv) | Implementation | +| --- | --- | --- | --- | +| Before | 33 | 25 | 8 | +| After | 42 | 34 | 8 | + +## Duplicate / alternate-URL handling + +Each addition is a distinct canonical arXiv `abs/` URI with a unique `source_id`; +none duplicates an existing paper or an alternate URL. The pre-existing alternate +URLs (DeepSeek-R1 Nature DOI, two OpenReview forum links) remain in `metadata` +rather than as separate rows. The manifest test asserts +`len({source_id}) == len({uri}) == len(rows)`. + +## Classification (nine additions) + +Metadata verified against arXiv on 2026-07-17. `implementation_status` follows the +research-lineage vocabulary (all additions are Adjacent or Adapted; none is +implemented by this contract). + +| arXiv | Work | Category | Status | LDI relevance | +| --- | --- | --- | --- | --- | +| [1810.04650](https://arxiv.org/abs/1810.04650) | MGDA (MTL as multi-objective opt.) | objective_geometry | Adjacent | Frames the objective/action-partition blocker | +| [2001.06782](https://arxiv.org/abs/2001.06782) | Gradient Surgery / PCGrad | objective_geometry | Adjacent | Diagnostic lens for the same blocker | +| [2106.09685](https://arxiv.org/abs/2106.09685) | LoRA | adapter_actuator | Adjacent | Removable causal actuator (later), not the event schema | +| [2109.05093](https://arxiv.org/abs/2109.05093) | PICARD | constrained_decoding | Adjacent | Prior art: hard constraints stay deployed | +| [2303.10512](https://arxiv.org/abs/2303.10512) | AdaLoRA | adapter_actuator | Adjacent | Deferred adaptive-rank actuator | +| [2402.03300](https://arxiv.org/abs/2402.03300) | DeepSeekMath / GRPO | verifiable_training | Adjacent | Deferred RLVR behind the readiness gate | +| [2405.21047](https://arxiv.org/abs/2405.21047) | Grammar-Aligned Decoding / ASAp | constrained_decoding | Adjacent | Legality ≠ preference (invariant 1) | +| [2407.01082](https://arxiv.org/abs/2407.01082) | Min-p sampling | decoding_sampling | Adjacent | Decoding-time baseline to beat | +| [2603.00025](https://arxiv.org/abs/2603.00025) | TAB-PO | local_preference | Adapted | Token-critical PO informs DecisionEventV2 weighting | + +## Honesty boundary + +Specification / inventory only. No experiment, train, eval, benchmark, checkpoint, +adapter, or model-card quality update. No paper result is represented as a +repository result; every source carries a lineage label, and constraint shadows +certify decoder legality only — never a semantic preference label. No ship gate is +weakened and no readiness or promotion claim is made. diff --git a/docs/design/local-decision-interventions.md b/docs/design/local-decision-interventions.md index 1829d73e6..055f9e05d 100644 --- a/docs/design/local-decision-interventions.md +++ b/docs/design/local-decision-interventions.md @@ -1,8 +1,11 @@ # Local decision interventions for TwoTower -**Status:** research intake plus measured E248/E249 campaign. E249 is rejected: -constraint-shadow ranking generalized locally but regressed semantic quality on -every suite. This document contains no model-quality or ship claim. +**Status:** LDI0-01 architecture/research contract plus the measured E248-E286 +local-decision campaign. The chain is negative: E249 is rejected (constraint-shadow +ranking generalized locally but regressed semantic quality on every suite), and no +local-preference intervention has cleared the unchanged ship gates. This document is +the canonical repository owner of the local-decision-intervention synthesis and +architectural boundaries; it contains no model-quality or ship claim. ## Source and audit @@ -12,9 +15,12 @@ The normal page reader returned only the application shell, so the server-render conversation payload was decoded and its citation records were normalized against primary arXiv and OpenReview pages. The reviewed inventory is committed as [`local-decision-sources.json`](../../src/slm_training/resources/autoresearch/local-decision-sources.json): -25 distinct academic works and eight implementation/documentation sources. The +**34 distinct academic works and eight implementation/documentation sources** — the +original 25 works plus nine added for this LDI0-01 contract (multi-objective / +gradient-conflict optimization, PEFT actuators, constrained decoding, and +token-critical preference lineage), each verified against arXiv on 2026-07-17. The DeepSeek-R1 Nature DOI and the two OpenReview URLs are retained as alternate URLs -rather than double-counted papers. +rather than double-counted papers, and no paper is duplicated under an alternate URL. | Cluster | Sources | Relevance here | | --- | --- | --- | @@ -32,6 +38,48 @@ Auto-Antislop was reviewed at `8fb98fdf019e6fcc20164f9bdec41f9008fcd632`; no license file was present, so its source is not copied. Antidoom was reviewed at `bd6a126476e18554b0cacaea3fd9f258fdde1f97` under Apache-2.0. +## Architecture contract (LDI0-01) + +This section is the canonical architectural boundary for the local-decision program. +It exists so future agents do not recreate a parallel harness, repeat the falsified +E249-E284 chain, or treat a local-metric gain as promotion evidence. The Linear +project document is planning context; this file plus the sources manifest are the +source of truth. + +**Separation of concerns (do not collapse).** Evidence, objective, actuator, +experiment, and promotion are distinct concerns with distinct owners — extend the +owner, never fork a second stack: + +| Concern | Owner (do not fork) | Rule | +| --- | --- | --- | +| Exact-state decision evidence | [`harnesses/preference/local_decisions.py`](../../src/slm_training/harnesses/preference/local_decisions.py) | Events carry exact masked-token state identity. A constraint shadow certifies decoder legality only. | +| Observation / replay | [`harnesses/distill/trace_store.py`](../../src/slm_training/harnesses/distill/trace_store.py) | The append-only decode trace is the observation/replay owner; it is not a trainer. | +| Objective | [`harnesses/preference/local_train.py`](../../src/slm_training/harnesses/preference/local_train.py) | Clipped-margin FTPO and preference losses. The objective is separate from the event schema. | +| Actuator | existing trainable parameters today; later LoRA / DoRA / PiSSA / TwoTower delta / ReFT / SAE | Adapter and representation form are actuator choices, never event schemas. | +| Experiment | [`scripts/run_quality_matrix.py`](../../scripts/run_quality_matrix.py) + [`quality-experiment-matrix.md`](quality-experiment-matrix.md) | The quality matrix / autoresearch is the bounded experiment owner. | +| Promotion | ship gates + [`docs/MODEL_CARD.md`](MODEL_CARD.md) | Only the unchanged five-suite scoreboard and ship gates promote. A local metric never does. | + +**Invariants:** + +1. Constraint shadows certify decoder legality only. They never become semantic + preference labels without same-state counterfactual verification. +2. Hard grammar/compiler constraints remain deployed. Interventions reduce error + mass and improve semantics *on top of* the deterministic guarantee; they do not + replace it. +3. The first new evidence contract is `DecisionEventV2` — an action-table extension + of `DecisionEventV1` with stable state identity and per-action verdicts (LDI0-02). + The first actuator experiments are causal PEFT and a removable TwoTower delta; SAE + work stays behind matched direct-supervision baselines. +4. New experiments use the `LDI` campaign name in prose and config but obtain + globally unique E IDs from the existing allocation process. Do not reserve or + assume the next E number (see the E-ID rule in + [`quality-experiment-matrix.md`](quality-experiment-matrix.md)). +5. Do not create a second orchestration or training stack. Extend the owners above. +6. No paper result is represented as a repository result. Every source is labeled + Faithful / Adapted / Surrogate / Adjacent in + [`research-lineage.md`](research-lineage.md) and carries an + `implementation_status` in the sources manifest. + ## Existing seam and missing evidence E228 already constructs a compiler decision canvas, restricts logits to legal @@ -68,6 +116,12 @@ Two evidence paths are allowed: Final-output failure alone never creates a token label. Splits group by prompt and record family so related counterfactuals cannot cross train/held-out boundaries. +**Next contract (LDI0-02):** `DecisionEventV2` adds stable state identity and a +per-action verdict table so multiple good/bad actions at one state each carry an +explicit counterfactual verdict, targeting the objective/action-partition blocker +below. It extends `DecisionEventV1`; it does not replace the trace store or the +objective, and it is specified — not implemented — by this contract. + ## Objectives and locality For good action `g`, bad action `b`, and decision logits `z`, define @@ -115,8 +169,9 @@ full-vocabulary drift. End-to-end authority remains the unchanged five-suite scoreboard and ship gates. Falsify a row if it does not improve held-out event metrics, exceeds the matched -reference-drift budget, or regresses any protected ship gate. No V10 intervention -row has run; there is no intervention checkpoint, model-card update, or promotion. +reference-drift budget, or regresses any protected ship gate. No V10/LDI +intervention row has cleared the unchanged ship gates, and there is no intervention +checkpoint, model-card update, or promotion. ## Measured event-mining prerequisite @@ -134,3 +189,28 @@ suite. Do not run E250/E251 on this corpus as quality labels. E252-E254 remain fail-closed because the corpus contains no counterfactual or set-valued evidence. Measured result: [`iter-e249-local-ce-margin-20260716.md`](iter-e249-local-ce-margin-20260716.md). + +## Measured chain and current blocker (E248-E286) + +E248-E286 is authoritative and lives in +[`quality-experiment-matrix.md`](quality-experiment-matrix.md) (the V10 rows +E248-E254 and the local-preference ledger through E286) and the per-run +`iter-e2*.md` iteration docs. The chain is negative: + +- **E248** — matched parent control, eval-only. +- **E249** — exact-event CE plus margin. The lexical decision objective generalized + (held-out chosen-win 0.7649) but semantic structure and reward regressed on every + suite; **rejected**. +- **E250-E284** — the registered bad-token, single-pair FTPO, verifier-backed set + FTPO, frozen-reference tether, and balanced-sampling levers (and the E265-E286 + local-preference ledger) were measured; none cleared the unchanged five-suite ship + gates or was promoted. + +**Current blocker: stable state support does not imply objective/action-partition +support.** Exact-state event identity — a stable, replayable decision state — is +necessary but not sufficient. It does not establish that the good/bad *action +partition* at that state is itself verifier-supported, so a locally-improved event +metric has repeatedly failed to transfer to semantic quality under the unchanged +gates. `DecisionEventV2` counterfactual action-verdict tables (LDI0-02) target +exactly this gap. No LDI intervention row is promoted; there is no intervention +checkpoint and no model-card quality update. diff --git a/docs/design/quality-experiment-matrix.md b/docs/design/quality-experiment-matrix.md index f613615ed..2de9aa3f5 100644 --- a/docs/design/quality-experiment-matrix.md +++ b/docs/design/quality-experiment-matrix.md @@ -1116,11 +1116,36 @@ full suites) is still required, and E241/E242's conflict machinery has not been exercised outside unit/integration tests because greedy decode never stalls on this checkpoint. +## LDI campaign index (local decision interventions) + +The **LDI** campaign is the local-decision-intervention line of work. Its canonical +architecture/research contract, invariants, named owners, and the 42-source manifest +are in [`local-decision-interventions.md`](local-decision-interventions.md). This +index is a namespace pointer, not a new set of rows: it claims **no** unrun row and +allocates **no** E ID. + +Measured record (authoritative): the **V10 exact-state local preference** rows +E248-E254 (below) and the measured **E265-E286 local-preference ledger** recorded in +this matrix and the per-run `iter-e2*.md` docs (broad/guarded FTPO, reference +tethers, balanced sampling). The chain is negative — E249 and E252 are rejected +(local metrics moved, semantic quality regressed), and no LDI intervention has +cleared the unchanged five-suite ship gates or been promoted. Current blocker: +**stable state support does not imply objective/action-partition support**; exact +state identity does not prove the good/bad action partition is verifier-supported. +`DecisionEventV2` action-verdict tables (LDI0-02) target this gap. + +**E-ID allocation rule.** New LDI experiments take a globally unique E ID from the +existing allocation process; the `LDI` name is prose/config only and reserves no ID. +As of 2026-07-17 the highest allocated ID is **E291** (B1/B3 tracks; see the +[`README.md`](../../README.md) run ledger), and E248-E291 plus the E263/E264 +local-preference rows are consumed. Do not assume "the next number after E286" is +free — the next free ID is **≥ E292**. + ## V10 exact-state local preference (E248 control measured) -The full 25-paper audit, source manifest, objective definition, and honesty boundary -are in [`local-decision-interventions.md`](local-decision-interventions.md). V10 -reuses the existing preference harness and append-only decode traces. It does not +The full source audit (34 works), source manifest, objective definition, and honesty +boundary are in [`local-decision-interventions.md`](local-decision-interventions.md). +V10 reuses the existing preference harness and append-only decode traces. It does not introduce an adapter/SAE trainer and does not claim that a local loss produces a local parameter update. diff --git a/docs/design/research-lineage.md b/docs/design/research-lineage.md index 032be3c4c..3e1bfedc9 100644 --- a/docs/design/research-lineage.md +++ b/docs/design/research-lineage.md @@ -166,15 +166,24 @@ of [XGrammar](https://arxiv.org/abs/2411.15100). Implementation and exact gates: | | | | --- | --- | -| **Papers** | [Unlikelihood](https://arxiv.org/abs/1908.04319), [Token-level DPO](https://arxiv.org/abs/2404.11999), [TIS-DPO](https://arxiv.org/abs/2410.04350), [ConfPO](https://arxiv.org/abs/2506.08712), [TGDPO](https://arxiv.org/abs/2506.14574), [Antislop](https://arxiv.org/abs/2510.15061), [TokenRatio](https://arxiv.org/abs/2605.12288) | +| **Papers** | [Unlikelihood](https://arxiv.org/abs/1908.04319), [Token-level DPO](https://arxiv.org/abs/2404.11999), [TIS-DPO](https://arxiv.org/abs/2410.04350), [ConfPO](https://arxiv.org/abs/2506.08712), [TGDPO](https://arxiv.org/abs/2506.14574), [Antislop](https://arxiv.org/abs/2510.15061), [TokenRatio](https://arxiv.org/abs/2605.12288), [TAB-PO](https://arxiv.org/abs/2603.00025) | | **Fidelity** | **Adapted** — exact masked-token states, verifier-backed good/bad action sets, clipped logit margins, and optional frozen-reference tethering; not sequence DPO or a faithful reproduction of any cited objective | -| **Code** | Existing preference/trace owners; detailed boundary and full 25-paper audit in [`local-decision-interventions.md`](local-decision-interventions.md) | -| **Matrix** | Proposed/unrun V10 rows E248-E254 in [`quality-experiment-matrix.md`](quality-experiment-matrix.md) | - -SAE/ReFT discovery, removable LoRA/DoRA/PiSSA actuators, adapter routing, iterative -remine, and RLVR are **Adjacent**. The first TwoTower implementation is a localized -loss with a global full-parameter update. It makes no locality or ship claim until -the registered controls run under unchanged gates. +| **Code** | Existing preference/trace owners; the LDI0-01 architecture contract, named owners, and full 34-source audit are in [`local-decision-interventions.md`](local-decision-interventions.md) | +| **Matrix** | Measured V10 rows E248-E254 and the E265-E286 local-preference ledger (LDI campaign index) in [`quality-experiment-matrix.md`](quality-experiment-matrix.md); the chain is negative (E249/E252 rejected) | + +SAE/ReFT discovery, removable LoRA/DoRA/PiSSA/[AdaLoRA](https://arxiv.org/abs/2303.10512) +actuators, adapter routing, iterative remine, and RLVR are **Adjacent**. +Objective-geometry lenses for the current blocker — +[Gradient Surgery/PCGrad](https://arxiv.org/abs/2001.06782) and +[MGDA](https://arxiv.org/abs/1810.04650) — and decoding baselines +([min-p](https://arxiv.org/abs/2407.01082)) are **Adjacent**: diagnostic framing and +baselines to beat, not implemented. [PICARD](https://arxiv.org/abs/2109.05093) and +[Grammar-Aligned Decoding/ASAp](https://arxiv.org/abs/2405.21047) frame the +constraint-legality invariant — constraint shadows certify legality only, never +semantic preference. These are lineage labels, not reproduced results. The first +TwoTower implementation is a localized loss with a global full-parameter update. It +makes no locality or ship claim until the registered controls run under unchanged +gates. ### GRPO-lite diff --git a/src/slm_training/resources/autoresearch/local-decision-sources.json b/src/slm_training/resources/autoresearch/local-decision-sources.json index a55ea02fb..dc59443b8 100644 --- a/src/slm_training/resources/autoresearch/local-decision-sources.json +++ b/src/slm_training/resources/autoresearch/local-decision-sources.json @@ -1,8 +1,17 @@ { "schema_version": 1, "retrieved_at": "2026-07-16T00:00:00Z", - "source_scope": "All 25 distinct academic works and 8 implementation/documentation sources cited by public ChatGPT share 6a593158-85c4-83ea-80b1-b6fb893b26bc. Canonical paper metadata was verified against arXiv or OpenReview; alternate URLs from the share are preserved in metadata.", + "source_scope": "All 34 distinct academic works and 8 implementation/documentation sources: the 25 works cited by public ChatGPT share 6a593158-85c4-83ea-80b1-b6fb893b26bc plus 9 works added for the LDI0-01 architecture contract (SLM-114) covering multi-objective/gradient-conflict optimization, PEFT actuators, constrained decoding, and token-critical preference lineage. Canonical paper metadata was verified against arXiv (original 25 on 2026-07-16; the 9 additions on 2026-07-17); alternate URLs from the share are preserved in metadata.", "sources": [ + { + "source_id": "arxiv-1810.04650", + "kind": "web", + "title": "Multi-Task Learning as Multi-Objective Optimization", + "uri": "https://arxiv.org/abs/1810.04650", + "published_at": "2018-10-10", + "summary": "Frames multi-task learning as multi-objective optimization and finds a Pareto-stationary descent direction with a multiple-gradient descent algorithm.", + "metadata": {"authors": ["Ozan Sener", "Vladlen Koltun"], "category": "objective_geometry", "local_decision_takeaway": "Locality preservation and semantic improvement are competing objectives to be measured, not blindly summed.", "repo_relevance": "Frames the E249-E284 objective/action-partition blocker", "implementation_status": "Adjacent", "limitations": "No MGDA solver is implemented; the TwoTower loss stays scalarized."} + }, { "source_id": "arxiv-1908.04319", "kind": "web", @@ -12,6 +21,42 @@ "summary": "Penalizes undesirable generations directly, including repetitive or otherwise disfavored tokens.", "metadata": {"authors": ["Sean Welleck", "Ilia Kulikov", "Stephen Roller", "Emily Dinan", "Kyunghyun Cho", "Jason Weston"], "category": "local_preference", "local_decision_takeaway": "Use bad-token unlikelihood as the simplest localized control.", "repo_relevance": "E250 bad-token control", "implementation_status": "Adapted", "limitations": "The paper studies autoregressive text, not masked OpenUI decisions."} }, + { + "source_id": "arxiv-2001.06782", + "kind": "web", + "title": "Gradient Surgery for Multi-Task Learning", + "uri": "https://arxiv.org/abs/2001.06782", + "published_at": "2020-01-19", + "summary": "Introduces PCGrad, which reduces destructive interference by projecting each conflicting task gradient onto the normal plane of the others.", + "metadata": {"authors": ["Tianhe Yu", "Saurabh Kumar", "Abhishek Gupta", "Sergey Levine", "Karol Hausman", "Chelsea Finn"], "category": "objective_geometry", "local_decision_takeaway": "Good/bad and locality/quality objectives can conflict; measure gradient geometry instead of assuming alignment.", "repo_relevance": "Diagnostic lens for the E249-E284 objective-support blocker", "implementation_status": "Adjacent", "limitations": "No gradient-surgery optimizer is implemented."} + }, + { + "source_id": "arxiv-2106.09685", + "kind": "web", + "title": "LoRA: Low-Rank Adaptation of Large Language Models", + "uri": "https://arxiv.org/abs/2106.09685", + "published_at": "2021-06-17", + "summary": "Freezes pretrained weights and injects trainable low-rank update matrices for parameter-efficient adaptation.", + "metadata": {"authors": ["Edward J. Hu", "Yelong Shen", "Phillip Wallis", "Zeyuan Allen-Zhu", "Yuanzhi Li", "Shean Wang", "Lu Wang", "Weizhu Chen"], "category": "adapter_actuator", "local_decision_takeaway": "A removable low-rank actuator is a later intervention choice, separate from the DecisionEventV2 objective.", "repo_relevance": "Removable causal adapter reused via the existing PEFT path (LDI2 TwoTower delta)", "implementation_status": "Adjacent", "limitations": "The first LDI actuator updates existing TwoTower parameters, not a LoRA adapter."} + }, + { + "source_id": "arxiv-2109.05093", + "kind": "web", + "title": "PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models", + "uri": "https://arxiv.org/abs/2109.05093", + "published_at": "2021-09-10", + "summary": "Constrains autoregressive decoding by incrementally parsing partial outputs and rejecting continuations that cannot complete a valid parse.", + "metadata": {"authors": ["Torsten Scholak", "Nathan Schucher", "Dzmitry Bahdanau"], "category": "constrained_decoding", "local_decision_takeaway": "Deterministic constrained decoding is retained; interventions reduce error mass on top of it, never replace it.", "repo_relevance": "Prior art for invariant 4 (hard grammar/compiler constraints stay deployed)", "implementation_status": "Adjacent", "limitations": "OpenUI uses its own incremental compiler forest, not PICARD's SQL parser."} + }, + { + "source_id": "arxiv-2303.10512", + "kind": "web", + "title": "AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning", + "uri": "https://arxiv.org/abs/2303.10512", + "published_at": "2023-03-18", + "summary": "Allocates the low-rank adaptation budget adaptively across weight matrices using importance-scored singular-value pruning.", + "metadata": {"authors": ["Qingru Zhang", "Minshuo Chen", "Alexander Bukharin", "Nikos Karampatziakis", "Pengcheng He", "Yu Cheng", "Weizhu Chen", "Tuo Zhao"], "category": "adapter_actuator", "local_decision_takeaway": "Rank budgeting is an actuator hyperparameter, not an event schema.", "repo_relevance": "Deferred adaptive-rank actuator ablation", "implementation_status": "Adjacent", "limitations": "No adaptive-rank adapter is introduced."} + }, { "source_id": "arxiv-2305.18290", "kind": "web", @@ -30,6 +75,15 @@ "summary": "Uses sparse autoencoders to decompose polysemantic language-model activations into interpretable features.", "metadata": {"authors": ["Hoagy Cunningham", "Aidan Ewart", "Logan Riggs", "Robert Huben", "Lee Sharkey"], "category": "representation_intervention", "local_decision_takeaway": "SAEs may diagnose states but should not replace explicit verifier evidence.", "repo_relevance": "Deferred structural-slop forensics", "implementation_status": "Adjacent", "limitations": "No SAE is trained or used as an oracle here."} }, + { + "source_id": "arxiv-2402.03300", + "kind": "web", + "title": "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models", + "uri": "https://arxiv.org/abs/2402.03300", + "published_at": "2024-02-05", + "summary": "Introduces Group Relative Policy Optimization (GRPO) and scales verifiable-reward reinforcement learning for mathematical reasoning.", + "metadata": {"authors": ["Zhihong Shao", "Peiyi Wang", "Qihao Zhu", "Runxin Xu", "Junxiao Song", "Xiao Bi", "Haowei Zhang", "Mingchuan Zhang", "Y. K. Li", "Y. Wu", "Daya Guo"], "category": "verifiable_training", "local_decision_takeaway": "Group-relative verifiable RL follows local correction and an explicit readiness gate; it is not a first LDI step.", "repo_relevance": "Deferred RLVR stage kept behind the RL-readiness gate", "implementation_status": "Adjacent", "limitations": "No GRPO/RL loop is implemented and no reasoning claim is made."} + }, { "source_id": "arxiv-2402.09353", "kind": "web", @@ -66,6 +120,15 @@ "summary": "Attributes preference updates to response tokens instead of weighting every token uniformly.", "metadata": {"authors": ["Yongcheng Zeng", "Guoqing Liu", "Weiyu Ma", "Ning Yang", "Haifeng Zhang", "Jun Wang"], "category": "local_preference", "local_decision_takeaway": "Train on the exact decision position rather than an entire generated program.", "repo_relevance": "DecisionEventV1 and E251-E254", "implementation_status": "Adapted", "limitations": "OpenUI decisions are masked-token states rather than autoregressive response tokens."} }, + { + "source_id": "arxiv-2405.21047", + "kind": "web", + "title": "Grammar-Aligned Decoding", + "uri": "https://arxiv.org/abs/2405.21047", + "published_at": "2024-05-31", + "summary": "Shows greedy grammar-constrained decoding is distributionally biased and corrects it with an adaptive sampling (ASAp) resampling scheme.", + "metadata": {"authors": ["Kanghee Park", "Jiayu Wang", "Taylor Berg-Kirkpatrick", "Nadia Polikarpova", "Loris D'Antoni"], "category": "constrained_decoding", "local_decision_takeaway": "Grammar alignment corrects the distribution but never itself creates semantic preference labels; constraint shadows stay legality-only evidence.", "repo_relevance": "Grounds invariant 2 (constraint shadows certify legality, not preference)", "implementation_status": "Adjacent", "limitations": "No ASAp resampling is implemented."} + }, { "source_id": "arxiv-2406.04093", "kind": "web", @@ -75,6 +138,15 @@ "summary": "Studies scaling laws and evaluation criteria for sparse autoencoders over language-model activations.", "metadata": {"authors": ["Leo Gao", "Tom Dupré la Tour", "Henk Tillman", "Gabriel Goh", "Rajan Troll", "Alec Radford", "Ilya Sutskever", "Jan Leike", "Jeffrey Wu"], "category": "representation_intervention", "local_decision_takeaway": "Any future SAE must have explicit coverage and causal evaluation.", "repo_relevance": "Deferred SAE audit", "implementation_status": "Adjacent", "limitations": "No SAE evaluation is run."} }, + { + "source_id": "arxiv-2407.01082", + "kind": "web", + "title": "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs", + "uri": "https://arxiv.org/abs/2407.01082", + "published_at": "2024-07-01", + "summary": "Introduces min-p truncation sampling, scaling the acceptance threshold by the top token's probability to balance coherence and diversity.", + "metadata": {"authors": ["Minh Nhat Nguyen", "Andrew Baker", "Clement Neo", "Allen Roush", "Andreas Kirsch", "Ravid Shwartz-Ziv"], "category": "decoding_sampling", "local_decision_takeaway": "Sampling-truncation controls are cheap baselines an intervention must beat, not a substitute for verified correction.", "repo_relevance": "Decoding-time control baseline distinct from a trained objective", "implementation_status": "Adjacent", "limitations": "TwoTower decode is compiler-constrained, not min-p sampled."} + }, { "source_id": "arxiv-2408.05147", "kind": "web", @@ -183,6 +255,15 @@ "summary": "Applies verifiable-reward reinforcement learning to structured cyber-threat outputs.", "metadata": {"authors": ["Md Tanvirul Alam", "Aritran Piplai", "Ionut Cardei", "Nidhi Rastogi", "Peter J Worth"], "category": "verifiable_training", "local_decision_takeaway": "Machine-checkable structure is useful supervision but does not remove the RL readiness gate.", "repo_relevance": "Deferred OpenUI RLVR", "implementation_status": "Adjacent", "limitations": "No Minerva recipe or CTI task is reproduced."} }, + { + "source_id": "arxiv-2603.00025", + "kind": "web", + "title": "TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation", + "uri": "https://arxiv.org/abs/2603.00025", + "published_at": "2026-02-03", + "summary": "Adds a token-level adaptive barrier to preference optimization so token-critical structured-generation decisions receive stronger, state-adaptive updates.", + "metadata": {"authors": ["Samah Fodeh", "Linhai Ma", "Ganesh Puthiaraju", "Srivani Talakokkul", "Afshan Khan", "Sreeraj Ramachandran", "Elyas Irankhah", "Muhammad Arif", "Ashley Hagaman", "Sarah R. Lowe", "Aimee Kendall Roundtree"], "category": "local_preference", "local_decision_takeaway": "Weight token-critical structured decisions with a state-adaptive barrier at the exact state; verify before labeling.", "repo_relevance": "Informs DecisionEventV2 action-table weighting for token-critical decisions", "implementation_status": "Adapted", "limitations": "The implemented clipped-margin FTPO is not TAB-PO's adaptive barrier."} + }, { "source_id": "arxiv-2603.15965", "kind": "web", diff --git a/tests/test_autoresearch/test_harness.py b/tests/test_autoresearch/test_harness.py index 5e79c630e..8810e0a76 100644 --- a/tests/test_autoresearch/test_harness.py +++ b/tests/test_autoresearch/test_harness.py @@ -742,19 +742,46 @@ def test_dsl_program_source_manifest_is_complete() -> None: def test_local_decision_source_manifest_is_complete() -> None: from scripts.autoresearch import _load_sources + allowed_status = {"Direct", "Adapted", "Adjacent", "Rejected"} + required_metadata = ( + "category", + "local_decision_takeaway", + "repo_relevance", + "implementation_status", + "limitations", + ) path = Path("src/slm_training/resources/autoresearch/local-decision-sources.json") manifest = json.loads(path.read_text()) rows = _load_sources(path) papers = [row for row in rows if row.uri.startswith("https://arxiv.org/abs/")] + assert "6a593158-85c4-83ea-80b1-b6fb893b26bc" in manifest["source_scope"] - assert len(rows) == 33 - assert len(papers) == 25 + # LDI0-01 (SLM-114) expanded the inventory from 33 rows / 25 papers to 42 / 34. + assert len(rows) == 42 + assert len(papers) == 34 + # Unique source IDs and canonical URIs: no paper duplicated under an alternate URL. + assert len({row.source_id for row in rows}) == len(rows) assert len({row.uri for row in rows}) == len(rows) - assert all(row.metadata.get("local_decision_takeaway") for row in rows) - assert {row.metadata.get("implementation_status") for row in rows} == { - "Adapted", - "Adjacent", + assert all(row.source_id for row in rows) + # Required metadata is present on every row. + for row in rows: + for field in required_metadata: + assert row.metadata.get(field), (row.source_id, field) + # Implementation-status vocabulary is constrained to the research-lineage set. + assert {row.metadata.get("implementation_status") for row in rows} <= allowed_status + # The nine LDI0-01 additions are present with verified arXiv IDs. + added = { + "arxiv-1810.04650", + "arxiv-2001.06782", + "arxiv-2106.09685", + "arxiv-2109.05093", + "arxiv-2303.10512", + "arxiv-2402.03300", + "arxiv-2405.21047", + "arxiv-2407.01082", + "arxiv-2603.00025", } + assert added <= {row.source_id for row in rows} class FakeResponses: From ec53e173c61b1eda97cc0c89f76eaa6a2b05c03b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 17 Jul 2026 23:28:04 +0000 Subject: [PATCH 2/2] Address review: align test vocabulary with research-lineage and read manifest UTF-8 (SLM-114) - allowed_status now matches the repo's canonical research-lineage vocabulary (Faithful/Adapted/Surrogate/Adjacent) rather than the issue text's Direct/Rejected; the manifest values {Adapted, Adjacent} remain a subset, so the check is unchanged. - read the manifest with explicit encoding="utf-8" to match _load_sources and avoid host-locale dependence (author names contain non-ASCII characters). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01KVT9Qe5YUhgZqvaHLCAYSS --- tests/test_autoresearch/test_harness.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/test_autoresearch/test_harness.py b/tests/test_autoresearch/test_harness.py index 8810e0a76..b5d8f1563 100644 --- a/tests/test_autoresearch/test_harness.py +++ b/tests/test_autoresearch/test_harness.py @@ -742,7 +742,7 @@ def test_dsl_program_source_manifest_is_complete() -> None: def test_local_decision_source_manifest_is_complete() -> None: from scripts.autoresearch import _load_sources - allowed_status = {"Direct", "Adapted", "Adjacent", "Rejected"} + allowed_status = {"Faithful", "Adapted", "Surrogate", "Adjacent"} required_metadata = ( "category", "local_decision_takeaway", @@ -751,7 +751,7 @@ def test_local_decision_source_manifest_is_complete() -> None: "limitations", ) path = Path("src/slm_training/resources/autoresearch/local-decision-sources.json") - manifest = json.loads(path.read_text()) + manifest = json.loads(path.read_text(encoding="utf-8")) rows = _load_sources(path) papers = [row for row in rows if row.uri.startswith("https://arxiv.org/abs/")]