Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/design/iter-ldi0-01-source-expansion-20260717.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Iter LDI0-01 — local-decision architecture contract + source expansion (2026-07-17)

**Issue:** SLM-114 (LDI0-01). **Type:** documentation / research-inventory / test
contract. **No code, model, checkpoint, adapter, or model-card quality change.**

**Reviewed base commit:** `c7cb099` (`origin/main`, "Cache exact choice completion
states (#315)").

## What changed

Made the local-decision-intervention (LDI) research synthesis and architectural
boundaries canonical in the repository so future agents cannot recreate a parallel
harness, repeat the falsified E249-E284 chain, or treat a local-metric gain as
promotion evidence.

- [`local-decision-interventions.md`](local-decision-interventions.md) — added the
**Architecture contract (LDI0-01)** (evidence / objective / actuator / experiment /
promotion separation with named owners; six invariants; forbids a second stack;
`DecisionEventV2` as the next contract) and a **Measured chain and current
blocker** section (E248-E286 negative chain; blocker = *stable state support does
not imply objective/action-partition support*).
- [`local-decision-sources.json`](../../src/slm_training/resources/autoresearch/local-decision-sources.json)
— added nine academic works (see table).
- [`research-lineage.md`](research-lineage.md) — extended the *Exact-state local
decision preference* entry with the new sources under the existing
Faithful/Adapted/Surrogate/Adjacent vocabulary; no duplicate entries.
- [`quality-experiment-matrix.md`](quality-experiment-matrix.md) — added the **LDI
campaign index** (namespace pointer only; claims no unrun row and mints no E ID;
records the E-ID allocation rule with next-free `≥ E292`).
- `tests/test_autoresearch/test_harness.py` — updated and extended
`test_local_decision_source_manifest_is_complete` (counts, source-ID uniqueness,
required metadata, unique canonical URIs, implementation-status vocabulary, the
nine additions).

## Source count before / after

| | Rows | Academic (arXiv) | Implementation |
| --- | --- | --- | --- |
| Before | 33 | 25 | 8 |
| After | 42 | 34 | 8 |

## Duplicate / alternate-URL handling

Each addition is a distinct canonical arXiv `abs/` URI with a unique `source_id`;
none duplicates an existing paper or an alternate URL. The pre-existing alternate
URLs (DeepSeek-R1 Nature DOI, two OpenReview forum links) remain in `metadata`
rather than as separate rows. The manifest test asserts
`len({source_id}) == len({uri}) == len(rows)`.

## Classification (nine additions)

Metadata verified against arXiv on 2026-07-17. `implementation_status` follows the
research-lineage vocabulary (all additions are Adjacent or Adapted; none is
implemented by this contract).

| arXiv | Work | Category | Status | LDI relevance |
| --- | --- | --- | --- | --- |
| [1810.04650](https://arxiv.org/abs/1810.04650) | MGDA (MTL as multi-objective opt.) | objective_geometry | Adjacent | Frames the objective/action-partition blocker |
| [2001.06782](https://arxiv.org/abs/2001.06782) | Gradient Surgery / PCGrad | objective_geometry | Adjacent | Diagnostic lens for the same blocker |
| [2106.09685](https://arxiv.org/abs/2106.09685) | LoRA | adapter_actuator | Adjacent | Removable causal actuator (later), not the event schema |
| [2109.05093](https://arxiv.org/abs/2109.05093) | PICARD | constrained_decoding | Adjacent | Prior art: hard constraints stay deployed |
| [2303.10512](https://arxiv.org/abs/2303.10512) | AdaLoRA | adapter_actuator | Adjacent | Deferred adaptive-rank actuator |
| [2402.03300](https://arxiv.org/abs/2402.03300) | DeepSeekMath / GRPO | verifiable_training | Adjacent | Deferred RLVR behind the readiness gate |
| [2405.21047](https://arxiv.org/abs/2405.21047) | Grammar-Aligned Decoding / ASAp | constrained_decoding | Adjacent | Legality ≠ preference (invariant 1) |
| [2407.01082](https://arxiv.org/abs/2407.01082) | Min-p sampling | decoding_sampling | Adjacent | Decoding-time baseline to beat |
| [2603.00025](https://arxiv.org/abs/2603.00025) | TAB-PO | local_preference | Adapted | Token-critical PO informs DecisionEventV2 weighting |

## Honesty boundary

Specification / inventory only. No experiment, train, eval, benchmark, checkpoint,
adapter, or model-card quality update. No paper result is represented as a
repository result; every source carries a lineage label, and constraint shadows
certify decoder legality only — never a semantic preference label. No ship gate is
weakened and no readiness or promotion claim is made.
94 changes: 87 additions & 7 deletions docs/design/local-decision-interventions.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Local decision interventions for TwoTower

**Status:** research intake plus measured E248/E249 campaign. E249 is rejected:
constraint-shadow ranking generalized locally but regressed semantic quality on
every suite. This document contains no model-quality or ship claim.
**Status:** LDI0-01 architecture/research contract plus the measured E248-E286
local-decision campaign. The chain is negative: E249 is rejected (constraint-shadow
ranking generalized locally but regressed semantic quality on every suite), and no
local-preference intervention has cleared the unchanged ship gates. This document is
the canonical repository owner of the local-decision-intervention synthesis and
architectural boundaries; it contains no model-quality or ship claim.

## Source and audit

Expand All @@ -12,9 +15,12 @@ The normal page reader returned only the application shell, so the server-render
conversation payload was decoded and its citation records were normalized against
primary arXiv and OpenReview pages. The reviewed inventory is committed as
[`local-decision-sources.json`](../../src/slm_training/resources/autoresearch/local-decision-sources.json):
25 distinct academic works and eight implementation/documentation sources. The
**34 distinct academic works and eight implementation/documentation sources** — the
original 25 works plus nine added for this LDI0-01 contract (multi-objective /
gradient-conflict optimization, PEFT actuators, constrained decoding, and
token-critical preference lineage), each verified against arXiv on 2026-07-17. The
DeepSeek-R1 Nature DOI and the two OpenReview URLs are retained as alternate URLs
rather than double-counted papers.
rather than double-counted papers, and no paper is duplicated under an alternate URL.

| Cluster | Sources | Relevance here |
| --- | --- | --- |
Expand All @@ -32,6 +38,48 @@ Auto-Antislop was reviewed at `8fb98fdf019e6fcc20164f9bdec41f9008fcd632`;
no license file was present, so its source is not copied. Antidoom was reviewed at
`bd6a126476e18554b0cacaea3fd9f258fdde1f97` under Apache-2.0.

## Architecture contract (LDI0-01)

This section is the canonical architectural boundary for the local-decision program.
It exists so future agents do not recreate a parallel harness, repeat the falsified
E249-E284 chain, or treat a local-metric gain as promotion evidence. The Linear
project document is planning context; this file plus the sources manifest are the
source of truth.

**Separation of concerns (do not collapse).** Evidence, objective, actuator,
experiment, and promotion are distinct concerns with distinct owners — extend the
owner, never fork a second stack:

| Concern | Owner (do not fork) | Rule |
| --- | --- | --- |
| Exact-state decision evidence | [`harnesses/preference/local_decisions.py`](../../src/slm_training/harnesses/preference/local_decisions.py) | Events carry exact masked-token state identity. A constraint shadow certifies decoder legality only. |
| Observation / replay | [`harnesses/distill/trace_store.py`](../../src/slm_training/harnesses/distill/trace_store.py) | The append-only decode trace is the observation/replay owner; it is not a trainer. |
| Objective | [`harnesses/preference/local_train.py`](../../src/slm_training/harnesses/preference/local_train.py) | Clipped-margin FTPO and preference losses. The objective is separate from the event schema. |
| Actuator | existing trainable parameters today; later LoRA / DoRA / PiSSA / TwoTower delta / ReFT / SAE | Adapter and representation form are actuator choices, never event schemas. |
| Experiment | [`scripts/run_quality_matrix.py`](../../scripts/run_quality_matrix.py) + [`quality-experiment-matrix.md`](quality-experiment-matrix.md) | The quality matrix / autoresearch is the bounded experiment owner. |
| Promotion | ship gates + [`docs/MODEL_CARD.md`](MODEL_CARD.md) | Only the unchanged five-suite scoreboard and ship gates promote. A local metric never does. |

**Invariants:**

1. Constraint shadows certify decoder legality only. They never become semantic
preference labels without same-state counterfactual verification.
2. Hard grammar/compiler constraints remain deployed. Interventions reduce error
mass and improve semantics *on top of* the deterministic guarantee; they do not
replace it.
3. The first new evidence contract is `DecisionEventV2` — an action-table extension
of `DecisionEventV1` with stable state identity and per-action verdicts (LDI0-02).
The first actuator experiments are causal PEFT and a removable TwoTower delta; SAE
work stays behind matched direct-supervision baselines.
4. New experiments use the `LDI` campaign name in prose and config but obtain
globally unique E IDs from the existing allocation process. Do not reserve or
assume the next E number (see the E-ID rule in
[`quality-experiment-matrix.md`](quality-experiment-matrix.md)).
5. Do not create a second orchestration or training stack. Extend the owners above.
6. No paper result is represented as a repository result. Every source is labeled
Faithful / Adapted / Surrogate / Adjacent in
[`research-lineage.md`](research-lineage.md) and carries an
`implementation_status` in the sources manifest.

## Existing seam and missing evidence

E228 already constructs a compiler decision canvas, restricts logits to legal
Expand Down Expand Up @@ -68,6 +116,12 @@ Two evidence paths are allowed:
Final-output failure alone never creates a token label. Splits group by prompt and
record family so related counterfactuals cannot cross train/held-out boundaries.

**Next contract (LDI0-02):** `DecisionEventV2` adds stable state identity and a
per-action verdict table so multiple good/bad actions at one state each carry an
explicit counterfactual verdict, targeting the objective/action-partition blocker
below. It extends `DecisionEventV1`; it does not replace the trace store or the
objective, and it is specified — not implemented — by this contract.

## Objectives and locality

For good action `g`, bad action `b`, and decision logits `z`, define
Expand Down Expand Up @@ -115,8 +169,9 @@ full-vocabulary drift. End-to-end authority remains the unchanged five-suite
scoreboard and ship gates.

Falsify a row if it does not improve held-out event metrics, exceeds the matched
reference-drift budget, or regresses any protected ship gate. No V10 intervention
row has run; there is no intervention checkpoint, model-card update, or promotion.
reference-drift budget, or regresses any protected ship gate. No V10/LDI
intervention row has cleared the unchanged ship gates, and there is no intervention
checkpoint, model-card update, or promotion.

## Measured event-mining prerequisite

Expand All @@ -134,3 +189,28 @@ suite. Do not run E250/E251 on this corpus as quality labels. E252-E254 remain
fail-closed because the corpus contains no counterfactual or set-valued evidence.
Measured result:
[`iter-e249-local-ce-margin-20260716.md`](iter-e249-local-ce-margin-20260716.md).

## Measured chain and current blocker (E248-E286)

E248-E286 is authoritative and lives in
[`quality-experiment-matrix.md`](quality-experiment-matrix.md) (the V10 rows
E248-E254 and the local-preference ledger through E286) and the per-run
`iter-e2*.md` iteration docs. The chain is negative:

- **E248** — matched parent control, eval-only.
- **E249** — exact-event CE plus margin. The lexical decision objective generalized
(held-out chosen-win 0.7649) but semantic structure and reward regressed on every
suite; **rejected**.
- **E250-E284** — the registered bad-token, single-pair FTPO, verifier-backed set
FTPO, frozen-reference tether, and balanced-sampling levers (and the E265-E286
local-preference ledger) were measured; none cleared the unchanged five-suite ship
gates or was promoted.

**Current blocker: stable state support does not imply objective/action-partition
support.** Exact-state event identity — a stable, replayable decision state — is
necessary but not sufficient. It does not establish that the good/bad *action
partition* at that state is itself verifier-supported, so a locally-improved event
metric has repeatedly failed to transfer to semantic quality under the unchanged
gates. `DecisionEventV2` counterfactual action-verdict tables (LDI0-02) target
exactly this gap. No LDI intervention row is promoted; there is no intervention
checkpoint and no model-card quality update.
31 changes: 28 additions & 3 deletions docs/design/quality-experiment-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -1116,11 +1116,36 @@ full suites) is still required, and E241/E242's conflict machinery has not been
exercised outside unit/integration tests because greedy decode never stalls on
this checkpoint.

## LDI campaign index (local decision interventions)

The **LDI** campaign is the local-decision-intervention line of work. Its canonical
architecture/research contract, invariants, named owners, and the 42-source manifest
are in [`local-decision-interventions.md`](local-decision-interventions.md). This
index is a namespace pointer, not a new set of rows: it claims **no** unrun row and
allocates **no** E ID.

Measured record (authoritative): the **V10 exact-state local preference** rows
E248-E254 (below) and the measured **E265-E286 local-preference ledger** recorded in
this matrix and the per-run `iter-e2*.md` docs (broad/guarded FTPO, reference
tethers, balanced sampling). The chain is negative — E249 and E252 are rejected
(local metrics moved, semantic quality regressed), and no LDI intervention has
cleared the unchanged five-suite ship gates or been promoted. Current blocker:
**stable state support does not imply objective/action-partition support**; exact
state identity does not prove the good/bad action partition is verifier-supported.
`DecisionEventV2` action-verdict tables (LDI0-02) target this gap.

**E-ID allocation rule.** New LDI experiments take a globally unique E ID from the
existing allocation process; the `LDI` name is prose/config only and reserves no ID.
As of 2026-07-17 the highest allocated ID is **E291** (B1/B3 tracks; see the
[`README.md`](../../README.md) run ledger), and E248-E291 plus the E263/E264
local-preference rows are consumed. Do not assume "the next number after E286" is
free — the next free ID is **≥ E292**.

## V10 exact-state local preference (E248 control measured)

The full 25-paper audit, source manifest, objective definition, and honesty boundary
are in [`local-decision-interventions.md`](local-decision-interventions.md). V10
reuses the existing preference harness and append-only decode traces. It does not
The full source audit (34 works), source manifest, objective definition, and honesty
boundary are in [`local-decision-interventions.md`](local-decision-interventions.md).
V10 reuses the existing preference harness and append-only decode traces. It does not
introduce an adapter/SAE trainer and does not claim that a local loss produces a
local parameter update.

Expand Down
25 changes: 17 additions & 8 deletions docs/design/research-lineage.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,15 +166,24 @@ of [XGrammar](https://arxiv.org/abs/2411.15100). Implementation and exact gates:

| | |
| --- | --- |
| **Papers** | [Unlikelihood](https://arxiv.org/abs/1908.04319), [Token-level DPO](https://arxiv.org/abs/2404.11999), [TIS-DPO](https://arxiv.org/abs/2410.04350), [ConfPO](https://arxiv.org/abs/2506.08712), [TGDPO](https://arxiv.org/abs/2506.14574), [Antislop](https://arxiv.org/abs/2510.15061), [TokenRatio](https://arxiv.org/abs/2605.12288) |
| **Papers** | [Unlikelihood](https://arxiv.org/abs/1908.04319), [Token-level DPO](https://arxiv.org/abs/2404.11999), [TIS-DPO](https://arxiv.org/abs/2410.04350), [ConfPO](https://arxiv.org/abs/2506.08712), [TGDPO](https://arxiv.org/abs/2506.14574), [Antislop](https://arxiv.org/abs/2510.15061), [TokenRatio](https://arxiv.org/abs/2605.12288), [TAB-PO](https://arxiv.org/abs/2603.00025) |
| **Fidelity** | **Adapted** — exact masked-token states, verifier-backed good/bad action sets, clipped logit margins, and optional frozen-reference tethering; not sequence DPO or a faithful reproduction of any cited objective |
| **Code** | Existing preference/trace owners; detailed boundary and full 25-paper audit in [`local-decision-interventions.md`](local-decision-interventions.md) |
| **Matrix** | Proposed/unrun V10 rows E248-E254 in [`quality-experiment-matrix.md`](quality-experiment-matrix.md) |

SAE/ReFT discovery, removable LoRA/DoRA/PiSSA actuators, adapter routing, iterative
remine, and RLVR are **Adjacent**. The first TwoTower implementation is a localized
loss with a global full-parameter update. It makes no locality or ship claim until
the registered controls run under unchanged gates.
| **Code** | Existing preference/trace owners; the LDI0-01 architecture contract, named owners, and full 34-source audit are in [`local-decision-interventions.md`](local-decision-interventions.md) |
| **Matrix** | Measured V10 rows E248-E254 and the E265-E286 local-preference ledger (LDI campaign index) in [`quality-experiment-matrix.md`](quality-experiment-matrix.md); the chain is negative (E249/E252 rejected) |

SAE/ReFT discovery, removable LoRA/DoRA/PiSSA/[AdaLoRA](https://arxiv.org/abs/2303.10512)
actuators, adapter routing, iterative remine, and RLVR are **Adjacent**.
Objective-geometry lenses for the current blocker —
[Gradient Surgery/PCGrad](https://arxiv.org/abs/2001.06782) and
[MGDA](https://arxiv.org/abs/1810.04650) — and decoding baselines
([min-p](https://arxiv.org/abs/2407.01082)) are **Adjacent**: diagnostic framing and
baselines to beat, not implemented. [PICARD](https://arxiv.org/abs/2109.05093) and
[Grammar-Aligned Decoding/ASAp](https://arxiv.org/abs/2405.21047) frame the
constraint-legality invariant — constraint shadows certify legality only, never
semantic preference. These are lineage labels, not reproduced results. The first
TwoTower implementation is a localized loss with a global full-parameter update. It
makes no locality or ship claim until the registered controls run under unchanged
gates.

### GRPO-lite

Expand Down
Loading
Loading