SLM-128 (LDI3-01): structured local-preference objectives (Legal-Set FTPO, TAB-PO barrier, TBPO ratio control)

[Tyler-R-Kendrick]

Implements SLM-128 / LDI3-01 — a shared, architecture-neutral objective library adding three OpenUI-native structured local-preference objectives over the exact grammar-legal action sets. One implementation (pure functions of a 1-D logit row + materialized action sets) that both the causal and TwoTower trainers call — the trainer extracts the row, this module owns the math, nothing duplicated in architecture-specific files. Builds on the SLM-116/117 materializers + objective-support admission now on main.

What landed

src/slm_training/harnesses/preference/structured_objectives.py:

• StructuredObjectiveConfig — typed, versioned, fail-closed-on-unknown-fields config with a deterministic content_sha fingerprint (part of run/adapter identity).
• StructuredObjectiveInput — logits + legal/good/bad/ambiguous/unobserved sets + per-action evidence weights, roles, criticality, optional same-state reference; rejects empty good/bad, out-of-legal ids, overlapping partitions.
• Objective A — Legal-Set FTPO (two variants): explicitly pair-weighted G × B set margin normalized per state, and a legal probability-mass margin softplus(margin + log P_B − log P_G) reporting good/bad/ambiguous/unobserved mass separately.
• Objective B — TAB-PO-inspired barrier: additive, separately-metered SFT anchor −log p_legal(g) for verifier-critical, under-confident good actions (zero for confident; structural roles default low) + a token_erosion_rate metric.
• Objective C — TBPO-inspired ratio control: bounded good-vs-bad log-ratio control against a same-state reference, advantage-centered or centered by a serializable StateBaseline fit only from train states; disabled + reported without a reference.
• Composition: preference + optional barrier + target/non-target locality tethers, each separately metered (no double-counting); per-state-normalized batch loss so large action sets can't dominate (un-normalized mode kept for ablation); a no-model-update fixture report generator.

Existing ce_margin/unlikelihood/ftpo_* are untouched (new names) — historical behavior and the 68 existing preference tests are unchanged.

Tests

tests/test_harnesses/preference/test_structured_objectives.py (16, green): both Legal-Set FTPO variants match hand-computed values, state normalization prevents large-set dominance, legal mass sums over only the legal set, barrier selectivity + structural-role weighting, erosion detection, ambiguous/unobserved normalized-but-never-targets, ratio control compares at identical states and disables without a reference, state-baseline fit + round-trip, mass-margin gradients match finite differences, tiny-probability numeric stability, config round-trip/fingerprint/fail-closed validation, architecture-neutral call over "causal"/"TwoTower" mock logits, and input validation. ruff, python -m scripts.repo_policy, git diff --check clean.

Evidence artifact

docs/design/iter-ldi3-01-structured-objectives-20260718.md + a deterministic fixture report. The four objectives are genuinely distinct on the same corpus (batch loss ≈ mass-margin 1.26 / pairwise 2.73 / barrier 2.36 / ratio 1.26); raw-vs-legal good mass differs (0.39 full-vocab vs 0.59 legal-space); the barrier activates only for the low-prob critical state.

Honesty

Adapted, not reproduced (names carry tab_po_inspired/tbpo_inspired). No matrix run, no checkpoint, no quality claim, no new event mining, no hidden eval gold; criticality comes from compiler/AST role + verified action evidence, never gold programs. Wiring the objectives into the trainer entry points behind a config switch is documented as deferred follow-on.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U5Fy8PnMTV54p675y9T3B8

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
slm-training	Building	Preview	Jul 18, 2026 4:37am

[coderabbitai]

Warning

Review limit reached

@Tyler-R-Kendrick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0fb08f9b-619c-475e-b6f4-41891a3918db

📥 Commits

Reviewing files that changed from the base of the PR and between 3847f45 and 9247608.

📒 Files selected for processing (4)

• docs/design/iter-ldi3-01-structured-objectives-20260718.md
• docs/design/iter-ldi3-01-structured-objectives-fixture-20260718.json
• src/slm_training/harnesses/preference/structured_objectives.py
• tests/test_harnesses/preference/test_structured_objectives.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/slm-training-issues-d8l0ba

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Tyler-R-Kendrick]

Superseded by #359, merged as the canonical LDI3-01 structured-objectives implementation for SLM-128.

#359 lands the same three objectives (Legal-Set FTPO / tab_barrier / tbpo_inspired) as a clean, gradcheck-verified 5-file additive change on current main (green CI), plus the no-model-update evidence report. This PR carries a full stale re-implementation of dsl/solver/*, dsl/analysis/arity/*, model code, docs, README/MODEL_CARD, and pyproject.toml that conflicts with or duplicates work already on main, so it cannot be merged as-is.

Closing as a duplicate to keep one canonical implementation per issue. If this branch's larger structured_objectives.py variant (+263/−558 vs #359) carries objective refinements worth keeping, they're best applied as a focused follow-up on top of #359.

---

Generated by Claude Code