refactor: extract frozen DSL-agnostic harness core (structural only)#484
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (38)
📝 WalkthroughWalkthroughIntroduces a DSL-agnostic ChangesHarness core extraction
Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Pull the frozen machinery every harness builds on into a new
slm_training.harness_core package that supports any DSL/AST; the OpenUI
harnesses keep their DSL-specific policy and optimizations at their
current paths and keep iterating there.
- moved (git mv): versioning.py, lineage/ (9 modules),
model_build/checkpoint_reference.py,
experiments/{efficiency_gain,scaling_fit}.py,
evals/{record_schema,eval_cache,score_policy}.py
- extracted engines: gate_engine.py (ship-gate check loop behind a
normalize_suite/default_min_n seam) and promotion_engine.py
(promotion checks behind hard_categories/gate_evaluator); the policy
owners model_build/ship_gates.py and experiments/promotion.py keep
every threshold, metric name, constant, and public signature
- every old import path remains a sys.modules alias shim: old and new
paths resolve to the same module objects (class identity and
monkeypatching unchanged); no call sites were touched
- hygiene enforced by tests/test_harness_core: no DSL-coupled
module-level imports in core, torch-free core import, shim identity,
and a golden characterization test pinning the ship-gate payload
byte-identical to a pre-extraction capture
- registry: new harness.core component claims harness_core/; gates.ship
additionally claims gate_engine.py; no-bump history notes on
gates.ship, harness.experiments, evals.scoring
- tooling/docs: check_changed prefix map, AGENTS.md norm,
repository-organization placement row, docs/design/harness-core.md
No functional change to harnesses or training: the full pre-existing
test suite passes/fails identically to the pre-change baseline in this
environment (the failing set is environment-dependent bridge/network
tests, byte-identical before and after).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011DARSmTTQtF2AzYBy6wosC
Tyler-R-Kendrick
force-pushed
the
claude/harness-core-package-d1vqiz
branch
from
July 19, 2026 20:45
9311adb to
36977a8
Compare
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pulls the frozen machinery every harness builds on into a new
slm_training.harness_corepackage that supports any arbitrary DSL/AST. The OpenUI harnesses keep their DSL-specific policy and optimizations at their current import paths and keep iterating there. Purely structural — zero functional change to harnesses or training.What moved (git mv, history preserved)
src/slm_training/versioning.pyharness_core/versioning.py(registry paths re-anchored)src/slm_training/lineage/(9 modules)harness_core/lineage/harnesses/model_build/checkpoint_reference.pyharness_core/checkpoint_reference.pyharnesses/experiments/{efficiency_gain,scaling_fit}.pyharness_core/{efficiency_gain,scaling_fit}.pyevals/{record_schema,eval_cache,score_policy}.pyharness_core/{record_schema,eval_cache,score_policy}.pyEngine seams (DSL specifics stay in the harnesses)
harness_core/gate_engine.py— the ship-gate check loop (missing-suite fail-closed, fallback certification,min_nevidence floor, threshold loop) extracted verbatim behind anormalize_suite/default_min_nseam.model_build/ship_gates.pyremains the OpenUI policy owner:DEFAULT_SHIP_GATES,MEANINGFUL_METRIC_POLICY, the slim-metric normalizer, payload assembly,gates.jsonwriting — thresholds and payload byte-identical.harness_core/promotion_engine.py— promotion checks extracted behindhard_categories/gate_evaluatorparameters.experiments/promotion.pykeepsHARD_CATEGORIES, the DSL-touchingcheck_data_integrity, the ship-gate binding, and every original public signature (including the pre-existingthreshold_version: custom:<sha>behavior when promotion invokes the default gate policy).Zero call-site churn
Every old import path remains a
sys.modulesalias shim: old and new paths resolve to the same module objects, so class identity and monkeypatching behave identically through either path. No consumer files (scripts, tests, web, autoresearch) were touched.Guarantees, enforced
tests/test_harness_core/test_import_hygiene.py— core has no DSL-coupledslm_training.*module-level imports (one grandfathered lazy exception allowlisted), and importing core never pulls torch.tests/test_harness_core/test_gate_engine_golden.py— ship-gate payloads pinned byte-for-byte against a pre-extraction capture (default policy, fallback/legacy paths, custom thresholds).tests/test_harness_core/test_shims.py— module identity across all 16 old/new path pairs.versions.json: newharness.corecomponent claimsharness_core/;gates.shipadditionally claimsgate_engine.pyso gate logic stays under gate discipline;no-bump:history notes ongates.ship,harness.experiments,evals.scoring.Verification
repo_policy,ruff check .,compileall,verify_version_stamps --check --base origin/main,validate_page_dsl --check,verify_checkpoint_references --checkall green.Docs
docs/design/harness-core.md(the freeze contract + seams + explicit non-goals/follow-ups), AGENTS.md engineering norm, repository-organization placement row.🤖 Generated with Claude Code
https://claude.ai/code/session_011DARSmTTQtF2AzYBy6wosC
Generated by Claude Code
Summary by CodeRabbit
New Features
Documentation
Tests