Add strict K-ary bottleneck phase-boundary experiment harness (CAP2-01, SLM-86)#336
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughAdds a strict K-ary bottleneck model, deterministic CAP2-01 arm matrix harness, CLI execution paths, serialized fixture results, design documentation, and regression tests covering capacity, robustness, leakage, and no-bypass behavior. ChangesCAP2 K-ary bottleneck experiment
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant run_matrix
participant KaryBottleneck
participant BottleneckMatrixReport
participant DesignArtifacts
CLI->>run_matrix: execute CAP2 arm matrix
run_matrix->>KaryBottleneck: train and evaluate learned arms
KaryBottleneck-->>run_matrix: return reconstruction metrics
run_matrix->>BottleneckMatrixReport: assemble arm results
BottleneckMatrixReport->>DesignArtifacts: write JSON and Markdown reports
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Tyler-R-Kendrick
force-pushed
the
slm-86-cap2-01-kary-bottleneck
branch
from
July 18, 2026 09:55
f7b9919 to
fb68e47
Compare
Tyler-R-Kendrick
changed the title
Tyler-R-Kendrick
changed the base branch from
slm-81-cap1-01-state-graph
to
main
July 18, 2026 10:01
Tyler-R-Kendrick
force-pushed
the
slm-86-cap2-01-kary-bottleneck
branch
from
July 18, 2026 10:02
fb68e47 to
36cbb68
Compare
…1, SLM-86) A self-contained K-ary discrete-bottleneck model plus a phase-boundary matrix harness that consumes CAP0-03's exact robust-code constructions (build_mds_7_4_2_3 / build_shortened_ternary_hamming_7_4_3) — no dependency on CAP1-01 (its optional --state-report JSON is duck-typed, requiring only a minimized_states integer; the default path uses --state-count). - models/kary_bottleneck.py: self-contained torch model. - harnesses/experiments/cap2_bottleneck.py + scripts/run_cap2_bottleneck.py: the matrix harness + CLI; experiments/__init__.py exposes them behind a torch-guarded import; run_scaling_ladder.py gains an opt-in --family discrete-bottleneck (defaults to scaling, existing behavior kept). - torch-gated tests (importorskip) + design doc + the author's recorded fixture-run evidence. No model trained or matrix executed here; mechanism increment, no ship claim. No-new-breakage vs a clean-main baseline; canonical arity unchanged. Torch tests run in CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017NWyjZwGMUPHXZhtqQVzUJ
Tyler-R-Kendrick
force-pushed
the
slm-86-cap2-01-kary-bottleneck
branch
from
July 18, 2026 10:20
36cbb68 to
da9ee50
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/design/iter-cap2-01-kary-bottleneck-20260717.md`:
- Around line 15-28: The documented Cap2 bottleneck evaluation recipe and
metadata must include the canonical AgentEvals JSONL and AgentV SDK result
bundle alongside the existing JSON and Markdown outputs. Update the recipe
metadata to explicitly record backend, learned training steps, matrix set, suite
size n, and honesty mode, and include the honest ship-gate or guardrail outcomes
for the measured run.
In `@src/slm_training/harnesses/experiments/__init__.py`:
- Around line 43-58: Update the optional import guard around cap2_bottleneck to
catch only the missing-torch import case, while allowing syntax errors and other
dependency regressions to propagate. Preserve the existing None assignments for
the supported torch-unavailable path and re-raise all other exceptions.
In `@src/slm_training/harnesses/experiments/cap2_bottleneck.py`:
- Around line 341-345: Update the learned-arm flow around KaryBottleneck,
train_kary_bottleneck, and evaluate_kary_bottleneck to execute
model.audit_no_bypass(...) before reporting no-bypass verification. Serialize
the audit result and gate the generated/committed report claims on its success;
otherwise label those claims as separate test evidence rather than fixture-run
verification.
- Around line 280-299: The robust-arm construction in the codebook selection
flow must reject requests exceeding the finite construction size instead of
silently producing fewer codewords. After building the codebook and before
corruption, validate that len(codebook) equals arm.state_count; raise a clear
ValueError identifying the arm and requested versus available codewords when it
does not, while preserving the existing corruption and metric calculations for
valid arms.
- Around line 491-505: Update build_matrix to validate state_count is positive
and seeds is nonempty before constructing arms, reject any arms_filter entries
not matching known arm_id values, and raise an appropriate input-validation
error when filtering leaves no arms. Preserve normal arm construction and
filtering for valid inputs so run_matrix cannot receive an empty matrix.
- Around line 332-345: The learned bottleneck runs are nondeterministic because
Torch is not seeded before initialization. In
src/slm_training/harnesses/experiments/cap2_bottleneck.py lines 332-345, apply
arm.seed before constructing KaryBottleneck and invoking train_kary_bottleneck;
in tests/test_models/test_kary_bottleneck.py lines 29-38, seed Torch before
model construction so the exact-reconstruction assertion is stable.
In `@src/slm_training/models/kary_bottleneck.py`:
- Around line 140-147: The audit logic in the semantic-backed
encoder/code_logits branch should no longer reject models based on
torch.equal(zero_code, code); remove the zero-logit argmax comparison and its
surrounding conditional while preserving decoder recomputation as the no-bypass
check. Add coverage for a semantic_trace model whose decoder depends only on the
hard code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 52229c49-dff1-4543-bb86-19d5c7d62959
📒 Files selected for processing (9)
docs/design/cap2-bottleneck-results.jsondocs/design/iter-cap2-01-kary-bottleneck-20260717.mdscripts/run_cap2_bottleneck.pyscripts/run_scaling_ladder.pysrc/slm_training/harnesses/experiments/__init__.pysrc/slm_training/harnesses/experiments/cap2_bottleneck.pysrc/slm_training/models/kary_bottleneck.pytests/test_harnesses/experiments/test_cap2_bottleneck.pytests/test_models/test_kary_bottleneck.py
Comment on lines
+15
to
+28
| ## Recipe | ||
|
|
||
| ```bash | ||
| python -m scripts.run_cap2_bottleneck \ | ||
| --out-dir outputs/runs/cap2_bottleneck \ | ||
| --seeds 0 | ||
| ``` | ||
|
|
||
| - Device: CPU | ||
| - State source: verified M=41 synthetic fixture | ||
| - Seeds: 0 | ||
| - Deterministic arms: injective assignment and robust-code controls | ||
| - Learned arms: `KaryBottleneck` with straight-through training | ||
| - Output JSON: [`cap2-bottleneck-results.json`](./cap2-bottleneck-results.json) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Record the required canonical evaluation artifacts and recipe metadata.
This measured fixture run includes custom JSON and Markdown only. Add the required AgentEvals JSONL and AgentV SDK result bundle, and explicitly record backend, learned steps, matrix set, suite size (n), and honesty mode.
As per coding guidelines: “Every evaluation run must emit AgentEvals JSONL and an AgentV SDK result bundle” and measured results must include “device, steps, backend, matrix set, suite size (n), honesty mode, and honest ship-gate or guardrail outcomes.” <coding_guidelines>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/design/iter-cap2-01-kary-bottleneck-20260717.md` around lines 15 - 28,
The documented Cap2 bottleneck evaluation recipe and metadata must include the
canonical AgentEvals JSONL and AgentV SDK result bundle alongside the existing
JSON and Markdown outputs. Update the recipe metadata to explicitly record
backend, learned training steps, matrix set, suite size n, and honesty mode, and
include the honest ship-gate or guardrail outcomes for the measured run.
Source: Coding guidelines
Comment on lines
+43
to
+58
| try: | ||
| from slm_training.harnesses.experiments.cap2_bottleneck import ( | ||
| BottleneckArm, | ||
| BottleneckMatrixReport, | ||
| BottleneckResult, | ||
| build_matrix, | ||
| evaluate_arm, | ||
| run_matrix, | ||
| ) | ||
| except Exception: # pragma: no cover - optional if torch unavailable | ||
| BottleneckArm = None # type: ignore[misc,assignment] | ||
| BottleneckMatrixReport = None # type: ignore[misc,assignment] | ||
| BottleneckResult = None # type: ignore[misc,assignment] | ||
| build_matrix = None # type: ignore[misc,assignment] | ||
| evaluate_arm = None # type: ignore[misc,assignment] | ||
| run_matrix = None # type: ignore[misc,assignment] |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== file outline ==\n'
ast-grep outline src/slm_training/harnesses/experiments/__init__.py --view expanded || true
printf '\n== relevant slice ==\n'
sed -n '1,140p' src/slm_training/harnesses/experiments/__init__.py
printf '\n== cap2_bottleneck outline ==\n'
ast-grep outline src/slm_training/harnesses/experiments/cap2_bottleneck.py --view expanded || true
printf '\n== import references ==\n'
rg -n "BottleneckArm|BottleneckMatrixReport|BottleneckResult|build_matrix|evaluate_arm|run_matrix|cap2_bottleneck" src/slm_training -g '!**/__pycache__/**' || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 12271
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== cap2_bottleneck imports ==\n'
sed -n '1,120p' src/slm_training/harnesses/experiments/cap2_bottleneck.py
printf '\n== direct torch references ==\n'
rg -n "\btorch\b|import\s+torch|from\s+torch\b" src/slm_training/harnesses/experiments/cap2_bottleneck.py || true
printf '\n== module-level optional dependency hints ==\n'
rg -n "try:|except|ImportError|ModuleNotFoundError|pragma: no cover" src/slm_training/harnesses/experiments/cap2_bottleneck.py || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 3889
Narrow the optional-import guard
cap2_bottleneck imports torch and several local modules at import time, so except Exception will also mask syntax errors and regressions in those dependencies by exporting these symbols as None. Catch only missing torch and re-raise everything else.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/experiments/__init__.py` around lines 43 - 58,
Update the optional import guard around cap2_bottleneck to catch only the
missing-torch import case, while allowing syntax errors and other dependency
regressions to propagate. Preserve the existing None assignments for the
supported torch-unavailable path and re-raise all other exceptions.
Source: Coding guidelines
Comment on lines
+280
to
+299
| if arm.K == 7 and arm.d == 4: | ||
| codebook = list(build_mds_7_4_2_3())[: arm.state_count] | ||
| elif arm.K == 3 and arm.d == 7: | ||
| codebook = list(build_shortened_ternary_hamming_7_4_3())[: arm.state_count] | ||
| else: | ||
| raise ValueError(f"no verified construction for robust arm {arm.arm_id}") | ||
|
|
||
| rng = random.Random(arm.seed) | ||
| corrupted: list[tuple[int, ...]] = [] | ||
| for code in codebook: | ||
| coord = rng.randrange(arm.d) | ||
| new_symbol = rng.choice([s for s in range(arm.K) if s != code[coord]]) | ||
| corrupted.append(code[:coord] + (new_symbol,) + code[coord + 1 :]) | ||
|
|
||
| decoded = [_nearest_codeword(c, codebook) for c in corrupted] | ||
| correct = sum(1 for orig, dec in zip(codebook, decoded) if orig == dec) | ||
| exact_rate = correct / len(states) | ||
| occupied = len(set(codebook)) | ||
| min_dist, mean_dist = _code_distance_stats(codebook) | ||
| entropy = _empirical_entropy(codebook) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reject robust arms larger than their construction codebook.
The [4,2,3]₇ and [7,4,3]₃ constructions contain finite codebooks. When state_count exceeds their size, slicing silently returns fewer codewords, yet the denominator remains len(states) and collision_count remains zero. Validate len(codebook) == arm.state_count before corruption and fail clearly.
As per coding guidelines: “Always perform input validation at trust boundaries.” <coding_guidelines>
🧰 Tools
🪛 ast-grep (0.44.1)
[info] 286-286: use secrets package over random package
Context: random.Random(arm.seed)
Note: [CWE-330] Use of Insufficiently Random Values.
(avoid-random-python)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/experiments/cap2_bottleneck.py` around lines 280 -
299, The robust-arm construction in the codebook selection flow must reject
requests exceeding the finite construction size instead of silently producing
fewer codewords. After building the codebook and before corruption, validate
that len(codebook) equals arm.state_count; raise a clear ValueError identifying
the arm and requested versus available codewords when it does not, while
preserving the existing corruption and metric calculations for valid arms.
Source: Coding guidelines
Comment on lines
+332
to
+345
| start = time.monotonic() | ||
| cfg = KaryBottleneckConfig( | ||
| num_states=arm.state_count, | ||
| K=arm.K, | ||
| d=arm.d, | ||
| hidden_dim=arm.hidden_dim, | ||
| mode="oracle_state", | ||
| train_steps=arm.train_steps, | ||
| ) | ||
| model = KaryBottleneck(cfg) | ||
| state_tensor = torch.tensor(states, dtype=torch.long) | ||
| target_tensor = torch.tensor(states, dtype=torch.long) | ||
| train_kary_bottleneck(model, state_tensor, target_tensor, steps=arm.train_steps) | ||
| eval_metrics = evaluate_kary_bottleneck(model, state_tensor, target_tensor) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## cap2_bottleneck outline\n'
ast-grep outline src/slm_training/harnesses/experiments/cap2_bottleneck.py --view expanded || true
printf '\n## test_kary_bottleneck outline\n'
ast-grep outline tests/test_models/test_kary_bottleneck.py --view expanded || true
printf '\n## search for seed handling around kary bottleneck\n'
rg -n "manual_seed|seed|torch\\.random|fork_rng|KaryBottleneck|train_kary_bottleneck|evaluate_kary_bottleneck" src/slm_training/harnesses/experiments/cap2_bottleneck.py tests/test_models/test_kary_bottleneck.py src tests -g '!**/__pycache__/**'Repository: Tyler-R-Kendrick/slm-training
Length of output: 50386
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '300,380p' src/slm_training/harnesses/experiments/cap2_bottleneck.py | cat -n
printf '\n----\n'
sed -n '1,120p' tests/test_models/test_kary_bottleneck.py | cat -nRepository: Tyler-R-Kendrick/slm-training
Length of output: 6224
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## model outline\n'
ast-grep outline src/slm_training/models/kary_bottleneck.py --view expanded || true
printf '\n## seed-related search in model file\n'
rg -n "manual_seed|seed|randn|randint|random|dropout|reset_parameters|torch\\.Generator" src/slm_training/models/kary_bottleneck.py
printf '\n## relevant model ranges\n'
sed -n '1,260p' src/slm_training/models/kary_bottleneck.py | cat -nRepository: Tyler-R-Kendrick/slm-training
Length of output: 9926
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## seed-related files\n'
rg -n "manual_seed|seed_everything|torch\.manual_seed|np\.random\.seed|random\.seed|pytest\.fixture" . \
-g 'conftest.py' -g 'pytest.ini' -g 'tests/**/*.py' -g 'src/**/*.py' -g '!**/__pycache__/**'
printf '\n## kary bottleneck callers\n'
rg -n "evaluate_learned_arm|KaryBottleneckConfig\\(|KaryBottleneck\\(" src tests -g '!**/__pycache__/**'Repository: Tyler-R-Kendrick/slm-training
Length of output: 4698
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## arm definition and seed usage\n'
rg -n "class BottleneckArm|seed:|arm\.seed|seed=" src/slm_training/harnesses/experiments/cap2_bottleneck.py
printf '\n## surrounding evaluate_learned_arm block\n'
sed -n '1,220p' src/slm_training/harnesses/experiments/cap2_bottleneck.py | cat -nRepository: Tyler-R-Kendrick/slm-training
Length of output: 9792
Seed the learned bottleneck runs.
src/slm_training/harnesses/experiments/cap2_bottleneck.py#L332-L345: applyarm.seedbefore constructingKaryBottleneckand training so the learned arm is reproducible.tests/test_models/test_kary_bottleneck.py#L29-L38: seed Torch before model construction; otherwise the exact-reconstruction assertion can flap across runs.
📍 Affects 2 files
src/slm_training/harnesses/experiments/cap2_bottleneck.py#L332-L345(this comment)tests/test_models/test_kary_bottleneck.py#L29-L38
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/experiments/cap2_bottleneck.py` around lines 332 -
345, The learned bottleneck runs are nondeterministic because Torch is not
seeded before initialization. In
src/slm_training/harnesses/experiments/cap2_bottleneck.py lines 332-345, apply
arm.seed before constructing KaryBottleneck and invoking train_kary_bottleneck;
in tests/test_models/test_kary_bottleneck.py lines 29-38, seed Torch before
model construction so the exact-reconstruction assertion is stable.
Comment on lines
+341
to
+345
| model = KaryBottleneck(cfg) | ||
| state_tensor = torch.tensor(states, dtype=torch.long) | ||
| target_tensor = torch.tensor(states, dtype=torch.long) | ||
| train_kary_bottleneck(model, state_tensor, target_tensor, steps=arm.train_steps) | ||
| eval_metrics = evaluate_kary_bottleneck(model, state_tensor, target_tensor) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not report no-bypass verification without executing the audit.
The learned-arm path trains and evaluates the model but never calls model.audit_no_bypass(...). Nevertheless, the generated and committed reports claim that the fixture run verifies this invariant. Either execute and serialize/gate the audit result or change those claims to identify it as separate test evidence.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/experiments/cap2_bottleneck.py` around lines 341 -
345, Update the learned-arm flow around KaryBottleneck, train_kary_bottleneck,
and evaluate_kary_bottleneck to execute model.audit_no_bypass(...) before
reporting no-bypass verification. Serialize the audit result and gate the
generated/committed report claims on its success; otherwise label those claims
as separate test evidence rather than fixture-run verification.
Comment on lines
+491
to
+505
| def build_matrix( | ||
| state_count: int, | ||
| *, | ||
| seeds: tuple[int, ...] = (0,), | ||
| arms_filter: tuple[str, ...] | None = None, | ||
| ) -> list[BottleneckArm]: | ||
| arms: list[BottleneckArm] = [] | ||
| arms.extend(build_boundary_arms(state_count, seeds)) | ||
| arms.extend(build_equal_capacity_arms(state_count, seeds)) | ||
| arms.extend(build_robust_arms(state_count, seeds)) | ||
| arms.extend(build_control_arms(state_count, seeds)) | ||
| if arms_filter: | ||
| wanted = set(arms_filter) | ||
| arms = [a for a in arms if a.arm_id in wanted] | ||
| return arms |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reject empty seeds and unknown arm filters instead of producing a vacuous PASS.
An empty --seeds value or entirely unknown --arms filter produces zero arms. run_matrix then returns an empty report and the CLI exits successfully because no leakage was observed.
Validate state_count > 0, require at least one seed, reject unknown arm IDs, and ensure the resulting matrix is nonempty.
As per coding guidelines: “Always perform input validation at trust boundaries” and “do not weaken ship gates merely to make CI pass.” <coding_guidelines>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/harnesses/experiments/cap2_bottleneck.py` around lines 491 -
505, Update build_matrix to validate state_count is positive and seeds is
nonempty before constructing arms, reject any arms_filter entries not matching
known arm_id values, and raise an appropriate input-validation error when
filtering leaves no arms. Preserve normal arm construction and filtering for
valid inputs so run_matrix cannot receive an empty matrix.
Source: Coding guidelines
Comment on lines
+140
to
+147
| if self.encoder is not None and self.code_logits is not None: | ||
| h = self.encoder(x) | ||
| zero_h = torch.zeros_like(h) | ||
| zero_logits = self.code_logits(zero_h).view(-1, self.d, self.K) | ||
| zero_code = zero_logits.argmax(dim=-1) | ||
| if x.numel() > 1 or (self.K > 1 and self.d > 0): | ||
| if torch.equal(zero_code, code): | ||
| return False |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the relevant code region with line numbers.
sed -n '110,180p' src/slm_training/models/kary_bottleneck.py | cat -n
# Locate related tests and audit references.
rg -n "audit_no_bypass|zero_code|zero_h|code_logits|semantic|bypass" src tests . -g '!**/.git/**' || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 50386
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== src/slm_training/models/kary_bottleneck.py ==\n'
sed -n '120,170p' src/slm_training/models/kary_bottleneck.py | cat -n
printf '\n== audit references ==\n'
rg -n "audit_no_bypass|zero_code|zero_h|code_logits|semantic-mode|semantic mode|bypass" src tests -g '!**/.git/**' || true
printf '\n== kary bottleneck tests ==\n'
fd -a -t f "kary_bottleneck" tests src || trueRepository: Tyler-R-Kendrick/slm-training
Length of output: 50386
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== model mode/setup ==\n'
sed -n '1,120p' src/slm_training/models/kary_bottleneck.py | cat -n
printf '\n== audit test body ==\n'
sed -n '1,120p' tests/test_models/test_kary_bottleneck.py | cat -n
printf '\n== semantic_trace references ==\n'
rg -n "semantic_trace|state_code_logits|code_logits|encoder is not None|decode_from_code|hard=True" src/slm_training/models/kary_bottleneck.py tests/test_models/test_kary_bottleneck.pyRepository: Tyler-R-Kendrick/slm-training
Length of output: 10398
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
path = Path("tests/test_models/test_kary_bottleneck.py")
text = path.read_text()
for needle in ["semantic_trace", "audit_no_bypass", "zero_code", "zero_h", "code_logits"]:
print(f"{needle}: {needle in text}")
PY
printf '\n== tests/test_models/test_kary_bottleneck.py ==\n'
sed -n '1,220p' tests/test_models/test_kary_bottleneck.py | cat -nRepository: Tyler-R-Kendrick/slm-training
Length of output: 3037
Drop the extra argmax-change check from the audit.
torch.equal(zero_code, code) can fail on a valid semantic_trace model even when the decoder still depends only on the hard code; the decoder recomputation already covers the no-bypass invariant. Add a test for the semantic-backed branch.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/slm_training/models/kary_bottleneck.py` around lines 140 - 147, The audit
logic in the semantic-backed encoder/code_logits branch should no longer reject
models based on torch.equal(zero_code, code); remove the zero-logit argmax
comparison and its surrounding conditional while preserving decoder
recomputation as the no-bypass check. Add coverage for a semantic_trace model
whose decoder depends only on the hard code.
Source: Coding guidelines
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CAP2-01 (SLM-86): strict K-ary bottleneck phase-boundary experiment
A self-contained K-ary discrete-bottleneck model + a phase-boundary matrix harness.
What's here (9 files, +1464/−2, on current
main)models/kary_bottleneck.py— self-contained torch model (no internalslm_trainingimports).harnesses/experiments/cap2_bottleneck.py+scripts/run_cap2_bottleneck.py— the phase-boundary matrix harness + CLI, consuming CAP0-03's exact robust-code constructions (build_mds_7_4_2_3/build_shortened_ternary_hamming_7_4_3).harnesses/experiments/__init__.py— exposes the CAP2 symbols behind a torch-guarded import (package still imports torch-free).scripts/run_scaling_ladder.py— opt-in--family discrete-bottleneck(defaults toscaling; existing behavior preserved).importorskip) + design doc + the author's recorded fixture-run evidence.No dependency on CAP1-01: the optional
--state-reportJSON is duck-typed (needs only aminimized_statesinteger); the default path uses--state-count. Verified: no import of CAP1-01'sprofiles/ divergentStateGraphor the retired stub. Canonical arity unchanged.Honesty
No model trained, no matrix executed here — mechanism increment; the results JSON is the PR author's recorded fixture evidence carried forward verbatim. No ship gate or test weakened.
Verification
mainbaseline (identical error set, +1 correct skip).test_arity_*49 passed.ruff/repo_policy/compileall/diff-checkclean. Torch tests run in CI.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation