fix(graduation): session-count-aware state for backfilled patterns#119
Conversation
_patterns_to_graduated_lessons previously teleported every qualifying correction_pattern directly to RULE@0.92, bypassing the INSTINCT→PATTERN →RULE graduation gates. A 2-session pattern ended up with the same standing as a 50-session rule. Result on live brain: 18 RULEs / 0 PATTERNs / 2 INSTINCTs — all 18 RULEs came from weak 2-session evidence. _state_for_sessions now maps evidence to state: - 2 sessions → PATTERN @ 0.70 - 3-4 → PATTERN @ 0.80 - 5+ → RULE @ 0.92 Aligns backfill with live graduation thresholds (MIN_APPLICATIONS_FOR_ PATTERN=3, MIN_APPLICATIONS_FOR_RULE=5). Backfill still filters single- session patterns (min_sessions=2) — those would be noise at the 2.1GB transcript scale. Co-Authored-By: Gradata <noreply@gradata.ai>
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughSummary
WalkthroughUpdated the rule pipeline to derive lesson state and confidence based on the count of distinct sessions rather than using fixed values. Added a tiered threshold function that maps session counts to state transitions and session-aware confidence scores, replacing the previous hard-coded RULE state and 0.92 confidence. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested labels
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Summary
Cherry-picked from a local commit on the main worktree (
40f33e5a, authored 2026-04-19 by Oliver) — promoting it to a reviewable PR so it can land cleanly before the v0.6.1 tag._patterns_to_graduated_lessonspreviously teleported every qualifyingcorrection_patterndirectly to RULE@0.92, bypassing the INSTINCT→PATTERN→RULE graduation gates. A 2-session pattern ended up with the same standing as a 50-session rule. Live brain showed 18 RULEs / 0 PATTERNs / 2 INSTINCTs — all 18 RULEs came from weak 2-session evidence.Fix
_state_for_sessionsnow maps evidence → state:Aligns backfill with live graduation thresholds (
MIN_APPLICATIONS_FOR_PATTERN=3,MIN_APPLICATIONS_FOR_RULE=5). Backfill still filters single-session patterns (min_sessions=2).Test plan
pytest tests/test_rule_pipeline.py— 26 passed in 0.56sWhy now
Needed before cutting the v0.6.1 tag — without this, any brain backfilled on 0.6.1 will replicate the over-promotion bug. Blocker for #117.
Generated with Gradata