feat: lower graduation thresholds + cloud_correction_event_id lineage

[Gradata]

Summary

• Lower low-volume graduation defaults: INSTINCT->PATTERN now requires 2 fires, PATTERN->RULE requires 3 fires, and Beta-LB defaults to 0.75 / 3 fires.
• Add BrainConfig.graduation_thresholds so advanced users can tune those gates back up per brain.
• Include cloud_correction_event_id on LESSON_CHANGE events, using the SDK event_id of the parent CORRECTION event for forward lineage.

Tests

• python3 -m py_compile src/gradata/_config.py src/gradata/_core.py src/gradata/enhancements/self_improvement/_confidence.py src/gradata/enhancements/self_improvement/_graduation.py src/gradata/enhancements/pipeline_rewriter.py tests/test_brain_config_recall.py tests/test_graduation_thresholds_low_volume.py tests/test_lesson_lineage_cloud_id.py tests/test_safety_assertion.py tests/test_spec_compliance.py tests/test_pipeline_rewriter.py
• python3 -m pytest tests/test_graduation_thresholds_low_volume.py tests/test_lesson_lineage_cloud_id.py tests/test_brain_config_recall.py tests/test_safety_assertion.py tests/test_spec_compliance.py tests/test_pipeline_rewriter.py -q

Notes

Cloud companion PR depends on this SDK release because it consumes LESSON_CHANGE.data.cloud_correction_event_id.

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: efa281fd-1196-4a44-ba16-cc7a0f6fec36

📥 Commits

Reviewing files that changed from the base of the PR and between 7cfb34d and 725845c.

📒 Files selected for processing (13)

• Gradata/docs/internal/SDK_GRADUATION_FORENSICS.md
• Gradata/src/gradata/_config.py
• Gradata/src/gradata/_core.py
• Gradata/src/gradata/enhancements/pipeline_rewriter.py
• Gradata/src/gradata/enhancements/self_improvement/__init__.py
• Gradata/src/gradata/enhancements/self_improvement/_confidence.py
• Gradata/src/gradata/enhancements/self_improvement/_graduation.py
• Gradata/tests/test_brain_config_recall.py
• Gradata/tests/test_graduation_thresholds_low_volume.py
• Gradata/tests/test_lesson_lineage_cloud_id.py
• Gradata/tests/test_pipeline_rewriter.py
• Gradata/tests/test_safety_assertion.py
• Gradata/tests/test_spec_compliance.py

---

📝 Walkthrough

Summary

• Lower graduation thresholds for low-volume categories: INSTINCT→PATTERN now requires 2 fires (previously 3); PATTERN→RULE now requires 3 fires (previously 5); Beta-LB default set to 0.75 with 3 fires
• New public API: GraduationThresholds dataclass with configurable fields: min_applications_for_pattern, min_applications_for_rule, beta_lb_threshold, beta_lb_min_fires
• New public API: graduation_thresholds() function that retrieves effective thresholds from current_brain_config() with fallback to defaults
• Extended BrainConfig with new graduation_thresholds field (default-initialized) supporting per-brain tuning via brain-config.json
• Forward lineage support: LESSON_CHANGE events now include cloud_correction_event_id field (populated from parent CORRECTION event) for cloud materializer lineage tracking
• Updated constants: MIN_APPLICATIONS_FOR_PATTERN (3→2) and MIN_APPLICATIONS_FOR_RULE (5→3) in gradata.enhancements.self_improvement
• Dynamic threshold application: Graduation logic now consults configured thresholds instead of hardcoded values across _confidence.py and _graduation.py
• Module refactoring: self_improvement/__init__.py converted to explicit barrel package with __all__ export list and backward-compatibility via __getattr__
• Documentation and testing: Added forensic analysis guide, new tests for low-volume graduation behavior, lesson lineage verification, and BrainConfig recall
• ⚠️ Breaking change: Lessons may graduate earlier due to lower fire-count thresholds; existing integration code relying on previous thresholds should be validated

Walkthrough

The pull request introduces configurable graduation thresholds for lesson promotion, allowing INSTINCT→PATTERN and PATTERN→RULE transitions to be parameterized via brain-config.json. Core changes include the GraduationThresholds dataclass, dynamic threshold lookup during promotion gating, cloud correction event ID propagation through lesson events, and updated defaults in pipeline analysis. Comprehensive tests and forensics documentation are included.

Changes

Configurable Graduation Thresholds

Layer / File(s)	Summary
Configuration & Schema src/gradata/_config.py	GraduationThresholds dataclass added with min_applications_for_pattern=2, min_applications_for_rule=3, beta_lb_threshold=0.75, beta_lb_min_fires=3. BrainConfig extended with graduation_thresholds field initialized via default factory. Config loading parses graduation_thresholds from brain-config.json with type coercion and range validation.
Graduation Promotion Gating src/gradata/enhancements/self_improvement/_confidence.py, src/gradata/enhancements/self_improvement/_graduation.py	graduation_thresholds() function fetches effective thresholds from current_brain_config() with fallback to GraduationThresholds(). Module constants updated: MIN_APPLICATIONS_FOR_PATTERN 3→2, MIN_APPLICATIONS_FOR_RULE 5→3. Promotion gating in update_confidence() and graduate() now uses threshold object attributes instead of hardcoded constants. _read_beta_lb_config() derives defaults from graduation_thresholds() with threshold clamping into [0.0, 1.0].
Event Lineage & Cloud ID Propagation src/gradata/_core.py	brain_correct() computes cloud_correction_event_id from correction event's event_id or cloud_event_id and propagates through LESSON_CHANGE payloads in both lesson reinforcement and creation paths, enabling traceability from corrections to lessons.
Public API Export src/gradata/enhancements/self_improvement/__init__.py	graduation_thresholds added to module's __all__ list and imported from _confidence, exposing configuration-driven threshold access as part of public barrel API.
Pipeline Analysis Defaults src/gradata/enhancements/pipeline_rewriter.py	analyze_pipeline() defaults changed: min_apps_pattern 3→2, min_apps_rule 5→3, affecting stuck-lesson detection and MIN_APPLICATIONS_FOR_RULE proposal generation.
Documentation docs/internal/SDK_GRADUATION_FORENSICS.md	New forensics guide documenting call-path breakdown (correction → events → lessons → graduation), filtering/dropping points, explicit threshold values, free-text lesson category taxonomy, ranked hypotheses for missing lessons, and five concrete SQL/observability experiments.
Tests & Verification tests/test_brain_config_recall.py, tests/test_graduation_thresholds_low_volume.py, tests/test_lesson_lineage_cloud_id.py, tests/test_pipeline_rewriter.py, tests/test_safety_assertion.py, tests/test_spec_compliance.py	New test test_brain_config_loads_graduation_thresholds verifies config loading with GraduationThresholds import. New test module test_graduation_thresholds_low_volume.py covers INSTINCT→PATTERN (3 fires), PATTERN→RULE (4 fires), and Beta lower-bound gating. test_lesson_change_created_carries_parent_cloud_event_id verifies cloud ID propagation. TestMinApplicationsForRule updated to assert MIN_APPLICATIONS_FOR_RULE == 3. Constants updated in test_spec_compliance.py. Test pipeline rewriter updated to pass explicit min_apps_rule=5.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Gradata/gradata#85: Both PRs modify brain_correct() in src/gradata/_core.py; this PR adds cloud correction event ID propagation while that PR adds provenance/adversarial metadata and review gating.
• Gradata/gradata#101: Both PRs modify the Beta lower-bound gate defaults and parsing in _graduation.py, changing how threshold and min_fires are read and applied.
• Gradata/gradata#165: Both PRs modify Beta-LB configuration and gating logic in _graduation.py, changing how beta gate defaults and parameters are derived and clamped.

Suggested labels

feature

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/graduation-thresholds-and-lineage

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 OpenGrep (1.20.0)

OpenGrep fatal error (exit code 2):
┌──────────────┐
│ Opengrep CLI │
└──────────────┘

�[32m✔�[39m �[1mOpengrep OSS�[0m
�[32m✔�[39m Basic security coverage for first-party code vulnerabilities.

�[1m Loading rules from local config...�[0m
[00.24][ERROR]: Error: exception Glob.Lexer.Syntax_error("malformed glob pattern: missing ']'")
Raised at Glob__Lexer.syntax_error in file "libs/glob/Lexer.mll", line 8, characters 2-26
Called from Glob__Lexer.__ocaml_lex_token_rec in file "libs/glob/Lexer.mll", line 29, characters 26-53
Cal

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}