Skip to content

feat(stage_router): corroborative tanh-scored routing with windowed self-clearing signals#68

Draft
sabhatinas wants to merge 6 commits into
mainfrom
sabhatinas/stage-router-scorer-corroborative-tanh
Draft

feat(stage_router): corroborative tanh-scored routing with windowed self-clearing signals#68
sabhatinas wants to merge 6 commits into
mainfrom
sabhatinas/stage-router-scorer-corroborative-tanh

Conversation

@sabhatinas

@sabhatinas sabhatinas commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Reworks the Python stage_router scorer/picker so the confidence dial is well-behaved and the signals are honest. No behavior is hard-coded to a single dominant axis anymore — decisions are corroborative, signals self-clear, and the score spreads smoothly across the range.

What changed

  • Windowed, self-clearing gatesstuck_exploring moved off cumulative counts (which latched on forever) to the recent window; no_progress re-differentiated so it's a distinct whole-task dead-end rather than a duplicate of stuck_exploring.
  • Two clean signal groups — "something's wrong" (severity, stuck, no-progress) → CAPABLE; "making progress" (recent writes, planning, clean streaks) → EFFICIENT. Picker-independent; capable_first/efficient_first differ only in the low-confidence fall-open.
  • Corroborative weights — a fixed unit weight so no single signal saturates; the confidence_threshold becomes the real dial (signals-to-clear = threshold / unit).
  • tanh spread — the raw weighted sum is squashed across (-1, 1) so a symmetric positive threshold sweeps the routing split smoothly instead of piling scores at the extremes.
  • tests_passed short-circuit — a settled run (recent test-pass + recent edit) routes to EFFICIENT decisively, instead of a fixed weight.

Validation

  • All stage_router unit tests pass (scorer, pickers, dimensions, request processor); ruff clean.
  • 20-task Terminal-Bench subset (matched Azure-Opus strong tier): matches the Opus baseline accuracy (~75%) while routing ~50% of steps to the cheap tier, with the hard/flaky tasks recovered.

Out of scope (follow-up)

  • The Rust v2 stage_router profile (crates/switchyard-components-v2/src/profiles/stage_router.rs) is a parallel implementation with the old weights and now diverges from Python. Porting these changes for parity is intentionally left as a separate follow-up.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Improved stage routing for settled runs, routing recent edits with passing tests to the Efficient tier.
    • Refined detection of stalled exploration and lack of progress using recent activity patterns.
    • Updated scoring to provide smoother confidence levels and better distinguish progress from unproductive activity.
  • Bug Fixes

    • Corrected routing behavior for low-confidence decisions and corroborated unproductive activity.
    • Separated passing-test decisions from general scoring for more consistent outcomes.

@sabhatinas
sabhatinas requested a review from a team as a code owner July 14, 2026 21:08
@sabhatinas sabhatinas self-assigned this Jul 14, 2026
@sabhatinas
sabhatinas marked this pull request as draft July 14, 2026 21:13
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d6605456-7b95-4bc7-a17a-9528671b5c1c

📥 Commits

Reviewing files that changed from the base of the PR and between 487fa4c and 3213847.

📒 Files selected for processing (6)
  • switchyard/lib/processors/stage_router/decision_log.py
  • switchyard/lib/processors/stage_router/dimensions.py
  • switchyard/lib/processors/stage_router/picker.py
  • switchyard/lib/processors/stage_router/scorer.py
  • tests/test_stage_router_pickers.py
  • tests/test_stage_router_scorer.py

Walkthrough

The stage router now separates tests_passed decisions, uses recent-window progress gates, routes settled runs directly to EFFICIENT, preserves critical overrides, and computes tanh-squashed scores with updated tests.

Changes

Stage router behavior

Layer / File(s) Summary
Parameterized tanh scoring
switchyard/lib/processors/stage_router/scorer.py, tests/test_stage_router_scorer.py
Grouped signal weights and tanh scoring replace fixed weights and hard clipping; tests cover direction, saturation, confidence, and raw contributions.
Routing gates and settled-run precedence
switchyard/lib/processors/stage_router/decision_log.py, switchyard/lib/processors/stage_router/dimensions.py, switchyard/lib/processors/stage_router/picker.py, tests/test_stage_router_pickers.py
Recent-window gates and the dedicated tests_passed source support early EFFICIENT routing, while critical severity continues to force CAPABLE; picker scenarios are updated accordingly.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

I’m a bunny with tests in my ear,
Hopping through scores that grow smooth and clear.
Settled runs dash to EFFICIENT light,
Critical thumps keep CAPABLE in sight.
Tanh curls the signals, soft as spring—
New paths bloom while green checks sing.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main stage_router changes: tanh-based scoring and windowed self-clearing signals.
Docstring Coverage ✅ Passed Docstring coverage is 88.24% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch from c43105a to 8c8c303 Compare July 16, 2026 17:08
@sabhatinas
sabhatinas force-pushed the sabhatinas/stage-router-scorer-corroborative-tanh branch from 0279579 to 220c474 Compare July 17, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant