Skip to content

feat(backend): pluggable Phase 2 provider (gemini ↔ MOSS sidecar), default-off - #145

Merged
slittycode merged 1 commit into
mainfrom
claude/nervous-kepler-7143f0
Jun 5, 2026
Merged

feat(backend): pluggable Phase 2 provider (gemini ↔ MOSS sidecar), default-off#145
slittycode merged 1 commit into
mainfrom
claude/nervous-kepler-7143f0

Conversation

@slittycode

Copy link
Copy Markdown
Owner

Summary

Adds a Phase2Provider abstraction so the producer_summary interpretation can be routed to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar instead of Gemini — fed the same audio + Phase-1 JSON, emitting the identical Phase2Result because both providers flow through the same parse → _validate_phase2_citation_pathsapply_live12_catalogue_gates → frozen recommendations.v1 tail. Default-off (ASA_PHASE2_PROVIDER=gemini); Gemini stays the product default and its path is behavior-preserving.

STEP ONE — licence gate (verified from primary sources)

The goal made a licence check mandatory and first. Split verdict:

  • Weights — every OpenMOSS-Team/MOSS-Audio-* repo is license: apache-2.0 (HF API), ungated, no NonCommercial rider.
  • ⚠️ Modeling code — the GitHub OpenMOSS/MOSS-Audio repo has no LICENSE file anywhere and its pyproject.toml declares license = { file = "LICENSE" } — a dangling reference to a file that doesn't exist. The HF weights repo has no modeling_*.py and no AutoModel auto_map, so the model can't run without that unlicensed GitHub code.

Not promotable to the product path (mirrors the MSST backend, which is even more encumbered). Built as a default-off research experiment: the sidecar ships no OpenMOSS code — mock mode for the experiment/eval, real-model path is a 501 stub. Full findings in docs/PHASE2_PROVIDER.md + the asa-moss-audio-licence-gate memory.

What's included

  • phase2_provider.pyPhase2Provider + MossSidecarProvider + env resolver (ASA_PHASE2_PROVIDER, default geminiNone → native path untouched).
  • server.py — one guarded asymmetric branch in _run_interpretation_request_with_profile_config. The Gemini client is still constructed before the try, so a construction error still propagates as INTERPRETATION_SETUP_FAILED. Most of the diff is re-indentation (the unchanged Gemini block nested under else).
  • moss_sidecar/ — standalone FastAPI sidecar (own venv, isolated like MSST) + a deterministic, schema-valid, Phase-1-grounded mock interpreter.
  • scripts/evaluate_phase2_providers.py + phase2_provider_evaluation.py — citation-accuracy eval reusing the production validators; documented "offline-good-enough vs Gemini-wins" split.
  • tests/test_phase2_provider.py — 22 tests incl. a MOSS-branch integration test that executes the modified server.py function end-to-end.

Honest blockers (the DoD's live numbers)

A live MOSS-vs-Gemini citation eval is blocked 3 ways: no GEMINI_API_KEY, no audio renders in the fixed recommendation_tracks/ corpus, and MOSS isn't runnable under the licence gate. The harness + scorer + fixed set + split framework ship; the live legs are labelled BLOCKED, not fabricated. The mock's perfect citation score is a contract proof, not a model-quality proxy.

Verification

🤖 Generated with Claude Code

…fault-off

Add a Phase2Provider abstraction so the producer_summary interpretation can be
routed to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar instead of Gemini,
flowing through the IDENTICAL parse/citation/catalogue validators (the "identical
recommendation schema" DoD). Default-off (ASA_PHASE2_PROVIDER=gemini); Gemini
stays the product default and its path is behavior-preserving (223 server
contract tests pass).

STEP ONE licence gate (verified from primary sources): MOSS-Audio *weights* are
Apache-2.0, but the *modeling code* you must run has no effective licence — the
OpenMOSS/MOSS-Audio GitHub repo has no LICENSE file and its pyproject points at a
non-existent one (dangling ref). So the sidecar ships NO OpenMOSS code: mock mode
for the experiment/eval, real-model path is a 501 stub. Research-only, not
promotable; mirrors the MSST backend treatment.

- phase2_provider.py: Phase2Provider + MossSidecarProvider + env resolver
- server.py: one guarded asymmetric branch in the interpretation request; the
  Gemini client is still built before the try so its errors still propagate as
  INTERPRETATION_SETUP_FAILED (most of the diff is re-indentation)
- moss_sidecar/: standalone FastAPI sidecar (own venv) + deterministic, schema-
  valid, Phase-1-grounded mock interpreter
- scripts/evaluate_phase2_providers.py + phase2_provider_evaluation.py: citation-
  accuracy eval reusing the production validators; documented offline-good-enough
  vs Gemini-wins split (live numbers BLOCKED: no GEMINI_API_KEY, no audio renders,
  model not runnable under the licence gate)
- tests/test_phase2_provider.py: 22 tests incl. a MOSS-branch integration test
  that executes the modified server.py function end-to-end
- docs/PHASE2_PROVIDER.md (STEP ONE findings + design) + CLAUDE.md env vars

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: COMMENT

Summary

Adds a Phase2Provider protocol and MossSidecarProvider HTTP client that lets _run_interpretation_request_with_profile_config route to a self-hosted MOSS sidecar instead of Gemini, while both paths flow through the identical parse → citation-validate → catalogue-gate → recommendations.v1 tail. Default-off (ASA_PHASE2_PROVIDER=gemini); the Gemini path is byte-for-byte unchanged when no external provider is selected. Frontend and Loudness WASM CI are green; Backend CI was still in progress at review time. The 22 new tests cover the resolver, mock contract, sidecar client, eval scorer, and a live integration branch through the real server function.

Findings

Should fix

stem_summary claims to stay on Gemini but doesn't. Three places in this PR assert it — CLAUDE.md, docs/PHASE2_PROVIDER.md, moss_sidecar/README.md: "stem_summary interpretations stay on Gemini; only the recommendation-emitting producer_summary profile routes to MOSS." The implementation disagrees: resolve_external_phase2_provider() is called unconditionally at the top of _run_interpretation_request_with_profile_config with no guard on profile_id. When ASA_PHASE2_PROVIDER=moss, every stem evaluation for every stem in _run_combined_stem_summary_request is also routed to MOSS. The mock sidecar returns a producer_summary-shaped dict — _is_valid_phase2_shape will reject it for the stem_summary schema, the salvage path kicks in, and you get silently empty stem interpretations. Not a product-path failure (MOSS is default-off) but it is wrong behavior for anyone exercising the experiment. Fix is one line:

# In _run_interpretation_request_with_profile_config, after line 1846:
external_provider = resolve_external_phase2_provider()
if profile_id != "producer_summary":   # MOSS is producer_summary-only
    external_provider = None

Worth considering

api_key scoping across two if external_provider is None: blocks (server.py ~L1848, ~L1893). The first block assigns api_key and has three early-return guards. The second block consumes it to construct the Gemini client. At runtime this is always safe (same guard, same condition), but the two-block structure means static analysis will flag api_key as potentially unbound in the second block. The comment "Construct the Gemini client BEFORE the try" explains why the split exists; the scoping awkwardness is worth noting for the next person touching this function.

Test results

Frontend: pass. Loudness WASM: pass. Backend: in progress at review time — no failures visible. The 22 unit/integration tests in test_phase2_provider.py are meaningful (they include a live integration test that executes the modified server function end-to-end with a mocked sidecar), not coverage padding.

Phase boundary check

Clean. Phase 1 (measurement_result) is passed read-only to Phase2ProviderRequest.phase1_result and to build_mock_phase2_result. No new code assigns to or mutates the measurement dict. The mock interpreter reads from phase1 to ground citations but produces an independent return value. The MOSS path flows through the same _validate_phase2_citation_paths + apply_live12_catalogue_gates + build_validated_recommendations tail as Gemini, so invariants #1 and #2 are enforced identically.


Generated by Claude Code

@slittycode
slittycode merged commit 8fbe874 into main Jun 5, 2026
3 checks passed
@slittycode
slittycode deleted the claude/nervous-kepler-7143f0 branch June 5, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant