Skip to content

feat: freeze versioned recommendations.v1 JSON contract for Phase 2 output #143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
slittycode merged 1 commit into from
Jun 5, 2026
Merged

feat: freeze versioned recommendations.v1 JSON contract for Phase 2 output #143

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ Operational and one-shot scripts live in two places. They are not on the request
2. `packages/loudness-spectro-wasm/` — browser-first WebAssembly DSP (ITU-R BS.1770-5 / EBU R128 loudness, A-weighted spectrum, and spectral-reassignment spectrogram). Rust lifted from [openmeters](https://github.com/httpsworldview/openmeters) and compiled via `wasm-bindgen`, GPL-3.0-or-later. **Phase 1: standalone, not yet imported by `apps/ui` or `apps/backend`.** Has its own Cargo workspace, `npm`/`cargo` build, and EBU/ebur128/pyloudnorm validation layers — see [`packages/loudness-spectro-wasm/README.md`](packages/loudness-spectro-wasm/README.md). When integration lands, the canonical Phase 1 LUFS contract still comes from the Essentia path until this is wired in and proven at parity. (The reassigned spectrogram on the product path today comes from librosa via the `reassigned` spectral-enhancement endpoint, not this package.)
3. `audits/` — dated, automated audit reports (e.g. `nightly-2026-05-19.md`, `phase2-recommendation-surface-2026-05-24.md`, `full-review-2026-05-30.md`). Past-tense paper trail; not imported by either app. (Older one-shot advisory deliverables like the phase 1 audit have been archived under `docs/history/phase1-audit/`.)
4. `incorporations/` — planning docs for incorporating upstream projects (e.g. `forking-plans-2026-05-14.md`, `beat-this-measurement-gate-2026-05-20.md`). Planning notes only; not code.
5. `docs/` — long-form rationale and architectural records. Key contents: `ARCHITECTURE_STRATEGY.md` (three-layer design rationale), `SETUP.md` (first-run setup), `LAYER2_EVALUATION.md` and `POLYPHONIC_TRANSCRIPTION_SPIKE.md` (Layer 2 research), `SAMPLE_GENERATION.md` (Phase 3 design), `adr/` (Architecture Decision Records — `0001-phase1-json-schema-v1.md` declares the v1 schema stability contract; `0002-phase1-loudness-units-v2.md` bumped `truePeak` to dBTP and `bpmConfidence` to normalized 0–1, schema now at `phase1.v2`), and `history/` (completed plans, one-shot audits, and deliverables — past-tense). Read the relevant doc *before* structural changes; do not treat them as living API docs.
5. `docs/` — long-form rationale and architectural records. Key contents: `ARCHITECTURE_STRATEGY.md` (three-layer design rationale), `SETUP.md` (first-run setup), `LAYER2_EVALUATION.md` and `POLYPHONIC_TRANSCRIPTION_SPIKE.md` (Layer 2 research), `SAMPLE_GENERATION.md` (Phase 3 design), `adr/` (Architecture Decision Records — `0001-phase1-json-schema-v1.md` declares the v1 schema stability contract; `0002-phase1-loudness-units-v2.md` bumped `truePeak` to dBTP and `bpmConfidence` to normalized 0–1, schema now at `phase1.v2`; `0003-recommendations-contract-v1.md` freezes the Phase 2 `recommendations` contract (`recommendations.v1`) — a normalized, schema-validated, citation-gated projection of the device cards), and `history/` (completed plans, one-shot audits, and deliverables — past-tense). Read the relevant doc *before* structural changes; do not treat them as living API docs.
6. `tests/ground_truth/` — labeled-corpus fixtures consumed by `scripts/calibrate_confidence.py` (see `tests/ground_truth/README.md`). Not a test suite — each app owns its own (`apps/backend/tests/`, `apps/ui/tests/`).

### Three-Layer Model
Expand Down Expand Up @@ -293,6 +293,8 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths

22. **`separation_backend.py`**: Selectable Phase 1 stem-separation backend (default-off experiment). `ASA_SEPARATION_BACKEND=msst` swaps torchaudio Hybrid Demucs (`analyze_audio_io.separate_stems`) for a stronger MSST/BS-RoFormer model from a `SUC-DriverOld/MSST-WebUI` checkout, while keeping the `{stem_name: wav_path}` contract (`vocals/bass/drums/other`, 44.1 kHz) unchanged. Because MSST pins deps that conflict with ASA's (`librosa==0.9.2`, `numpy<2`, its own torch), it runs in its **own** venv and ASA shells out to `scripts/msst_separate_runner.py` under `ASA_MSST_PYTHON` — mirroring how `loudness_backend.py` shells out to `measure-cli`. The subprocess boundary also keeps MSST's stdout/logging off `analyze.py`'s JSON contract (tripwire #1) and isolates its `utils`/`inference` packages. A small model registry (`ASA_MSST_MODEL`, default `scnet_4stem`) maps an id → `{model_type, config, checkpoint}`. Any failure (missing venv/checkout/checkpoint, non-zero exit, unparseable output) degrades back to Demucs. `separate_stems_backend` is the entry point called from `analyze.py`'s three separation sites (measurement `--separate`, `--pitch-note-only`, `--mt3-only`).

23. **`recommendations_contract.py` + `schemas/recommendations.v1.schema.json`**: Frozen, versioned Phase 2 recommendation contract (ADR 0003). `project_recommendations` normalizes the three Phase 2 device-card arrays (`abletonRecommendations`, `mixAndMasterChain`, `secretSauce.workflowSteps`) into a flat `{device, parameter, value, unit, range, cited_measurements[]}` envelope (`version: "recommendations.v1"`); `validate_envelope` checks it against the committed JSON Schema via `jsonschema` (the real file, not a hand-rolled mirror — see ADR 0001's drift warning). Derived and additive — it never overrides Phase 1 (invariant #1) and admits ONLY cited cards (`cited_measurements.minItems: 1`); uncited cards stay in the raw arrays where the warn-and-keep catalogue gate flags them. `server.py` attaches the validated envelope to the producer_summary interpretation result as a `recommendations` field (degrades to absent on error). TS mirror: `RecommendationsContract` in [src/types/interpretation.ts](apps/ui/src/types/interpretation.ts). CI gate: `tests/test_recommendations_contract.py` (schema validity + projection-validates + round-trip + freeze).

The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details.

**Phase 2 (`POST /api/phase2`, legacy compat):** Uploads audio to Gemini inline if ≤100 MiB, or via the Gemini Files API if larger. Phase 1 JSON is appended to the system prompt from `prompts/phase2_system.txt`. Also relevant: `prompts/stem_summary_system.txt` and `prompts/live12_device_catalog.json` (the *prompt-injected* device catalogue — distinct from the runtime-validation `data/live12_catalogue.json` consumed by `phase2_catalogue_gates.py`). Backend defense-in-depth: `server_phase2.py`'s `_validate_phase2_citation_paths` mirrors the frontend citation-existence check and emits `validationWarnings` when a recommendation cites a Phase 1 path that doesn't exist — it flags invented citations rather than failing, since Phase 1 stays authoritative (invariant #1). `phase2_catalogue_gates.apply_live12_catalogue_gates` runs immediately after for source-catalogue checks (see backend file #20 above).
Expand Down Expand Up @@ -435,6 +437,6 @@ This repo carries parallel guidance for non-Claude agents:
3. **`docs/ARCHITECTURE_STRATEGY.md`** — *why* the three-layer architecture is shaped the way it is.
4. **`GOAL.md`** — the recommendation-proof campaign (north-star goal). `apps/backend/NEEDS.md` is its living status doc; `apps/backend/RECOMMENDATION_VERDICT.md` is the provisional Gemini-vs-deterministic write-up (proxy-render caveats inside).
5. **`docs/history/`** — completed plans and one-shot audits. Past-tense, not living docs.
6. **`docs/adr/`** — Architecture Decision Records. `0001-phase1-json-schema-v1.md` declares the Phase 1 JSON schema v1 stability contract (field-rename policy, CSV column pinning, `publicStatus` collapse). `0002-phase1-loudness-units-v2.md` accepted a breaking unit change: `truePeak` is now dBTP (was linear), `bpmConfidence` is now 0–1 normalized (was raw Essentia ~0–5.32), and a `phase1Version: "phase1.v2"` top-level field was added. Consult both before proposing changes to the Phase 1 payload shape.
6. **`docs/adr/`** — Architecture Decision Records. `0001-phase1-json-schema-v1.md` declares the Phase 1 JSON schema v1 stability contract (field-rename policy, CSV column pinning, `publicStatus` collapse). `0002-phase1-loudness-units-v2.md` accepted a breaking unit change: `truePeak` is now dBTP (was linear), `bpmConfidence` is now 0–1 normalized (was raw Essentia ~0–5.32), and a `phase1Version: "phase1.v2"` top-level field was added. `0003-recommendations-contract-v1.md` freezes the Phase 2 recommendation surface as the versioned JSON Schema `recommendations.v1` — each entry `{device, parameter, value, unit, range, cited_measurements[]}`, validated in CI against `apps/backend/schemas/recommendations.v1.schema.json`. Consult these before proposing changes to the Phase 1 payload shape or the recommendation contract.

When information conflicts: `PURPOSE.md` > `CLAUDE.md` > `GOAL.md` > per-app `AGENTS.md`.
Loading
Loading