From dc93bd856e254016d9ef9177fcc2988e0cf1f28c Mon Sep 17 00:00:00 2001 From: slitty-codes Date: Fri, 5 Jun 2026 18:29:29 +1200 Subject: [PATCH] feat: freeze versioned recommendations.v1 contract for Phase 2 output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 emits device recommendations across three free-shaped arrays (abletonRecommendations, mixAndMasterChain, secretSauce.workflowSteps) with free-text values and no machine-checkable freeze. This adds a normalized, versioned, schema-validated contract for that surface. - schemas/recommendations.v1.schema.json: Draft 2020-12 schema. Envelope {version, recommendations[]}; each entry {device, parameter, value, unit, range, cited_measurements[]}, additionalProperties:false, cited_measurements minItems:1. - recommendations_contract.py: deterministic projection of the three Phase 2 card arrays into the normalized shape, validated against the literal schema file via jsonschema (not a hand-rolled mirror — avoids the drift ADR 0001 warned about). Derived/additive: never overrides Phase 1 (invariant #1); admits only cited cards (invariant #2); unit/range are nullable best-effort since the static Live 12 catalogue carries no min/max (invariant #4). - server.py: attaches the validated envelope to the producer_summary interpretation result; it travels into GET /api/analysis-runs/{run_id} under stages.interpretation.result.recommendations (degrades to absent on error). - interpretation.ts: TS mirror (RecommendationsContract). - ADR 0003 + CLAUDE.md index document the contract and its semver/compat policy. Tests: test_recommendations_contract.py (schema validity, projection-validates, round-trip, freeze), a server wiring test, and a runtime snapshot-survival test. Full backend suite green (1153), tsc --noEmit green. jsonschema==4.26.0 pinned. Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 6 +- apps/backend/recommendations_contract.py | 322 +++++++++++++++ apps/backend/requirements.txt | 9 + .../schemas/recommendations.v1.schema.json | 72 ++++ apps/backend/server.py | 19 + apps/backend/tests/test_analysis_runtime.py | 68 ++++ .../tests/test_recommendations_contract.py | 367 ++++++++++++++++++ apps/backend/tests/test_server.py | 55 +++ apps/ui/src/types/interpretation.ts | 37 ++ docs/adr/0003-recommendations-contract-v1.md | 194 +++++++++ 10 files changed, 1147 insertions(+), 2 deletions(-) create mode 100644 apps/backend/recommendations_contract.py create mode 100644 apps/backend/schemas/recommendations.v1.schema.json create mode 100644 apps/backend/tests/test_recommendations_contract.py create mode 100644 docs/adr/0003-recommendations-contract-v1.md diff --git a/CLAUDE.md b/CLAUDE.md index 268c1f86..f7f4b1c5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -208,7 +208,7 @@ Operational and one-shot scripts live in two places. They are not on the request 2. `packages/loudness-spectro-wasm/` — browser-first WebAssembly DSP (ITU-R BS.1770-5 / EBU R128 loudness, A-weighted spectrum, and spectral-reassignment spectrogram). Rust lifted from [openmeters](https://github.com/httpsworldview/openmeters) and compiled via `wasm-bindgen`, GPL-3.0-or-later. **Phase 1: standalone, not yet imported by `apps/ui` or `apps/backend`.** Has its own Cargo workspace, `npm`/`cargo` build, and EBU/ebur128/pyloudnorm validation layers — see [`packages/loudness-spectro-wasm/README.md`](packages/loudness-spectro-wasm/README.md). When integration lands, the canonical Phase 1 LUFS contract still comes from the Essentia path until this is wired in and proven at parity. (The reassigned spectrogram on the product path today comes from librosa via the `reassigned` spectral-enhancement endpoint, not this package.) 3. `audits/` — dated, automated audit reports (e.g. `nightly-2026-05-19.md`, `phase2-recommendation-surface-2026-05-24.md`, `full-review-2026-05-30.md`). Past-tense paper trail; not imported by either app. (Older one-shot advisory deliverables like the phase 1 audit have been archived under `docs/history/phase1-audit/`.) 4. `incorporations/` — planning docs for incorporating upstream projects (e.g. `forking-plans-2026-05-14.md`, `beat-this-measurement-gate-2026-05-20.md`). Planning notes only; not code. -5. `docs/` — long-form rationale and architectural records. Key contents: `ARCHITECTURE_STRATEGY.md` (three-layer design rationale), `SETUP.md` (first-run setup), `LAYER2_EVALUATION.md` and `POLYPHONIC_TRANSCRIPTION_SPIKE.md` (Layer 2 research), `SAMPLE_GENERATION.md` (Phase 3 design), `adr/` (Architecture Decision Records — `0001-phase1-json-schema-v1.md` declares the v1 schema stability contract; `0002-phase1-loudness-units-v2.md` bumped `truePeak` to dBTP and `bpmConfidence` to normalized 0–1, schema now at `phase1.v2`), and `history/` (completed plans, one-shot audits, and deliverables — past-tense). Read the relevant doc *before* structural changes; do not treat them as living API docs. +5. `docs/` — long-form rationale and architectural records. Key contents: `ARCHITECTURE_STRATEGY.md` (three-layer design rationale), `SETUP.md` (first-run setup), `LAYER2_EVALUATION.md` and `POLYPHONIC_TRANSCRIPTION_SPIKE.md` (Layer 2 research), `SAMPLE_GENERATION.md` (Phase 3 design), `adr/` (Architecture Decision Records — `0001-phase1-json-schema-v1.md` declares the v1 schema stability contract; `0002-phase1-loudness-units-v2.md` bumped `truePeak` to dBTP and `bpmConfidence` to normalized 0–1, schema now at `phase1.v2`; `0003-recommendations-contract-v1.md` freezes the Phase 2 `recommendations` contract (`recommendations.v1`) — a normalized, schema-validated, citation-gated projection of the device cards), and `history/` (completed plans, one-shot audits, and deliverables — past-tense). Read the relevant doc *before* structural changes; do not treat them as living API docs. 6. `tests/ground_truth/` — labeled-corpus fixtures consumed by `scripts/calibrate_confidence.py` (see `tests/ground_truth/README.md`). Not a test suite — each app owns its own (`apps/backend/tests/`, `apps/ui/tests/`). ### Three-Layer Model @@ -293,6 +293,8 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths 22. **`separation_backend.py`**: Selectable Phase 1 stem-separation backend (default-off experiment). `ASA_SEPARATION_BACKEND=msst` swaps torchaudio Hybrid Demucs (`analyze_audio_io.separate_stems`) for a stronger MSST/BS-RoFormer model from a `SUC-DriverOld/MSST-WebUI` checkout, while keeping the `{stem_name: wav_path}` contract (`vocals/bass/drums/other`, 44.1 kHz) unchanged. Because MSST pins deps that conflict with ASA's (`librosa==0.9.2`, `numpy<2`, its own torch), it runs in its **own** venv and ASA shells out to `scripts/msst_separate_runner.py` under `ASA_MSST_PYTHON` — mirroring how `loudness_backend.py` shells out to `measure-cli`. The subprocess boundary also keeps MSST's stdout/logging off `analyze.py`'s JSON contract (tripwire #1) and isolates its `utils`/`inference` packages. A small model registry (`ASA_MSST_MODEL`, default `scnet_4stem`) maps an id → `{model_type, config, checkpoint}`. Any failure (missing venv/checkout/checkpoint, non-zero exit, unparseable output) degrades back to Demucs. `separate_stems_backend` is the entry point called from `analyze.py`'s three separation sites (measurement `--separate`, `--pitch-note-only`, `--mt3-only`). +23. **`recommendations_contract.py` + `schemas/recommendations.v1.schema.json`**: Frozen, versioned Phase 2 recommendation contract (ADR 0003). `project_recommendations` normalizes the three Phase 2 device-card arrays (`abletonRecommendations`, `mixAndMasterChain`, `secretSauce.workflowSteps`) into a flat `{device, parameter, value, unit, range, cited_measurements[]}` envelope (`version: "recommendations.v1"`); `validate_envelope` checks it against the committed JSON Schema via `jsonschema` (the real file, not a hand-rolled mirror — see ADR 0001's drift warning). Derived and additive — it never overrides Phase 1 (invariant #1) and admits ONLY cited cards (`cited_measurements.minItems: 1`); uncited cards stay in the raw arrays where the warn-and-keep catalogue gate flags them. `server.py` attaches the validated envelope to the producer_summary interpretation result as a `recommendations` field (degrades to absent on error). TS mirror: `RecommendationsContract` in [src/types/interpretation.ts](apps/ui/src/types/interpretation.ts). CI gate: `tests/test_recommendations_contract.py` (schema validity + projection-validates + round-trip + freeze). + The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details. **Phase 2 (`POST /api/phase2`, legacy compat):** Uploads audio to Gemini inline if ≤100 MiB, or via the Gemini Files API if larger. Phase 1 JSON is appended to the system prompt from `prompts/phase2_system.txt`. Also relevant: `prompts/stem_summary_system.txt` and `prompts/live12_device_catalog.json` (the *prompt-injected* device catalogue — distinct from the runtime-validation `data/live12_catalogue.json` consumed by `phase2_catalogue_gates.py`). Backend defense-in-depth: `server_phase2.py`'s `_validate_phase2_citation_paths` mirrors the frontend citation-existence check and emits `validationWarnings` when a recommendation cites a Phase 1 path that doesn't exist — it flags invented citations rather than failing, since Phase 1 stays authoritative (invariant #1). `phase2_catalogue_gates.apply_live12_catalogue_gates` runs immediately after for source-catalogue checks (see backend file #20 above). @@ -435,6 +437,6 @@ This repo carries parallel guidance for non-Claude agents: 3. **`docs/ARCHITECTURE_STRATEGY.md`** — *why* the three-layer architecture is shaped the way it is. 4. **`GOAL.md`** — the recommendation-proof campaign (north-star goal). `apps/backend/NEEDS.md` is its living status doc; `apps/backend/RECOMMENDATION_VERDICT.md` is the provisional Gemini-vs-deterministic write-up (proxy-render caveats inside). 5. **`docs/history/`** — completed plans and one-shot audits. Past-tense, not living docs. -6. **`docs/adr/`** — Architecture Decision Records. `0001-phase1-json-schema-v1.md` declares the Phase 1 JSON schema v1 stability contract (field-rename policy, CSV column pinning, `publicStatus` collapse). `0002-phase1-loudness-units-v2.md` accepted a breaking unit change: `truePeak` is now dBTP (was linear), `bpmConfidence` is now 0–1 normalized (was raw Essentia ~0–5.32), and a `phase1Version: "phase1.v2"` top-level field was added. Consult both before proposing changes to the Phase 1 payload shape. +6. **`docs/adr/`** — Architecture Decision Records. `0001-phase1-json-schema-v1.md` declares the Phase 1 JSON schema v1 stability contract (field-rename policy, CSV column pinning, `publicStatus` collapse). `0002-phase1-loudness-units-v2.md` accepted a breaking unit change: `truePeak` is now dBTP (was linear), `bpmConfidence` is now 0–1 normalized (was raw Essentia ~0–5.32), and a `phase1Version: "phase1.v2"` top-level field was added. `0003-recommendations-contract-v1.md` freezes the Phase 2 recommendation surface as the versioned JSON Schema `recommendations.v1` — each entry `{device, parameter, value, unit, range, cited_measurements[]}`, validated in CI against `apps/backend/schemas/recommendations.v1.schema.json`. Consult these before proposing changes to the Phase 1 payload shape or the recommendation contract. When information conflicts: `PURPOSE.md` > `CLAUDE.md` > `GOAL.md` > per-app `AGENTS.md`. diff --git a/apps/backend/recommendations_contract.py b/apps/backend/recommendations_contract.py new file mode 100644 index 00000000..a7ae1870 --- /dev/null +++ b/apps/backend/recommendations_contract.py @@ -0,0 +1,322 @@ +"""Recommendations contract v1 — frozen, versioned, schema-validated projection. + +ASA's Phase 2 (interpretation) layer emits device recommendations across three +free-shaped arrays — ``abletonRecommendations``, ``mixAndMasterChain``, and +``secretSauce.workflowSteps`` — each carrying ``{device, parameter, value, +phase1Fields}`` plus prose. ``value`` is a free-text string ("10 ms", "-18 dB", +"3:1", "Sine") and there is no separate ``unit``/``range``. + +This module freezes a **normalized, machine-actionable view** of those cards: +a flat list of entries shaped ``{device, parameter, value, unit, range, +cited_measurements[]}``, validated against the committed JSON Schema at +``schemas/recommendations.v1.schema.json``. The contract version string is +``recommendations.v1`` (ADR 0003). + +Design properties (all load-bearing): + + 1. **Derived, never authoritative.** The projection only reads Phase 2 output; + it never overrides or re-estimates a Phase 1 measurement (PURPOSE.md + invariant #1). It is additive — the raw Phase 2 arrays are untouched. + 2. **Citation-gated.** A card is admitted ONLY if it cites >=1 Phase 1 + measurement (invariant #2 / schema ``cited_measurements.minItems: 1``). + Uncited cards are excluded *from this view* by design; they remain in the + raw arrays, where the warn-and-keep catalogue gate + (``phase2_catalogue_gates.py``) already flags them. Nothing user-facing is + dropped — exclusion is from the normalized contract only. + 3. **Honest range.** ``range`` is a best-effort working neighborhood derived + from a per-unit tolerance band, emitted only when the value is numeric AND + the unit is known; otherwise ``null``. The static Live 12 catalogue carries + no min/max (see ``data/live12_catalogue.schema.json`` extraction_notes), so + a hard device range is deliberately NOT claimed (invariant #4). + 4. **Validated against the file, not a mirror.** ``validate_envelope`` runs the + real JSON Schema (``jsonschema``) against the committed file — there is no + hand-rolled structural mirror that could drift from the published schema + (the failure mode ADR 0001 warned about). + +Provenance: the value parser and per-unit tolerance bands are a faithful copy of +``recommendation_evaluation.py`` (the research-only recommendation scorer). They +are duplicated here on purpose so this product-path module carries no dependency +on the deletable research harness — the same deliberate cross-module duplication +pattern as ``audio_mime.py`` / ``audioFile.ts``. Keep the two band tables in +sync; if they ever need to diverge, that is a scoring vs. contract decision worth +a comment in both files. +""" + +from __future__ import annotations + +import json +import math +import re +from dataclasses import dataclass +from functools import lru_cache +from pathlib import Path +from typing import Any, Mapping + +import jsonschema + +CONTRACT_VERSION = "recommendations.v1" + +SCHEMA_PATH = Path(__file__).resolve().parent / "schemas" / "recommendations.v1.schema.json" + + +# --------------------------------------------------------------------------- +# Value parsing + per-unit tolerance bands. +# Faithful copy of recommendation_evaluation.py — see module docstring. +# Internal unit tokens are lowercase ("hz", "db", ...); the contract maps them +# to display units ("Hz", "dB", ...) via _DISPLAY_UNIT below. +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class ParsedValue: + """A numeric magnitude extracted from a free-text value string.""" + + number: float + unit: str # normalized: "hz", "db", "ms", "s", "ratio", "pct", "st", "" + + +@dataclass(frozen=True) +class UnitBand: + abs_tol: float | None + rel_tol: float | None + neutral: float + + +# tolerance is absolute in the unit unless `rel_tol` is set (fraction of value). +UNIT_BANDS: dict[str, UnitBand] = { + "hz": UnitBand(abs_tol=None, rel_tol=0.20, neutral=1000.0), # +/-20% + "db": UnitBand(abs_tol=3.0, rel_tol=None, neutral=0.0), # +/-3 dB + "ms": UnitBand(abs_tol=None, rel_tol=0.30, neutral=20.0), # +/-30% + "s": UnitBand(abs_tol=None, rel_tol=0.30, neutral=1.0), # +/-30% + "ratio": UnitBand(abs_tol=1.0, rel_tol=None, neutral=1.0), # +/-1:1 + "pct": UnitBand(abs_tol=15.0, rel_tol=None, neutral=50.0), # +/-15% + "st": UnitBand(abs_tol=1.0, rel_tol=None, neutral=0.0), # +/-1 semitone + "": UnitBand(abs_tol=None, rel_tol=0.20, neutral=0.0), # unitless: +/-20% +} + +# Internal token -> display unit. "" (unitless) maps to None (absent here). +_DISPLAY_UNIT: dict[str, str] = { + "hz": "Hz", + "db": "dB", + "ms": "ms", + "s": "s", + "ratio": "ratio", + "pct": "%", + "st": "st", +} + +# Longer unit tokens must precede their prefixes in the alternation (`st` before +# `s`, `sec`/`semitones` before `s`, `ms` before `s`) so regex first-match wins. +_VALUE_RE = re.compile( + r"(-?\d+(?:\.\d+)?)\s*" + r"(k?hz|db|ms|sec|semitones?|st|s|%|:1|x)?", + re.IGNORECASE, +) + + +def _normalize_unit(raw: str) -> str: + if raw in ("khz", "hz"): + return "hz" + if raw == "db": + return "db" + if raw == "ms": + return "ms" + if raw in ("s", "sec"): + return "s" + if raw == "%": + return "pct" + if raw in ("st", "semitone", "semitones"): + return "st" + if raw in (":1", "x"): + return "ratio" + return "" + + +def parse_value(text: Any) -> ParsedValue | None: + """Extract a numeric magnitude + normalized unit from a value string. + + Handles ``"4 kHz"``, ``"-15 dB"``, ``"200 ms"``, ``"3:1"``, ``"30%"``, + ``"0.6"``, ``"+12st"``. Returns ``None`` for non-numeric values (e.g. + ``"Sine"``, ``"Auto"``). + """ + if text is None: + return None + if isinstance(text, bool): # bool is an int subclass — exclude. + return None + if isinstance(text, (int, float)): + if not math.isfinite(float(text)): + return None + return ParsedValue(number=float(text), unit="") + s = str(text).strip() + if not s: + return None + # Ratio form "3:1" -> 3.0 ratio. + ratio_match = re.search(r"(-?\d+(?:\.\d+)?)\s*:\s*1\b", s) + if ratio_match: + return ParsedValue(number=float(ratio_match.group(1)), unit="ratio") + match = _VALUE_RE.search(s) + if not match: + return None + number = float(match.group(1)) + raw_unit = (match.group(2) or "").lower() + unit = _normalize_unit(raw_unit) + if unit == "hz" and raw_unit.startswith("k"): + number *= 1000.0 + return ParsedValue(number=number, unit=unit) + + +def _derive_range(number: float, token: str) -> list[float] | None: + """Suggested working range [min, max] from the per-unit tolerance band. + + Returns ``None`` for the unitless band — a range without a unit is not + meaningful enough to publish (the contract emits ``range: null`` there). + """ + if token not in _DISPLAY_UNIT: + return None + band = UNIT_BANDS.get(token) + if band is None: + return None + if band.abs_tol is not None: + tol = band.abs_tol + else: + tol = abs(number) * (band.rel_tol or 0.0) + return [round(number - tol, 4), round(number + tol, 4)] + + +# --------------------------------------------------------------------------- +# Projection: Phase 2 result -> recommendations.v1 envelope. +# --------------------------------------------------------------------------- + + +def _clean_str(value: Any) -> str: + return value.strip() if isinstance(value, str) else "" + + +def _citations(raw: Any) -> list[str]: + if not isinstance(raw, list): + return [] + return [s.strip() for s in raw if isinstance(s, str) and s.strip()] + + +def _project_card(card: Any) -> dict[str, Any] | None: + """Project one Phase 2 device card into a contract entry, or ``None``. + + Returns ``None`` (card excluded from the contract) when the card lacks a + usable device/parameter, has no usable value, or — critically — carries no + Phase 1 citation. Exclusion here is from the normalized view only; the card + remains in its raw Phase 2 array. + """ + if not isinstance(card, Mapping): + return None + device = _clean_str(card.get("device")) + parameter = _clean_str(card.get("parameter")) + if not device or not parameter: + return None + citations = _citations(card.get("phase1Fields")) + if not citations: # citation-chain invariant — uncited cards are not admitted + return None + + parsed = parse_value(card.get("value")) + if parsed is not None: + value: Any = parsed.number + unit = _DISPLAY_UNIT.get(parsed.unit) # None for the unitless token + rng = _derive_range(parsed.number, parsed.unit) if unit is not None else None + else: + value = _clean_str(card.get("value")) + if not value: # no usable value -> cannot form a valid entry + return None + unit = None + rng = None + + return { + "device": device, + "parameter": parameter, + "value": value, + "unit": unit, + "range": rng, + "cited_measurements": citations, + } + + +# Phase 2 list-valued sources that carry device cards. secretSauce.workflowSteps +# is nested and handled separately. +_LIST_SOURCES = ("abletonRecommendations", "mixAndMasterChain") + + +def project_recommendations(phase2_result: Any) -> dict[str, Any]: + """Project a Phase 2 result dict into a ``recommendations.v1`` envelope. + + Deterministic and side-effect-free. The returned envelope always validates + against the committed schema (an empty ``recommendations`` list is valid). + """ + recs: list[dict[str, Any]] = [] + if isinstance(phase2_result, Mapping): + for key in _LIST_SOURCES: + items = phase2_result.get(key) + if isinstance(items, list): + for item in items: + entry = _project_card(item) + if entry is not None: + recs.append(entry) + secret = phase2_result.get("secretSauce") + if isinstance(secret, Mapping): + steps = secret.get("workflowSteps") + if isinstance(steps, list): + for item in steps: + entry = _project_card(item) + if entry is not None: + recs.append(entry) + return {"version": CONTRACT_VERSION, "recommendations": recs} + + +# --------------------------------------------------------------------------- +# Validation against the committed schema FILE (not a hand-rolled mirror). +# --------------------------------------------------------------------------- + + +@lru_cache(maxsize=1) +def load_schema() -> dict[str, Any]: + """Load and cache the committed recommendations.v1 JSON Schema.""" + with SCHEMA_PATH.open("r", encoding="utf-8") as handle: + return json.load(handle) + + +@lru_cache(maxsize=1) +def _validator() -> jsonschema.protocols.Validator: + schema = load_schema() + validator_cls = jsonschema.validators.validator_for(schema) + # Fail fast if the committed schema is itself malformed. + validator_cls.check_schema(schema) + return validator_cls(schema) + + +def validate_envelope(envelope: Mapping[str, Any]) -> None: + """Validate an envelope against the committed schema. Raises on the first + violation (``jsonschema.exceptions.ValidationError``).""" + _validator().validate(envelope) + + +def iter_validation_errors(envelope: Mapping[str, Any]) -> list[str]: + """Return human-readable messages for every schema violation (empty if + valid). Used by tests and the runtime degrade path.""" + errors = sorted( + _validator().iter_errors(envelope), key=lambda err: str(err.json_path) + ) + return [f"{err.json_path}: {err.message}" for err in errors] + + +def build_validated_recommendations(phase2_result: Any) -> dict[str, Any] | None: + """Project and validate in one call for runtime use. + + Returns the validated envelope, or ``None`` if projection/validation fails. + Degrade-on-error mirrors the catalogue-gate / loudness-backend philosophy: + a derived view must never break the response it rides on. The projection is + deterministic, so a ``None`` here signals a genuine bug worth logging at the + call site, not an expected outcome. + """ + try: + envelope = project_recommendations(phase2_result) + validate_envelope(envelope) + return envelope + except Exception: # noqa: BLE001 — additive view must not break the response + return None diff --git a/apps/backend/requirements.txt b/apps/backend/requirements.txt index 91fa9f57..b11d7b01 100644 --- a/apps/backend/requirements.txt +++ b/apps/backend/requirements.txt @@ -87,3 +87,12 @@ pytheory>=0.42,<1.0 # offline render-to-buffer synthesis, so pyaudio is unneeded; 1.3.4 is the # latest release that depends on numpy alone. pyfluidsynth>=1.3,<1.3.6 +# Recommendations contract v1 (recommendations_contract.py): validate the +# normalized Phase 2 recommendation envelope against the committed JSON Schema +# (schemas/recommendations.v1.schema.json) — the CI gate for ADR 0003. Pure +# Python; the three lines below are its transitive deps (attrs is already +# pinned above). See docs/adr/0003-recommendations-contract-v1.md. +jsonschema==4.26.0 +jsonschema-specifications==2025.9.1 +referencing==0.37.0 +rpds-py==2026.5.1 diff --git a/apps/backend/schemas/recommendations.v1.schema.json b/apps/backend/schemas/recommendations.v1.schema.json new file mode 100644 index 00000000..f1731435 --- /dev/null +++ b/apps/backend/schemas/recommendations.v1.schema.json @@ -0,0 +1,72 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://ableton-sonic-analyzer.local/schemas/recommendations.v1.schema.json", + "title": "ASA Recommendations Contract v1", + "description": "Frozen, versioned contract for ASA's normalized recommendation surface. Each entry is a deterministic, server-side projection of a Phase 2 (interpretation) device card into a flat, machine-actionable shape. The contract is a DERIVED VIEW: it never overrides Phase 1 measurements (PURPOSE.md invariant #1) and only admits cards that cite at least one Phase 1 measurement (invariant #2). Uncited cards are excluded by design and remain in the raw Phase 2 arrays (mixAndMasterChain / abletonRecommendations / secretSauce.workflowSteps), where the warn-and-keep catalogue gate flags them. Version semantics follow ADR 0003: additive changes (new optional entry field, new unit enum value) are minor; renaming/removing/retyping a field or changing range/unit semantics is a major bump to recommendations.v2.", + "type": "object", + "additionalProperties": false, + "required": ["version", "recommendations"], + "properties": { + "version": { + "type": "string", + "const": "recommendations.v1", + "description": "Contract version identifier. Mirrors the phase1Version convention (ADR 0002). A breaking change bumps this to recommendations.v2." + }, + "recommendations": { + "type": "array", + "description": "Zero or more normalized, measurement-cited recommendations. Empty is valid: a track may yield no cited device cards.", + "items": { "$ref": "#/$defs/recommendation" } + } + }, + "$defs": { + "recommendation": { + "type": "object", + "additionalProperties": false, + "required": ["device", "parameter", "value", "unit", "range", "cited_measurements"], + "properties": { + "device": { + "type": "string", + "minLength": 1, + "description": "Ableton Live 12 device name as recommended (e.g. \"Glue Compressor\", \"EQ Eight\"). Not catalogue-rewritten — the value is whatever Phase 2 emitted, trimmed." + }, + "parameter": { + "type": "string", + "minLength": 1, + "description": "Device parameter name as recommended (e.g. \"Attack\", \"Threshold\"). Not catalogue-rewritten." + }, + "value": { + "description": "Normalized value. A number when a single magnitude was parsed from the Phase 2 value string (\"10 ms\" -> 10, \"-18 dB\" -> -18, \"3:1\" -> 3); otherwise the original non-numeric string (\"Sine\", \"4-Pole\", \"Auto\").", + "oneOf": [ + { "type": "string", "minLength": 1 }, + { "type": "number" } + ] + }, + "unit": { + "description": "Normalized unit for a numeric value, or null when the value is non-numeric / unitless. \"ratio\" denotes a compression-style ratio (value 3 == \"3:1\").", + "oneOf": [ + { "type": "null" }, + { "type": "string", "enum": ["Hz", "dB", "ms", "s", "ratio", "%", "st"] } + ] + }, + "range": { + "description": "Best-effort suggested working range [min, max] for the value, derived from a per-unit tolerance band around the recommended value. Null whenever no band applies (non-numeric value or unknown unit). This is a measurement-informed neighborhood, NOT a Live device hard-limit: the static Live 12 catalogue carries no min/max (see data/live12_catalogue.schema.json extraction_notes), so a hard device range cannot be supplied here. Honors invariant #4 by not pretending a precise range exists where it does not.", + "oneOf": [ + { "type": "null" }, + { + "type": "array", + "items": { "type": "number" }, + "minItems": 2, + "maxItems": 2 + } + ] + }, + "cited_measurements": { + "type": "array", + "minItems": 1, + "items": { "type": "string", "minLength": 1 }, + "description": "Dotted Phase 1 measurement paths that justify this recommendation (projected from the source card's phase1Fields). minItems:1 encodes the citation-chain invariant (#2): a recommendation with no Phase 1 citation is never admitted to this contract." + } + } + } + } +} diff --git a/apps/backend/server.py b/apps/backend/server.py index 4b95ff32..8d492209 100644 --- a/apps/backend/server.py +++ b/apps/backend/server.py @@ -147,6 +147,8 @@ apply_live12_catalogue_gates, ) +from recommendations_contract import build_validated_recommendations + import server_samples @@ -2007,6 +2009,23 @@ def _generate_files_api() -> Any: "requestId": request_id, } ] + # Frozen recommendations.v1 contract (ADR 0003): attach a normalized, + # citation-gated projection of the Phase 2 device cards as an additive + # `recommendations` field on the interpretation result. Derived, never + # authoritative — it cannot override Phase 1 (invariant #1). Only the + # producer_summary profile carries device cards (stem_summary does not), + # and it degrades to absent on any error (build_validated_recommendations + # returns None), exactly like the catalogue gate above. + if profile_id == "producer_summary" and isinstance(interpretation_result, dict): + recommendations_envelope = build_validated_recommendations(interpretation_result) + if recommendations_envelope is not None: + interpretation_result["recommendations"] = recommendations_envelope + else: + logger.warning( + "Recommendations contract projection produced no valid " + "envelope (request_id=%s)", + request_id, + ) validation_warnings = ( parse_validation_warnings + style_profile_warnings diff --git a/apps/backend/tests/test_analysis_runtime.py b/apps/backend/tests/test_analysis_runtime.py index 2080ae78..f3385782 100644 --- a/apps/backend/tests/test_analysis_runtime.py +++ b/apps/backend/tests/test_analysis_runtime.py @@ -638,6 +638,74 @@ def test_interpretation_attempts_store_grounding_columns(self) -> None: self.assertEqual(row["grounded_measurement_output_id"], grounding["measurementOutputId"]) self.assertEqual(row["grounded_pitch_note_attempt_id"], pitch_note_attempt_id) + def test_run_snapshot_surfaces_recommendations_contract_verbatim(self) -> None: + """ADR 0003: the recommendations.v1 envelope server.py attaches to the + interpretation result must survive persistence and reach the run + snapshot (the payload behind GET /api/analysis-runs/{run_id}) unstripped. + This closes the chain of custody the wiring test in test_server.py starts + — that one stops at the helper return; this one checks the snapshot.""" + import recommendations_contract as rc + + runtime = self._runtime() + created = runtime.create_run( + filename="track.mp3", + content=b"fake-audio", + mime_type="audio/mpeg", + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="async", + interpretation_profile="producer_summary", + interpretation_model="gemini-2.5-flash", + ) + runtime.complete_measurement( + created["runId"], + payload={"bpm": 128, "key": "A minor", "durationSeconds": 184.2}, + provenance={"schemaVersion": "measure.v1", "engineVersion": "analyze.py"}, + diagnostics={"backendDurationMs": 1200}, + ) + attempt_id = runtime.create_interpretation_attempt( + created["runId"], + profile_id="producer_summary", + model_name="gemini-2.5-flash", + status="queued", + ) + grounding = runtime.get_interpretation_grounding(created["runId"]) + + envelope = rc.build_validated_recommendations( + { + "abletonRecommendations": [ + { + "device": "Glue Compressor", + "parameter": "Attack", + "value": "10 ms", + "phase1Fields": ["dynamicsDetail.crestFactor"], + } + ] + } + ) + self.assertIsNotNone(envelope) + runtime.complete_interpretation_attempt( + attempt_id, + result={"trackCharacter": "Grounded summary", "recommendations": envelope}, + provenance={}, + diagnostics={"backendDurationMs": 250}, + grounded_measurement_output_id=grounding["measurementOutputId"], + grounded_pitch_note_attempt_id=grounding["pitchNoteAttemptId"], + ) + + interpretation = runtime.get_run(created["runId"])["stages"]["interpretation"] + # Preferred-attempt result carries the contract verbatim ... + self.assertEqual(interpretation["result"]["recommendations"], envelope) + self.assertEqual( + interpretation["result"]["recommendations"]["version"], + "recommendations.v1", + ) + # ... as does the per-profile result. + self.assertEqual( + interpretation["profiles"]["producer_summary"]["result"]["recommendations"], + envelope, + ) + def test_interpretation_grounding_exposes_mt3_result(self) -> None: """F3: get_interpretation_grounding surfaces a completed MT3 attempt so _execute_interpretation_attempt can forward it to Gemini. When no MT3 diff --git a/apps/backend/tests/test_recommendations_contract.py b/apps/backend/tests/test_recommendations_contract.py new file mode 100644 index 00000000..62b61816 --- /dev/null +++ b/apps/backend/tests/test_recommendations_contract.py @@ -0,0 +1,367 @@ +"""Recommendations contract v1 — schema, projection, and round-trip gate. + +This is the CI gate for ADR 0003. It proves three things the goal's DoD names: + + 1. the committed schema (schemas/recommendations.v1.schema.json) is a valid + JSON Schema and is the artifact actually validated against — not a + hand-rolled mirror that could drift from it; + 2. every projected Phase 2 result validates against that schema (the + "validate all Phase-2 output against it" requirement); and + 3. a projected envelope survives a JSON round-trip unchanged. + +It also pins the contract's invariants: citation-gating (uncited cards are +excluded), value/unit normalization, best-effort range, and the schema's +freeze (additionalProperties:false, cited_measurements minItems:1, version const). +""" + +import json +import unittest + +import jsonschema + +import recommendations_contract as rc + + +# A realistic Phase 2 result covering all three recommendation sources plus the +# prose fields the projection must ignore. Every device card here is cited. +PHASE2_FIXTURE = { + "trackCharacter": "Driving, sidechained techno with a saturated low end.", + "sonicElements": { + "kick": "Punchy analog kick, fundamental ~55 Hz.", + "bass": "Rolling reese bass, sidechained to the kick.", + "melodicArp": "Sparse detuned stab.", + "grooveAndTiming": "Straight 4/4, tight quantize.", + "effectsAndTexture": "Plate reverb on the stab return.", + }, + "abletonRecommendations": [ + { + "device": "Glue Compressor", + "category": "DYNAMICS", + "trackContext": "Kick bus", + "parameter": "Attack", + "value": "10 ms", + "reason": "Preserve the kick transient given crest factor 8.2 dB.", + "phase1Fields": ["dynamicsDetail.crestFactor"], + }, + { + "device": "EQ Eight", + "category": "EQ", + "trackContext": "Bass", + "parameter": "Band 1 Frequency", + "value": "4 kHz", + "reason": "Carve presence to seat the bass under the kick.", + "phase1Fields": ["spectralBalance.lowBass", "spectralBalance.mids"], + }, + { + "device": "Operator", + "category": "SYNTHESIS", + "trackContext": "Bass", + "parameter": "Oscillator Waveform", + "value": "Sine", + "reason": "Sub energy concentrated at the fundamental.", + "phase1Fields": ["kickDetail.fundamentalHz"], + }, + ], + "mixAndMasterChain": [ + { + "order": 1, + "device": "Glue Compressor", + "trackContext": "Master", + "parameter": "Ratio", + "value": "3:1", + "reason": "Gentle master glue; dynamics are already controlled.", + "phase1Fields": ["dynamicsDetail.crestFactor"], + }, + { + "order": 2, + "device": "Limiter", + "trackContext": "Master", + "parameter": "Ceiling", + "value": "-1 dB", + "reason": "True-peak headroom for streaming.", + "phase1Fields": ["truePeak"], + }, + ], + "secretSauce": { + "title": "The pump", + "explanation": "Sidechain everything to the kick.", + "implementationSteps": ["Route kick to a send", "Trigger the compressor"], + "workflowSteps": [ + { + "step": 1, + "trackContext": "Bass", + "device": "Compressor", + "parameter": "Threshold", + "value": "-24 dB", + "instruction": "Sidechain the bass to the kick.", + "measurementJustification": "Sidechain depth measured at -6 dB.", + "phase1Fields": ["sidechainDetail.depthDb"], + } + ], + }, +} + + +class SchemaIsValidTests(unittest.TestCase): + def test_committed_schema_is_a_valid_json_schema(self): + schema = rc.load_schema() + # Raises SchemaError if the committed file is not a valid Draft 2020-12 + # schema. This is what makes "validate against the file" trustworthy. + validator_cls = jsonschema.validators.validator_for(schema) + validator_cls.check_schema(schema) + + def test_schema_version_const_matches_module_constant(self): + # Drift guard: the version in the schema file and the module must agree. + schema = rc.load_schema() + self.assertEqual( + schema["properties"]["version"]["const"], rc.CONTRACT_VERSION + ) + + def test_validator_actually_rejects_bad_input(self): + # Guards against a no-op validator: a missing required field must fail. + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope({"version": rc.CONTRACT_VERSION}) + + +class ProjectionValidatesTests(unittest.TestCase): + def test_projected_fixture_validates_against_committed_schema(self): + envelope = rc.project_recommendations(PHASE2_FIXTURE) + # Must not raise. + rc.validate_envelope(envelope) + self.assertEqual(envelope["version"], "recommendations.v1") + # 3 abletonRecommendations + 2 mixAndMasterChain + 1 workflowStep. + self.assertEqual(len(envelope["recommendations"]), 6) + + def test_empty_phase2_yields_valid_empty_envelope(self): + envelope = rc.project_recommendations({}) + rc.validate_envelope(envelope) + self.assertEqual(envelope["recommendations"], []) + + def test_non_dict_input_is_safe(self): + for junk in (None, [], "x", 7): + envelope = rc.project_recommendations(junk) + rc.validate_envelope(envelope) + self.assertEqual(envelope["recommendations"], []) + + def test_projection_is_deterministic(self): + a = rc.project_recommendations(PHASE2_FIXTURE) + b = rc.project_recommendations(PHASE2_FIXTURE) + self.assertEqual(a, b) + + +class RoundTripTests(unittest.TestCase): + def test_envelope_survives_json_round_trip_unchanged(self): + envelope = rc.project_recommendations(PHASE2_FIXTURE) + rc.validate_envelope(envelope) + + serialized = json.dumps(envelope, sort_keys=True) + reloaded = json.loads(serialized) + + # Still valid after a full serialize -> parse cycle ... + rc.validate_envelope(reloaded) + # ... structurally identical ... + self.assertEqual(reloaded, envelope) + # ... and serialization is stable (re-dump matches). + self.assertEqual(json.dumps(reloaded, sort_keys=True), serialized) + + +class CitationGatingTests(unittest.TestCase): + def test_uncited_cards_are_excluded(self): + phase2 = { + "abletonRecommendations": [ + {"device": "EQ Eight", "parameter": "Gain", "value": "+3 dB"}, + { + "device": "EQ Eight", + "parameter": "Gain", + "value": "+3 dB", + "phase1Fields": [], + }, + { + "device": "EQ Eight", + "parameter": "Gain", + "value": "+3 dB", + "phase1Fields": ["spectralBalance.highs"], + }, + ] + } + envelope = rc.project_recommendations(phase2) + rc.validate_envelope(envelope) + # Only the third (cited) card survives. + self.assertEqual(len(envelope["recommendations"]), 1) + self.assertEqual( + envelope["recommendations"][0]["cited_measurements"], + ["spectralBalance.highs"], + ) + + def test_card_missing_device_or_parameter_is_excluded(self): + phase2 = { + "abletonRecommendations": [ + {"parameter": "Gain", "value": "1", "phase1Fields": ["bpm"]}, + {"device": "EQ Eight", "value": "1", "phase1Fields": ["bpm"]}, + ] + } + envelope = rc.project_recommendations(phase2) + self.assertEqual(envelope["recommendations"], []) + + def test_blank_citation_strings_are_dropped(self): + phase2 = { + "abletonRecommendations": [ + { + "device": "EQ Eight", + "parameter": "Gain", + "value": "1", + "phase1Fields": [" ", "", "spectralBalance.highs", " "], + } + ] + } + envelope = rc.project_recommendations(phase2) + self.assertEqual( + envelope["recommendations"][0]["cited_measurements"], + ["spectralBalance.highs"], + ) + + +class ValueUnitRangeTests(unittest.TestCase): + def _project_one(self, value): + phase2 = { + "abletonRecommendations": [ + { + "device": "D", + "parameter": "P", + "value": value, + "phase1Fields": ["bpm"], + } + ] + } + return rc.project_recommendations(phase2)["recommendations"][0] + + def test_milliseconds(self): + entry = self._project_one("10 ms") + self.assertEqual(entry["value"], 10.0) + self.assertEqual(entry["unit"], "ms") + self.assertEqual(entry["range"], [7.0, 13.0]) + + def test_decibels(self): + entry = self._project_one("-18 dB") + self.assertEqual(entry["value"], -18.0) + self.assertEqual(entry["unit"], "dB") + self.assertEqual(entry["range"], [-21.0, -15.0]) + + def test_kilohertz_normalizes_to_hz(self): + entry = self._project_one("4 kHz") + self.assertEqual(entry["value"], 4000.0) + self.assertEqual(entry["unit"], "Hz") + self.assertEqual(entry["range"], [3200.0, 4800.0]) + + def test_ratio(self): + entry = self._project_one("3:1") + self.assertEqual(entry["value"], 3.0) + self.assertEqual(entry["unit"], "ratio") + self.assertEqual(entry["range"], [2.0, 4.0]) + + def test_percent(self): + entry = self._project_one("30%") + self.assertEqual(entry["value"], 30.0) + self.assertEqual(entry["unit"], "%") + self.assertEqual(entry["range"], [15.0, 45.0]) + + def test_semitones(self): + entry = self._project_one("+12 st") + self.assertEqual(entry["value"], 12.0) + self.assertEqual(entry["unit"], "st") + self.assertEqual(entry["range"], [11.0, 13.0]) + + def test_non_numeric_value_has_null_unit_and_range(self): + entry = self._project_one("Sine") + self.assertEqual(entry["value"], "Sine") + self.assertIsNone(entry["unit"]) + self.assertIsNone(entry["range"]) + + def test_unitless_number_has_null_unit_and_range(self): + entry = self._project_one("0.6") + self.assertEqual(entry["value"], 0.6) + self.assertIsNone(entry["unit"]) + self.assertIsNone(entry["range"]) + + +class SchemaFreezeTests(unittest.TestCase): + """The schema must reject anything outside the frozen v1 shape.""" + + def _valid_entry(self): + return { + "device": "EQ Eight", + "parameter": "Gain", + "value": 3.0, + "unit": "dB", + "range": [0.0, 6.0], + "cited_measurements": ["spectralBalance.highs"], + } + + def _envelope(self, entry): + return {"version": rc.CONTRACT_VERSION, "recommendations": [entry]} + + def test_valid_entry_passes(self): + rc.validate_envelope(self._envelope(self._valid_entry())) + + def test_empty_citation_array_is_rejected(self): + entry = self._valid_entry() + entry["cited_measurements"] = [] + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(self._envelope(entry)) + + def test_additional_property_is_rejected(self): + entry = self._valid_entry() + entry["reason"] = "not part of the frozen contract" + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(self._envelope(entry)) + + def test_missing_required_field_is_rejected(self): + entry = self._valid_entry() + del entry["range"] # range is required (may be null, but must be present) + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(self._envelope(entry)) + + def test_unknown_unit_token_is_rejected(self): + entry = self._valid_entry() + entry["unit"] = "decibels" # not in the unit enum + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(self._envelope(entry)) + + def test_wrong_version_string_is_rejected(self): + envelope = self._envelope(self._valid_entry()) + envelope["version"] = "recommendations.v2" + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(envelope) + + def test_range_must_be_two_numbers(self): + entry = self._valid_entry() + entry["range"] = [1.0, 2.0, 3.0] + with self.assertRaises(jsonschema.exceptions.ValidationError): + rc.validate_envelope(self._envelope(entry)) + + def test_null_unit_and_range_are_accepted(self): + entry = self._valid_entry() + entry["value"] = "Sine" + entry["unit"] = None + entry["range"] = None + rc.validate_envelope(self._envelope(entry)) + + +class BuildValidatedHelperTests(unittest.TestCase): + def test_build_validated_returns_envelope(self): + envelope = rc.build_validated_recommendations(PHASE2_FIXTURE) + self.assertIsNotNone(envelope) + self.assertEqual(envelope["version"], "recommendations.v1") + + def test_iter_validation_errors_empty_when_valid(self): + envelope = rc.project_recommendations(PHASE2_FIXTURE) + self.assertEqual(rc.iter_validation_errors(envelope), []) + + def test_iter_validation_errors_reports_problems(self): + bad = {"version": "wrong", "recommendations": [{"device": "X"}]} + errors = rc.iter_validation_errors(bad) + self.assertTrue(errors) + + +if __name__ == "__main__": + unittest.main() diff --git a/apps/backend/tests/test_server.py b/apps/backend/tests/test_server.py index 3c92554e..1e3c1a75 100644 --- a/apps/backend/tests/test_server.py +++ b/apps/backend/tests/test_server.py @@ -845,6 +845,61 @@ def test_run_interpretation_request_with_profile_config_surfaces_salvage_warning self.assertEqual(warnings[0]["originalValue"], "SYNTHESIS") self.assertEqual(warnings[0]["coercedValue"], "SOUND_DESIGN") + def test_run_interpretation_request_attaches_recommendations_contract(self) -> None: + # ADR 0003: the interpretation result must carry the frozen + # recommendations.v1 contract as an additive, derived field. + import recommendations_contract as rc + + payload = _valid_phase2_result() + mock_response = unittest.mock.MagicMock() + mock_response.text = json.dumps(payload) + mock_model = unittest.mock.MagicMock() + mock_model.generate_content.return_value = mock_response + mock_client = unittest.mock.MagicMock() + mock_client.models = mock_model + with tempfile.TemporaryDirectory(prefix="asa_phase2_reccontract_") as temp_dir: + audio_path = Path(temp_dir) / "track.wav" + audio_path.write_bytes(b"fake-audio") + profile_config = server._resolve_interpretation_profile_config("producer_summary") + with ( + patch.object(server, "_GENAI_AVAILABLE", True), + patch.dict(server.os.environ, {"GEMINI_API_KEY": "fake-key"}), + patch.object(server, "_genai") as mock_genai, + patch.object(server, "_genai_types") as mock_genai_types, + ): + mock_genai.Client.return_value = mock_client + mock_genai_types.GenerateContentConfig.return_value = unittest.mock.MagicMock() + execution = server._run_interpretation_request_with_profile_config( + source_path=str(audio_path), + filename=audio_path.name, + file_size_bytes=audio_path.stat().st_size, + profile_id="producer_summary", + profile_config=profile_config, + measurement_result={"bpm": 128}, + pitch_note_result=None, + grounding_metadata={"profileId": "producer_summary"}, + model_name="gemini-3.1-pro-preview", + request_id="rec-contract-test", + ) + + self.assertTrue(execution["ok"]) + contract = execution["interpretationResult"]["recommendations"] + self.assertEqual(contract["version"], "recommendations.v1") + # Re-validate the live-attached envelope against the committed schema. + rc.validate_envelope(contract) + # mixAndMasterChain + secretSauce.workflowStep + abletonRecommendation, + # all three cited in _valid_phase2_result(). + self.assertEqual(len(contract["recommendations"]), 3) + for entry in contract["recommendations"]: + self.assertTrue(entry["cited_measurements"]) # every entry is cited + # Spot-check the parsed master EQ band ("-1.5 dB @ 35 Hz" -> -1.5 dB). + eq_band = next( + e for e in contract["recommendations"] if e["device"] == "EQ Eight" + ) + self.assertEqual(eq_band["value"], -1.5) + self.assertEqual(eq_band["unit"], "dB") + self.assertEqual(eq_band["cited_measurements"], ["spectralBalance.subBass"]) + def test_finalize_style_profile_authoritative_measurements_overwrites_mismatches(self) -> None: interpretation_result = _valid_phase2_result() interpretation_result["styleProfile"] = _valid_style_profile() diff --git a/apps/ui/src/types/interpretation.ts b/apps/ui/src/types/interpretation.ts index 60cefe03..d39c4c47 100644 --- a/apps/ui/src/types/interpretation.ts +++ b/apps/ui/src/types/interpretation.ts @@ -135,6 +135,38 @@ export interface StyleProfile { authoritativeMeasurements: StyleProfileAuthoritativeMeasurements; } +export type RecommendationUnit = + | "Hz" + | "dB" + | "ms" + | "s" + | "ratio" + | "%" + | "st"; + +/** + * One entry of the frozen recommendations.v1 contract (ADR 0003): a normalized, + * citation-gated projection of a Phase 2 device card into a flat, machine- + * actionable shape. `value` is a number when a magnitude was parsed ("10 ms" -> + * 10), else the original non-numeric string ("Sine"); `unit`/`range` are null + * when no numeric magnitude/known unit applies. Mirrors the backend projection + * in apps/backend/recommendations_contract.py and the JSON Schema at + * apps/backend/schemas/recommendations.v1.schema.json — keep all three in sync. + */ +export interface RecommendationContractEntry { + device: string; + parameter: string; + value: string | number; + unit: RecommendationUnit | null; + range: [number, number] | null; + cited_measurements: string[]; +} + +export interface RecommendationsContract { + version: "recommendations.v1"; + recommendations: RecommendationContractEntry[]; +} + export interface Phase2Result { trackCharacter: string; projectSetup?: Phase2ProjectSetup; @@ -197,6 +229,11 @@ export interface Phase2Result { reason: string; }[]; abletonRecommendations: AbletonRecommendation[]; + // Frozen recommendations.v1 contract (ADR 0003) — additive, derived, + // citation-gated normalization of the device cards above. Attached server-side + // for the producer_summary profile; optional because older stored results and + // the stem_summary profile do not carry it. + recommendations?: RecommendationsContract; } export interface StemSummaryBar { diff --git a/docs/adr/0003-recommendations-contract-v1.md b/docs/adr/0003-recommendations-contract-v1.md new file mode 100644 index 00000000..398a53ea --- /dev/null +++ b/docs/adr/0003-recommendations-contract-v1.md @@ -0,0 +1,194 @@ +# ADR 0003 — Recommendations Contract v1 + +**Status:** Accepted +**Date:** 2026-06-05 +**Supersedes:** — +**Anchor:** [PURPOSE.md](../../PURPOSE.md) quality invariants #1 (measurement authority) and #2 (citation chain). + +## Context + +ASA's Phase 2 (interpretation) layer emits device recommendations across three +free-shaped arrays in the `Phase2Result` payload: + +1. `abletonRecommendations[]` — general device cards, carry `category` + `trackContext`. +2. `mixAndMasterChain[]` — the ordered mastering pipeline. +3. `secretSauce.workflowSteps[]` — the signature-technique steps. + +Each card carries `{device, parameter, value, …, phase1Fields[]}` plus prose +(`reason`, `instruction`, `measurementJustification`). Two properties make this +shape awkward as a stable API surface: + +1. **`value` is free text.** Gemini emits `"10 ms"`, `"-18 dB"`, `"3:1"`, + `"Sine"` — a single string with the magnitude, unit, and sometimes shape all + fused. There is no separate `unit`, no `range`, and the citation is named + `phase1Fields`, not the producer-facing `cited_measurements`. +2. **There is no machine-checkable freeze.** The Phase 2 shape is typed on the + frontend (`interpretation.ts`) and guarded at runtime by `phase2Validator.ts` + and the backend catalogue/citation gates, but no committed JSON Schema + declares a versioned recommendation contract an external consumer (a REAPER + script, a Max patch, a downstream pipeline, or ASA's own UI) can validate + against — the same gap ADR 0001 closed for Phase 1's *measurements*, now for + Phase 2's *recommendations*. + +The campaign goal is to **freeze a versioned `recommendations` JSON contract** — +each entry `{device, parameter, value, unit, range, cited_measurements[]}`, every +recommendation citing the Phase 1 measurement(s) that justify it and never +overriding them — with a semver'd JSON Schema, CI validation of Phase 2 output +against it, and a round-trip test. + +## Decision + +Declare the **Recommendations Contract v1**, version string `"recommendations.v1"`. + +The contract is a **deterministic, server-side projection** of the existing +Phase 2 cards into a normalized envelope. It is additive: it never overrides a +Phase 1 measurement (invariant #1) and never mutates the raw Phase 2 arrays — it +only *reads* them. + +**Envelope:** + +```json +{ "version": "recommendations.v1", "recommendations": [ , … ] } +``` + +**Entry** (exactly six fields, `additionalProperties: false`): + +| Field | Type | Meaning | +| --- | --- | --- | +| `device` | `string` (non-empty) | Live 12 device name as recommended, trimmed. Not catalogue-rewritten. | +| `parameter` | `string` (non-empty) | Device parameter name as recommended. Not catalogue-rewritten. | +| `value` | `string \| number` | The magnitude when one was parsed from the value string (`"10 ms"`→`10`, `"3:1"`→`3`); else the original non-numeric string (`"Sine"`). | +| `unit` | enum \| `null` | One of `Hz, dB, ms, s, ratio, %, st`, or `null` when the value is non-numeric/unitless. `ratio` denotes a compression-style ratio (`3` ≡ `3:1`). | +| `range` | `[number, number] \| null` | Best-effort suggested working range, derived from a per-unit tolerance band around `value`. `null` when no band applies. **Not** a device hard-limit (see *Honest range* below). | +| `cited_measurements` | `string[]` (`minItems: 1`) | Dotted Phase 1 measurement paths justifying the recommendation, projected from `phase1Fields`. | + +**Authoritative artifacts:** + +- Schema: [`apps/backend/schemas/recommendations.v1.schema.json`](../../apps/backend/schemas/recommendations.v1.schema.json) (Draft 2020-12). +- Projection + validation: [`apps/backend/recommendations_contract.py`](../../apps/backend/recommendations_contract.py) + (`project_recommendations`, `validate_envelope`, `build_validated_recommendations`). +- TypeScript mirror: `RecommendationContractEntry` / `RecommendationsContract` in + [`apps/ui/src/types/interpretation.ts`](../../apps/ui/src/types/interpretation.ts). + +**API exposure:** the validated envelope is attached additively to the +interpretation stage result as `recommendations` (alongside the existing rich +Phase 2 fields), so the normalized contract travels in the live +`GET /api/analysis-runs/{run_id}` snapshot, not just in tests. + +## Why a projection, not a change to Gemini's emission + +Reshaping Gemini's structured output to emit these six fields directly was +rejected: it is unverifiable without a live Gemini call, touches the prompt, the +response schema, every existing validator, the UI, and all stored older results +— the opposite of a surgical change. The projection is additive, deterministic, +testable in the cheap inner loop with no Gemini spend, and reuses the proven +value-parsing logic from the recommendation scorer +(`recommendation_evaluation.py`). It honors invariant #1 by construction: a view +derived from Phase 2 output cannot override a Phase 1 measurement. + +## Citation-gating (invariant #2) + +The schema requires `cited_measurements` with `minItems: 1`. The projection +admits a card **only if it cites ≥1 Phase 1 measurement**; uncited cards are +**excluded from this normalized view by design**. + +This is *not* a violation of the warn-and-keep contract +(`phase2_catalogue_gates.py`): nothing user-facing is dropped. Uncited cards +remain in the raw Phase 2 arrays, where the existing warn-and-keep gate already +flags them with `RECOMMENDATION_UNVERIFIED` / `citation_missing`. The exclusion +is from the *contract* only, and it is surfaced (the gate warning), not silent. +`minItems: 1` is therefore honest: every entry that *is* in the contract carries +a citation. It is **not** a runtime hard-fail against live Gemini — Phase 1 stays +authoritative, and a missing citation degrades to a warning, never a rejection. + +## Honest range (invariant #4) + +`range` is a **measurement-informed neighborhood**, not a Live device limit. The +static Live 12 catalogue carries no `min`/`max` (see +[`data/live12_catalogue.schema.json`](../../data/live12_catalogue.schema.json) +`extraction_notes` — those live on runtime `Live.DeviceParameter` objects), so a +true device range cannot be supplied here. The contract derives `range` from the +same per-unit tolerance bands the recommendation scorer uses for value accuracy +(`±20%` for Hz, `±3 dB`, `±30%` for ms/s, `±1` for ratio/st, `±15%`), and emits +it **only** when the value is numeric *and* the unit is known — otherwise `null`. +This keeps the contract from pretending a precise range exists where it does not. + +## Validation is against the file, not a mirror + +`validate_envelope` runs the real `jsonschema` validator against the **committed +schema file**. There is deliberately **no** hand-rolled structural validator that +"matches" the schema — that is exactly the drift failure mode ADR 0001 called out +(a payload validated against a function that *resembles* the schema is not +validated against the schema). The `live12_catalogue.py` precedent avoided a +`jsonschema` dependency because it had no consumer mandating schema-as-gate; this +contract's whole point is the CI gate, so the dependency (`jsonschema==4.26.0`, +pure-Python) is earned. + +## Compatibility policy + +`version: "recommendations.v1"` identifies the **major** version; the committed +`recommendations.v1.schema.json` is the current v1 contract. With +`additionalProperties: false`, evolution classifies as: + +| Change | Class | Allowed in v1? | +| --- | --- | --- | +| Add a new **optional** entry field | Additive (minor) | **Yes** — add to `properties` (not `required`), keep `additionalProperties: false`, keep the `version` const. Consumers must validate against the *current* v1 schema; payloads emitted before the addition still validate. | +| Add a new value to the `unit` enum | Additive (minor) | **Yes** — extend the enum and `_DISPLAY_UNIT` / `UNIT_BANDS`. | +| Make an existing optional field required | Breaking (major) | **No** — bump to `recommendations.v2`. | +| Rename / remove / retype a field | Breaking (major) | **No** — v2. | +| Change `value`/`unit`/`range` semantics or a tolerance band | Breaking (major) | **No** — v2. | +| Loosen `additionalProperties` to `true` | Breaking (major) | **No** — it changes the validation contract; v2. | + +A v2 bump means a new `recommendations.v2.schema.json`, a new `version` const, +and a new ADR — mirroring ADR 0002's `phase1.v2` precedent. + +## Where v1 is enforced + +| Layer | Enforcement | If you break v1 here | +| --- | --- | --- | +| Schema validity | `test_recommendations_contract.SchemaIsValidTests` runs `check_schema` on the file | Test fails; CI red. | +| Phase 2 output conforms | `test_recommendations_contract.ProjectionValidatesTests` projects a representative Phase 2 result and validates each entry against the file | Test fails; CI red. | +| Round-trip stability | `test_recommendations_contract.RoundTripTests` | Test fails; CI red. | +| Frozen entry shape | `test_recommendations_contract.SchemaFreezeTests` (additionalProperties, minItems, required, enum, version const) | Test fails; CI red. | +| Attached to the interpretation result | `test_server.…test_run_interpretation_request_attaches_recommendations_contract` | Test fails; CI red. | +| Survives into the run snapshot | `test_analysis_runtime.test_run_snapshot_surfaces_recommendations_contract_verbatim` (the payload behind `GET /api/analysis-runs/{run_id}`) | Test fails; CI red. | +| TS ↔ Python agreement | `RecommendationContractEntry` in `interpretation.ts` | UI type-check (`npm run lint`) fails. | + +CI runs these via `python -m unittest discover -s tests` in the backend job +([`.github/workflows/ci.yml`](../../.github/workflows/ci.yml)). + +## What this ADR does *not* do + +- **Does not change Gemini's emission.** The raw Phase 2 arrays and the prompt + are untouched; `recommendations` is a derived sibling field. +- **Does not deduplicate.** A device/parameter appearing in more than one source + array yields more than one entry. Dedup is deferrable to a minor revision if a + consumer needs it. +- **Does not rewrite devices/parameters against the catalogue.** Catalogue + verification stays warn-and-keep on the raw arrays (`phase2_catalogue_gates.py`). +- **Does not guarantee a non-empty contract.** A track that yields no cited cards + produces `{"version": "recommendations.v1", "recommendations": []}`, which is + valid. + +## Consequences + +- External consumers and the UI gain a stable, citation-first recommendation + shape they can validate against, independent of the prose-heavy Phase 2 fields. +- Adding an optional entry field or a unit enum value is low-friction (update the + schema file + the projection + the freeze tests). Renaming/removing/retyping + requires a v2 schema and a new ADR. +- The contract makes the citation chain (invariant #2) a *validated* property of + the recommendation surface, not just a runtime guardrail. + +## Alternatives considered + +- **Reshape Gemini's structured output to emit the six fields directly.** + Rejected — unverifiable without live Gemini, large blast radius, violates the + surgical-change principle. +- **Hand-rolled stdlib validator (the `live12_catalogue.py` pattern).** Rejected + — reintroduces the schema/validator drift ADR 0001 warned about and makes + "validate against the schema" a fiction. The `jsonschema` dependency is earned + by the CI-gate mandate. +- **Include uncited cards with an empty `cited_measurements`.** Rejected — it + would make `minItems: 1` impossible and dilute the contract's promise that + every entry is measurement-justified. Uncited cards stay in the raw arrays.