From 0a12a62200d4acdb9695dc364bf9ec1919ae633e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 4 Jul 2026 04:53:21 +0000 Subject: [PATCH] feat(key): multi-profile key ensemble cross-check (accuracy PR-B3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit analyze_key now runs KeyExtractor for edma + temperley + krumhansl in full mode and emits an additive, full-only keyEnsemble field: {method:'profile_vote.v1', agreement, profiles[], alternates[]}. SURFACING-ONLY: the shipped key/keyConfidence/keyProfile are unchanged — still EDMA (golden diff confirms only keyEnsemble was added; the key value is byte-identical). The vote is evidence until the GiantSteps Key gate proves it beats EDMA-alone. Pre-registered decision rule (frozen before the run) in incorporations/key-ensemble-decision-2026-07-04.md: adopt iff mirexWeighted gain >= +0.02 AND no exact-rate regression beyond 0.01, on >= 400 evaluable clips. agreement (0-3) is a ready confidence calibrator; alternates hedge the relative-major/minor error that is EDMA's typical miss. Gated on full mode (like tuning*): fast mode is unchanged and pays no extra KeyExtractor cost. Any/all profile failures degrade gracefully (null ensemble; EDMA key still ships). Contract mirrors (tripwire #4): analyze.py + server_phase1 forwarding, JSON_SCHEMA.md inventory/full-only/field-row, test_audio_fixture full-mode key set, types/measurement.ts KeyEnsemble, backendPhase1Client forwarding, phase1FullPayload fixture, golden re-baselined (new key, structural). Tests: test_key_ensemble (agreement/alternates, full-vs-fast, all-fail). Backend 1302 OK, fundamentals gate green, frontend verify green. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01T4wfz87k6kzJqkKLKZE7YL --- apps/backend/JSON_SCHEMA.md | 5 +- apps/backend/analyze.py | 2 + apps/backend/analyze_core.py | 52 ++++++++++++++++++- apps/backend/server_phase1.py | 1 + .../tests/fixtures/golden/phase1_default.json | 14 +++++ apps/backend/tests/test_audio_fixture.py | 2 +- apps/backend/tests/test_key_ensemble.py | 51 ++++++++++++++++++ apps/ui/src/services/backendPhase1Client.ts | 3 ++ apps/ui/src/types/measurement.ts | 23 ++++++++ apps/ui/tests/fixtures/phase1FullPayload.ts | 10 ++++ .../key-ensemble-decision-2026-07-04.md | 51 ++++++++++++++++++ 11 files changed, 210 insertions(+), 4 deletions(-) create mode 100644 apps/backend/tests/test_key_ensemble.py create mode 100644 incorporations/key-ensemble-decision-2026-07-04.md diff --git a/apps/backend/JSON_SCHEMA.md b/apps/backend/JSON_SCHEMA.md index 71f03ddb..b294e004 100644 --- a/apps/backend/JSON_SCHEMA.md +++ b/apps/backend/JSON_SCHEMA.md @@ -15,9 +15,9 @@ Conventions: Top-level keys: -`phase1Version`, `fundamentalsQuality`, `bpm`, `bpmConfidence`, `bpmPercival`, `bpmAgreement`, `bpmDoubletime`, `bpmSource`, `bpmRawOriginal`, `key`, `keyConfidence`, `keyProfile`, `tuningFrequency`, `tuningCents`, `timeSignature`, `timeSignatureSource`, `timeSignatureConfidence`, `timeSignatureCandidates`, `durationSeconds`, `sampleRate`, `lufsIntegrated`, `lufsRange`, `lufsMomentaryMax`, `lufsShortTermMax`, `lufsCurve`, `truePeak`, `crestFactor`, `dynamicSpread`, `monoCompatible`, `plr`, `dynamicCharacter`, `textureCharacter`, `stereoDetail`, `spectralBalance`, `spectralBalanceTimeSeries`, `spectralDetail`, `stemAnalysis`, `transientDensityDetail`, `saturationDetail`, `snareDetail`, `hihatDetail`, `rhythmDetail`, `melodyDetail`, `transcriptionDetail`, `pitchDetail`, `grooveDetail`, `beatsLoudness`, `rhythmTimeline`, `sidechainDetail`, `reverbDetail`, `vocalDetail`, `acidDetail`, `supersawDetail`, `bassDetail`, `kickDetail`, `genreDetail`, `effectsDetail`, `synthesisCharacter`, `danceability`, `structure`, `arrangementDetail`, `segmentLoudness`, `segmentSpectral`, `segmentStereo`, `segmentKey`, `chordDetail`, `perceptual`, `essentiaFeatures`. +`phase1Version`, `fundamentalsQuality`, `bpm`, `bpmConfidence`, `bpmPercival`, `bpmAgreement`, `bpmDoubletime`, `bpmSource`, `bpmRawOriginal`, `key`, `keyConfidence`, `keyProfile`, `keyEnsemble`, `tuningFrequency`, `tuningCents`, `timeSignature`, `timeSignatureSource`, `timeSignatureConfidence`, `timeSignatureCandidates`, `durationSeconds`, `sampleRate`, `lufsIntegrated`, `lufsRange`, `lufsMomentaryMax`, `lufsShortTermMax`, `lufsCurve`, `truePeak`, `crestFactor`, `dynamicSpread`, `monoCompatible`, `plr`, `dynamicCharacter`, `textureCharacter`, `stereoDetail`, `spectralBalance`, `spectralBalanceTimeSeries`, `spectralDetail`, `stemAnalysis`, `transientDensityDetail`, `saturationDetail`, `snareDetail`, `hihatDetail`, `rhythmDetail`, `melodyDetail`, `transcriptionDetail`, `pitchDetail`, `grooveDetail`, `beatsLoudness`, `rhythmTimeline`, `sidechainDetail`, `reverbDetail`, `vocalDetail`, `acidDetail`, `supersawDetail`, `bassDetail`, `kickDetail`, `genreDetail`, `effectsDetail`, `synthesisCharacter`, `danceability`, `structure`, `arrangementDetail`, `segmentLoudness`, `segmentSpectral`, `segmentStereo`, `segmentKey`, `chordDetail`, `perceptual`, `essentiaFeatures`. -**Shared (fast + full) vs full-only.** Fast-mode output is asserted byte-for-byte against the `EXPECTED_TOP_LEVEL_KEYS` set in [`tests/test_analyze.py`](tests/test_analyze.py). Full mode emits those keys *plus* a handful of detail-only fields that are deliberately absent from the shared snapshot: `keyProfile`, `tuningFrequency`, `tuningCents`, `lufsMomentaryMax`, `lufsShortTermMax`, `pitchDetail`, and `timeSignatureCandidates`. When changing the schema, update both this list and `EXPECTED_TOP_LEVEL_KEYS`; full-only fields stay out of that set on purpose. See CLAUDE.md tripwire #4. Schema changes are also cross-checked executably against the frontend fixture and parser by `apps/ui/tests/services/phase1ContractParity.test.ts`, driven by the golden snapshot's `topLevelKeys`/`keyTree` — regenerating the golden (`UPDATE_PHASE1_GOLDEN=1`) is what arms the nested half of that gate. +**Shared (fast + full) vs full-only.** Fast-mode output is asserted byte-for-byte against the `EXPECTED_TOP_LEVEL_KEYS` set in [`tests/test_analyze.py`](tests/test_analyze.py). Full mode emits those keys *plus* a handful of detail-only fields that are deliberately absent from the shared snapshot: `keyProfile`, `tuningFrequency`, `tuningCents`, `lufsMomentaryMax`, `lufsShortTermMax`, `pitchDetail`, `timeSignatureCandidates`, and `keyEnsemble`. When changing the schema, update both this list and `EXPECTED_TOP_LEVEL_KEYS`; full-only fields stay out of that set on purpose. See CLAUDE.md tripwire #4. Schema changes are also cross-checked executably against the frontend fixture and parser by `apps/ui/tests/services/phase1ContractParity.test.ts`, driven by the golden snapshot's `topLevelKeys`/`keyTree` — regenerating the golden (`UPDATE_PHASE1_GOLDEN=1`) is what arms the nested half of that gate. ## Relationship To `POST /api/analyze` @@ -162,6 +162,7 @@ Current server behavior that affects schema expectations: | `durationSeconds` | `float \| null` | Track duration from sample count. | seconds | Useful for arrangement section planning and timeline mapping. | | `sampleRate` | `int \| null` | Effective analysis sample rate. | Hz | Ensures downstream feature interpretation uses correct temporal/frequency scaling. | | `keyProfile` | `string \| null` | Key profile used by `KeyExtractor` (e.g. `"edma"`). | categorical | Indicates which pitch template corpus was used for key detection. | +| `keyEnsemble` | `object \| null` | **Full mode only.** Multi-profile key cross-check (accuracy program PR-B3): `{method: "profile_vote.v1", agreement, profiles: [{profile, key, strength}], alternates: [{key, strength}]}`. EDMA stays the authoritative shipped `key`; this records temperley/krumhansl reads. `null` when all profiles failed. | object | Surfacing-only until the GiantSteps gate proves the vote beats EDMA-alone (`incorporations/key-ensemble-decision-2026-07-04.md`). `agreement` (0-3) calibrates key confidence; `alternates` hedge the display. Never an override of `key`. | | `tuningFrequency` | `float \| null` | Estimated tuning reference frequency from spectral peak analysis. | Hz | Deviation from 440 Hz helps detect detuned material or concert-pitch variants. | | `tuningCents` | `float \| null` | Tuning offset from A440 in cents. | cents | Positive = sharp of A440, negative = flat. Useful for pitch-correcting reconstructions. | diff --git a/apps/backend/analyze.py b/apps/backend/analyze.py index fa476652..66f82d01 100644 --- a/apps/backend/analyze.py +++ b/apps/backend/analyze.py @@ -1895,6 +1895,8 @@ def main(): "key": result.get("key"), "keyConfidence": result.get("keyConfidence"), "keyProfile": result.get("keyProfile"), + # Full-only (like keyProfile/tuning*): multi-profile key cross-check. + "keyEnsemble": result.get("keyEnsemble"), "tuningFrequency": result.get("tuningFrequency"), "tuningCents": result.get("tuningCents"), "timeSignature": result.get("timeSignature"), diff --git a/apps/backend/analyze_core.py b/apps/backend/analyze_core.py index ba0e78f2..f7e85218 100644 --- a/apps/backend/analyze_core.py +++ b/apps/backend/analyze_core.py @@ -135,13 +135,61 @@ def analyze_bpm( } +# Key profiles run in the full-mode ensemble. EDMA stays the authoritative +# shipped label (tuned for electronic dance music); temperley and krumhansl +# are cross-checks whose agreement calibrates confidence and whose +# disagreements surface as alternates. Order fixed so the emitted list is +# deterministic. +_KEY_ENSEMBLE_PROFILES = ("edma", "temperley", "krumhansl") + + +def _run_key_profile(mono: np.ndarray, profile: str) -> dict | None: + """Run one KeyExtractor profile → {profile, key, strength} or None on failure.""" + try: + key, scale, strength = es.KeyExtractor(profileType=profile)(mono) + return { + "profile": profile, + "key": f"{key} {scale.capitalize()}", + "strength": round(float(strength), 3), + } + except Exception: + return None + + +def _build_key_ensemble(mono: np.ndarray, primary_key: str | None) -> dict | None: + """Cross-check EDMA against temperley/krumhansl (accuracy program PR-B3). + + Surfacing-only: records each profile's read, how many agree with the + shipped EDMA label, and the distinct alternates. Does NOT override the + shipped ``key`` — the vote is evidence until the GiantSteps gate proves it + beats EDMA-alone (see incorporations/key-ensemble-decision-2026-07-04.md). + """ + profiles = [p for p in (_run_key_profile(mono, name) for name in _KEY_ENSEMBLE_PROFILES) if p] + if not profiles: + return None + agreement = sum(1 for p in profiles if primary_key is not None and p["key"] == primary_key) + seen: set[str] = set() + alternates: list[dict] = [] + for p in profiles: + if p["key"] != primary_key and p["key"] not in seen: + seen.add(p["key"]) + alternates.append({"key": p["key"], "strength": p["strength"]}) + return { + "method": "profile_vote.v1", + "agreement": agreement, + "profiles": profiles, + "alternates": alternates, + } + + def analyze_key(mono: np.ndarray, *, include_tuning: bool = True) -> dict: """Extract musical key and confidence using KeyExtractor with EDMA profile. ``include_tuning`` gates the per-frame spectral-peak tuning-frequency pass, which is the dominant cost of key analysis on a full track. Fast mode does not surface ``tuningFrequency``/``tuningCents``, so it passes ``False`` to - skip that work while keeping the return shape identical. + skip that work while keeping the return shape identical. It also gates the + full-only ``keyEnsemble`` cross-check for the same reason. """ try: extractor = es.KeyExtractor(profileType="edma") @@ -158,6 +206,8 @@ def analyze_key(mono: np.ndarray, *, include_tuning: bool = True) -> dict: result["tuningCents"] = None return result + result["keyEnsemble"] = _build_key_ensemble(mono, key_str) + try: frame_size = 2048 hop_size = 1024 diff --git a/apps/backend/server_phase1.py b/apps/backend/server_phase1.py index 1ad2cf4a..368cc830 100644 --- a/apps/backend/server_phase1.py +++ b/apps/backend/server_phase1.py @@ -196,6 +196,7 @@ def _build_phase1(payload: dict[str, Any]) -> dict[str, Any]: "key": _coerce_nullable_string(payload.get("key")), "keyConfidence": _coerce_number(payload.get("keyConfidence")), "keyProfile": payload.get("keyProfile"), + "keyEnsemble": payload.get("keyEnsemble"), "tuningFrequency": _coerce_nullable_number(payload.get("tuningFrequency")), "tuningCents": _coerce_nullable_number(payload.get("tuningCents")), "timeSignature": _coerce_string(payload.get("timeSignature"), "4/4"), diff --git a/apps/backend/tests/fixtures/golden/phase1_default.json b/apps/backend/tests/fixtures/golden/phase1_default.json index 29901b91..b31c866d 100644 --- a/apps/backend/tests/fixtures/golden/phase1_default.json +++ b/apps/backend/tests/fixtures/golden/phase1_default.json @@ -236,6 +236,18 @@ "hihatDetail": "null", "key": "str", "keyConfidence": "number", + "keyEnsemble": { + "agreement": "number", + "alternates": "list", + "method": "str", + "profiles": { + "[]": { + "key": "str", + "profile": "str", + "strength": "number" + } + } + }, "keyProfile": "str", "kickDetail": { "fundamentalHz": "number", @@ -575,6 +587,7 @@ "hihatDetail", "key", "keyConfidence", + "keyEnsemble", "keyProfile", "kickDetail", "lufsCurve", @@ -645,6 +658,7 @@ "hihatDetail": "null", "key": "str", "keyConfidence": "number", + "keyEnsemble": "dict", "keyProfile": "str", "kickDetail": "dict", "lufsCurve": "dict", diff --git a/apps/backend/tests/test_audio_fixture.py b/apps/backend/tests/test_audio_fixture.py index 0566cd02..265c4b37 100644 --- a/apps/backend/tests/test_audio_fixture.py +++ b/apps/backend/tests/test_audio_fixture.py @@ -14,7 +14,7 @@ "fundamentalsQuality", "bpm", "bpmConfidence", "bpmPercival", "bpmAgreement", "bpmDoubletime", "bpmSource", "bpmRawOriginal", - "key", "keyConfidence", "keyProfile", "tuningFrequency", "tuningCents", + "key", "keyConfidence", "keyProfile", "keyEnsemble", "tuningFrequency", "tuningCents", "timeSignature", "timeSignatureSource", "timeSignatureConfidence", "timeSignatureCandidates", "durationSeconds", "sampleRate", diff --git a/apps/backend/tests/test_key_ensemble.py b/apps/backend/tests/test_key_ensemble.py new file mode 100644 index 00000000..64f8d558 --- /dev/null +++ b/apps/backend/tests/test_key_ensemble.py @@ -0,0 +1,51 @@ +import unittest +from unittest import mock + +import numpy as np + +import analyze_core + + +class KeyEnsembleTests(unittest.TestCase): + def _fake_extractor(self, table): + """Return a KeyExtractor stand-in whose output depends on profileType.""" + def factory(profileType="edma"): + key, scale, strength = table[profileType] + return lambda mono: (key, scale, strength) + return factory + + def test_ensemble_records_agreement_and_alternates(self) -> None: + table = { + "edma": ("C", "major", 0.90), + "temperley": ("C", "major", 0.85), # agrees + "krumhansl": ("A", "minor", 0.70), # relative -> alternate + } + with mock.patch.object(analyze_core.es, "KeyExtractor", self._fake_extractor(table)): + ensemble = analyze_core._build_key_ensemble(np.zeros(4096, dtype=np.float32), "C Major") + self.assertEqual(ensemble["method"], "profile_vote.v1") + self.assertEqual(ensemble["agreement"], 2) # edma + temperley + self.assertEqual([p["profile"] for p in ensemble["profiles"]], ["edma", "temperley", "krumhansl"]) + self.assertEqual(ensemble["alternates"], [{"key": "A Minor", "strength": 0.7}]) + + def test_full_key_result_carries_ensemble_but_not_fast(self) -> None: + table = {p: ("F", "minor", 0.8) for p in ("edma", "temperley", "krumhansl")} + with mock.patch.object(analyze_core.es, "KeyExtractor", self._fake_extractor(table)): + full = analyze_core.analyze_key(np.zeros(4096, dtype=np.float32), include_tuning=True) + fast = analyze_core.analyze_key(np.zeros(4096, dtype=np.float32), include_tuning=False) + # Shipped key stays EDMA-derived; ensemble is additive and full-only. + self.assertEqual(full["key"], "F Minor") + self.assertEqual(full["keyProfile"], "edma") + self.assertEqual(full["keyEnsemble"]["agreement"], 3) + self.assertNotIn("keyEnsemble", fast) + + def test_all_profiles_failing_yields_none(self) -> None: + def boom(profileType="edma"): + def run(mono): + raise RuntimeError("extractor down") + return run + with mock.patch.object(analyze_core.es, "KeyExtractor", boom): + self.assertIsNone(analyze_core._build_key_ensemble(np.zeros(4096, dtype=np.float32), "C Major")) + + +if __name__ == "__main__": + unittest.main() diff --git a/apps/ui/src/services/backendPhase1Client.ts b/apps/ui/src/services/backendPhase1Client.ts index 86d966f1..3c37adac 100644 --- a/apps/ui/src/services/backendPhase1Client.ts +++ b/apps/ui/src/services/backendPhase1Client.ts @@ -591,6 +591,9 @@ export function parsePhase1Result(value: unknown): Phase1Result { key: expectNullableString(phase1, "key"), keyConfidence: expectNumber(phase1, "keyConfidence"), keyProfile: toOptionalStringOrNull(phase1.keyProfile), + keyEnsemble: isRecord(phase1.keyEnsemble) + ? (phase1.keyEnsemble as unknown as Phase1Result["keyEnsemble"]) + : null, tuningFrequency: toNumber(phase1.tuningFrequency), tuningCents: toNumber(phase1.tuningCents), timeSignature: expectString(phase1, "timeSignature"), diff --git a/apps/ui/src/types/measurement.ts b/apps/ui/src/types/measurement.ts index 48f77780..27224aa3 100644 --- a/apps/ui/src/types/measurement.ts +++ b/apps/ui/src/types/measurement.ts @@ -656,6 +656,27 @@ export interface TimeSignatureCandidate { positionMeans: number[]; } +/** + * Multi-profile key cross-check (accuracy program PR-B3). Full mode only. + * EDMA stays the authoritative shipped `key`; this records what temperley + * and krumhansl also read, how many agree, and the distinct alternates. + * Surfacing-only until the GiantSteps gate proves the vote beats EDMA-alone + * (incorporations/key-ensemble-decision-2026-07-04.md). + */ +export interface KeyEnsembleProfile { + profile: string; + key: string; + strength: number; +} + +export interface KeyEnsemble { + method: string; + /** How many profiles (0-3) match the shipped EDMA label. */ + agreement: number; + profiles: KeyEnsembleProfile[]; + alternates: Array<{ key: string; strength: number }>; +} + export type FundamentalsQualityStatus = "authoritative" | "ambiguous" | "failed" | "not_run"; export interface FundamentalsQualityDomain { @@ -692,6 +713,8 @@ export interface Phase1Result { key: string | null; keyConfidence: number; keyProfile?: string | null; + /** Full-only multi-profile key cross-check (surfacing-only). */ + keyEnsemble?: KeyEnsemble | null; tuningFrequency?: number | null; tuningCents?: number | null; timeSignature: string; diff --git a/apps/ui/tests/fixtures/phase1FullPayload.ts b/apps/ui/tests/fixtures/phase1FullPayload.ts index 1a573046..e7d2f597 100644 --- a/apps/ui/tests/fixtures/phase1FullPayload.ts +++ b/apps/ui/tests/fixtures/phase1FullPayload.ts @@ -140,6 +140,16 @@ export const phase1EnvelopeFixture = { key: 'A minor', keyConfidence: 0.91, keyProfile: 'edma', + keyEnsemble: { + method: 'profile_vote.v1', + agreement: 2, + profiles: [ + { profile: 'edma', key: 'A minor', strength: 0.91 }, + { profile: 'temperley', key: 'A minor', strength: 0.86 }, + { profile: 'krumhansl', key: 'C major', strength: 0.7 }, + ], + alternates: [{ key: 'C major', strength: 0.7 }], + }, tuningFrequency: 440.12, tuningCents: 0.05, timeSignature: '4/4', diff --git a/incorporations/key-ensemble-decision-2026-07-04.md b/incorporations/key-ensemble-decision-2026-07-04.md new file mode 100644 index 00000000..fd97d70c --- /dev/null +++ b/incorporations/key-ensemble-decision-2026-07-04.md @@ -0,0 +1,51 @@ +# Key-ensemble decision record (pre-registration) + +**Status: PENDING — gate not yet run (awaiting GiantSteps Key audio).** + +Accuracy program PR-B3. Records — *before* the deciding run — the rule for +whether ASA's shipped key label should switch from EDMA-alone to a +multi-profile vote. Pre-registered so the outcome can't be rationalised after +seeing the numbers. + +## What shipped in PR-B3 (surfacing-only) + +`analyze_key` now runs `KeyExtractor` for three profiles — `edma`, +`temperley`, `krumhansl` — in full mode and emits an additive, full-only +`keyEnsemble` field: `{method: "profile_vote.v1", agreement, profiles[], +alternates[]}`. **The shipped `key`/`keyConfidence`/`keyProfile` are +unchanged — still EDMA.** The vote is evidence only until this gate passes. + +## The measurement + +Run `scripts/evaluate_giantsteps.py --subset key` (GiantSteps Key: ~600 +expert-annotated Beatport clips) twice: + +1. **Baseline** — the shipped EDMA label. +2. **Vote** — a candidate rule that resolves the label from the three + profiles: majority-exact wins; no majority → EDMA stays. + +Metrics: MIREX weighted key score (`giantsteps_evaluation.mirex_key_score`), +exact rate, exact-or-relative rate. + +## Frozen decision rule + +**Adopt the vote as the shipped label iff BOTH:** + +1. `mirexWeighted(vote) − mirexWeighted(edma) ≥ +0.02` (a real ≥2-point gain + on the 0–1 MIREX scale), AND +2. `keyExactRate(vote) ≥ keyExactRate(edma) − 0.01` (no exact-match + regression beyond noise). + +Otherwise **keep EDMA** as the shipped label; `keyEnsemble` stays a +surfacing-only cross-check (still useful: `agreement` calibrates confidence, +`alternates` hedge the UI). + +**Power:** GiantSteps Key has ~604 clips; require ≥ 400 evaluable (audio +present) or the run is `underpowered` and must not be finalised. + +## If adopted + +Switch the shipped `key` to the vote result, set `keyProfile` to the winning +profile (or `"profile_vote.v1"`), recalibrate `keyConfidence` from +`(agreement, winning strength)`, re-baseline the golden (curated value), and +sync the frontend parity fixture. Record the numbers here.