diff --git a/audits/genre-coverage-2026-07-13.md b/audits/genre-coverage-2026-07-13.md new file mode 100644 index 00000000..b7c4a636 --- /dev/null +++ b/audits/genre-coverage-2026-07-13.md @@ -0,0 +1,192 @@ +# Genre coverage audit — where the measurements are only proven on techno/house (2026-07-13) + +Coverage audit for the genre-generalization program +(`plans/genre-generalization-program.md`). Maps the three genre-aware +surfaces against the priority electronic genres and names the gaps. No code +changes; every claim below carries the file it was read from. + +The three surfaces: + +1. **Genre classifier signatures** — `_GENRE_SIGNATURES`, + `apps/backend/analyze_detection.py:1031` (34 signatures) plus + `_GENRE_FAMILY_MAP` at `analyze_detection.py:1078`. +2. **Synthetic fundamentals corpus** — + `apps/backend/scripts/build_synthetic_corpus.py` (29 clips, + `fundamentals_eval_manifest.synthetic.json`). +3. **Mix-doctor genre profiles** — `apps/ui/src/data/genreProfiles.ts` + (34 profiles, same id set as the backend signatures), resolved by + `apps/ui/src/services/mixDoctor.ts` via `genreDetail.genre` → + `genreDetail.genreFamily` → default. + +Priority genres (program mandate): UK bass / garage / speed garage / 2-step; +halftime / trap / dubstep / bass; hard dance / hardstyle / gabber; IDM / +ambient / electro; plus footwork, breakbeat, downtempo. + +## Summary verdict + +The accuracy gates are real but narrow: the synthetic corpus proves the +pipeline on **steady kick-per-beat 4/4 material, mostly 96–140 BPM**, and +the genre surfaces enumerate house/techno lineages in depth while entire +electronic families (trap, hardstyle, gabber, footwork, electro, halftime, +2-step-as-rhythm) have **no signature, no profile, and no corpus clip**. +Nothing in the active gate set exercises a broken kick pattern, a halftime +feel, 16th-note shuffle, beatless material, or any tempo above 174. + +## Surface 1 — `_GENRE_SIGNATURES` (analyze_detection.py:1031) + +### What exists (34 signatures) + +| Family | Signatures | BPM span | +|---|---|---| +| Ambient/downtempo | ambient-drone, ambient-techno, dub-techno, ambient | 40–125 | +| House | deep/organic/classic/tech/progressive/afro/bass-house, house | 115–132 | +| Techno | minimal/melodic/driving/industrial/hard/acid/detroit, techno | 122–160 | +| Trance | trance, psytrance | 136–148 | +| Bass | dubstep | 138–145 | +| D&B/breaks | drum-bass, neurofunk, dnb, breaks | 125–180 | +| UK garage | uk-garage, bassline, garage | 128–142 | +| Legacy broad | edm, hiphop, rock, pop, acoustic | 70–160 | + +Feature axes per signature: `bpm`, `subBassDb`, `crestFactor`, +`onsetDensity`, `spectralCentroid`, `sidechainStrength`, `bassDecay`, +optional `rt60` and `kickDistortion` (kickDistortion used by exactly two: +industrial-techno, hard-techno). Weights (`analyze_detection.py:1180`): +sidechain 0.95 and bass decay 0.85 are the primary discriminators — a +**four-on-the-floor-centric assumption**, since both features encode the +kick-pumping relationship that broken-beat genres don't have. + +### Priority-genre gaps + +1. **trap** — nothing covers it. `hiphop` tops out at 110 BPM and assumes + the felt (half-time) tempo; a trap track measured at its notated ~140 + BPM (see the tempo-octave trap below) lands nearest `dubstep` + (138–145), whose sub/decay/centroid ranges then feed wrong mix advice. +2. **footwork / juke** (155–165) — falls between `hard-techno` (145–160, + wrong character: footwork has low sidechain, sparse sub-heavy toms) and + `dnb` (160–180, wrong rhythm). No signature. +3. **hardstyle** (150–160) — `hard-techno` will absorb it, but hardstyle's + reverse-bass kick and much higher kick THD are a different production + blueprint; the Phase 2 advice chain treats them identically today. +4. **gabber / hardcore** (170–200+) — **no signature of any kind above + 180 BPM.** A 190 BPM gabber track's best match is `dnb`/`drum-bass`, + which is flatly wrong (different kick, different sub, different swing). +5. **electro** (100–135, broken 808 machine funk) — no signature. The + broken-beat + high-crest + moderate-tempo combination scores weakly + everywhere; likely misread as `breaks` or `pop`. +6. **IDM** — no signature. Deliberately heterogeneous; the honest behavior + is a *confident abstain* (`genreDetail: None` / low confidence), which + the classifier already supports (`primary_score < 0.25` → + abstain, `analyze_detection.py:1217`). The audit recommendation is to + verify abstention on IDM-like material rather than force a signature. +7. **2-step** — `uk-garage` (128–136) exists, but its + `sidechainStrength: (0.35, 0.65)` encodes 4×4-garage pumping; classic + 2-step has a broken kick and little sidechain, so it scores *against* + the one signature meant to catch it. No separate 2-step signature; no + **speed garage** signature (4×4 130–140, warehouse sub) either. +8. **halftime (D&B tempo, half feel)** — no signature; at the measured + ~85–87 BPM (post-octave-halving) it reads as `hiphop`. +9. **breakbeat** — `breaks` (125–135) exists; adequate for nu-skool/big + beat. The gap is *measurement-level* (downbeats on syncopated kicks), + not signature-level. +10. **downtempo / trip-hop** (60–110 with beats) — falls in the crack + between `ambient` (onsetDensity 0–3, i.e. nearly beatless) and + `hiphop` (centroid ≤ 2500). No dedicated signature. + +### Family-map and cross-boundary gaps + +1. `_GENRE_FAMILY_MAP` (`analyze_detection.py:1078`) has **no entries for + uk-garage, bassline, or garage** — all three fall to `"other"`, so the + mix-doctor family fallback (`mixDoctor.ts:77`) never fires for garage. +2. The TS side pins `genreFamily` to a closed union of 8 values — + `apps/ui/src/types/measurement.ts:652` and the `validFamilies` allowlist + in `apps/ui/src/services/backendPhase1Client.ts:1134`. **Any new family + must land on both sides** (tripwire #3) or it silently degrades to + `"other"`. +3. The vocal boost list (`analyze_detection.py:1205`) and supersaw/acid + boosts don't reference any of the missing genres — fine today, but new + signatures for trap (808 + vocal) and hardstyle (kick THD) should reuse + the existing `kickDistortion` axis rather than invent new ones. + +## Surface 2 — synthetic corpus span (build_synthetic_corpus.py) + +### What the 29 clips cover + +| Axis | Coverage | +|---|---| +| Tempo (grid clips) | 70, 90, 110, 128, 140, 174 BPM | +| Tempo (other kinds) | chords 96–140, swing 124, multi 122/128 | +| Meter | 4/4, 3/4, 6/8, 7/8 (odd meters are registered `knownGaps`) | +| Rhythm pattern | **one pattern only: kick on every beat, accented downbeat** (`render_grid_pattern`) | +| Swing | straight-vs-swung **8th-note** hats, 50–66%, at 124 BPM only | +| Key/chords | 12 roots × major/minor, clean sine triads | +| Texture | drums-only, chords-only, bass-only, 2 multi-layer 4/4 clips | + +Already-registered `knownGaps` (`_KNOWN_GAPS_BY_ID`, +`build_synthetic_corpus.py:395`): odd meters read as 4/4 (downbeats follow), +7/8 tempo smears, and **174 BPM halves to 86.9** (octave preference). + +### What is absent — the priority-genre rhythm surface + +1. **Broken kick patterns (2-step, breakbeat, electro).** Every grid clip + places a kick on every beat. No active check exercises the downbeat + heuristic (`_compute_downbeat_phase`, `analyze_rhythm.py:160` — it + *documents* that four-on-the-floor carries no phase information) or the + meter autocorrelation on syncopation. +2. **Halftime feel (kick 1 / snare 3).** The canonical tempo-octave trap — + a 140 BPM halftime clip whose truth is 140, not 70 — does not exist. + Both estimators (RhythmExtractor2013 + Percival) plus the ratio-based + `apply_bpm_correction` (`analyze_core.py:39`, which *always* sides with + Percival on a ~2×/1.5× ratio) are unexercised on this case. +3. **Triplet / shuffle swing and 16th-note shuffle.** Swing clips stop at + 66% on the 8th grid. UKG/2-step shuffle lives on the **16th** grid — + which `compute_swing_detail` (`analyze_rhythm.py:191`) cannot see: it + keeps only IOIs in 0.30–0.70 beats and hardwires + `gridResolution: "8th"`. +4. **Sparse / beatless material.** No clip verifies honest abstention — + tempo confidence collapsing, `swingDetail: None`, meter falling back to + `assumed_four_four`, key ensemble hedging on drones. (The chords clips + are beatless but only check key/chords; nothing asserts what the rhythm + stack *should not* claim on them.) +5. **Tempo extremes 85 / 150 / 190.** 85 (halftime/trap felt tempo) and + 150 (hardstyle) are absent; 174 exists only as a knownGap; **nothing + above 174 at all** (gabber territory, and the octave-halving failure + is *worse* there). + +## Surface 3 — mix-doctor genre profiles (genreProfiles.ts) + +The 34 profile ids mirror the backend signature table one-for-one, so every +signature gap above is also a profile gap: **no trap, footwork, hardstyle, +gabber, electro, IDM, 2-step, speed garage, halftime, or downtempo +spectral/loudness targets.** + +Consequence chain: a trap track misclassified as `dubstep` receives +dubstep's `spectralTargets` and LUFS/crest ranges from +`resolveMixDoctorGenreId` (`mixDoctor.ts:93`); the advice is specific, +confident, and wrong — the exact failure mode PURPOSE.md invariant #4 +exists to prevent. The genre-agnostic loudness guardrails +(`loudnessGuardrails.ts`) still hold, but every genre-relative suggestion +inherits the misclassification. + +Profile shape (`genreProfiles.ts:28`): crest/PLR/LUFS ranges + 7 spectral +band targets. New profiles are additive data entries; the risk is not +mechanism but **unvalidated targets** — new-genre targets must be sourced +from reference-track measurements, not invented (same discipline as the +2026-05-16 audit applied to the existing table). + +## Measurement-level implications (what the fix PRs target) + +| # | Measurement | Failure mode on priority genres | Evidence | +|---|---|---|---| +| 1 | Tempo octave | 174 → 86.9 measured; 190 unmeasured but same mechanism; halftime trap ambiguous by construction | `_KNOWN_GAPS_BY_ID`, `apply_bpm_correction` always prefers Percival on ratio hits | +| 2 | Meter / downbeats | odd meters read 4/4 (measured); syncopated/broken kicks unmeasured — kick-accent phase heuristic assumes kick-on-downbeat | `analyze_time_signature` 20% margin (`analyze_core.py:1292`), `_compute_downbeat_phase` | +| 3 | Swing | 16th-grid shuffle invisible; `gridResolution` hardwired `"8th"` | `compute_swing_detail` IOI window 0.30–0.70 beats | +| 4 | Key on sparse/ambient | ensemble landed (PR-B3, keep EDMA) but no abstention/hedge check exists on beatless or drone material | `_build_key_ensemble` (`analyze_core.py:159`), corpus gap #4 | +| 5 | Genre classifier | 10 priority families unrepresented or mis-tuned (see Surface 1); family map + TS union must move together | `analyze_detection.py:1031/1078`, `measurement.ts:652`, `backendPhase1Client.ts:1134` | + +## Recommended sequence + +See `plans/genre-generalization-program.md` for the PR-by-PR program this +audit feeds. Ordering rationale: corpus first (you can't gate what you +can't measure), then measurement fixes each behind the new clips' +pre-registered checks, classifier/profile widening last (it depends on the +measurements being right for the new material). diff --git a/plans/genre-generalization-program.md b/plans/genre-generalization-program.md new file mode 100644 index 00000000..eb33607f --- /dev/null +++ b/plans/genre-generalization-program.md @@ -0,0 +1,161 @@ +# Genre-generalization program — from "passes on techno" to "passes across electronic" + +Successor to the Phase A/B accuracy program (ground-truth harnesses; meter +evidence, real swing, key ensemble, stem-aware chords — all merged). The +accuracy gates currently over-index on four-on-the-floor techno/house: the +synthetic corpus is steady-kick 4/4 at mostly 96–140 BPM, and entire +electronic families have no signature, no mix-doctor profile, and no corpus +clip. The measured evidence is in +`audits/genre-coverage-2026-07-13.md` — read it first; this plan is the +PR-by-PR execution of its findings. + +Priority genres: UK bass / garage / speed garage / 2-step; halftime / trap / +dubstep / bass; hard dance / hardstyle / gabber; IDM / ambient / electro; +plus footwork, breakbeat, downtempo. + +## Non-negotiable discipline (how every prior PR shipped) + +1. **Additive or surfacing-only, behind a pre-registered gate.** Never + override an authoritative Phase 1 value (PURPOSE.md invariant #1). A fix + that would change a shipped value (e.g. the octave-corrected `bpm`) + first ships as surfaced evidence + a frozen decision doc in + `incorporations/`, exactly like the key-ensemble gate + (`incorporations/key-ensemble-decision-2026-07-04.md`). +2. **Every new field lands on both sides**: `analyze.py` camelCase → + `JSON_SCHEMA.md` + `EXPECTED_TOP_LEVEL_KEYS` (and the full-only list) → + `src/types/measurement.ts` → `backendPhase1Client.ts` reconstructor → + `phase1FullPayload.ts` fixture → `phase1CitationContract.test.ts` if + citable. Then re-baseline the golden deliberately: + `UPDATE_PHASE1_GOLDEN=1 ./venv/bin/python -m unittest tests.test_phase1_golden`. +3. **Green before every push** (from repo root): + ```bash + cd apps/backend && ./venv/bin/python -m unittest discover -s tests + ./venv/bin/python scripts/build_synthetic_corpus.py --check --audio-only \ + --out-dir tests/fixtures/fundamentals_tracks \ + --manifest tests/fixtures/fundamentals_eval_manifest.json + ./venv/bin/python scripts/evaluate_fundamentals.py --fail-on-skip + cd ../ui && npm run verify + ``` +4. **One coherent change per PR**, stacked for review — behavior changes + are shown before merging. Honest PR bodies: what is validated + synthetically vs. what still needs real audio. +5. New fundamentals clips that fail at baseline enter + `_KNOWN_GAPS_BY_ID` (report-but-don't-gate), exactly like the odd-meter + and 174 BPM entries — the gap list is the program's burn-down chart. + +## PR sequence + +### PR-G1 — Coverage audit (docs only) — this PR + +`audits/genre-coverage-2026-07-13.md` + this plan. No behavior change. + +### PR-G2 — Synthetic corpus expansion + gate promotion + +Extend `scripts/build_synthetic_corpus.py` with known-truth clips spanning +the missing rhythm/tempo surface, and promote them into the active +fundamentals manifest (`fundamentals_eval_manifest.synthetic.json`): + +1. **2-step broken kick** (~132 BPM): kick on 1 and the "and" of 2 (or + equivalent classic pattern), snare on 2 and 4 — truth: bpm, 4/4, + beatGrid, downbeats. Exercises the kick-accent downbeat phase heuristic + off its four-on-the-floor assumption. +2. **Halftime feel** (kick 1 / snare 3) at 140 and 85 — truth pins the + *notated* tempo; this is the deliberate tempo-octave trap. +3. **Triplet/shuffle swing** (66.7% triplet target) and **16th-note + shuffle** clips (UKG-style) — expected initially to land in + `knownGaps` because `compute_swing_detail` only reads the 8th grid. +4. **Breakbeat syncopation**: sampled-breaks-style pattern (syncopated + kick, ghost snares) — beat/downbeat truth. +5. **Sparse/beatless ambient**: slow pad, no percussion — the *abstention* + clip. Active checks assert what the pipeline should NOT claim: + low/absent tempo confidence, `swingDetail` None or `direction: + straight` with low confidence, meter on the assumed-4/4 fallback, key + still correct on tonal pads. +6. **Tempo extremes**: grid clips at 85, 150, 174 (exists), 190. 190 and + likely 150+ enter `knownGaps` at baseline (octave halving). + +Every new clip kind gets `_EXPECTED_KEYS_BY_KIND` / `_THRESHOLDS_BY_KIND` +entries and deterministic-render `--check` coverage. Measured baseline +failures are registered in `_KNOWN_GAPS_BY_ID` with the owning fix PR named +in a comment. This PR changes no product code. + +### PR-G3 — Tempo octave handling at extremes (surfacing first) + +Surface octave evidence additively: candidate tempos (measured, ×2, ÷2 with +their supporting evidence — onset-rate consistency, beat-grid density, +genre-prior band) as a new full-mode field; never rewrite `bpm`. Gate: the +PR-G2 halftime + extreme clips flip from `knownGaps` to active passes on +the *evidence* field (does the true octave appear as a candidate with +dominant support?). Promotion of any evidence-driven correction into the +shipped `bpm` requires a frozen `incorporations/` decision doc measured on +GiantSteps tempo (staged locally, 664 clips) — same protocol as the +key-ensemble gate. + +### PR-G4 — Meter/downbeats on non-4/4 and broken kicks + +The measured weak layer (audit baseline 2026-07-03). Improve +`analyze_time_signature`'s discrimination (the 20% margin conservatism) and +`_compute_downbeat_phase`'s kick-on-downbeat assumption using the meter +evidence field (Phase B) plus snare/backbeat evidence. Gate: PR-G2's +2-step/breakbeat clips + the existing odd-meter knownGaps burn down without +regressing the 29 existing clips or the golden. beat_this (venv-eval, +adopt_pending_asa_slice) remains the candidate backend for the bigger jump; +its gate stays the frozen beat-this doc, not this PR. + +### PR-G5 — Swing on the 16th grid (2-step shuffle) + +Extend `compute_swing_detail` to detect the dominant swing grid (8th vs +16th) instead of hardwiring `"8th"` — additive: `gridResolution` already +exists as a field; new evidence lands beside it. Gate: PR-G2 shuffle/16th +clips activate; existing 8th-grid swing clips (50–66%) must not move. + +### PR-G6 — Key honesty on sparse/ambient material + +No algorithm change expected: the ensemble (keep-EDMA) already ships. This +PR adds the abstention/hedge checks from the PR-G2 ambient clips (key +confidence + ensemble disagreement behavior on drones/beatless pads) and +fixes only what those checks prove wrong, surfacing-only. + +### PR-G7 — Genre classifier + mix-doctor widening (both sides, one PR) + +Add signatures for the missing families — trap, footwork, hardstyle, +gabber/hardcore, electro, 2-step, speed garage, halftime, downtempo — and +tune uk-garage's sidechain assumption. IDM is a verified-abstain, not a +signature. Must move together (tripwire #3): + +1. `_GENRE_SIGNATURES` + `_GENRE_FAMILY_MAP` (backend), +2. the `genreFamily` union in `src/types/measurement.ts:652` + the + `validFamilies` allowlist in `backendPhase1Client.ts:1134`, +3. `genreProfiles.ts` mix-doctor profiles with *sourced* targets + (reference-track measurements, not invented numbers), +4. golden re-baseline if any fixture's classification shifts. + +Gate: classification checks on PR-G2 clips where the synthetic render is +genre-representative; for families a synthetic clip can't honestly +represent (gabber kick THD, trap 808 character), the PR body says so and +the check waits for real-audio fixtures. + +## Real-audio validation (operator machine, network available) + +From `plans/owner-actions-accuracy-program.md`: GiantSteps key/tempo (604 + +664 clips) and GTZAN + GTZAN-Rhythm are **already staged locally** +(2026-07-05). Remaining: + +1. **Demucs weights fetch** — first `--separate` run on this machine pulls + the torchaudio Hybrid Demucs weights, activating the stem-aware chord + path (PR-B4) that cloud sessions could never exercise (falls back to + full-mix without weights). Then re-run the fundamentals multi clips and + record whether the stem path holds its ≥0.45 floor. +2. **GiantSteps tempo re-run** whenever PR-G3 surfaces octave evidence — + the real-audio octave-error rate is the gate metric. +3. **ASA 18-clip beat slice** stays the owner's hand-annotation task (the + one deliberately non-automatable item). + +## Definition of done + +The fundamentals summary gates on clips spanning broken-kick, halftime, +shuffle, breakbeat, beatless, and 85–190 BPM material — with +`_KNOWN_GAPS_BY_ID` empty or every remaining entry tied to a named, +frozen-gate decision doc — and the genre surfaces (signatures, family map, +TS union, mix-doctor profiles) cover the priority families on both sides of +the contract.