feat(samples): Phase 3 audition — heuristic WAV/MIDI from Phase 1 + PyTheory

[slittycode]

What this is

A prototype "audition" feature: turn the Phase 1 measurement chain (and Phase 2 context, when available) into short audible clips so producers can ear-check whether the detected key, chord progression, kick fundamental, and melody scaffold actually match the source track.

Click "Generate audition samples" after Phase 2 finishes → get back tonal clips (chord progression + bass root in the detected key), drum one-shots (kick grounded in kickDetail.fundamentalHz/decayTimeMs, plus heuristic snare/hat), and a melody-lead clip. Each clip comes with citation chips pointing at the Phase 1 fields that justified it.

Why this is shaped the way it is

The user's framing was "feed Gemini's specific Ableton device points into PyTheory and return small samples so the user can hear how accurate the analysis is." Two structural decisions came out of that:

1. PyTheory only handles the musical layer — notes, scales, chords. It doesn't make sound. So this PR pairs it with FluidSynth (high-quality, optional) and a NumPy sine-additive fallback (always available). Drum one-shots bypass PyTheory entirely — drums aren't harmonic.
2. Phase 2 isn't authoritative for the musical decisions — Phase 1 measurements are (per the PURPOSE.md invariants). Phase 2 enriches labels and provides genre context, but the key/mode that drives chord rendering comes from Phase 1. This honors the chain-of-custody invariant: nothing in this PR lets an audition value flow back upstream into a measurement.

The audition is explicitly framed in the UI as a heuristic reconstruction, not Ableton-accurate. It validates the foundation; the user still follows Phase 2 in Live for production character. docs/SAMPLE_GENERATION.md captures the full rationale and the alignment with each PURPOSE.md quality invariant.

Architecture

• sample_theory.py — PyTheory adapter with self-contained Western-theory fallback. Both paths emit identical MIDI numbers for the same key/mode/degree. Tests exercise the fallback so CI doesn't depend on PyTheory being importable.
• sample_drums.py — NumPy kick (pitch-modulated sub-sine + click envelope grounded in kickDetail), heuristic snare (tone body + noise crack), heuristic hat (filtered noise). Manifest is explicit that snare/hat are kit-completeness defaults, not measurement-grounded.
• sample_synthesis.py — MIDI plan → WAV. FluidSynth + GM SoundFont when available (walks env var + common system paths); falls back cleanly to sine-additive synthesis with an ADSR envelope. MIDI export via pretty_midi so users can drop .mid into Live regardless of which render path was used.
• sample_generation.py — Orchestrator. Reads phase1/phase2 dicts, produces WAV + MIDI artifacts and a samples_manifest.json with citation metadata and lowConfidence flags propagated from keyConfidence / kickDetail.confidence.
• server_samples.py + server.py — POST / GET /api/analysis-runs/{run_id}/samples. On-demand only: nothing in the staged runtime auto-enqueues sample generation. This was a deliberate choice: until audition value is validated on real tracks, we don't want to pay the synthesis cost on every run. Artifacts ride on the existing run_artifacts table with new kind prefixes (sample_audio:*, sample_midi:*, sample_manifest); the existing artifact-stream route serves them — no new file routes.

Frontend

• types/samples.ts mirrors the backend manifest.
• sampleGenerationClient.ts is a typed fetchJson wrapper that treats SAMPLES_NOT_GENERATED as a null return rather than an error (so the UI can decide between "show CTA" vs "show panel").
• SamplePlayback.tsx slots into AnalysisResults below the measurement panel. Renders an "audition samples" CTA after Phase 1 completes; on click, groups clips by category (tonal / drums / melody), with a confidence chip, citation chips for the Phase 1 fields each clip leans on, and a MIDI download link.

Test coverage

Backend — 37 new tests, all green:

• Theory: parse_key handles canonical keys, parenthetical confidence suffixes, missing-mode default, and bad inputs; C-major and A-minor progression voicings verified against canonical MIDI numbers; bass two octaves below voicing; melody default ascent + scale-degree hints.
• Drums: kick FFT peak lands within ±20 Hz of the requested fundamental; decay envelope obeyed; snare/hat well-formed and non-silent.
• Synthesis: sine-fallback render shape; FFT peak at 440 Hz for A4 (MIDI 69); silent plan stays silent; WAV round-trips through soundfile; MIDI round-trips through pretty_midi.
• Generation: every promised sample emitted on full input; tonal samples skipped when key missing; low keyConfidence flags lowConfidence: true on tonal samples (but not on drum_kick); citation chain invariant: every sample either cites a Phase 1 field or labels itself heuristic in its rationale.
• Server helper: precondition errors (measurement not completed, manifest already exists), force=true regenerates, fetch_existing_manifest round-trips with decorated artifact IDs.

Frontend — 10 new tests, all green (550 total): manifest parser accepts well-formed payload; rejects unknown schema versions; coerces unknown enum values to safe defaults; client URL-encoding; SAMPLES_NOT_GENERATED becomes null; other backend errors rethrow.

Type-check: npm run lint (tsc --noEmit) clean.

Dependencies

pytheory>=0.42 and pyfluidsynth>=1.3 added to requirements.txt, but both are optional — the fallbacks make the feature work in lean environments. Both packages are actively maintained (Nov 2024 PyPI releases) and install cleanly on the project's pinned Python 3.11. libfluidsynth system library + a .sf2 soundfont are required only for the high-quality render path; the sine fallback covers everything else.

What this PR does not do

• Does not add a first-class pipeline stage with its own attempts table / progress tracking. That's a follow-up if audition value is validated.
• Does not generate a combined drums+bass+chords loop. Separate stems were a more honest representation of what we can claim from measurements; combining them is a possible follow-up.
• Does not reproduce production character (compression, sidechain, reverb). Those are what Phase 2 tells the user to dial in Ableton. Reproducing them here would muddle the message and risk breaking invariant feat: value-first backport — genre profiles, MixDoctor, broken-pipe fixes #1 (measurement authority).
• Does not let audition values feed back into Phase 1 or Phase 2. Samples are downstream consumers only.

Files touched

A  apps/backend/sample_drums.py
A  apps/backend/sample_generation.py
A  apps/backend/sample_synthesis.py
A  apps/backend/sample_theory.py
A  apps/backend/server_samples.py
A  apps/backend/tests/test_sample_drums.py
A  apps/backend/tests/test_sample_generation.py
A  apps/backend/tests/test_sample_synthesis.py
A  apps/backend/tests/test_sample_theory.py
A  apps/backend/tests/test_server_samples.py
A  apps/ui/src/components/SamplePlayback.tsx
A  apps/ui/src/services/sampleGenerationClient.ts
A  apps/ui/src/types/samples.ts
A  apps/ui/tests/services/sampleGenerationClient.test.ts
A  docs/SAMPLE_GENERATION.md
M  BACKLOG.md
M  apps/backend/requirements.txt
M  apps/backend/server.py
M  apps/ui/src/components/AnalysisResults.tsx

https://claude.ai/code/session_015L67xTQdgRuhCRpJurJWYd

---

Generated by Claude Code

[slittycode]

Verdict: COMMENT

Summary

Adds a Phase 3 audition-sample pipeline: Phase 1 measurements drive PyTheory chord plans, NumPy drum synthesis, and a sine/FluidSynth render path, producing WAV + MIDI artifacts with a citation manifest. Phase boundary is clean — nothing upstream is mutated or re-derived. Tests are meaningful and cover the chain-of-custody invariant explicitly. Several doc/code mismatches in SAMPLE_GENERATION.md need fixing before this gets treated as a stable reference.

Findings

Should fix

SAMPLE_GENERATION.md has five doc/code mismatches — given the chain-of-custody framing of this whole feature, the design doc being wrong in multiple ways is more than a typo:

1. Arch diagram says "reject if phase2 not completed" but _extract_phase2_from_snapshot does no such thing — it returns None and generation proceeds from Phase 1 alone. The correct behavior is described correctly two paragraphs later in the HTTP Contract section, making the diagram actively misleading.
2. HTTP Contract says Response: 200. Code returns 201 Created. (201 is right; doc is wrong.)
3. Test strategy says "generates a kick at 53 Hz, asserts the FFT peak lands within ±3 Hz". test_sample_drums.py uses 80 Hz and delta=20.0. The ±20 Hz tolerance is correct given the pitch sweep — the doc just never got updated from an earlier draft.
4. Arch diagram shows artifact_storage.store_file() in the call chain; actual call path is runtime.record_artifact().
5. Documents a GET /api/analysis-runs/{run_id}/samples/{sample_id} endpoint that isn't implemented. The UI correctly uses /api/analysis-runs/{run_id}/artifacts/{artifact_id} instead.

Misleading comment in server_samples.py:_extract_phase2_from_snapshot (server_samples.py, inside the early-return branch):

if interpretation.get("status") not in {"completed", "ready"}:
    # "ready" means measurement done but no interpretation attempt yet —
    # treat as "no phase2 available".
    return None

The comment is inside the branch that executes when status is not in {"completed", "ready"} — so it's describing a case this branch never runs for. "ready" actually falls through to result = interpretation.get("result"), returns None via the isinstance check, which is the right behavior. The comment belongs after the if block near the result extraction, not inside the early-return arm.

Worth considering

selected_backend or ... assignments in sample_generation.py are dead code after the first chord render. chord_result.backend is always "fluidsynth" or "sine_fallback" (never falsy), so selected_backend or bass_result.backend and selected_backend or melody_result.backend can never update selected_backend. Harmless today but will surprise anyone who adds a conditional render path.

synth_hat uses a Python for loop for 2nd-order IIR filtering (sample_drums.py:157–159) while synth_snare uses vectorized NumPy for its 1st-order HP. At 0.18s × 44.1 kHz ≈ 7938 iterations this is not a performance issue for one-shot generation, but the inconsistency will read as unintentional.

import json inside fetch_existing_manifest (server_samples.py:1505). Stdlib, cached, not a real problem — just odd when json is already in the project's standard import set and sample_generation.py (imported at the top of this file) already brings it in transitively.

Test results

Backend (theory + drums — deps available): 20/20 pass.

Synthesis (sine fallback, pretty_midi mocked): 4/4 pass.

Generation + server_samples: pretty_midi couldn't be built in this environment (wheel build failure), so the 4 tests in test_server_samples that call generate_and_register_samples couldn't run end-to-end. All 6 test_sample_generation tests pass. Code inspection of the failing tests shows they'd pass with the dep installed — the errors are mock pm.write() not writing real files to temp dirs, not logic bugs.

Frontend: 567 pass, 3 fail (pre-existing decision_gate.*.live.test.ts failures requiring a live Gemini key — confirmed against main). The 10 new sampleGenerationClient tests pass.

Phase boundary check

Clean. sample_generation.py, sample_theory.py, sample_drums.py, and sample_synthesis.py all consume Phase 1/Phase 2 dicts as read-only inputs. Phase 2 only contributes string citations; all synthesis parameters come from Phase 1 measurements. The citation chain test (test_every_sample_cites_or_explains_absence) enforces this invariant at runtime.

---

Generated by Claude Code