Skip to content

test(samples): end-to-end audio-content assertions for audition output - #46

Merged
slittycode merged 2 commits into
mainfrom
claude/gemini-sample-generation-U753E
May 14, 2026
Merged

test(samples): end-to-end audio-content assertions for audition output#46
slittycode merged 2 commits into
mainfrom
claude/gemini-sample-generation-U753E

Conversation

@slittycode

Copy link
Copy Markdown
Owner

What this is

Follow-up to #45. Adds end-to-end audio-content test coverage for the audition-sample feature — the layer that was missing.

Why

#45's tests verified the audition pipeline in segments:

  • the plantest_sample_theory.py asserts correct MIDI numbers for a key
  • the primitivestest_sample_synthesis.py (one note → right frequency), test_sample_drums.py (synth_kick → right fundamental)
  • the orchestrated output existstest_sample_generation.py checks the WAVs are written, non-empty, and have correct metadata/citations

But nothing loaded the orchestrated WAV files back off disk and checked the audio itself reflected the Phase 1 input. test_sample_generation.py asserts the kick's label says "55 Hz" — it never asserted the kick audio was at 55 Hz. The chain was tested link by link but never end-to-end as sound.

What this adds

apps/backend/tests/test_sample_audio_content.py — 6 tests that run the real orchestrator, load the WAVs it wrote, FFT them, and assert on spectral content:

Test Verifies
test_c_major_chord_audio_contains_the_triad The C-major chord WAV carries real energy at C4/E4/G4, and the weakest chord tone is >20× louder than a non-chord tone (C#4) — the audio is in the key, not loud noise
test_bass_root_audio_sits_on_the_tonic The bass WAV's dominant frequency is C2 (~65.4 Hz) within 4 Hz
test_different_keys_produce_different_audio The load-bearing test. Generates C major and F# minor; proves C-natural is >20× louder in the C-major render than the F#-minor render, and F# is >20× louder the other way. The render genuinely tracks the measured key — it's not a canned clip
test_orchestrated_kick_lands_on_measured_fundamental The kick WAV the endpoint serves peaks at the measured kickDetail.fundamentalHz (extends test_sample_drums, which only tested synth_kick directly)
test_kick_fundamental_tracks_the_measurement A 45 Hz vs 95 Hz measurement yield audibly different kicks, each on its own fundamental
test_melody_follows_scale_degree_hints Hints [1,5] in C major render C5 then G5, in that order, ascending

Scope / honest caveats

  • Runs against the sine-additive fallback (prefer_fluidsynth=False) so assertions are deterministic and don't depend on a system soundfont — same reason CI can't assume FluidSynth is installed. The FluidSynth path is structurally exercised by the same orchestrator code but not asserted on.
  • The bar is "in the right key / on the right fundamental" — not "sounds like the source track." That's the deliberate scope from the design doc: audition validates the measurement chain, not Ableton timbre.
  • Snare/hat aren't asserted here — Phase 1 has no measured ground truth for them, so there's nothing to assert against beyond "well-formed" (already covered in test_sample_drums.py).

All 6 tests pass locally. Pure addition — one new file, no changes to existing code.

https://claude.ai/code/session_015L67xTQdgRuhCRpJurJWYd


Generated by Claude Code

The existing sample tests verify the plan (correct MIDI numbers), the
primitives (one note renders at the right pitch, one kick lands on its
fundamental), and that the orchestrated WAVs exist and are well-formed —
but nothing loaded those WAVs back to check the audio *content* reflects
the Phase 1 input. This closes that loop.

test_sample_audio_content.py runs the real orchestrator, loads the WAV
files it wrote to disk, FFTs them, and asserts on spectral content:

- C-major chord WAV carries real energy at C4/E4/G4, with the weakest
  chord tone >20x louder than a non-chord tone (C#4) — the audio is in
  the key, not loud noise.
- Bass-root WAV's dominant frequency is C2 within 4 Hz.
- Different keys produce different audio: C-natural is >20x louder in a
  C-major render than an F#-minor render, and F# is >20x louder the other
  way. This is the load-bearing test — it proves the generator is
  responsive to the measured key, not emitting a canned clip.
- The orchestrated kick WAV peaks at the measured kickDetail.fundamentalHz
  (extends test_sample_drums, which only tested synth_kick directly).
- A 45 Hz vs 95 Hz measurement yield audibly different kicks, each on its
  own fundamental.
- Melody hints [1,5] in C major render C5 then G5, in order, ascending.

Runs against the sine-additive fallback (prefer_fluidsynth=False) so the
assertions are deterministic and don't depend on a system soundfont. The
FluidSynth path is structurally exercised by the same code but not
asserted on, for the same reason CI can't assume a soundfont is installed.

All 6 tests pass locally.

https://claude.ai/code/session_015L67xTQdgRuhCRpJurJWYd

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

Pure test addition (one new file, no production code touched) that closes a real coverage gap: the audition pipeline was tested link-by-link but never end-to-end as audio. All six tests are methodologically sound. Frontend CI passes; backend CI was still in-progress at review time — the code is clean and I'd be surprised if it fails.

Findings

Worth considering (not blocking):

  • test_melody_follows_scale_degree_hints — the timing comment says "note 0 spans ~0–3.4 s" but at 120 BPM with 2 bars/note, note 0 spans 0–4 s. The sampling windows (0.5–3.0 s and 4.5–7.0 s) are correct and safely inside both notes, so the test works fine; the comment is just slightly off.

  • test_orchestrated_kick_lands_on_measured_fundamental with delta=20.0 is generous (72 ± 20 Hz), but it's consistent with the delta=20.0 tolerance already established in test_sample_drums.py and justified by the pitch-envelope settling behavior called out in the comment.

Nothing else. The Hann windowing is the right call for inter-bin leakage; the 20× energy ratio is a meaningful threshold (not just > 0); prefer_fluidsynth=False correctly isolates the tests from system soundfont state; the generate_samples call signature matches the actual implementation exactly.

Test results

Frontend: pass. Backend CI in-progress at review time; no issues visible in the code.

Phase boundary check

Clean. _phase1() constructs a mock input; generate_samples consumes it. No Phase 1 data is mutated or re-derived anywhere in the new file.


Generated by Claude Code

Review nit on #46: the comment conflated a note's sounding duration with
its slot. Each note gets an 8-beat (4 s) slot at 120 BPM; the 0.85
note-length gate means it actually sounds for ~3.4 s of that. The
sampling windows were already correct — only the comment was imprecise.
Comment-only change; no test behavior affected.

https://claude.ai/code/session_015L67xTQdgRuhCRpJurJWYd
@slittycode
slittycode merged commit e0af77c into main May 14, 2026
2 checks passed
@slittycode
slittycode deleted the claude/gemini-sample-generation-U753E branch May 14, 2026 08:44
slittycode added a commit that referenced this pull request May 16, 2026
…#49)

Documentation drifted vs the code in several places. This sweep brings
the living docs back in sync and archives one-shot review docs whose
action items have all shipped.

Updated in place:
- BACKLOG.md: move audition samples from "In Progress" to "Shipped"
  (landed in #45, #46) and link the merged backend/frontend modules.
- docs/SAMPLE_GENERATION.md: header changes "Status: Prototype on
  branch X" -> "Status: Shipped".
- apps/ui/README.md: drop "Google Gen AI SDK" from the Tech Stack —
  @google/genai was removed from package.json; Gemini is fully
  backend-mediated.
- apps/backend/ARCHITECTURE.md: expand the Components table to cover
  csv_export.py, dsp_utils.py, dsp_bandbank.py, sample_*.py,
  server_samples.py, stage_status.py, symbolic_extract.py,
  url_ingest.py, phase1_evaluation.py, phase1_report_html.py,
  utils/cleanup.py. Add the missing routes: /interrupt, /samples
  (POST + GET).
- apps/backend/README.md: extend the canonical-routes list with
  DELETE /runs, /interrupt, /source-audio, /export/csv,
  /spectral-enhancements, /pitch-note-translations,
  /interpretations, /samples.
- apps/backend/AGENTS.md: refresh the File Map with the same new
  modules, add an Operator/Research Scripts section covering
  scripts/, and bump the stale 2026-03-10 date stamp.
- apps/ui/AGENTS.md: expand the File Map with analysisRunsClient,
  phase2Validator, mixDoctor, spectralArtifactsClient,
  sampleGenerationClient, fieldAnalytics, midi/, sessionMusician/,
  and the types/ barrel. Note that the UI does not import an AI SDK.
- CLAUDE.md: extend the backend "Core files" list with the same new
  modules; explicitly call out the audition-samples surface.

Archived (moved to docs/history/):
- docs/external-repo-review-2026-05-13.md — completed review; Tracks
  1, 2, 3 all shipped or deferred. Outcomes summarized in an
  archive-note header.
- docs/track1-spike-outcome-2026-05-13.md — completed loudness
  verification spike; fix landed and regression test is live.

Cross-references updated:
- docs/adr/0001-phase1-json-schema-v1.md
- apps/backend/csv_export.py (module docstring)
- apps/backend/tests/test_loudness_r128.py (module docstring)
- docs/history/README.md index
- relative links inside the two archived docs

https://claude.ai/code/session_01YNsCxdcxQiTieUNvAgKMd9

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants