feat(spectral): librosa.reassigned_spectrogram as an opt-in enhancement - #43
Conversation
Last item from the merged external-repo review's adopt list. Adds a
new on-demand spectral-enhancement kind, 'reassigned', that produces
a sharper spectrogram via per-bin (time, frequency) reassignment.
Why reassignment: vanilla STFT spectrograms smear transients across
the analyzing window (~46 ms for n_fft=2048 at 44.1 kHz). Reassignment
relocates each STFT bin to the local centroid of energy in the
(time, freq) plane — transients land at their true onset and stable
partials collapse to sharp horizontal lines. Useful when a producer
is trying to read attack timing or harmonic detail from the existing
mel/STFT views and finding them too blurry.
Implementation:
- apps/backend/spectral_viz.py — new generate_reassigned_spectrogram
function. Calls librosa.reassigned_spectrogram with both frequency
and time reassignment enabled, fill_nan=True (handles silent-region
numerical instability without dropping points), and renders via
matplotlib scatter (not librosa.display.specshow — specshow would
re-rasterize to a uniform grid and undo the sharpening). Magnitude
floor of -60 dBFS below peak filters the noise wash without losing
attack detail.
- apps/backend/server.py — registered as 'reassigned' in the existing
_ENHANCEMENT_GENERATORS dict. Inherits all existing wiring: auth,
measurement-completed precondition, idempotency, artifact storage,
MIME types, JSON response envelope.
- apps/backend/tests/test_spectral_viz.py — 3 new tests:
- produces_single_reassigned_png: shape + non-trivial size
- output_is_valid_png: magic bytes
- handles_silent_input_without_raising: regression gate against
NaN propagation on near-silent fixtures
No new dependencies — librosa is already in requirements.txt
(librosa==0.11.0 exposes reassigned_spectrogram).
Usage:
POST /api/analysis-runs/{run_id}/spectral-enhancements/reassigned
Returns the new 'spectrogram_reassigned' artifact ID alongside the
existing PNG-style spectrograms. Idempotent: re-posting returns the
existing artifact without regenerating.
Out of scope (deferred):
- Frontend UI toggle to display the reassigned PNG. The backend now
produces it on demand; the UI wiring is a separate follow-up.
- Raw (time, freq, mag) JSON artifact for client-side custom
visualization. Could be added as a sibling 'reassigned_data'
artifact if a future consumer needs it.
Closes the last review-adopt item. Full session summary in the
merge commits for #33 through #42.
https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg
slittycode
left a comment
There was a problem hiding this comment.
Verdict: APPROVE
Summary
Adds an opt-in reassigned spectrogram artifact via the existing enhancement route. Purely additive — one new function in spectral_viz.py, one new entry in _ENHANCEMENT_GENERATORS, three tests. Phase 1 output is untouched. Tests could not be run (no venv in reviewer workspace), but code-review of ReassignedSpectrogramTests shows meaningful coverage of the three cases that matter: normal output, valid PNG header, and NaN propagation on near-silent input. Architecture is clean.
Findings
Should fix — scatter point ceiling is missing (spectral_viz.py, the ax.scatter(...) call)
The rest of the file guards time-series output with DEFAULT_MAX_POINTS = 500 and _downsample_time_series. This function has no equivalent cap. For a 3-minute track at default n_fft=2048 / hop_length=1024: 1025 bins × ~7,750 frames ≈ 7.9M points before the floor mask; expect 4–5M survivors. For a 10-minute track it's north of 15M. matplotlib.Axes.scatter at that scale takes tens of seconds for the render step alone — the librosa compute cost is not the bottleneck here.
Fix is small: add max_scatter_points: int = 300_000 parameter, then after building the mask:
idx = np.arange(mask.sum())
if idx.size > max_scatter_points:
idx = np.random.default_rng(0).choice(idx.size, max_scatter_points, replace=False)
flat_times_vis = flat_times[mask][idx]
flat_freqs_vis = flat_freqs[mask][idx]
flat_mags_vis = flat_mags_db[mask][idx]300K points renders in under a second and still looks sharp at FIG_DPI=100. Could be a follow-up PR if this is close to merge.
Worth noting — CLAUDE.md says "inferno colormap" but the codebase already uses magma everywhere
Not introduced by this PR — every existing generator uses cmap="magma" and this PR follows that. The CLAUDE.md rule is stale and will confuse future contributors. Worth a one-line correction in any nearby docs pass.
Test results
Could not execute (no venv). Code review of ReassignedSpectrogramTests: all three tests are meaningful. The silent-fixture test correctly uses imperceptible dither (1e-6 amplitude noise) rather than true zeros, which exercises the fill_nan path and np.isfinite mask without making librosa.amplitude_to_db(ref=np.max) undefined.
Phase boundary check
Clean. No Phase 1 fields read, mutated, or re-derived. The new artifact is produced from the raw audio path exactly the same way every other enhancement generator works.
Generated by Claude Code
PR #43 review caught a real perf issue: the original generate_reassigned_spectrogram fed every STFT cell into matplotlib.scatter unchecked. At default params (n_fft=2048, hop=1024, sr=44100), that's: ~8M raw points on a 3-minute track ~15M raw points on a 10-minute track Both well above the floor-mask survival count and well above the threshold where matplotlib's render step starts taking tens of seconds. The librosa compute cost isn't the bottleneck — the scatter render is. Fix: - New REASSIGNED_MAX_SCATTER_POINTS = 300_000 constant. At FIG_DPI=100 / ~1200x400 px figure, 300K is the visual sweet spot — sharper-looking than vanilla STFT, renders in under a second. - After the floor mask, the post-mask arrays (times_visible, freqs_visible, mags_visible) get subsampled via a seeded RNG (np.random.default_rng(0)). Seeded so re-running on the same audio produces a byte-identical PNG — matters for idempotency tests and content-hash artifact caching. - New test test_scatter_point_cap_is_enforced_for_long_inputs: generates a 60 s multi-component fixture (~1M+ post-mask points, comfortably above the 300K cap), patches matplotlib.scatter to capture call args without rendering, and asserts the actual point count passed to scatter is <= REASSIGNED_MAX_SCATTER_POINTS. The 'inferno colormap' worth-noting from the review was checked — no 'inferno' reference exists in CLAUDE.md or anywhere in the codebase. The reviewer's worth-noting was itself stale; nothing to fix there. https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg
Last item from the merged external-repo review's adopt list. Adds an opt-in spectral-reassignment enhancement that produces a sharper spectrogram than the existing mel/STFT views.
Why reassignment
Vanilla STFT smears transients across the analyzing window (~46 ms for
n_fft=2048at 44.1 kHz). Reassignment relocates each STFT bin to the local centroid of energy in the (time, frequency) plane:Useful when a producer is trying to read attack timing or harmonic detail from the existing mel/STFT views and finding them too blurry.
How it integrates
Reuses the existing on-demand enhancement route — no new HTTP surface:
The route already supports auth, measurement-completed precondition, idempotency, artifact storage, MIME types, and JSON envelopes. Adding a new kind is a one-line entry in
_ENHANCEMENT_GENERATORS. Returns the newspectrogram_reassignedartifact ID; re-posting is idempotent.Implementation
apps/backend/spectral_viz.pygenerate_reassigned_spectrogramfunction (~95 lines). Callslibrosa.reassigned_spectrogramwith both frequency and time reassignment enabled,fill_nan=True(handles silent-region instability), and renders via matplotlib scatter — notlibrosa.display.specshow, which would re-rasterize to a uniform grid and undo the sharpening. Magnitude floor of -60 dBFS below peak filters the noise wash without losing attack detail.apps/backend/server.py_ENHANCEMENT_GENERATORS:"reassigned": ("generate_reassigned_spectrogram", ["spectrogram_reassigned"], True).apps/backend/tests/test_spectral_viz.pyReassignedSpectrogramTests: shape + size, valid PNG magic bytes, regression gate against NaN propagation on near-silent input.apps/backend/ARCHITECTURE.mdreassignedas a validkindin the route list.No new dependencies
librosa==0.11.0is already inrequirements.txtand exposeslibrosa.reassigned_spectrogram.What this PR does NOT do
(time, freq, mag)JSON artifact. Could be added as a siblingreassigned_datakind if a future consumer needs to do client-side custom visualization. Out of scope for v1.Session closeout: review-adopt list complete
This PR completes the merged external-repo review's full adopt list:
After this lands, every adopt item from the review is on
main.https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg
Generated by Claude Code