Skip to content

feat(spectral): librosa.reassigned_spectrogram as an opt-in enhancement - #43

Merged
slittycode merged 2 commits into
mainfrom
claude/spectral-reassignment-5XA5r
May 14, 2026
Merged

feat(spectral): librosa.reassigned_spectrogram as an opt-in enhancement#43
slittycode merged 2 commits into
mainfrom
claude/spectral-reassignment-5XA5r

Conversation

@slittycode

Copy link
Copy Markdown
Owner

Last item from the merged external-repo review's adopt list. Adds an opt-in spectral-reassignment enhancement that produces a sharper spectrogram than the existing mel/STFT views.

Why reassignment

Vanilla STFT smears transients across the analyzing window (~46 ms for n_fft=2048 at 44.1 kHz). Reassignment relocates each STFT bin to the local centroid of energy in the (time, frequency) plane:

  • Transients land at their true onset instead of being smeared across a window.
  • Stable partials collapse to sharp horizontal lines instead of fuzzy ridges.

Useful when a producer is trying to read attack timing or harmonic detail from the existing mel/STFT views and finding them too blurry.

How it integrates

Reuses the existing on-demand enhancement route — no new HTTP surface:

POST /api/analysis-runs/{run_id}/spectral-enhancements/reassigned

The route already supports auth, measurement-completed precondition, idempotency, artifact storage, MIME types, and JSON envelopes. Adding a new kind is a one-line entry in _ENHANCEMENT_GENERATORS. Returns the new spectrogram_reassigned artifact ID; re-posting is idempotent.

Implementation

File Change
apps/backend/spectral_viz.py New generate_reassigned_spectrogram function (~95 lines). Calls librosa.reassigned_spectrogram with both frequency and time reassignment enabled, fill_nan=True (handles silent-region instability), and renders via matplotlib scatter — not librosa.display.specshow, which would re-rasterize to a uniform grid and undo the sharpening. Magnitude floor of -60 dBFS below peak filters the noise wash without losing attack detail.
apps/backend/server.py One-line addition to _ENHANCEMENT_GENERATORS: "reassigned": ("generate_reassigned_spectrogram", ["spectrogram_reassigned"], True).
apps/backend/tests/test_spectral_viz.py Three new tests in ReassignedSpectrogramTests: shape + size, valid PNG magic bytes, regression gate against NaN propagation on near-silent input.
apps/backend/ARCHITECTURE.md Documents reassigned as a valid kind in the route list.

No new dependencies

librosa==0.11.0 is already in requirements.txt and exposes librosa.reassigned_spectrogram.

What this PR does NOT do

  • No frontend UI to display the reassigned PNG. The backend produces it on demand; the UI work to show it next to (or replacing) the existing mel spectrogram is a separate follow-up.
  • No raw (time, freq, mag) JSON artifact. Could be added as a sibling reassigned_data kind if a future consumer needs to do client-side custom visualization. Out of scope for v1.
  • Doesn't change the default mel spectrogram pipeline. Existing artifacts are untouched; this is purely additive.
  • No CPU-cost gate. Reassignment is more expensive than vanilla STFT but still well-bounded for ASA's 100 MiB / ~10 min track ceiling. If it ever shows up in profile traces for typical workloads, downsampling the input before reassigning is the obvious fix.

Session closeout: review-adopt list complete

This PR completes the merged external-repo review's full adopt list:

After this lands, every adopt item from the review is on main.

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg


Generated by Claude Code

Last item from the merged external-repo review's adopt list. Adds a
new on-demand spectral-enhancement kind, 'reassigned', that produces
a sharper spectrogram via per-bin (time, frequency) reassignment.

Why reassignment: vanilla STFT spectrograms smear transients across
the analyzing window (~46 ms for n_fft=2048 at 44.1 kHz). Reassignment
relocates each STFT bin to the local centroid of energy in the
(time, freq) plane — transients land at their true onset and stable
partials collapse to sharp horizontal lines. Useful when a producer
is trying to read attack timing or harmonic detail from the existing
mel/STFT views and finding them too blurry.

Implementation:

- apps/backend/spectral_viz.py — new generate_reassigned_spectrogram
  function. Calls librosa.reassigned_spectrogram with both frequency
  and time reassignment enabled, fill_nan=True (handles silent-region
  numerical instability without dropping points), and renders via
  matplotlib scatter (not librosa.display.specshow — specshow would
  re-rasterize to a uniform grid and undo the sharpening). Magnitude
  floor of -60 dBFS below peak filters the noise wash without losing
  attack detail.

- apps/backend/server.py — registered as 'reassigned' in the existing
  _ENHANCEMENT_GENERATORS dict. Inherits all existing wiring: auth,
  measurement-completed precondition, idempotency, artifact storage,
  MIME types, JSON response envelope.

- apps/backend/tests/test_spectral_viz.py — 3 new tests:
  - produces_single_reassigned_png: shape + non-trivial size
  - output_is_valid_png: magic bytes
  - handles_silent_input_without_raising: regression gate against
    NaN propagation on near-silent fixtures

No new dependencies — librosa is already in requirements.txt
(librosa==0.11.0 exposes reassigned_spectrogram).

Usage:

  POST /api/analysis-runs/{run_id}/spectral-enhancements/reassigned

Returns the new 'spectrogram_reassigned' artifact ID alongside the
existing PNG-style spectrograms. Idempotent: re-posting returns the
existing artifact without regenerating.

Out of scope (deferred):

- Frontend UI toggle to display the reassigned PNG. The backend now
  produces it on demand; the UI wiring is a separate follow-up.
- Raw (time, freq, mag) JSON artifact for client-side custom
  visualization. Could be added as a sibling 'reassigned_data'
  artifact if a future consumer needs it.

Closes the last review-adopt item. Full session summary in the
merge commits for #33 through #42.

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

Adds an opt-in reassigned spectrogram artifact via the existing enhancement route. Purely additive — one new function in spectral_viz.py, one new entry in _ENHANCEMENT_GENERATORS, three tests. Phase 1 output is untouched. Tests could not be run (no venv in reviewer workspace), but code-review of ReassignedSpectrogramTests shows meaningful coverage of the three cases that matter: normal output, valid PNG header, and NaN propagation on near-silent input. Architecture is clean.

Findings

Should fix — scatter point ceiling is missing (spectral_viz.py, the ax.scatter(...) call)

The rest of the file guards time-series output with DEFAULT_MAX_POINTS = 500 and _downsample_time_series. This function has no equivalent cap. For a 3-minute track at default n_fft=2048 / hop_length=1024: 1025 bins × ~7,750 frames ≈ 7.9M points before the floor mask; expect 4–5M survivors. For a 10-minute track it's north of 15M. matplotlib.Axes.scatter at that scale takes tens of seconds for the render step alone — the librosa compute cost is not the bottleneck here.

Fix is small: add max_scatter_points: int = 300_000 parameter, then after building the mask:

idx = np.arange(mask.sum())
if idx.size > max_scatter_points:
    idx = np.random.default_rng(0).choice(idx.size, max_scatter_points, replace=False)
flat_times_vis = flat_times[mask][idx]
flat_freqs_vis = flat_freqs[mask][idx]
flat_mags_vis  = flat_mags_db[mask][idx]

300K points renders in under a second and still looks sharp at FIG_DPI=100. Could be a follow-up PR if this is close to merge.

Worth noting — CLAUDE.md says "inferno colormap" but the codebase already uses magma everywhere

Not introduced by this PR — every existing generator uses cmap="magma" and this PR follows that. The CLAUDE.md rule is stale and will confuse future contributors. Worth a one-line correction in any nearby docs pass.

Test results

Could not execute (no venv). Code review of ReassignedSpectrogramTests: all three tests are meaningful. The silent-fixture test correctly uses imperceptible dither (1e-6 amplitude noise) rather than true zeros, which exercises the fill_nan path and np.isfinite mask without making librosa.amplitude_to_db(ref=np.max) undefined.

Phase boundary check

Clean. No Phase 1 fields read, mutated, or re-derived. The new artifact is produced from the raw audio path exactly the same way every other enhancement generator works.


Generated by Claude Code

PR #43 review caught a real perf issue: the original
generate_reassigned_spectrogram fed every STFT cell into
matplotlib.scatter unchecked. At default params (n_fft=2048,
hop=1024, sr=44100), that's:

  ~8M raw points on a 3-minute track
  ~15M raw points on a 10-minute track

Both well above the floor-mask survival count and well above the
threshold where matplotlib's render step starts taking tens of
seconds. The librosa compute cost isn't the bottleneck — the
scatter render is.

Fix:

- New REASSIGNED_MAX_SCATTER_POINTS = 300_000 constant. At
  FIG_DPI=100 / ~1200x400 px figure, 300K is the visual sweet
  spot — sharper-looking than vanilla STFT, renders in under a
  second.

- After the floor mask, the post-mask arrays (times_visible,
  freqs_visible, mags_visible) get subsampled via a seeded RNG
  (np.random.default_rng(0)). Seeded so re-running on the same
  audio produces a byte-identical PNG — matters for idempotency
  tests and content-hash artifact caching.

- New test test_scatter_point_cap_is_enforced_for_long_inputs:
  generates a 60 s multi-component fixture (~1M+ post-mask points,
  comfortably above the 300K cap), patches matplotlib.scatter to
  capture call args without rendering, and asserts the actual
  point count passed to scatter is <= REASSIGNED_MAX_SCATTER_POINTS.

The 'inferno colormap' worth-noting from the review was checked —
no 'inferno' reference exists in CLAUDE.md or anywhere in the
codebase. The reviewer's worth-noting was itself stale; nothing
to fix there.

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg
@slittycode
slittycode merged commit 3ee2582 into main May 14, 2026
2 checks passed
@slittycode
slittycode deleted the claude/spectral-reassignment-5XA5r branch May 14, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants