WS3a: asa-dsp ↔ Essentia loudness parity harness (checkpoint) - #129
Conversation
Before the loudness-spectro-wasm core (asa-dsp, openmeters-derived BS.1770) can back any ASA product path (WS3b backend, WS3c browser readout), it has to agree with Essentia — ASA's authoritative Phase 1 loudness source, since Phase 1 measurements are ground truth (PURPOSE.md invariant #1). scripts/essentia_parity.py runs a deterministic synthetic corpus (tones, sweep, white/pink noise, decorrelated stereo) through BOTH paths at each file's native rate (no resampling): - asa-dsp via the native measure-cli binary (source-identical to the WASM core, so WS3a needs no wasm toolchain), and - Essentia via AudioLoader -> LoudnessEBUR128, mirroring analyze_core.analyze_loudness, plus a dBTP true peak from TruePeakDetector. Primary gate: integrated LUFS within +/-0.1 LU on every file. Result (docs/essentia-parity-report.md): 7/8 signals agree within +/-0.02 LU (most within +/-0.002) — essentially perfect BS.1770 parity on tones and noise. The sole breach is the 20 Hz->20 kHz log sweep at -0.102 LU (0.002 over the gate), a pathological non-program signal. Verdict is reported as the harness computed it (FAIL by that single borderline case); the realistic-signal agreement is the substantive finding. Surfaced at the WS3a hard checkpoint. README documents the harness as Validation helper #4. https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q
slittycode
left a comment
There was a problem hiding this comment.
Verdict: COMMENT
Summary
Harness + docs only — packages/loudness-spectro-wasm/scripts/essentia_parity.py, the committed checkpoint report, and a README addition. No product code touched; asa-dsp remains unwired. Phase 1 contract unchanged, no phase boundary violations. One genuine code oddity in run() and one secondary-metric gap worth flagging before WS3c. CI was still in progress at review time.
Findings
Worth considering — run() calls build_report() and discards the result (scripts/essentia_parity.py:289–290)
def run(...):
...
_text, passed = build_report(rows, tolerance) # result thrown away
return rows, passedThen in main() (~line 310):
rows, passed = run(corpus, args.tolerance)
...
text, passed = build_report(rows, args.tolerance) # called again, shadows passedbuild_report() runs twice and passed from run() is immediately overwritten. Not a bug — the function is pure so both calls agree — but it's confusing and the _text discard is a hint that run() shouldn't own this call at all. Simplest fix: drop the build_report() call from run() and have it return only rows.
Worth considering — white_noise true-peak gap (+0.59 dBTP)
From the committed report secondary table: Essentia −6.61 dBTP vs asa-dsp −6.02 dBTP on white_noise. The caveats section correctly flags true peak as "a sanity signal, not a gate," but if WS3c lands a browser true-peak readout the 0.59 dBTP gap on broadband content needs a gate or at minimum an explicit "known divergence, do not expose" note before that work proceeds. The gap is benign for WS3a but should be addressed in the WS3b/WS3c scoping.
Test results
CI was in progress (Backend, Frontend, Loudness WASM all in_progress) at review time. All changed files are under packages/loudness-spectro-wasm/, which is standalone and unwired. No unit tests for the harness — consistent with the established pattern for every evaluate_*.py script under apps/backend/scripts/; the harness is itself the measurement gate.
Phase boundary check
Clean. No Phase 1 output types modified. No apps/backend/ or apps/ui/ files touched. The reported FAIL verdict is an honest, measurement-cited finding, not an override of Phase 1 data.
Generated by Claude Code
Addresses the PR #129 review: - run() no longer calls build_report() and discards the text; it returns rows only. build_report() now runs exactly once, in main(). Pure refactor — the harness reproduces identical numbers. - Report caveats now explicitly flag the broadband true-peak divergence (white_noise ~0.6 dBTP gap): WS3c must not expose a browser true-peak readout without its own gate. Integrated loudness is unaffected. https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q
…default-off) (#132) * WS3b: optional asa-dsp loudness backend behind ASA_LOUDNESS_BACKEND Wires the loudness-spectro-wasm core (asa-dsp) into the backend as a default-off alternative loudness source for the LUFS scalars, selected by ASA_LOUDNESS_BACKEND=essentia|wasm (default essentia). Architecture (deviates from the plan's pyo3 sketch, deliberately): - Invokes the already-built native measure-cli binary on a temp WAV rather than binding via pyo3. This sidesteps the workspace's `panic = "abort"` profile (which blocks pyo3), needs no separate Cargo workspace, and — being pure Python on this side — cannot red the loudness-wasm CI job. The backend has already decoded the audio, so the temp WAV makes it format-agnostic. - Scope is the four LUFS scalars (integrated/range/momentary-max/short-term-max) — the EBU loudness the WS3a parity harness validated at ±0.1 LU. truePeak stays Essentia (asa-dsp's true peak diverges on broadband per #129) and lufsCurve stays Essentia (the CLI emits scalars, not per-frame curves). So this is a scalar override, not a full engine replacement. - Default stays Essentia (PURPOSE.md invariant #1); any failure (missing binary, non-zero exit, unparseable/null output) degrades gracefully back to Essentia. apps/backend/loudness_backend.py + a call after analyze_loudness in analyze.py. Tests mock the subprocess (the binary isn't built in the Python CI job): backend selection, scalar override + curve passthrough, and every fall-back path. The default path is unchanged — test_analyze + the golden snapshot still pass. https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q * WS3b review: fix repo-root path off-by-one; don't clobber Essentia with null Addresses the PR #132 REQUEST CHANGES: - BLOCKING: _REPO_ROOT used parents[1] (= apps/), so the default measure-cli path was apps/packages/loudness-spectro-wasm/... which doesn't exist — the feature silently degraded to Essentia unless ASA_MEASURE_CLI was set. Fixed to parents[2] (repo root). Regression test asserts the default path resolves to the real <repo>/packages/loudness-spectro-wasm dir. - Should-fix: only override a LUFS scalar when the CLI value is non-null, so a null lra (undefined for short content where integrated is still valid) can't clobber Essentia's reading. Regression test covers it. 12/12 loudness_backend tests pass; the warn path now correctly points at <repo>/packages/... (no apps/ prefix). https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q --------- Co-authored-by: Claude <noreply@anthropic.com>
* WS3c: in-browser loudness parity readout (flag-gated, WAV-only) Adds an additive browser-side loudness readout that measures the uploaded WAV with the asa-dsp (WASM core) and compares the LUFS scalars against the authoritative Phase 1 Essentia values. Phase 1 stays the source of truth (PURPOSE.md invariant #1); this is a diagnostic, default-off behind VITE_ENABLE_BROWSER_LOUDNESS. Services (pure, fully unit-tested): - wavDecoder.ts: decodes uncompressed WAV (PCM 16/24/32 + float32, incl. WAVE_FORMAT_EXTENSIBLE) to interleaved f32 at the file's NATIVE rate — no AudioContext.decodeAudioData resampling, which would break ±0.1 LU EBU conformance. WAV only; FLAC/MP3 report "WAV only" rather than guessing. - parity.ts: computeLoudnessParity → per-field deltas vs Phase 1, gated at ±0.1 LU. True peak is deliberately excluded (asa-dsp diverges on broadband per #129). - loader.ts: guarded dynamic import of the WASM module from a runtime URL (VITE_BROWSER_LOUDNESS_WASM_URL) with @vite-ignore so the build never tries to resolve the (unbuilt, gitignored) package; returns null → "unavailable" on any failure. The result adapter (NaN→null, free()) is unit-tested. UI: BrowserLoudnessPanel (flag-gated mount in AnalysisResults) reuses the retained App `audioFile`; graceful states for non-WAV, wasm-unavailable, error. Verification limits (honest): the WAV decoder + parity math + result adapter are unit-tested; the actual WASM run and browser behavior are NOT verified here (no browser, and building web-wasm needs wasm-bindgen-cli). Activation requires building packages/loudness-spectro-wasm and serving pkg/ — documented in the unavailable state. 729 unit tests pass; build clean (no static wasm import). https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q * WS3c review: fix EXTENSIBLE guard off-by-2, drop redundant import.meta cast - wavDecoder: WAVE_FORMAT_EXTENSIBLE guard needs chunkSize >= 26 (not 24) to safely read the 2-byte SubFormat tag at body+24; add a test that decodes a real 40-byte EXTENSIBLE fmt chunk (the branch was previously untested). - loader: VITE_BROWSER_LOUDNESS_WASM_URL is declared in vite-env.d.ts, so the import.meta cast is unnecessary — read import.meta.env directly. https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q --------- Co-authored-by: Claude <noreply@anthropic.com>
What this is
The WS3a parity checkpoint for wiring
packages/loudness-spectro-wasm(asa-dsp, openmeters-derived BS.1770) into the product. Before asa-dsp can back any ASA loudness path (WS3b backend, WS3c browser readout), it must agree with Essentia — ASA's authoritative Phase 1 loudness source, since Phase 1 measurements are ground truth (PURPOSE.md invariant #1).scripts/essentia_parity.pyruns a deterministic synthetic corpus (tones, 20 Hz→20 kHz sweep, white/pink noise, decorrelated stereo) through both paths at each file's native rate (no resampling — that alone breaks ±0.1 LU EBU conformance):measure-clibinary (source-identical to the WASM core, so WS3a needs no wasm toolchain — per the plan).AudioLoader → LoudnessEBUR128, mirroringapps/backend/analyze_core.analyze_loudness, plus a dBTP true peak fromTruePeakDetector.Primary gate: integrated LUFS within ±0.1 LU on every file.
Result — borderline (see
docs/essentia-parity-report.md)log_sweep_20_20kat −0.102 LU — 0.002 over the gateThe one miss is a full-range log sweep — a pathological, non-program stress signal sweeping into the near-Nyquist region, not music. The verdict in the committed report is reported exactly as the harness computed it (FAIL by that single borderline case); the substantive finding is that the cores agree on every realistic signal.
Decision pending (the hard checkpoint)
Per the plan, WS3a is a STOP point. The question for the owner: treat the sweep breach as (a) a pathological-edge artifact and clear WS3b/WS3c to proceed behind a default-off flag (loudness default never flips to asa-dsp until real-program parity is also proven), or (b) a strict hard stop. Awaiting direction before stacking WS3b/WS3c.
Notes
main.https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q
Generated by Claude Code