Skip to content

WS3a: asa-dsp ↔ Essentia loudness parity harness (checkpoint) - #129

Merged
slittycode merged 2 commits into
mainfrom
claude/wasm-loudness-parity
Jun 2, 2026
Merged

WS3a: asa-dsp ↔ Essentia loudness parity harness (checkpoint)#129
slittycode merged 2 commits into
mainfrom
claude/wasm-loudness-parity

Conversation

@slittycode

Copy link
Copy Markdown
Owner

What this is

The WS3a parity checkpoint for wiring packages/loudness-spectro-wasm (asa-dsp, openmeters-derived BS.1770) into the product. Before asa-dsp can back any ASA loudness path (WS3b backend, WS3c browser readout), it must agree with Essentia — ASA's authoritative Phase 1 loudness source, since Phase 1 measurements are ground truth (PURPOSE.md invariant #1).

scripts/essentia_parity.py runs a deterministic synthetic corpus (tones, 20 Hz→20 kHz sweep, white/pink noise, decorrelated stereo) through both paths at each file's native rate (no resampling — that alone breaks ±0.1 LU EBU conformance):

  • asa-dsp via the native measure-cli binary (source-identical to the WASM core, so WS3a needs no wasm toolchain — per the plan).
  • Essentia via AudioLoader → LoudnessEBUR128, mirroring apps/backend/analyze_core.analyze_loudness, plus a dBTP true peak from TruePeakDetector.

Primary gate: integrated LUFS within ±0.1 LU on every file.

Result — borderline (see docs/essentia-parity-report.md)

Realistic signals (tones, noise, stereo) 7/8 within ±0.02 LU (most ±0.002) — essentially perfect BS.1770 parity
Sole breach log_sweep_20_20k at −0.102 LU — 0.002 over the gate
Worst delta 0.102 LU

The one miss is a full-range log sweep — a pathological, non-program stress signal sweeping into the near-Nyquist region, not music. The verdict in the committed report is reported exactly as the harness computed it (FAIL by that single borderline case); the substantive finding is that the cores agree on every realistic signal.

Decision pending (the hard checkpoint)

Per the plan, WS3a is a STOP point. The question for the owner: treat the sweep breach as (a) a pathological-edge artifact and clear WS3b/WS3c to proceed behind a default-off flag (loudness default never flips to asa-dsp until real-program parity is also proven), or (b) a strict hard stop. Awaiting direction before stacking WS3b/WS3c.

Notes

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q


Generated by Claude Code

Before the loudness-spectro-wasm core (asa-dsp, openmeters-derived BS.1770)
can back any ASA product path (WS3b backend, WS3c browser readout), it has
to agree with Essentia — ASA's authoritative Phase 1 loudness source, since
Phase 1 measurements are ground truth (PURPOSE.md invariant #1).

scripts/essentia_parity.py runs a deterministic synthetic corpus (tones,
sweep, white/pink noise, decorrelated stereo) through BOTH paths at each
file's native rate (no resampling):
- asa-dsp via the native measure-cli binary (source-identical to the WASM
  core, so WS3a needs no wasm toolchain), and
- Essentia via AudioLoader -> LoudnessEBUR128, mirroring
  analyze_core.analyze_loudness, plus a dBTP true peak from TruePeakDetector.

Primary gate: integrated LUFS within +/-0.1 LU on every file.

Result (docs/essentia-parity-report.md): 7/8 signals agree within +/-0.02 LU
(most within +/-0.002) — essentially perfect BS.1770 parity on tones and
noise. The sole breach is the 20 Hz->20 kHz log sweep at -0.102 LU (0.002
over the gate), a pathological non-program signal. Verdict is reported as the
harness computed it (FAIL by that single borderline case); the realistic-signal
agreement is the substantive finding. Surfaced at the WS3a hard checkpoint.

README documents the harness as Validation helper #4.

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: COMMENT

Summary

Harness + docs only — packages/loudness-spectro-wasm/scripts/essentia_parity.py, the committed checkpoint report, and a README addition. No product code touched; asa-dsp remains unwired. Phase 1 contract unchanged, no phase boundary violations. One genuine code oddity in run() and one secondary-metric gap worth flagging before WS3c. CI was still in progress at review time.

Findings

Worth considering — run() calls build_report() and discards the result (scripts/essentia_parity.py:289–290)

def run(...):
    ...
    _text, passed = build_report(rows, tolerance)  # result thrown away
    return rows, passed

Then in main() (~line 310):

rows, passed = run(corpus, args.tolerance)
...
text, passed = build_report(rows, args.tolerance)  # called again, shadows passed

build_report() runs twice and passed from run() is immediately overwritten. Not a bug — the function is pure so both calls agree — but it's confusing and the _text discard is a hint that run() shouldn't own this call at all. Simplest fix: drop the build_report() call from run() and have it return only rows.

Worth considering — white_noise true-peak gap (+0.59 dBTP)

From the committed report secondary table: Essentia −6.61 dBTP vs asa-dsp −6.02 dBTP on white_noise. The caveats section correctly flags true peak as "a sanity signal, not a gate," but if WS3c lands a browser true-peak readout the 0.59 dBTP gap on broadband content needs a gate or at minimum an explicit "known divergence, do not expose" note before that work proceeds. The gap is benign for WS3a but should be addressed in the WS3b/WS3c scoping.

Test results

CI was in progress (Backend, Frontend, Loudness WASM all in_progress) at review time. All changed files are under packages/loudness-spectro-wasm/, which is standalone and unwired. No unit tests for the harness — consistent with the established pattern for every evaluate_*.py script under apps/backend/scripts/; the harness is itself the measurement gate.

Phase boundary check

Clean. No Phase 1 output types modified. No apps/backend/ or apps/ui/ files touched. The reported FAIL verdict is an honest, measurement-cited finding, not an override of Phase 1 data.


Generated by Claude Code

Addresses the PR #129 review:
- run() no longer calls build_report() and discards the text; it returns
  rows only. build_report() now runs exactly once, in main(). Pure refactor —
  the harness reproduces identical numbers.
- Report caveats now explicitly flag the broadband true-peak divergence
  (white_noise ~0.6 dBTP gap): WS3c must not expose a browser true-peak
  readout without its own gate. Integrated loudness is unaffected.

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q
@slittycode
slittycode merged commit 97f36ef into main Jun 2, 2026
3 checks passed
@slittycode
slittycode deleted the claude/wasm-loudness-parity branch June 2, 2026 03:44
slittycode added a commit that referenced this pull request Jun 2, 2026
…default-off) (#132)

* WS3b: optional asa-dsp loudness backend behind ASA_LOUDNESS_BACKEND

Wires the loudness-spectro-wasm core (asa-dsp) into the backend as a default-off
alternative loudness source for the LUFS scalars, selected by
ASA_LOUDNESS_BACKEND=essentia|wasm (default essentia).

Architecture (deviates from the plan's pyo3 sketch, deliberately):
- Invokes the already-built native measure-cli binary on a temp WAV rather than
  binding via pyo3. This sidesteps the workspace's `panic = "abort"` profile
  (which blocks pyo3), needs no separate Cargo workspace, and — being pure
  Python on this side — cannot red the loudness-wasm CI job. The backend has
  already decoded the audio, so the temp WAV makes it format-agnostic.
- Scope is the four LUFS scalars (integrated/range/momentary-max/short-term-max)
  — the EBU loudness the WS3a parity harness validated at ±0.1 LU. truePeak stays
  Essentia (asa-dsp's true peak diverges on broadband per #129) and lufsCurve
  stays Essentia (the CLI emits scalars, not per-frame curves). So this is a
  scalar override, not a full engine replacement.
- Default stays Essentia (PURPOSE.md invariant #1); any failure (missing binary,
  non-zero exit, unparseable/null output) degrades gracefully back to Essentia.

apps/backend/loudness_backend.py + a call after analyze_loudness in analyze.py.
Tests mock the subprocess (the binary isn't built in the Python CI job): backend
selection, scalar override + curve passthrough, and every fall-back path. The
default path is unchanged — test_analyze + the golden snapshot still pass.

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q

* WS3b review: fix repo-root path off-by-one; don't clobber Essentia with null

Addresses the PR #132 REQUEST CHANGES:
- BLOCKING: _REPO_ROOT used parents[1] (= apps/), so the default measure-cli
  path was apps/packages/loudness-spectro-wasm/... which doesn't exist — the
  feature silently degraded to Essentia unless ASA_MEASURE_CLI was set. Fixed
  to parents[2] (repo root). Regression test asserts the default path resolves
  to the real <repo>/packages/loudness-spectro-wasm dir.
- Should-fix: only override a LUFS scalar when the CLI value is non-null, so a
  null lra (undefined for short content where integrated is still valid) can't
  clobber Essentia's reading. Regression test covers it.

12/12 loudness_backend tests pass; the warn path now correctly points at
<repo>/packages/... (no apps/ prefix).

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q

---------

Co-authored-by: Claude <noreply@anthropic.com>
slittycode added a commit that referenced this pull request Jun 2, 2026
* WS3c: in-browser loudness parity readout (flag-gated, WAV-only)

Adds an additive browser-side loudness readout that measures the uploaded WAV
with the asa-dsp (WASM core) and compares the LUFS scalars against the
authoritative Phase 1 Essentia values. Phase 1 stays the source of truth
(PURPOSE.md invariant #1); this is a diagnostic, default-off behind
VITE_ENABLE_BROWSER_LOUDNESS.

Services (pure, fully unit-tested):
- wavDecoder.ts: decodes uncompressed WAV (PCM 16/24/32 + float32, incl.
  WAVE_FORMAT_EXTENSIBLE) to interleaved f32 at the file's NATIVE rate — no
  AudioContext.decodeAudioData resampling, which would break ±0.1 LU EBU
  conformance. WAV only; FLAC/MP3 report "WAV only" rather than guessing.
- parity.ts: computeLoudnessParity → per-field deltas vs Phase 1, gated at
  ±0.1 LU. True peak is deliberately excluded (asa-dsp diverges on broadband
  per #129).
- loader.ts: guarded dynamic import of the WASM module from a runtime URL
  (VITE_BROWSER_LOUDNESS_WASM_URL) with @vite-ignore so the build never tries
  to resolve the (unbuilt, gitignored) package; returns null → "unavailable"
  on any failure. The result adapter (NaN→null, free()) is unit-tested.

UI: BrowserLoudnessPanel (flag-gated mount in AnalysisResults) reuses the
retained App `audioFile`; graceful states for non-WAV, wasm-unavailable, error.

Verification limits (honest): the WAV decoder + parity math + result adapter are
unit-tested; the actual WASM run and browser behavior are NOT verified here (no
browser, and building web-wasm needs wasm-bindgen-cli). Activation requires
building packages/loudness-spectro-wasm and serving pkg/ — documented in the
unavailable state. 729 unit tests pass; build clean (no static wasm import).

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q

* WS3c review: fix EXTENSIBLE guard off-by-2, drop redundant import.meta cast

- wavDecoder: WAVE_FORMAT_EXTENSIBLE guard needs chunkSize >= 26 (not 24) to
  safely read the 2-byte SubFormat tag at body+24; add a test that decodes a
  real 40-byte EXTENSIBLE fmt chunk (the branch was previously untested).
- loader: VITE_BROWSER_LOUDNESS_WASM_URL is declared in vite-env.d.ts, so the
  import.meta cast is unnecessary — read import.meta.env directly.

https://claude.ai/code/session_01Wq1vE1D7o3W9xZKSHXz43Q

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants