diff --git a/apps/backend/AGENTS.md b/apps/backend/AGENTS.md index 871fcbd9..fa717fc9 100644 --- a/apps/backend/AGENTS.md +++ b/apps/backend/AGENTS.md @@ -7,7 +7,7 @@ - The repo is a local Python audio-analysis service with two entry points: - `analyze.py`: raw CLI analyzer - `server.py`: FastAPI wrapper around the CLI -- There are no repo-local Cursor rules, `.cursorrules`, or Copilot instruction files in this repo as of 2026-03-10. +- There are no repo-local Cursor rules, `.cursorrules`, or Copilot instruction files in this repo as of 2026-05-15. ## Working Style For Agents @@ -110,7 +110,9 @@ python3.11 -m venv venv - `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`: Feature modules (BPM/key/LUFS/stereo, rhythm/melody/groove, segments, structure, transcription, the `--fast` pipeline). Split from the original monolith in commit `5c40dd44` — keep the split when adding features. - `server.py` + `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`: FastAPI app + route modules. Multipart/URL upload handling, subprocess execution, envelope normalization, and the on-demand Phase 3 audition-sample routes. - `analysis_runtime.py`: SQLite-backed run state, stage queue, artifact metadata. +- `stage_status.py`: Stage-status helpers, including the `publicStatus` projection used by the snapshot API. - `worker.py`, `runtime_profile.py`, `auth_context.py`, `artifact_storage.py`: Hosted-mode foundation. Local mode shouldn't branch through these unless it has to. +- `url_ingest.py`: SSRF-guarded URL ingestion for `POST /api/analysis-runs`. - `upload_limits.py`: Canonical 100 MiB raw-audio / 101 MiB request-envelope limits. Operator contract is generated, not hand-edited. - `url_ingest.py`: SSRF-guarded URL-mode ingestion for `POST /api/analysis-runs`. - `csv_export.py`: CSV exporters for Phase 1 time-series fields; backs `GET /api/analysis-runs/{run_id}/export/csv/{field_path}`. @@ -120,11 +122,23 @@ python3.11 -m venv venv - `spectral_viz.py`: Librosa spectrogram and spectral time-series artifacts. Non-critical — failures don't break a run. - `phase1_evaluation.py` + `phase1_report_html.py`: Offline Phase 1 evaluation harness and HTML render. Not on the product path; driven by `scripts/evaluate_phase1.py`. - `polyphonic_evaluation.py` + `scripts/evaluate_polyphonic.py`: **Research-only.** Offline polyphonic-transcription evaluation harness, not part of the shipped product path. +- `phase1_evaluation.py`, `phase1_report_html.py`: Offline evaluation harness for Phase 1 measurement quality. Research-only. +- `utils/cleanup.py`: Periodic artifact cleanup helpers used by the server background-task loop. - `tests/test_server.py`: OpenAPI and envelope contract tests. - `tests/test_analyze.py`: generated WAV fixture, `EXPECTED_TOP_LEVEL_KEYS` snapshot, raw payload assertions. +- `tests/test_csv_export.py`, `tests/test_sample_*.py`, `tests/test_server_samples.py`: Coverage for CSV export and Phase 3 audition samples. - `ARCHITECTURE.md`: backend responsibilities and request flow. - `JSON_SCHEMA.md`: raw CLI schema plus HTTP mapping notes. +## Operator and Research Scripts + +Under `apps/backend/scripts/` (not on the product path): + +- `bootstrap.sh`: create/recreate the venv with the pinned Python 3.11 dependencies. +- `dev.sh`: backend-only dev launcher used by the monorepo `./scripts/dev.sh`. +- `render_upload_limit_contract.py`: re-renders the operator-facing upload-limit contract whenever `upload_limits.py` numbers change. +- `evaluate_phase1.py`, `evaluate_structure_sweep.py`, `evaluate_polyphonic.py`, `genre_check.py`, `audit_pass1.py`, `replay_catalog_validation.py`: research and audit harnesses for measurement quality and prompt-output review. Outputs land under `.runtime/reports/` and are intentionally not wired into the live API. + ## Code Style - Follow the surrounding file style instead of introducing a new formatter profile. diff --git a/apps/backend/ARCHITECTURE.md b/apps/backend/ARCHITECTURE.md index b0028d87..8351a5b6 100644 --- a/apps/backend/ARCHITECTURE.md +++ b/apps/backend/ARCHITECTURE.md @@ -7,10 +7,12 @@ | `analyze.py` | Raw CLI analyzer entry point. Loads audio, coordinates the `analyze_*.py` feature modules (see [Analyzer Submodules](#analyzer-submodules) below), optionally separates stems and transcribes notes through torchcrepe, then prints JSON to `stdout`. | | `server.py` + `server_phase1.py` / `server_phase2.py` / `server_upload.py` / `server_samples.py` | FastAPI app and router composition. Accepts uploads, computes estimates, manages the canonical staged run API, normalizes measurement results, serves artifact access, and exposes the on-demand Phase 3 audition-sample routes. | | `analysis_runtime.py` | Run-state persistence and staged-analysis orchestration. Owns run snapshots, stage status, artifact metadata, and ownership checks. | +| `stage_status.py` | Helpers for normalizing stage execution status across the runtime and HTTP surfaces (e.g. `publicStatus` projection). | | `artifact_storage.py` | Artifact storage boundary. The current implementation uses the local filesystem, but the runtime now talks to a storage service interface instead of assuming every artifact is a local disk path forever. | | `runtime_profile.py` | Runtime/profile switchboard for `local` vs `hosted` behavior and `all` vs `api` vs `worker` process roles. | | `auth_context.py` | Hosted-mode user-context resolution. Establishes the current run owner in the canonical API path. | | `worker.py` | Dedicated worker-process entry point for hosted-style background stage execution. | +| `url_ingest.py` | URL-mode ingestion for `POST /api/analysis-runs`. SSRF-guarded against private/loopback/link-local addresses; streams to enforce the shared 100 MiB cap. | | `upload_limits.py` | Canonical raw-audio (100 MiB) and request-envelope (101 MiB) limits, plus the protected-route list. Operator contract is generated, not hand-edited — see `scripts/render_upload_limit_contract.py`. | | `spectral_viz.py` | Librosa-based spectrogram generation and spectral time-series extraction. Produces mel/chroma PNG spectrograms and per-frame spectral evolution JSON. Called after successful measurement; failures are non-critical. | | `url_ingest.py` | SSRF-guarded URL-mode ingestion for `POST /api/analysis-runs`. Fetches a public `http`/`https` audio file and feeds the bytes through the same downstream pipeline as a multipart upload. | @@ -20,9 +22,13 @@ | `dsp_bandbank.py` + `dsp_utils.py` | Shared DSP primitives: `BatchedBandpass` (4th-order Butterworth bandpass bank with zero-phase `filtfilt`) and cross-module utility functions. | | `phase1_evaluation.py` + `phase1_report_html.py` | Offline Phase 1 evaluation harness — deterministic-metric and detector-stability reporting, with a standalone HTML render. Not on the product path; driven by `scripts/evaluate_phase1.py`. | | `polyphonic_evaluation.py` + `scripts/evaluate_polyphonic.py` | Research-only offline polyphonic-transcription evaluation harness. Not on the product path. | +| `phase1_evaluation.py` + `phase1_report_html.py` | Offline evaluation harness for Phase 1 measurement quality. Research-only. | +| `utils/cleanup.py` | Periodic artifact cleanup helpers used by the server background-task loop. | | `tests/test_server.py` | Contract tests for estimate, timeout, and success envelopes. | | `tests/test_analyze.py` | Structural snapshot tests for the raw analyzer JSON output. Owns `EXPECTED_TOP_LEVEL_KEYS` — update it whenever you add a root field. | | `tests/test_spectral_viz.py` | Unit tests for spectrogram generation, time-series computation, and artifact orchestration. | +| `tests/test_csv_export.py` | Round-trip CSV-exporter tests against the registry in `csv_export.py`. | +| `tests/test_sample_*.py` + `tests/test_server_samples.py` | Phase 3 audition-sample synthesis and HTTP contract tests. | ### Analyzer Submodules @@ -76,6 +82,7 @@ Custom routes: - `POST /api/analysis-runs/estimate` - `POST /api/analysis-runs` — multipart upload OR URL ingestion. Provide *exactly one* of `track` (multipart `UploadFile`) or `url` (form field with a public `http`/`https` URL). URL mode is SSRF-guarded against private/loopback/link-local addresses and enforces the same 100 MiB cap via streaming. See [`url_ingest.py`](url_ingest.py). - `GET /api/analysis-runs/{run_id}` +- `POST /api/analysis-runs/{run_id}/interrupt` — terminate any active child processes for the run and mark stages interrupted. - `DELETE /api/analysis-runs/{run_id}` — owner can delete their own run; operators with `SONIC_ANALYZER_ADMIN_KEY` set can supply `X-Admin-Key` to delete any run. Admin path is closed when the env var is unset. - `GET /api/analysis-runs/{run_id}/artifacts` and `…/artifacts/{artifact_id}` - `GET /api/analysis-runs/{run_id}/source-audio` — re-serves the original ingested audio for the run. Owner-only (no admin bypass). Saves a round-trip vs looking up the source artifact id first. diff --git a/apps/backend/README.md b/apps/backend/README.md index 016bb6eb..10a7dfdd 100644 --- a/apps/backend/README.md +++ b/apps/backend/README.md @@ -30,6 +30,7 @@ Canonical live-analysis routes: Legacy compatibility routes: - `POST /api/analyze` (legacy compatibility wrapper) +- `POST /api/analyze/estimate` (legacy compatibility wrapper) - `POST /api/phase2` (legacy compatibility wrapper) FastAPI also serves the usual generated endpoints at `/openapi.json`, `/docs`, and `/redoc`. diff --git a/apps/backend/csv_export.py b/apps/backend/csv_export.py index 84eca5d2..5841231e 100644 --- a/apps/backend/csv_export.py +++ b/apps/backend/csv_export.py @@ -1,10 +1,11 @@ """CSV exporters for Phase 1 time-series fields. Patterned on the ``(time, duration, value)`` tabular shape used by Partiels -and similar Vamp-output tools (see ``docs/external-repo-review-2026-05-13.md`` -Track 2 for the rationale and the rejected-alternatives discussion). Each -exporter maps one dotted JSON path inside the Phase 1 measurement payload to -a small CSV with documented columns. +and similar Vamp-output tools (see +``docs/history/external-repo-review-2026-05-13.md`` Track 2 for the rationale +and the rejected-alternatives discussion). Each exporter maps one dotted +JSON path inside the Phase 1 measurement payload to a small CSV with +documented columns. The exporters are kept here, outside ``server.py``, so the route handler can stay a thin lookup-and-serve. diff --git a/apps/backend/tests/test_loudness_r128.py b/apps/backend/tests/test_loudness_r128.py index a6c025a2..d4f9b4cb 100644 --- a/apps/backend/tests/test_loudness_r128.py +++ b/apps/backend/tests/test_loudness_r128.py @@ -1,6 +1,6 @@ """EBU R128 verification + regression gate for ASA's loudness path. -Track 1 of the external-repo incorporation plan (docs/external-repo-review-2026-05-13.md) +Track 1 of the external-repo incorporation plan (docs/history/external-repo-review-2026-05-13.md) asked: is ASA's integrated-LUFS implementation correct, or does it lag the BS.1770-5 revision? This module answers that with the EBU Tech 3341 compliance signals — the canonical loudness conformance test set — and now also gates the sample-rate @@ -160,7 +160,7 @@ class TestLoudnessR128AtNon441kHz(unittest.TestCase): bug* (sample rate not threaded), not an algorithm bug. Originally the open finding from the verification spike documented - in docs/external-repo-review-2026-05-13.md (Track 1 follow-up). The + in docs/history/external-repo-review-2026-05-13.md (Track 1 follow-up). The follow-up fix lives in this branch's companion edits to ``analyze_core.analyze_loudness`` (now takes ``sample_rate``) and its call sites in ``analyze.py``; the regression test below diff --git a/apps/ui/AGENTS.md b/apps/ui/AGENTS.md index 1dbce3c9..094a2386 100644 --- a/apps/ui/AGENTS.md +++ b/apps/ui/AGENTS.md @@ -4,8 +4,8 @@ - This file applies to `apps/ui` inside the `asa` monorepo. - Stack: React 19, TypeScript, Vite 6, Tailwind CSS v4, Vitest, Playwright. -- The app talks to the local `sonic-analyzer` backend. -- No repo-local `.cursorrules`, `.cursor/rules/`, or `.github/copilot-instructions.md` exist here as of 2026-03-10. +- The app talks to the local `sonic-analyzer` backend. Gemini is backend-mediated; the UI does **not** import an AI SDK. +- No repo-local `.cursorrules`, `.cursor/rules/`, or `.github/copilot-instructions.md` exist here as of 2026-05-15. ## Working Style For Agents @@ -95,6 +95,7 @@ RUN_GEMINI_LIVE_SMOKE=true VITE_ENABLE_PHASE2_GEMINI=true GEMINI_API_KEY=your_ke - `src/index.css`: Tailwind theme tokens and visual language. - `tests/services/*`: unit and service tests. - `tests/smoke/*`: smoke and live smoke coverage. +- `tests/e2e/*`: canonical no-Gemini integration spec plus live full-stack Gemini specs (Playwright). ## Code Style diff --git a/apps/ui/README.md b/apps/ui/README.md index fc851cd1..12ec114d 100644 --- a/apps/ui/README.md +++ b/apps/ui/README.md @@ -37,11 +37,14 @@ The app uploads a track to the local DSP backend, shows the estimate and executi - Vite 6 - Tailwind CSS v4 - WaveSurfer.js -- Google Gen AI SDK - MIDI Writer JS - Vitest - Playwright +Gemini is backend-mediated: the UI never imports an AI SDK. The backend reads +`GEMINI_API_KEY` at runtime and the UI posts to +`POST /api/analysis-runs/{run_id}/interpretations`. + ## Prerequisites - Node.js 20+ diff --git a/docs/adr/0001-phase1-json-schema-v1.md b/docs/adr/0001-phase1-json-schema-v1.md index e702c2fe..078a2abe 100644 --- a/docs/adr/0001-phase1-json-schema-v1.md +++ b/docs/adr/0001-phase1-json-schema-v1.md @@ -14,7 +14,7 @@ ASA's HTTP contract has two layers that matter to external consumers: These two layers share the same field shape — Python emits camelCase JSON directly; there is no rename layer between them ([CLAUDE.md tripwire #3](../../CLAUDE.md)). That coupling has served ASA well, but it carries no explicit version or compatibility promise. Until this ADR, "Phase 1 schema v1" was implicit in `EXPECTED_TOP_LEVEL_KEYS` and `Phase1Result` — provable from code but not stated as a contract. -The external-repo review at [`docs/external-repo-review-2026-05-13.md`](../external-repo-review-2026-05-13.md) flagged this gap as Track 2 work. Partiels exports a documented schema (CSV/JSON/SDIF), and consumers of ASA's output (REAPER scripts, Max patches, downstream pipelines) need a stable shape to write against. +The external-repo review at [`docs/history/external-repo-review-2026-05-13.md`](../history/external-repo-review-2026-05-13.md) flagged this gap as Track 2 work. Partiels exports a documented schema (CSV/JSON/SDIF), and consumers of ASA's output (REAPER scripts, Max patches, downstream pipelines) need a stable shape to write against. ## Decision @@ -95,6 +95,6 @@ There is no single "phase1.v1.json" file. The schema is the four-test-suite inte ## Alternatives considered -- **Mirror Partiels' export schema verbatim.** Rejected in [external-repo-review-2026-05-13.md](../external-repo-review-2026-05-13.md). Partiels' CSV is `time, duration, label/value` per Vamp track — too flat for ASA's domain-named measurements. Would have collapsed the citation chain (Quality Invariant #2). +- **Mirror Partiels' export schema verbatim.** Rejected in [external-repo-review-2026-05-13.md](../history/external-repo-review-2026-05-13.md). Partiels' CSV is `time, duration, label/value` per Vamp track — too flat for ASA's domain-named measurements. Would have collapsed the citation chain (Quality Invariant #2). - **Generate a JSON Schema artifact.** Deferred. No current consumer needs it; the test-snapshot enforcement already prevents drift. - **Version the schema via HTTP `Accept` header.** Overkill for one runtime. Re-evaluate at v2. diff --git a/docs/history/README.md b/docs/history/README.md index 6e91375c..2e4bb837 100644 --- a/docs/history/README.md +++ b/docs/history/README.md @@ -16,4 +16,7 @@ Everything in this directory is past-tense. Treat it as a paper trail, not a sou - `phase1-hardening-plan.md` — completed Phase 1 hardening plan. - `library-review-torchfx-2026-05-13.md` — one-shot library evaluation of torchfx against ASA's DSP path. - `phase1-audit/` — one-shot advisory deliverable: audit, decks, evidence index, visual story pack. +- `library-review-torchfx-2026-05-13.md` — completed library review. +- `external-repo-review-2026-05-13.md` — completed external-repo incorporation review (openmeters / soundscope / Partiels / forever-jukebox). All three tracks resolved. +- `track1-spike-outcome-2026-05-13.md` — completed loudness verification spike. Fix shipped; regression test retained. - `archive/` — older archived plans and result stubs. diff --git a/docs/external-repo-review-2026-05-13.md b/docs/history/external-repo-review-2026-05-13.md similarity index 87% rename from docs/external-repo-review-2026-05-13.md rename to docs/history/external-repo-review-2026-05-13.md index f99afb08..3f2355d3 100644 --- a/docs/external-repo-review-2026-05-13.md +++ b/docs/history/external-repo-review-2026-05-13.md @@ -1,11 +1,18 @@ # External Repo Incorporation Review — 2026-05-13 +> **Archived 2026-05-15.** Completed review. Outcomes: +> - **Track 1 (loudness)** — verification spike confirmed Essentia's `LoudnessEBUR128` is correct on stereo program material; sample-rate threading fix shipped in [PR #34](https://github.com/slittycode/ableton-sonic-analyzer/pull/34). Reassigned spectrogram landed as an opt-in spectral enhancement in [PR #43](https://github.com/slittycode/ableton-sonic-analyzer/pull/43). +> - **Track 2 (schema + CSV export)** — ADR ratified at [`docs/adr/0001-phase1-json-schema-v1.md`](../adr/0001-phase1-json-schema-v1.md); CSV exporter shipped in [PR #35](https://github.com/slittycode/ableton-sonic-analyzer/pull/35). +> - **Track 3 (REST adopt list)** — URL-mode ingestion ([PR #36](https://github.com/slittycode/ableton-sonic-analyzer/pull/36)), `X-Admin-Key` ([PR #37](https://github.com/slittycode/ableton-sonic-analyzer/pull/37)), source-audio re-serve ([PR #40](https://github.com/slittycode/ableton-sonic-analyzer/pull/40)), and additive `publicStatus` ([PR #42](https://github.com/slittycode/ableton-sonic-analyzer/pull/42)) all landed. SDIF deferred. +> +> Kept here for the rationale trail; the action items are no longer live. + **Subject:** Evaluating openmeters, soundscope, Partiels, and forever-jukebox against ASA's mission and current implementation. **Authored by:** Claude (review-only branch `claude/review-asa-repos-5XA5r`). No code changes; this is an evaluation document. -**Anchor:** [PURPOSE.md](../PURPOSE.md) — the user-value test and the +**Anchor:** [PURPOSE.md](../../PURPOSE.md) — the user-value test and the decision framework. Every recommendation below traces back to one of its three "build it" criteria or its "stop and reconsider" branch. @@ -27,13 +34,13 @@ its three "build it" criteria or its "stop and reconsider" branch. | Repo | License | Implication for ASA | | --- | --- | --- | -| **ASA** (this repo) | MIT ([LICENSE](../LICENSE)) | Permissive; can adopt MIT/BSD/Apache code with attribution. **Cannot vendor GPL code.** | +| **ASA** (this repo) | MIT ([LICENSE](../../LICENSE)) | Permissive; can adopt MIT/BSD/Apache code with attribution. **Cannot vendor GPL code.** | | **openmeters** | GPL-3.0 | Algorithms are not copyrightable, but copied/translated *code* is. Read the **ITU-R BS.1770-5 standard PDF** for the algorithm; do not read openmeters' source if you intend to write a clean-room port. Treat as inspiration-only. | | **soundscope** | MIT | Code-compatible. But it's TUI-only with no JSON/CLI numeric output, so it can only serve as a *manual* cross-check, not an automated oracle. | | **Partiels** | GPL-3.0 | File-format schemas are facts, not copyrightable expression; mirroring its CSV column shape is safe. Do not vendor Partiels code. | | **forever-jukebox** | MIT | Code-compatible; safe to read and selectively port. | -**The plan's framing of "port algorithms (NOT fork — Rust → Essentia.js / JS)" misses one thing:** Rust → JS is not the boundary that matters here. ASA's measurement layer is **Python (Essentia)**, not JS/WASM. There is no Essentia.js in this codebase ([analyze_core.py:8](../apps/backend/analyze_core.py)). Re-implementing in JS/WASM would be a second port to the wrong runtime — see Track 1 below. +**The plan's framing of "port algorithms (NOT fork — Rust → Essentia.js / JS)" misses one thing:** Rust → JS is not the boundary that matters here. ASA's measurement layer is **Python (Essentia)**, not JS/WASM. There is no Essentia.js in this codebase ([analyze_core.py:8](../../apps/backend/analyze_core.py)). Re-implementing in JS/WASM would be a second port to the wrong runtime — see Track 1 below. --- @@ -41,9 +48,9 @@ its three "build it" criteria or its "stop and reconsider" branch. ### Where ASA's loudness lives today -- Implementation: [`analyze_core.py:186-230`](../apps/backend/analyze_core.py) — calls `essentia.standard.LoudnessEBUR128()`. -- True-peak: separate path via `essentia.standard.TruePeakDetector()` at [`analyze_core.py:233-`](../apps/backend/analyze_core.py). -- Fields emitted: `lufsIntegrated`, `lufsRange`, `lufsMomentaryMax`, `lufsShortTermMax`, `lufsCurve.{shortTerm,momentary}[]`, `truePeak`, `crestFactor`, `plr`. All gated by `EXPECTED_TOP_LEVEL_KEYS` in [`tests/test_analyze.py:26-43`](../apps/backend/tests/test_analyze.py). +- Implementation: [`analyze_core.py:186-230`](../../apps/backend/analyze_core.py) — calls `essentia.standard.LoudnessEBUR128()`. +- True-peak: separate path via `essentia.standard.TruePeakDetector()` at [`analyze_core.py:233-`](../../apps/backend/analyze_core.py). +- Fields emitted: `lufsIntegrated`, `lufsRange`, `lufsMomentaryMax`, `lufsShortTermMax`, `lufsCurve.{shortTerm,momentary}[]`, `truePeak`, `crestFactor`, `plr`. All gated by `EXPECTED_TOP_LEVEL_KEYS` in [`tests/test_analyze.py:26-43`](../../apps/backend/tests/test_analyze.py). - Curves are downsampled via `_downsample_lufs_array` in `dsp_utils.py` for payload size. ### What openmeters offers @@ -68,7 +75,7 @@ The plan asserts ASA "likely tracks an older BS.1770 rev." Two facts make this l - `-4` (2015): multichannel weighting extended to NHK 22.2. - `-5` (2023): primarily editorial clarifications + true-peak measurement refinements. -ASA processes stereo audio. The functional delta between Essentia's `LoudnessEBUR128` and an ITU-R BS.1770-5 reference implementation, *on stereo material*, is likely below the precision ASA reports (`round(integrated, 1)` — one decimal place of a LU, per [`analyze_core.py:216`](../apps/backend/analyze_core.py)). +ASA processes stereo audio. The functional delta between Essentia's `LoudnessEBUR128` and an ITU-R BS.1770-5 reference implementation, *on stereo material*, is likely below the precision ASA reports (`round(integrated, 1)` — one decimal place of a LU, per [`analyze_core.py:216`](../../apps/backend/analyze_core.py)). This does not mean "do nothing." It means **measure before rewriting**: @@ -111,9 +118,9 @@ But: ### Where ASA's schema lives today -- Spec: [`apps/backend/JSON_SCHEMA.md`](../apps/backend/JSON_SCHEMA.md) — 60+ top-level keys, deeply nested per-domain detail objects. -- Test snapshot: `EXPECTED_TOP_LEVEL_KEYS` at [`tests/test_analyze.py:26-43`](../apps/backend/tests/test_analyze.py). -- Frontend mirror: [`apps/ui/src/types/measurement.ts`](../apps/ui/src/types/measurement.ts) — `Phase1Result` and its detail interfaces. +- Spec: [`apps/backend/JSON_SCHEMA.md`](../../apps/backend/JSON_SCHEMA.md) — 60+ top-level keys, deeply nested per-domain detail objects. +- Test snapshot: `EXPECTED_TOP_LEVEL_KEYS` at [`tests/test_analyze.py:26-43`](../../apps/backend/tests/test_analyze.py). +- Frontend mirror: [`apps/ui/src/types/measurement.ts`](../../apps/ui/src/types/measurement.ts) — `Phase1Result` and its detail interfaces. - Time-tagged fields today: `lufsCurve.{shortTerm,momentary}[]` (`{t, lufs}`), `spectralBalanceTimeSeries[]` (`{t, subBass, lowBass, …}`), `rhythmDetail.tempoCurve[]` (`{t, bpm}`), `segmentLoudness`/`segmentSpectral`/`segmentStereo`/`segmentKey` (per-section summary objects, not per-frame). - Export formats: JSON (over HTTP); MIDI for melody/transcription via `apps/ui/src/services/midi/midiExport.ts`. **No CSV, SDIF, LAB, or REAPER export today.** @@ -182,7 +189,7 @@ SDIF is IRCAM's binary frame format. The full SDIF spec involves frame/matrix si ### Where ASA's API lives today -From the survey of [`server.py`](../apps/backend/server.py), [`server_phase1.py`](../apps/backend/server_phase1.py), [`server_phase2.py`](../apps/backend/server_phase2.py), [`server_upload.py`](../apps/backend/server_upload.py), and [`analysis_runtime.py`](../apps/backend/analysis_runtime.py): +From the survey of [`server.py`](../../apps/backend/server.py), [`server_phase1.py`](../../apps/backend/server_phase1.py), [`server_phase2.py`](../../apps/backend/server_phase2.py), [`server_upload.py`](../../apps/backend/server_upload.py), and [`analysis_runtime.py`](../../apps/backend/analysis_runtime.py): **Canonical (run-oriented):** - `POST /api/analysis-runs` — create run (multipart upload). @@ -195,11 +202,11 @@ From the survey of [`server.py`](../apps/backend/server.py), [`server_phase1.py` - `POST /api/analysis-runs/{run_id}/pitch-note-translations`. - `POST /api/analysis-runs/{run_id}/interpretations`. -**Legacy:** `POST /api/analyze`, `POST /api/analyze/estimate`, `POST /api/phase2` — wrappers kept for compatibility only ([CLAUDE.md](../CLAUDE.md), "Staged Analysis Runs"). +**Legacy:** `POST /api/analyze`, `POST /api/analyze/estimate`, `POST /api/phase2` — wrappers kept for compatibility only ([CLAUDE.md](../../CLAUDE.md), "Staged Analysis Runs"). **Error envelope:** Always includes `requestId`, `error.code`, `error.message`, `error.retryable`, `diagnostics`. Stage states: `queued`, `running`, `blocked`, `ready`, `completed`, `failed`, `interrupted`, `not_requested`. -**Upload limit:** 100 MiB raw / 101 MiB request envelope ([`upload_limits.py:7-16`](../apps/backend/upload_limits.py) — `MAX_UPLOAD_SIZE_BYTES` 104,857,600 + `UPLOAD_REQUEST_SIZE_SLACK_BYTES` 1,048,576 = 105,906,176 bytes = 101 MiB). +**Upload limit:** 100 MiB raw / 101 MiB request envelope ([`upload_limits.py:7-16`](../../apps/backend/upload_limits.py) — `MAX_UPLOAD_SIZE_BYTES` 104,857,600 + `UPLOAD_REQUEST_SIZE_SLACK_BYTES` 1,048,576 = 105,906,176 bytes = 101 MiB). ### What forever-jukebox exposes @@ -230,7 +237,7 @@ From [`api/README.md`](https://github.com/creightonlinza/forever-jukebox/blob/ma | Job retry | Auto-creates a new job on resubmission of a retryable failure | Client must explicitly recreate the run; `error.retryable` is advisory | **forever-jukebox slightly leads.** ASA's `retryable` flag is informative; turning it into automatic retry on resubmit would close the loop. | | Discovery (top/trending/recent) | Yes | No | **Out of scope** for ASA's target user — they have a specific reference track, not a feed. | | Search (Spotify/YouTube) | Yes | No | **Out of scope** for ASA — pulling rights-restricted audio is a separate product question. | -| Admin auth | `X-Admin-Key` header | None at the canonical-API layer ([`auth_context.py`](../apps/backend/auth_context.py) handles hosted-mode user context but not admin) | **forever-jukebox leads operationally**; ASA has a forward design for hosted auth but no admin-key surface for operator tasks (purge a run, audit a job). Worth filing. | +| Admin auth | `X-Admin-Key` header | None at the canonical-API layer ([`auth_context.py`](../../apps/backend/auth_context.py) handles hosted-mode user context but not admin) | **forever-jukebox leads operationally**; ASA has a forward design for hosted auth but no admin-key surface for operator tasks (purge a run, audit a job). Worth filing. | | Audio re-serve | `GET /api/audio/` | No | **Worth considering** — ASA already persists artifacts; re-serving the original audio at the staged endpoint avoids re-uploading on Phase 2 reruns. | ### Adopt / Adapt / Reject per endpoint @@ -244,13 +251,13 @@ From [`api/README.md`](https://github.com/creightonlinza/forever-jukebox/blob/ma | Collapsed public state machine | **Adapt** | Keep the 8 internal states; expose 5 in the public snapshot. Map `blocked` and `not_requested` to `pending`; map `ready` to `queued`. Document the collapse in JSON_SCHEMA.md. | | `X-Admin-Key` for admin ops | **Adopt** | Apply to `DELETE /api/analysis-runs/{run_id}` and a forthcoming `GET /api/admin/runs` listing. Gate via existing `auth_context.py`. | | Search / discovery / favorites / play tracking | **Reject** | Out of scope for the recreation-blueprint mission. PURPOSE.md decision-framework branch 5: "stop and reconsider." | -| Echo Nest-style beats/segments/sections output | **Reject** | ASA emits richer, named domain measurements ([`JSON_SCHEMA.md`](../apps/backend/JSON_SCHEMA.md)). Adopting Echo Nest's flat segment schema would collapse the citation chain (Quality Invariant #2). | +| Echo Nest-style beats/segments/sections output | **Reject** | ASA emits richer, named domain measurements ([`JSON_SCHEMA.md`](../../apps/backend/JSON_SCHEMA.md)). Adopting Echo Nest's flat segment schema would collapse the citation chain (Quality Invariant #2). | ### Definition of done (revised) - [ ] Side-by-side endpoint comparison committed (this section is the seed; expand into `docs/api-comparison-2026-05-13.md` if it gets longer). - [ ] ADR per "adopt" decision: URL input, admin key, audio re-serve, collapsed public state machine. -- [ ] API doc updated (currently lives in [`apps/backend/ARCHITECTURE.md`](../apps/backend/ARCHITECTURE.md)) — specifically the public state-machine collapse, since it's the only contract-shape change. +- [ ] API doc updated (currently lives in [`apps/backend/ARCHITECTURE.md`](../../apps/backend/ARCHITECTURE.md)) — specifically the public state-machine collapse, since it's the only contract-shape change. --- @@ -286,7 +293,7 @@ The plan's "Out of scope" list is correct: - **spleeter-web** — revisit when ASA adds stems beyond Demucs. - **Olaf** — revisit if cross-take alignment becomes a feature. - **nightingale** — desktop shell; not relevant until/unless ASA grows a desktop app. -- **ChordMiniApp / chordonomicon / rawl** — harmonic-domain tools; flagged separately in [`BACKLOG.md`](../BACKLOG.md) as a Harmonia track. +- **ChordMiniApp / chordonomicon / rawl** — harmonic-domain tools; flagged separately in [`BACKLOG.md`](../../BACKLOG.md) as a Harmonia track. No change to the deferral logic. @@ -294,7 +301,7 @@ No change to the deferral logic. ## Sources -- Anchoring docs in this repo: [PURPOSE.md](../PURPOSE.md), [CLAUDE.md](../CLAUDE.md), [apps/backend/JSON_SCHEMA.md](../apps/backend/JSON_SCHEMA.md), [apps/backend/analyze_core.py](../apps/backend/analyze_core.py), [apps/backend/upload_limits.py](../apps/backend/upload_limits.py). +- Anchoring docs in this repo: [PURPOSE.md](../../PURPOSE.md), [CLAUDE.md](../../CLAUDE.md), [apps/backend/JSON_SCHEMA.md](../../apps/backend/JSON_SCHEMA.md), [apps/backend/analyze_core.py](../../apps/backend/analyze_core.py), [apps/backend/upload_limits.py](../../apps/backend/upload_limits.py). - [openmeters on GitHub](https://github.com/httpsworldview/openmeters) (GPL-3.0). - [bananaofhappiness/soundscope on GitHub](https://github.com/bananaofhappiness/soundscope) (MIT). - [Ircam-Partiels/Partiels on GitHub](https://github.com/Ircam-Partiels/Partiels) (GPL-3.0). diff --git a/docs/track1-spike-outcome-2026-05-13.md b/docs/history/track1-spike-outcome-2026-05-13.md similarity index 84% rename from docs/track1-spike-outcome-2026-05-13.md rename to docs/history/track1-spike-outcome-2026-05-13.md index a8c8ba90..75b90da9 100644 --- a/docs/track1-spike-outcome-2026-05-13.md +++ b/docs/history/track1-spike-outcome-2026-05-13.md @@ -1,11 +1,13 @@ # Track 1 verification spike outcome — 2026-05-13 +> **Archived 2026-05-15.** Completed verification spike. The sample-rate threading follow-up shipped in [PR #34](https://github.com/slittycode/ableton-sonic-analyzer/pull/34); the loudness regression test now lives at `apps/backend/tests/test_loudness_r128.py`. Kept here for the rationale trail. + **Scope:** verify the review's premise (in [`external-repo-review-2026-05-13.md`](external-repo-review-2026-05-13.md)) that ASA's existing loudness path is already correct on its operating sample rate, so no port of openmeters' BS.1770-5 implementation is needed. **Result (pending CI):** The added test -[`apps/backend/tests/test_loudness_r128.py`](../apps/backend/tests/test_loudness_r128.py) +[`apps/backend/tests/test_loudness_r128.py`](../../apps/backend/tests/test_loudness_r128.py) exercises EBU Tech 3341 Cases 1 and 2 — the canonical sine-tone conformance signals — through ASA's `analyze_loudness` at 44.1 kHz. If those assert green, the premise check is positive and Track 1's @@ -47,8 +49,8 @@ The original finding is preserved below for context. While tracing the loudness path, the spike surfaced an asymmetry between ASA's two call sites for `LoudnessEBUR128`: -- [`analyze_core.py:197`](../apps/backend/analyze_core.py) — `analyze_loudness(stereo)` instantiates `es.LoudnessEBUR128()` with **no** `sampleRate` argument (Essentia's default is 44100). -- [`analyze_fast.py:100`](../apps/backend/analyze_fast.py) — `analyze_fast` instantiates `es.LoudnessEBUR128(sampleRate=sample_rate)` with the actual sample rate. +- [`analyze_core.py:197`](../../apps/backend/analyze_core.py) — `analyze_loudness(stereo)` instantiates `es.LoudnessEBUR128()` with **no** `sampleRate` argument (Essentia's default is 44100). +- [`analyze_fast.py:100`](../../apps/backend/analyze_fast.py) — `analyze_fast` instantiates `es.LoudnessEBUR128(sampleRate=sample_rate)` with the actual sample rate. The full pipeline (`analyze.py:1448`) loads stereo via `load_stereo`, which preserves the **source file's native sample rate** (no upstream resample).