From 45d3c28d4cf8762ea0e7bec1bb973f2e59edbf34 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 30 May 2026 20:50:46 +0000 Subject: [PATCH 1/2] docs: refresh stale items vs current code; archive completed hosting record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sweep against the 2026-05-30 audit findings — the doc surface had drifted through the MT3, Live 12 catalogue, pianoroll/symusic, and recommendation-proof merges. Adds those features to the inventories where they were missing, fixes a wrong line reference, removes a dead commit hash, refreshes date stamps, and archives a completed-work record. - MT3 polyphonic transcription: add to apps/backend/AGENTS.md (file map + guidance), apps/backend/ARCHITECTURE.md (Components table + mt3-transcriptions route), apps/backend/README.md (Polyphonic section now reflects opt-in shipping), apps/ui/AGENTS.md + apps/ui/README.md (mt3Client.ts + UI surface), CLAUDE.md (#19 backend file, #6a frontend service), and all three CHANGELOG.md files. - Live 12 catalogue + phase2_catalogue_gates, transcription pianoroll + symusic, recommendation-proof campaign: add to CHANGELOG.md (root) and apps/backend/CHANGELOG.md and apps/ui/CHANGELOG.md. - CLAUDE.md: fix wrong analysis_runtime.py line ref (685 → ~800), add ASA_ENABLE_MT3 and ASA_SAMPLE_SYNTH_BACKEND env vars, correct the tsconfig excludes description, drop the unreachable 5c40dd44 commit hash. - apps/backend/AGENTS.md + ARCHITECTURE.md: drop the same dead 5c40dd44 hash. - docs/ARCHITECTURE_STRATEGY.md + docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md: reflect that MT3 has shipped as an opt-in, additive stage (the spike harness remains the offline comparison rig for other candidates). - docs/SAMPLE_GENERATION.md: fix the schematic node label (synth_kit() → the real synth_kick/snare/hat() trio). - apps/backend/JSON_SCHEMA.md: annotate the shared-vs-full carve-out so the full-mode-only keys (keyProfile, tuningFrequency, tuningCents, lufsMomentaryMax, lufsShortTermMax, pitchDetail) are explicit per CLAUDE.md tripwire #4. - Archive docs/PUBLIC_HOSTING_FOUNDATION.md → docs/history/public-hosting-foundation-2026-04-01.md with an archived-banner; update inbound references in CHANGELOG.md, docs/SETUP.md, docs/history/README.md, and docs/history/archive/refactor-state-2026-03-18.md. - incorporations/forking-plans-2026-05-14.md: add a 2026-05-30 status update noting Plan 6 substantially landed via the Live 12 catalogue + gates work. No code touched — markdown only. https://claude.ai/code/session_016m6cUFNwb3kD2EzAbgqCHe --- CHANGELOG.md | 6 ++++- CLAUDE.md | 14 +++++++----- apps/backend/AGENTS.md | 10 +++++---- apps/backend/ARCHITECTURE.md | 6 +++-- apps/backend/CHANGELOG.md | 4 ++++ apps/backend/JSON_SCHEMA.md | 2 ++ apps/backend/README.md | 13 ++++++++--- apps/ui/AGENTS.md | 3 ++- apps/ui/CHANGELOG.md | 3 +++ apps/ui/README.md | 1 + docs/ARCHITECTURE_STRATEGY.md | 2 +- docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md | 22 ++++++++++--------- docs/SAMPLE_GENERATION.md | 4 ++-- docs/SETUP.md | 4 ++-- docs/history/README.md | 1 + .../archive/refactor-state-2026-03-18.md | 2 +- .../public-hosting-foundation-2026-04-01.md} | 4 +++- incorporations/forking-plans-2026-05-14.md | 2 ++ 18 files changed, 70 insertions(+), 33 deletions(-) rename docs/{PUBLIC_HOSTING_FOUNDATION.md => history/public-hosting-foundation-2026-04-01.md} (96%) diff --git a/CHANGELOG.md b/CHANGELOG.md index a85b2da0..6b392f05 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ All notable changes to `asa` are documented here. ## Unreleased ### Added +- **Optional MT3 polyphonic transcription stage** ([`apps/backend/mt3_transcription.py`](apps/backend/mt3_transcription.py), `apps/ui/src/services/mt3Client.ts`): Google MT3 (T5X) wired as an opt-in, additive staged backend — peer of pitch/note translation. Gated by run-level form field `mt3_mode='enabled'` on `POST /api/analysis-runs` (canonical) and env var `ASA_ENABLE_MT3=1` on the legacy `analyze.py` CLI. Emits per-stem MIDI as artifacts and an `mt3` namespace on the run snapshot. Never overrides measurement (PURPOSE.md invariant #1); heavy dependencies pinned separately in `apps/backend/requirements-mt3.txt`. New endpoint: `POST /api/analysis-runs/{run_id}/mt3-transcriptions`. See [`JSON_SCHEMA.md` "Optional MT3 Namespace"](apps/backend/JSON_SCHEMA.md#optional-mt3-namespace). +- **Live 12 source-catalogue + Phase 2 output-validation gates** ([`apps/backend/live12_catalogue.py`](apps/backend/live12_catalogue.py), [`apps/backend/phase2_catalogue_gates.py`](apps/backend/phase2_catalogue_gates.py), [`data/live12_catalogue.json`](data/live12_catalogue.json) + schema, [`scripts/build_live12_catalogue.py`](scripts/build_live12_catalogue.py)): regenerable static device/parameter catalogue extracted from the upstream `gluon/AbletonLive12_MIDIRemoteScripts`. Backend cross-checks every `{device, parameter, value, phase1Fields}` record in `mixAndMasterChain`, `abletonRecommendations`, and `secretSauce.workflowSteps` against the catalogue and emits **warn-and-keep** `RECOMMENDATION_UNVERIFIED` events on `validationWarnings` (reasons: `device_unknown`, `parameter_unknown`, `value_out_of_range`, `citation_missing`). Never drops a recommendation, never rewrites a parameter — an earlier fuzzy-rewrite path produced confidently-wrong output, so the contract is warn-only. +- **Transcription pianoroll endpoint + symusic consolidation** ([`apps/backend/transcription_pianoroll.py`](apps/backend/transcription_pianoroll.py), `apps/ui/src/services/transcriptionPianorollClient.ts`, `apps/ui/src/components/TranscriptionPianoroll.tsx` / `TranscriptionPianorollBlock.tsx`): velocity-encoded `(pitch, time)` uint8 matrix backed by `symusic`. Backs `GET /api/analysis-runs/{run_id}/transcription/pianoroll`; mounted in the Session Musician suite when `transcriptionDetail.noteCount > 0`. Derived view — Phase 1 stays authoritative; response cites the Phase 1 `bpm` and `timeSignature` so every cell traces back to a measurement. +- **Recommendation-proof campaign** ([`GOAL.md`](GOAL.md) + [`apps/backend/NEEDS.md`](apps/backend/NEEDS.md) + [`apps/backend/RECOMMENDATION_VERDICT.md`](apps/backend/RECOMMENDATION_VERDICT.md)): recommendation-quality scorer ([`apps/backend/recommendation_evaluation.py`](apps/backend/recommendation_evaluation.py) + `scripts/evaluate_recommendations.py`), Node 23+ deterministic source bridge (`scripts/emit_deterministic_recs.ts`), per-recommendation UI corpus-verification badge (`apps/ui/src/services/recommendationVerification.ts`, `apps/ui/src/data/recommendationVerification.ts`, `apps/ui/src/components/RecommendationVerificationBadge.tsx`), and the known-settings fixture corpus at `apps/backend/tests/fixtures/recommendation_tracks/`. Off the product path; badge hidden until corpus has scored fixtures. - **Backend Phase 2 citation-path verification** (defense-in-depth): [`_validate_phase2_citation_paths`](apps/backend/server_phase2.py) (called from [`server.py`](apps/backend/server.py)) flags any cited `phase1Fields` path that doesn't resolve against the authoritative measurement payload, mirroring the frontend's `validatePhase1FieldCitations`. WARNING-only — rides the existing `validationWarnings` channel and surfaces in the UI's interpretation-caution section; Phase 1 stays authoritative and citations are never rejected. - **`packages/loudness-spectro-wasm`** (Phase 1): browser-first Rust→WASM DSP that lifts openmeters' ITU-R BS.1770-5 / EBU R128 loudness (K-weighting, momentary/short-term, 4× true-peak) and adds integrated loudness + gating + LRA, plus an A-weighted whole-file spectrum and a spectral-reassignment spectrogram. Loudness is validated against absolute EBU Tech 3341/3342 conformance signals, the `ebur128` crate, and a pyloudnorm cross-check, with CI. Standalone — not yet imported by `apps/ui` or `apps/backend`; the Essentia path remains the authoritative Phase 1 loudness source and the product reassigned-spectrogram endpoint runs on librosa. GPL-3.0-or-later (inherited from openmeters). See [`packages/loudness-spectro-wasm/README.md`](packages/loudness-spectro-wasm/README.md). - **Loudness-action guardrail** in Phase 2 validation: a `MISSING_LOUDNESS_ACTION` consistency check (`validateLoudnessActionPresence`) backed by [`apps/ui/src/services/loudnessGuardrails.ts`](apps/ui/src/services/loudnessGuardrails.ts). When Phase 1 measures an objective loudness defect (digital clipping via `saturationDetail.clippedSampleCount`, or a true-peak over via `truePeak`), Phase 2 must include a MASTERING/DYNAMICS card citing the triggering measurement. The Phase 2 system prompt was updated to require it. @@ -41,7 +45,7 @@ All notable changes to `asa` are documented here. - **UI design-system migration ("D-series").** Introduced a shared primitive layer in `apps/ui/src/components/ui/` (`Button`, `Panel`, `DeviceRack`, `SectionHeader`, `MetricBar`/`MetricBarRow`/`MetricTile`, `DataTable`, `EmptyState`, `LedIndicator`, `Pill`, `SignalChain`, `ChainSeparator`, `TimeReadout`, `Checkbox`, `Tooltip`) with Storybook stories and semantic design tokens in `src/index.css`, then migrated feature components onto it — inline hex colors replaced with tokens and bespoke layout boxes replaced with primitives. Also: TypeScript strict mode, `prefers-reduced-motion` support, and a source-SR-preserving STFT spectrogram as the default spectrogram tab. - **Backend monolith split** into domain modules (commit `5c40dd44`): `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`. Server routes likewise split into `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`. - **`BatchedBandpass` centralization** ([`apps/backend/dsp_bandbank.py`](apps/backend/dsp_bandbank.py)): per-band 4th-order Butterworth filtering with zero-phase `filtfilt` now lives in one place instead of being duplicated across detectors. -- **Hosted runtime foundation** landed without disturbing local mode: [`runtime_profile.py`](apps/backend/runtime_profile.py), [`worker.py`](apps/backend/worker.py), [`artifact_storage.py`](apps/backend/artifact_storage.py), [`auth_context.py`](apps/backend/auth_context.py). See [`docs/PUBLIC_HOSTING_FOUNDATION.md`](docs/PUBLIC_HOSTING_FOUNDATION.md). +- **Hosted runtime foundation** landed without disturbing local mode: [`runtime_profile.py`](apps/backend/runtime_profile.py), [`worker.py`](apps/backend/worker.py), [`artifact_storage.py`](apps/backend/artifact_storage.py), [`auth_context.py`](apps/backend/auth_context.py). See [`docs/history/public-hosting-foundation-2026-04-01.md`](docs/history/public-hosting-foundation-2026-04-01.md). - Synthesis character labels aligned to phase2 prompt thresholds (three-tier: clean subtractive / FM-acid / wavetable-noise) - Removed citation instructions from Phase 2 system prompt (citations added noise, not value) - `dynamicCharacter` forwarded through `_build_phase1()` to Gemini Phase 2 diff --git a/CLAUDE.md b/CLAUDE.md index 6a390af7..c37d96d6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -178,12 +178,13 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths 15. **`dsp_bandbank.py` + `dsp_utils.py`**: Shared DSP primitives — `BatchedBandpass` (4th-order Butterworth bandpass bank) and cross-module utility functions. 16. **`phase1_evaluation.py` + `phase1_report_html.py`, `polyphonic_evaluation.py`, `beat_evaluation.py` + `beat_report_html.py`, `loudness_rec_evaluation.py`, `recommendation_evaluation.py`**: Offline evaluation harnesses (deterministic-metric / detector-stability reporting, research-only polyphonic transcription, the beat/downbeat measurement gate that benchmarks CPJKU/beat_this against the shipping kick-accent heuristic, loudness-recommendation reachability, and the recommendation-quality scorer that grades Phase 2 recommendations against known-settings fixtures — see [`apps/backend/NEEDS.md`](apps/backend/NEEDS.md)). Not on the product path; driven by `scripts/evaluate_*.py`. Deleting them restores the product exactly. 17. **`utils/cleanup.py`**: Periodic artifact-cleanup helpers used by the server background-task loop. -18. **`transcription_pianoroll.py`**: Renders the pitch-note translation stage's `transcriptionDetail` as a velocity-encoded `(pitch, time)` uint8 matrix via [`symusic`](https://github.com/Yikai-Liao/symusic). Backs `GET /api/analysis-runs/{run_id}/transcription/pianoroll`. Derived view, never overrides Phase 1; the response cites Phase 1's `bpm` + `timeSignature` so chain of custody is preserved. Note: `transcriptionDetail` is *stripped* from `measurement.result` (see `analysis_runtime.py:685`) and lives in the `pitchNoteTranslation` stage instead — the route reads from there. -19. **`live12_catalogue.py` + `phase2_catalogue_gates.py`**: Source-extracted Live 12 catalogue lookup and Phase 2 output-validation gates. `live12_catalogue.py` loads `data/live12_catalogue.json` (generated by `scripts/build_live12_catalogue.py`), validates it against the published schema, and exposes a `Live12Catalogue` API with case-insensitive `has_device`, exact-match parameter lookup, and a `fuzzy_resolve` escape hatch. `phase2_catalogue_gates.py` cross-checks every `{device, parameter, value, phase1Fields}` record in `mixAndMasterChain`, `abletonRecommendations`, and `secretSauce.workflowSteps` against the catalogue and emits **warn-and-keep** `RECOMMENDATION_UNVERIFIED` events on `validationWarnings` (reasons: `device_unknown`, `parameter_unknown`, `value_out_of_range`, `citation_missing`). NEVER drops a recommendation, NEVER rewrites a parameter — an earlier fuzzy-rewrite path produced confidently-wrong output (wrong EQ band + wrong A/B curve), so the contract is now warn-only. Mirrors `_validate_phase2_citation_paths` in spirit. Wired into `server.py` after `_validate_phase2_citation_paths`; load/parse errors degrade to a single skip warning instead of failing the response. +18. **`transcription_pianoroll.py`**: Renders the pitch-note translation stage's `transcriptionDetail` as a velocity-encoded `(pitch, time)` uint8 matrix via [`symusic`](https://github.com/Yikai-Liao/symusic). Backs `GET /api/analysis-runs/{run_id}/transcription/pianoroll`. Derived view, never overrides Phase 1; the response cites Phase 1's `bpm` + `timeSignature` so chain of custody is preserved. Note: `transcriptionDetail` is *stripped* from `measurement.result` (see `analysis_runtime.py` ~L800) and lives in the `pitchNoteTranslation` stage instead — the route reads from there. +19. **`mt3_transcription.py`**: Optional polyphonic transcription via Google MT3 (T5X). Additive only — never overrides measurement (PURPOSE.md invariant #1). Gated on the env var `ASA_ENABLE_MT3=1` for the legacy CLI path and the run-level form field `mt3_mode='enabled'` for the canonical staged API. Driven from `_execute_mt3_attempt`/`_mt3_worker_loop` in `server.py`; emits per-stem MIDI as artifacts and an `mt3` namespace on the run snapshot. Heavy dependencies pinned in `requirements-mt3.txt`. See [`JSON_SCHEMA.md` "Optional MT3 Namespace"](apps/backend/JSON_SCHEMA.md#optional-mt3-namespace). +20. **`live12_catalogue.py` + `phase2_catalogue_gates.py`**: Source-extracted Live 12 catalogue lookup and Phase 2 output-validation gates. `live12_catalogue.py` loads `data/live12_catalogue.json` (generated by `scripts/build_live12_catalogue.py`), validates it against the published schema, and exposes a `Live12Catalogue` API with case-insensitive `has_device`, exact-match parameter lookup, and a `fuzzy_resolve` escape hatch. `phase2_catalogue_gates.py` cross-checks every `{device, parameter, value, phase1Fields}` record in `mixAndMasterChain`, `abletonRecommendations`, and `secretSauce.workflowSteps` against the catalogue and emits **warn-and-keep** `RECOMMENDATION_UNVERIFIED` events on `validationWarnings` (reasons: `device_unknown`, `parameter_unknown`, `value_out_of_range`, `citation_missing`). NEVER drops a recommendation, NEVER rewrites a parameter — an earlier fuzzy-rewrite path produced confidently-wrong output (wrong EQ band + wrong A/B curve), so the contract is now warn-only. Mirrors `_validate_phase2_citation_paths` in spirit. Wired into `server.py` after `_validate_phase2_citation_paths`; load/parse errors degrade to a single skip warning instead of failing the response. The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details. -**Phase 2 (`POST /api/phase2`, legacy compat):** Uploads audio to Gemini inline if ≤100 MiB, or via the Gemini Files API if larger. Phase 1 JSON is appended to the system prompt from `prompts/phase2_system.txt`. Also relevant: `prompts/stem_summary_system.txt` and `prompts/live12_device_catalog.json` (the *prompt-injected* device catalogue — distinct from the runtime-validation `data/live12_catalogue.json` consumed by `phase2_catalogue_gates.py`). Backend defense-in-depth: `server_phase2.py`'s `_validate_phase2_citation_paths` mirrors the frontend citation-existence check and emits `validationWarnings` when a recommendation cites a Phase 1 path that doesn't exist — it flags invented citations rather than failing, since Phase 1 stays authoritative (invariant #1). `phase2_catalogue_gates.apply_live12_catalogue_gates` runs immediately after for source-catalogue checks (see backend file #19 above). +**Phase 2 (`POST /api/phase2`, legacy compat):** Uploads audio to Gemini inline if ≤100 MiB, or via the Gemini Files API if larger. Phase 1 JSON is appended to the system prompt from `prompts/phase2_system.txt`. Also relevant: `prompts/stem_summary_system.txt` and `prompts/live12_device_catalog.json` (the *prompt-injected* device catalogue — distinct from the runtime-validation `data/live12_catalogue.json` consumed by `phase2_catalogue_gates.py`). Backend defense-in-depth: `server_phase2.py`'s `_validate_phase2_citation_paths` mirrors the frontend citation-existence check and emits `validationWarnings` when a recommendation cites a Phase 1 path that doesn't exist — it flags invented citations rather than failing, since Phase 1 stays authoritative (invariant #1). `phase2_catalogue_gates.apply_live12_catalogue_gates` runs immediately after for source-catalogue checks (see backend file #20 above). **Python version constraint:** Python 3.11.x required on macOS arm64. Essentia 2.1b6 wheels are only published for 3.11; this constraint may be relaxable if Essentia publishes 3.12+ wheels. @@ -199,6 +200,7 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V 4. **`src/services/httpClient.ts`**: Shared fetch helpers and request-header injection used by the run/artifact/sample clients. 5. **`src/services/spectralArtifactsClient.ts`**: Fetches spectrogram/spectral-evolution artifacts via `/api/analysis-runs/{run_id}/artifacts/…`. 6. **`src/services/transcriptionPianorollClient.ts`**: Fetches the velocity-encoded transcription pianoroll matrix via `/api/analysis-runs/{run_id}/transcription/pianoroll`. Backed by `apps/backend/transcription_pianoroll.py`. Rendered by `TranscriptionPianoroll.tsx` (canvas heatmap) inside `TranscriptionPianorollBlock.tsx`, which `AnalysisResults.tsx` mounts in the Session Musician suite when `transcriptionDetail.noteCount > 0`. +6a. **`src/services/mt3Client.ts`**: Opt-in MT3 polyphonic transcription client. POSTs `/api/analysis-runs/{run_id}/mt3-transcriptions` and reads the additive `mt3` namespace from the run snapshot. Only fires when the user enables MT3 at upload time (form field `mt3_mode='enabled'`); measurement stays authoritative. 7. **`src/services/sampleGenerationClient.ts`**: Phase 3 audition-sample POST/GET against `/api/analysis-runs/{run_id}/samples`, plus per-clip artifact streaming. 8. **`src/services/mixDoctor.ts`**: Mix advisory logic — client-side scoring and suggestions against measured spectral balance. 9. **`src/services/phase2Validator.ts`** + **`loudnessGuardrails.ts`**: Runtime guardrail. Validates Phase 2 consistency against Phase 1 (`validateBPMConsistency`, `validateKeyConsistency`, `validateLUFSConsistency`, `validateGenreDSPConsistency`, `validateNumericBounds`, `validateLoudnessActionPresence`). `loudnessGuardrails.ts` defines the objective loudness defects (digital clipping via `saturationDetail.clippedSampleCount`, true-peak overs via `truePeak`) that a Phase 2 mastering/dynamics card *must* address — a missing action surfaces as a `MISSING_LOUDNESS_ACTION` violation. The aggregate `validatePhase2Consistency` drives **`Phase2ConsistencyReport.tsx`**, which renders the chain-of-custody report on the results surface (`AnalysisResults.tsx`, `hideWhenClean`) and in full inside the diagnostic log (`DiagnosticLog.tsx`); `App.tsx` computes the report and passes it down. @@ -235,6 +237,8 @@ DISABLE_HMR="false" # set "true" for dev environments that need HMR SONIC_ANALYZER_PORT=8100 GEMINI_API_KEY="your_key_here" # read by server.py at runtime, not in browser bundle SONIC_ANALYZER_ADMIN_KEY="optional" # if set, DELETE /api/analysis-runs/{run_id} accepts an X-Admin-Key header that bypasses ownership for operator-level purge. Unset by default; admin path is closed. +ASA_ENABLE_MT3="0" # set to "1" on the legacy analyze.py CLI path to run the optional MT3 polyphonic transcription pass. Canonical staged API uses the run-level form field `mt3_mode='enabled'` instead. Heavy deps in apps/backend/requirements-mt3.txt. +ASA_SAMPLE_SYNTH_BACKEND="auto" # Phase 3 audition-sample synth backend: `auto` (default, prefers FluidSynth and falls back to symusic), `symusic`, or `fluidsynth` to pin one. See apps/backend/sample_synthesis.py. ``` Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-only. `SONIC_ANALYZER_ADMIN_KEY` is backend-only and never exposed to clients. @@ -245,7 +249,7 @@ Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-onl - **Architecture strategy:** Read `docs/ARCHITECTURE_STRATEGY.md` before proposing structural changes to the dependency stack, transcription pipeline, or layer boundaries. - **No linter/formatter:** No ESLint, Prettier, or Ruff configured. Follow the style of the surrounding code. - **Backend tests use stdlib `unittest`**, not pytest. Frontend tests use Vitest in `node` environment (not jsdom). -- **`npm run lint`** only type-checks `src/`; test files and `playwright.config.ts` are excluded from `tsconfig.json`. +- **`npm run lint`** only type-checks `src/`; `tests/`, `dist/`, `node_modules/`, `playwright.config.ts`, and `vitest.config.ts` are excluded from `tsconfig.json`. - **Canonical ports:** UI on 3100, backend on 8100. `./scripts/dev.sh` fails loudly if either port is occupied. - **`--fast` flag** runs a streamlined pipeline (BPM, key, loudness, basic dynamics) via `analyze_fast.py`. It is forwarded through the HTTP API via form field or query param. - **`dsp_json_override`** is accepted by the server but ignored. It's a legacy field; don't repurpose it. @@ -288,7 +292,7 @@ A quick map from intent to the right place to start: ## Recent Refactors (don't undo) -- **Backend and frontend monoliths were intentionally split** (commit `5c40dd44`, "refactor: split monoliths into domain modules") into domain modules — on the backend, `analyze.py` coordinates sibling modules `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, and `analyze_fast.py`; on the frontend, focused service files under `apps/ui/src/services/`. The split is the current target shape; resist consolidating it back. +- **Backend and frontend monoliths were intentionally split** into domain modules — on the backend, `analyze.py` coordinates sibling modules `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, and `analyze_fast.py`; on the frontend, focused service files under `apps/ui/src/services/`. The split is the current target shape; resist consolidating it back. - **Hosted runtime foundation landed without disturbing local mode.** `runtime_profile.py`, `worker.py`, `artifact_storage.py`, and `auth_context.py` are the seams. Local-mode code should not branch on profile unless it has to; the boundary handles it. - **UI design-system migration (the "D-series").** A shared primitive layer landed in `apps/ui/src/components/ui/` (with Storybook stories and semantic design tokens in `src/index.css`), and feature components were migrated onto it — inline hex colors were replaced with tokens and bespoke layout boxes with primitives like `DeviceRack`/`SectionHeader`. Build on the primitives and tokens; don't reintroduce one-off styled boxes or raw hex. - **WASM loudness/spectro library (Phase 1).** `packages/loudness-spectro-wasm/` lifts openmeters' BS.1770-5 / EBU R128 loudness, an A-weighted spectrum, and a spectral-reassignment spectrogram into a Rust→WASM package with EBU 3341/3342, `ebur128`, and pyloudnorm conformance layers. It's standalone and **not yet wired into either app** — leave the Essentia loudness path authoritative until integration is proven at parity, and note the product reassigned-spectrogram endpoint runs on librosa, not this package. Don't reimplement this DSP in JS; the package is the home for it. diff --git a/apps/backend/AGENTS.md b/apps/backend/AGENTS.md index 1f4fa35a..1762cfe7 100644 --- a/apps/backend/AGENTS.md +++ b/apps/backend/AGENTS.md @@ -7,7 +7,7 @@ - The repo is a local Python audio-analysis service with two entry points: - `analyze.py`: raw CLI analyzer - `server.py`: FastAPI wrapper around the CLI -- There are no repo-local Cursor rules, `.cursorrules`, or Copilot instruction files in this repo as of 2026-05-27. +- There are no repo-local Cursor rules, `.cursorrules`, or Copilot instruction files in this repo as of 2026-05-30. ## Working Style For Agents @@ -37,8 +37,8 @@ python3.11 -m venv venv - Main runtime dependencies are pinned in `requirements.txt`. - Python `3.12+` is not a supported full-feature local bootstrap target on macOS arm64 because Essentia 2.1b6 wheels are only published for 3.11 on arm64. -- Basic Pitch has been removed. `TorchcrepeBackend` is the canonical Layer 2 backend. PENN was assessed and removed after local benchmarks showed no meaningful win over torchcrepe for ASA's stem-aware note workflow. -- Do not add Basic Pitch or MT3 as production backends in `analyze.py` or `server.py`. If you need to investigate polyphonic full-track transcription, use `polyphonic_evaluation.py` and `scripts/evaluate_polyphonic.py` as a research-only harness. +- Basic Pitch has been removed. `TorchcrepeBackend` is the canonical Layer 2 (monophonic pitch/note) backend. PENN was assessed and removed after local benchmarks showed no meaningful win over torchcrepe for ASA's stem-aware note workflow. +- MT3 polyphonic transcription is now an **optional, opt-in staged backend** (`mt3_transcription.py`, gated by run-level `mt3_mode=enabled` and the env var `ASA_ENABLE_MT3=1` for the legacy CLI path). It is additive — measurement remains authoritative (PURPOSE.md invariant #1). Do not add Basic Pitch as a production backend; use `polyphonic_evaluation.py` and `scripts/evaluate_polyphonic.py` for research-only comparison of other candidates. - If audio/DSP imports fail, check local native dependencies before editing code. ## Main Commands @@ -107,7 +107,8 @@ python3.11 -m venv venv ## File Map - `analyze.py`: CLI entry point. Coordinates the split `analyze_*.py` feature modules and emits the raw JSON. -- `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`: Feature modules (BPM/key/LUFS/stereo, rhythm/melody/groove, segments, structure, transcription, the `--fast` pipeline). Split from the original monolith in commit `5c40dd44` — keep the split when adding features. +- `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`: Feature modules (BPM/key/LUFS/stereo, rhythm/melody/groove, segments, structure, transcription, the `--fast` pipeline). Split from the original `analyze.py` monolith — keep the split when adding features, don't merge them back in. +- `mt3_transcription.py`: Optional polyphonic transcription via Google MT3 (research-grade T5X model). Gated on the env var `ASA_ENABLE_MT3=1` for the legacy CLI path and on run-level `mt3_mode=enabled` for the staged API. Additive only — never overrides measurement (PURPOSE.md invariant #1). Heavy dependency footprint pinned separately in `requirements-mt3.txt`. Driven from `_execute_mt3_attempt`/`_mt3_worker_loop` in `server.py`; the staged-run handler emits per-stem MIDI as artifacts and surfaces an `mt3` namespace per the "Optional MT3 Namespace" section of `JSON_SCHEMA.md`. - `server.py` + `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`: FastAPI app + route modules. Multipart/URL upload handling, subprocess execution, envelope normalization, and the on-demand Phase 3 audition-sample routes. - `analysis_runtime.py`: SQLite-backed run state, stage queue, artifact metadata. - `worker.py`, `runtime_profile.py`, `auth_context.py`, `artifact_storage.py`: Hosted-mode foundation. Local mode shouldn't branch through these unless it has to. @@ -132,6 +133,7 @@ python3.11 -m venv venv - `tests/test_csv_export.py`, `tests/test_sample_*.py`, `tests/test_server_samples.py`: Coverage for CSV export and Phase 3 audition samples. - `tests/test_phase1_golden.py`: golden-snapshot regression gate over measured Phase 1 values (`tests/fixtures/golden/phase1_default.json`). Re-baseline deliberately when a measurement change is intended. - `tests/test_beat_evaluation.py`, `tests/test_loudness_rec_evaluation.py`: unit coverage for the research/eval-only beat and loudness-recommendation harnesses. +- `tests/test_mt3_transcription.py`: unit coverage for the optional MT3 polyphonic backend module. - `ARCHITECTURE.md`: backend responsibilities and request flow. - `JSON_SCHEMA.md`: raw CLI schema plus HTTP mapping notes. diff --git a/apps/backend/ARCHITECTURE.md b/apps/backend/ARCHITECTURE.md index ce765ef5..1a5ed608 100644 --- a/apps/backend/ARCHITECTURE.md +++ b/apps/backend/ARCHITECTURE.md @@ -4,7 +4,8 @@ | Component | Role | | --- | --- | -| `analyze.py` | Raw CLI analyzer entry point. Loads audio, coordinates the `analyze_*.py` feature modules (see [Analyzer Submodules](#analyzer-submodules) below), optionally separates stems and transcribes notes through torchcrepe, then prints JSON to `stdout`. | +| `analyze.py` | Raw CLI analyzer entry point. Loads audio, coordinates the `analyze_*.py` feature modules (see [Analyzer Submodules](#analyzer-submodules) below), optionally separates stems and transcribes notes through torchcrepe, optionally runs MT3 polyphonic transcription (gated by `ASA_ENABLE_MT3=1`), then prints JSON to `stdout`. | +| `mt3_transcription.py` | Optional polyphonic transcription via Google MT3 (T5X). Additive only — never overrides measurement. Gated by `ASA_ENABLE_MT3=1` for the CLI path and by run-level `mt3_mode=enabled` for the staged API. Driven by `_execute_mt3_attempt`/`_mt3_worker_loop` in `server.py`. Optional dependencies in `requirements-mt3.txt`. | | `server.py` + `server_phase1.py` / `server_phase2.py` / `server_upload.py` / `server_samples.py` | FastAPI app and router composition. Accepts uploads, computes estimates, manages the canonical staged run API, normalizes measurement results, serves artifact access, and exposes the on-demand Phase 3 audition-sample routes. | | `analysis_runtime.py` | Run-state persistence and staged-analysis orchestration. Owns run snapshots, stage status, artifact metadata, and ownership checks. | | `artifact_storage.py` | Artifact storage boundary. The current implementation uses the local filesystem, but the runtime now talks to a storage service interface instead of assuming every artifact is a local disk path forever. | @@ -37,7 +38,7 @@ ### Analyzer Submodules -`analyze.py` was split from a monolith in commit `5c40dd44` and now imports from the modules below. Add new measurements in the module that matches the domain, not back into `analyze.py`. +`analyze.py` was split from a monolith and now imports from the modules below. Add new measurements in the module that matches the domain, not back into `analyze.py`. | Module | Domain | | --- | --- | @@ -95,6 +96,7 @@ Custom routes: - `GET /api/analysis-runs/{run_id}/transcription/pianoroll` — velocity-encoded pianoroll matrix derived from the pitch-note translation stage's `transcriptionDetail`. Query params: `mode` (`frame`|`onset`, default `frame`), `pitchLow` (default 21), `pitchHigh` (default 109, exclusive), `tpq` (default 4). Response cites the Phase 1 `bpm` and `timeSignature` so every cell traces back to a measurement. Status codes: 200 (payload), 400 (`INVALID_MODE`/`INVALID_PITCH_RANGE`/`INVALID_TPQ`), 404 (`RUN_NOT_FOUND`/`TRANSCRIPTION_NOT_REQUESTED`/`TRANSCRIPTION_NOT_AVAILABLE`), 409 (`MEASUREMENT_NOT_COMPLETED`/`TRANSCRIPTION_NOT_COMPLETED`). Implementation in [`transcription_pianoroll.py`](transcription_pianoroll.py). - `POST /api/analysis-runs/{run_id}/spectral-enhancements/{kind}` — on-demand spectral artifacts. `kind` is one of `cqt`, `hpss`, `onset`, `chroma_interactive`, or `reassigned` (sharper transient/frequency localization via `librosa.reassigned_spectrogram`). - `POST /api/analysis-runs/{run_id}/pitch-note-translations` +- `POST /api/analysis-runs/{run_id}/mt3-transcriptions` — opt-in MT3 polyphonic transcription, peer of pitch-note translation. Runs only when the create-run request had `mt3_mode='enabled'` (or this route is hit explicitly to re-attempt). Additive — never overrides measurement; emits per-stem MIDI as artifacts. See the "Optional MT3 Namespace" section of [`JSON_SCHEMA.md`](JSON_SCHEMA.md). - `POST /api/analysis-runs/{run_id}/interpretations` - `POST /api/analysis-runs/{run_id}/samples` and `GET /api/analysis-runs/{run_id}/samples` — on-demand Phase 3 audition-sample generation and retrieval. Nothing in the staged-execution loop runs these automatically; the UI POSTs after interpretation completes. See [`server_samples.py`](server_samples.py) and [`docs/SAMPLE_GENERATION.md`](../../docs/SAMPLE_GENERATION.md). - `POST /api/analyze` (legacy compatibility) diff --git a/apps/backend/CHANGELOG.md b/apps/backend/CHANGELOG.md index a437d04e..d7f2847a 100644 --- a/apps/backend/CHANGELOG.md +++ b/apps/backend/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes to `sonic-analyzer` are documented here in reverse chronolog ## Unreleased +- Added an **optional MT3 polyphonic transcription stage** (`mt3_transcription.py`, `requirements-mt3.txt`, `_execute_mt3_attempt`/`_mt3_worker_loop` in `server.py`). Peer of pitch/note translation; gated by env var `ASA_ENABLE_MT3=1` on the legacy CLI and by the run-level form field `mt3_mode='enabled'` on `POST /api/analysis-runs`. Adds `POST /api/analysis-runs/{run_id}/mt3-transcriptions` and an `mt3` namespace on the run snapshot. Emits per-stem MIDI as artifacts. Additive only — never overrides measurement. +- Added the **Live 12 source-extracted catalogue + Phase 2 output-validation gates** (`live12_catalogue.py`, `phase2_catalogue_gates.py`, `data/live12_catalogue.json` + schema, `scripts/build_live12_catalogue.py`). Cross-checks every Phase 2 `{device, parameter, value, phase1Fields}` record against the catalogue and emits **warn-and-keep** `RECOMMENDATION_UNVERIFIED` events on `validationWarnings`. Never drops or rewrites — replaces an earlier fuzzy-rewrite path that landed on the wrong EQ band and wrong A/B curve. +- Added the **transcription pianoroll endpoint** (`transcription_pianoroll.py` backed by `symusic`) at `GET /api/analysis-runs/{run_id}/transcription/pianoroll`. Velocity-encoded `(pitch, time)` uint8 matrix; response cites the Phase 1 `bpm` and `timeSignature` for chain of custody. `transcriptionDetail` now lives in the `pitchNoteTranslation` stage and is stripped from `measurement.result` (see `analysis_runtime.py`). +- Added the **recommendation-proof campaign** offline machinery (`recommendation_evaluation.py`, `scripts/evaluate_recommendations.py`, `scripts/emit_deterministic_recs.ts`, fixtures at `tests/fixtures/recommendation_tracks/`). Grades Phase 2 recs against known-settings tracks with a role/parameter/direction-band rubric and a chain-of-custody penalty that mirrors `phase2Validator.ts`. Off the product path; emits a `--verification-artifact` consumed by the UI badge module. - Added a Phase 1 golden-snapshot regression gate (`tests/test_phase1_golden.py`, fixture `tests/fixtures/golden/phase1_default.json`) that pins measured analyzer values so unintended numeric drift fails CI. - Added a research-only beat/downbeat measurement gate (`beat_evaluation.py` + `beat_report_html.py`, `scripts/evaluate_beats.py`, `scripts/build_beat_manifest.py`) benchmarking CPJKU/beat_this against the shipping kick-accent downbeat heuristic. Off the product path — deleting it restores the product exactly. - Added an eval/test-only loudness-recommendation reachability harness (`loudness_rec_evaluation.py`, `scripts/evaluate_loudness_recs.py`) that re-measures rendered audio to confirm the deterministic subset of a loudness recommendation is physically reachable. Not imported by `analyze.py` or `server.py`. diff --git a/apps/backend/JSON_SCHEMA.md b/apps/backend/JSON_SCHEMA.md index 49953921..d3cfe44f 100644 --- a/apps/backend/JSON_SCHEMA.md +++ b/apps/backend/JSON_SCHEMA.md @@ -17,6 +17,8 @@ Top-level keys: `bpm`, `bpmConfidence`, `bpmPercival`, `bpmAgreement`, `bpmDoubletime`, `bpmSource`, `bpmRawOriginal`, `key`, `keyConfidence`, `keyProfile`, `tuningFrequency`, `tuningCents`, `timeSignature`, `timeSignatureSource`, `timeSignatureConfidence`, `durationSeconds`, `sampleRate`, `lufsIntegrated`, `lufsRange`, `lufsMomentaryMax`, `lufsShortTermMax`, `lufsCurve`, `truePeak`, `crestFactor`, `dynamicSpread`, `monoCompatible`, `plr`, `dynamicCharacter`, `textureCharacter`, `stereoDetail`, `spectralBalance`, `spectralBalanceTimeSeries`, `spectralDetail`, `stemAnalysis`, `transientDensityDetail`, `saturationDetail`, `snareDetail`, `hihatDetail`, `rhythmDetail`, `melodyDetail`, `transcriptionDetail`, `pitchDetail`, `grooveDetail`, `beatsLoudness`, `rhythmTimeline`, `sidechainDetail`, `reverbDetail`, `vocalDetail`, `acidDetail`, `supersawDetail`, `bassDetail`, `kickDetail`, `genreDetail`, `effectsDetail`, `synthesisCharacter`, `danceability`, `structure`, `arrangementDetail`, `segmentLoudness`, `segmentSpectral`, `segmentStereo`, `segmentKey`, `chordDetail`, `perceptual`, `essentiaFeatures`. +**Shared (fast + full) vs full-only.** Fast-mode output is asserted byte-for-byte against the `EXPECTED_TOP_LEVEL_KEYS` set in [`tests/test_analyze.py`](tests/test_analyze.py). Full mode emits those keys *plus* a handful of detail-only fields that are deliberately absent from the shared snapshot: `keyProfile`, `tuningFrequency`, `tuningCents`, `lufsMomentaryMax`, `lufsShortTermMax`, and `pitchDetail`. When changing the schema, update both this list and `EXPECTED_TOP_LEVEL_KEYS`; full-only fields stay out of that set on purpose. See CLAUDE.md tripwire #4. + ## Relationship To `POST /api/analyze` The HTTP success envelope is: diff --git a/apps/backend/README.md b/apps/backend/README.md index 89d4ab8b..a715db97 100644 --- a/apps/backend/README.md +++ b/apps/backend/README.md @@ -119,11 +119,18 @@ PENN was evaluated as an alternative Layer 2 backend and then removed. In plain English: it did not produce a useful quality win over the existing stem-aware torchcrepe path, it was slower in local benchmarks, and it added extra setup cost and first-run model downloads. ASA stays on `torchcrepe-viterbi` for pitch/note translation. -### Polyphonic full-track research spike +### Polyphonic full-track transcription -ASA does not ship a production polyphonic full-track transcription backend. +ASA now ships **optional, opt-in** polyphonic transcription via Google MT3 (T5X). It is gated on: -In plain English: for dense mixed songs, current public models still do not clear the quality bar needed for a reliable producer feature. If you want to compare research candidates anyway, use the offline harness: +- the env var `ASA_ENABLE_MT3=1` for the legacy `analyze.py` CLI path +- the run-level form field `mt3_mode='enabled'` on `POST /api/analysis-runs` for the canonical staged API + +When enabled, the staged `mt3` stage runs as a **peer of pitch/note translation** and emits per-stem MIDI as artifacts plus an `mt3` namespace on the run snapshot. It is **additive only** — measurement remains authoritative (PURPOSE.md invariant #1), and the UI prefers the deterministic torchcrepe note view for Session Musician unless the user explicitly opts into MT3. + +In plain English: full-track polyphonic transcription is available, but off by default. For dense mixed songs the quality varies, so the user (or the operator) has to opt in deliberately, and the result is offered alongside Phase 1 — never instead of it. + +For offline comparison of other research candidates (e.g. `basic-pitch`, alternate MT3 wrappers, future models) without touching the runtime, use the eval harness: ```bash ./venv/bin/python scripts/evaluate_polyphonic.py --manifest /absolute/path/to/polyphonic_manifest.json diff --git a/apps/ui/AGENTS.md b/apps/ui/AGENTS.md index 705755b8..d53cffbb 100644 --- a/apps/ui/AGENTS.md +++ b/apps/ui/AGENTS.md @@ -5,7 +5,7 @@ - This file applies to `apps/ui` inside the `asa` monorepo. - Stack: React 19, TypeScript, Vite 6, Tailwind CSS v4, Vitest, Playwright. - The app talks to the local `sonic-analyzer` backend. Gemini is backend-mediated; the UI does **not** import an AI SDK. -- No repo-local `.cursorrules`, `.cursor/rules/`, or `.github/copilot-instructions.md` exist here as of 2026-05-27. +- No repo-local `.cursorrules`, `.cursor/rules/`, or `.github/copilot-instructions.md` exist here as of 2026-05-30. ## Working Style For Agents @@ -96,6 +96,7 @@ RUN_GEMINI_LIVE_SMOKE=true VITE_ENABLE_PHASE2_GEMINI=true GEMINI_API_KEY=your_ke - `src/services/httpClient.ts`: shared fetch helpers and request-header injection used by the run/artifact/sample clients. - `src/services/spectralArtifactsClient.ts`: spectrogram and spectral-evolution artifact fetches via `/api/analysis-runs/{run_id}/artifacts/…`. - `src/services/transcriptionPianorollClient.ts`: velocity-encoded transcription pianoroll matrix via `/api/analysis-runs/{run_id}/transcription/pianoroll`. Backed by `apps/backend/transcription_pianoroll.py`; rendered in `components/TranscriptionPianoroll.tsx` / `TranscriptionPianorollBlock.tsx`. +- `src/services/mt3Client.ts`: opt-in MT3 polyphonic transcription client. POSTs `/api/analysis-runs/{run_id}/mt3-transcriptions` and polls the run snapshot for the additive `mt3` namespace. Surfaced only when the user opts in; measurement remains authoritative. - `src/services/sampleGenerationClient.ts`: Phase 3 audition-sample POST/GET against `/api/analysis-runs/{run_id}/samples` plus per-clip artifact streaming. - `src/services/audioFile.ts`: client-side audio validation, blank-MIME extension fallback, and preview-URL lifecycle. - `src/services/mixDoctor.ts`: client-side spectral-balance scoring against genre profiles. diff --git a/apps/ui/CHANGELOG.md b/apps/ui/CHANGELOG.md index 5507813a..b5a315ec 100644 --- a/apps/ui/CHANGELOG.md +++ b/apps/ui/CHANGELOG.md @@ -4,6 +4,9 @@ All notable changes to `sonic-analyzer-UI` are documented here in reverse chrono ## Unreleased +- **Optional MT3 polyphonic transcription client.** Added `src/services/mt3Client.ts`. POSTs `/api/analysis-runs/{run_id}/mt3-transcriptions` and reads the additive `mt3` namespace from the run snapshot. Only fires when the user opts in at upload time (form field `mt3_mode='enabled'`); the deterministic Session Musician view remains the default and measurement stays authoritative. +- **Transcription pianoroll UI.** Added `src/services/transcriptionPianorollClient.ts` and the `TranscriptionPianoroll` / `TranscriptionPianorollBlock` components. Velocity-encoded canvas heatmap mounted in the Session Musician suite when `transcriptionDetail.noteCount > 0`. Grounded in the Phase 1 `bpm` + `timeSignature` so every cell traces back to a measurement. +- **Per-recommendation corpus-verification badge** (`GOAL.md` sub-goal 4). Added `src/services/recommendationVerification.ts`, the generated `src/data/recommendationVerification.ts` artifact (from `apps/backend/scripts/evaluate_recommendations.py --verification-artifact`), and `components/RecommendationVerificationBadge.tsx`. Renders the corpus confidence band on each recommendation card; hidden when the corpus has no scored fixtures so the surface degrades gracefully. - **Loudness-action guardrail.** Added `src/services/loudnessGuardrails.ts` (objective loudness defects: digital clipping via `saturationDetail.clippedSampleCount`, true-peak overs via `truePeak`) and a `MISSING_LOUDNESS_ACTION` check (`validateLoudnessActionPresence`) in `phase2Validator.ts` — flags when Phase 2 fails to address a measured loudness defect with a MASTERING/DYNAMICS card. - **Design-system migration ("D-series").** Added a shared primitive layer under `src/components/ui/` (`Button`, `Panel`, `DeviceRack`, `SectionHeader`, `MetricBar`/`MetricBarRow`/`MetricTile`, `DataTable`, `EmptyState`, `LedIndicator`, `Pill`, `SignalChain`, `ChainSeparator`, `TimeReadout`, `Checkbox`, `Tooltip`), each with a `*.stories.tsx`, barrel-exported from `src/components/ui/index.ts` (`variants.ts` for variants, `cn.ts` for class merging). Migrated feature components (FileUpload, Results header + metric strip, Mix Chain, Patches, MeasurementDashboard, Signal Monitor → DeviceRack, SessionMusicianPanel, MixDoctorPanel, SpectrogramViewer, DenseDawConcept) onto the primitives and onto semantic design tokens, replacing inline hex with tokens. Dropped unused `.ableton-{panel,device,header}` styles. - Enabled TypeScript strict mode; added `prefers-reduced-motion` support and exhaustive `switch` handling. diff --git a/apps/ui/README.md b/apps/ui/README.md index 0b9f493f..2c9a3a18 100644 --- a/apps/ui/README.md +++ b/apps/ui/README.md @@ -24,6 +24,7 @@ The app uploads a track to the local DSP backend, shows the estimate and executi - quantize grid and swing controls - browser preview and `.mid` download - transcription pianoroll heatmap (velocity-encoded `pitch × time`) fetched from the backend; rendered when `transcriptionDetail.noteCount > 0` and grounded in the Phase 1 `bpm` + `timeSignature` so each cell traces back to a measurement +- optional MT3 polyphonic transcription view (`mt3Client.ts`, `mt3_mode` form field on the create-run request) — additive only, surfaced alongside the deterministic Session Musician view; measurement remains authoritative - JSON export and markdown report export - Phase 3 audition-sample playback panel — on-demand heuristic WAV/MIDI clips with citation metadata, requested after interpretation completes - chain-of-custody report (`Phase2ConsistencyReport`) that surfaces `phase2Validator` violations directly under the Phase 2 results diff --git a/docs/ARCHITECTURE_STRATEGY.md b/docs/ARCHITECTURE_STRATEGY.md index a7697b16..fd33958e 100644 --- a/docs/ARCHITECTURE_STRATEGY.md +++ b/docs/ARCHITECTURE_STRATEGY.md @@ -146,7 +146,7 @@ The Codex architecture hardening plan (SQLite + job queue + async Phase 2) is th | Done | Backport genreProfiles, abletonDevices, 8 detection services from sonic-architect-app | Grounds Phase 2 Gemini in spectral targets | | Now | Experiment A — gather a broader real producer corpus for torchcrepe validation | Quality work on real target material | | Now | Legacy endpoint removal (`/api/analyze`, `/api/phase2`) | Reduce maintenance surface | -| Next | Polyphonic full-track research spike stays offline-only | Compare `basic-pitch` and optional `MT3` through `apps/backend/scripts/evaluate_polyphonic.py`; do not expose a product backend unless the corpus clears the manual usefulness gates | +| Done | MT3 polyphonic transcription shipped as an **opt-in, additive** staged backend | Gated on run-level `mt3_mode='enabled'` (and `ASA_ENABLE_MT3=1` on the legacy CLI); never overrides measurement. Other candidates still gated behind `scripts/evaluate_polyphonic.py` before any runtime exposure. | | +3 months | Ship Session Musician v2 — whichever path(s) produced usable output, honestly labelled | Product decision based on experiment results | | +6 months | Re-evaluate polyphonic transcription landscape | The field is moving, but new candidates should first go through the offline research harness instead of going straight into the product path. | diff --git a/docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md b/docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md index 928fd053..a7b35e32 100644 --- a/docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md +++ b/docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md @@ -1,25 +1,27 @@ # Polyphonic Full-Track Research Spike -**Status:** Research only. Not a production backend. -**Updated:** March 23, 2026 +**Status:** Offline research harness. The MT3 candidate has since shipped as an **opt-in** product stage; this harness remains the comparison rig for other candidates. +**Updated:** 2026-05-30 ## What This Is -ASA does not currently ship full-track polyphonic audio-to-MIDI as a product feature. +The original spike (March 2026) asked: should ASA ship full-track polyphonic audio-to-MIDI as a product feature, and which candidate? The answer landed in two parts: -In plain English: on dense mixed producer tracks, current public models can generate note data, but they do not reliably generate note data that a producer can trust without heavy cleanup. That makes this a research question, not a product toggle. +- **MT3 was promoted** to an opt-in staged backend after this evaluation (`apps/backend/mt3_transcription.py`, run-level `mt3_mode='enabled'`, env var `ASA_ENABLE_MT3=1` on the legacy CLI). It is additive — measurement remains authoritative (PURPOSE.md invariant #1) — and off by default because dense-mix quality still varies. +- **The offline harness below is preserved** for comparing future candidates (or alternate MT3 wrappers, or `basic-pitch`) against a fixed corpus **without** touching the runtime. New candidates should clear this harness before any consideration of a runtime path. -The repo now includes a separate offline harness for comparing polyphonic candidates on a fixed corpus: +In plain English: MT3 made it from spike to shipping (opt-in, additive only). Everything else still lives behind this offline harness. + +The harness: - module: `apps/backend/polyphonic_evaluation.py` - CLI entry point: `apps/backend/scripts/evaluate_polyphonic.py` -This harness is intentionally **not** wired into: +It is intentionally **not** wired into: -- `apps/backend/analyze.py` -- `apps/backend/server.py` -- the public API -- the UI +- `apps/backend/analyze.py` (the `--transcribe` flag still uses torchcrepe; MT3 is a separate optional pass gated by `ASA_ENABLE_MT3`) +- the canonical product path inside `apps/backend/server.py` for new candidates +- the UI for new candidates ## Current Candidates diff --git a/docs/SAMPLE_GENERATION.md b/docs/SAMPLE_GENERATION.md index b9e63473..43e01caf 100644 --- a/docs/SAMPLE_GENERATION.md +++ b/docs/SAMPLE_GENERATION.md @@ -70,8 +70,8 @@ Each WAV is ≤ 5 seconds, mono or stereo, 44.1 kHz, 16-bit PCM — small enough │ ├── PyTheory if available │ │ └── pure-Python fallback │ │ │ - │ sample_drums.synth_kit() │ - │ └── NumPy oscillators │ + │ sample_drums.synth_kick/snare/ │ + │ hat() └── NumPy oscillators │ │ │ │ sample_synthesis.render_midi() │ │ ├── FluidSynth if available │ diff --git a/docs/SETUP.md b/docs/SETUP.md index a1b7d346..8a399c4d 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -178,8 +178,8 @@ The backend supports two profiles, selected via `SONIC_ANALYZER_RUNTIME_PROFILE` - **`local`** — SQLite + local artifact files + in-process workers. Default for development. - **`hosted`** — Adds auth-context resolution and worker-process separation. The local product path is unaffected. -Implementation record for the hosted foundation: -[`docs/PUBLIC_HOSTING_FOUNDATION.md`](PUBLIC_HOSTING_FOUNDATION.md). +Implementation record for the hosted foundation (archived): +[`docs/history/public-hosting-foundation-2026-04-01.md`](history/public-hosting-foundation-2026-04-01.md). ## Release position diff --git a/docs/history/README.md b/docs/history/README.md index 1d136fd0..5f853d0d 100644 --- a/docs/history/README.md +++ b/docs/history/README.md @@ -19,5 +19,6 @@ Everything in this directory is past-tense. Treat it as a paper trail, not a sou - `external-repo-review-2026-05-13.md` — completed external-repo incorporation review (openmeters / soundscope / Partiels / forever-jukebox). All three tracks resolved. - `track1-spike-outcome-2026-05-13.md` — completed loudness verification spike. Fix shipped; regression test retained. - `audio-analyzer-rs-decision-2026-05-20.md` — completed evaluation/decision on the audio-analyzer-rs path. +- `public-hosting-foundation-2026-04-01.md` — completion record for the hosted runtime foundation (`runtime_profile.py`, `worker.py`, `artifact_storage.py`, `auth_context.py`, plus frontend runtime overrides). The implementation it describes is current code; the "Remaining work before real public hosting" punch list inside is aspirational — re-open items in `BACKLOG.md` if needed. - `field_utilization_report.md` — point-in-time (2026-03-26) snapshot of which Phase 1 fields appear in Phase 2 recommendations. Field names and removal recommendations are stale; see `apps/backend/JSON_SCHEMA.md` for the authoritative field list. - `archive/` — older archived plans and result stubs. diff --git a/docs/history/archive/refactor-state-2026-03-18.md b/docs/history/archive/refactor-state-2026-03-18.md index 2b68c275..df165ea5 100644 --- a/docs/history/archive/refactor-state-2026-03-18.md +++ b/docs/history/archive/refactor-state-2026-03-18.md @@ -1,6 +1,6 @@ # ASA Refactor — Architecture State -> **Archived 2026-05-12.** This is a completion record for the Phase A / B1 / B2 refactor finished on 2026-03-18. The "What's Left" items it lists (artifact TTL, terminology sweep) were not separately tracked — re-open them in `BACKLOG.md` if still needed. For the current architecture see [`docs/ARCHITECTURE_STRATEGY.md`](../ARCHITECTURE_STRATEGY.md) and [`docs/PUBLIC_HOSTING_FOUNDATION.md`](../PUBLIC_HOSTING_FOUNDATION.md). +> **Archived 2026-05-12.** This is a completion record for the Phase A / B1 / B2 refactor finished on 2026-03-18. The "What's Left" items it lists (artifact TTL, terminology sweep) were not separately tracked — re-open them in `BACKLOG.md` if still needed. For the current architecture see [`docs/ARCHITECTURE_STRATEGY.md`](../../ARCHITECTURE_STRATEGY.md) and [`docs/history/public-hosting-foundation-2026-04-01.md`](../public-hosting-foundation-2026-04-01.md). _Last updated: 2026-03-18_ diff --git a/docs/PUBLIC_HOSTING_FOUNDATION.md b/docs/history/public-hosting-foundation-2026-04-01.md similarity index 96% rename from docs/PUBLIC_HOSTING_FOUNDATION.md rename to docs/history/public-hosting-foundation-2026-04-01.md index 20dbd1ff..720f77fd 100644 --- a/docs/PUBLIC_HOSTING_FOUNDATION.md +++ b/docs/history/public-hosting-foundation-2026-04-01.md @@ -1,6 +1,8 @@ # ASA Public Hosting Foundation -_Last updated: 2026-04-01_ +_Archived 2026-05-30 — moved from `docs/` to `docs/history/`. This is a record of completed work, not a living spec. The hosted-mode code itself (`runtime_profile.py`, `worker.py`, `artifact_storage.py`, `auth_context.py`, `apps/ui/src/config.ts` runtime overrides) is current; the "Remaining work before real public hosting" section near the end is the live punch list and lives outside this archived doc._ + +_Last updated (pre-archive): 2026-04-01_ ## Why this work exists diff --git a/incorporations/forking-plans-2026-05-14.md b/incorporations/forking-plans-2026-05-14.md index 8b6e3d7f..9322a0d7 100644 --- a/incorporations/forking-plans-2026-05-14.md +++ b/incorporations/forking-plans-2026-05-14.md @@ -22,6 +22,8 @@ proceeded using the task description as the spec, the actual upstream repos as the authoritative source for license / API / schema, and the prior `external-repo-review-2026-05-13.md` for house style. +> **STATUS UPDATE (2026-05-30):** Plan 6 (jivetalking — Phase 2 prompt-level measurement→parameter heuristics) has **substantially landed** via the Live 12 source-catalogue (`apps/backend/live12_catalogue.py`, `data/live12_catalogue.json`), the warn-and-keep output gates (`apps/backend/phase2_catalogue_gates.py`), and the Phase 2 prompt hardening pass. Plans 4 (audio-analyzer-rs) and 5 (resonators) remain open. Plans 7 / 8 remain re-scoped per the 2026-05-22 correction below and have not been worked. When all five plans are resolved (landed, declined, or rolled into BACKLOG.md), this doc moves to `docs/history/`. +> > **CORRECTION (2026-05-22): "Harmonia" is a phantom — it does not exist.** > A later investigation (4 independent agents + first-hand repo/web > forensics, including an adversarial pass tasked with proving it real) From 08de12acb6540ee6392ee949bac3c3deacb56cc2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 30 May 2026 20:53:41 +0000 Subject: [PATCH 2/2] docs: fix self-contradictory archive banner phrasing The banner said the "Remaining work" section "lives outside this archived doc" but that section is in the doc itself. Aligns with the clearer phrasing already used in docs/history/README.md. https://claude.ai/code/session_016m6cUFNwb3kD2EzAbgqCHe --- docs/history/public-hosting-foundation-2026-04-01.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/history/public-hosting-foundation-2026-04-01.md b/docs/history/public-hosting-foundation-2026-04-01.md index 720f77fd..cc132246 100644 --- a/docs/history/public-hosting-foundation-2026-04-01.md +++ b/docs/history/public-hosting-foundation-2026-04-01.md @@ -1,6 +1,6 @@ # ASA Public Hosting Foundation -_Archived 2026-05-30 — moved from `docs/` to `docs/history/`. This is a record of completed work, not a living spec. The hosted-mode code itself (`runtime_profile.py`, `worker.py`, `artifact_storage.py`, `auth_context.py`, `apps/ui/src/config.ts` runtime overrides) is current; the "Remaining work before real public hosting" section near the end is the live punch list and lives outside this archived doc._ +_Archived 2026-05-30 — moved from `docs/` to `docs/history/`. This is a record of completed work, not a living spec. The hosted-mode code itself (`runtime_profile.py`, `worker.py`, `artifact_storage.py`, `auth_context.py`, `apps/ui/src/config.ts` runtime overrides) is current; the "Remaining work before real public hosting" punch list inside is aspirational — re-open items in `BACKLOG.md` if needed._ _Last updated (pre-archive): 2026-04-01_