Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,8 @@ All eight detectors emit fields visible in [`apps/backend/tests/test_analyze.py`
- ✅ `timeSignatureSource` / `timeSignatureConfidence` — surfaced through HTTP `phase1` via `server.py`.
- ✅ Vibrato display follow-up — `melodyDetail.vibratoExtent` is labeled in cents and present-branch sub-1% confidence renders as `< 1%` rather than `VIBRATO: PRESENT … 0%`.

## Shipped — Phase 3 Audition
- ✅ **Audition samples.** Heuristic WAV/MIDI clips derived from Phase 1 measurements (and Phase 2 context when available) so producers can ear-check the measurement chain. PyTheory generates the musical plan, FluidSynth (with sine-additive fallback) renders audio, NumPy synthesizes drum one-shots, manifest carries citations. **Landed:** [`apps/backend/sample_generation.py`](apps/backend/sample_generation.py), [`sample_theory.py`](apps/backend/sample_theory.py), [`sample_synthesis.py`](apps/backend/sample_synthesis.py), [`sample_drums.py`](apps/backend/sample_drums.py), [`server_samples.py`](apps/backend/server_samples.py); UI [`SamplePlayback.tsx`](apps/ui/src/components/SamplePlayback.tsx) + [`sampleGenerationClient.ts`](apps/ui/src/services/sampleGenerationClient.ts). On-demand endpoints `POST/GET /api/analysis-runs/{run_id}/samples` (not part of the staged-execution queue). Design doc: [`docs/SAMPLE_GENERATION.md`](docs/SAMPLE_GENERATION.md). Adjacent to `patchSmith.ts` but solves a different problem — audition validates the measurements rather than producing a saveable preset.

## Open
- 🔲 `services/patchSmith.ts` — generates Vital/Operator patch parameters from detected features. Slot: Phase 3 (not yet built); unique differentiator (download-ready preset output). Evaluate against `PURPOSE.md` decision framework before scheduling.
12 changes: 10 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ The backend supports staged execution via `analysis_runtime.py`, which persists

Run-oriented endpoints (`/api/analysis-runs*`) are the canonical interface for staged execution. Legacy `POST /api/analyze`, `POST /api/analyze/estimate`, and `POST /api/phase2` remain only as temporary compatibility wrappers — do not build new functionality on them.

Phase 3 audition-sample generation (`POST/GET /api/analysis-runs/{run_id}/samples`) is **on-demand**, not a queue stage — the UI explicitly requests it after interpretation completes. See `server_samples.py` and the `sample_*.py` modules.

Frontend polling: `src/services/analysisRunsClient.ts` creates runs and polls stage snapshots. `src/services/analyzer.ts` orchestrates the create-run + poll loop and projects display payloads.

### Runtime Profiles
Expand All @@ -129,14 +131,20 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths

1. **`analyze.py`**: Pure DSP pipeline entry point. Runs as a subprocess invoked by `server.py`. Coordinates the split feature modules below. **Writes JSON to stdout, diagnostics to stderr** — this contract is load-bearing.
2. **`analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`**: Feature modules. Loadouts: BPM/key/LUFS/stereo/spectral balance, rhythm/melody detail, segment boundaries, transcription. `analyze_fast.py` is the streamlined pipeline used by `--fast`.
3. **`server.py`**: FastAPI app and router composition. Routes are organized into `server_phase1.py`, `server_phase2.py`, `server_upload.py`. Handles multipart uploads, invokes `analyze.py` (or worker), normalizes raw output into the `phase1` HTTP contract.
3. **`server.py`**: FastAPI app and router composition. Routes are organized into `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`. Handles multipart uploads, invokes `analyze.py` (or worker), normalizes raw output into the `phase1` HTTP contract.
4. **`analysis_runtime.py`**: SQLite-backed run state and stage queue management. Run state in `.runtime/analysis_runs.sqlite3`; artifacts in `.runtime/artifacts/`.
5. **`worker.py`**: Dedicated worker-process entry point for hosted-style background stage execution. In `local` profile, work runs in-process; in `hosted` profile, this is the worker role.
6. **`runtime_profile.py`**: Switchboard for `local` vs `hosted` profile and `all` vs `api` vs `worker` process roles (env: `SONIC_ANALYZER_RUNTIME_PROFILE`, `SONIC_ANALYZER_PROCESS_ROLE`).
7. **`artifact_storage.py`**: Storage-service boundary. Today writes to local disk; the interface is designed so callers do not assume disk paths forever.
8. **`auth_context.py`**: Hosted-mode user-context resolution and ownership checks on canonical run routes.
9. **`upload_limits.py`**: Canonical 100 MiB raw-audio / 101 MiB request-envelope limits. Regenerate the operator contract via `scripts/render_upload_limit_contract.py` if numbers change.
10. **`spectral_viz.py`**: Librosa-based spectrogram/time-series artifacts. Called after measurement; failures are non-critical.
11. **`url_ingest.py`**: SSRF-guarded URL-mode ingestion for `POST /api/analysis-runs` — fetches a public `http`/`https` audio URL and feeds it through the same downstream pipeline as a multipart upload.
12. **`csv_export.py`**: CSV exporters for Phase 1 time-series fields, backing `GET /api/analysis-runs/{run_id}/export/csv/{field_path}`. Keeps the route handler a thin lookup-and-serve.
13. **`stage_status.py`**: Collapses the eight internal stage statuses into the additive client-facing `publicStatus` field on every stage snapshot.
14. **`server_samples.py` + `sample_generation.py`, `sample_theory.py`, `sample_synthesis.py`, `sample_drums.py`**: Phase 3 audition-sample generation. `sample_theory.py` builds the PyTheory musical plan, `sample_synthesis.py` renders audio (FluidSynth with sine-additive fallback), `sample_drums.py` synthesizes drum one-shots, `sample_generation.py` orchestrates and emits the citation manifest. On-demand only.
15. **`dsp_bandbank.py` + `dsp_utils.py`**: Shared DSP primitives — `BatchedBandpass` (4th-order Butterworth bandpass bank) and cross-module utility functions.
16. **`phase1_evaluation.py` + `phase1_report_html.py`, `polyphonic_evaluation.py`**: Offline evaluation harnesses (deterministic-metric / detector-stability reporting and research-only polyphonic transcription). Not on the product path; driven by `scripts/evaluate_*.py`.

The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details.

Expand Down Expand Up @@ -236,7 +244,7 @@ A quick map from intent to the right place to start:

## Backport Candidates

Most of the original `sonic-architect-app` port (genre profiles, Ableton device mappings, mix doctor, eight detectors) is **shipped**. The remaining open item in [`BACKLOG.md`](BACKLOG.md) is `patchSmith.ts` (Phase 3 synth-patch generation). Consult `BACKLOG.md` before re-implementing genre detection, mix analysis, acid/reverb/vocal/supersaw/bass/kick detection — they're already in [`apps/backend/analyze_detection.py`](apps/backend/analyze_detection.py) and emit fields visible in `EXPECTED_TOP_LEVEL_KEYS`.
Most of the original `sonic-architect-app` port (genre profiles, Ableton device mappings, mix doctor, eight detectors) is **shipped**, as is Phase 3 audition-sample generation. The remaining open item in [`BACKLOG.md`](BACKLOG.md) is `patchSmith.ts` (Phase 3 synth-patch generation — distinct from audition samples, which validate measurements rather than producing a saveable preset). Consult `BACKLOG.md` before re-implementing genre detection, mix analysis, acid/reverb/vocal/supersaw/bass/kick detection — they're already in [`apps/backend/analyze_detection.py`](apps/backend/analyze_detection.py) and emit fields visible in `EXPECTED_TOP_LEVEL_KEYS`.

## Companion Agent Docs

Expand Down
8 changes: 7 additions & 1 deletion apps/backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,11 +108,17 @@ python3.11 -m venv venv

- `analyze.py`: CLI entry point. Coordinates the split `analyze_*.py` feature modules and emits the raw JSON.
- `analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`: Feature modules (BPM/key/LUFS/stereo, rhythm/melody/groove, segments, structure, transcription, the `--fast` pipeline). Split from the original monolith in commit `5c40dd44` — keep the split when adding features.
- `server.py` + `server_phase1.py`, `server_phase2.py`, `server_upload.py`: FastAPI app + route modules. Multipart upload handling, subprocess execution, envelope normalization.
- `server.py` + `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`: FastAPI app + route modules. Multipart/URL upload handling, subprocess execution, envelope normalization, and the on-demand Phase 3 audition-sample routes.
- `analysis_runtime.py`: SQLite-backed run state, stage queue, artifact metadata.
- `worker.py`, `runtime_profile.py`, `auth_context.py`, `artifact_storage.py`: Hosted-mode foundation. Local mode shouldn't branch through these unless it has to.
- `upload_limits.py`: Canonical 100 MiB raw-audio / 101 MiB request-envelope limits. Operator contract is generated, not hand-edited.
- `url_ingest.py`: SSRF-guarded URL-mode ingestion for `POST /api/analysis-runs`.
- `csv_export.py`: CSV exporters for Phase 1 time-series fields; backs `GET /api/analysis-runs/{run_id}/export/csv/{field_path}`.
- `stage_status.py`: Collapses the eight internal stage statuses into the additive client-facing `publicStatus` field.
- `sample_generation.py`, `sample_theory.py`, `sample_synthesis.py`, `sample_drums.py`: Phase 3 audition-sample generation — PyTheory plan, FluidSynth/sine-additive render, NumPy drum one-shots, citation manifest. On-demand only.
- `dsp_bandbank.py`, `dsp_utils.py`: Shared DSP primitives — `BatchedBandpass` Butterworth bank and cross-module utilities.
- `spectral_viz.py`: Librosa spectrogram and spectral time-series artifacts. Non-critical — failures don't break a run.
- `phase1_evaluation.py` + `phase1_report_html.py`: Offline Phase 1 evaluation harness and HTML render. Not on the product path; driven by `scripts/evaluate_phase1.py`.
- `polyphonic_evaluation.py` + `scripts/evaluate_polyphonic.py`: **Research-only.** Offline polyphonic-transcription evaluation harness, not part of the shipped product path.
- `tests/test_server.py`: OpenAPI and envelope contract tests.
- `tests/test_analyze.py`: generated WAV fixture, `EXPECTED_TOP_LEVEL_KEYS` snapshot, raw payload assertions.
Expand Down
11 changes: 9 additions & 2 deletions apps/backend/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,20 @@
| Component | Role |
| --- | --- |
| `analyze.py` | Raw CLI analyzer entry point. Loads audio, coordinates the `analyze_*.py` feature modules (see [Analyzer Submodules](#analyzer-submodules) below), optionally separates stems and transcribes notes through torchcrepe, then prints JSON to `stdout`. |
| `server.py` + `server_phase1.py` / `server_phase2.py` / `server_upload.py` | FastAPI app and router composition. Accepts uploads, computes estimates, manages the canonical staged run API, normalizes measurement results, and serves artifact access. |
| `server.py` + `server_phase1.py` / `server_phase2.py` / `server_upload.py` / `server_samples.py` | FastAPI app and router composition. Accepts uploads, computes estimates, manages the canonical staged run API, normalizes measurement results, serves artifact access, and exposes the on-demand Phase 3 audition-sample routes. |
| `analysis_runtime.py` | Run-state persistence and staged-analysis orchestration. Owns run snapshots, stage status, artifact metadata, and ownership checks. |
| `artifact_storage.py` | Artifact storage boundary. The current implementation uses the local filesystem, but the runtime now talks to a storage service interface instead of assuming every artifact is a local disk path forever. |
| `runtime_profile.py` | Runtime/profile switchboard for `local` vs `hosted` behavior and `all` vs `api` vs `worker` process roles. |
| `auth_context.py` | Hosted-mode user-context resolution. Establishes the current run owner in the canonical API path. |
| `worker.py` | Dedicated worker-process entry point for hosted-style background stage execution. |
| `upload_limits.py` | Canonical raw-audio (100 MiB) and request-envelope (101 MiB) limits, plus the protected-route list. Operator contract is generated, not hand-edited — see `scripts/render_upload_limit_contract.py`. |
| `spectral_viz.py` | Librosa-based spectrogram generation and spectral time-series extraction. Produces mel/chroma PNG spectrograms and per-frame spectral evolution JSON. Called after successful measurement; failures are non-critical. |
| `url_ingest.py` | SSRF-guarded URL-mode ingestion for `POST /api/analysis-runs`. Fetches a public `http`/`https` audio file and feeds the bytes through the same downstream pipeline as a multipart upload. |
| `csv_export.py` | CSV exporters for Phase 1 time-series fields, keyed by dotted JSON path. Backs `GET /api/analysis-runs/{run_id}/export/csv/{field_path}` and keeps the route handler a thin lookup-and-serve. |
| `stage_status.py` | Collapses the eight internal stage statuses into the additive client-facing `publicStatus` field carried on every stage in the run snapshot. |
| `server_samples.py` + `sample_generation.py` / `sample_theory.py` / `sample_synthesis.py` / `sample_drums.py` | Phase 3 audition-sample generation. PyTheory musical plan, FluidSynth (sine-additive fallback) audio render, NumPy drum one-shots, and a citation manifest tying every clip back to a Phase 1 field. On-demand only — not part of the staged-execution queue. See [`docs/SAMPLE_GENERATION.md`](../../docs/SAMPLE_GENERATION.md). |
| `dsp_bandbank.py` + `dsp_utils.py` | Shared DSP primitives: `BatchedBandpass` (4th-order Butterworth bandpass bank with zero-phase `filtfilt`) and cross-module utility functions. |
| `phase1_evaluation.py` + `phase1_report_html.py` | Offline Phase 1 evaluation harness — deterministic-metric and detector-stability reporting, with a standalone HTML render. Not on the product path; driven by `scripts/evaluate_phase1.py`. |
| `polyphonic_evaluation.py` + `scripts/evaluate_polyphonic.py` | Research-only offline polyphonic-transcription evaluation harness. Not on the product path. |
| `tests/test_server.py` | Contract tests for estimate, timeout, and success envelopes. |
| `tests/test_analyze.py` | Structural snapshot tests for the raw analyzer JSON output. Owns `EXPECTED_TOP_LEVEL_KEYS` — update it whenever you add a root field. |
Expand Down Expand Up @@ -74,9 +80,10 @@ Custom routes:
- `GET /api/analysis-runs/{run_id}/artifacts` and `…/artifacts/{artifact_id}`
- `GET /api/analysis-runs/{run_id}/source-audio` — re-serves the original ingested audio for the run. Owner-only (no admin bypass). Saves a round-trip vs looking up the source artifact id first.
- `GET /api/analysis-runs/{run_id}/export/csv/{field_path}` — CSV export of a Phase 1 time-series field. See [`docs/adr/0001-phase1-json-schema-v1.md`](../../docs/adr/0001-phase1-json-schema-v1.md) and the registry in [`csv_export.py`](csv_export.py).
- `POST /api/analysis-runs/{run_id}/spectral-enhancements/{kind}`
- `POST /api/analysis-runs/{run_id}/spectral-enhancements/{kind}` — on-demand spectral artifacts. `kind` is one of `cqt`, `hpss`, `onset`, `chroma_interactive`, or `reassigned` (sharper transient/frequency localization via `librosa.reassigned_spectrogram`).
- `POST /api/analysis-runs/{run_id}/pitch-note-translations`
- `POST /api/analysis-runs/{run_id}/interpretations`
- `POST /api/analysis-runs/{run_id}/samples` and `GET /api/analysis-runs/{run_id}/samples` — on-demand Phase 3 audition-sample generation and retrieval. Nothing in the staged-execution loop runs these automatically; the UI POSTs after interpretation completes. See [`server_samples.py`](server_samples.py) and [`docs/SAMPLE_GENERATION.md`](../../docs/SAMPLE_GENERATION.md).
- `POST /api/analyze` (legacy compatibility)
- `POST /api/analyze/estimate` (legacy compatibility)
- `POST /api/phase2` (legacy compatibility)
Expand Down
13 changes: 11 additions & 2 deletions apps/backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,16 @@ This repo contains two entry points:
Canonical live-analysis routes:

- `POST /api/analysis-runs/estimate`
- `POST /api/analysis-runs`
- `POST /api/analysis-runs` — multipart upload or SSRF-guarded URL ingestion
- `GET /api/analysis-runs/{run_id}`
- `DELETE /api/analysis-runs/{run_id}` — owner delete; operator `X-Admin-Key` bypass when `SONIC_ANALYZER_ADMIN_KEY` is set
- `GET /api/analysis-runs/{run_id}/artifacts...`
- `GET /api/analysis-runs/{run_id}/source-audio`
- `GET /api/analysis-runs/{run_id}/export/csv/{field_path}`
- `POST /api/analysis-runs/{run_id}/spectral-enhancements/{kind}`
- `POST /api/analysis-runs/{run_id}/pitch-note-translations`
- `POST /api/analysis-runs/{run_id}/interpretations`
- `POST /api/analysis-runs/{run_id}/samples` and `GET /api/analysis-runs/{run_id}/samples` — on-demand Phase 3 audition samples

Legacy compatibility routes:

Expand All @@ -34,7 +41,9 @@ FastAPI also serves the usual generated endpoints at `/openapi.json`, `/docs`, a
- NumPy
- Demucs
- torchcrepe (pitch/note translation)
- mido
- librosa (spectrogram / spectral time-series artifacts)
- mido / pretty_midi
- pytheory + pyfluidsynth (Phase 3 audition-sample generation)
- FastAPI
- Uvicorn

Expand Down
Loading