Skip to content
14 changes: 9 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ npm run verify # lint + test:unit + build + test:smoke (ful
npm run lint # TypeScript type-check only (no ESLint/Prettier)
npm test # All Vitest tests (vitest.config.ts include: tests/**/*.test.ts)
npm run test:unit # Vitest, restricted to tests/services/ (the unit subset)
npx playwright install chromium # first-time Playwright setup (run once from apps/ui/)
npm run test:smoke # Playwright smoke suite (tests/smoke/, default config)
npm run test:smoke:live-gemini # Playwright smoke against the real Gemini Files API (tests/smoke/upload-phase2-live-gemini.spec.ts)
npm run test:e2e # Playwright full e2e suite (playwright.full.config.ts)
Expand Down Expand Up @@ -186,7 +187,7 @@ Operational and one-shot scripts live in two places. They are not on the request
1. `dev.sh` — full-stack dev launcher (covered above). Covered by `apps/backend/tests/test_root_dev_script.py`.
2. `test-e2e.sh` / `test-e2e-integration.sh` — e2e harnesses (covered above). Covered by `apps/backend/tests/test_root_e2e_script.py`.
3. `calibrate_confidence.py` — threshold-sweep harness for the pitch / chord / sidechain detectors. Research-only. Its unit test lives in `scripts/tests/test_calibrate_confidence.py`.
4. `build_live12_catalogue.py` — regenerates `data/live12_catalogue.json` (the source-extracted Live 12 device/parameter catalogue) from the upstream `gluon/AbletonLive12_MIDIRemoteScripts` checkout. Static AST extraction; carries no proprietary code. Re-run when the upstream commit shifts or the schema bumps; the published `data/live12_catalogue.schema.json` validates the output.
4. `build_live12_catalogue.py` — regenerates `data/live12_catalogue.json` (the source-extracted Live 12 device/parameter catalogue) from the upstream `gluon/AbletonLive12_MIDIRemoteScripts` checkout. Static AST extraction; carries no proprietary code. Re-run when the upstream commit shifts or the schema bumps; the published `data/live12_catalogue.schema.json` validates the output. Invocation: `scripts/build_live12_catalogue.py --source /path/to/gluon/AbletonLive12_MIDIRemoteScripts --output data/live12_catalogue.json` (or set `SONIC_ANALYZER_LIVE12_SOURCE` env var to omit `--source`).

`apps/backend/scripts/`:
1. `bootstrap.sh` — recreate the Python 3.11 venv (covered above). `dev.sh` here is a thin shim that `exec`s the repo-root `scripts/dev.sh` full-stack launcher. `bin/asa` script contracts (install, bootstrap, cleanup) are covered by `apps/backend/tests/test_bootstrap_scripts.py`.
Expand Down Expand Up @@ -271,7 +272,7 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths
1. **`analyze.py`**: Pure DSP pipeline entry point. Runs as a subprocess invoked by `server.py`. Coordinates the split feature modules below. **Writes JSON to stdout, diagnostics to stderr** — this contract is load-bearing.
2. **`analyze_core.py`, `analyze_audio_io.py`, `analyze_detection.py`, `analyze_estimate.py`, `analyze_rhythm.py`, `analyze_segments.py`, `analyze_structure.py`, `analyze_transcription.py`, `analyze_fast.py`**: Feature modules. Loadouts: BPM/key/LUFS/stereo/spectral balance, rhythm/melody detail, segment boundaries, transcription. `analyze_fast.py` is the streamlined pipeline used by `--fast`.
3. **`server.py`**: FastAPI app and router composition. Routes are organized into `server_phase1.py`, `server_phase2.py`, `server_upload.py`, `server_samples.py`. Handles multipart uploads, invokes `analyze.py` (or worker), normalizes raw output into the `phase1` HTTP contract.
4. **`analysis_runtime.py`**: SQLite-backed run state and stage queue management. Run state in `.runtime/analysis_runs.sqlite3`; artifacts in `.runtime/artifacts/`.
4. **`analysis_runtime.py`**: SQLite-backed run state and stage queue management. Run state in `.runtime/analysis_runs.sqlite3`; artifacts in `.runtime/artifacts/`. New artifact-producing code calls `record_artifact` — do not write `.runtime/` paths directly (see also Tripwire #6 and `artifact_storage.py`).
5. **`worker.py`**: Dedicated worker-process entry point for hosted-style background stage execution. In `local` profile, work runs in-process; in `hosted` profile, this is the worker role.
6. **`runtime_profile.py`**: Switchboard for `local` vs `hosted` profile and `all` vs `api` vs `worker` process roles (env: `SONIC_ANALYZER_RUNTIME_PROFILE`, `SONIC_ANALYZER_PROCESS_ROLE`).
7. **`artifact_storage.py`**: Storage-service boundary. Today writes to local disk; the interface is designed so callers do not assume disk paths forever.
Expand All @@ -295,7 +296,7 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths

23. **`recommendations_contract.py` + `schemas/recommendations.v1.schema.json`**: Frozen, versioned Phase 2 recommendation contract (ADR 0003). `project_recommendations` normalizes the three Phase 2 device-card arrays (`abletonRecommendations`, `mixAndMasterChain`, `secretSauce.workflowSteps`) into a flat `{device, parameter, value, unit, range, cited_measurements[]}` envelope (`version: "recommendations.v1"`); `validate_envelope` checks it against the committed JSON Schema via `jsonschema` (the real file, not a hand-rolled mirror — see ADR 0001's drift warning). Derived and additive — it never overrides Phase 1 (invariant #1) and admits ONLY cited cards (`cited_measurements.minItems: 1`); uncited cards stay in the raw arrays where the warn-and-keep catalogue gate flags them. `server.py` attaches the validated envelope to the producer_summary interpretation result as a `recommendations` field (degrades to absent on error). TS mirror: `RecommendationsContract` in [src/types/interpretation.ts](apps/ui/src/types/interpretation.ts). CI gate: `tests/test_recommendations_contract.py` (schema validity + projection-validates + round-trip + freeze).

24. **`phase2_provider.py` + `moss_sidecar/`**: Selectable Phase 2 interpretation provider (default-off experiment). `ASA_PHASE2_PROVIDER=moss` routes the producer_summary interpretation to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar instead of Gemini, while both paths flow through the identical parse/citation/catalogue validators. STEP ONE licence gate: MOSS-Audio *weights* are Apache-2.0 but the *modeling code* has no effective licence, so the sidecar ships no OpenMOSS code and the real-model path is a 501 stub — research-only, not promotable. `moss_sidecar/app.py` is the FastAPI sidecar; `moss_sidecar/mock_interpreter.py` generates deterministic schema-valid Phase-1-grounded output for the default `ASA_MOSS_SIDECAR_MODE=mock` path. See `docs/PHASE2_PROVIDER.md` for the full design rationale and licence analysis.
24. **`phase2_provider.py` + `moss_sidecar/`**: Selectable Phase 2 interpretation provider (default-off experiment). `ASA_PHASE2_PROVIDER=moss` routes the producer_summary interpretation to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar instead of Gemini; `ASA_PHASE2_PROVIDER=claude` routes it to `ClaudeCliProvider` — a text-only interpreter that runs the local Claude Code CLI headless (sandboxed: `--safe-mode`, `--tools ""`, `--no-session-persistence`, response schema enforced via `--json-schema`), grounds purely on the prompt's embedded Phase 1 JSON, and needs no `GEMINI_API_KEY`. All paths flow through the identical parse/citation/catalogue validators. STEP ONE licence gate: MOSS-Audio *weights* are Apache-2.0 but the *modeling code* has no effective licence, so the sidecar ships no OpenMOSS code and the real-model path is a 501 stub — research-only, not promotable. `moss_sidecar/app.py` is the FastAPI sidecar; `moss_sidecar/mock_interpreter.py` generates deterministic schema-valid Phase-1-grounded output for the default `ASA_MOSS_SIDECAR_MODE=mock` path. See `docs/PHASE2_PROVIDER.md` for the full design rationale and licence analysis.

The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details.

Expand Down Expand Up @@ -367,11 +368,14 @@ ASA_MSST_ROOT="" # path to the MSST-WebUI checkout (adde
ASA_MSST_MODEL="scnet_4stem" # MSST model registry id in separation_backend._MSST_MODEL_REGISTRY. Default `scnet_4stem` (4-stem parity). `bs_roformer_vocals` is research/A-B-only (2-stem, leaves bass/drums empty). Unknown ids fall back to the default.
ASA_MSST_MODEL_DIR="" # optional config/checkpoint root (MSST configs/ + pretrain/ layout); defaults to ASA_MSST_ROOT.
ASA_MSST_DEVICE="" # optional device for MSST inference (auto|cpu|cuda|mps); defaults to auto.
ASA_PHASE2_PROVIDER="gemini" # Phase 2 interpretation provider: `gemini` (default, product path, unchanged) or `moss` to route the producer_summary interpretation to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar. Default-off experiment; both providers flow through the identical parse/citation/catalogue validators. STEP ONE licence gate: MOSS-Audio *weights* are Apache-2.0 but the *modeling code* has no effective licence (the GitHub repo's pyproject points at a non-existent LICENSE), so the sidecar ships no OpenMOSS code and the real-model path is a 501 stub — research-only, not promotable. See apps/backend/phase2_provider.py + docs/PHASE2_PROVIDER.md.
ASA_PHASE2_PROVIDER="gemini" # Phase 2 interpretation provider: `gemini` (default, product path, unchanged), `moss` to route the producer_summary interpretation to a self-hosted OpenMOSS MOSS-Audio FastAPI sidecar, or `claude` to run the local Claude Code CLI headless as a TEXT-ONLY interpreter (no audio sent; grounds on the prompt's embedded Phase 1 JSON; needs no GEMINI_API_KEY — rides the operator's existing Claude Code login). Default-off experiments; all providers flow through the identical parse/citation/catalogue validators. MOSS STEP ONE licence gate: MOSS-Audio *weights* are Apache-2.0 but the *modeling code* has no effective licence (the GitHub repo's pyproject points at a non-existent LICENSE), so the sidecar ships no OpenMOSS code and the real-model path is a 501 stub — research-only, not promotable. See apps/backend/phase2_provider.py + docs/PHASE2_PROVIDER.md.
ASA_MOSS_SIDECAR_URL="http://127.0.0.1:8200" # base URL of the MOSS sidecar (apps/backend/moss_sidecar/app.py) when ASA_PHASE2_PROVIDER=moss.
ASA_MOSS_SIDECAR_TIMEOUT_SECONDS="180" # per-request timeout for the MOSS sidecar call.
ASA_MOSS_MODEL_ID="" # optional MOSS model id forwarded to the sidecar; defaults to the run's model_name.
ASA_MOSS_SIDECAR_MODE="mock" # sidecar-side (its own venv): `mock` (default, deterministic schema-valid Phase-1-grounded output) or `model` (a 501 licence-gated stub that executes no OpenMOSS code).
ASA_CLAUDE_CLI="claude" # path to the Claude Code CLI binary used when ASA_PHASE2_PROVIDER=claude; defaults to `claude` on PATH.
ASA_CLAUDE_MODEL="" # optional model override for the claude provider (e.g. `sonnet`); empty = the CLI's default model. The subprocess runs sandboxed: --safe-mode, --tools "", --no-session-persistence, schema enforced via --json-schema.
ASA_CLAUDE_TIMEOUT_SECONDS="600" # per-call timeout for the claude provider subprocess (a full producer_summary prompt on the CLI default model measures ~6 min).
```

Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-only. `SONIC_ANALYZER_ADMIN_KEY` is backend-only and never exposed to clients.
Expand All @@ -394,7 +398,7 @@ Things that look like normal code changes but silently break the contract. Most
1. **`print(...)` in [analyze.py](apps/backend/analyze.py) without `file=sys.stderr`.** Stdout is the JSON contract. Any stray print corrupts it and the server reports a parse error with no useful trace. The existing code is consistent about this — match the pattern (`print(f"[warn] ...", file=sys.stderr)`).
2. **Calling `analyze.py` as a subprocess without `--yes`.** The CLI prompts for confirmation when stdin is a TTY. Subprocess invocations must pass `--yes` or hang waiting for input. `server.py` already does; new callers must too.
3. **Renaming a field on only one side.** Python emits *camelCase* JSON directly (`bpmConfidence`, not `bpm_confidence`) — there is no conversion layer. A rename in [analyze.py](apps/backend/analyze.py) without a matching update in [src/types.ts](apps/ui/src/types.ts) is undetectable by either type system; the field just disappears from the UI.
4. **Adding a top-level key without updating `EXPECTED_TOP_LEVEL_KEYS`.** [tests/test_analyze.py](apps/backend/tests/test_analyze.py) holds a snapshot of every root key. New fields require updating that set *and* [JSON_SCHEMA.md](apps/backend/JSON_SCHEMA.md). The same test enforces that `--fast` only populates `FAST_MODE_POPULATED_FIELDS`. A change to *measured values* (not just keys) can also trip the golden-snapshot regression gate in [tests/test_phase1_golden.py](apps/backend/tests/test_phase1_golden.py) (fixture `tests/fixtures/golden/phase1_default.json`) — re-baseline it deliberately, never blindly.
4. **Adding a top-level key without updating `EXPECTED_TOP_LEVEL_KEYS`.** [tests/test_analyze.py](apps/backend/tests/test_analyze.py) holds a snapshot of every root key. New fields require updating that set *and* [JSON_SCHEMA.md](apps/backend/JSON_SCHEMA.md). The same test enforces that `--fast` only populates `FAST_MODE_POPULATED_FIELDS`. A change to *measured values* (not just keys) can also trip the golden-snapshot regression gate in [tests/test_phase1_golden.py](apps/backend/tests/test_phase1_golden.py) (fixture `tests/fixtures/golden/phase1_default.json`) — re-baseline it deliberately, never blindly (from `apps/backend/`): `UPDATE_PHASE1_GOLDEN=1 ./venv/bin/python -m unittest tests.test_phase1_golden`
5. **Using `document` or `window` in `tests/services/`.** Vitest runs in `node`, not `jsdom`. Service-layer tests are pure logic; if you need DOM, it's a Playwright test in `tests/smoke/` instead.
6. **Hard-coding `Path(...)` for artifacts in new code.** Artifact access must go through [artifact_storage.py](apps/backend/artifact_storage.py). Direct paths work in `local` profile and break silently in `hosted`.
7. **Editing `apps/ui/.env` and expecting `dev.sh` to honor it.** `dev.sh` reads `apps/ui/.env` but *overrides* `VITE_API_BASE_URL` for the spawned UI process so stale `.env` files don't break the stack. To point the UI at a non-canonical backend, edit `dev.sh` or run the UI directly with the env var on the command line.
Expand Down
4 changes: 2 additions & 2 deletions apps/backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ python3.11 -m venv venv
- Run one test module:

```bash
./venv/bin/python -m unittest tests/test_server.py
./venv/bin/python -m unittest tests/test_analyze.py
./venv/bin/python -m unittest tests.test_server
./venv/bin/python -m unittest tests.test_analyze
```

- Run one test class:
Expand Down
Loading
Loading