diff --git a/.gitignore b/.gitignore index f6ae1e5f..987a3013 100644 --- a/.gitignore +++ b/.gitignore @@ -31,6 +31,11 @@ apps/backend/.env # Backend runtime state (artifacts, sqlite, reports) apps/backend/.runtime/ +.runtime/ + +# Generated/binary design docs kept locally, not in git +ASA_System_Design.docx +docs/PHASE2_TRUTHFULNESS_PHASES_A_B_C.docx # UI prototypes apps/ui/redesign-concept.html diff --git a/.runtime/analysis_runs.sqlite3 b/.runtime/analysis_runs.sqlite3 deleted file mode 100644 index e69de29b..00000000 diff --git a/AGENTS.md b/AGENTS.md index a224df50..61eb9713 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,512 +1,19 @@ # AGENTS.md -This file provides guidance to AI coding agents working with the ASA (Sonic Analyzer) codebase. +Pointer for AI coding agents that look for `AGENTS.md` by convention (Codex, +OpenHands, and others). -Codex-specific instructions live in `CODEX.md` at the repo root, with app overlays in `apps/ui/CODEX.md` and `apps/backend/CODEX.md`. +**The canonical agent guidance for this repo is [`CLAUDE.md`](CLAUDE.md).** Read it first. -## Project Overview +Order of precedence when guidance conflicts: -ASA is a local audio analysis tool for music producers. It analyzes audio files to extract measurable properties (tempo, key, loudness, spectral characteristics) and provides AI-powered interpretation for arrangement advice and musical descriptions. +1. [`PURPOSE.md`](PURPOSE.md) — why ASA exists and the non-negotiable quality invariants. +2. [`CLAUDE.md`](CLAUDE.md) — canonical agent guide: commands, architecture, tripwires, change map. +3. [`docs/ARCHITECTURE_STRATEGY.md`](docs/ARCHITECTURE_STRATEGY.md) — why the three-layer design is shaped the way it is. -### Core Philosophy +App-local entry points: -The system follows a **three-layer hybrid architecture** that separates deterministic measurement from AI interpretation: +- [`apps/backend/AGENTS.md`](apps/backend/AGENTS.md) +- [`apps/ui/AGENTS.md`](apps/ui/AGENTS.md) -1. **Measurement is authoritative** - DSP results from Essentia are the system's ground truth -2. **Pitch/note translation is best-effort** - Monophonic pitch tracking on separated stems, honest about uncertainty -3. **Interpretation is contextual** - Gemini provides musical insights grounded in measurements, not replacements for them - -This split exists because frontier audio-language models (as of early 2026) still degrade on measurement tasks like BPM estimation and key detection. The hybrid approach leverages the strengths of each layer. - -In plain English: ASA can give you a usable pitch-note sketch and a melody guide, but it does not currently promise reliable full-track polyphonic audio-to-MIDI from dense mixed songs. - -### System Architecture - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ LAYER 1 — MEASUREMENT (Essentia/DSP) │ -│ Deterministic, repeatable, authoritative │ -│ BPM, LUFS, key, spectral balance, stereo, dynamics │ -└─────────────────────────────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────────────────────────────┐ -│ LAYER 2 — PITCH/NOTE TRANSLATION (torchcrepe) │ -│ Best-effort monophonic pitch on Demucs stems │ -│ Bass + Other stems → MIDI notes with confidence │ -└─────────────────────────────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────────────────────────────────┐ -│ LAYER 3 — INTERPRETATION (Gemini) │ -│ Grounded by Layer 1 measurements │ -│ Arrangement advice, device mappings, musical descriptions │ -└─────────────────────────────────────────────────────────────────┘ -``` - -## Technology Stack - -### Frontend (`apps/ui`) - -| Technology | Version | Purpose | -|------------|---------|---------| -| React | 19 | UI framework | -| TypeScript | 5.8 | Type safety | -| Vite | 6 | Build tool and dev server | -| Tailwind CSS | 4.1.14 | Styling with semantic tokens | -| WaveSurfer.js | 7.12.1 | Audio waveform visualization | -| midi-writer-js | 3.2.1 | MIDI file export | -| Vitest | 4.0.18 | Unit testing | -| Playwright | 1.58.2 | E2E/smoke testing | - -### Backend (`apps/backend`) - -| Technology | Version | Purpose | -|------------|---------|---------| -| Python | 3.11.x | Runtime (3.12+ not supported for full setup) | -| FastAPI | 0.135.1 | HTTP API framework | -| Uvicorn | 0.41.0 | ASGI server | -| Essentia | 2.1b6.dev1389 | DSP analysis library | -| Demucs | 4.0.1 | Source separation | -| PyTorch | 2.10.0 | Deep learning backend | -| Google GenAI | 1.14.0+ | Gemini API client | -| SQLite | (builtin) | Run state persistence | - -### Development Tools - -- **Node.js**: 20+ for frontend -- **Python**: 3.11.x specifically for backend -- **Bash**: For orchestration scripts - -## Project Structure - -``` -asa/ -├── apps/ -│ ├── ui/ # React frontend -│ │ ├── src/ -│ │ │ ├── components/ # React components -│ │ │ ├── services/ # API clients, business logic -│ │ │ ├── hooks/ # Custom React hooks -│ │ │ ├── utils/ # Utility functions -│ │ │ ├── types.ts # Shared TypeScript types -│ │ │ └── config.ts # App configuration -│ │ ├── tests/ -│ │ │ ├── services/ # Vitest unit tests -│ │ │ └── smoke/ # Playwright E2E tests -│ │ ├── package.json -│ │ └── tsconfig.json -│ └── backend/ # Python backend -│ ├── analyze.py # CLI entry; coordinates analyze_*.py modules -│ ├── analyze_core.py, analyze_detection.py, analyze_rhythm.py, -│ │ analyze_segments.py, analyze_structure.py, -│ │ analyze_transcription.py, analyze_audio_io.py, -│ │ analyze_estimate.py, analyze_fast.py # Feature modules (split from monolith) -│ ├── server.py + server_phase1.py / server_phase2.py / server_upload.py -│ ├── analysis_runtime.py # SQLite persistence layer -│ ├── worker.py, runtime_profile.py, auth_context.py, artifact_storage.py -│ │ # Hosted-mode foundation -│ ├── upload_limits.py # 100/101 MiB upload contract -│ ├── spectral_viz.py # Spectrogram artifacts (non-critical) -│ ├── polyphonic_evaluation.py # Research-only; not on product path -│ ├── requirements.txt -│ ├── scripts/ -│ │ ├── bootstrap.sh # Environment setup -│ │ └── evaluate_polyphonic.py # Research-only evaluator entry point -│ ├── tests/ # unittest suite -│ └── prompts/ # Gemini system prompts + Live 12 device catalog -├── scripts/ -│ ├── dev.sh # Full-stack dev launcher -│ ├── test-e2e-integration.sh # Local-only integration E2E -│ └── test-e2e.sh # Live Gemini E2E -├── docs/ -│ ├── ARCHITECTURE_STRATEGY.md # Why the architecture is shaped this way -│ ├── PUBLIC_HOSTING_FOUNDATION.md # Hosted-mode boundaries -│ ├── POLYPHONIC_TRANSCRIPTION_SPIKE.md # Research notes -│ └── archive/ # Historical/invalidated docs -└── AGENTS.md # This file -``` - -### Key Frontend Modules - -| File/Directory | Purpose | -|----------------|---------| -| `src/App.tsx` | Main application, upload flow, phase orchestration | -| `src/services/analysisRunsClient.ts` | Canonical transport for `/api/analysis-runs*` | -| `src/services/analyzer.ts` | Orchestration: run creation, polling, display payload projection | -| `src/services/backendPhase1Client.ts` | Legacy multipart transport for `/api/analyze` (compat path) | -| `src/services/phase2Validator.ts` | Runtime chain-of-custody validator (Phase 2 vs Phase 1) | -| `src/types.ts` + `src/types/` | Shared response contracts (barrel re-export of `types/{measurement,backend,interpretation}.ts`) | -| `src/components/AnalysisResults.tsx` | Results display | -| `src/components/SessionMusicianPanel.tsx` | MIDI/piano roll UI | - -### Key Backend Modules - -| File | Purpose | -|------|---------| -| `analyze.py` | DSP pipeline entry point; coordinates the `analyze_*.py` modules below | -| `analyze_core/_detection/_rhythm/_segments/_structure/_transcription/_audio_io/_estimate/_fast.py` | Feature modules (split from the monolith in commit `5c40dd44`) | -| `server.py` (+ `server_phase1.py`, `server_phase2.py`, `server_upload.py`) | HTTP transport, temp file handling, response normalization | -| `analysis_runtime.py` | SQLite persistence, stage queues, artifact metadata | -| `worker.py` / `runtime_profile.py` / `auth_context.py` / `artifact_storage.py` | Hosted-mode runtime | -| `polyphonic_evaluation.py` | **Research-only.** Offline harness, not on product path | -| `tests/test_server.py` | API contract tests | -| `tests/test_analyze.py` | Structural snapshot tests; owns `EXPECTED_TOP_LEVEL_KEYS` | - -## Build and Development Commands - -### Full Stack (Recommended) - -Start both backend and UI with proper synchronization: - -```bash -./scripts/dev.sh -``` - -This script: -1. Starts backend on `127.0.0.1:8100` -2. Waits for OpenAPI contract verification -3. Starts UI on `127.0.0.1:3100` -4. Handles graceful shutdown on Ctrl-C - -### Backend Only - -Setup (first time or after dependency changes): - -```bash -./apps/backend/scripts/bootstrap.sh -``` - -Run the server: - -```bash -./apps/backend/venv/bin/python apps/backend/server.py -# Or with custom port: -SONIC_ANALYZER_PORT=8100 ./apps/backend/venv/bin/python apps/backend/server.py -``` - -Run CLI analyzer directly: - -```bash -./apps/backend/venv/bin/python apps/backend/analyze.py [--separate] [--transcribe] [--yes] -``` - -### Frontend Only - -Setup: - -```bash -cd apps/ui -npm install -``` - -Development server: - -```bash -npm run dev:local # Port 3100, localhost only -npm run dev # Port 3000, host 0.0.0.0 -``` - -Build: - -```bash -npm run build -npm run preview # Preview production build -``` - -### Verification Commands - -Frontend full validation: - -```bash -cd apps/ui -npm run verify # lint + unit tests + build + smoke tests -``` - -Backend tests: - -```bash -cd apps/backend -./venv/bin/python -m unittest discover -s tests -``` - -Syntax checks: - -```bash -cd apps/backend -./venv/bin/python -m py_compile server.py -./venv/bin/python -m py_compile analyze.py -``` - -### Single Test Commands - -Frontend unit test: - -```bash -cd apps/ui -npx vitest run tests/services/backendPhase1Client.test.ts -npx vitest run tests/services/backendPhase1Client.test.ts -t "test name" -``` - -Frontend smoke test: - -```bash -cd apps/ui -npm run test:smoke -- tests/smoke/upload-phase1.spec.ts -``` - -Backend single test: - -```bash -cd apps/backend -./venv/bin/python -m unittest tests.test_server -./venv/bin/python -m unittest tests.test_server.ServerContractTests -./venv/bin/python -m unittest tests.test_server.ServerContractTests.test_analyze_endpoint_combines_separate_and_transcribe_in_subprocess -``` - -## Testing Strategy - -### Frontend Testing - -| Test Type | Tool | Location | Purpose | -|-----------|------|----------|---------| -| Unit | Vitest | `tests/services/` | Test business logic, parsers, clients | -| Smoke | Playwright | `tests/smoke/` | Critical path E2E tests | -| Live | Playwright | `tests/smoke/*-live*.spec.ts` | Tests against real backend/Gemini | - -**Test Environment**: Vitest runs in `node` environment (not `jsdom`). Tests use mocks, fake timers, and payload fixtures. - -**Important**: Playwright boots the app on `127.0.0.1:3100` for smoke tests. - -### Backend Testing - -| Test Type | Tool | Location | Purpose | -|-----------|------|----------|---------| -| Contract | unittest | `tests/test_server.py` | API envelope, error handling | -| Structural | unittest | `tests/test_analyze.py` | Raw analyzer JSON output | - -**Testing Framework**: Uses stdlib `unittest`, not pytest. - -### Test Data - -- Backend tests generate temporary WAV fixtures -- Smoke tests may use `TEST_FLAC_PATH` environment variable for live backend tests -- Gemini live tests require `RUN_GEMINI_LIVE_SMOKE=true` and API key - -## Code Style Guidelines - -### Python (Backend) - -- **Indentation**: 4 spaces -- **Quotes**: Prefer double quotes -- **Type hints**: Use Python 3.10+ style (`str | None`, `dict[str, Any]`) -- **Import order**: stdlib → third-party → local (separated by blank lines) -- **Naming**: `snake_case` functions/variables, `PascalCase` classes, `UPPER_SNAKE_CASE` constants -- **Private helpers**: Prefix with `_` when internal to module - -Example: -```python -import json -from typing import Any - -import numpy as np -from fastapi import FastAPI - -from analysis_runtime import AnalysisRun - - -def _normalize_value(value: float | None) -> float | None: - return round(value, 4) if value is not None else None -``` - -### TypeScript/React (Frontend) - -- Follow the local style of the file you're editing -- **Naming**: `PascalCase` components/interfaces, `camelCase` functions/variables -- **Imports**: External packages first, then local modules -- **Components**: Use function components and hooks -- **Cleanup**: Always clean up timers, object URLs, audio resources in `useEffect` - -### Tailwind CSS - -- Use semantic tokens from `src/index.css` (`bg-bg-panel`, `text-text-secondary`) -- Maintain Ableton-inspired dark visual language -- Keep motion purposeful and lightweight - -## Security Considerations - -### API Keys - -- **Gemini API Key**: Stored in backend environment, NOT exposed to frontend -- Frontend Phase 2 uses backend-mediated Gemini calls -- Never commit API keys to the repository - -### CORS - -Backend allows these origins: -- `http://localhost:3000`, `http://127.0.0.1:3000` -- `http://localhost:3100`, `http://127.0.0.1:3100` -- `http://localhost:5173`, `http://127.0.0.1:5173` - -### File Uploads - -- Temporary files are written to disk during analysis -- Files are cleaned up after processing (success or error) -- Raw audio files at or below 100 MiB go inline to Gemini -- Larger files use Gemini Files API - -### Local Development Only - -Current quality bar is for local development. Do not present as production-ready security until: -- Proper authentication is implemented -- Input validation is hardened -- Database and artifact storage move beyond the local machine - -## Environment Configuration - -### Frontend Environment Variables - -Create `apps/ui/.env` from `.env.example`: - -```bash -# Required for backend connection -VITE_API_BASE_URL="http://127.0.0.1:8100" - -# Enable Phase 2 Gemini features -VITE_ENABLE_PHASE2_GEMINI="true" - -# Disable HMR for testing -DISABLE_HMR="true" -``` - -### Backend Environment Variables - -```bash -# Server port (default: 8100) -SONIC_ANALYZER_PORT=8100 - -# Gemini API key for Phase 2 -GEMINI_API_KEY="your_key_here" -``` - -### Test Environment Variables - -```bash -# Live backend smoke tests -TEST_FLAC_PATH=/path/to/track.flac -VITE_API_BASE_URL=http://127.0.0.1:8100 - -# Live Gemini smoke tests -RUN_GEMINI_LIVE_SMOKE=true -VITE_ENABLE_PHASE2_GEMINI=true -GEMINI_API_KEY=your_key_here -``` - -## Network Configuration - -### Canonical Local Ports - -| Service | Port | URL | -|---------|------|-----| -| UI dev server | 3100 | http://127.0.0.1:3100 | -| Backend API | 8100 | http://127.0.0.1:8100 | - -### API Endpoints - -Backend exposes: - -- `POST /api/analysis-runs/estimate` - Get the canonical runtime estimate -- `POST /api/analysis-runs` - Create a staged analysis run -- `GET /api/analysis-runs/{run_id}` - Poll the canonical run snapshot -- `POST /api/analyze` - Legacy compatibility wrapper for full analysis -- `POST /api/phase2` - Legacy compatibility wrapper for Gemini interpretation -- `GET /openapi.json` - OpenAPI schema -- `GET /docs` - Swagger UI -- `GET /redoc` - ReDoc documentation - -## Deployment - -### Current Status - -The system is designed for **local development** use. Current limitations: - -- SQLite database stored locally (`.runtime/analysis_runs.sqlite3`) -- Artifact storage on local filesystem -- No authentication/authorization layer -- Python 3.11.x requirement for full setup - -### Deployment Considerations - -Before production deployment: - -1. Move to proper database (PostgreSQL) -2. Implement cloud storage for artifacts -3. Add authentication layer -4. Containerize with Docker -5. Implement proper secret management -6. Add rate limiting and resource controls - -## Sub-Project Documentation - -For detailed information specific to each app, see: - -- `apps/ui/AGENTS.md` - Frontend-specific guidance, React patterns, styling rules -- `apps/backend/AGENTS.md` - Backend-specific guidance, DSP pipeline, testing expectations -- `apps/backend/ARCHITECTURE.md` - Backend component responsibilities -- `apps/backend/JSON_SCHEMA.md` - Raw CLI and HTTP schema documentation -- `docs/ARCHITECTURE_STRATEGY.md` - Architecture decisions and roadmap - -## Important Constraints - -1. **Python Version**: Backend requires Python 3.11.x for full-feature local setup. Python 3.12+ is not yet supported because Essentia 2.1b6 wheels are only published for 3.11 on macOS arm64. - -2. **No Repo-Wide Formatting**: No ESLint/Prettier/Ruff baseline is enforced. Follow the style of the surrounding file. - -3. **Contract Boundaries**: - - Measurement result is authoritative - - Pitch/note transcription is injected from pitch/note translation stage (not copied from measurement) - - UI/backend contract is strict and strongly typed - -4. **Before Structural Changes**: Read `docs/ARCHITECTURE_STRATEGY.md` first. It contains the reasoning behind the current design and planned experiments. - -## Common Tasks - -### Adding a New DSP Feature - -1. Add function in `apps/backend/analyze.py` -2. Update raw JSON output schema (document in `JSON_SCHEMA.md`) -3. Update `server.py` normalization if needed -4. Update frontend `types.ts` if new fields exposed via HTTP -5. Add tests in `tests/test_analyze.py` - -### Adding a New API Endpoint - -1. Add route in `apps/backend/server.py` -2. Run contract tests: `./venv/bin/python -m unittest tests.test_server` -3. Update OpenAPI will be automatic -4. Add frontend client method in `src/services/` -5. Update frontend types in `src/types.ts` - -### Debugging Backend Issues - -1. Check logs on stderr (timing, diagnostics) -2. Run CLI directly: `./venv/bin/python analyze.py --yes` -3. Verify JSON output is valid -4. Run contract tests to isolate issue - -## Change Checklist - -Before submitting changes: - -- [ ] If changing API request parsing, run `tests/test_server.py` -- [ ] If changing raw analyzer output, run `tests/test_analyze.py` and update docs -- [ ] If changing timeout or diagnostics, inspect both tests and `ARCHITECTURE.md` -- [ ] If adding a new field, document whether it belongs to raw CLI output, HTTP `phase1`, or both -- [ ] Run narrowest relevant test first, then full suite for broad changes -- [ ] Frontend: run `npm run verify` for app-wide changes -- [ ] Backend: run `./venv/bin/python -m unittest discover -s tests` - -## Getting Help - -- Review `docs/ARCHITECTURE_STRATEGY.md` for architecture reasoning -- Check `apps/backend/JSON_SCHEMA.md` for API contracts -- See `README.md` at repo root and in each app directory -- Review existing tests for usage examples +Historical plan and audit documents live in [`docs/history/`](docs/history/) — past-tense, not living docs. diff --git a/ASA_System_Design.docx b/ASA_System_Design.docx deleted file mode 100644 index ede1fc56..00000000 Binary files a/ASA_System_Design.docx and /dev/null differ diff --git a/CHANGELOG.md b/CHANGELOG.md index 7699dcb5..90d89c60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,7 +38,7 @@ All notable changes to `asa` are documented here. - **TypeScript limiter fallback card**: `deviceFamily` and `workflowStage` narrowed with `as const` to satisfy `DeviceFamily` / `WorkflowStage` union types in the `satisfies` clause - MixDoctor null-genre fallback: prompts for manual selection instead of silently using first profile - Genre abstention logic with tests for empty, sparse, ambiguous, and fast-mode inputs -- **Confidence calibration invalidated**: `docs/confidence_calibration_results.md` (now archived at [`docs/archive/confidence-calibration-results-stubs.md`](docs/archive/confidence-calibration-results-stubs.md)) was generated from hand-crafted cache stubs with no real audio. All F1=1.0 results and threshold recommendations were artefacts of the stub data. Thresholds reverted to original engineering-judgment values (`pitchConfidence=0.15`, `chordStrength=0.70`, `pumpingConfidence=0.40`) in `apps/backend/prompts/phase2_system.txt`. Calibration script now aborts if all tracks are cache-only with no audio files present, and warns when only a partial real-audio subset is available. +- **Confidence calibration invalidated**: `docs/confidence_calibration_results.md` (now archived at [`docs/history/archive/confidence-calibration-results-stubs.md`](docs/history/archive/confidence-calibration-results-stubs.md)) was generated from hand-crafted cache stubs with no real audio. All F1=1.0 results and threshold recommendations were artefacts of the stub data. Thresholds reverted to original engineering-judgment values (`pitchConfidence=0.15`, `chordStrength=0.70`, `pumpingConfidence=0.40`) in `apps/backend/prompts/phase2_system.txt`. Calibration script now aborts if all tracks are cache-only with no audio files present, and warns when only a partial real-audio subset is available. ## v2.1.0 diff --git a/CLAUDE.md b/CLAUDE.md index f6391d60..b198e7d5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -239,10 +239,11 @@ Most of the original `sonic-architect-app` port (genre profiles, Ableton device ## Companion Agent Docs -This repo carries parallel guidance for non-Claude agents. The files largely cover the same ground but with different framings; read whichever matches the context you need: +This repo carries parallel guidance for non-Claude agents: -1. **`AGENTS.md`** (root + `apps/*/AGENTS.md`) — general AI-agent policy, technology stack tables, change checklist, common task recipes. -2. **`CODEX.md`** (root + `apps/*/CODEX.md`) — Codex-tailored read-order and mission-gate questions, derived from `PURPOSE.md`. +1. **`AGENTS.md`** (root) — pointer for Codex / OpenHands / any tool that looks for `AGENTS.md` by name. Defers to this file. +2. **`apps/backend/AGENTS.md`**, **`apps/ui/AGENTS.md`** — per-app overlays with technology-stack details and app-specific change checklists. 3. **`docs/ARCHITECTURE_STRATEGY.md`** — *why* the three-layer architecture is shaped the way it is. +4. **`docs/history/`** — completed plans and one-shot audits. Past-tense, not living docs. -When information conflicts, `PURPOSE.md` > `AGENTS.md` chain > this file. +When information conflicts: `PURPOSE.md` > this file > per-app `AGENTS.md`. diff --git a/CODEX.md b/CODEX.md deleted file mode 100644 index fc833a43..00000000 --- a/CODEX.md +++ /dev/null @@ -1,84 +0,0 @@ -# CODEX.md - -This file provides guidance to Codex when working in this repository. - -## Source Mapping - -- Product mission and quality bar: `PURPOSE.md` and `ASA_System_Design.docx`. -- Repo workflow and policy: `AGENTS.md` at root, then app-local `apps/ui/AGENTS.md` or `apps/backend/AGENTS.md`. -- Command and runtime specifics: `CLAUDE.md`. - -When guidance differs: - -1. Mission and quality invariants from `PURPOSE.md` and `ASA_System_Design.docx` win. -2. Repo workflow and contract rules from the `AGENTS.md` chain win next. -3. `CODEX.md` files provide Codex-tailored execution guidance. - -## Read Order For Codex - -1. `PURPOSE.md` -2. `ASA_System_Design.docx` -3. `AGENTS.md` -4. app-local `CODEX.md` + app-local `AGENTS.md` for the area you edit -5. `CLAUDE.md` for command details and additional guardrails - -## Mission Gate - -Before implementing changes, run this test: - -1. Does this improve measurement accuracy? -2. Does this improve recommendation specificity/quality? -3. Does this improve a producer's ability to act in Ableton? -4. If it is maintenance-only, does it clearly unblock one of the above? -5. If none apply, stop and reconsider. - -## Non-Negotiable Invariants - -- Phase 1 measurements are ground truth; Phase 2 does not override measured values. -- Phase 2 recommendations must cite specific Phase 1 measurements. -- Recommendations must be Ableton Live 12 specific (device, parameter, value). -- Low-confidence measurements must lead to hedged recommendations. -- Reconstruction guidance must cover the full production surface. -- Output must remain usable for intermediate producers without DSP expertise. - -## Architecture Snapshot - -- Layer 1 (`apps/backend/analyze.py`): deterministic DSP measurement engine. -- Layer 2 (`apps/backend/server.py` + `/api/phase2`): interpretation using measured data plus audio. -- Layer 3 (`apps/ui`): upload, estimate, analysis, and reconstruction-facing presentation. -- Contract boundary: `phase1`/`phase2` shapes consumed by UI types must remain aligned with backend responses. - -## Codex Workflow Expectations - -- Treat monorepo root as entrypoint for stack orchestration and release context. -- Prefer surgical edits; avoid broad rewrites unless explicitly requested. -- Preserve `analyze.py` `stdout` JSON vs `stderr` diagnostics behavior. -- Keep frontend/backend contracts in sync when adding/removing fields. -- Read `docs/ARCHITECTURE_STRATEGY.md` before proposing structural architecture or pipeline changes. - -## Canonical Commands - -From repo root: - -```bash -./scripts/dev.sh -``` - -Frontend verification: - -```bash -cd apps/ui -npm run verify -``` - -Backend verification: - -```bash -cd apps/backend -./venv/bin/python -m unittest discover -s tests -``` - -## App Routing - -- UI work: read `apps/ui/CODEX.md` and `apps/ui/AGENTS.md`. -- Backend work: read `apps/backend/CODEX.md` and `apps/backend/AGENTS.md`. diff --git a/README.md b/README.md index b3bfd5b5..b474e501 100644 --- a/README.md +++ b/README.md @@ -1,222 +1,73 @@ -# asa +# Ableton Sonic Analyzer (ASA) -Local/dev monorepo for the Sonic Analyzer project. +A local-first tool that answers **"how do I make something that sounds like +this?"** for intermediate Ableton Live 12 producers. -This repo preserves the history of the existing UI and backend repos and brings -them together under one roof: +ASA runs deterministic DSP measurements on a track (Phase 1) and feeds them to +an AI interpreter (Phase 2) that produces specific, measurement-cited Ableton +device recommendations. The chain of custody from measured number to +recommendation is the product — see [`PURPOSE.md`](PURPOSE.md) for the design +brief and quality invariants. -- `apps/ui` contains the React/Vite frontend -- `apps/backend` contains the Python/FastAPI local DSP backend -- `scripts/dev.sh` starts the full local stack on the canonical ports +## Architecture -Migration note: - -- `apps/ui` and `apps/backend` were imported with history from the former standalone repos. -- The monorepo root is now the source of truth for release notes, local-stack commands, and push workflow. -- App-level changelogs remain imported app history rather than monorepo release history. -- App-specific editing and test guidance lives in `apps/ui/AGENTS.md` and `apps/backend/AGENTS.md`. - -## AI Agent Docs - -- `AGENTS.md`: monorepo policy and workflow rules. -- `CODEX.md`: Codex-tailored guidance derived from `PURPOSE.md`, `ASA_System_Design.docx`, and `CLAUDE.md`. -- App-local overlays: - - `apps/ui/CODEX.md` - - `apps/backend/CODEX.md` - -## Canonical Local Stack - -- UI: `http://127.0.0.1:3100` -- backend: `http://127.0.0.1:8100` - -## Canonical Runtime Flow - -- `POST /api/analysis-runs/estimate` -- `POST /api/analysis-runs` -- `GET /api/analysis-runs/{run_id}` -- `GET /api/analysis-runs/{run_id}/artifacts...` - -Runtime profiles: - -- `local`: current local/dev mode with SQLite + local artifact files + in-process workers. -- `hosted`: hosted-service mode with auth hooks and worker separation boundaries. - -In plain English: the analysis engine is still shared, but the repo now has an explicit split between local mode and hosted mode so public-hosting work does not have to change the local product path. - -Artifact storage now sits behind a backend storage service boundary. In plain English: ASA still writes files locally today, but the code is no longer hard-wired to assume that every stored artifact is just a disk path on the same machine. - -Implementation record: - -- see `docs/PUBLIC_HOSTING_FOUNDATION.md` for the full summary of the hosted-foundation work, the follow-up fixes, the verification that was run, and the remaining work before any true public deployment. - -Legacy `POST /api/analyze`, `POST /api/analyze/estimate`, and `POST /api/phase2` remain available only as temporary compatibility wrappers during the migration window. - -## Local Setup - -Frontend dependencies: - -```bash -cd apps/ui -npm install ``` - -Backend environment: - -```bash -./apps/backend/scripts/bootstrap.sh -``` - -The backend bootstrap path is verified on Python `3.11.x`. The bootstrap -script recreates `apps/backend/venv` from scratch and is the supported recovery -path if the local backend environment becomes stale or broken. - -Manual equivalent: - -```bash -cd apps/backend -python3.11 -m venv venv -./venv/bin/python -m pip install --upgrade pip -./venv/bin/python -m pip install -r requirements.txt -``` - -The backend dependency stack is pinned and validated on Python `3.11.x` for -full-feature local development on macOS arm64. - -Current limitation: Python `3.12+` is not yet supported because Essentia -2.1b6 wheels are only published for 3.11 on macOS arm64. - -Run the full stack from the repo root: - -```bash -./scripts/dev.sh +Layer 1 — MEASUREMENT (Essentia/DSP) → deterministic, authoritative +Layer 2 — PITCH/NOTE TRANSLATION (torchcrepe) → best-effort on separated stems +Layer 3 — INTERPRETATION (Gemini) → grounded in Layer 1 measurements ``` -### Phase 2 Local Setup - -`./scripts/dev.sh` now reads `apps/ui/.env` before starting Vite. This is the -recommended persistent way to enable Gemini Phase 2 locally. - -Persistent `.env` setup: +Phase 2 never overrides Phase 1. See [`docs/ARCHITECTURE_STRATEGY.md`](docs/ARCHITECTURE_STRATEGY.md) for *why* the stack is shaped this way. -```bash -cd apps/ui -cp .env.example .env -``` - -Then set: +## Repo layout -```bash -VITE_API_BASE_URL="http://127.0.0.1:8100" -VITE_ENABLE_PHASE2_GEMINI="true" ``` - -Optional hosted-mode request-header bootstrap for private beta testing: - -```bash -VITE_API_REQUEST_HEADERS_JSON='{"X-ASA-User-Id":"beta-user-123"}' +apps/backend/ Python 3.11 + FastAPI + Essentia DSP pipeline +apps/ui/ React 19 + Vite + TypeScript + Tailwind +scripts/ dev.sh, e2e harnesses +docs/ ARCHITECTURE_STRATEGY, SETUP, topic docs +docs/history/ Completed plans and one-shot audits (reference only) ``` -Supported shell-based overrides: +## Quickstart -```bash -export GEMINI_API_KEY="your_real_key_here" -./scripts/dev.sh -``` +Requires **Python 3.11.x** (Essentia 2.1b6 wheels aren't published for 3.12+) and **Node.js 20+**. ```bash -GEMINI_API_KEY="your_real_key_here" ./scripts/dev.sh -``` +# One-time backend setup +./apps/backend/scripts/bootstrap.sh -This does **not** work because the variable is not exported to the next -command: +# One-time frontend setup +cd apps/ui && npm install && cd - -```bash -GEMINI_API_KEY="your_real_key_here" +# Start the full stack (UI on :3100, backend on :8100) ./scripts/dev.sh ``` -Manual equivalent: - -```bash -cd apps/backend -SONIC_ANALYZER_PORT=8100 ./venv/bin/python server.py -``` - -Hosted worker process: - -```bash -cd apps/backend -SONIC_ANALYZER_RUNTIME_PROFILE=hosted SONIC_ANALYZER_PROCESS_ROLE=worker ./venv/bin/python worker.py -``` - -```bash -cd apps/ui -VITE_API_BASE_URL=http://127.0.0.1:8100 npm run dev:local -``` +Full setup, environment variables, Phase 2 (Gemini) configuration, and +verification commands live in [`docs/SETUP.md`](docs/SETUP.md). ## Verification -Frontend: - ```bash -cd apps/ui -npm run verify +cd apps/ui && npm run verify # lint + unit + build + smoke +cd apps/backend && ./venv/bin/python -m unittest discover -s tests +./scripts/test-e2e-integration.sh # local-only e2e, no Gemini key ``` -Backend: - -```bash -cd apps/backend -./venv/bin/python -m unittest discover -s tests -``` - -Canonical local end-to-end verification is local-only, boots the real backend, drives the UI against the canonical `analysis-runs` routes, and does not require Gemini credentials or a user-provided track: - -```bash -./scripts/test-e2e-integration.sh -``` - -Full live Gemini end-to-end verification stays separate and requires a real audio file plus backend Gemini credentials: - -```bash -TEST_FLAC_PATH=/path/to/track.flac \ -GEMINI_API_KEY=your_real_key_here \ -VITE_ENABLE_PHASE2_GEMINI=true \ -./scripts/test-e2e.sh -``` - -## Upload Limits - -For the backend upload routes, ASA now distinguishes between: - -- raw audio limit: `100 MiB` -- HTTP request envelope limit: `101 MiB` - -In plain English: the audio file itself must stay at or below 100 MiB, but the -whole multipart request is allowed to be slightly larger so filenames and form -wrapping do not cause false `413` errors. - -The canonical operator view is generated from backend code, not maintained by -hand. To see the current edge contract and proxy examples: - -```bash -cd apps/backend -./venv/bin/python scripts/render_upload_limit_contract.py -``` - -If you later put this local stack behind a reverse proxy or load balancer, -mirror the generated `101 MiB` request-body limit there for the protected -upload routes instead of copying stale numbers from old docs. - -## Release Position - -The initial monorepo cut was **local/dev `v1.0.0`**. Current tags: `v1.2.0` (root), `ui-v1.6.0` (frontend). +## Documentation -The current quality bar is met for local development and iterative product work. -It should not be presented as a stronger production/security milestone until -authentication, stronger input hardening, and non-local artifact/database infrastructure are in place. +| Where | What | +|---|---| +| [`PURPOSE.md`](PURPOSE.md) | Why ASA exists; non-negotiable quality invariants. | +| [`CLAUDE.md`](CLAUDE.md) | Canonical guide for AI coding agents and contributors: commands, architecture, tripwires, change map. | +| [`docs/ARCHITECTURE_STRATEGY.md`](docs/ARCHITECTURE_STRATEGY.md) | Why the three-layer design is shaped the way it is. | +| [`docs/SETUP.md`](docs/SETUP.md) | Detailed local setup, env vars, Phase 2 wiring. | +| [`apps/backend/ARCHITECTURE.md`](apps/backend/ARCHITECTURE.md) | Backend HTTP flow and contract. | +| [`apps/backend/JSON_SCHEMA.md`](apps/backend/JSON_SCHEMA.md) | Phase 1 stdout JSON schema. | +| [`BACKLOG.md`](BACKLOG.md) | What's next. | +| [`CHANGELOG.md`](CHANGELOG.md) | What's shipped. | -Keep the backend bootstrap limitation in mind when handing the repo to another machine: +## License -- prefer Python `3.11.x` -- run `./apps/backend/scripts/bootstrap.sh` from the repo root before starting the local stack +[MIT](LICENSE). diff --git a/apps/backend/AGENTS.md b/apps/backend/AGENTS.md index 30cc8922..4ae460d4 100644 --- a/apps/backend/AGENTS.md +++ b/apps/backend/AGENTS.md @@ -3,7 +3,7 @@ ## Scope - This file applies to `apps/backend` inside the `asa` monorepo. -- Codex-specific instructions for this app live in `CODEX.md`. +- Root-level agent guidance lives in `../../CLAUDE.md`; this file is the backend overlay. - The repo is a local Python audio-analysis service with two entry points: - `analyze.py`: raw CLI analyzer - `server.py`: FastAPI wrapper around the CLI diff --git a/apps/backend/CODEX.md b/apps/backend/CODEX.md deleted file mode 100644 index aee27b78..00000000 --- a/apps/backend/CODEX.md +++ /dev/null @@ -1,46 +0,0 @@ -# CODEX.md - -Codex instructions for `apps/backend`. - -## Source Mapping - -- Product intent and quality bar: `../../PURPOSE.md` and `../../ASA_System_Design.docx`. -- Repo and app policy: `../../AGENTS.md` and `./AGENTS.md`. -- Runtime command details and additional guardrails: `../../CLAUDE.md`. - -When guidance differs, keep mission and quality invariants from `PURPOSE.md` and the system design document as the primary decision filter. - -## Backend Mission In This Repo - -- Keep deterministic measurement quality high and trustworthy. -- Preserve the chain of custody from Phase 1 metrics to Phase 2 advice. -- Protect producer-facing reliability over internal abstraction complexity. - -## Contract-Critical Rules - -- `analyze.py` emits machine-readable JSON to `stdout`; diagnostics/logs go to `stderr`. -- `server.py` normalizes raw analyzer output into stable HTTP envelopes for UI consumption. -- Treat backend output shape as contract; update tests/docs with any intentional schema change. -- Keep Phase 1 measurement authority intact; never add behavior that lets Phase 2 override measured values. - -## Canonical Commands - -```bash -./scripts/bootstrap.sh -./venv/bin/python server.py -./venv/bin/python analyze.py [--separate] [--transcribe] [--fast] [--yes] -./venv/bin/python -m unittest discover -s tests -``` - -Preferred synced stack from repo root: - -```bash -./scripts/dev.sh -``` - -## Codex Change Checklist - -- If request parsing, subprocess behavior, or envelopes change: run `tests/test_server.py` or broader. -- If raw analyzer output changes: run `tests/test_analyze.py` and sync docs. -- Preserve bounded diagnostics and structured error responses. -- Keep edits surgical unless an explicit broader refactor is requested. diff --git a/apps/backend/LICENSE b/apps/backend/LICENSE deleted file mode 100644 index 5a1ef31b..00000000 --- a/apps/backend/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 Christian Smith - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/apps/backend/tests/test_upload_limits.py b/apps/backend/tests/test_upload_limits.py index e3941627..84502e36 100644 --- a/apps/backend/tests/test_upload_limits.py +++ b/apps/backend/tests/test_upload_limits.py @@ -63,18 +63,21 @@ def test_generator_outputs_plain_english_and_proxy_snippets(self) -> None: self.assertIn(str(upload_limits.MAX_UPLOAD_REQUEST_BYTES), result.stdout) def test_docs_reference_generator_and_current_contract_values(self) -> None: - root_readme = (REPO_ROOT / "README.md").read_text(encoding="utf-8") + # Operator-facing upload-limit detail lives in docs/SETUP.md (the + # canonical setup/operator doc that the root README links to). The + # backend README and ARCHITECTURE.md keep their app-local copies. + setup_doc = (REPO_ROOT / "docs" / "SETUP.md").read_text(encoding="utf-8") backend_readme = (BACKEND_DIR / "README.md").read_text(encoding="utf-8") architecture_doc = (BACKEND_DIR / "ARCHITECTURE.md").read_text(encoding="utf-8") expected_command = "./venv/bin/python scripts/render_upload_limit_contract.py" - self.assertIn(expected_command, root_readme) + self.assertIn(expected_command, setup_doc) self.assertIn(expected_command, backend_readme) self.assertIn("upload limit contract", architecture_doc.lower()) self.assertIn(str(upload_limits.MAX_UPLOAD_REQUEST_BYTES), backend_readme) self.assertIn(f"{upload_limits.MAX_UPLOAD_REQUEST_BYTES:,}".replace(",", ""), backend_readme) - self.assertIn(f"{upload_limits.MAX_UPLOAD_SIZE_BYTES // (1024 * 1024)} MiB", root_readme) + self.assertIn(f"{upload_limits.MAX_UPLOAD_SIZE_BYTES // (1024 * 1024)} MiB", setup_doc) if __name__ == "__main__": diff --git a/apps/ui/CODEX.md b/apps/ui/CODEX.md deleted file mode 100644 index fa54e149..00000000 --- a/apps/ui/CODEX.md +++ /dev/null @@ -1,50 +0,0 @@ -# CODEX.md - -Codex instructions for `apps/ui`. - -## Source Mapping - -- Product intent and quality bar: `../../PURPOSE.md` and `../../ASA_System_Design.docx`. -- Repo and app policy: `../../AGENTS.md` and `./AGENTS.md`. -- Runtime command details and additional guardrails: `../../CLAUDE.md`. - -When guidance differs, keep mission and quality invariants from `PURPOSE.md` and the system design document as the primary decision filter. - -## UI Mission In This Repo - -- Present deterministic Phase 1 measurements clearly and faithfully. -- Present Phase 2 interpretation as measurement-cited Ableton reconstruction guidance. -- Improve producer actionability over visual novelty. - -## Contract-Critical Rules - -- Preserve backend client and shared type contracts in: - - `src/services/backendPhase1Client.ts` - - `src/types.ts` -- Do not silently rename fields expected by backend envelopes. -- Keep diagnostics behavior stable unless intentionally changing contract + tests/docs together. -- Respect the Phase 1 ground-truth model when rendering or explaining results. - -## Canonical Commands - -```bash -npm run dev -npm run dev:local -npm run lint -npm run test:unit -npm run test:smoke -npm run verify -``` - -Preferred synced stack from repo root: - -```bash -./scripts/dev.sh -``` - -## Codex Change Checklist - -- Run focused tests first (single file/spec), then broaden as needed. -- If editing upload/orchestration/rendering flow, run relevant smoke specs. -- If editing shared types or transport parsing, run lint + targeted service tests. -- Avoid style-only churn in mixed-style files. diff --git a/apps/ui/LICENSE b/apps/ui/LICENSE deleted file mode 100644 index 5a1ef31b..00000000 --- a/apps/ui/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 Christian Smith - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/docs/PHASE2_TRUTHFULNESS_PHASES_A_B_C.docx b/docs/PHASE2_TRUTHFULNESS_PHASES_A_B_C.docx deleted file mode 100644 index b2316d32..00000000 Binary files a/docs/PHASE2_TRUTHFULNESS_PHASES_A_B_C.docx and /dev/null differ diff --git a/docs/SETUP.md b/docs/SETUP.md new file mode 100644 index 00000000..a1b7d346 --- /dev/null +++ b/docs/SETUP.md @@ -0,0 +1,189 @@ +# Local Setup + +Full setup, environment variables, and verification details for ASA. For a +30-second overview, see the root [`README.md`](../README.md). + +## Prerequisites + +- **Python `3.11.x`** — Essentia 2.1b6 wheels are only published for 3.11. + Python 3.12+ is not yet supported. +- **Node.js 20+**. +- macOS arm64 is the validated platform; Linux works for most paths. + +## Backend + +```bash +./apps/backend/scripts/bootstrap.sh +``` + +`bootstrap.sh` recreates `apps/backend/venv` from scratch. It is also the +supported recovery path if the local backend environment becomes stale. + +Manual equivalent: + +```bash +cd apps/backend +python3.11 -m venv venv +./venv/bin/python -m pip install --upgrade pip +./venv/bin/python -m pip install -r requirements.txt +``` + +## Frontend + +```bash +cd apps/ui +npm install +``` + +## Full stack + +```bash +./scripts/dev.sh +``` + +This boots backend on `127.0.0.1:8100` and UI on `127.0.0.1:3100`. It waits +for the backend `/openapi.json` contract before launching Vite, and overrides +`VITE_API_BASE_URL` for the spawned UI process so stale `apps/ui/.env` files +don't break the stack. + +## Phase 2 (Gemini) setup + +Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. The Gemini API key is +backend-only — it never reaches the browser bundle. + +Persistent `.env` setup: + +```bash +cd apps/ui +cp .env.example .env +``` + +Then set: + +```bash +VITE_API_BASE_URL="http://127.0.0.1:8100" +VITE_ENABLE_PHASE2_GEMINI="true" +``` + +Optional hosted-mode request-header bootstrap for private beta testing: + +```bash +VITE_API_REQUEST_HEADERS_JSON='{"X-ASA-User-Id":"beta-user-123"}' +``` + +Backend Gemini key — must be exported in the same shell that runs `dev.sh`: + +```bash +export GEMINI_API_KEY="your_real_key_here" +./scripts/dev.sh +``` + +Inline on a single command line also works: + +```bash +GEMINI_API_KEY="your_real_key_here" ./scripts/dev.sh +``` + +This does **not** work (the variable is not exported to the next command): + +```bash +GEMINI_API_KEY="your_real_key_here" +./scripts/dev.sh +``` + +## Running services individually + +Backend only: + +```bash +cd apps/backend +SONIC_ANALYZER_PORT=8100 ./venv/bin/python server.py +``` + +Hosted worker process: + +```bash +cd apps/backend +SONIC_ANALYZER_RUNTIME_PROFILE=hosted SONIC_ANALYZER_PROCESS_ROLE=worker \ + ./venv/bin/python worker.py +``` + +UI only: + +```bash +cd apps/ui +VITE_API_BASE_URL=http://127.0.0.1:8100 npm run dev:local +``` + +## Verification + +Frontend gate: + +```bash +cd apps/ui +npm run verify # lint + test:unit + build + test:smoke +``` + +Backend tests: + +```bash +cd apps/backend +./venv/bin/python -m unittest discover -s tests +``` + +Local end-to-end (boots the real backend, drives the UI against canonical +`/api/analysis-runs` routes, no Gemini credentials required): + +```bash +./scripts/test-e2e-integration.sh +``` + +Full live Gemini end-to-end (requires a real audio file and backend Gemini +credentials): + +```bash +TEST_FLAC_PATH=/path/to/track.flac \ +GEMINI_API_KEY=your_real_key_here \ +VITE_ENABLE_PHASE2_GEMINI=true \ +./scripts/test-e2e.sh +``` + +## Upload limits + +The backend distinguishes between: + +- raw audio limit: **100 MiB** +- HTTP request envelope limit: **101 MiB** + +The audio file itself must stay at or below 100 MiB; the whole multipart +request is allowed to be slightly larger so filenames and form wrapping don't +cause false `413` errors. + +The canonical operator view is generated from backend code: + +```bash +cd apps/backend +./venv/bin/python scripts/render_upload_limit_contract.py +``` + +If you put the local stack behind a reverse proxy or load balancer, mirror the +generated `101 MiB` request-body limit on the protected upload routes rather +than copying numbers from old docs. + +## Runtime profiles + +The backend supports two profiles, selected via `SONIC_ANALYZER_RUNTIME_PROFILE` +(`local` | `hosted`) and `SONIC_ANALYZER_PROCESS_ROLE` (`all` | `api` | `worker`): + +- **`local`** — SQLite + local artifact files + in-process workers. Default for development. +- **`hosted`** — Adds auth-context resolution and worker-process separation. The local product path is unaffected. + +Implementation record for the hosted foundation: +[`docs/PUBLIC_HOSTING_FOUNDATION.md`](PUBLIC_HOSTING_FOUNDATION.md). + +## Release position + +The current local quality bar is met for local development and iterative +product work. It should not be presented as a stronger production/security +milestone until authentication, stronger input hardening, and non-local +artifact/database infrastructure are in place. diff --git a/docs/history/README.md b/docs/history/README.md new file mode 100644 index 00000000..d18167fd --- /dev/null +++ b/docs/history/README.md @@ -0,0 +1,18 @@ +# docs/history + +One-shot deliverables and completed plan docs. Kept for reference; not living docs. + +If you need to know *why* something looks the way it does today, prefer: + +1. `PURPOSE.md` (root) — why ASA exists and the quality invariants. +2. `docs/ARCHITECTURE_STRATEGY.md` — why the three-layer architecture is shaped the way it is. +3. `apps/backend/ARCHITECTURE.md` + `apps/backend/JSON_SCHEMA.md` — current backend contract. + +Everything in this directory is past-tense. Treat it as a paper trail, not a source of truth. + +## Contents + +- `optimization-plan.md` — completed optimization workstream. +- `phase1-hardening-plan.md` — completed Phase 1 hardening plan. +- `phase1-audit/` — one-shot advisory deliverable: audit, decks, evidence index, visual story pack. +- `archive/` — older archived plans and result stubs. diff --git a/docs/archive/README.md b/docs/history/archive/README.md similarity index 100% rename from docs/archive/README.md rename to docs/history/archive/README.md diff --git a/docs/archive/confidence-calibration-results-stubs.md b/docs/history/archive/confidence-calibration-results-stubs.md similarity index 100% rename from docs/archive/confidence-calibration-results-stubs.md rename to docs/history/archive/confidence-calibration-results-stubs.md diff --git a/docs/archive/phase1-hardening-plan-alt.md b/docs/history/archive/phase1-hardening-plan-alt.md similarity index 100% rename from docs/archive/phase1-hardening-plan-alt.md rename to docs/history/archive/phase1-hardening-plan-alt.md diff --git a/docs/archive/refactor-state-2026-03-18.md b/docs/history/archive/refactor-state-2026-03-18.md similarity index 100% rename from docs/archive/refactor-state-2026-03-18.md rename to docs/history/archive/refactor-state-2026-03-18.md diff --git a/docs/archive/stage3-reality-audit-2026-03-18.md b/docs/history/archive/stage3-reality-audit-2026-03-18.md similarity index 100% rename from docs/archive/stage3-reality-audit-2026-03-18.md rename to docs/history/archive/stage3-reality-audit-2026-03-18.md diff --git a/OPTIMIZATION_PLAN.md b/docs/history/optimization-plan.md similarity index 100% rename from OPTIMIZATION_PLAN.md rename to docs/history/optimization-plan.md diff --git a/advisory/phase1_audit/.node/package-lock.json b/docs/history/phase1-audit/.node/package-lock.json similarity index 100% rename from advisory/phase1_audit/.node/package-lock.json rename to docs/history/phase1-audit/.node/package-lock.json diff --git a/advisory/phase1_audit/.node/package.json b/docs/history/phase1-audit/.node/package.json similarity index 100% rename from advisory/phase1_audit/.node/package.json rename to docs/history/phase1-audit/.node/package.json diff --git a/advisory/phase1_audit/build_phase1_executive_deck.js b/docs/history/phase1-audit/build_phase1_executive_deck.js similarity index 100% rename from advisory/phase1_audit/build_phase1_executive_deck.js rename to docs/history/phase1-audit/build_phase1_executive_deck.js diff --git a/advisory/phase1_audit/ceo_visual_deck.md b/docs/history/phase1-audit/ceo_visual_deck.md similarity index 100% rename from advisory/phase1_audit/ceo_visual_deck.md rename to docs/history/phase1-audit/ceo_visual_deck.md diff --git a/advisory/phase1_audit/deck_build_notes.md b/docs/history/phase1-audit/deck_build_notes.md similarity index 100% rename from advisory/phase1_audit/deck_build_notes.md rename to docs/history/phase1-audit/deck_build_notes.md diff --git a/advisory/phase1_audit/evidence_index.md b/docs/history/phase1-audit/evidence_index.md similarity index 100% rename from advisory/phase1_audit/evidence_index.md rename to docs/history/phase1-audit/evidence_index.md diff --git a/advisory/phase1_audit/phase1_audit.md b/docs/history/phase1-audit/phase1_audit.md similarity index 100% rename from advisory/phase1_audit/phase1_audit.md rename to docs/history/phase1-audit/phase1_audit.md diff --git a/advisory/phase1_audit/phase1_executive_deck.pptx b/docs/history/phase1-audit/phase1_executive_deck.pptx similarity index 100% rename from advisory/phase1_audit/phase1_executive_deck.pptx rename to docs/history/phase1-audit/phase1_executive_deck.pptx diff --git a/advisory/phase1_audit/phase1_flow.mmd b/docs/history/phase1-audit/phase1_flow.mmd similarity index 100% rename from advisory/phase1_audit/phase1_flow.mmd rename to docs/history/phase1-audit/phase1_flow.mmd diff --git a/advisory/phase1_audit/phase1_flow_doc.md b/docs/history/phase1-audit/phase1_flow_doc.md similarity index 100% rename from advisory/phase1_audit/phase1_flow_doc.md rename to docs/history/phase1-audit/phase1_flow_doc.md diff --git a/advisory/phase1_audit/phase1_flow_preview.html b/docs/history/phase1-audit/phase1_flow_preview.html similarity index 100% rename from advisory/phase1_audit/phase1_flow_preview.html rename to docs/history/phase1-audit/phase1_flow_preview.html diff --git a/advisory/phase1_audit/phase1_visual_story_v2/build_visual_story_v2.js b/docs/history/phase1-audit/phase1_visual_story_v2/build_visual_story_v2.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/build_visual_story_v2.js rename to docs/history/phase1-audit/phase1_visual_story_v2/build_visual_story_v2.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/notes.md b/docs/history/phase1-audit/phase1_visual_story_v2/notes.md similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/notes.md rename to docs/history/phase1-audit/phase1_visual_story_v2/notes.md diff --git a/advisory/phase1_audit/phase1_visual_story_v2/phase1_executive_deck_v2.pptx b/docs/history/phase1-audit/phase1_visual_story_v2/phase1_executive_deck_v2.pptx similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/phase1_executive_deck_v2.pptx rename to docs/history/phase1-audit/phase1_visual_story_v2/phase1_executive_deck_v2.pptx diff --git a/advisory/phase1_audit/phase1_visual_story_v2/phase1_visual_story_v2.html b/docs/history/phase1-audit/phase1_visual_story_v2/phase1_visual_story_v2.html similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/phase1_visual_story_v2.html rename to docs/history/phase1-audit/phase1_visual_story_v2/phase1_visual_story_v2.html diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-01-core-boundary.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-01-core-boundary.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-01-core-boundary.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-01-core-boundary.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-02-truth-vs-compatibility.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-02-truth-vs-compatibility.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-02-truth-vs-compatibility.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-02-truth-vs-compatibility.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-03-measurement-engine.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-03-measurement-engine.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-03-measurement-engine.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-03-measurement-engine.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-04-duplicate-work.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-04-duplicate-work.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-04-duplicate-work.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-04-duplicate-work.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-05-value-density.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-05-value-density.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-05-value-density.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-05-value-density.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-06-resource-allocation.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-06-resource-allocation.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-06-resource-allocation.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-06-resource-allocation.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/previews/slide-07-roadmap.svg.png b/docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-07-roadmap.svg.png similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/previews/slide-07-roadmap.svg.png rename to docs/history/phase1-audit/phase1_visual_story_v2/previews/slide-07-roadmap.svg.png diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-01-core-boundary.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-01-core-boundary.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-01-core-boundary.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-01-core-boundary.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-02-truth-vs-compatibility.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-02-truth-vs-compatibility.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-02-truth-vs-compatibility.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-02-truth-vs-compatibility.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-03-measurement-engine.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-03-measurement-engine.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-03-measurement-engine.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-03-measurement-engine.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-04-duplicate-work.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-04-duplicate-work.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-04-duplicate-work.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-04-duplicate-work.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-05-value-density.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-05-value-density.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-05-value-density.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-05-value-density.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-06-resource-allocation.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-06-resource-allocation.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-06-resource-allocation.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-06-resource-allocation.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/slides/slide-07-roadmap.svg b/docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-07-roadmap.svg similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/slides/slide-07-roadmap.svg rename to docs/history/phase1-audit/phase1_visual_story_v2/slides/slide-07-roadmap.svg diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/code.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/code.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/code.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/code.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/image.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/image.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/image.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/image.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/index.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/index.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/index.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/index.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/latex.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/latex.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/latex.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/latex.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout_builders.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout_builders.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout_builders.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/layout_builders.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/svg.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/svg.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/svg.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/svg.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/text.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/text.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/text.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/text.js diff --git a/advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/util.js b/docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/util.js similarity index 100% rename from advisory/phase1_audit/phase1_visual_story_v2/work/pptxgenjs_helpers/util.js rename to docs/history/phase1-audit/phase1_visual_story_v2/work/pptxgenjs_helpers/util.js diff --git a/advisory/phase1_audit/pipeline_map.md b/docs/history/phase1-audit/pipeline_map.md similarity index 100% rename from advisory/phase1_audit/pipeline_map.md rename to docs/history/phase1-audit/pipeline_map.md diff --git a/advisory/phase1_audit/resource_priority_matrix.md b/docs/history/phase1-audit/resource_priority_matrix.md similarity index 100% rename from advisory/phase1_audit/resource_priority_matrix.md rename to docs/history/phase1-audit/resource_priority_matrix.md diff --git a/advisory/phase1_audit/visual_prompt_pack.md b/docs/history/phase1-audit/visual_prompt_pack.md similarity index 100% rename from advisory/phase1_audit/visual_prompt_pack.md rename to docs/history/phase1-audit/visual_prompt_pack.md diff --git a/PLAN-phase1-hardening.md b/docs/history/phase1-hardening-plan.md similarity index 100% rename from PLAN-phase1-hardening.md rename to docs/history/phase1-hardening-plan.md diff --git a/scripts/calibrate_confidence.py b/scripts/calibrate_confidence.py index 7de7b41d..99b3e3c7 100755 --- a/scripts/calibrate_confidence.py +++ b/scripts/calibrate_confidence.py @@ -32,7 +32,7 @@ DEFAULT_OUTPUT_PATH = "docs/confidence_calibration_results.md" ANALYZE_SCRIPT_PATH = "apps/backend/analyze.py" -# Threshold ranges to test (as specified in OPTIMIZATION_PLAN.md) +# Threshold ranges to test (as specified in docs/history/optimization-plan.md) PITCH_CONFIDENCE_THRESHOLDS = [0.05, 0.10, 0.15, 0.20, 0.25] CHORD_STRENGTH_THRESHOLDS = [0.50, 0.60, 0.70, 0.80, 0.90] PUMPING_CONFIDENCE_THRESHOLDS = [0.20, 0.30, 0.40, 0.50] diff --git a/tests/ground_truth/README.md b/tests/ground_truth/README.md index 06194b8d..fd28821b 100644 --- a/tests/ground_truth/README.md +++ b/tests/ground_truth/README.md @@ -43,5 +43,5 @@ python3 scripts/calibrate_confidence.py \ ## Related files - `scripts/calibrate_confidence.py` — the calibration runner (writes to `docs/confidence_calibration_results.md` by default; create that file fresh on the next real-audio run) -- [`docs/archive/confidence-calibration-results-stubs.md`](../../docs/archive/confidence-calibration-results-stubs.md) — invalidated historical run against stub data, kept for reference +- [`docs/history/archive/confidence-calibration-results-stubs.md`](../../docs/history/archive/confidence-calibration-results-stubs.md) — invalidated historical run against stub data, kept for reference - `apps/backend/scripts/genre_corpus.md` — genre selection criteria