Harden backend uploads, add integration tests, remove dead fields

AGENTS.md

@@ -333,7 +333,7 @@ Backend allows these origins:
 
 - Temporary files are written to disk during analysis
 - Files are cleaned up after processing (success or error)
-- Maximum inline upload size: 100MB (base64 encoded)
+- Raw audio files at or below 100 MiB go inline to Gemini
 - Larger files use Gemini Files API
 
 ### Local Development Only

CLAUDE.md

@@ -76,11 +76,12 @@ Frontend polling: `src/services/analysisRunsClient.ts` creates runs and polls st
 
 ### Backend (`apps/backend`)
 
-**Two-file core plus runtime:**
+**Core files:**
 
 1. **`analyze.py`** (~112KB): Pure DSP pipeline. Runs as a subprocess invoked by `server.py`. Extracts BPM, key, LUFS, stereo width, spectral balance, rhythm/melody detail, transcription, stem separation. **Writes JSON to stdout, diagnostics to stderr** — this contract is load-bearing.
 2. **`server.py`** (~24KB): FastAPI HTTP wrapper. Accepts multipart uploads, invokes `analyze.py` as a subprocess, normalizes raw output into the `phase1` HTTP contract, returns structured JSON. Also hosts the staged run endpoints.
 3. **`analysis_runtime.py`**: SQLite-backed run state and stage queue management. Artifacts stored in `.runtime/artifacts/`.
+4. **`analyze_fast.py`**: Streamlined analysis pipeline (BPM, key, loudness, basic dynamics) invoked when `--fast` is passed to `analyze.py`.
 
 The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details.
 
@@ -99,7 +100,12 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V
 3. **`src/services/backendPhase2Client.ts`**: Phase 2 transport to `/api/phase2`.
 4. **`src/services/analyzer.ts`**: Phase orchestration entry point — sequences run creation, polling, and display payload projection.
 5. **`src/types.ts`**: Source of truth for `Phase1Result`, `Phase2Result`, `AnalysisRunSnapshot`, and all backend response shapes.
-6. **`src/config.ts`**: Runtime resolution of `VITE_API_BASE_URL` and feature flags; falls back to `http://127.0.0.1:8100`.
+6. **`src/config.ts`**: Runtime resolution of `VITE_API_BASE_URL` and feature flags; falls back to `http://127.0.0.1:8100`. Supports window-level overrides (`window.__VITE_API_BASE_URL_OVERRIDE__`, `window.__VITE_ENABLE_PHASE2_GEMINI_OVERRIDE__`) for hosted deployments that inject config at runtime without a rebuild.
+
+7. **`src/services/spectralArtifactsClient.ts`**: Fetches spectral artifact payloads from the backend.
+8. **`src/services/mixDoctor.ts`**: Mix advisory logic — client-side scoring and suggestions.
+9. **`src/services/phase2Validator.ts`**: Validates Phase 2 consistency against Phase 1.
+10. **`src/services/midi/`**: MIDI export, preview, and quantization utilities (`midiExport.ts`, `midiPreview.ts`, `quantization.ts`).
 
 `AnalysisResults.tsx` (~45KB) is lazy-loaded via Suspense. Manual vendor chunks in `vite.config.ts` control bundle splitting.
 
@@ -113,6 +119,8 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V
 # apps/ui/.env (copy from .env.example)
 VITE_API_BASE_URL="http://127.0.0.1:8100"
 VITE_ENABLE_PHASE2_GEMINI="true"
+RUN_GEMINI_LIVE_SMOKE="false"    # set "true" to run live Playwright tests against real Gemini Files API
+DISABLE_HMR="false"              # set "true" for dev environments that need HMR disabled
 
 # Backend (env var, no .env file)
 SONIC_ANALYZER_PORT=8100
@@ -129,7 +137,7 @@ Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-onl
 - **Backend tests use stdlib `unittest`**, not pytest. Frontend tests use Vitest in `node` environment (not jsdom).
 - **`npm run lint`** only type-checks `src/`; test files and `playwright.config.ts` are excluded from `tsconfig.json`.
 - **Canonical ports:** UI on 3100, backend on 8100. `./scripts/dev.sh` fails loudly if either port is occupied.
-- **`--fast` flag** is currently a no-op in `analyze.py` but is forwarded through the HTTP API via form field or query param.
+- **`--fast` flag** runs a streamlined pipeline (BPM, key, loudness, basic dynamics) via `analyze_fast.py`. It is forwarded through the HTTP API via form field or query param.
 - **`dsp_json_override`** is accepted by the server but ignored.
 
 ## Backport Candidates

README.md

@@ -162,7 +162,13 @@ cd apps/backend
 ./venv/bin/python -m unittest discover -s tests
 ```
 
-Canonical live end-to-end verification is local-only and requires a real audio file plus backend Gemini credentials:
+Canonical local end-to-end verification is local-only, boots the real backend, drives the UI against the canonical `analysis-runs` routes, and does not require Gemini credentials or a user-provided track:
+
+```bash
+./scripts/test-e2e-integration.sh
+```
+
+Full live Gemini end-to-end verification stays separate and requires a real audio file plus backend Gemini credentials:
 
 ```bash
 TEST_FLAC_PATH=/path/to/track.flac \
@@ -171,6 +177,29 @@ VITE_ENABLE_PHASE2_GEMINI=true \
 ./scripts/test-e2e.sh
 ```
 
+## Upload Limits
+
+For the backend upload routes, ASA now distinguishes between:
+
+- raw audio limit: `100 MiB`
+- HTTP request envelope limit: `101 MiB`
+
+In plain English: the audio file itself must stay at or below 100 MiB, but the
+whole multipart request is allowed to be slightly larger so filenames and form
+wrapping do not cause false `413` errors.
+
+The canonical operator view is generated from backend code, not maintained by
+hand. To see the current edge contract and proxy examples:
+
+```bash
+cd apps/backend
+./venv/bin/python scripts/render_upload_limit_contract.py
+```
+
+If you later put this local stack behind a reverse proxy or load balancer,
+mirror the generated `101 MiB` request-body limit there for the protected
+upload routes instead of copying stale numbers from old docs.
+
 ## Release Position
 
 The initial monorepo cut was **local/dev `v1.0.0`**. Current tags: `v1.2.0` (root), `ui-v1.6.0` (frontend).

apps/backend/AGENTS.md

@@ -166,7 +166,7 @@ python3.11 -m venv venv
 
 ## Known Gotchas
 
-- `--fast` is accepted by `analyze.py` but is currently a no-op.
+- `--fast` runs the reduced fast-analysis preset: core fields stay populated, while most detail-heavy fields remain `null`.
 - `dsp_json_override` is accepted by the server but ignored.
 - The server always appends `--yes` when invoking `analyze.py`.
 - README CORS docs have drifted before; trust `server.py` constants over prose if they disagree.

apps/backend/ARCHITECTURE.md

@@ -59,6 +59,11 @@ Custom routes:
 
 FastAPI-generated routes remain available at `/openapi.json`, `/docs`, and `/redoc`.
 
+The upload limit contract is the canonical source for the raw-audio limit, the
+request-envelope limit, the protected route list, and the edge proxy examples.
+In plain English: if those numbers ever change, operators should regenerate the
+contract instead of trusting old documentation.
+
 ## CLI Flow
 
 1. Parse the positional audio path and the optional flags `--separate`, `--transcribe`, `--fast`, and `--yes`.
@@ -89,42 +94,46 @@ Important sections:
 
 ### `POST /api/analysis-runs/estimate`
 
-1. Persist the uploaded file to a temporary path.
-2. Read duration metadata with `get_audio_duration_seconds()`.
-3. Resolve staged estimate flags from the requested run shape.
-4. Call `build_analysis_estimate(duration, run_separation, run_transcribe)`.
-5. Normalize stage keys into the server contract:
+1. Reject requests above the `101 MiB` request-envelope limit when `Content-Length` is present.
+2. For valid multipart uploads, count only the `track` part bytes toward the shared `100 MiB` raw-audio limit.
+3. Persist the uploaded file to a temporary path.
+4. Read duration metadata with `get_audio_duration_seconds()`.
+5. Resolve staged estimate flags from the requested run shape.
+6. Call `build_analysis_estimate(duration, run_separation, run_transcribe)`.
+7. Normalize stage keys into the server contract:
    - `dsp` -> `local_dsp`
    - `separation` -> `demucs_separation`
-6. Return:
+8. Return:
    - `requestId`
    - `estimate.durationSeconds`
    - `estimate.totalLowMs`
    - `estimate.totalHighMs`
    - `estimate.stages[]`
-7. Close the upload and delete the temporary file.
+9. Close the upload and delete the temporary file.
 
 ### `POST /api/analyze` (legacy compatibility wrapper)
 
-1. Persist the uploaded file to a temporary path.
-2. Build the same estimate object used by the estimate route.
-3. Convert the estimated upper bound into a timeout with a 15-second buffer.
-4. Build the subprocess command:
+1. Reject requests above the `101 MiB` request-envelope limit when `Content-Length` is present.
+2. For valid multipart uploads, count only the `track` part bytes toward the shared `100 MiB` raw-audio limit.
+3. Persist the uploaded file to a temporary path.
+4. Build the same estimate object used by the estimate route.
+5. Convert the estimated upper bound into a timeout with a 15-second buffer.
+6. Build the subprocess command:
    - base command: `./venv/bin/python analyze.py <temp_path> --yes`
    - add `--separate` when the query parameter is present
    - add `--transcribe` when the multipart form field is truthy
-5. Run the subprocess with `capture_output=True`.
-6. Handle failures with structured JSON error envelopes:
+7. Run the subprocess with `capture_output=True`.
+8. Handle failures with structured JSON error envelopes:
    - timeout
    - internal subprocess launch failure
    - non-zero exit
    - empty stdout
    - malformed JSON
    - non-object JSON
-7. Build `diagnostics.timings` from request wall time, subprocess wall time, flag usage, upload size, and analyzer-reported duration.
-8. Emit a `[TIMING]` summary line to `stderr` for every completed request, including structured errors.
-9. On success, normalize the raw payload into `phase1` and attach diagnostics.
-10. Close the upload and delete the temporary file.
+9. Build `diagnostics.timings` from request wall time, subprocess wall time, flag usage, upload size, and analyzer-reported duration.
+10. Emit a `[TIMING]` summary line to `stderr` for every completed request, including structured errors.
+11. On success, normalize the raw payload into `phase1` and attach diagnostics.
+12. Close the upload and delete the temporary file.
 
 ### Hosted foundation additions
 
@@ -255,14 +264,15 @@ When the analyzer never produces a valid JSON object, `timings.fileDurationSecon
 
 ### `POST /api/phase2` (legacy compatibility wrapper)
 
-1. Optionally use a cached temp file from a prior Phase 1 request (keyed by `phase1_request_id` form field); fall back to persisting the uploaded file if no cache hit.
-2. Parse `phase1_json` form field — used as-is (already normalized by the frontend).
-3. Build the Gemini prompt: system prompt from `prompts/phase2_system.txt` + Phase 1 JSON appended.
-4. Upload the audio inline (≤100 MiB) or via the Gemini Files API (>100 MiB).
-5. Call `generateContent` with structured output schema; retry on transient errors.
-6. Parse and validate the response against the Phase 2 schema.
-7. Return `{ requestId, phase2: Phase2Result | null, message, diagnostics }`.
-8. Clean up the temporary file in the `finally` block.
+1. Validate the uploaded audio against the shared backend upload limit.
+2. Resolve the server-owned analysis run from `analysis_run_id` or `phase1_request_id`.
+3. Parse `phase1_json` form field for compatibility only; canonical grounding comes from the server-owned run state.
+4. Build the Gemini prompt: system prompt from `prompts/phase2_system.txt` + grounded analysis data.
+5. Upload the audio inline (≤100 MiB) or via the Gemini Files API (>100 MiB).
+6. Call `generateContent` with structured output schema; retry on transient errors.
+7. Parse and validate the response against the Phase 2 schema.
+8. Return `{ requestId, phase2: Phase2Result | null, message, diagnostics }`.
+9. Clean up the temporary file in the `finally` block.
 
 ## Transcription Pipeline
 

apps/backend/README.md

@@ -73,7 +73,7 @@ Bootstrap contract for this monorepo `v1.0.0` cut:
 | `--separate` | Runs Demucs before melody analysis. If `--transcribe` is also enabled, the selected pitch backend uses the `bass` and `other` stems when they exist. |
 | `--transcribe` | Runs the selected pitch backend and returns `transcriptionDetail`. Without Demucs it transcribes the full mix; with Demucs it transcribes `bass` and `other` separately and merges the notes. |
 | `--pitch-note-backend BACKEND` | Selects the Layer 2 backend for `--pitch-note-only`. Supported values are `auto`, `torchcrepe-viterbi`, and alias `torchcrepe`. |
-| `--fast` | Accepted, but currently a no-op parser stub. |
+| `--fast` | Runs the reduced fast-analysis preset. Core fields such as BPM, key, duration, LUFS, true peak, and crest factor are populated; most detail-heavy fields remain `null`. |
 | `--yes` | Skips the interactive confirmation prompt after the CLI prints its runtime estimate. |
 
 ### Runtime Behavior
@@ -374,7 +374,7 @@ When the analyzer does not produce a valid payload, `diagnostics.timings.fileDur
 - `dsp_json_override` is accepted by both endpoints but is currently ignored by the backend.
 - The server timeout budget is derived from the estimate path and now reflects requested separation and transcription work.
 - `transcriptionDetail` is only present when `analyze.py` runs with `--transcribe`; otherwise it is `null`.
-- `--fast` is accepted by the CLI but does not change analysis behavior yet.
+- `--fast` runs the reduced fast-analysis preset instead of the full descriptor pass.
 
 ## Validation
 
@@ -383,7 +383,13 @@ When the analyzer does not produce a valid payload, `diagnostics.timings.fileDur
 ./venv/bin/python -m unittest discover -s tests
 ```
 
-Canonical live end-to-end verification is local-only and runs from the repo root with a real audio fixture plus backend Gemini credentials:
+Canonical local end-to-end verification is local-only and runs from the repo root without Gemini credentials or a user-provided audio file:
+
+```bash
+./scripts/test-e2e-integration.sh
+```
+
+The separate full live Gemini end-to-end verification still requires a real audio fixture plus backend Gemini credentials:
 
 ```bash
 TEST_FLAC_PATH=/path/to/track.flac \

apps/backend/analysis_runtime.py

@@ -211,11 +211,6 @@ def create_run(
         legacy_request_id: str | None = None,
         analysis_mode: str = "full",
     ) -> dict[str, Any]:
-        if self._count_active_measurement_runs() >= self.max_pending_per_stage:
-            raise RuntimeError("Measurement queue is full.")
-        resolved_analysis_mode = self._resolve_analysis_mode(analysis_mode)
-        self._resolve_pitch_note_backend(pitch_note_backend)
-        run_id = str(uuid4())
         artifact_id = str(uuid4())
         created_at = _utc_now_iso()
         stored_artifact = self.artifact_storage.store_bytes(

apps/backend/scripts/render_upload_limit_contract.py

@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+
+BACKEND_DIR = Path(__file__).resolve().parents[1]
+if str(BACKEND_DIR) not in sys.path:
+    sys.path.insert(0, str(BACKEND_DIR))
+
+import upload_limits
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="Render the canonical backend upload-limit contract.",
+    )
+    parser.add_argument(
+        "--format",
+        choices=("text", "json"),
+        default="text",
+        help="Output format. Use json for machine-readable output.",
+    )
+    args = parser.parse_args()
+
+    if args.format == "json":
+        print(json.dumps(upload_limits.build_upload_limit_contract(), indent=2))
+        return 0
+
+    print(upload_limits.render_upload_limit_contract_text())
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())