diff --git a/AGENTS.md b/AGENTS.md index 34a1d4fb..ddcd2c50 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -333,7 +333,7 @@ Backend allows these origins: - Temporary files are written to disk during analysis - Files are cleaned up after processing (success or error) -- Maximum inline upload size: 100MB (base64 encoded) +- Raw audio files at or below 100 MiB go inline to Gemini - Larger files use Gemini Files API ### Local Development Only diff --git a/CLAUDE.md b/CLAUDE.md index cc4b00d5..0fe43636 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,11 +76,12 @@ Frontend polling: `src/services/analysisRunsClient.ts` creates runs and polls st ### Backend (`apps/backend`) -**Two-file core plus runtime:** +**Core files:** 1. **`analyze.py`** (~112KB): Pure DSP pipeline. Runs as a subprocess invoked by `server.py`. Extracts BPM, key, LUFS, stereo width, spectral balance, rhythm/melody detail, transcription, stem separation. **Writes JSON to stdout, diagnostics to stderr** — this contract is load-bearing. 2. **`server.py`** (~24KB): FastAPI HTTP wrapper. Accepts multipart uploads, invokes `analyze.py` as a subprocess, normalizes raw output into the `phase1` HTTP contract, returns structured JSON. Also hosts the staged run endpoints. 3. **`analysis_runtime.py`**: SQLite-backed run state and stage queue management. Artifacts stored in `.runtime/artifacts/`. +4. **`analyze_fast.py`**: Streamlined analysis pipeline (BPM, key, loudness, basic dynamics) invoked when `--fast` is passed to `analyze.py`. The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details. @@ -99,7 +100,12 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V 3. **`src/services/backendPhase2Client.ts`**: Phase 2 transport to `/api/phase2`. 4. **`src/services/analyzer.ts`**: Phase orchestration entry point — sequences run creation, polling, and display payload projection. 5. **`src/types.ts`**: Source of truth for `Phase1Result`, `Phase2Result`, `AnalysisRunSnapshot`, and all backend response shapes. -6. **`src/config.ts`**: Runtime resolution of `VITE_API_BASE_URL` and feature flags; falls back to `http://127.0.0.1:8100`. +6. **`src/config.ts`**: Runtime resolution of `VITE_API_BASE_URL` and feature flags; falls back to `http://127.0.0.1:8100`. Supports window-level overrides (`window.__VITE_API_BASE_URL_OVERRIDE__`, `window.__VITE_ENABLE_PHASE2_GEMINI_OVERRIDE__`) for hosted deployments that inject config at runtime without a rebuild. + +7. **`src/services/spectralArtifactsClient.ts`**: Fetches spectral artifact payloads from the backend. +8. **`src/services/mixDoctor.ts`**: Mix advisory logic — client-side scoring and suggestions. +9. **`src/services/phase2Validator.ts`**: Validates Phase 2 consistency against Phase 1. +10. **`src/services/midi/`**: MIDI export, preview, and quantization utilities (`midiExport.ts`, `midiPreview.ts`, `quantization.ts`). `AnalysisResults.tsx` (~45KB) is lazy-loaded via Suspense. Manual vendor chunks in `vite.config.ts` control bundle splitting. @@ -113,6 +119,8 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V # apps/ui/.env (copy from .env.example) VITE_API_BASE_URL="http://127.0.0.1:8100" VITE_ENABLE_PHASE2_GEMINI="true" +RUN_GEMINI_LIVE_SMOKE="false" # set "true" to run live Playwright tests against real Gemini Files API +DISABLE_HMR="false" # set "true" for dev environments that need HMR disabled # Backend (env var, no .env file) SONIC_ANALYZER_PORT=8100 @@ -129,7 +137,7 @@ Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-onl - **Backend tests use stdlib `unittest`**, not pytest. Frontend tests use Vitest in `node` environment (not jsdom). - **`npm run lint`** only type-checks `src/`; test files and `playwright.config.ts` are excluded from `tsconfig.json`. - **Canonical ports:** UI on 3100, backend on 8100. `./scripts/dev.sh` fails loudly if either port is occupied. -- **`--fast` flag** is currently a no-op in `analyze.py` but is forwarded through the HTTP API via form field or query param. +- **`--fast` flag** runs a streamlined pipeline (BPM, key, loudness, basic dynamics) via `analyze_fast.py`. It is forwarded through the HTTP API via form field or query param. - **`dsp_json_override`** is accepted by the server but ignored. ## Backport Candidates diff --git a/README.md b/README.md index 42d9490e..7f3ffae6 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,13 @@ cd apps/backend ./venv/bin/python -m unittest discover -s tests ``` -Canonical live end-to-end verification is local-only and requires a real audio file plus backend Gemini credentials: +Canonical local end-to-end verification is local-only, boots the real backend, drives the UI against the canonical `analysis-runs` routes, and does not require Gemini credentials or a user-provided track: + +```bash +./scripts/test-e2e-integration.sh +``` + +Full live Gemini end-to-end verification stays separate and requires a real audio file plus backend Gemini credentials: ```bash TEST_FLAC_PATH=/path/to/track.flac \ @@ -171,6 +177,29 @@ VITE_ENABLE_PHASE2_GEMINI=true \ ./scripts/test-e2e.sh ``` +## Upload Limits + +For the backend upload routes, ASA now distinguishes between: + +- raw audio limit: `100 MiB` +- HTTP request envelope limit: `101 MiB` + +In plain English: the audio file itself must stay at or below 100 MiB, but the +whole multipart request is allowed to be slightly larger so filenames and form +wrapping do not cause false `413` errors. + +The canonical operator view is generated from backend code, not maintained by +hand. To see the current edge contract and proxy examples: + +```bash +cd apps/backend +./venv/bin/python scripts/render_upload_limit_contract.py +``` + +If you later put this local stack behind a reverse proxy or load balancer, +mirror the generated `101 MiB` request-body limit there for the protected +upload routes instead of copying stale numbers from old docs. + ## Release Position The initial monorepo cut was **local/dev `v1.0.0`**. Current tags: `v1.2.0` (root), `ui-v1.6.0` (frontend). diff --git a/apps/backend/AGENTS.md b/apps/backend/AGENTS.md index 16491be5..7149dabe 100644 --- a/apps/backend/AGENTS.md +++ b/apps/backend/AGENTS.md @@ -166,7 +166,7 @@ python3.11 -m venv venv ## Known Gotchas -- `--fast` is accepted by `analyze.py` but is currently a no-op. +- `--fast` runs the reduced fast-analysis preset: core fields stay populated, while most detail-heavy fields remain `null`. - `dsp_json_override` is accepted by the server but ignored. - The server always appends `--yes` when invoking `analyze.py`. - README CORS docs have drifted before; trust `server.py` constants over prose if they disagree. diff --git a/apps/backend/ARCHITECTURE.md b/apps/backend/ARCHITECTURE.md index a983d03d..c99acbb9 100644 --- a/apps/backend/ARCHITECTURE.md +++ b/apps/backend/ARCHITECTURE.md @@ -59,6 +59,11 @@ Custom routes: FastAPI-generated routes remain available at `/openapi.json`, `/docs`, and `/redoc`. +The upload limit contract is the canonical source for the raw-audio limit, the +request-envelope limit, the protected route list, and the edge proxy examples. +In plain English: if those numbers ever change, operators should regenerate the +contract instead of trusting old documentation. + ## CLI Flow 1. Parse the positional audio path and the optional flags `--separate`, `--transcribe`, `--fast`, and `--yes`. @@ -89,42 +94,46 @@ Important sections: ### `POST /api/analysis-runs/estimate` -1. Persist the uploaded file to a temporary path. -2. Read duration metadata with `get_audio_duration_seconds()`. -3. Resolve staged estimate flags from the requested run shape. -4. Call `build_analysis_estimate(duration, run_separation, run_transcribe)`. -5. Normalize stage keys into the server contract: +1. Reject requests above the `101 MiB` request-envelope limit when `Content-Length` is present. +2. For valid multipart uploads, count only the `track` part bytes toward the shared `100 MiB` raw-audio limit. +3. Persist the uploaded file to a temporary path. +4. Read duration metadata with `get_audio_duration_seconds()`. +5. Resolve staged estimate flags from the requested run shape. +6. Call `build_analysis_estimate(duration, run_separation, run_transcribe)`. +7. Normalize stage keys into the server contract: - `dsp` -> `local_dsp` - `separation` -> `demucs_separation` -6. Return: +8. Return: - `requestId` - `estimate.durationSeconds` - `estimate.totalLowMs` - `estimate.totalHighMs` - `estimate.stages[]` -7. Close the upload and delete the temporary file. +9. Close the upload and delete the temporary file. ### `POST /api/analyze` (legacy compatibility wrapper) -1. Persist the uploaded file to a temporary path. -2. Build the same estimate object used by the estimate route. -3. Convert the estimated upper bound into a timeout with a 15-second buffer. -4. Build the subprocess command: +1. Reject requests above the `101 MiB` request-envelope limit when `Content-Length` is present. +2. For valid multipart uploads, count only the `track` part bytes toward the shared `100 MiB` raw-audio limit. +3. Persist the uploaded file to a temporary path. +4. Build the same estimate object used by the estimate route. +5. Convert the estimated upper bound into a timeout with a 15-second buffer. +6. Build the subprocess command: - base command: `./venv/bin/python analyze.py --yes` - add `--separate` when the query parameter is present - add `--transcribe` when the multipart form field is truthy -5. Run the subprocess with `capture_output=True`. -6. Handle failures with structured JSON error envelopes: +7. Run the subprocess with `capture_output=True`. +8. Handle failures with structured JSON error envelopes: - timeout - internal subprocess launch failure - non-zero exit - empty stdout - malformed JSON - non-object JSON -7. Build `diagnostics.timings` from request wall time, subprocess wall time, flag usage, upload size, and analyzer-reported duration. -8. Emit a `[TIMING]` summary line to `stderr` for every completed request, including structured errors. -9. On success, normalize the raw payload into `phase1` and attach diagnostics. -10. Close the upload and delete the temporary file. +9. Build `diagnostics.timings` from request wall time, subprocess wall time, flag usage, upload size, and analyzer-reported duration. +10. Emit a `[TIMING]` summary line to `stderr` for every completed request, including structured errors. +11. On success, normalize the raw payload into `phase1` and attach diagnostics. +12. Close the upload and delete the temporary file. ### Hosted foundation additions @@ -255,14 +264,15 @@ When the analyzer never produces a valid JSON object, `timings.fileDurationSecon ### `POST /api/phase2` (legacy compatibility wrapper) -1. Optionally use a cached temp file from a prior Phase 1 request (keyed by `phase1_request_id` form field); fall back to persisting the uploaded file if no cache hit. -2. Parse `phase1_json` form field — used as-is (already normalized by the frontend). -3. Build the Gemini prompt: system prompt from `prompts/phase2_system.txt` + Phase 1 JSON appended. -4. Upload the audio inline (≤100 MiB) or via the Gemini Files API (>100 MiB). -5. Call `generateContent` with structured output schema; retry on transient errors. -6. Parse and validate the response against the Phase 2 schema. -7. Return `{ requestId, phase2: Phase2Result | null, message, diagnostics }`. -8. Clean up the temporary file in the `finally` block. +1. Validate the uploaded audio against the shared backend upload limit. +2. Resolve the server-owned analysis run from `analysis_run_id` or `phase1_request_id`. +3. Parse `phase1_json` form field for compatibility only; canonical grounding comes from the server-owned run state. +4. Build the Gemini prompt: system prompt from `prompts/phase2_system.txt` + grounded analysis data. +5. Upload the audio inline (≤100 MiB) or via the Gemini Files API (>100 MiB). +6. Call `generateContent` with structured output schema; retry on transient errors. +7. Parse and validate the response against the Phase 2 schema. +8. Return `{ requestId, phase2: Phase2Result | null, message, diagnostics }`. +9. Clean up the temporary file in the `finally` block. ## Transcription Pipeline diff --git a/apps/backend/README.md b/apps/backend/README.md index 74850fed..c71b9fbe 100644 --- a/apps/backend/README.md +++ b/apps/backend/README.md @@ -73,7 +73,7 @@ Bootstrap contract for this monorepo `v1.0.0` cut: | `--separate` | Runs Demucs before melody analysis. If `--transcribe` is also enabled, the selected pitch backend uses the `bass` and `other` stems when they exist. | | `--transcribe` | Runs the selected pitch backend and returns `transcriptionDetail`. Without Demucs it transcribes the full mix; with Demucs it transcribes `bass` and `other` separately and merges the notes. | | `--pitch-note-backend BACKEND` | Selects the Layer 2 backend for `--pitch-note-only`. Supported values are `auto`, `torchcrepe-viterbi`, and alias `torchcrepe`. | -| `--fast` | Accepted, but currently a no-op parser stub. | +| `--fast` | Runs the reduced fast-analysis preset. Core fields such as BPM, key, duration, LUFS, true peak, and crest factor are populated; most detail-heavy fields remain `null`. | | `--yes` | Skips the interactive confirmation prompt after the CLI prints its runtime estimate. | ### Runtime Behavior @@ -374,7 +374,7 @@ When the analyzer does not produce a valid payload, `diagnostics.timings.fileDur - `dsp_json_override` is accepted by both endpoints but is currently ignored by the backend. - The server timeout budget is derived from the estimate path and now reflects requested separation and transcription work. - `transcriptionDetail` is only present when `analyze.py` runs with `--transcribe`; otherwise it is `null`. -- `--fast` is accepted by the CLI but does not change analysis behavior yet. +- `--fast` runs the reduced fast-analysis preset instead of the full descriptor pass. ## Validation @@ -383,7 +383,13 @@ When the analyzer does not produce a valid payload, `diagnostics.timings.fileDur ./venv/bin/python -m unittest discover -s tests ``` -Canonical live end-to-end verification is local-only and runs from the repo root with a real audio fixture plus backend Gemini credentials: +Canonical local end-to-end verification is local-only and runs from the repo root without Gemini credentials or a user-provided audio file: + +```bash +./scripts/test-e2e-integration.sh +``` + +The separate full live Gemini end-to-end verification still requires a real audio fixture plus backend Gemini credentials: ```bash TEST_FLAC_PATH=/path/to/track.flac \ diff --git a/apps/backend/analysis_runtime.py b/apps/backend/analysis_runtime.py index b51f159f..7d27cfbb 100644 --- a/apps/backend/analysis_runtime.py +++ b/apps/backend/analysis_runtime.py @@ -211,11 +211,6 @@ def create_run( legacy_request_id: str | None = None, analysis_mode: str = "full", ) -> dict[str, Any]: - if self._count_active_measurement_runs() >= self.max_pending_per_stage: - raise RuntimeError("Measurement queue is full.") - resolved_analysis_mode = self._resolve_analysis_mode(analysis_mode) - self._resolve_pitch_note_backend(pitch_note_backend) - run_id = str(uuid4()) artifact_id = str(uuid4()) created_at = _utc_now_iso() stored_artifact = self.artifact_storage.store_bytes( diff --git a/apps/backend/scripts/render_upload_limit_contract.py b/apps/backend/scripts/render_upload_limit_contract.py new file mode 100644 index 00000000..d664165f --- /dev/null +++ b/apps/backend/scripts/render_upload_limit_contract.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + + +BACKEND_DIR = Path(__file__).resolve().parents[1] +if str(BACKEND_DIR) not in sys.path: + sys.path.insert(0, str(BACKEND_DIR)) + +import upload_limits + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Render the canonical backend upload-limit contract.", + ) + parser.add_argument( + "--format", + choices=("text", "json"), + default="text", + help="Output format. Use json for machine-readable output.", + ) + args = parser.parse_args() + + if args.format == "json": + print(json.dumps(upload_limits.build_upload_limit_contract(), indent=2)) + return 0 + + print(upload_limits.render_upload_limit_contract_text()) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/apps/backend/server.py b/apps/backend/server.py index 68476d78..57490273 100644 --- a/apps/backend/server.py +++ b/apps/backend/server.py @@ -10,7 +10,7 @@ import sys import tempfile import threading -from datetime import datetime, timedelta +from datetime import datetime from math import ceil from math import isfinite from pathlib import Path @@ -29,6 +29,7 @@ from fastapi import FastAPI, File, Form, Header, Query, UploadFile from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import FileResponse, JSONResponse +from python_multipart.multipart import MultipartParseError, MultipartParser, parse_options_header from auth_context import AuthenticationRequiredError, UserContext, resolve_api_user_context from analysis_runtime import ( @@ -62,6 +63,7 @@ DEFAULT_SERVER_HOST = "0.0.0.0" DEFAULT_SERVER_PORT = 8100 INLINE_SIZE_LIMIT = 104_857_600 # 100 MiB — confirmed by Google on 2026-01-12 +ARTIFACT_CLEANUP_INTERVAL_SECONDS = 3_600 WORKER_IDLE_SECONDS = 0.25 ALLOWED_GEMINI_MODELS = { "gemini-2.5-flash", @@ -131,6 +133,14 @@ LEGACY_ENDPOINT_SUNSET = "Wed, 31 Dec 2026 23:59:59 GMT" +class UploadTooLargeError(ValueError): + def __init__(self, limit_bytes: int): + self.limit_bytes = limit_bytes + super().__init__( + f"Uploaded audio exceeds the backend upload limit of {limit_bytes} bytes." + ) + + def _load_prompt_template(name: str) -> str: path = _PROMPTS_DIR / name try: @@ -212,6 +222,180 @@ def _mark_legacy_endpoint_response(response: JSONResponse, *, endpoint: str) -> return response +def _scope_header_value(scope: dict[str, Any], header_name: bytes) -> str | None: + for name, value in scope.get("headers", []): + if name.lower() != header_name: + continue + try: + return value.decode("latin-1") + except UnicodeDecodeError: + return None + return None + + +class _MultipartTrackSizeCounter: + def __init__(self, *, content_type_header: str | None, track_field_name: str, limit_bytes: int) -> None: + self.limit_bytes = limit_bytes + self.track_field_name = track_field_name + self.track_bytes = 0 + self.overflowed = False + self.disabled = False + self._current_headers: dict[bytes, bytes] = {} + self._current_header_field = bytearray() + self._current_header_value = bytearray() + self._current_part_is_track_file = False + self._parser = self._build_parser(content_type_header) + + def _build_parser(self, content_type_header: str | None) -> MultipartParser | None: + content_type, options = parse_options_header(content_type_header) + if content_type != b"multipart/form-data": + return None + boundary = options.get(b"boundary") + if not boundary: + return None + return MultipartParser( + boundary, + callbacks={ + "on_part_begin": self.on_part_begin, + "on_header_begin": self.on_header_begin, + "on_header_field": self.on_header_field, + "on_header_value": self.on_header_value, + "on_header_end": self.on_header_end, + "on_headers_finished": self.on_headers_finished, + "on_part_data": self.on_part_data, + "on_part_end": self.on_part_end, + }, + ) + + def feed(self, chunk: bytes) -> None: + if not chunk or self._parser is None or self.disabled or self.overflowed: + return + try: + self._parser.write(chunk) + except MultipartParseError: + self.disabled = True + + def on_part_begin(self) -> None: + self._current_headers = {} + self._current_header_field = bytearray() + self._current_header_value = bytearray() + self._current_part_is_track_file = False + + def on_header_begin(self) -> None: + self._current_header_field = bytearray() + self._current_header_value = bytearray() + + def on_header_field(self, data: bytes, start: int, end: int) -> None: + self._current_header_field.extend(data[start:end]) + + def on_header_value(self, data: bytes, start: int, end: int) -> None: + self._current_header_value.extend(data[start:end]) + + def on_header_end(self) -> None: + if not self._current_header_field: + return + self._current_headers[ + bytes(self._current_header_field).lower() + ] = bytes(self._current_header_value) + + def on_headers_finished(self) -> None: + disposition = self._current_headers.get(b"content-disposition") + if disposition is None: + return + _, options = parse_options_header(disposition) + field_name = options.get(b"name") + self._current_part_is_track_file = ( + field_name == self.track_field_name.encode("utf-8") + and b"filename" in options + ) + + def on_part_data(self, data: bytes, start: int, end: int) -> None: + if not self._current_part_is_track_file: + return + self.track_bytes += end - start + if self.track_bytes > self.limit_bytes: + self.overflowed = True + + def on_part_end(self) -> None: + self._current_part_is_track_file = False + + +class UploadSizeLimitMiddleware: + def __init__(self, app: Callable[..., Any]) -> None: + self.app = app + + async def __call__(self, scope: dict[str, Any], receive: Any, send: Any) -> None: + if ( + scope.get("type") != "http" + or scope.get("method") != "POST" + or scope.get("path") not in upload_limits.UPLOAD_LIMITED_POST_PATHS + ): + await self.app(scope, receive, send) + return + + raw_limit_bytes = upload_limits.MAX_UPLOAD_SIZE_BYTES + request_limit_bytes = upload_limits.MAX_UPLOAD_REQUEST_BYTES + content_length = _scope_header_value(scope, b"content-length") + try: + declared_length = int(content_length) if content_length is not None else None + except ValueError: + declared_length = None + if declared_length is not None and declared_length > request_limit_bytes: + response = _upload_too_large_response_for_path(scope["path"], limit_bytes=raw_limit_bytes) + await response(scope, receive, send) + return + + size_counter = _MultipartTrackSizeCounter( + content_type_header=_scope_header_value(scope, b"content-type"), + track_field_name="track", + limit_bytes=raw_limit_bytes, + ) + overflowed = False + response_started = False + synthetic_response_sent = False + + def send_upload_limit_response() -> JSONResponse: + return _upload_too_large_response_for_path(scope["path"], limit_bytes=raw_limit_bytes) + + async def limited_receive() -> dict[str, Any]: + nonlocal overflowed + if overflowed: + return {"type": "http.disconnect"} + message = await receive() + if message.get("type") != "http.request": + return message + body = message.get("body", b"") + if isinstance(body, bytes): + size_counter.feed(body) + if size_counter.overflowed: + overflowed = True + return {"type": "http.disconnect"} + return message + + async def tracked_send(message: dict[str, Any]) -> None: + nonlocal response_started, synthetic_response_sent + if overflowed: + if not synthetic_response_sent: + synthetic_response_sent = True + response_started = True + await send_upload_limit_response()(scope, receive, send) + return + if message.get("type") == "http.response.start": + response_started = True + await send(message) + + try: + await self.app(scope, limited_receive, tracked_send) + except UploadTooLargeError: + if response_started: + raise + await send_upload_limit_response()(scope, receive, send) + return + + if overflowed and not synthetic_response_sent and not response_started: + await send_upload_limit_response()(scope, receive, send) + + def _register_active_child_process( run_id: str, stage_key: str, @@ -668,6 +852,7 @@ def _interrupt_active_child_processes(run_id: str) -> list[str]: "http://127.0.0.1:5173", ] +app.add_middleware(UploadSizeLimitMiddleware) app.add_middleware( CORSMiddleware, allow_origins=ALLOWED_ORIGINS, @@ -739,7 +924,7 @@ def _create_background_tasks( @app.on_event("startup") -async def _start_cache_eviction() -> None: +async def _start_background_tasks() -> None: runtime = get_analysis_runtime() runtime_profile = resolve_runtime_profile() process_role = resolve_runtime_process_role(runtime_profile=runtime_profile) @@ -770,6 +955,15 @@ def _run_artifact_cleanup() -> None: ) +async def _artifact_cleanup_loop(runtime_dir: Path) -> None: + while True: + try: + await asyncio.to_thread(cleanup_artifacts, runtime_dir) + except Exception as exc: + logger.warning("[warn] artifact cleanup failed: %s", exc) + await asyncio.sleep(ARTIFACT_CLEANUP_INTERVAL_SECONDS) + + @app.on_event("shutdown") async def _stop_background_tasks() -> None: global _BACKGROUND_TASKS @@ -1063,18 +1257,36 @@ def _build_phase1(payload: dict[str, Any]) -> dict[str, Any]: } -def _persist_upload(track: UploadFile) -> tuple[str, int]: +def _persist_upload( + track: UploadFile, + *, + max_bytes: int | None = None, +) -> tuple[str, int]: + effective_max_bytes = ( + upload_limits.MAX_UPLOAD_SIZE_BYTES if max_bytes is None else max_bytes + ) suffix = Path(track.filename or "upload.bin").suffix or ".bin" - with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as temp_file: - temp_path = temp_file.name - total_bytes = 0 - while True: - chunk = track.file.read(1024 * 1024) - if not chunk: - break - total_bytes += len(chunk) - temp_file.write(chunk) - return temp_path, total_bytes + temp_path: str | None = None + try: + track.file.seek(0) + except Exception: + pass + try: + with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as temp_file: + temp_path = temp_file.name + total_bytes = 0 + while True: + chunk = track.file.read(1024 * 1024) + if not chunk: + break + total_bytes += len(chunk) + if total_bytes > effective_max_bytes: + raise UploadTooLargeError(effective_max_bytes) + temp_file.write(chunk) + return temp_path, total_bytes + except Exception: + _cleanup_temp_path(temp_path) + raise def _cleanup_temp_path(temp_path: str | None) -> None: @@ -1204,7 +1416,7 @@ async def _create_analysis_run_record( interpretation_model: str | None, legacy_request_id: str | None = None, ) -> tuple[AnalysisRuntime, str]: - content = await track.read() + temp_path: str | None = None runtime = get_analysis_runtime() analysis_mode = _resolve_analysis_mode_value(analysis_mode) if interpretation_mode != "off": @@ -1267,6 +1479,62 @@ def _value_error_code(value_error: ValueError) -> str: return "INTERPRETATION_PROFILE_UNSUPPORTED" +def _canonical_upload_too_large_response(exc: UploadTooLargeError) -> JSONResponse: + return JSONResponse( + status_code=413, + content={ + "error": { + "code": "UPLOAD_TOO_LARGE", + "message": str(exc), + } + }, + ) + + +def _legacy_upload_too_large_response( + *, + request_id: str, + phase: str, + endpoint: str, + analysis_run_id: str | None = None, +) -> JSONResponse: + content: dict[str, Any] = { + "requestId": request_id, + "error": { + "code": "UPLOAD_TOO_LARGE", + "message": upload_limits.upload_too_large_message(), + "phase": phase, + "retryable": False, + }, + } + if analysis_run_id is not None: + content["analysisRunId"] = analysis_run_id + return _mark_legacy_endpoint_response( + JSONResponse(status_code=413, content=content), + endpoint=endpoint, + ) + + +def _upload_too_large_response_for_path( + path: str, + *, + limit_bytes: int, +) -> JSONResponse: + if path in {"/api/analyze", "/api/analyze/estimate"}: + return _legacy_upload_too_large_response( + request_id=str(uuid4()), + phase=ERROR_PHASE_LOCAL_DSP, + endpoint=path, + ) + if path == "/api/phase2": + return _legacy_upload_too_large_response( + request_id=str(uuid4()), + phase=ERROR_PHASE_GEMINI, + endpoint=path, + ) + return _canonical_upload_too_large_response(UploadTooLargeError(limit_bytes)) + + def _resolve_analysis_mode_value(value: Any) -> str: if not isinstance(value, str): return "full" @@ -2560,6 +2828,8 @@ async def estimate_analysis_run( interpretation_profile=interpretation_profile, interpretation_model=interpretation_model, ) + except UploadTooLargeError as exc: + return _canonical_upload_too_large_response(exc) except UnsupportedPitchNoteModeError as exc: return JSONResponse( status_code=400, @@ -4942,18 +5212,25 @@ async def estimate_analysis( await track.close() return _mark_legacy_endpoint_response(user_context, endpoint="/api/analyze/estimate") _ = dsp_json_override - response = await _estimate_analysis_run( - track=track, - analysis_mode=analysis_mode, - pitch_note_mode=_resolve_pitch_note_mode_for_legacy(transcribe), - pitch_note_backend="auto", - interpretation_mode="off", - interpretation_profile="producer_summary", - interpretation_model=None, - run_separation_override=bool(separate or separate_query or separate_flag), - run_transcribe_override=bool(transcribe), - ) - return _mark_legacy_endpoint_response(response, endpoint="/api/analyze/estimate") + try: + response = await _estimate_analysis_run( + track=track, + analysis_mode=analysis_mode, + pitch_note_mode=_resolve_pitch_note_mode_for_legacy(transcribe), + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + run_separation_override=bool(separate or separate_query or separate_flag), + run_transcribe_override=bool(transcribe), + ) + return _mark_legacy_endpoint_response(response, endpoint="/api/analyze/estimate") + except UploadTooLargeError: + return _legacy_upload_too_large_response( + request_id=str(uuid4()), + phase=ERROR_PHASE_LOCAL_DSP, + endpoint="/api/analyze/estimate", + ) @app.post("/api/analyze") @@ -5036,6 +5313,12 @@ async def analyze_audio( "diagnostics": execution["diagnostics"], } ), endpoint="/api/analyze") + except UploadTooLargeError: + return _legacy_upload_too_large_response( + request_id=request_id, + phase=ERROR_PHASE_LOCAL_DSP, + endpoint="/api/analyze", + ) except RuntimeError as exc: return _mark_legacy_endpoint_response(JSONResponse( status_code=429, @@ -5085,12 +5368,15 @@ async def analyze_phase2( Infrastructure failures (timeout, auth, quota) return 4xx/5xx. """ request_id = str(uuid4()) + temp_path: str | None = None logger.warning("Legacy compatibility endpoint hit: /api/phase2 request_id=%s", request_id) user_context = _resolve_route_user_context(x_asa_user_id, x_asa_user_email) if isinstance(user_context, JSONResponse): await track.close() return _mark_legacy_endpoint_response(user_context, endpoint="/api/phase2") try: + temp_path, _ = _persist_upload(track) + if not _GENAI_AVAILABLE: return _mark_legacy_endpoint_response(JSONResponse( status_code=500, @@ -5227,6 +5513,12 @@ async def analyze_phase2( }, ), endpoint="/api/phase2") + except UploadTooLargeError: + return _legacy_upload_too_large_response( + request_id=request_id, + phase=ERROR_PHASE_GEMINI, + endpoint="/api/phase2", + ) except Exception as exc: return _build_phase2_error_response( request_id=request_id, @@ -5243,6 +5535,7 @@ async def analyze_phase2( stderr=str(exc), ) finally: + _cleanup_temp_path(temp_path) await track.close() diff --git a/apps/backend/tests/test_analysis_runtime.py b/apps/backend/tests/test_analysis_runtime.py index 998d6c5e..e7b85af2 100644 --- a/apps/backend/tests/test_analysis_runtime.py +++ b/apps/backend/tests/test_analysis_runtime.py @@ -1,6 +1,9 @@ +import hashlib +import sqlite3 import tempfile import unittest from pathlib import Path +from unittest.mock import patch class AnalysisRuntimeTests(unittest.TestCase): @@ -178,6 +181,64 @@ def resolve_local_path(self, storage_ref: str) -> Path | None: runtime.delete_run(created["runId"]) self.assertEqual(storage.deleted_refs, [f"memory://{source['artifactId']}/track.mp3"]) + def test_create_run_from_source_path_persists_streamed_source_artifact(self) -> None: + runtime = self._runtime() + source_path = Path(self.temp_dir.name) / "source-track.mp3" + source_bytes = b"streamed-audio-data" + source_path.write_bytes(source_bytes) + + created = runtime.create_run_from_source_path( + filename="track.mp3", + source_path=str(source_path), + mime_type="audio/mpeg", + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + ) + + snapshot = runtime.get_run(created["runId"]) + source_audio = snapshot["artifacts"]["sourceAudio"] + self.assertEqual(source_audio["filename"], "track.mp3") + self.assertEqual(source_audio["sizeBytes"], len(source_bytes)) + self.assertEqual( + source_audio["contentSha256"], + hashlib.sha256(source_bytes).hexdigest(), + ) + self.assertEqual(snapshot["stages"]["measurement"]["status"], "queued") + self.assertEqual(Path(source_audio["path"]).read_bytes(), source_bytes) + + def test_create_run_from_source_path_removes_partial_artifact_on_db_failure(self) -> None: + runtime = self._runtime() + source_path = Path(self.temp_dir.name) / "source-track.mp3" + source_path.write_bytes(b"streamed-audio-data") + + class FailingConnection: + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def execute(self, *_args, **_kwargs): + raise sqlite3.OperationalError("db write failed") + + with patch.object(runtime, "_connect", return_value=FailingConnection()): + with self.assertRaisesRegex(sqlite3.OperationalError, "db write failed"): + runtime.create_run_from_source_path( + filename="track.mp3", + source_path=str(source_path), + mime_type="audio/mpeg", + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + ) + + self.assertEqual(list(runtime.artifacts_dir.iterdir()), []) + def test_measurement_completion_strips_transcription_and_enqueues_pitch_note_stage( self, ) -> None: diff --git a/apps/backend/tests/test_cleanup.py b/apps/backend/tests/test_cleanup.py index 01ab9187..4cab631d 100644 --- a/apps/backend/tests/test_cleanup.py +++ b/apps/backend/tests/test_cleanup.py @@ -4,7 +4,7 @@ import unittest from datetime import datetime, timedelta from pathlib import Path -from unittest.mock import Mock, patch +from unittest.mock import AsyncMock, Mock, patch import server @@ -88,58 +88,82 @@ def _close_coro_and_return_task(self, coro): coro.close() return Mock(name="task") - def test_startup_schedules_artifact_cleanup_in_daemon_thread(self) -> None: + def test_artifact_cleanup_loop_runs_immediately_and_then_hourly(self) -> None: runtime = Mock() runtime.runtime_dir = Path("/tmp/asa-runtime") - thread = Mock() - captured_target = {} - def build_thread(*args, **kwargs): - captured_target["target"] = kwargs["target"] - return thread + async def run_to_thread(func, *args): + func(*args) - with patch.object(server, "_BACKGROUND_TASKS", []): - with patch.object(server, "get_analysis_runtime", return_value=runtime): - with patch.object(server.threading, "Thread", side_effect=build_thread) as thread_cls: - with patch.object(server.asyncio, "create_task", side_effect=self._close_coro_and_return_task): - with patch.object(server, "cleanup_artifacts", create=True) as cleanup_mock: - asyncio.run(server._start_cache_eviction()) - captured_target["target"]() + with ( + patch.object(server, "cleanup_artifacts") as cleanup_mock, + patch.object(server.asyncio, "to_thread", side_effect=run_to_thread), + patch.object( + server.asyncio, + "sleep", + side_effect=[None, asyncio.CancelledError()], + ), + ): + with self.assertRaises(asyncio.CancelledError): + asyncio.run(server._artifact_cleanup_loop(runtime.runtime_dir)) - runtime.recover_incomplete_attempts.assert_called_once_with() - thread_cls.assert_called_once() - self.assertTrue(thread_cls.call_args.kwargs["daemon"]) - self.assertEqual(thread_cls.call_args.kwargs["name"], "artifact-cleanup") - thread.start.assert_called_once_with() - cleanup_mock.assert_called_once_with(runtime.runtime_dir) + cleanup_mock.assert_any_call(runtime.runtime_dir) + self.assertEqual(cleanup_mock.call_count, 2) - def test_startup_cleanup_target_swallows_exceptions_with_warning_log(self) -> None: + def test_artifact_cleanup_loop_swallows_exceptions_with_warning_log(self) -> None: runtime = Mock() runtime.runtime_dir = Path("/tmp/asa-runtime") - thread = Mock() - captured_target = {} - def build_thread(*args, **kwargs): - captured_target["target"] = kwargs["target"] - return thread + cleanup_mock = Mock(side_effect=[RuntimeError("boom"), None]) - with patch.object(server, "_BACKGROUND_TASKS", []): - with patch.object(server, "get_analysis_runtime", return_value=runtime): - with patch.object(server.threading, "Thread", side_effect=build_thread): - with patch.object(server.asyncio, "create_task", side_effect=self._close_coro_and_return_task): - with patch.object( - server, - "cleanup_artifacts", - side_effect=RuntimeError("boom"), - create=True, - ): - with patch.object(server.logger, "warning") as warning_mock: - asyncio.run(server._start_cache_eviction()) - captured_target["target"]() + async def run_to_thread(func, *args): + return func(*args) + with ( + patch.object(server, "cleanup_artifacts", cleanup_mock), + patch.object(server.asyncio, "to_thread", side_effect=run_to_thread), + patch.object( + server.asyncio, + "sleep", + side_effect=[None, asyncio.CancelledError()], + ), + patch.object(server.logger, "warning") as warning_mock, + ): + with self.assertRaises(asyncio.CancelledError): + asyncio.run(server._artifact_cleanup_loop(runtime.runtime_dir)) + + self.assertEqual(cleanup_mock.call_count, 2) warning_mock.assert_called_once() self.assertIn("artifact cleanup failed", warning_mock.call_args.args[0].lower()) + def test_startup_schedules_recurring_cleanup_task_and_no_file_cache(self) -> None: + runtime = Mock() + runtime.runtime_dir = Path("/tmp/asa-runtime") + cleanup_loop = AsyncMock(return_value=None) + + with patch.object(server, "_BACKGROUND_TASKS", []): + with patch.object(server, "get_analysis_runtime", return_value=runtime): + with patch.object(server, "_artifact_cleanup_loop", cleanup_loop, create=True): + with ( + patch.object( + server.asyncio, + "create_task", + side_effect=self._close_coro_and_return_task, + ) as create_task_mock, + patch.object(server.logger, "info") as info_mock, + ): + asyncio.run(server._start_background_tasks()) + + runtime.recover_incomplete_attempts.assert_called_once_with() + cleanup_loop.assert_called_once_with(runtime.runtime_dir) + self.assertEqual(create_task_mock.call_count, 4) + info_mock.assert_called_once_with( + "Upload limits configured: raw_audio_limit_bytes=%s edge_request_limit_bytes=%s", + server.upload_limits.MAX_UPLOAD_SIZE_BYTES, + server.upload_limits.MAX_UPLOAD_REQUEST_BYTES, + ) + self.assertFalse(hasattr(server, "_FILE_CACHE")) + def test_hosted_startup_skips_in_process_worker_loops_by_default(self) -> None: runtime = Mock() runtime.runtime_dir = Path("/tmp/asa-runtime") diff --git a/apps/backend/tests/test_root_e2e_script.py b/apps/backend/tests/test_root_e2e_script.py index c2f8bd59..e6dac3f1 100644 --- a/apps/backend/tests/test_root_e2e_script.py +++ b/apps/backend/tests/test_root_e2e_script.py @@ -19,7 +19,9 @@ def setUp(self) -> None: self.workspace = Path(self.temp_dir.name) / "repo" self.track_path = Path(self.temp_dir.name) / "reference-track.flac" self.track_path.write_bytes(b"fLaC") + self.npm_log_path = Path(self.temp_dir.name) / "npm-call.log" + (self.workspace / "bin").mkdir(parents=True, exist_ok=True) (self.workspace / "scripts").mkdir(parents=True, exist_ok=True) (self.workspace / "apps" / "backend" / "venv" / "bin").mkdir(parents=True, exist_ok=True) (self.workspace / "apps" / "ui").mkdir(parents=True, exist_ok=True) @@ -30,6 +32,18 @@ def setUp(self) -> None: self.workspace / "apps" / "backend" / "venv" / "bin" / "python", "#!/usr/bin/env bash\nexit 0\n", ) + _write_executable( + self.workspace / "bin" / "python3", + "#!/usr/bin/env bash\nexit 0\n", + ) + _write_executable( + self.workspace / "bin" / "npm", + ( + "#!/usr/bin/env bash\n" + f'printf "%s|%s" "$*" "${{VITE_ENABLE_PHASE2_GEMINI:-}}" > "{self.npm_log_path}"\n' + "exit 0\n" + ), + ) def tearDown(self) -> None: self.temp_dir.cleanup() @@ -37,6 +51,7 @@ def tearDown(self) -> None: def _run_script(self, extra_env: dict[str, str] | None = None) -> subprocess.CompletedProcess[str]: env = os.environ.copy() env.update(extra_env or {}) + env["PATH"] = f"{self.workspace / 'bin'}:{env.get('PATH', '')}" return subprocess.run( ["/bin/bash", str(self.workspace / "scripts" / "test-e2e.sh")], @@ -47,6 +62,29 @@ def _run_script(self, extra_env: dict[str, str] | None = None) -> subprocess.Com check=False, ) + def _copy_script_to_workspace(self, script_name: str) -> Path: + source = self.repo_root / "scripts" / script_name + destination = self.workspace / "scripts" / script_name + if source.exists(): + _write_executable(destination, source.read_text(encoding="utf-8")) + return destination + + def _run_named_script( + self, script_name: str, extra_env: dict[str, str] | None = None + ) -> subprocess.CompletedProcess[str]: + env = os.environ.copy() + env.update(extra_env or {}) + env["PATH"] = f"{self.workspace / 'bin'}:{env.get('PATH', '')}" + + return subprocess.run( + ["/bin/bash", str(self.workspace / "scripts" / script_name)], + cwd=self.workspace, + capture_output=True, + text=True, + env=env, + check=False, + ) + def test_requires_phase2_flag_before_running(self) -> None: completed = self._run_script( { @@ -87,6 +125,36 @@ def test_script_verifies_canonical_analysis_run_routes(self) -> None: self.assertIn("/api/analysis-runs", script_text) self.assertIn("/api/analysis-runs/{run_id}", script_text) + def test_integration_script_exists_and_runs_without_live_gemini_prerequisites(self) -> None: + integration_script = self._copy_script_to_workspace("test-e2e-integration.sh") + self.assertTrue( + integration_script.exists(), + "Expected scripts/test-e2e-integration.sh to exist at the repo root.", + ) + + completed = self._run_named_script("test-e2e-integration.sh") + + self.assertEqual(completed.returncode, 0, completed.stderr) + self.assertFalse("GEMINI_API_KEY must be set to a real Gemini API key" in completed.stderr) + self.assertFalse("TEST_FLAC_PATH must point to a readable audio file" in completed.stderr) + self.assertFalse("VITE_ENABLE_PHASE2_GEMINI must be set to true" in completed.stderr) + self.assertEqual( + self.npm_log_path.read_text(encoding="utf-8"), + "run test:e2e:integration|false", + ) + + def test_integration_script_verifies_canonical_analysis_run_routes(self) -> None: + integration_script = self._copy_script_to_workspace("test-e2e-integration.sh") + self.assertTrue( + integration_script.exists(), + "Expected scripts/test-e2e-integration.sh to exist at the repo root.", + ) + + script_text = integration_script.read_text(encoding="utf-8") + self.assertIn("/api/analysis-runs/estimate", script_text) + self.assertIn("/api/analysis-runs", script_text) + self.assertIn("/api/analysis-runs/{run_id}", script_text) + if __name__ == "__main__": unittest.main() diff --git a/apps/backend/tests/test_server.py b/apps/backend/tests/test_server.py index e514171f..11743e22 100644 --- a/apps/backend/tests/test_server.py +++ b/apps/backend/tests/test_server.py @@ -10,6 +10,7 @@ from fastapi.responses import JSONResponse from fastapi import UploadFile +from starlette.testclient import TestClient import server @@ -783,6 +784,49 @@ def test_get_analysis_run_rejects_wrong_owner_in_hosted_mode(self) -> None: self.assertEqual(response.status_code, 404) self.assertEqual(payload["error"]["code"], "RUN_NOT_FOUND") + def test_analysis_runs_endpoint_rejects_oversized_upload(self) -> None: + with patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 4): + response = asyncio.run( + server.create_analysis_run( + track=UploadFile(filename="track.mp3", file=io.BytesIO(b"12345")), + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + ) + ) + + self.assertEqual(response.status_code, 413) + payload = self._decode_json_response(response) + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertIn("4", payload["error"]["message"]) + + def test_analysis_runs_endpoint_streams_upload_without_async_read(self) -> None: + from analysis_runtime import AnalysisRuntime + + with tempfile.TemporaryDirectory(prefix="asa_server_runtime_") as temp_dir: + runtime = AnalysisRuntime(Path(temp_dir) / "runtime") + track = UploadFile(filename="track.mp3", file=io.BytesIO(b"fake-audio")) + track.read = AsyncMock(side_effect=AssertionError("track.read should not be used")) + with patch.object(server, "get_analysis_runtime", return_value=runtime): + response = asyncio.run( + server.create_analysis_run( + track=track, + analysis_mode="full", + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + ) + ) + + self.assertEqual(response.status_code, 200) + payload = self._decode_json_response(response) + self.assertEqual(payload["stages"]["measurement"]["status"], "queued") + track.read.assert_not_awaited() + @patch.object(server, "get_audio_duration_seconds", return_value=214.6, create=True) @patch.object( server, @@ -834,6 +878,25 @@ def test_analysis_runs_estimate_endpoint_uses_staged_request_fields( ) build_estimate_mock.assert_called_once_with(214.6, True, True, run_standard=True) + def test_analysis_runs_estimate_endpoint_rejects_oversized_upload(self) -> None: + with patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 4): + response = asyncio.run( + server.estimate_analysis_run( + track=UploadFile(filename="track.mp3", file=io.BytesIO(b"12345")), + analysis_mode="full", + pitch_note_mode="off", + pitch_note_backend="auto", + interpretation_mode="off", + interpretation_profile="producer_summary", + interpretation_model=None, + ) + ) + + self.assertEqual(response.status_code, 413) + payload = self._decode_json_response(response) + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertIn("4", payload["error"]["message"]) + def test_analysis_runs_estimate_rejects_unknown_pitch_note_mode(self) -> None: response = asyncio.run( server.estimate_analysis_run( @@ -2855,6 +2918,363 @@ def test_response_includes_request_id(self) -> None: self.assertIsInstance(body["requestId"], str) self.assertGreater(len(body["requestId"]), 0) + def test_phase2_rejects_oversized_upload_with_legacy_error_envelope(self) -> None: + with patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 4): + response = self._call(content=b"12345") + + self.assertEqual(response.status_code, 413) + body = self._decode(response) + self.assertIn("requestId", body) + self.assertEqual(body["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(body["error"]["phase"], server.ERROR_PHASE_GEMINI) + self.assertFalse(body["error"]["retryable"]) + + +class UploadSizeLimitMiddlewareTests(unittest.TestCase): + _ALLOWED_ORIGIN = "http://127.0.0.1:3100" + + def _client(self) -> TestClient: + return TestClient( + server.app, + raise_server_exceptions=False, + headers={"origin": self._ALLOWED_ORIGIN}, + ) + + def _post_file( + self, + client: TestClient, + path: str, + *, + content: bytes = b"12345", + data: dict[str, str] | None = None, + ): + return client.post( + path, + data=data or {}, + files={"track": ("track.mp3", content, "audio/mpeg")}, + ) + + def _chunked_multipart_body(self, boundary: str, file_chunks: list[bytes]) -> list[bytes]: + return [ + ( + f"--{boundary}\r\n" + 'Content-Disposition: form-data; name="track"; filename="track.mp3"\r\n' + "Content-Type: audio/mpeg\r\n\r\n" + ).encode("utf-8"), + *file_chunks, + f"\r\n--{boundary}--\r\n".encode("utf-8"), + ] + + def _dispatch_asgi( + self, + *, + path: str, + headers: list[tuple[bytes, bytes]], + request_messages: list[dict[str, object]], + ) -> tuple[int, dict[str, str], bytes]: + sent_messages: list[dict[str, object]] = [] + + async def invoke() -> None: + pending_messages = list(request_messages) + + async def receive() -> dict[str, object]: + if pending_messages: + return pending_messages.pop(0) + return {"type": "http.disconnect"} + + async def send(message: dict[str, object]) -> None: + sent_messages.append(message) + + await server.app( + { + "type": "http", + "asgi": {"version": "3.0"}, + "http_version": "1.1", + "method": "POST", + "scheme": "http", + "path": path, + "raw_path": path.encode("utf-8"), + "query_string": b"", + "headers": headers, + "client": ("testclient", 50000), + "server": ("testserver", 80), + }, + receive, + send, + ) + + asyncio.run(invoke()) + + start_message = next( + message for message in sent_messages if message["type"] == "http.response.start" + ) + body = b"".join( + message.get("body", b"") + for message in sent_messages + if message["type"] == "http.response.body" + ) + response_headers = { + name.decode("latin-1"): value.decode("latin-1") + for name, value in start_message["headers"] + } + return start_message["status"], response_headers, body + + def test_middleware_rejects_header_based_oversized_upload_for_analysis_runs(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 1_024), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4), + patch.object( + server, + "_persist_upload", + side_effect=AssertionError("_persist_upload should not be called"), + ), + ): + client = self._client() + response = self._post_file(client, "/api/analysis-runs") + client.close() + + self.assertEqual(response.status_code, 413) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertNotIn("requestId", payload) + + def test_middleware_rejects_header_based_oversized_upload_for_analysis_run_estimate(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 1_024), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4), + patch.object( + server, + "_persist_upload", + side_effect=AssertionError("_persist_upload should not be called"), + ), + ): + client = self._client() + response = self._post_file(client, "/api/analysis-runs/estimate") + client.close() + + self.assertEqual(response.status_code, 413) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertNotIn("requestId", payload) + + def test_middleware_rejects_header_based_oversized_upload_for_analyze(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 1_024), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4), + ): + client = self._client() + response = self._post_file(client, "/api/analyze") + client.close() + + self.assertEqual(response.status_code, 413) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + self.assertEqual(response.headers["Deprecation"], "true") + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(payload["error"]["phase"], server.ERROR_PHASE_LOCAL_DSP) + self.assertFalse(payload["error"]["retryable"]) + self.assertIn("requestId", payload) + + def test_middleware_rejects_header_based_oversized_upload_for_analyze_estimate(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 1_024), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4), + ): + client = self._client() + response = self._post_file(client, "/api/analyze/estimate") + client.close() + + self.assertEqual(response.status_code, 413) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + self.assertEqual(response.headers["Deprecation"], "true") + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertIn("requestId", payload) + + def test_middleware_rejects_header_based_oversized_upload_for_phase2(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 1_024), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4), + ): + client = self._client() + response = self._post_file(client, "/api/phase2") + client.close() + + self.assertEqual(response.status_code, 413) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + self.assertEqual(response.headers["Deprecation"], "true") + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(payload["error"]["phase"], server.ERROR_PHASE_GEMINI) + self.assertFalse(payload["error"]["retryable"]) + self.assertIn("requestId", payload) + + def test_middleware_rejects_streamed_overage_without_content_length(self) -> None: + boundary = "transport-limit-boundary" + stream_chunks = self._chunked_multipart_body( + boundary, + [b"a" * 100, b"b" * 100, b"c" * 100], + ) + request_messages = [ + { + "type": "http.request", + "body": chunk, + "more_body": index < len(stream_chunks) - 1, + } + for index, chunk in enumerate(stream_chunks) + ] + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 256), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 4_096), + patch.object( + server, + "_persist_upload", + side_effect=AssertionError("_persist_upload should not be called"), + ), + ): + status_code, headers, body = self._dispatch_asgi( + path="/api/analysis-runs", + headers=[ + (b"origin", self._ALLOWED_ORIGIN.encode("utf-8")), + ( + b"content-type", + f"multipart/form-data; boundary={boundary}".encode("utf-8"), + ), + ], + request_messages=request_messages, + ) + + self.assertEqual(status_code, 413) + self.assertEqual(headers["access-control-allow-origin"], self._ALLOWED_ORIGIN) + payload = json.loads(body.decode("utf-8")) + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + + def test_middleware_allows_near_limit_track_even_with_multipart_overhead(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 256), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 1_024), + patch.object(server, "_persist_upload", return_value=("/tmp/near-limit.wav", 256)), + patch.object(server, "get_audio_duration_seconds", return_value=60.0, create=True), + patch.object( + server, + "build_analysis_estimate", + return_value={ + "durationSeconds": 60.0, + "totalSeconds": {"min": 10, "max": 20}, + "stages": [ + { + "key": "dsp", + "label": "DSP analysis", + "seconds": {"min": 10, "max": 20}, + } + ], + }, + create=True, + ), + ): + client = self._client() + response = client.post( + "/api/analysis-runs/estimate", + data={"analysis_mode": "full", "note": "x" * 40}, + files={ + "track": ( + "track-with-extra-multipart-overhead-and-a-very-long-name.mp3", + b"a" * 256, + "audio/mpeg", + ) + }, + ) + client.close() + + self.assertEqual(response.status_code, 200) + self.assertEqual( + response.headers["access-control-allow-origin"], + self._ALLOWED_ORIGIN, + ) + + def test_middleware_rejects_track_one_byte_over_raw_limit_for_canonical_route(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 256), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 2_048), + patch.object( + server, + "_persist_upload", + side_effect=AssertionError("_persist_upload should not be called"), + ), + ): + client = self._client() + response = self._post_file( + client, + "/api/analysis-runs", + content=b"a" * 257, + ) + client.close() + + self.assertEqual(response.status_code, 413) + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertNotIn("requestId", payload) + + def test_middleware_rejects_track_one_byte_over_raw_limit_for_legacy_route(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 256), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 2_048), + patch.object( + server, + "_persist_upload", + side_effect=AssertionError("_persist_upload should not be called"), + ), + ): + client = self._client() + response = self._post_file( + client, + "/api/analyze", + content=b"a" * 257, + ) + client.close() + + self.assertEqual(response.status_code, 413) + payload = response.json() + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(payload["error"]["phase"], server.ERROR_PHASE_LOCAL_DSP) + self.assertFalse(payload["error"]["retryable"]) + self.assertIn("requestId", payload) + + def test_middleware_leaves_malformed_multipart_as_parse_error(self) -> None: + with ( + patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 256), + patch.object(server.upload_limits, "MAX_UPLOAD_REQUEST_BYTES", 2_048), + ): + client = self._client() + response = client.post( + "/api/analysis-runs", + content=b"not-a-valid-multipart-body", + headers={"content-type": "multipart/form-data"}, + ) + client.close() + + self.assertEqual(response.status_code, 400) + payload = response.json() + self.assertIn("detail", payload) + self.assertNotIn("error", payload) + class AnalysisRunCompatibilityTests(unittest.TestCase): """Tests for legacy wrappers backed by analysis runs.""" @@ -2928,6 +3348,51 @@ def test_analyze_returns_analysis_run_id_and_persists_measurement(self, *_mocks) self.assertEqual(snapshot["stages"]["measurement"]["status"], "completed") self.assertEqual(snapshot["stages"]["measurement"]["result"]["bpm"], 128) + def test_analyze_rejects_oversized_upload_with_legacy_error_envelope(self) -> None: + with patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 4): + response = asyncio.run( + server.analyze_audio( + track=UploadFile(filename="track.mp3", file=io.BytesIO(b"12345")), + dsp_json_override=None, + analysis_mode="full", + transcribe=False, + separate=False, + separate_query=False, + separate_flag=False, + fast=False, + fast_query=False, + ) + ) + + self.assertEqual(response.status_code, 413) + payload = self._decode(response) + self.assertIn("requestId", payload) + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(payload["error"]["phase"], server.ERROR_PHASE_LOCAL_DSP) + self.assertFalse(payload["error"]["retryable"]) + self.assertEqual(response.headers["Deprecation"], "true") + + def test_analyze_estimate_rejects_oversized_upload_with_legacy_headers(self) -> None: + with patch.object(server.upload_limits, "MAX_UPLOAD_SIZE_BYTES", 4): + response = asyncio.run( + server.estimate_analysis( + track=UploadFile(filename="track.mp3", file=io.BytesIO(b"12345")), + dsp_json_override=None, + analysis_mode="full", + transcribe=False, + separate=False, + separate_query=False, + separate_flag=False, + ) + ) + + self.assertEqual(response.status_code, 413) + payload = self._decode(response) + self.assertIn("requestId", payload) + self.assertEqual(payload["error"]["code"], "UPLOAD_TOO_LARGE") + self.assertEqual(response.headers["Deprecation"], "true") + self.assertEqual(response.headers["Link"], '; rel="successor-version"') + @patch.object(server, "get_audio_duration_seconds", return_value=60.0, create=True) @patch.object( server, diff --git a/apps/backend/tests/test_upload_limits.py b/apps/backend/tests/test_upload_limits.py new file mode 100644 index 00000000..e3941627 --- /dev/null +++ b/apps/backend/tests/test_upload_limits.py @@ -0,0 +1,81 @@ +import json +import subprocess +import sys +import unittest +from pathlib import Path + +import upload_limits + + +BACKEND_DIR = Path(__file__).resolve().parents[1] +REPO_ROOT = BACKEND_DIR.parents[1] + + +class UploadLimitContractTests(unittest.TestCase): + def test_contract_constants_match_expected_values(self) -> None: + self.assertEqual(upload_limits.MAX_UPLOAD_SIZE_BYTES, 104_857_600) + self.assertEqual(upload_limits.UPLOAD_REQUEST_SIZE_SLACK_BYTES, 1_048_576) + self.assertEqual(upload_limits.MAX_UPLOAD_REQUEST_BYTES, 105_906_176) + self.assertEqual( + upload_limits.UPLOAD_LIMITED_POST_PATHS, + ( + "/api/analysis-runs", + "/api/analysis-runs/estimate", + "/api/analyze", + "/api/analyze/estimate", + "/api/phase2", + ), + ) + + def test_generator_outputs_machine_readable_json(self) -> None: + result = subprocess.run( + [ + sys.executable, + "scripts/render_upload_limit_contract.py", + "--format", + "json", + ], + cwd=BACKEND_DIR, + capture_output=True, + text=True, + check=True, + ) + payload = json.loads(result.stdout) + self.assertEqual(payload["rawAudioLimitBytes"], upload_limits.MAX_UPLOAD_SIZE_BYTES) + self.assertEqual(payload["requestEnvelopeLimitBytes"], upload_limits.MAX_UPLOAD_REQUEST_BYTES) + self.assertEqual(payload["protectedPostRoutes"], list(upload_limits.UPLOAD_LIMITED_POST_PATHS)) + + def test_generator_outputs_plain_english_and_proxy_snippets(self) -> None: + result = subprocess.run( + [ + sys.executable, + "scripts/render_upload_limit_contract.py", + ], + cwd=BACKEND_DIR, + capture_output=True, + text=True, + check=True, + ) + self.assertIn("Plain English", result.stdout) + self.assertIn("nginx", result.stdout) + self.assertIn("Caddy", result.stdout) + self.assertIn("Traefik", result.stdout) + self.assertIn(str(upload_limits.MAX_UPLOAD_REQUEST_BYTES), result.stdout) + + def test_docs_reference_generator_and_current_contract_values(self) -> None: + root_readme = (REPO_ROOT / "README.md").read_text(encoding="utf-8") + backend_readme = (BACKEND_DIR / "README.md").read_text(encoding="utf-8") + architecture_doc = (BACKEND_DIR / "ARCHITECTURE.md").read_text(encoding="utf-8") + + expected_command = "./venv/bin/python scripts/render_upload_limit_contract.py" + self.assertIn(expected_command, root_readme) + self.assertIn(expected_command, backend_readme) + self.assertIn("upload limit contract", architecture_doc.lower()) + + self.assertIn(str(upload_limits.MAX_UPLOAD_REQUEST_BYTES), backend_readme) + self.assertIn(f"{upload_limits.MAX_UPLOAD_REQUEST_BYTES:,}".replace(",", ""), backend_readme) + self.assertIn(f"{upload_limits.MAX_UPLOAD_SIZE_BYTES // (1024 * 1024)} MiB", root_readme) + + +if __name__ == "__main__": + unittest.main() diff --git a/apps/backend/upload_limits.py b/apps/backend/upload_limits.py new file mode 100644 index 00000000..d3aacd72 --- /dev/null +++ b/apps/backend/upload_limits.py @@ -0,0 +1,105 @@ +from __future__ import annotations + +import json +from typing import Any + + +MAX_UPLOAD_SIZE_BYTES = 104_857_600 +UPLOAD_REQUEST_SIZE_SLACK_BYTES = 1_048_576 +MAX_UPLOAD_REQUEST_BYTES = MAX_UPLOAD_SIZE_BYTES + UPLOAD_REQUEST_SIZE_SLACK_BYTES +UPLOAD_LIMITED_POST_PATHS = ( + "/api/analysis-runs", + "/api/analysis-runs/estimate", + "/api/analyze", + "/api/analyze/estimate", + "/api/phase2", +) + +_BYTES_PER_MIB = 1024 * 1024 + + +def bytes_to_mib(value: int) -> int: + return value // _BYTES_PER_MIB + + +def bytes_to_mib_label(value: int) -> str: + return f"{bytes_to_mib(value)} MiB" + + +def upload_too_large_message(limit_bytes: int = MAX_UPLOAD_SIZE_BYTES) -> str: + return f"Uploaded audio exceeds the backend upload limit of {limit_bytes} bytes." + + +def build_proxy_snippets() -> dict[str, str]: + request_limit_mib = bytes_to_mib(MAX_UPLOAD_REQUEST_BYTES) + return { + "nginx": f"client_max_body_size {request_limit_mib}m;", + "Caddy": ( + "request_body {\n" + f" max_size {MAX_UPLOAD_REQUEST_BYTES}\n" + "}" + ), + "Traefik": ( + "http:\n" + " middlewares:\n" + " asa-upload-limit:\n" + " buffering:\n" + f" maxRequestBodyBytes: {MAX_UPLOAD_REQUEST_BYTES}" + ), + } + + +def build_upload_limit_contract() -> dict[str, Any]: + return { + "rawAudioLimitBytes": MAX_UPLOAD_SIZE_BYTES, + "rawAudioLimitMiB": bytes_to_mib(MAX_UPLOAD_SIZE_BYTES), + "requestEnvelopeLimitBytes": MAX_UPLOAD_REQUEST_BYTES, + "requestEnvelopeLimitMiB": bytes_to_mib(MAX_UPLOAD_REQUEST_BYTES), + "requestSlackBytes": UPLOAD_REQUEST_SIZE_SLACK_BYTES, + "requestSlackMiB": bytes_to_mib(UPLOAD_REQUEST_SIZE_SLACK_BYTES), + "protectedPostRoutes": list(UPLOAD_LIMITED_POST_PATHS), + "proxySnippets": build_proxy_snippets(), + } + + +def render_upload_limit_contract_text() -> str: + contract = build_upload_limit_contract() + route_lines = "\n".join( + f"- POST {route}" + for route in contract["protectedPostRoutes"] + ) + proxy_snippets = contract["proxySnippets"] + json_summary = json.dumps(contract, indent=2) + return ( + "Upload Limit Contract\n" + "=====================\n\n" + "Plain English\n" + "-------------\n" + f"- Raw audio uploads are capped at {MAX_UPLOAD_SIZE_BYTES} bytes " + f"({bytes_to_mib_label(MAX_UPLOAD_SIZE_BYTES)}).\n" + f"- Full HTTP upload requests are allowed up to {MAX_UPLOAD_REQUEST_BYTES} bytes " + f"({bytes_to_mib_label(MAX_UPLOAD_REQUEST_BYTES)}) so multipart framing does not " + "cause false rejects.\n" + "- Only the protected upload POST routes below should be capped at the edge.\n\n" + "Protected POST Routes\n" + "---------------------\n" + f"{route_lines}\n\n" + "Machine-readable JSON\n" + "---------------------\n" + f"{json_summary}\n\n" + "Proxy Snippets\n" + "--------------\n" + "nginx\n" + f"{proxy_snippets['nginx']}\n\n" + "Caddy\n" + f"{proxy_snippets['Caddy']}\n\n" + "Traefik\n" + f"{proxy_snippets['Traefik']}\n\n" + "Deployment Checklist\n" + "--------------------\n" + f"- Edge request-body limit matches {MAX_UPLOAD_REQUEST_BYTES} bytes " + f"({bytes_to_mib_label(MAX_UPLOAD_REQUEST_BYTES)}).\n" + "- Only the five protected upload POST routes are capped.\n" + "- Backend startup log shows the same raw and edge limits after deploy.\n" + "- A near-limit valid upload succeeds and an oversized upload returns HTTP 413.\n" + ) diff --git a/apps/ui/README.md b/apps/ui/README.md index 3f33e346..44dabe6b 100644 --- a/apps/ui/README.md +++ b/apps/ui/README.md @@ -307,15 +307,29 @@ Notes about tests: - most smoke tests stub the backend and Gemini calls - `tests/smoke` remains the mocked, CI-friendly UI-contract layer -- `tests/e2e` is the canonical always-live suite and uses the current staged runtime contract: `POST /api/analysis-runs/estimate`, `POST /api/analysis-runs`, `GET /api/analysis-runs/{run_id}`, and artifact endpoints -- the live E2E suite is local-only and requires: +- `tests/e2e/analysis-runs-integration.spec.ts` is the canonical no-Gemini integration proof and uses the current staged runtime contract: `POST /api/analysis-runs/estimate`, `POST /api/analysis-runs`, `GET /api/analysis-runs/{run_id}`, and artifact endpoints +- the no-Gemini integration lane generates its own local WAV, boots the real backend, and keeps AI interpretation off +- the full live Gemini lane stays separate and requires: - `TEST_FLAC_PATH` pointing at a readable audio file - `GEMINI_API_KEY` in the backend environment - `VITE_ENABLE_PHASE2_GEMINI=true` - `tests/smoke/upload-phase1-live.spec.ts` is a lightweight opt-in proof against the canonical `analysis-runs` flow - `tests/smoke/upload-phase2-live-gemini.spec.ts` is an opt-in Files API proof using a generated `>100MB` WAV and backend-mediated Gemini -Run the canonical local-only live suite: +Run the canonical no-Gemini local integration suite: + +```bash +./scripts/test-e2e-integration.sh +``` + +Run the same no-Gemini lane directly from `apps/ui` when the backend is already up on `http://127.0.0.1:8100`: + +```bash +VITE_API_BASE_URL=http://127.0.0.1:8100 \ +npm run test:e2e:integration +``` + +Run the separate full live Gemini suite: ```bash TEST_FLAC_PATH=/path/to/track.flac \ diff --git a/apps/ui/package.json b/apps/ui/package.json index 311d82fd..b6b59330 100644 --- a/apps/ui/package.json +++ b/apps/ui/package.json @@ -14,6 +14,7 @@ "test": "vitest run", "test:unit": "vitest run tests/services", "test:e2e": "playwright test --config=playwright.full.config.ts --reporter=list", + "test:e2e:integration": "playwright test tests/e2e/analysis-runs-integration.spec.ts --config=playwright.integration.config.ts --reporter=list", "test:e2e:headed": "playwright test --config=playwright.full.config.ts --headed --reporter=list", "test:smoke": "playwright test --reporter=list", "test:smoke:live-gemini": "playwright test tests/smoke/upload-phase2-live-gemini.spec.ts --reporter=list", diff --git a/apps/ui/playwright.integration.config.ts b/apps/ui/playwright.integration.config.ts new file mode 100644 index 00000000..ddca0e5c --- /dev/null +++ b/apps/ui/playwright.integration.config.ts @@ -0,0 +1,35 @@ +import { defineConfig } from '@playwright/test'; + +const INTEGRATION_PORT = 3100; +const integrationBaseUrl = `http://127.0.0.1:${INTEGRATION_PORT}`; +const integrationBackendUrl = process.env.VITE_API_BASE_URL ?? 'http://127.0.0.1:8100'; + +export default defineConfig({ + testDir: './tests/e2e', + fullyParallel: false, + globalSetup: './tests/e2e/support/integrationGlobalSetup.ts', + reporter: 'list', + timeout: 12 * 60 * 1_000, + expect: { + timeout: 30_000, + }, + use: { + acceptDownloads: true, + baseURL: integrationBaseUrl, + headless: true, + screenshot: 'only-on-failure', + trace: 'retain-on-failure', + }, + webServer: { + command: 'npm run dev:local', + env: { + ...process.env, + VITE_API_BASE_URL: integrationBackendUrl, + VITE_ENABLE_PHASE2_GEMINI: 'false', + }, + url: integrationBaseUrl, + reuseExistingServer: false, + timeout: 120_000, + }, + workers: 1, +}); diff --git a/apps/ui/src/components/AnalysisResults.tsx b/apps/ui/src/components/AnalysisResults.tsx index 391f2a65..b2ab6512 100644 --- a/apps/ui/src/components/AnalysisResults.tsx +++ b/apps/ui/src/components/AnalysisResults.tsx @@ -517,9 +517,9 @@ export function AnalysisResults({
@@ -789,10 +789,10 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate')} > - {formatDisplayText(item.name, 'title')} + {item.name}

- {formatDisplayText(item.type, 'eyebrow')} + {item.type}

@@ -831,7 +831,7 @@ export function AnalysisResults({

Sidechain Source

- {formatDisplayText(routingBlueprint.sidechainSource ?? 'Not specified', 'title')} + {routingBlueprint.sidechainSource ?? 'Not specified'}

@@ -855,7 +855,7 @@ export function AnalysisResults({

- {formatDisplayText(returnTrack.name, 'title')} + {returnTrack.name}

{returnTrack.deviceFocus} @@ -951,7 +951,7 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate pr-2')} > - {formatDisplayText(item.name, 'title')} + {item.name} - {formatDisplayText(stem.label, 'title')} + {stem.label}

{truncateAtSentenceBoundary(stem.summary, 220)} @@ -1323,7 +1323,7 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate')} > - {formatDisplayText(card.title, 'title')} + {card.title} {card.id === 'harmonicContent' && lowConfidenceIndicator(chordsAreApproximate)} {card.transcriptionDerived && ( @@ -1413,7 +1413,7 @@ export function AnalysisResults({ data-text-role="meta" className={textRoleClassName('meta', 'border-b border-border/70 pb-1')} > - {groupIcon(group.name)} {formatDisplayText(group.name, 'eyebrow')} + {groupIcon(group.name)} {group.name} {group.annotation && (

@@ -1443,7 +1443,7 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate')} > - {formatDisplayText(card.device, 'title')} + {card.device} {card.category} @@ -1537,7 +1537,7 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate')} > - {formatDisplayText(patch.device, 'title')} + {patch.device} {patch.transcriptionDerived && ( @@ -1630,7 +1630,7 @@ export function AnalysisResults({ data-text-role="item-title" className={[getTextRoleClassName('item-title'), 'text-lg'].join(' ')} > - {formatDisplayText(phase2.secretSauce.title, 'title')} + {phase2.secretSauce.title}

{truncateAtSentenceBoundary(phase2.secretSauce.explanation, 600)} @@ -1650,7 +1650,7 @@ export function AnalysisResults({ data-text-role="item-title" className={textRoleClassName('item-title', 'truncate')} > - {formatDisplayText(step.device, 'title')} + {step.device}

{step.parameter}: {step.value} diff --git a/apps/ui/src/components/MeasurementDashboard.tsx b/apps/ui/src/components/MeasurementDashboard.tsx index 319bb5a8..8de68a96 100644 --- a/apps/ui/src/components/MeasurementDashboard.tsx +++ b/apps/ui/src/components/MeasurementDashboard.tsx @@ -1433,9 +1433,9 @@ export function MeasurementDashboard({ @@ -1659,13 +1659,13 @@ export function MeasurementDashboard({ {/* Zone 3 — Dynamics & Texture */}

- + Dynamics & Texture {dynamicCharacter && textureCharacter ? (
- + Dynamics
- + Texture
- - {formatDisplayText('Spectral Balance', 'eyebrow')} + + Spectral Balance
{SPECTRAL_ROW_CONFIG.map((row) => ( @@ -2082,7 +2082,7 @@ export function MeasurementDashboard({ {phase1.essentiaFeatures && ( <>
- + Essentia Features
@@ -2583,7 +2583,7 @@ export function MeasurementDashboard({ {phase1.segmentLoudness && phase1.segmentLoudness.length > 0 && ( <>
- + Segment Loudness
@@ -2700,7 +2700,7 @@ export function MeasurementDashboard({ {phase1.synthesisCharacter && (
- + Synthesis Character {phase1.synthesisCharacter.analogLike !== undefined && @@ -2739,7 +2739,7 @@ export function MeasurementDashboard({ {phase1.perceptual && (
- + Perceptual
- + Sidechain / Pumping {phase1.sidechainDetail.pumpingRate && ( diff --git a/apps/ui/src/components/MixDoctorPanel.tsx b/apps/ui/src/components/MixDoctorPanel.tsx index 694712f1..a737b2c3 100644 --- a/apps/ui/src/components/MixDoctorPanel.tsx +++ b/apps/ui/src/components/MixDoctorPanel.tsx @@ -59,7 +59,7 @@ export function MixDoctorPanel({ report }: MixDoctorPanelProps) { Target Genre
- {formatDisplayText(report.genreName, 'title')} + {report.genreName}
diff --git a/apps/ui/src/components/SessionMusicianPanel.tsx b/apps/ui/src/components/SessionMusicianPanel.tsx index 65588cb0..339816c3 100644 --- a/apps/ui/src/components/SessionMusicianPanel.tsx +++ b/apps/ui/src/components/SessionMusicianPanel.tsx @@ -399,15 +399,12 @@ export function SessionMusicianPanel({ phase1, sourceFileName }: SessionMusician
-
-

- {formatDisplayText('Session Musician', 'title')} -

-

- {formatDisplayText('Pitch detection and melody guide', 'eyebrow')} +

+

+ Pitch detection and melody guide

-

- {formatDisplayText('Draft notes for MIDI cleanup', 'none')} +

+ Draft notes for MIDI cleanup

diff --git a/apps/ui/src/index.css b/apps/ui/src/index.css index 2277ec77..51aeb2b4 100644 --- a/apps/ui/src/index.css +++ b/apps/ui/src/index.css @@ -95,10 +95,20 @@ body { .text-role-section-title { font-family: var(--font-mono); - font-size: 0.875rem; + font-size: 1.125rem; font-weight: 500; - letter-spacing: 0.06em; + letter-spacing: 0.05em; line-height: 1.3; + color: var(--color-text-primary); +} + +.text-role-subsection-title { + font-family: var(--font-mono); + font-size: 0.75rem; + font-weight: 500; + letter-spacing: 0.1em; + line-height: 1.3; + text-transform: uppercase; color: var(--color-text-secondary); } diff --git a/apps/ui/src/services/analysisRunsClient.ts b/apps/ui/src/services/analysisRunsClient.ts index 934ff4c8..186f2380 100644 --- a/apps/ui/src/services/analysisRunsClient.ts +++ b/apps/ui/src/services/analysisRunsClient.ts @@ -32,8 +32,6 @@ interface CreateAnalysisRunOptions extends AnalysisRunsClientOptions { analysisMode?: 'full' | 'standard'; pitchNoteMode: string; pitchNoteBackend: string; - symbolicMode?: string; - symbolicBackend?: string; interpretationMode: string; interpretationProfile: string; interpretationModel?: string | null; @@ -71,8 +69,6 @@ export async function estimateAnalysisRun( body.append('analysis_mode', options.analysisMode ?? 'full'); body.append('pitch_note_mode', options.pitchNoteMode); body.append('pitch_note_backend', options.pitchNoteBackend); - body.append('symbolic_mode', options.symbolicMode ?? options.pitchNoteMode); - body.append('symbolic_backend', options.symbolicBackend ?? options.pitchNoteBackend); body.append('interpretation_mode', options.interpretationMode); body.append('interpretation_profile', options.interpretationProfile); if (options.interpretationModel) { @@ -97,8 +93,6 @@ export async function createAnalysisRun( body.append('analysis_mode', options.analysisMode ?? 'full'); body.append('pitch_note_mode', options.pitchNoteMode); body.append('pitch_note_backend', options.pitchNoteBackend); - body.append('symbolic_mode', options.symbolicMode ?? options.pitchNoteMode); - body.append('symbolic_backend', options.symbolicBackend ?? options.pitchNoteBackend); body.append('interpretation_mode', options.interpretationMode); body.append('interpretation_profile', options.interpretationProfile); if (options.interpretationModel) { @@ -370,12 +364,6 @@ function parseRequestedStages(value: unknown): AnalysisRunRequestedStages { requested.analysisMode == null ? 'full' : expectAnalysisMode(requested.analysisMode), pitchNoteMode: expectString(requested.pitchNoteMode, 'requestedStages.pitchNoteMode'), pitchNoteBackend: expectString(requested.pitchNoteBackend, 'requestedStages.pitchNoteBackend'), - symbolicMode: - asString(requested.symbolicMode) ?? - expectString(requested.pitchNoteMode, 'requestedStages.pitchNoteMode'), - symbolicBackend: - asString(requested.symbolicBackend) ?? - expectString(requested.pitchNoteBackend, 'requestedStages.pitchNoteBackend'), interpretationMode: expectString(requested.interpretationMode, 'requestedStages.interpretationMode'), interpretationProfile: expectString(requested.interpretationProfile, 'requestedStages.interpretationProfile'), interpretationModel: asString(requested.interpretationModel), diff --git a/apps/ui/src/services/analyzer.ts b/apps/ui/src/services/analyzer.ts index 2d6232cb..bcf420fc 100644 --- a/apps/ui/src/services/analyzer.ts +++ b/apps/ui/src/services/analyzer.ts @@ -192,8 +192,6 @@ export async function analyzeAudio( analysisMode: analysisOptions?.analysisMode ?? 'full', pitchNoteMode: resolvePitchNoteRequested(analysisOptions) ? 'stem_notes' : 'off', pitchNoteBackend: 'auto', - symbolicMode: resolvePitchNoteRequested(analysisOptions) ? 'stem_notes' : 'off', - symbolicBackend: 'auto', interpretationMode: resolveInterpretationMode(analysisOptions), interpretationProfile: 'producer_summary', interpretationModel: resolveInterpretationMode(analysisOptions) === 'off' ? null : modelName, diff --git a/apps/ui/src/types.ts b/apps/ui/src/types.ts index 35c1eda1..7811aac5 100644 --- a/apps/ui/src/types.ts +++ b/apps/ui/src/types.ts @@ -711,8 +711,6 @@ export interface AnalysisRunRequestedStages { analysisMode: 'full' | 'standard'; pitchNoteMode: string; pitchNoteBackend: string; - symbolicMode: string; - symbolicBackend: string; interpretationMode: string; interpretationProfile: string; interpretationModel: string | null; diff --git a/apps/ui/src/utils/displayText.ts b/apps/ui/src/utils/displayText.ts index b7a6d00e..cc2ba9aa 100644 --- a/apps/ui/src/utils/displayText.ts +++ b/apps/ui/src/utils/displayText.ts @@ -1,6 +1,7 @@ export type TextRole = | 'page-title' | 'section-title' + | 'subsection-title' | 'item-title' | 'eyebrow' | 'body' diff --git a/apps/ui/tests/e2e/analysis-runs-integration.spec.ts b/apps/ui/tests/e2e/analysis-runs-integration.spec.ts new file mode 100644 index 00000000..5466ef49 --- /dev/null +++ b/apps/ui/tests/e2e/analysis-runs-integration.spec.ts @@ -0,0 +1,75 @@ +import { expect, test } from '@playwright/test'; + +import { writeMusicalReferenceWav } from './support/audioFixtures'; +import { + downloadBinaryArtifact, + expectNoCommonConnectivityErrors, + gotoUploadPage, + listRunArtifacts, + openDiagnosticLog, + setToggle, + startAnalysisAndCaptureRunId, + uploadAudioFile, + waitForAnalysisResults, + waitForEstimate, + waitForRunToReachTerminalState, +} from './support/liveHarness'; + +test('local integration flow uses canonical analysis-runs routes without Gemini credentials', async ({ + page, +}, testInfo) => { + test.setTimeout(12 * 60 * 1_000); + + const fixturePath = testInfo.outputPath('analysis-runs-integration-reference.wav'); + const fixture = await writeMusicalReferenceWav(fixturePath); + expect(fixture.byteLength).toBeGreaterThan(0); + + await gotoUploadPage(page); + await expect(page.getByTestId('phase2-status-inline')).toHaveText('INTERPRETATION CONFIG OFF'); + + const estimateResponsePromise = page.waitForResponse( + (response) => + response.request().method() === 'POST' && + response.url().includes('/api/analysis-runs/estimate'), + { timeout: 60_000 }, + ); + + await uploadAudioFile(page, fixturePath); + + const estimateResponse = await estimateResponsePromise; + expect(estimateResponse.ok()).toBeTruthy(); + + await setToggle(page, 'PITCH/NOTE TRANSLATION', true); + await waitForEstimate(page, 60_000); + + const runId = await startAnalysisAndCaptureRunId(page); + const finalSnapshot = await waitForRunToReachTerminalState(runId, { timeoutMs: 12 * 60 * 1_000 }); + + await waitForAnalysisResults(page, 12 * 60 * 1_000); + await openDiagnosticLog(page); + await expectNoCommonConnectivityErrors(page); + + await expect(page.getByText(/Measurement complete\./i)).toBeVisible({ timeout: 45_000 }); + await expect(page.getByText(/Pitch\/Note Translation complete\./i)).toBeVisible({ timeout: 45_000 }); + await expect(page.getByTestId('analysis-results-root')).toBeVisible(); + await expect(page.getByTestId('measurement-dashboard')).toBeVisible(); + await expect(page.getByTestId('session-musician-panel')).toBeVisible(); + + expect(finalSnapshot.runId).toBe(runId); + expect(finalSnapshot.stages.measurement.status).toBe('completed'); + expect(finalSnapshot.stages.pitchNoteTranslation.status).toBe('completed'); + expect(finalSnapshot.stages.interpretation.status).toBe('not_requested'); + expect(finalSnapshot.artifacts.sourceAudio.filename).toBeTruthy(); + expect(finalSnapshot.artifacts.spectral?.spectrograms.length ?? 0).toBeGreaterThan(0); + expect(finalSnapshot.artifacts.spectral?.timeSeries).toBeTruthy(); + + const artifacts = await listRunArtifacts(runId); + const artifactKinds = new Set(artifacts.map((artifact) => String(artifact.kind))); + expect(artifactKinds).toContain('source_audio'); + expect(artifactKinds).toContain('spectrogram_mel'); + expect(artifactKinds).toContain('spectral_time_series'); + + const midiArtifact = await downloadBinaryArtifact(page, /Download \.mid/i); + expect(midiArtifact.download.suggestedFilename()).toBe('track-analysis.mid'); + expect(midiArtifact.sizeBytes).toBeGreaterThan(0); +}); diff --git a/apps/ui/tests/e2e/support/integrationGlobalSetup.ts b/apps/ui/tests/e2e/support/integrationGlobalSetup.ts new file mode 100644 index 00000000..6ac6c734 --- /dev/null +++ b/apps/ui/tests/e2e/support/integrationGlobalSetup.ts @@ -0,0 +1,5 @@ +import { assertIntegrationE2EPreflight } from './preflight'; + +export default async function globalSetup(): Promise { + await assertIntegrationE2EPreflight(); +} diff --git a/apps/ui/tests/e2e/support/preflight.ts b/apps/ui/tests/e2e/support/preflight.ts index ca2970fc..12575497 100644 --- a/apps/ui/tests/e2e/support/preflight.ts +++ b/apps/ui/tests/e2e/support/preflight.ts @@ -77,7 +77,32 @@ export async function validateLiveE2EEnv(env: Record return issues; } -export async function backendSupportsLiveE2ERoutes( +interface LiveE2EPreflightOptions { + env?: Record; + fetchImpl?: FetchLike; +} + +async function assertCanonicalAnalysisRunsBackend( + backendBaseUrl: string, + suiteLabel: string, + fetchImpl: FetchLike = fetch, +): Promise { + const backendReady = await backendSupportsCanonicalAnalysisRunRoutes(backendBaseUrl, fetchImpl); + + if (!backendReady) { + throw new Error( + `Backend at ${backendBaseUrl} must expose /api/analysis-runs/estimate, /api/analysis-runs, and /api/analysis-runs/{run_id} for ${suiteLabel}.`, + ); + } +} + +export async function validateIntegrationE2EEnv( + _env: Record, +): Promise { + return []; +} + +export async function backendSupportsCanonicalAnalysisRunRoutes( baseUrl: string, fetchImpl: FetchLike = fetch, ): Promise { @@ -103,9 +128,11 @@ export async function backendSupportsLiveE2ERoutes( } } -interface LiveE2EPreflightOptions { - env?: Record; - fetchImpl?: FetchLike; +export async function backendSupportsLiveE2ERoutes( + baseUrl: string, + fetchImpl: FetchLike = fetch, +): Promise { + return backendSupportsCanonicalAnalysisRunRoutes(baseUrl, fetchImpl); } export async function assertLiveE2EPreflight(options: LiveE2EPreflightOptions = {}): Promise { @@ -117,11 +144,19 @@ export async function assertLiveE2EPreflight(options: LiveE2EPreflightOptions = } const backendBaseUrl = resolveLiveBackendBaseUrl(env); - const backendReady = await backendSupportsLiveE2ERoutes(backendBaseUrl, options.fetchImpl); + await assertCanonicalAnalysisRunsBackend(backendBaseUrl, 'the full live E2E suite', options.fetchImpl); +} - if (!backendReady) { - throw new Error( - `Backend at ${backendBaseUrl} must expose /api/analysis-runs/estimate, /api/analysis-runs, and /api/analysis-runs/{run_id} for the full live E2E suite.`, - ); +export async function assertIntegrationE2EPreflight( + options: LiveE2EPreflightOptions = {}, +): Promise { + const env = options.env ?? (process.env as Record); + const issues = await validateIntegrationE2EEnv(env); + + if (issues.length > 0) { + throw new Error(issues.join('\n')); } + + const backendBaseUrl = resolveLiveBackendBaseUrl(env); + await assertCanonicalAnalysisRunsBackend(backendBaseUrl, 'the local integration E2E suite', options.fetchImpl); } diff --git a/apps/ui/tests/services/analysisResultsUi.test.ts b/apps/ui/tests/services/analysisResultsUi.test.ts index 3c5fa9f0..e376a337 100644 --- a/apps/ui/tests/services/analysisResultsUi.test.ts +++ b/apps/ui/tests/services/analysisResultsUi.test.ts @@ -437,8 +437,9 @@ describe('AnalysisResults UI wiring', () => { expect(html).toContain('data-text-role="item-title"'); expect(html).toContain('>Harmonic Content'); expect(html).toContain('>Drum Buss'); + expect(html).toContain('data-text-role="subsection-title"'); + expect(html).toContain('>Spectral Balance'); expect(html).toContain('data-text-role="eyebrow"'); - expect(html).toContain('>SPECTRAL BALANCE'); expect(html).toContain('>SUB BASS'); expect(html).toContain('data-text-role="body"'); expect(html).toContain('Shapes drum impact by adds punch to drums.'); @@ -496,8 +497,8 @@ describe('AnalysisResults UI wiring', () => { ); expect((html.match(/border-l-2 border-accent/g) ?? []).length).toBeGreaterThanOrEqual(4); - expect(html).toContain('>HOUSE<'); - expect(html).toContain('>TECHNO<'); + expect(html).toContain('>house<'); + expect(html).toContain('>techno<'); }); it('renders character scanning fallback when phase2 is unavailable', () => { diff --git a/apps/ui/tests/services/e2ePreflight.test.ts b/apps/ui/tests/services/e2ePreflight.test.ts index c1e04c97..c03c5779 100644 --- a/apps/ui/tests/services/e2ePreflight.test.ts +++ b/apps/ui/tests/services/e2ePreflight.test.ts @@ -5,9 +5,12 @@ import path from 'node:path'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { + assertIntegrationE2EPreflight, + backendSupportsCanonicalAnalysisRunRoutes, assertLiveE2EPreflight, backendSupportsLiveE2ERoutes, isPlaceholderGeminiApiKey, + validateIntegrationE2EEnv, validateLiveE2EEnv, } from '../e2e/support/preflight'; @@ -26,6 +29,16 @@ afterEach(async () => { }); describe('e2e preflight', () => { + it('accepts missing Gemini credentials and track path for the no-Gemini integration lane', async () => { + expect( + await validateIntegrationE2EEnv({ + TEST_FLAC_PATH: '', + VITE_ENABLE_PHASE2_GEMINI: 'false', + GEMINI_API_KEY: '', + }), + ).toEqual([]); + }); + it('treats empty and placeholder Gemini keys as invalid', () => { expect(isPlaceholderGeminiApiKey('')).toBe(true); expect(isPlaceholderGeminiApiKey(' ')).toBe(true); @@ -86,10 +99,33 @@ describe('e2e preflight', () => { }), }); + await expect(backendSupportsCanonicalAnalysisRunRoutes('http://127.0.0.1:8100', fetchImpl)).resolves.toBe(true); await expect(backendSupportsLiveE2ERoutes('http://127.0.0.1:8100', fetchImpl)).resolves.toBe(true); expect(fetchImpl).toHaveBeenCalledWith('http://127.0.0.1:8100/openapi.json', expect.any(Object)); }); + it('fails fast for the integration lane when the backend is unreachable or missing canonical routes', async () => { + const fetchImpl = vi.fn().mockResolvedValue({ + ok: true, + json: async () => ({ + paths: { + '/api/analysis-runs/estimate': {}, + }, + }), + }); + + await expect( + assertIntegrationE2EPreflight({ + env: { + VITE_API_BASE_URL: 'http://127.0.0.1:8100', + }, + fetchImpl, + }), + ).rejects.toThrow( + 'Backend at http://127.0.0.1:8100 must expose /api/analysis-runs/estimate, /api/analysis-runs, and /api/analysis-runs/{run_id} for the local integration E2E suite.', + ); + }); + it('fails fast when the backend is unreachable or missing required routes', async () => { const trackPath = await createTempFile('fixture.wav'); const fetchImpl = vi.fn().mockResolvedValue({ diff --git a/apps/ui/tests/smoke/ui-details.spec.ts b/apps/ui/tests/smoke/ui-details.spec.ts index 0495ba77..359138c5 100644 --- a/apps/ui/tests/smoke/ui-details.spec.ts +++ b/apps/ui/tests/smoke/ui-details.spec.ts @@ -515,7 +515,7 @@ test('results panels expose shared typography roles for spectral, band diagnosti await expect(page.getByText('Analysis Results')).toBeVisible(); - await expect(page.locator('[data-text-role="eyebrow"]').filter({ hasText: 'SPECTRAL BALANCE' }).first()).toBeVisible(); + await expect(page.locator('[data-text-role="subsection-title"]').filter({ hasText: /spectral balance/i }).first()).toBeVisible(); await expect(page.locator('[data-text-role="eyebrow"]').filter({ hasText: 'SUB BASS' }).first()).toBeVisible(); await expect(page.locator('[data-text-role="item-title"]').filter({ hasText: 'Harmonic Content' }).first()).toBeVisible(); await expect(page.locator('[data-text-role="item-title"]').filter({ hasText: 'Drum Buss' }).first()).toBeVisible(); diff --git a/docs/ARCHITECTURE_STRATEGY.md b/docs/ARCHITECTURE_STRATEGY.md index cb09d9ad..ca73963d 100644 --- a/docs/ARCHITECTURE_STRATEGY.md +++ b/docs/ARCHITECTURE_STRATEGY.md @@ -94,7 +94,7 @@ Both outputs are honest about what they are. Neither pretends to be Ableton's au **Phase 2 grounding rule:** Inject Layer 1 deterministic measurements (tempo, time signature, key, section boundaries) into every Gemini call. Instruct Gemini explicitly not to re-estimate those values. Benchmarks show numeric drift and omission occur when models aren't anchored — grounding Gemini with the measurements substantially reduces that failure mode but does not guarantee it. -**Inline size limit — confirmed January 12 2026.** Google officially increased the Gemini API inline limit from 20MB to 100MB. The `INLINE_SIZE_LIMIT = 20_971_520` constant in server.py should be updated to `104_857_600`. Demucs stems at 30-120s are well under 100MB uncompressed — the Files API routing for stems is likely unnecessary overhead. Note: the Firebase AI Logic SDK still enforces 20MB at its own layer, but the direct google-genai SDK used in server.py gets the full 100MB. Base64 encoding adds ~33% to raw file size; verify that a 75MB raw stem stays under 100MB encoded before removing the Files API path entirely. +**Inline size limit — confirmed January 12 2026.** Google officially increased the Gemini API inline limit from 20MB to 100MB, and `server.py` already uses `INLINE_SIZE_LIMIT = 104_857_600`. ASA now sends raw audio files at or below 100 MiB inline and uses the Gemini Files API above that threshold. Note: the Firebase AI Logic SDK still enforces 20MB at its own layer, but the direct google-genai SDK used in `server.py` gets the full 100MB. Base64 encoding still adds ~33% within the inline path, but the current code contract is the raw-file threshold implemented in `server.py`. **Structured Outputs for stem listening:** Use a minimal JSON schema (bar grid, inferred scale degrees, rhythmic pattern class, uncertainty flags). Structured Outputs constrain the response to what's parseable. This is the right interface for the Gemini stem-listening experiment. diff --git a/scripts/test-e2e-integration.sh b/scripts/test-e2e-integration.sh new file mode 100755 index 00000000..78c69bb0 --- /dev/null +++ b/scripts/test-e2e-integration.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +BACKEND_PYTHON="$ROOT_DIR/apps/backend/venv/bin/python" +BACKEND_URL="${VITE_API_BASE_URL:-http://127.0.0.1:8100}" +BACKEND_LOG="$(mktemp -t sonic-analyzer-e2e-integration-backend.XXXXXX.log)" +BACKEND_PID="" + +verify_backend_contract() { + python3 - "$BACKEND_URL" <<'PY' +import json +import sys +import urllib.request + +base_url = sys.argv[1].rstrip("/") +request = urllib.request.Request(f"{base_url}/openapi.json", method="GET") + +try: + with urllib.request.urlopen(request, timeout=2.5) as response: + payload = json.load(response) +except Exception: + sys.exit(1) + +info = payload.get("info") or {} +paths = payload.get("paths") or {} + +if ( + info.get("title") == "Sonic Analyzer Local API" + and "/api/analysis-runs/estimate" in paths + and "/api/analysis-runs" in paths + and "/api/analysis-runs/{run_id}" in paths +): + sys.exit(0) + +sys.exit(1) +PY +} + +cleanup() { + if [[ -n "${BACKEND_PID}" ]]; then + kill "${BACKEND_PID}" 2>/dev/null || true + wait "${BACKEND_PID}" 2>/dev/null || true + fi +} + +trap cleanup EXIT + +if [[ ! -x "${BACKEND_PYTHON}" ]]; then + echo "Missing backend interpreter at ${BACKEND_PYTHON}. Run ./apps/backend/scripts/bootstrap.sh first." >&2 + exit 1 +fi + +if [[ "${BACKEND_URL}" != "http://127.0.0.1:8100" ]]; then + echo "scripts/test-e2e-integration.sh expects VITE_API_BASE_URL to be http://127.0.0.1:8100 for the local backend run." >&2 + exit 1 +fi + +( + cd "${ROOT_DIR}/apps/backend" + SONIC_ANALYZER_PORT=8100 "${BACKEND_PYTHON}" server.py >"${BACKEND_LOG}" 2>&1 +) & +BACKEND_PID=$! + +for _ in $(seq 1 30); do + if verify_backend_contract; then + break + fi + sleep 1 +done + +if ! verify_backend_contract; then + echo "Backend did not become ready on http://127.0.0.1:8100 with the canonical analysis-runs contract within 30 seconds." >&2 + echo "Backend log:" >&2 + cat "${BACKEND_LOG}" >&2 + exit 1 +fi + +( + cd "${ROOT_DIR}/apps/ui" + export VITE_API_BASE_URL="http://127.0.0.1:8100" + export VITE_ENABLE_PHASE2_GEMINI="false" + npm run test:e2e:integration +)