Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Source: `active/sonic-architect-app` (keep in `active/` as reference). Most of t

## Shipped — Data
- ✅ `data/genreProfiles.ts` — 35 EDM genre spectral targets, LUFS/crest factor/PLR ranges. **Landed:** [`apps/ui/src/data/genreProfiles.ts`](apps/ui/src/data/genreProfiles.ts).
- ✅ `data/abletonDevices.ts` — 14 spectral-band → Ableton Live 12 device mappings + FX rule engine. **Landed:** [`apps/ui/src/data/abletonDevices.ts`](apps/ui/src/data/abletonDevices.ts).
- ✅ `data/abletonDevices.ts` — 14 spectral-band → Ableton Live 12 device mappings + FX rule engine. **Landed:** [`apps/ui/src/data/abletonDevices.ts`](apps/ui/src/data/abletonDevices.ts). *Ported but demoted to a research-only scoring baseline (owner decision 2026-06-11): imported by no product code, consumed only by the recommendation-eval bridge. See the NEEDS-WIRING decision in [`apps/backend/NEEDS.md`](apps/backend/NEEDS.md).*

## Shipped — Mix Analysis
- ✅ `services/mixDoctor.ts` + `components/MixDoctorPanel.tsx` — spectral-balance scoring against genre profiles. **Landed:** [`apps/ui/src/services/mixDoctor.ts`](apps/ui/src/services/mixDoctor.ts), [`apps/ui/src/components/MixDoctorPanel.tsx`](apps/ui/src/components/MixDoctorPanel.tsx).
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,7 @@ ASA_MOSS_MODEL_ID="" # optional MOSS model id forwarded to t
ASA_MOSS_SIDECAR_MODE="mock" # sidecar-side (its own venv): `mock` (default, deterministic schema-valid Phase-1-grounded output) or `model` (a 501 licence-gated stub that executes no OpenMOSS code).
ASA_CLAUDE_CLI="claude" # path to the Claude Code CLI binary used when ASA_PHASE2_PROVIDER=claude; defaults to `claude` on PATH.
ASA_CLAUDE_MODEL="" # optional model override for the claude provider (e.g. `sonnet`); empty = the CLI's default model. The subprocess runs sandboxed: --safe-mode, --tools "", --no-session-persistence, schema enforced via --json-schema.
ASA_CLAUDE_TIMEOUT_SECONDS="600" # per-call timeout for the claude provider subprocess (a full producer_summary prompt on the CLI default model measures ~6 min).
ASA_CLAUDE_TIMEOUT_SECONDS="600" # per-call timeout for the claude provider subprocess (a full producer_summary prompt on the CLI default model measures ~6 min). For headless runs also set MAX_THINKING_TOKENS=0 — a thinking-enabled model can spend the entire budget deliberating before the structured output starts (see docs/PHASE2_PROVIDER.md, 2026-06-11 addendum).
```

Phase 2 is gated by `VITE_ENABLE_PHASE2_GEMINI`. `GEMINI_API_KEY` is backend-only. `SONIC_ANALYZER_ADMIN_KEY` is backend-only and never exposed to clients.
Expand All @@ -400,7 +400,7 @@ Things that look like normal code changes but silently break the contract. Most

1. **`print(...)` in [analyze.py](apps/backend/analyze.py) without `file=sys.stderr`.** Stdout is the JSON contract. Any stray print corrupts it and the server reports a parse error with no useful trace. The existing code is consistent about this — match the pattern (`print(f"[warn] ...", file=sys.stderr)`).
2. **Calling `analyze.py` as a subprocess without `--yes`.** The CLI prompts for confirmation when stdin is a TTY. Subprocess invocations must pass `--yes` or hang waiting for input. `server.py` already does; new callers must too.
3. **Renaming a field on only one side.** Python emits *camelCase* JSON directly (`bpmConfidence`, not `bpm_confidence`) — there is no conversion layer. A rename in [analyze.py](apps/backend/analyze.py) without a matching update in [src/types.ts](apps/ui/src/types.ts) is undetectable by either type system; the field just disappears from the UI.
3. **Renaming a field on only one side.** Python emits *camelCase* JSON directly (`bpmConfidence`, not `bpm_confidence`) — there is no conversion layer. A rename in [analyze.py](apps/backend/analyze.py) without a matching update in [src/types.ts](apps/ui/src/types.ts) is undetectable by either type system; the field just disappears from the UI. A subtler variant: the ~12 `parseOptional*` reconstructors in [src/services/backendPhase1Client.ts](apps/ui/src/services/backendPhase1Client.ts) rebuild their block field-by-field, so a backend field one of them forgets to forward is *silently dropped* — and if Phase 2 cites it, the citation fails the existence check (this bit `reverbDetail.preDelayMs` + the vocal stem proxies). The executable guard is [tests/services/phase1CitationContract.test.ts](apps/ui/tests/services/phase1CitationContract.test.ts): it parses a comprehensive payload and asserts every citable path survives `collectPhase1FieldPaths`. Add a new citable field to both its fixture and its `CITABLE_DETAIL_PATHS` list.
4. **Adding a top-level key without updating `EXPECTED_TOP_LEVEL_KEYS`.** [tests/test_analyze.py](apps/backend/tests/test_analyze.py) holds a snapshot of every root key. New fields require updating that set *and* [JSON_SCHEMA.md](apps/backend/JSON_SCHEMA.md). The same test enforces that `--fast` only populates `FAST_MODE_POPULATED_FIELDS`. A change to *measured values* (not just keys) can also trip the golden-snapshot regression gate in [tests/test_phase1_golden.py](apps/backend/tests/test_phase1_golden.py) (fixture `tests/fixtures/golden/phase1_default.json`) — re-baseline it deliberately, never blindly (from `apps/backend/`): `UPDATE_PHASE1_GOLDEN=1 ./venv/bin/python -m unittest tests.test_phase1_golden`
5. **Using `document` or `window` in `tests/services/`.** Vitest runs in `node`, not `jsdom`. Service-layer tests are pure logic; if you need DOM, it's a Playwright test in `tests/smoke/` instead.
6. **Hard-coding `Path(...)` for artifacts in new code.** Artifact access must go through [artifact_storage.py](apps/backend/artifact_storage.py). Direct paths work in `local` profile and break silently in `hosted`.
Expand Down Expand Up @@ -440,7 +440,7 @@ A quick map from intent to the right place to start:

## Backport Candidates

The full original `sonic-architect-app` backlog is **shipped** — genre profiles, Ableton device mappings, mix doctor, eight detectors, Phase 3 audition-sample generation, and `patchSmith.ts` (Phase 3 Vital preset generation, [`apps/ui/src/services/patchSmith.ts`](apps/ui/src/services/patchSmith.ts)). See [`BACKLOG.md`](BACKLOG.md) for the landed inventory. Consult it before re-implementing genre detection, mix analysis, acid/reverb/vocal/supersaw/bass/kick detection — they're already in [`apps/backend/analyze_detection.py`](apps/backend/analyze_detection.py) and emit fields visible in `EXPECTED_TOP_LEVEL_KEYS`.
The full original `sonic-architect-app` backlog is **shipped** — genre profiles, Ableton device mappings (`abletonDevices.ts` — ported but **research-only** since the 2026-06-11 demotion decision: a scoring baseline for the recommendation-proof harness, imported by no product code; see the NEEDS-WIRING decision in `apps/backend/NEEDS.md`), mix doctor, eight detectors, Phase 3 audition-sample generation, and `patchSmith.ts` (Phase 3 Vital preset generation, [`apps/ui/src/services/patchSmith.ts`](apps/ui/src/services/patchSmith.ts)). See [`BACKLOG.md`](BACKLOG.md) for the landed inventory. Consult it before re-implementing genre detection, mix analysis, acid/reverb/vocal/supersaw/bass/kick detection — they're already in [`apps/backend/analyze_detection.py`](apps/backend/analyze_detection.py) and emit fields visible in `EXPECTED_TOP_LEVEL_KEYS`.

## Companion Agent Docs

Expand Down
53 changes: 33 additions & 20 deletions apps/backend/NEEDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,9 @@ citations**, so the chain-of-custody penalty drives its adjusted aggregate to
#2). The rules **are** feature-triggered, so attaching the triggering measurement
as a citation is a concrete, harness-rewarded improvement — a candidate
sub-goal-3 "score-driven change" (frontend edit to `abletonDevices.ts` →
`npm run verify`). But see the dead-code finding under NEEDS-WIRING before
investing in this path.
`npm run verify`). Moot since 2026-06-11: the NEEDS-WIRING decision below
demoted `abletonDevices.ts` to a research-only baseline, so this candidate is
no longer a *product* improvement path.

This is the first real signal toward the Gemini verdict — likely Gemini earns its
place on citation + full-surface coverage. Confirm on real renders; the synthetic
Expand Down Expand Up @@ -169,22 +170,23 @@ factor, onset density) is the app's own projection (`analyzer.ts`); reuse it rat
than re-deriving, and only then is the deterministic score a real verdict input
rather than the synthetic illustration above.

**FINDING — the deterministic source is currently dead code (affects sub-goal 3
framing).** `apps/ui/src/data/abletonDevices.ts` is imported **nowhere** in the UI
(no component, no test) — verified by grep. ASA's product recommendations come
entirely from Gemini (`Phase2Result`); the deterministic engine is unwired,
ported-but-unused. Consequence: a "score-driven improvement" to `abletonDevices.ts`
(e.g. the citation-emit fix flagged above) would **not reach any user** — it would
raise a harness number for code nobody runs, which is harness-gaming, not a product
improvement (PURPOSE.md decision #5). So before sub-goal 3 treats the deterministic
path as a real recommendation source, decide one of:
1. **Wire `abletonDevices.ts` into the product** (so its recs — and any citation
improvement — actually reach users), then score + improve it; or
2. **Treat it as a research-only baseline** in the three-source comparison (score
it, but land score-driven *product* improvements on the **Gemini** path
instead — which needs the `chore/...` branch + renders).
This is the honest reason no score-driven improvement was "landed" this session:
the only render-free candidate would have improved dead code.
**FINDING — the deterministic source is dead code on the product path. ✅ DECIDED
2026-06-11: demoted to research-only baseline (option 2).**
`apps/ui/src/data/abletonDevices.ts` is imported **nowhere** in the UI (no
component, no test) — verified by grep. ASA's product recommendations come
entirely from the Phase 2 providers (`Phase2Result`); the deterministic engine is
unwired, ported-but-unused. Consequence: a "score-driven improvement" to
`abletonDevices.ts` (e.g. the citation-emit fix flagged above) would **not reach
any user** — it would raise a harness number for code nobody runs, which is
harness-gaming, not a product improvement (PURPOSE.md decision #5). The owner
resolved the wire-or-demote choice: **demote**. Standing consequences:
1. The module stays where it is as the scored free baseline in the three-source
comparison (its file header now records the status); it is **not** to be
wired into the product.
2. Score-driven *product* improvements land on the **Phase 2 provider** path
instead. The citation-emit candidate above is off the table as a product
change (it remains a legitimate baseline-fairness tweak if the comparison
ever needs it).

---

Expand All @@ -201,8 +203,19 @@ known-settings axes (role recall / value accuracy) — those need rendered fixtu
so the *complete* "does Gemini recover the actual settings?" verdict remains
render-gated.

Mechanism is in place: `--source baseline|gemini|deterministic` on the same
corpus. To deliver the **full** verdict:
**Claude provider scored on the proxy corpus — 2026-06-11, zero Gemini cost.**
The `--source claude` harness path plus `scripts/gen_claude_phase2.py` generated
and scored fully-cited, zero-warning recommendations on the three
proxy-fingerprint fixtures: **acid 0.485, house 0.424, melodic_techno 0.424**
vs recorded Gemini 0.172 / 0.343 / 0.000 on identical fingerprints (text-only
vs audio+JSON — full table and caveats in `RECOMMENDATION_VERDICT.md`). The
melodic_techno "Gemini 0 recs" outlier is resolved as Gemini-side: Claude
produced 13 cards from the same fingerprint. Evidence committed as
`phase2.claude.json` in each fixture dir. Real-render re-runs remain the
authoritative step for both providers.

Mechanism is in place: `--source baseline|gemini|claude|deterministic` on the
same corpus. To deliver the **full** verdict:
1. Render the fixtures (NEEDS-FIXTURE #1–2).
2. Wire the deterministic adapter (NEEDS-WIRING).
3. For each fixture, produce a live Gemini `phase2.json` (needs `GEMINI_API_KEY`;
Expand Down
57 changes: 55 additions & 2 deletions apps/backend/RECOMMENDATION_VERDICT.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,65 @@ recs cited and path-valid** against the real fingerprint (custody 1.000), full-s
coverage. Consistent with the corpus verdict — Gemini's custody advantage is real,
not a synthesis artifact.

## Claude provider head-to-head (2026-06-11 — same proxy corpus, zero Gemini cost)

The selectable Claude provider (`ASA_PHASE2_PROVIDER=claude`,
`phase2_provider.ClaudeCliProvider`) was scored on the three fixtures whose proxy
fingerprints survived the 2026-06-10 corpus re-authoring, via the exact server
path (`server._run_interpretation_request`) and the same scorer. Generation:
`scripts/gen_claude_phase2.py`; scoring: `--source claude` (reads the committed
`phase2.claude.json` evidence in each fixture dir). Model `sonnet`
(claude-sonnet-4-6), **text-only** (grounds purely on the embedded Phase 1 JSON,
no audio), `MAX_THINKING_TOKENS=0`, 300–365 s per fixture.

| Fixture | Claude | Gemini (recorded above) |
|---|---|---|
| acid_303_128 | **0.485** | 0.172 |
| house_sidechain_pluck_124 | **0.424** | 0.343 |
| melodic_techno_arp_124 | **0.424** | 0.000 (zero cards) |
| **Mean (shared subset)** | **0.444** | 0.172 |

All three Claude results are fully cited (custody penalty 1.000) with **zero**
`validationWarnings` — both the citation-path check and the Live 12 catalogue
gates came back clean.

Findings:
1. **The melodic_techno outlier was Gemini-side, not fixture-side.** From the
identical fingerprint where Gemini returned 0 structured cards, Claude
produced 13 rec cards + 14 chain cards + 5 workflow steps (32 envelope
entries) and scored 0.424. The fixture's fingerprint is interpretable.
2. **Kick role recall 1.00 on all three** (the recorded gemini/deterministic
table above shows kick 0.00/0.00): Claude's kick cards name `Operator`,
matching the specs' source synth. Whether genuine inference or
electronic-genre convention, it softens the "source instrument is
unrecoverable from audio" ceiling — notably, Claude never heard any audio.
3. The groove/fx/stereo zeros persist for Claude too (cards exist — `Drum Buss`
groove cards, a `Hybrid Reverb` that lands in `unknown` domain — but miss
the spec devices), consistent with the domain-attribution limitation in
`NEEDS.md`. Provider-agnostic and scorer-consistent.

Caveats, honestly: the SYNTHETIC-PROXY warning at the top applies in full. The
comparison is like-for-like on *inputs* (identical fingerprints, same scorer)
but not on *modality*: Gemini received the unrepresentative proxy audio
alongside the Phase 1 JSON, while Claude's text-only grounding may have
shielded it from proxy-audio artifacts. Model classes also differ (sonnet vs
gemini-2.5-flash). Re-run both on real renders before treating this as a
provider verdict. What it does establish: the zero-Gemini-cost route produces
fully-cited, catalogue-clean, schema-valid recommendations at
competitive-or-better measured quality on every fixture it has seen.

## Where real input is needed (retroactive)
1. **Real Ableton renders** of the 5 specs (48 kHz/24-bit) replace the proxies →
makes the known-settings axes (role recall, value accuracy) authoritative. The
proxies have artifacts: dnb BPM read half-time (174→116), `acidDetail` fires on
all (synth is saw-heavy), melodic_techno key mis-detected.
2. **`melodic_techno_arp_124` Gemini 0 recs** — investigate the parse/shape outlier
on a real render before trusting that fixture's contribution.
on a real render before trusting that fixture's contribution. *(Partially
resolved 2026-06-11: Claude produced a full 13-card result from the same
fingerprint — see the head-to-head section above — so the outlier is
Gemini-side, not fixture-side. Still verify Gemini on a real render.)*
3. The deterministic `AudioFeatures` projection is approximate (not the app's
`analyzer.ts`); reconcile when wiring the deterministic path for real.
`analyzer.ts`). The wire-or-demote question was resolved 2026-06-11: the
deterministic engine is a research-only baseline and will not be wired into
the product (see `NEEDS.md`), so reconciling the projection matters only for
baseline fairness in this comparison, not for any product path.
6 changes: 4 additions & 2 deletions apps/backend/scripts/emit_deterministic_recs.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
/**
* Deterministic recommendation source bridge (GOAL.md sub-goal 3 inner loop).
*
* EVAL / RESEARCH ONLY. Wraps the product's deterministic recommendation engine
* (`apps/ui/src/data/abletonDevices.ts`) and emits the scorer's normalized rec
* EVAL / RESEARCH ONLY. Wraps the deterministic recommendation engine
* (`apps/ui/src/data/abletonDevices.ts` — itself research-only since the
* 2026-06-11 demotion decision; see apps/backend/NEEDS.md) and emits the
* scorer's normalized rec
* shape so `scripts/evaluate_recommendations.py --source deterministic` can grade
* the *free* path without a Gemini call. Importing the real module keeps a single
* source of truth — no Python re-port to drift from the TS.
Expand Down
Loading