Skip to content

feat: land the recommendation-proof campaign (GOAL.md + scorer + corpus + UI badge) #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
slittycode merged 3 commits into from
May 27, 2026
Merged

feat: land the recommendation-proof campaign (GOAL.md + scorer + corpus + UI badge) #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
404c6cd
docs: land GOAL.md — the recommendation-proof campaign
slittycode May 27, 2026
a9185cc
feat(eval): recommendation-quality scorer + answer-key fixture corpus
slittycode May 27, 2026
08c08e8
feat(ui): per-recommendation verification badge mounted on result cards
slittycode May 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Operational and one-shot scripts live in two places. They are not on the request
`apps/backend/scripts/`:
1. `bootstrap.sh` — recreate the Python 3.11 venv (covered above). `dev.sh` here is a thin shim that `exec`s the repo-root `scripts/dev.sh` full-stack launcher.
2. `render_upload_limit_contract.py` — regenerate the operator-facing upload-limit contract text from `upload_limits.py`. Run after changing the canonical limits.
3. `evaluate_phase1.py`, `evaluate_polyphonic.py`, `evaluate_structure_sweep.py`, `evaluate_beats.py`, `evaluate_loudness_recs.py` — offline evaluation harnesses for the Phase 1 detector battery, the research polyphonic transcriber, structure-segmentation parameter sweeps, the beat/downbeat measurement gate, and loudness-recommendation reachability. Wired to `phase1_evaluation.py` / `polyphonic_evaluation.py` / `beat_evaluation.py` / `loudness_rec_evaluation.py`. `build_beat_manifest.py` assembles the beat-eval corpus manifest. All research-only. The beat gate's optional neural deps (`beat_this`, `mir_eval`) live in `apps/backend/requirements-eval.txt` — install into a separate venv, never the product venv.
3. `evaluate_phase1.py`, `evaluate_polyphonic.py`, `evaluate_structure_sweep.py`, `evaluate_beats.py`, `evaluate_loudness_recs.py`, `evaluate_recommendations.py` — offline evaluation harnesses for the Phase 1 detector battery, the research polyphonic transcriber, structure-segmentation parameter sweeps, the beat/downbeat measurement gate, loudness-recommendation reachability, and the recommendation-quality scorer (`GOAL.md`'s recommendation-proof campaign — scores Phase 2 recs against the `tests/fixtures/recommendation_tracks/` answer-key corpus). Wired to `phase1_evaluation.py` / `polyphonic_evaluation.py` / `beat_evaluation.py` / `loudness_rec_evaluation.py` / `recommendation_evaluation.py`. `build_beat_manifest.py` assembles the beat-eval corpus manifest. `emit_deterministic_recs.ts` is the recommendation harness's deterministic-source bridge (Node 23+ native TS; wraps `apps/ui/src/data/abletonDevices.ts` to score the free path). All research-only. The beat gate's optional neural deps (`beat_this`, `mir_eval`) live in `apps/backend/requirements-eval.txt` — install into a separate venv, never the product venv.
4. `audit_pass1.py`, `genre_check.py`, `replay_catalog_validation.py` — corpus auditing and Live 12 device-catalog validation. `genre_corpus.md` is the corpus manifest.
5. `import_midi_to_ground_truth.py`, `score_polyphonic_clip.py` — corpus-building helpers for the transcription/polyphonic ground-truth fixtures (`tests/fixtures/transcription_tracks/`, `tests/fixtures/polyphonic_tracks/`). Research-only; see those fixtures' READMEs and `docs/LAYER2_EVALUATION.md` / `docs/POLYPHONIC_TRANSCRIPTION_SPIKE.md`.

Expand Down Expand Up @@ -172,7 +172,7 @@ Artifact access goes through `artifact_storage.py` rather than direct disk paths
13. **`stage_status.py`**: Collapses the eight internal stage statuses into the additive client-facing `publicStatus` field on every stage snapshot.
14. **`server_samples.py` + `sample_generation.py`, `sample_theory.py`, `sample_synthesis.py`, `sample_drums.py`**: Phase 3 audition-sample generation. `sample_theory.py` builds the PyTheory musical plan, `sample_synthesis.py` renders audio (FluidSynth with sine-additive fallback), `sample_drums.py` synthesizes drum one-shots, `sample_generation.py` orchestrates and emits the citation manifest. On-demand only.
15. **`dsp_bandbank.py` + `dsp_utils.py`**: Shared DSP primitives — `BatchedBandpass` (4th-order Butterworth bandpass bank) and cross-module utility functions.
16. **`phase1_evaluation.py` + `phase1_report_html.py`, `polyphonic_evaluation.py`, `beat_evaluation.py` + `beat_report_html.py`, `loudness_rec_evaluation.py`**: Offline evaluation harnesses (deterministic-metric / detector-stability reporting, research-only polyphonic transcription, the beat/downbeat measurement gate that benchmarks CPJKU/beat_this against the shipping kick-accent heuristic, and loudness-recommendation reachability). Not on the product path; driven by `scripts/evaluate_*.py`. Deleting them restores the product exactly.
16. **`phase1_evaluation.py` + `phase1_report_html.py`, `polyphonic_evaluation.py`, `beat_evaluation.py` + `beat_report_html.py`, `loudness_rec_evaluation.py`, `recommendation_evaluation.py`**: Offline evaluation harnesses (deterministic-metric / detector-stability reporting, research-only polyphonic transcription, the beat/downbeat measurement gate that benchmarks CPJKU/beat_this against the shipping kick-accent heuristic, loudness-recommendation reachability, and the recommendation-quality scorer that grades Phase 2 recommendations against known-settings fixtures — see [`apps/backend/NEEDS.md`](apps/backend/NEEDS.md)). Not on the product path; driven by `scripts/evaluate_*.py`. Deleting them restores the product exactly.
17. **`utils/cleanup.py`**: Periodic artifact-cleanup helpers used by the server background-task loop.

The subprocess isolation means `analyze.py` works as a standalone CLI. Check `apps/backend/JSON_SCHEMA.md` before adding new analyzer output fields. Check `apps/backend/ARCHITECTURE.md` for the full HTTP flow and contract details.
Expand All @@ -196,6 +196,7 @@ Single-page React 19 + Vite + TypeScript + Tailwind CSS v4 app with no router. V
7. **`src/services/mixDoctor.ts`**: Mix advisory logic — client-side scoring and suggestions against measured spectral balance.
8. **`src/services/phase2Validator.ts`** + **`loudnessGuardrails.ts`**: Runtime guardrail. Validates Phase 2 consistency against Phase 1 (`validateBPMConsistency`, `validateKeyConsistency`, `validateLUFSConsistency`, `validateGenreDSPConsistency`, `validateNumericBounds`, `validateLoudnessActionPresence`). `loudnessGuardrails.ts` defines the objective loudness defects (digital clipping via `saturationDetail.clippedSampleCount`, true-peak overs via `truePeak`) that a Phase 2 mastering/dynamics card *must* address — a missing action surfaces as a `MISSING_LOUDNESS_ACTION` violation. The aggregate `validatePhase2Consistency` drives **`Phase2ConsistencyReport.tsx`**, which renders the chain-of-custody report on the results surface (`AnalysisResults.tsx`, `hideWhenClean`) and in full inside the diagnostic log (`DiagnosticLog.tsx`); `App.tsx` computes the report and passes it down.
9. **`src/services/appliedRecommendations.ts`** + **`userLabels.ts`**: Applied-recommendations tracker and persisted-label state used by the audit overhaul.
9a. **`src/services/recommendationVerification.ts`** + **`src/data/recommendationVerification.ts`** + **`components/RecommendationVerificationBadge.tsx`**: Per-recommendation corpus-verification badge (`GOAL.md` sub-goal 4). The data module is a generated artifact from `apps/backend/scripts/evaluate_recommendations.py --verification-artifact` (all-`NONE` until the ground-truth corpus has renders); the service infers a card's domain (mirroring the backend scorer's `infer_domain`) and looks up its confidence band; the badge renders on `AnalysisResults.tsx` recommendation cards, hidden when there is no corpus evidence. Research/proof surface — see [`apps/backend/NEEDS.md`](apps/backend/NEEDS.md).
10. **`src/services/phase1Picker.ts`** + **`phaseLabels.ts`**: Phase-snapshot projection helpers consumed by the results surface.
11. **`src/services/audioFile.ts`**: Client-side audio validation, blank-MIME extension fallback, preview-URL lifecycle.
12. **`src/services/fieldAnalytics.ts`** + **`diagnosticLogs.ts`**: Instrumentation hooks and diagnostic-log capture for the request panel.
Expand Down
Loading
Loading