Nightly audit: 1 issue(s)

[slittycode]

Nightly audit — 2026-05-14

Automated Phase-boundary + test-guardrail audit.

Result

Step	Outcome
Unit suite (vitest run)	3 failed / 557 passed / 561 total
Playwright smoke (playwright test)	44 passed / 2 skipped / 0 failed
Phase-boundary scan	No violations found
Phase 1 → Phase 2 contract	Consumed read-only downstream — OK

Repo-structure note (task assumptions vs. reality)

The task brief assumes src/phase2/ and src/phase3/ directories and a root-level
npm ci / Essentia.js. The actual repo does not match:

1. There is no root package.json. Frontend deps live in apps/ui; the setup
   script's npm ci was run there instead.
2. Phase 1 DSP is Python/Essentia in apps/backend (analyze*.py), not
   Essentia.js. It is the ground-truth measurement layer.
3. Phase 2 (interpretation) is apps/backend/server_phase2.py + Gemini, with the
   frontend guardrail in apps/ui/src/services/phase2Validator.ts.
4. Phase 3 is not implemented (patchSmith.ts is still the open item in
   BACKLOG.md). Nothing to audit there.

The audit was performed against the real structure.

Failing tests

All 3 failures are in fixture-dependent live decision-gate comparators, all
introduced together in commit 94a26fbd (PR #18):

1. apps/ui/tests/decision_gate.multi.live.test.ts:147
   — decision-gate (multi-model) compares validator output across Gemini variants
2. apps/ui/tests/decision_gate.real.live.test.ts:152
   — decision-gate (multi-model, real track) ... on Vtss-CantCatchMe.mp3
3. apps/ui/tests/decision_gate.stems.live.test.ts:168
   — decision-gate (multi-model, stem-aware, real track) ... with stemAnalysis populated

Failure mode (all three): AssertionError: expected 0 to be greater than 0
at expect(ok.length).toBeGreaterThan(0). Each test loads pre-generated Gemini
snapshots (/tmp/decision_gate_<model>.json or apps/backend/.runtime/reports)
produced by live multi-model Gemini runs. With no snapshots present, ok.length
is 0 and the assertion fails.

Root cause: test-harness bug, not a product regression. Each file's own
docstring claims it "Skips when no snapshots are present, so the suite stays
clean elsewhere" — but the code asserts toBeGreaterThan(0) instead of calling
it.skip / this.skip(). The intended skip-when-no-fixtures behavior was never
implemented.

Not a Phase-boundary violation. These tests exercise the boundary guardrail
(phase2Validator.ts) read-only; they don't cross it.

Note: npm run test:unit (scoped to tests/services) is green — these three
live tests sit at tests/ root and are only picked up by the unscoped npm test.

Phase-boundary scan — no violations

Checked every path where Phase 2 / interpretation touches Phase 1 / measurement
data:

1. Storage layer (apps/backend/analysis_runtime.py) — measurement results
   live in the measurement_outputs table; interpretation results live in the
   separate interpretation_attempts table. _update_measurement_row is called
   only from the measurement stage (analysis_runtime.py:615,686,704) and
   progress updaters in server.py. The interpretation stage references the
   measurement via FK grounded_measurement_output_id — a read-only pointer,
   never a write-back.
2. server_phase2.py:742 _normalize_measurement_result_for_gemini — builds
   dict(payload) / dict(stem_entry) copies and only renames spectral keys
   (spectralCentroid → spectralCentroidMean, etc.) for prompt presentation so
   Gemini's field names match the validator contract. Operates on copies; the
   stored measurement is untouched. No recompute of values.
3. apps/ui/src/services/analysisRunsClient.ts:184 projectPhase1FromRun —
   returns { ...measurement, transcriptionDetail: pitchNote }: a new object that
   attaches the Layer-2 pitch/note result as an additive field. No measured DSP
   value is overwritten; no mutation of the source object.
4. apps/ui/src/services/phase2Validator.ts — reads Phase 1 strictly
   read-only to check Phase 2 consistency. This is the boundary guardrail.

Phase 1 → Phase 2 contract — OK

Phase1Result (apps/ui/src/types/measurement.ts) flows into Phase 2 only as:
(a) read-only input to the Gemini prompt builder, and (b) read-only input to
validatePhase2Consistency. No downstream writer assigns to tonal-balance,
dynamics-envelope, stereo-image, loudness (LUFS/LRA/peak), or spectral
centroid/rolloff fields.

Needs human review

1. decision_gate.*.live.test.ts skip logic. Decide intended behavior:
   either gate these behind an env flag (e.g. RUN_GEMINI_LIVE_SMOKE, as the
   smoke suite already does) or have them it.skip when no snapshots/fixtures are
   present — matching their own docstrings. Until then npm test is red on any
   machine without pre-generated Gemini snapshots, including CI.

Suggested next steps (no auto-fix applied)

1. Fix the three live tests to skip cleanly when fixtures are absent, or move them
   under an env gate so the default npm test stays green.
2. Optional: align npm test and npm run test:unit scope, or document that the
   nightly guardrail should run test:unit for the deterministic subset.

---

Generated by the asa-nightly-phase-1-invariant--test-guardrail scheduled task.
No source files were modified; no fixes were applied.

[slittycode]

Verdict: COMMENT

Summary

Large mixed PR: a backend grammar post-processor for Phase 2 text, substantial frontend UX improvements (CitationBlock, applied-recommendations tracker, progress-card honesty fixes, nav reorder, idle panel), and the nightly audit report documenting 3 pre-existing test failures from PR #18. All new logic has test coverage. Phase boundary is clean. Phase 1 schema is untouched. The 3 failing tests predate this PR and are flagged correctly in the description — they're the only thing standing between npm test being green.

Findings

Should fix (not introduced by this PR, but blocks CI)

apps/ui/tests/decision_gate.*.live.test.ts (three files from PR #18) — the docstrings say "Skips when no snapshots are present" but the code unconditionally asserts expect(ok.length).toBeGreaterThan(0), which fails on any machine without pre-generated Gemini snapshots. npm test is red on CI until this is resolved. The fix is one of: (a) gate behind RUN_GEMINI_LIVE_SMOKE=true (matching the existing smoke pattern), or (b) add a guard that calls it.skip when the fixture directory is absent. Either is fine; the current state is not.

Worth considering

server_phase2.py — _to_gerund doesn't handle consonant doubling (controls → "controling" instead of "controlling"). Documented in the test comments. If Gemini starts emitting "by controls" the output gets worse, not better. A small exception dict at module level for the 5–10 most common affected verbs would close the gap without trying to implement English orthographic rules.

server_phase2.py:111 — return record if updated or True else record — the updated flag is computed but the or True makes it a no-op. Dead code; consider just return record.

appliedRecommendations.ts — no max-entries or TTL guard on the localStorage store. Orphaned records accumulate over many sessions. Low priority for V1 (acknowledged in the docstring), but if a producer runs ASA on 50 tracks, the key will hold 50 stale entries forever. A simple prune-to-N-most-recent on write would contain it.

Test results

Unit: 3 failed / 557 passed — all 3 failures are in decision_gate.*.live.test.ts (pre-existing, PR #18). npm run test:unit is green.
Playwright smoke: 44 passed / 2 skipped / 0 failed.

Phase boundary check

Clean.

• _apply_phase2_grammar_fixes mutates the normalized Phase 2 dict in-place. Never touches Phase 1 data.
• CitationBlock / phase1Picker.ts read Phase 1 via dotted-path traversal, strictly read-only.
• appliedRecommendations.ts is a UI concern (localStorage); no contact with either measurement layer.
• CONFIDENCE_PAIRS export from phase2Validator.ts is a correct refactor — single source of truth now shared by the picker and the validator.
• Math.round(phase1.bpm) in MeasurementDashboard.tsx is display-only; the underlying phase1.bpm float is unchanged.

---

Generated by Claude Code