Nightly audit: 8 issue(s)

[slittycode]

Summary

Automated nightly audit against the Phase 1 ground-truth invariant.

• Vitest: 426/428 pass (2 correctly skipped via decision-gate snapshot guards)
• Playwright smoke: 37 pass / 8 unexpected fail / 1 skipped / 0 flaky
• Phase boundary violations: 0 confirmed, 0 new review items

The eight Playwright failures are independent of the Phase 1 contract:

• Seven share page.goto('/', { waitUntil: 'networkidle' }) against the shared reused Vite dev server (webServer.reuseExistingServer: true) and look like server/HMR flakiness — the simplest one ("header shows SonicAnalyzer brand", no upload, no stubs) still times out at 90 s.
• One (upload-phase1-live.spec.ts:44) needs a real backend to finish analyzing silence.wav within 35 s.

Notable: upload-estimate-phase1.spec.ts:13 has now failed in two consecutive nightlies (2026-05-16, 2026-05-19) without code change — repeats the 2026-05-16 suggestion to inspect recent changes to analysisRunsClient.ts / analyzer.ts for an interpretation-fetch gating change.

The 2026-05-16 estimatePlr boundary review item was closed by adb715ef and verified read-only in this audit. The three previously failing decision-gate suites were skip-gated by e9e97e1e and now correctly fall under the 2 skipped count.

Full triage in audits/nightly-2026-05-19.md. No auto-fixes (per audit policy).

Test plan

• Read the boundary-audit section to confirm the chain-of-custody invariant remains intact
• Triage the eight Playwright failures using the per-file pass/fail breakdown and the waitUntil: 'networkidle' hypothesis
• Decide whether upload-phase1-live.spec.ts should move to a live-only config alongside upload-phase2-live-gemini.spec.ts
• Decide whether the second consecutive upload-estimate-phase1.spec.ts failure warrants a git blame walk on analysisRunsClient.ts / analyzer.ts

🤖 Generated with Claude Code

[slittycode]

Verdict: APPROVE

Summary

Docs-only PR adding audits/nightly-2026-05-19.md, the third entry in an established audit series. Unit tests green (426/428, 2 correctly skipped). Phase boundary invariant clean — the regex audit methodology and the manual per-file enumeration in §3 are both sound, and the estimatePlr open item from 2026-05-16 is correctly confirmed closed. No production code touches.

Findings

Worth considering

1. networkidle scope understated. The suggested fix in §2 says "the five networkidle waits are at [error-states:340/465, file-validation:350, ui-details:360/396, upload-estimate-phase1, upload-phase1-midi:84]". A grep across the smoke directory shows waitUntil: 'networkidle' in at least 20 locations across error-states, file-validation, upload-estimate-phase1, upload-phase1, upload-phase1-live, phase2-degradation, responsive-layout, and upload-phase1-midi. Anyone acting on suggestion #1 will miss most of the instances — the fix scope should say "all occurrences repo-wide" rather than listing five.

2. upload-estimate-phase1.spec.ts:13 is 2-for-2 across consecutive nightlies. The audit correctly flags this under "re-check" and the git blame recommendation still stands. Worth emphasising: at 100% failure rate across all observed runs, this is trending toward "latent regression" rather than flakiness, and treating it as "intermittent" may be letting a real gating change go undetected longer than necessary.

3. _fix_grammar_in_record line number: audit cites line 1037; actual def is at 1034. Three-line slip, probably citing a line inside the function body. Cosmetic but will mislead a reader navigating by line number.

Test results

426 pass / 0 fail / 2 skipped (vitest). 37 pass / 8 fail / 1 skipped (Playwright). All 8 Playwright failures are infra-level (7× networkidle HMR sensitivity, 1× real-backend timeout) with no production-code causation identified.

Phase boundary check

Clean. The field-assignment regex scan returned zero matches in production code. Per-file manual review in §3 covers all Phase 2/3 backend modules and the relevant frontend services, with correct identification of the test-fixture false positive at test_sample_generation.py:106. No new review items.

---

Generated by Claude Code