Skip to content

test: cross-boundary Phase 1 contract gate — kill the silent-field-drop bug class - #161

Merged
slittycode merged 3 commits into
mainfrom
claude/quirky-sagan-jaz8hm
Jun 11, 2026
Merged

test: cross-boundary Phase 1 contract gate — kill the silent-field-drop bug class#161
slittycode merged 3 commits into
mainfrom
claude/quirky-sagan-jaz8hm

Conversation

@slittycode

Copy link
Copy Markdown
Owner

Why

The Phase 1 contract lives in three hand-maintained reconstructions — analyze.py output, _build_phase1 in server_phase1.py, and parsePhase1Result in backendPhase1Client.ts — plus two declarative mirrors (JSON_SCHEMA.md, types/measurement.ts), and nothing executable defended the boundary. The 2026-05-30 full review named this the structural root cause of the dropped reverbDetail.perBandRt60/preDelayMs and vocalDetail.stemEnergyRatio/stemOtherCorrelation citation fields ("no generated source of truth and no cross-app test"); CLAUDE.md tripwires #3/#4 exist because this keeps biting. This PR makes the bug class fail loudly in CI instead of relying on the tripwire being remembered.

What

Backend — golden snapshot becomes the generated cross-app source of truth (apps/backend/tests/test_phase1_golden.py):

  • build_golden now records a recursive, structure-only keyTree: dicts recurse, lists of objects record the union of element keys, scalars reduce to type categories. Comparison tolerates the known runner-noise axes (nested null↔scalar flips, empty↔populated lists) and bites on every container-shape change (key added/removed, dict↔scalar, objects↔scalars). KEYTREE_PRUNE_PATHS (seeded empty) is the documented one-line loosening path if a flaky subtree ever surfaces.
  • New test_phase1_keytree_matches_golden gate; goldens predating keyTree skip visibly until the next UPDATE_PHASE1_GOLDEN=1 re-baseline arms it. The golden here is regenerated, so the gate ships armed (+426 purely additive lines).
  • New BuildPhase1KeySupersetTests: pins that _build_phase1 forwards every golden top-level key — the third reconstruction layer, previously unguarded (a key missed there never reaches the frontend at all).
  • 14 KeyTreeMetaTests prove the comparator bites and doesn't over-bite.

Frontend — the parity gate (apps/ui):

  • tests/fixtures/phase1FullPayload.ts: the inline validPayload promoted to a shared, envelope-shaped fixture, enriched to cover all 66 golden top-level keys (9 were missing: phase1Version, lufsCurve, spectralBalanceTimeSeries, saturationDetail, snareDetail, hihatDetail, transientDensityDetail, pitchDetail, stemAnalysis) with realistic nested values.
  • tests/services/phase1ContractParity.test.ts — five gates driven by the committed backend golden:
    • A1: every golden top-level key exists (non-null) in the fixture.
    • A2 (reverse): every fixture key is a golden key or a declared envelope addition — catches fixture rot when the backend deletes a field.
    • A3: every nested keyTree path exists in the fixture. Auto-arms off the golden; active in this PR.
    • B: the fixture survives the real parsePhase1Result with every non-null input path intact — the silent-drop check itself.
    • C: the canonical run path strips exactly transcriptionDetail and nothing else.
    • Server-side transforms (stereo hoist, spectral *Mean renames, staged-path strip) are a small declared map with server_phase1.py/analysis_runtime.py citations — an undeclared future transform fails the gate rather than passing silently.

Two real catches before merge (the gate proving itself):

  1. parsePhase1Result silently dropped phase1Version — emitted by the server, declared in the type, never forwarded. Fixed (one line).
  2. Arming Gate A3 surfaced stale segmentLoudness/segmentSpectral fixture shapes ({start, value} vs the real {segmentIndex, start, end, lufs, lra} contract) — corrected.

Docs: tripwires #3/#4, JSON_SCHEMA.md, and apps/ui/AGENTS.md now point at the executable gates.

Verification

  • apps/ui: npm run verify green — lint, 756 unit tests (all six new gates active, zero skips), build, 49 smoke tests.
  • apps/backend (fresh py3.11 venv): tests.test_phase1_golden 27 tests OK with the keytree gate active; tests.test_analyze + tests.test_server 374 tests OK.
  • Mutation checks observed red→green: parser-side drop (reverbDetail.preDelayMs forwarding removed → Gate B names exactly that path), fixture-side gap (vocalDetail removed → Gates A1+A2 name it), and the live phase1Version drop (Gate B red on first run, green after the fix).

https://claude.ai/code/session_014eU638tNAHuV3bXDZStuNU


Generated by Claude Code

claude added 3 commits June 11, 2026 21:04
…op class

The Phase 1 contract lives in three hand-maintained reconstructions
(analyze.py output, _build_phase1 in server_phase1.py, parsePhase1Result in
backendPhase1Client.ts) plus two declarative mirrors (JSON_SCHEMA.md,
types/measurement.ts), and nothing executable defended the boundary — the
2026-05-30 full review traced the dropped reverbDetail/vocalDetail citation
fields to exactly this gap (CLAUDE.md tripwires #3/#4). This makes the
boundary executable end to end:

Backend (tests/test_phase1_golden.py):
- The golden snapshot now records a recursive structure-only `keyTree`
  (dicts recurse; lists of objects record the union of element keys; scalars
  reduce to type categories; nested null<->scalar flips are tolerated as
  runner noise, container-shape changes always bite). Regenerated golden
  arms it (+426 lines, purely additive).
- New keytree gate compares every nested key the analyzer emits against the
  committed tree; goldens predating keyTree skip visibly, not silently.
- New superset gate pins that _build_phase1 forwards every golden top-level
  key — the third reconstruction layer, previously unguarded.
- KeyTreeMetaTests prove the comparator bites (and doesn't over-bite).

Frontend (apps/ui):
- tests/fixtures/phase1FullPayload.ts: the inline validPayload promoted to a
  shared, enriched envelope-shaped fixture covering all 66 golden top-level
  keys (9 were missing) with realistic nested values.
- tests/services/phase1ContractParity.test.ts: five gates driven by the
  backend golden — A1 every golden key exists in the fixture; A2 reverse
  (fixture rot on backend deletions); A3 every nested keyTree path exists in
  the fixture (auto-arms off the golden); B the fixture survives the real
  parsePhase1Result with every non-null path intact (the silent-drop check);
  C the canonical run path strips exactly transcriptionDetail and nothing
  else. Server-side transforms (stereo hoist, spectral *Mean renames, staged
  strip) are a declared, cited map — an undeclared transform fails the gate.
- Fix the live instance the gate caught on first run: parsePhase1Result
  silently dropped phase1Version (emitted by the server, declared in the
  type, never forwarded). Gate A3's arming also surfaced stale
  segmentLoudness/segmentSpectral fixture shapes — corrected to the real
  analyzer contract.

Docs: tripwires #3/#4, JSON_SCHEMA.md, and apps/ui/AGENTS.md now point at
the executable gates.

Verified: apps/ui `npm run verify` green (756 unit incl. the 6 new gates
active with zero skips, 49 smoke); backend test_phase1_golden (27),
test_analyze + test_server (374) green in a fresh py3.11 venv; mutation
checks observed red-then-green for a parser-side drop (preDelayMs), a
fixture-side gap (vocalDetail), and the live phase1Version drop.

https://claude.ai/code/session_014eU638tNAHuV3bXDZStuNU
The review job authenticates with the subscription OAuth token
(CLAUDE_CODE_OAUTH_TOKEN), which expires roughly yearly; an expired
token 401s the whole job and must not block otherwise-green PRs.

https://claude.ai/code/session_014eU638tNAHuV3bXDZStuNU
Job-level continue-on-error greens the workflow run but the job's
check run still reports failure in the PR checks list (verified on
this PR). Step-level keeps the check itself green — genuinely
advisory.

https://claude.ai/code/session_014eU638tNAHuV3bXDZStuNU
@slittycode
slittycode merged commit a9317f2 into main Jun 11, 2026
5 checks passed
@slittycode
slittycode deleted the claude/quirky-sagan-jaz8hm branch June 11, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants