Skyfall/In the End construction research spike (parallel research layer)

[samscarrow]

What this is

A research spike, strictly parallel to the evidence-first roadmap, turning a piece of trusted artistic ground truth — In the End entering at a Skyfall chorus with the stressed "hard" landing on the sung "fall" (±~1 beat) — into a reproducible empirical case.

What's in it

• src/mashpad/research/ — new package; nothing in analysis/scoring/report/cli imports it, and no weights, thresholds, provenance semantics, or qualification gates changed.
  • construction.py: MashupConstruction — ground truth for a directed, section-anchored, event-aligned arrangement (host/guest as a structural axis distinct from TrackRole; anchor events; convergences with bounded offset/tolerance hypotheses). Every empirical field carries a resolution state (measured/annotated/hypothesis/unresolved) with anti-laundering guards: MEASURED refuses laundering methods, event times refuse MEASURED entirely (no sanctioned seam exists), annotation is the research twin of USER_ASSERTED.
  • alignment_basin.py: the smallest offset-aware scorer — point-process nearest-neighbor distance over annotated event times, blind to titles by API.
• tests/fixtures/construction_skyfall_in_the_end.json — the construction recorded as identity + bounded hypotheses only; real event times live in a local, uncommitted annotation file.
• tests/test_construction_case.py (11 tests) — locks the load-bearing executable negative result: production evaluate_move is structurally offset-blind (shifting every guest section time by any offset yields a byte-identical CompatibilityProfile), plus the basin's two findings: beat-grid events alone leave the intended offset tied with whole-bar shifts (periodic ridge); one aperiodic lyric-anchor pair breaks the tie.
• docs/design-memo-skyfall-construction-case.md — full diagnosis, representation, experimental protocol, success/failure/overfitting criteria, and roadmap feedback (no taxonomy change now; ML judged premature — alignment search → calibration → within-pair learning-to-rank).
• Decision-log entry + CLAUDE.md pointer.

Verification

uv run pytest → 195 passed, 1 skipped (11 new; baseline 184 unchanged). ruff check and ruff format --check clean. No audio committed; no real paths; no new dependencies.

🤖 Generated with Claude Code

https://claude.ai/code/session_014N7eKM6RRxM4uzQ47XALfG

[samscarrow]

Update: human-auditioned witness (djay Pro session, 2026-07-09)

Commit 23632e6 sharpens the case with real session evidence: host read at ~75 BPM (not djay's initial doubled inference — a live instance of the octave-ambiguity failure mode the verdict layer abstains on), 74 BPM shared grid, measure offset host = guest + 22 (77↔55, 78↔56), effective from ~Skyfall chorus 2 through the bridge and final chorus, guest apparently +2 semitones (hypothesis — verify from the session, not the screenshot).

• Three-level hypothesis in the schema: global conformance / structural alignment (GridAlignment + judged AlignedWindows) / local convergence events. "hard"-on-"fall" is now one salient convergence within the window, not the sole target.
• Witness, not target: claim_scope is schema-enforced to "witness" — an existence proof of one working arrangement, never a uniqueness claim Mashpad must recover.
• Honest session provenance: human listening judgments are annotated; djay display/beat-grid readouts are hypothesis with source-naming methods.
• New artifact: measure-keyed construction timeline (research/timeline.py + fixture) with an OffsetAudition ledger (+22 annotated; ±1-measure neighbors explicitly unresolved).
• Protocol revised: correct beat grids first → offset exactness/constancy → window entry/exit → +2 st question → hard/fall landing on the corrected grid → other convergences → neighbor-offset degradation.

203 tests pass (8 new); ruff clean; production scoring/verdict/provenance/qualification untouched.

[samscarrow]

Correction: tempo evidence + viability region (commit b81f1b8)

Two corrections to the session evidence, per the user:

1. The octave error was the host's. djay correctly measured In the End at 105 BPM (now research-layer measured, method djay_tempo_analysis, ordinary source re-verification pending — not an unbounded estimate). Skyfall was the metrical-octave issue: djay read ~148, user-corrected to ~74 (annotated). The correction changed the available transformation path — guest slowed 105→74 (−29.5%) instead of accelerated toward 148 (+41%). Witnessed ratio 74/105 ≈ 0.705 replaces the broad 0.67–0.74 hypothesis, explicitly grid-choice dependent and non-definitional. Severity minimization is recorded as a candidate-selection feature that helped find the construction, not a universal rule.

2. Region, not point. 74 BPM is the witnessed working point, not the unique/optimal grid. New GridAlignment.viable_grid_bpm_region (hypothesis, ~74–90 BPM): asymmetric constraint — host character sets the upper bound (Skyfall increasingly sounds rushed), guest tolerates deep slowing; acceptability is host preservation, not minimal aggregate transformation (at 90, aggregate change is smaller yet viability less certain). The case is one witnessed construction family.

Pre-registered next experiment: TempoAudition ledger + TEMPO_SWEEP_ASPECTS (host naturalness, guest intelligibility, groove, dramatic weight, overall effectiveness) — semi-blind sweep at the same +22 offset: 74 annotated witness, 78/82/86/90 unresolved candidates, 96 beyond-region probe to locate the failure boundary. Goal: estimate a viability curve/interval, never fit one BPM.

Modeling requirements recorded (fixture taxonomy_gap_notes + memo): octave-differing tempo estimates require comparing octave-equivalent interpretations and their implied transformation costs before rejecting a pairing; transformation cost is role-asymmetric and cannot be judged by absolute percentage.

206 tests pass (3 new); ruff clean; production untouched.

[samscarrow]

Refinement: downbeat anchor + aligned-but-muted + selective entrance (commit af12388)

Primary anchor is now downbeat-to-downbeat. New GridAnchor joins each recording's first metrically established downbeat (ground truth = source timestamps + musical function; djay's "Skyfall bar 3 = In the End bar 1" kept only as session_bar_labels — Skyfall's brass opening makes bar counting ambiguous). Measure-offset bookkeeping is now secondary and frame-dependent. Honestly flagged, not resolved: the anchor frame implies ~+2 in djay labels, contradicting the earlier +22 chorus-region readout on a single grid — reconciliation pending (numbering-frame difference vs distinct family member), locked by test_offset_frames_await_reconciliation.

Synchronized ≠ audible. New GuestAudibility (muted/entering/audible) makes aligned-but-muted a first-class state: the guest's first ~7 bars clash (piano intro vs piano intro — meters align, active material doesn't) and stay muted; the guest enters ~its bar 8, with the cadential-convergence enabler recorded as its own hypothesis (cadential_entrance, new EventKind.CADENCE). Four distinctions now structural: alignment ≠ harmonic compatibility; synchronized ≠ audible; valid grid ≠ valid full-duration overlay; app numbering ≠ source structure.

Timeline expanded per aligned measure: source-relative downbeat timestamps (unresolved GroundTruthFields — the grid's ground truth), app bar labels kept separately, sections, audibility, harmonic judgment, texture conflict, cadential events. Re-keyed to the anchor strategy with its numbering frame stated.

Experiment split: (A) can Mashpad recover the correct downbeat alignment despite the ambiguous opening — a natural adversarial case for a future downbeat analyzer; (B) given the grid, can it distinguish inadmissible guest bars 1–7 from the viable bar-8 entrance — production's single-global-key harmonic evidence is window-blind by construction (the harmonic analogue of the offset-blindness result). Layered search model recorded: grid → admissible windows → entrances/mutes/stems → local convergences.

Witness framing preserved. 211 tests pass (5 new); ruff clean; production untouched.

[samscarrow]

Resolution: +2 user-attested; annotation gap stated plainly (latest commit)

Frame question settled by the user. grid.measure_offset = +2 (djay Skyfall bar 3 = djay In the End bar 1), state annotated, method user_attested. The earlier "+22" was a local measure-number readout from a different numbering frame — not a rival alignment. "Reconciliation pending" removed everywhere; stale different-frame ledger entries dropped. Epistemic layering kept: musical anchor = downbeat-to-downbeat (source timestamps still the ground truth), djay labels session-specific, +2 is the witnessed relation in the djay frame.

Tooling honesty (memo + CLAUDE.md). Plain answers: no code loads both audio files (load_track never decodes; only tempo backends decode, one file, tempo-only); no playback exists; no input interface exists; nothing persists annotations into the research fixtures (hand-edited JSON; the referenced local annotation file has no loader); no command/UI exposes any annotation flow. "Annotate the audio" is not an available in-repo step and the docs no longer imply it is.

Smallest missing executable path — identified, deliberately not built: a label-import seam. djay already covers aligned audition; any label editor (e.g. Audacity label-track export, plain TSV) covers click-to-timestamp. The repo needs only: parse export → match label text to event_ids → write local annotations JSON → flip matched construction event times to ANNOTATED (never MEASURED) → emit TimedEvents for the alignment basin. One small module + CLI, no new deps, no annotation app.

211 tests pass; ruff clean; production untouched.

[samscarrow]

Built: the label-import seam (commit 23c9735)

The executable path from real audio to the research structures now exists — and nothing more (still no audio loading, playback, or UI; djay and an external label editor remain the interactive tools).

• mashpad/research/annotations.py — stdlib-only Audacity-style label parsing (loud failures with line numbers); side-aware matching: a label naming an event_id annotates that event (cross-side matches and duplicate named-event labels are loud errors), a label naming an EventKind value ("downbeat", "cadence", …) becomes a grid event, everything else is reported unmatched, never dropped; AnnotationSet local JSON store under fixtures/local/ (gitignored — real timestamps, audio_index.json policy); apply_annotations flips matched event times to ANNOTATED (never MEASURED); basin_events emits TimedEvents with explicit per-kind weights.
• scripts/import_labels.py — thin CLI shim (eval_tempo.py convention): per-side import, match/grid/unmatched/still-unresolved reporting, --dry-run, nonzero exit when nothing matches.
• tests/test_annotation_import.py (11 tests) — includes one end-to-end test: synthetic label files for both sides → CLI twice into one store → applied to the committed construction fixture → basin distinguishes the intended offset from whole-bar corruptions. Fully executable with no audio and no real timestamps in the repo.

The honest workflow is now: audition/decide in djay → click timestamps in a label editor (label text = event_id or a kind like downbeat) → export → uv run scripts/import_labels.py --side host|guest … → run the basin experiment on the result.

222 tests pass; ruff clean; production untouched.

[samscarrow]

Re-centered: automatic discovery slice built (commit 506466c)

Objective restated per the user: two audio files in, ranked construction hypotheses out — no manual pins. The label-import seam is demoted to evaluation-only infrastructure; witnessed values are acceptance evidence (fixture-driven witness_agreement), never constants in the discovery rules.

Review of prior pieces under this standard: construction schema/fixture, alignment basin, offset-blindness negatives, and timeline remain (target representation + capability boundary + evaluation record); annotations/import_labels are evaluation-only; the manual audition ledgers as primary protocol are deprioritized (still valid human acceptance evidence).

The slice (research/discovery.py + scripts/propose_construction.py): pure hypothesis core over beat-granular TrackFeatures (fully unit-testable without audio) + one thin librosa extractor. Pipeline: decode both files → octave-aware metrical interpretations (tracked + half-time from the sanctioned backend's candidates; double-time declared unsearched) → shared-tempo candidates ranked by role-asymmetric transformation cost (host stretch weighted 3× guest — uncalibrated policy default; this is what makes the octave-corrected host reading beat the doubled reading with no hard-coded slower-is-better rule) → pitch shift by mean-chroma rotation → structural anchor = first stable downbeat per side (irregular openings fall out mechanically) → per-aligned-bar chroma/energy admissibility → ranked entry windows + derived aligned-but-muted window → both host/guest assignments searched. Every hypothesis carries explicit evidence + uncertainty; nothing claims MEASURED.

librosa scope: research-layer expansion beyond tempo candidates (onset/beats/chroma/RMS), authorized by the re-centering directive; optional tempo-librosa extra, lazy import, production guardrail unchanged.

Verified: 12 new tests (234 total) incl. a librosa-gated end-to-end on in-test synthesized WAVs. CLI smoke run on synthetic 74/105 BPM chord/click pairs: top hypothesis chose the slow track as host, preserved it (grid ~73.8, guest ×0.714), aligned first stable downbeats, recovered the built-in transposition, and produced a correct AGREES/DIFFERS witness report.

Acceptance run (needs the two local files):

uv run --extra tempo-librosa scripts/propose_construction.py \
  <skyfall> <in_the_end> --witness tests/fixtures/construction_skyfall_in_the_end.json

Success = a top hypothesis recognizably related to the witnessed arrangement (host ~74 not 148, guest ~105, guest slowed onto the grid, downbeat anchor, muted opening, entrance near guest bar 8, +2 st) — every DIFFERS line localizes which signal is too crude.

[samscarrow]

Acceptance run: discovery recovers the witnessed construction ✅

The real recordings were present locally (fixtures/local/skyfall.wav, in_the_end.wav). Running the discovery CLI against them — no manual pins, no annotations, no song-specific constants — the top-ranked hypothesis AGREES with the witnessed construction on every comparable field:

element	witnessed	proposed (#1)
roles	Skyfall host / In the End guest	same (assignment search)
host BPM	~74 (user-corrected from djay's ~148)	76.0 — half-time reading of the tracker's 143.6 (librosa made the same octave error; the asymmetric cost model made the same correction)
guest BPM	105	103.4
shared grid	74; viable region 74–90	76.0, host preserved, guest −26.5%
pitch shift	+2 st	+2 st (chroma score 0.977)
opening	guest aligned-but-muted (piano clash)	no sustained admissible window in bars 1–7 → muted through bar 7
entrance	~guest bar 8	guest bar 8, opening an unbroken 77-bar window (mean fit 0.862)

The per-bar profile shows bars 1–7 oscillating 0.30–0.79 (volatile, no sustained run) and a sustained admissible run beginning exactly at bar 8 — the entrance came from the generic sustained-run rule, not tuning.

Caveats (recorded in the memo): the winning half-time candidate carried backend confidence 0.08 — the octave interpretation won on transformation cost, not tempo evidence, and remains the most fragile link; role and pitch decisions are close leans (0.092 vs 0.115; +2 vs −3 at 0.968); the host-side anchor timestamp is unverified by ear; n=1 — next falsification is the same run on other pairs, including deliberately incompatible ones.

This meets the restated success criterion: Mashpad received the two recordings and independently proposed a construction recognizably related to the witnessed arrangement, with explicit uncertainty and without manual pins.

[samscarrow]

Registration search: the "+22" family member, machine-corroborated

Question: does discovery also propose the original ~+22 relation (user-attested as structurally compatible)? Before: no — the slice evaluated exactly one registration (anchors coincident). Now: yes — search_alignments scores guest-delay offsets 0–48 host bars (admissible coverage × mean window fit), proposes the top registrations, and always proposes the anchor-coincident registration alongside them (it's the registration the anchors define). The witness report now surfaces the best-agreeing proposal, not only #1.

On the real recordings (Skyfall host @ 76 half-time, +2 st):

rank	registration	fit	arrangement
#1–3	guest delayed 25/36/38 bars	0.90/0.895/0.895	guest plays from its bar 1 over mid-song host material
#4	anchors coincide (offset 0)	0.772	muted intro, entrance at guest bar 8 — full 5-AGREES witness match

The delayed region is the global fit maximum: anchor-frame offsets 20–26 all score 0.88–0.90 — corroborating the user's attestation and the alignment basin's periodic-ridge prediction at section scale (a broad compatible ridge, not one sharp optimum). djay's +22 maps to anchor-frame ~+20 if our host bar 1 = djay bar 3; offsets 20/22 rank 14th/12th of 49, within 0.02 of the peak at 25 — exact witnessed offset within the ridge pends host bar-1 frame verification.

Recorded insight, not patched: the coverage metric structurally penalizes the witnessed muted-intro arrangement (silent bars count against coverage; delayed registrations skip them free). Arrangement-aware scoring should measure coverage over the audible plan — and which family member is artistically preferable (dramatic muted-intro build vs. immediate overlay) stays a human judgment the ledgers capture. Timeline ledger records the +22 attestation (user_attested, machine-corroborated).

3 new tests (237 total); ruff clean; production untouched. (Local commits currently blocked by the SSH signing agent — changes staged, will push once signing is approved.)

[samscarrow]

Phrase-class constraint: +19/+20/+21 are now structurally excluded

Probe: discovery should NOT propose the loose-bar neighbors (djay +19/+20/+21) of the valid +22 relation. Before: it couldn't exclude them — the chroma-coverage ridge was nearly flat (anchor-frame 17–26 all 0.87–0.89) because bar-level chroma is blind to phrase grouping.

Now: registrations must differ from the anchor by whole 4-bar phrases (offset ≡ 0 mod PHRASE_BARS) — the anchors' first stable downbeats are assumed to open phrases (a declared assumption in every hypothesis's uncertainty, with the witnessed family's own geometry as internal evidence: its two attested members, anchor 0 and ~+20, differ by exactly five phrases). Off-phrase offsets are structurally wrong, not merely weaker, and are not searched. A strength-based hypermetric estimate runs as corroboration only; on this pair it disagrees (residue 2) at near-chance confidence (0.29/0.28 vs 0.25 floor) and the disagreement is printed in every hypothesis rather than silently resolved.

Real-recording result: proposals are now anchor-frame 36 / 24 / 40 (all ≡ 0 mod 4) plus the always-proposed anchor registration (0 — still the 5-AGREES witness match at #4). djay +19/+20/+21 (anchor 17/18/19 ≡ 1/2/3) are excluded by the general rule, no witnessed constants involved. The attested ~+22 member (anchor 20, fit 0.882) is in-class and searchable, currently rank 4 within the class, 0.013 under the class peak — a ranking question (arrangement-aware coverage), not admissibility.

Locked by test_off_phrase_registrations_are_not_proposed (guest material matching at an off-phrase offset 18 → 17/18/19 must not be proposed; nearest phrase-consistent registration proposed instead, partial clash accepted). 238 tests pass; ruff clean; production untouched. (Commits still blocked by the SSH signing agent — three turns of work staged locally.)

[samscarrow]

Redirect applied: phrase-class gate reverted; joint-overlay feature program started (591798c; the previously blocked three turns landed as af435b0 once the signing agent recovered).

The offset ≡ 0 (mod 4) restriction was justified from the witness pair's own attested members — an overfit workaround, not analytical capability. It is no longer a gate: discovery evaluates every offset again (−1/−2/−3 neighbors included), with phrase-class membership kept as descriptive metadata only. test_off_phrase_registrations_are_not_proposed → test_off_phrase_registrations_stay_evaluated.

New in this commit:

• Experimental design + dataset schema — docs/experiment-joint-registration-features.md + tests/fixtures/registration_corpus_v1.json: success / near-offset-negative / hard-harmonic-negative / random-negative labels with resolution states (most near negatives on pair 1 are unauditioned hypotheses, and the fixture says so), leave-one-song-pair-out protocol, labels never readable by probe/discovery code.
• Minimal joint probe — mashpad.research.joint_features + scripts/probe_registration_features.py: guest frames time-warped onto the host timeline per registration, then transient coincidence, LF interference, spectral-band overlap, heuristic harmonic roughness, and bar-level energy/density complementarity measured from the synchronized cross-source pairs. Measurements only (no rank/fit/verdict fields); every requested offset measured.

First run on the real pair (offsets −3..26, none excluded): no feature discriminates the attested successes (0, 20) from their neighbors. Reported as a failure per the ground rules, with one structural insight: a whole-bar shift lands on another beat of the same grid, so sub-beat transient alignment is preserved at every whole-bar offset by construction — frame-scale statistics measure grid quality, which all candidates share. Full-mix chroma roughness is offset-invariant (0.453–0.457), and the two successes disagree in sign on bar-density correlation (contradictory example on record). Hypothesized next probes: time-resolved per-bar series compared as curves; phrase-boundary/cadence co-occurrence; and auditioning the near negatives, without which there is no discrimination test at all.

253 tests pass; production scoring untouched.

🤖 Generated with Claude Code

https://claude.ai/code/session_014N7eKM6RRxM4uzQ47XALfG

[samscarrow]

Second slice landed (c0853b6): grounded-label audition workflow + phrase-scale joint structure. Production ranking/scoring/gates untouched; 280 tests pass.

Blinded audition workflow (research/audition.py, scripts/audition_registrations.py): identical host comparison windows per offset, per-side RMS matching + common peak, seeded blind IDs (mapping sealed in key.json), structured responses (viable true/false/unsure — multiple viable allowed — plus 1–5 rhythmic/harmonic/phrase-section/masking ratings, confidence, notes), full provenance, strict unseal. Two sessions are rendered and waiting for ears (gitignored, never committed):

• fixtures/local/auditions/anchor_neighborhood/ — offsets −3..+3, host bars 8–16, 7 blinded clips, seed 41
• fixtures/local/auditions/delayed_neighborhood/ — offsets 17..23, host bars 28–36, 7 blinded clips, seed 87

The −3..+3 labels for this pair stay unresolved until those are auditioned; nothing assumes neighbors are negatives.

Phrase-scale trajectories (research/trajectories.py): ordered per-bar series (onset density, band energies, chroma, harmonic-change, tension, novelty, repetition, crude midband-salience/build/drop/cadence proxies) compared as shapes — local windowed correlation, complementarity, change-point co-occurrence (±1 bar), foreground-density collision, localized conflict maxima with bar location.

Stem-aware path (research/stems.py): instrumentation only — external stems as data (no new dependency), crude pseudo_ HPSS fallback with deliberately no vocal pseudo-stem; every measurement names its stem sources and abstains when they're missing.

Within-pair ranking (research/evaluation.py) on the real pair, reported honestly: the strict run (successes 0/20 vs attested negatives 17/18/19) shows five features at pairwise 1.0 in the lower-is-better direction — but four of six comparisons ride cross-region position drift, the direction is in-sample, and those negative labels lack per-offset auditions. The provisional run's top feature (novelty.peak_cooccurrence, lower-better, 0.854) inverts the intuitive structural story — a textbook witness-specific artifact, promoted to nothing. The two successes still disagree in sign on density agreement. Conclusion: the binding constraint is labels, not features; everything gets recomputed after the sessions are unsealed.

Benchmark plan (in the experiment doc): 10–15 pairs stratified by move family (overlay / transition_blend / rhythmic_graft / hook_collision / genre_contrast_blend), sourced from published mashup recipes over owned recordings, discovery-proposed library pairs, and deliberately incompatible controls; song-pair identity grouped; leave-one-pair-out before any production feature proposal.

🤖 Generated with Claude Code

https://claude.ai/code/session_014N7eKM6RRxM4uzQ47XALfG

[samscarrow]

Third slice landed (1edda1d, 3daf122): local audition workbench. No compatibility features added or tuned; production scoring, ranking, gates, and feature definitions untouched. 292 tests pass.

research/workbench.py + scripts/audition_workbench.py — a stdlib-http.server web UI over the existing blinded-session format (no new dependencies, no accounts/auth/cloud):

• One blinded clip at a time: play/pause, replay, prev/next, keyboard controls (space/r/←→/c/y/n/u), and A/B comparison against the immediately previous clip at the matched playback position.
• Captures the existing fields (viable yes/no/unsure — multiple viable allowed — the four 1–5 ratings, confidence, notes) with atomic autosave after every response (temp file + os.replace under a lock).
• Progress shown without offsets. The blind is enforced server-side: key.json is never read before finalization, any request touching it returns 403, and no API payload contains an offset pre-finalization — locked by tests at both the app and HTTP layers.
• Finalize refuses incomplete sessions (same validation as unseal), then writes decoded labels.json + ranking_refreshed.json beside the untouched sealed artifacts and displays the by-offset judgment table plus the refreshed strict ranking grounded in the session's own blinded labels (unsure = excluded, never a negative). Corpus fixture updates remain manual.
• Phone-usable over the local network via --lan (byte-range audio serving for mobile Safari).

The workbench is running now against both rendered sessions. To conduct the auditions (or relaunch it later), from the repo root:

uv run scripts/audition_workbench.py \
    fixtures/local/auditions/anchor_neighborhood \
    fixtures/local/auditions/delayed_neighborhood \
    --lan --port 8765 \
    --trajectories fixtures/local/skyfall_in_the_end.trajectories.json \
    --span fixtures/local/skyfall_in_the_end.joint_probe.json

• This Mac: http://127.0.0.1:8765/
• Phone (same Wi-Fi): http://10.10.10.103:8765/

14 clips total (7 per session, ~26 s each). The sealed offset mappings have not been opened or summarized; finalize from the UI when every clip is answered and the by-offset comparison appears only then.

🤖 Generated with Claude Code

https://claude.ai/code/session_014N7eKM6RRxM4uzQ47XALfG