Skip to content

feat(export): Track 2 — CSV export of Phase 1 time-series fields + schema ADR v1 - #35

Merged
slittycode merged 3 commits into
mainfrom
claude/track2-csv-export-schema-adr-5XA5r
May 13, 2026
Merged

feat(export): Track 2 — CSV export of Phase 1 time-series fields + schema ADR v1#35
slittycode merged 3 commits into
mainfrom
claude/track2-csv-export-schema-adr-5XA5r

Conversation

@slittycode

Copy link
Copy Markdown
Owner

Implements Track 2 from the merged external-repo review (docs/external-repo-review-2026-05-13.md). This is the first net-new user-facing feature delivered from that planning work.

What this adds

A new HTTP endpoint that lets external tooling (REAPER scripts, Max patches, downstream pipelines) pull individual ASA time-series measurements as CSV without parsing the full JSON envelope:

GET /api/analysis-runs/{run_id}/export/csv/{field_path}

{field_path} is a simple dot-path (e.g. lufsCurve.shortTerm) — not JSONPath. Arbitrary nested-key descent isn't supported; only allowlisted fields are exportable. Adding a new exportable field is a deliberate code change in apps/backend/csv_export.py.

v1 exportable fields

Path CSV columns
lufsCurve.shortTerm time, duration, lufs (duration=3.0 — EBU R128 short-term window)
lufsCurve.momentary time, duration, lufs (duration=0.4 — momentary window)
rhythmDetail.tempoCurve time, bpm
spectralBalanceTimeSeries time, subBass, lowBass, lowMids, mids, upperMids, highs, brilliance

Why this shape (and not Partiels' shape verbatim)

The review rejected mirroring Partiels' flat (time, duration, label/value) schema directly — that would collapse ASA's domain-named measurements into per-track Vamp-style tracks, breaking the citation chain (PURPOSE.md Quality Invariant #2). What this PR adopts instead: the tabular pattern, with ASA's domain field names preserved. The CSV columns come straight from the v1 JSON schema. Same data, lower-friction consumption.

What this PR contains

File Role
apps/backend/csv_export.py (new, ~200 lines) Allowlisted registry + per-field serializers + dot-path resolver. Pure stdlib; no external deps.
apps/backend/server.py (+88 lines) New route handler. Same auth + 404 patterns as the existing artifact routes. Three distinct 404 codes (RUN_NOT_FOUND, EXPORT_FIELD_NOT_SUPPORTED, EXPORT_FIELD_NOT_AVAILABLE).
apps/backend/tests/test_csv_export.py (new, 22 tests) Per-field unit tests covering column order, edge cases, empty arrays, missing keys, partial rows. All pass locally.
apps/backend/tests/test_server.py (+133 lines, 4 tests) Route tests: success path with Content-Disposition; each 404 code; runs through real AnalysisRuntime + complete_measurement.
docs/adr/0001-phase1-json-schema-v1.md (new) Ratifies the current Phase 1 shape as v1 with an additive-only compatibility policy. Pins the four exportable time-series field shapes.
apps/backend/ARCHITECTURE.md Route list updated — also adds previously-undocumented routes (/artifacts, /spectral-enhancements, /pitch-note-translations, /interpretations).

Error model

Condition Status Code
Successful export 200 text/csv; charset=utf-8
Run doesn't exist or not owned by caller 404 RUN_NOT_FOUND
Field path not in registry 404 EXPORT_FIELD_NOT_SUPPORTED (message lists all supported paths)
Field is null/missing/empty for this run 404 EXPORT_FIELD_NOT_AVAILABLE

The "not supported" vs "not available" split lets clients distinguish "I asked for the wrong thing" from "this run doesn't have that data populated."

ADR 0001 — Phase 1 JSON Schema v1

Separately landed in this PR because the CSV endpoint's column contract depends on the schema being explicitly versioned. Key decisions:

  • The schema is v1 as of today. Defined by the union of EXPECTED_TOP_LEVEL_KEYS, JSON_SCHEMA.md, and Phase1Result. No separate machine-readable schema artifact — deferred to v2 if an external consumer ever needs one.
  • Additive changes are minor; renames/removals/retypes are breaking (major). Compatibility classification table in the ADR.
  • No HTTP version header. When v2 lands (if ever), the canonical move is a new top-level phase1Version field rather than Accept header negotiation.

Verification

  • 22 csv_export unit tests pass locally (Python stdlib only — no Essentia/FastAPI needed):

    $ python -m unittest tests.test_csv_export -v
    ...
    Ran 22 tests in 0.002s
    OK
    
  • 4 route tests in CsvExportRouteTests run via the existing AnalysisRuntime + patch.object(server, "get_analysis_runtime", ...) pattern that the rest of test_server.py uses. CI will run them.

What this PR does NOT do

  • No frontend UI for export. This is a backend-only addition. Adding a "Download CSV" button in the results view would be a follow-up.
  • No SDIF, LAB, REAPER export. Deferred per the review.
  • No beat/downbeat tabular export. The existing rhythmDetail.beatGrid / downbeats / beatPositions are sample indices, not seconds — converting requires threading the analyzer's sample rate, which is a separate decision.
  • No CLI export tool. The HTTP endpoint is the v1 entry point.

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg


Generated by Claude Code

…hema ADR v1

Implements Track 2 from the external-repo incorporation review
(docs/external-repo-review-2026-05-13.md). Patterned on Partiels'
(time, duration, value) tabular shape, but per-ASA-field with the
domain names preserved — does not flatten ASA's schema into a
Partiels-style flat shape (the review rejected that as a chain-of-
custody regression).

What's new:

- apps/backend/csv_export.py — allowlisted registry of four exportable
  time-series fields with per-field serializers and a small dot-path
  resolver (not JSONPath). 200 lines, no external deps beyond stdlib.
  Public surface: export_field_to_csv, is_supported_field,
  list_supported_fields.

- apps/backend/server.py — new route
    GET /api/analysis-runs/{run_id}/export/csv/{field_path}
  Three 404 error codes (RUN_NOT_FOUND, EXPORT_FIELD_NOT_SUPPORTED,
  EXPORT_FIELD_NOT_AVAILABLE) for the three distinguishable failure
  modes. Sets Content-Disposition: attachment with a sensible filename.
  Auth model matches the existing /api/analysis-runs/.../artifacts route.

- apps/backend/tests/test_csv_export.py — 22 unit tests covering the
  registry, the dot-path resolver, and each serializer's column layout
  and edge cases (empty arrays, missing keys, partial rows). All pass
  locally.

- apps/backend/tests/test_server.py — 4 new tests in CsvExportRouteTests
  covering: success path with content-disposition + body shape;
  unsupported field path returns EXPORT_FIELD_NOT_SUPPORTED with the
  supported list in the message; supported-but-null field returns
  EXPORT_FIELD_NOT_AVAILABLE; unknown run returns RUN_NOT_FOUND.

- docs/adr/0001-phase1-json-schema-v1.md — ratifies the current Phase 1
  shape as v1, defines the additive-only compatibility policy, and pins
  the four exportable time-series field shapes. The schema is the union
  of EXPECTED_TOP_LEVEL_KEYS, JSON_SCHEMA.md, and Phase1Result — no
  separate machine-readable JSON Schema artifact (deferred to v2 if a
  consumer ever needs one).

- apps/backend/ARCHITECTURE.md — updated route list to include the new
  endpoint plus three pre-existing routes that were missing from the doc
  (artifacts, spectral-enhancements, pitch-note-translations,
  interpretations).

Exportable fields in v1:
  lufsCurve.shortTerm         → time, duration=3.0, lufs
  lufsCurve.momentary         → time, duration=0.4, lufs
  rhythmDetail.tempoCurve     → time, bpm
  spectralBalanceTimeSeries   → time, subBass, lowBass, lowMids, mids,
                                upperMids, highs, brilliance

Out of scope (deferred, documented in the review):
- SDIF / LAB / REAPER export formats
- Beat/downbeat tabular export (the existing fields are sample indices,
  not seconds; adding this requires a frame-rate threading decision)
- CLI export tool

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

Clean backend addition: a new allowlisted CSV export endpoint for Phase 1 time-series fields, plus an ADR that formally ratifies the existing schema as v1. All 22 unit tests pass locally. CI was still running at review time. No phase boundary violations, no breaking schema changes.

Findings

Worth considering (non-blocking)

  1. NaN passthrough in serializers (csv_export.py:160–163, 182, 212). float(value) on a numeric field doesn't guard against NaN or Infinity. If the Phase 1 analyzer ever emits a NaN (possible from FFT edge cases on pathological input — silent audio with a DC offset, for instance), it will write the string nan to the CSV. Verified locally:

    'time,duration,lufs\n0.000000,3.0,nan\n'
    

    Most REAPER and Max CSV parsers will treat nan as a string, not a float, and silently misread the column. This is not blocking because (a) JSON doesn't have a NaN literal so the analyzer would need to deliberately emit null or skip the field — both of which are already handled — and (b) the current Phase 1 fields are normalized before emission. But it's the kind of thing that bites you once on a silent 0.0-amplitude stem.

  2. test_empty_array_returns_none only exercises lufsCurve.shortTerm (test_csv_export.py:98–101). The same empty-list path through _serialize_spectral_balance_time_series and _serialize_tempo_curve is untested. Both return None correctly (verified), but the test suite doesn't assert it. Trivial to add; worth it because the empty-time-series case is the most common "measurement ran but produced nothing useful" scenario.

Test results

22 / 22 pass (tests.test_csv_export). CI (Backend + Frontend jobs) was in-progress at review time — no failures reported yet.

Phase boundary check

Clean. The route handler is read-only: it pulls snapshot["stages"]["measurement"]["result"] and serializes it. Nothing in csv_export.py or the new route modifies, re-derives, or overrides any Phase 1 value. _resolve_dot_path is a pure descent with no side effects.

ADR note

ADR 0001 correctly pins the v1 schema as the union of EXPECTED_TOP_LEVEL_KEYS, JSON_SCHEMA.md, and Phase1Result. The "no separate JSON Schema artifact" decision is sound — a drifting .schema.json would be a new failure mode with no current consumer to justify it.


Generated by Claude Code

claude and others added 2 commits May 13, 2026 06:43
Per PR #35 review (both non-blocking, both worth doing):

1. NaN passthrough. The serializers did float(value) on numeric fields
   with no guard. Python's json.loads accepts NaN/Infinity by default
   (non-standard JSON extension), so a pathological analyzer output
   could in principle round-trip a NaN into a snapshot and from there
   into a CSV cell as 'nan' — which most downstream tools (REAPER,
   Max, pandas) mis-parse as a string.

   Added _as_finite_float() helper that converts to float and returns
   None for NaN/Inf/non-numeric. The three serializers now use it
   instead of bare float(); a row with any non-finite field is skipped,
   matching the existing 'missing key -> skip row' behavior.

2. Empty-array coverage was only tested for lufsCurve.shortTerm. Added
   the same assertion for rhythmDetail.tempoCurve and
   spectralBalanceTimeSeries. Trivial; closes the most common
   'measurement ran but produced nothing useful' case.

New tests (14 total):
- 2 empty-array tests for tempo/spectral
- 8 unit tests of _as_finite_float covering None, normal numbers,
  NaN, +Inf, -Inf, non-numeric strings, numeric-string coercion
- 4 end-to-end tests that NaN/Inf in real fields causes the row to
  be skipped from the CSV

Test count: 22 -> 36, all passing locally.

https://claude.ai/code/session_01Uh2eNT1oqjixcox8UX3BTg
@slittycode
slittycode merged commit 5123dc9 into main May 13, 2026
2 checks passed
@slittycode
slittycode deleted the claude/track2-csv-export-schema-adr-5XA5r branch May 13, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants