Add rendering-verification design spec: LOD invariants and a visual-regression corpus#133
Conversation
Two workstreams that make renderer churn safe during the refine-and-optimize phase: - Property-based LOD invariant tests (Hypothesis through the real ctypes ABI against O(n) NumPy oracles): M4 extrema-exactness, density/hexbin count fidelity, worker/kernel re-bin equivalence, re-bin mass conservation, NaN containment, offset-encoding error bounds, view round-trips, and tier-decision consistency. - A golden visual-regression corpus of canvas-readback baselines per chart family x interaction state, pinned to one Chromium engine, with committed baselines, a size budget, and diff-triptych CI artifacts. Includes fail-first calibration against the shipped regressions (#118, #87, #79) and a phased advisory-to-hard-gate rollout.
Merging this PR will not alter performance
Comparing Footnotes
|
Greptile SummaryThis PR defines two test surfaces for catching silent rendering regressions. The main changes are:
Confidence Score: 5/5This looks safe to merge. The histogram conservation rule limits cross-resolution checks to nested subdivisions. The offset-encoding invariant defines a measurable error bound. The visual harness makes the settle counter an explicit client deliverable. No blocking issues remain in the changed specification.
What T-Rex did
Important Files Changed
Reviews (2): Last reviewed commit: "Keep the visual corpus binary-free: mani..." | Re-trigger Greptile |
- A5: per-bin oracle fidelity at the implementation's own edges replaces the nested-conservation framing; nesting is checked only on constructed k-times subdivisions so independent nice edges cannot fail a correct implementation. - A9: state the offset-encoding error bound as a formula (one f32 ULP of the offset-relative magnitude, <= 2^-23 x max(window span, |value - offset|)) instead of an undefined epsilon. - 4.2a: the settled signal the corpus waits on does not exist in the client today; specify it as a Phase-3 deliverable — a CDP-observable monotonic settle counter gating on no pending kernel round-trips, worker re-bins, or scheduled frames.
…PNGs Replace committed golden PNGs with a committed text manifest of per-tile signatures (hash + per-channel means) and thresholds. The repo uses no Git LFS and should not start for regenerable data, and committed PNGs under tests/ would ship in the sdist. Images exist only as on-demand CI artifacts rendered from the PR merge-base; regen output is git-ignored, and the check fails if a PNG appears under tests/visual/. The comparison metric is restated to be computable from the manifest alone, with pixel-level diffing demoted to a CI-artifact diagnostic.
Summary
Adds
spec/process/rendering-verification.md(plus thespec/README.mdindex entry): the design for the two test surfaces that make constant renderer churn safe during the refine-and-optimize phase. Design only — no implementation in this PR.The motivating pattern: the recent regression run — density flicker on pan/zoom-out (#118), scatter collapse after repeated box-zoom (#87), dense scatter blanking on double-click (#79), example-asset styling drift (#60) — is all one bug class: pixels went wrong silently, past gates that check "did it render something" rather than "is it still right".
Workstream A — LOD invariant property tests
Hypothesis-driven (already a dev dependency, pattern proven in
tests/test_framing_property.py), exercising the native kernels through the real ctypes ABI against O(n) NumPy oracles — no new Rust crates, so no vendoring problem. Ten invariants specified as a contract table, including:Runs inside the existing
pytest -qgate with a bounded CI Hypothesis profile (≤ 60 s added).Workstream B — golden visual-regression corpus
Canvas-readback goldens per chart family × interaction state — first paint is the least interesting entry; the states are where the shipped bugs lived (repeated box-zoom + zoom-out, double-click reset, pan-then-zoom-out-past-home, drill in/out, kernel-less worker re-bin). Gestures driven over CDP with settled-signal waits. Determinism contract: one pinned Chromium, DPR 1, canvas pixels only — DOM chrome stays under the existing conformance layout-box approach; Firefox/WebKit keep the tolerant signature. The repo stays binary-free — no committed PNGs and no Git LFS: the committed baseline is a small text manifest of per-tile signatures and thresholds; images exist only as on-demand CI artifacts (expected/actual/diff triptychs rendered from the PR merge-base); manifest updates ship in the same PR as the rendering change.
Discipline
Related direction: follows the refine-and-optimize focus alongside #129 (view-state layer); the corpus's interaction states are also the safety net for implementing that spec.
Validation
uv run ruff check ./ targeted pre-commit on the changed files — clean