Skip to content

Add rendering-verification design spec: LOD invariants and a visual-regression corpus#133

Merged
FarhanAliRaza merged 3 commits into
mainfrom
alek/spec-rendering-verification
Jul 22, 2026
Merged

Add rendering-verification design spec: LOD invariants and a visual-regression corpus#133
FarhanAliRaza merged 3 commits into
mainfrom
alek/spec-rendering-verification

Conversation

@Alek99

@Alek99 Alek99 commented Jul 21, 2026

Copy link
Copy Markdown
Member

Summary

Adds spec/process/rendering-verification.md (plus the spec/README.md index entry): the design for the two test surfaces that make constant renderer churn safe during the refine-and-optimize phase. Design only — no implementation in this PR.

The motivating pattern: the recent regression run — density flicker on pan/zoom-out (#118), scatter collapse after repeated box-zoom (#87), dense scatter blanking on double-click (#79), example-asset styling drift (#60) — is all one bug class: pixels went wrong silently, past gates that check "did it render something" rather than "is it still right".

Workstream A — LOD invariant property tests

Hypothesis-driven (already a dev dependency, pattern proven in tests/test_framing_property.py), exercising the native kernels through the real ctypes ABI against O(n) NumPy oracles — no new Rust crates, so no vendoring problem. Ten invariants specified as a contract table, including:

  • M4 extrema-exactness per pixel column (the Tier-1 truthfulness claim)
  • density/hexbin count fidelity + the occupied-bins-only wire contract
  • standalone worker re-bin ≡ kernel re-bin (the Fix standalone density flicker on pan/zoom-out #118 shape)
  • re-bin mass conservation for histogram/bar
  • NaN containment (dossier §19) and f32 offset-encoding error bounds (§4)
  • f64 view round-trips and tier-decision/hysteresis consistency (L1/L2/L3)

Runs inside the existing pytest -q gate with a bounded CI Hypothesis profile (≤ 60 s added).

Workstream B — golden visual-regression corpus

Canvas-readback goldens per chart family × interaction state — first paint is the least interesting entry; the states are where the shipped bugs lived (repeated box-zoom + zoom-out, double-click reset, pan-then-zoom-out-past-home, drill in/out, kernel-less worker re-bin). Gestures driven over CDP with settled-signal waits. Determinism contract: one pinned Chromium, DPR 1, canvas pixels only — DOM chrome stays under the existing conformance layout-box approach; Firefox/WebKit keep the tolerant signature. The repo stays binary-free — no committed PNGs and no Git LFS: the committed baseline is a small text manifest of per-tile signatures and thresholds; images exist only as on-demand CI artifacts (expected/actual/diff triptychs rendered from the PR merge-base); manifest updates ship in the same PR as the rendering change.

Discipline

Related direction: follows the refine-and-optimize focus alongside #129 (view-state layer); the corpus's interaction states are also the safety net for implementing that spec.

Validation

  • uv run ruff check . / targeted pre-commit on the changed files — clean
  • codespell — clean (repo's own "LOD" terminology excepted)

Two workstreams that make renderer churn safe during the
refine-and-optimize phase:

- Property-based LOD invariant tests (Hypothesis through the real ctypes
  ABI against O(n) NumPy oracles): M4 extrema-exactness, density/hexbin
  count fidelity, worker/kernel re-bin equivalence, re-bin mass
  conservation, NaN containment, offset-encoding error bounds, view
  round-trips, and tier-decision consistency.
- A golden visual-regression corpus of canvas-readback baselines per
  chart family x interaction state, pinned to one Chromium engine, with
  committed baselines, a size budget, and diff-triptych CI artifacts.

Includes fail-first calibration against the shipped regressions (#118,
#87, #79) and a phased advisory-to-hard-gate rollout.
@Alek99 Alek99 added the spec label Jul 21, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 21, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 94 untouched benchmarks
⏩ 1 skipped benchmark1


Comparing alek/spec-rendering-verification (2fbcfc9) with main (79b0a3f)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

@greptile-apps

greptile-apps Bot commented Jul 21, 2026

Copy link
Copy Markdown

Greptile Summary

This PR defines two test surfaces for catching silent rendering regressions. The main changes are:

  • Property-based checks for LOD kernels and rendering invariants.
  • A manifest-based visual corpus covering chart families and interaction states.
  • A client settle counter for deterministic browser captures.
  • CI rollout, calibration, runtime budgets, and baseline regeneration rules.
  • An index entry for the new design document.

Confidence Score: 5/5

This looks safe to merge.

The histogram conservation rule limits cross-resolution checks to nested subdivisions. The offset-encoding invariant defines a measurable error bound. The visual harness makes the settle counter an explicit client deliverable. No blocking issues remain in the changed specification.

T-Rex T-Rex Logs

What T-Rex did

  • The pre-validation state shows that the codespell module was missing, as documented in docs-validation-01-before.log.
  • The post-validation state shows Ruff success and that the repository-configured codespell blocker is active, as documented in docs-validation-02-after.log, including the commands run, the working directory, outputs, and exit codes.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
spec/process/rendering-verification.md Defines the LOD property suite, visual corpus, deterministic capture contract, and phased CI rollout.
spec/README.md Adds the rendering-verification design to the specification index.

Reviews (2): Last reviewed commit: "Keep the visual corpus binary-free: mani..." | Re-trigger Greptile

Comment thread spec/process/rendering-verification.md Outdated
Comment thread spec/process/rendering-verification.md Outdated
Comment thread spec/process/rendering-verification.md Outdated
Alek99 added 2 commits July 21, 2026 12:02
- A5: per-bin oracle fidelity at the implementation's own edges replaces
  the nested-conservation framing; nesting is checked only on
  constructed k-times subdivisions so independent nice edges cannot fail
  a correct implementation.
- A9: state the offset-encoding error bound as a formula (one f32 ULP of
  the offset-relative magnitude, <= 2^-23 x max(window span, |value -
  offset|)) instead of an undefined epsilon.
- 4.2a: the settled signal the corpus waits on does not exist in the
  client today; specify it as a Phase-3 deliverable — a CDP-observable
  monotonic settle counter gating on no pending kernel round-trips,
  worker re-bins, or scheduled frames.
…PNGs

Replace committed golden PNGs with a committed text manifest of
per-tile signatures (hash + per-channel means) and thresholds. The repo
uses no Git LFS and should not start for regenerable data, and
committed PNGs under tests/ would ship in the sdist. Images exist only
as on-demand CI artifacts rendered from the PR merge-base; regen output
is git-ignored, and the check fails if a PNG appears under
tests/visual/. The comparison metric is restated to be computable from
the manifest alone, with pixel-level diffing demoted to a CI-artifact
diagnostic.
@FarhanAliRaza
FarhanAliRaza merged commit 7566e21 into main Jul 22, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants