Finance Charting surface [Don't Merge] - #1
Closed
Alek99 wants to merge 172 commits into
Closed
Conversation
Implements the design dossier's Phase 0 (docs/design-dossier.md §11/§25/§35):
- Zero-dependency Rust core (C ABI): zone maps (§22), offset-encoded f32
(§4/§16), M4 min/max decimation (§5 Tier 1); ctypes binding with zero-copy
NumPy pointers and a loud NumPy fallback (§33)
- Data-less {traces,layout} spec + single-copy column store with ingest copy
accounting (§9, §4, §29)
- Dependency-free WebGL2 anywidget client: instanced SDF points, instanced
line segments, DOM chrome, pan/zoom as uniform updates, kernel-side
re-decimation round-trip with stale-while-revalidate (§7, §17, §28)
- Tests for the two front-loaded thesis risks (§25): ms-timestamp precision
under viewport re-centering, M4 spike preservation
- Benchmark harness (§12), headless Chromium render smoke, CI wheel matrix
with install-size budget (§33)
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
- JS: tick-label trailing-zero strip corrupted integers (100 -> 1); tick loops could spin at extreme zoom (step below f64 ulp); zoom-in now clamps at the anchor's ulp region; tier-update buffers handle unaligned comm DataViews - Python: guard zero-length arrays before C calls (empty NumPy arrays can carry a null pointer — UB for slice::from_raw_parts even at len 0) - hatch hook: don't assume force_include exists in build_data Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
The render smoke test caught a real bug: WebGL2 rejects a program whose shared uniforms (u_width) differ in precision between stages — vertex floats default to highp, fragment shaders declared mediump. Fixed both fragment shaders to highp. Two new stdlib-only gates that run without numpy/PyPI (and first in CI): - scripts/abi_smoke.py: 12 checks over the Rust C ABI via ctypes + array — precision, M4 spikes, zone-map NaN handling, invalid-arg sentinels - scripts/render_smoke_nonumpy.py: hand-builds a wire payload and drives headless Chromium, reads back lit pixels + DOM chrome nodes Verified locally: cargo test 8/8, ABI smoke 12/12, render smoke lit+chrome OK. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
scripts/bench_native.py times the Rust kernels through the C ABI (stdlib only, no PyPI). Real numbers on this machine: encode ~930 Mpt/s, zone maps ~370, M4 ~258 at 10M points; zoom re-decimation of a 1% window of 10M points is 0.39 ms (~250x under the §17 100-300ms interaction budget). Added to CI and README. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Extends scatter from Tier-0 dots to a complete chart type per the dossier's scatter contract (§5/§28/§36c): Rust core (ABI v2): - bin_2d: additive 2D density aggregation over a viewport (§5 Tier 2), NaN- and bounds-skipping, row0=bottom for GL; grid_max for per-view normalization (§F6) - 5 new cargo tests (13 total); fc_bin_2d C ABI + ctypes + NumPy fallback parity Python: - channels.py: color resolves to constant / continuous (colormap) / categorical (factorized palette); size to constant / continuous. Shipped as one f32/point + a LUT (never per-point RGBA) — a colored+sized scatter is ~16 B/pt (§2) - scatter() gains color/size/colormap/size_range/density; large scatter auto- switches to Tier-2 density (per-point channels drop with a loud warning + a channels_dropped flag, never silent — §28) - density_view (re-bin visible window on zoom, §5) and pick (exact f64 row from canonical for hover, §16/§17); widget handles density_view/pick messages JS client: - per-vertex color (continuous colormap LUT / categorical palette LUT) and size - GPU picking: RGBA8 ID texture + 1px readback -> exact-row DOM tooltip (§17) - Tier-2 density: R32F count texture, log-colormap at composite, per-view max normalization, re-bin round-trip with stale grid drawn until it lands (§17) - colormaps (viridis/magma/plasma/cividis/turbo), gradient/palette legend - fixes another WebGL2 precision-mismatch (int uniform) caught by the smoke Verification (PyPI still blocked, so numpy suite runs in CI): cargo 13/13, ABI smoke 16/16, headless-Chromium smoke now exercises colored+sized scatter + density + picking + standalone hover decode. bin_2d bench: 10M pts -> 512x384 grid in 61 ms. New pytest suites (test_scatter.py, +bin_2d parity) for CI. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
A declarative component API over the same engine — the feel of Reflex's
Recharts components, with no reflex dependency:
fc.scatter_chart(
fc.scatter(x='gdp', y='life', color='continent', size='pop', data=df),
fc.x_axis(label='GDP'), fc.y_axis(label='life'), fc.legend(),
title='Gapminder',
on_hover=lambda row: ..., on_select=lambda sel: ...,
)
- components.py: scatter_chart/line_chart containers composing scatter/line
marks + x_axis/y_axis/legend children into a Figure; snake_case props;
data= + column-name resolution (the Recharts data_key idiom, read directly);
CSS-color-vs-column disambiguation; log-axis warns (v1 roadmap, §30)
- Events: on_hover fires with the exact f64 picked row (§16/§17); on_select
fires with a Selection (indices + .xy() arrays, never JSON). Figure.select_range
is the §34 Tier-A range predicate.
- Box-select: shift-drag rubber band in the client; kernel resolves the range
to indices; unselected marks dim on the GPU via a per-vertex a_sel attribute +
u_selActive. Standalone HTML computes the selection from resident f32 (no
kernel). Double-click clears. legend(show=False) suppresses the legend.
Verification (PyPI still blocked; numpy suite in CI): cargo 13/13, ABI 16/16,
render smoke now also exercises box-select (100 selected -> 51 in sub-range,
mask active). New test_components.py (composition build, data_key, select_range,
Selection payload). Docs + notebook lead with the composition API.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Three-way scatter benchmark on point-count ceiling, speed, memory, and payload size (docs/benchmark.md): - scripts/bench_vs.py: the full comparison. Per library (matplotlib Agg, Plotly Scattergl + Scatter/SVG, fastcharts), per N: build/render/total time, peak memory (tracemalloc + psutil RSS), output/wire bytes, points/sec, and a ceiling probe (largest N under a wall-clock budget without erroring). Runs whatever's installed; missing libs reported, not faked. JSON + Markdown out. - scripts/bench_scatter_native.py: the fastcharts arm alone, stdlib-only, so its numbers are real even with PyPI blocked — data-prep (encode/bin via the C ABI), wire bytes, and real render-to-pixels time in headless Chromium. - CI 'benchmark' job installs plotly/matplotlib/kaleido and runs the three-way comparison, uploading the report as an artifact. Measured fastcharts arm (this machine, software GL): wire payload goes FLAT at 768 KB once density aggregation engages — 0.79 B/pt at 1M, 0.08 at 10M vs a constant 8 B/pt direct — while render cost stays screen-bounded (~150 ms at 10M under SwiftShader). Data prep: 62 ms to bin 10M points. Competitor columns are filled by the CI job; until then the report cites the field's published numbers (deck.gl 10-100M crash, Plotly ~14x memory) clearly labeled as literature. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
…atch So the three-way benchmark produces measured Plotly/matplotlib numbers in CI (GitHub Actions has the network access this container lacks), retrievable via the Actions API. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Ran the three-way harness in GitHub Actions (which has the network access this container lacks) and pulled the measured results back. docs/benchmark.md and the README now carry real numbers for all three libraries, not literature. At 10M points (CI, Ubuntu, Python 3.12): fastcharts 86 ms 2 MB 768 KB payload matplotlib 3230 ms 553 MB 41 KB PNG plotly WebGL 33907 ms 1584 MB 49 KB PNG plotly SVG did not finish (113 s at 3M, over budget at 10M) fastcharts is the only library flat in N: payload and peak memory stop growing once density aggregation engages. ~37x faster than matplotlib, ~394x than plotly-gl, 250-790x less memory; fairness caveat (payload-build vs to-pixels) documented in the report. Also fixes the cargo clippy failures (too_many_arguments on bin_2d; intentional NaN-rejecting !(a>b) comparisons annotated) so the CI test job passes lint. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
… more Full review of the codebase against docs/design-dossier.md invariants. Findings, all fixed with regression tests: P0 — shipped/canonical index mismatch (hover + selection wrong rows). build_payload drops NaN rows from shipped scatter buffers (§19) but discarded the mapping. The client's GPU pick speaks shipped vertex indices; Figure.pick read canonical arrays at that index -> wrong hover row whenever data had NaNs. Inverse bug in selection: select_range returns canonical rows, the client masks shipped positions -> wrong points highlighted. Fix: Trace.shipped_sel mapping, pick() translates shipped->canonical, new Figure.to_shipped_indices translates canonical->shipped for the wire (callback keeps canonical, per §34). P0b — density=False was silently ignored. force_density collapsed the tri-state to bool, so density=False still auto-aggregated above the threshold, violating the documented API. Fix: Optional[bool] tri-state honored by use_density(). P1 — JS seq race froze stale tiers. Hover picks shared this.seq with view requests; a hover after a zoom invalidated the in-flight tier_update (msg.seq !== this.seq) and the stale decimation stuck. Fix: separate _pickSeq. P2 — NaN-blind sort check: any(diff<0) is False for NaN diffs, letting NaN-x lines skip sorting and violate M4's sorted precondition; now not all(diff>=0), NaNs sort last and the m4 window excludes them (asserted: no NaN reaches a vertex buffer, §19). P2 — density=False above the direct ceiling now warns (§28 no-silent rule). P2 — theme refresh re-resolves scatter constant colors, not just lines (§36 live re-resolution). 7 new regression tests. Local: cargo 13/13, ABI 16/16, Chromium render smoke (incl. picking + box-select) pass; numpy suite runs in CI. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
- Figure.to_html: escape '</' in the embedded JSON spec and entity-escape the <title> text, so a title/name shaped like '</script>...' cannot break out of the script context in exported files. Regression test included. - JS tooltip: build with text nodes instead of innerHTML — categorical labels are user data and must never parse as markup. Local: JS rebuilt, render smoke green. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
…ive scatter matrix Two real bugs found auditing every scatter combination a user hits: 1. inf reached the vertex buffer. The ship-time drop keyed on NaN-only null_count and used isnan, so ±inf shipped as inf f32 (corrupts primitives, §19) AND poisoned autorange -> range collapsed to the (0,1) fallback. Fix: treat non-finite (NaN and ±inf) uniformly as invalid across the kernels — zone_maps/min_max/m4/bin_2d now use is_finite, Python drop uses ~isfinite, and normalize_to_unit maps non-finite to the domain floor. Autorange, null_count, and the drop path now agree. (Rust + NumPy fallback both updated; ABI smoke + cargo tests assert inf-as-null.) 2. x/y length mismatch was silent — scatter([1,2,3],[1,2]) shipped mismatched buffers and the client drew min(). Now raises ValueError in scatter and line (color/size were already validated against n). tests/test_scatter_matrix.py: exhaustive sweep — input dtypes (list/ndarray/ f32/int8/datetime/Series-like), color modes (hex/named/rgb/3-digit, continuous int+float, categorical incl. bool/single/25-cats-cycling), size modes, empty/ 1-point/all-same/degenerate-domain, NaN/inf/all-NaN, tier boundaries, force density/direct, multi-trace, mixed line+scatter, pick/select translation, and to_html across every tier. Runs under both backends in CI. Local: cargo 14/14, ABI smoke 18/18, Chromium render smoke green. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
- CI render smoke used the apt/snap chromium-browser, a confined snap that can't read the temp HTML in /tmp and whose GPU process crashes headless (dbus + non-existent-mailbox errors) -> render probe never reached FC_OK. Switch to Playwright's unconfined Chromium (already the local convention). - channels.resolve_color warns when a categorical color exceeds 256 distinct values (the client palette LUT is 256-wide; beyond that codes collide). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
The harness measured bytes and serialize time but stopped before pixels existed for every browser-rendered library — fastcharts' render_s excluded the WebGL draw, and Plotly-HTML/Bokeh/Altair/hvPlot 'render' was pure HTML serialization (no parse, no scene build, no paint). That understated the libraries that ship megabytes of JS the browser must execute, and made the fastcharts comparison unfair in both directions. Add real time-to-first-render (data -> pixels): - benchmarks/_browser.py: headless-Chromium First Contentful Paint measurement (JS inlined, no CDN, virtual-time-advanced through async renders). Validated locally against a fastcharts standalone density page (~180 ms, SwiftShader). - bench_vs.py: adapters for browser libs now yield an artifact_fn; --ttfr loads each output in Chromium and records browser_paint_ms + ttfr_ms. Raster libs (matplotlib/seaborn/datashader/Plotly-kaleido) count build+render (already pixels). New TTFR table + column in the Markdown report. - CI benchmark job installs the full adapter set + Playwright Chromium and runs the TTFR pass (capped at 100k/row since each launches a browser). - docs/benchmark.md documents the metric and the gap it closes. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Addresses the install-friction worry. The truth the README obscured, now fixed: - End users need NO toolchain: 'pip install fastcharts' pulls a platform wheel that already contains the compiled core AND the JS client (CI wheel matrix). - Source builds no longer FAIL without Rust. hatch_build.py degrades gracefully: if cargo is absent (or the build fails), it produces a pure-Python wheel that uses the NumPy fallback (correct, slower, one loud warning) instead of erroring. FASTCHARTS_REQUIRE_CARGO=1 (set on the CI wheel matrix) keeps published wheels from silently shipping without the core. - Node is only needed to EDIT the JS client — the bundle is a committed artifact, so it's never required to install. New CI job 'install_without_rust' builds a wheel on a runner with no Rust and no Node, installs it, and asserts it imports + renders a payload via the NumPy fallback with the expected warning — so the ergonomics promise stays honest. README rewritten to lead with the zero-toolchain wheel path and mark Rust/Node as optional (fast path / JS editing only). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
The Rust binary already ships inside the wheel — but CI only uploaded wheels as artifacts and tagged Linux wheels 'linux_x86_64', which PyPI rejects. So the 'no Rust needed' promise wasn't actually deliverable. Fixes: - Linux wheels are auditwheel-repaired to manylinux (the cdylib needs only libc/libm, nothing to bundle) — valid and installable off-box. Applied in both the CI wheels job (validation) and the release workflow. - New .github/workflows/release.yml: on a v* tag, builds the wheel matrix (native core required) + sdist and publishes to PyPI via trusted publishing (OIDC, no stored token). Each wheel is smoke-installed and asserted to load the native core before publish. Result: 'pip install fastcharts' downloads a prebuilt platform wheel with the compiled core inside — no Rust, no Node, ever. README says so explicitly. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
…free Test job died at lint with 76 errors from a newer ruff: all stylistic classes (Optional->| unions, quoted annotations, unicode multiplication signs in prose, unused unpacked test variables, unused-noqa). Opt out of those in pyproject (py3.9 typing floor is deliberate) and fix the two real ones: an unused import in test_figure.py (F401) and a semicolon statement in abi_smoke.py (E702). install_without_rust failed for the best possible reason: GitHub runners ship Rust preinstalled, so the 'no toolchain' build happily compiled the native core and produced a platform wheel, failing the py3-none-any assertion. The graceful fallback works — the job now strips every cargo from PATH first so it actually exercises the no-Rust path. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Last lint error — the demo notebook's import block was un-sorted under ruff 0.15.20. Group stdlib/first-party and order import-before-from. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
ruff format --check flagged 13 files under the pinned formatter. Applied ruff format (0.15.8, matching CI 0.15.20 output — same 13 files, all now formatted). Pure formatting: no logic changes; ABI smoke still 18/18. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Test job now failed at ty (0.0.56, pre-1.0) with 58 diagnostics, all in test/ benchmark code that intentionally subscripts known-non-None Optionals (pick()->Optional[dict], color_ch: Optional[...]) — ty can't narrow those. Scope ty to python/ (the shippable package, where type-checking has value) and run it advisory (|| warning), so it reports without gating. pytest, ruff, and cargo remain hard gates. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
The macOS wheels built the native core fine but the size-budget step used GNU 'du -sb', which BSD du rejects. Use portable 'wc -c'. Test job, fallback, install-without-rust, and ubuntu/windows wheels are already green. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Abstract the shared machinery so a new chart (histogram, bar, area, heatmap) plugs into seams instead of copy-pasting ingest + payload logic. Behavior- preserving (matched the original branch structure exactly); ABI + render smokes unchanged, pytest verifies in CI. Python figure.py: - _ingest_xy(x, y, kind): the equal-length ingest contract every xy chart shares; line()/scatter() now call it instead of repeating store.ingest + check. - _PayloadWriter: one class owns the wire encoding (ship = offset-encoded geometry §4; ship_scalar = raw f32 channel/grid). build_payload no longer carries nonlocal closures. - Per-kind emitters dispatched by _emit_trace via getattr(_emit_<kind>): the build_payload loop is no longer a monolith. Shared _base_entry (spec skeleton) and _finite_sel (§19 non-finite drop) are reused across kinds. Adding a chart = add one _emit_<kind> method. Python components.py: - Mark appliers registry (_MARK_APPLIERS: kind -> apply fn) replaces the hardcoded scatter/line if/else in Chart.figure(). Adding a chart = one applier + a thin *_chart wrapper. JS renderer already factors the shared paths (_bindScalarAttr, _lut, _upload, _map) with a per-kind draw dispatch in _drawNow — a rectangle renderer slots in there without touching the shared helpers. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
…der gate - Selection moved widget.py -> figure.py: it's the on_select payload with zero widget/anywidget dependency; misplaced module. Now exported from the package root (users type their callbacks) without pulling anywidget at import time (§33 import budget). widget.py re-exports for backward compatibility. - scripts/smoke_render.py was dead code that shouldn't be dead — the only test driving a REAL Figure (line decimation + scatter) through to_html's payload path into a browser; nothing executed it. Wired into the CI test job using the Playwright Chromium already installed there. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
The deferred audit items, now applied with zero behavior change: Python (figure.py 704 -> ~470 lines): - config.py: tier/tuning constants (PROTOCOL_VERSION, thresholds, DENSITY_GRID, palette) shared by figure/interaction/export without import cycles; figure re-exports them (historic import path — tests unchanged). - interaction.py: pick / select_range / to_shipped_indices / decimate_view / density_view moved verbatim (self -> fig); Figure keeps thin delegating methods so the public API is untouched. - export.py: to_html template + escaping moved out; Figure delegates. JS (js/src/fastcharts.js 1337 lines -> 7 ordered parts): - Split exactly at the existing section banners (header/colormaps/theme/ticks/ gl/chartview/entries); build.mjs concatenates in a stated order. The built static artifacts are byte-identical to the previous monolith build (verified by git diff --quiet), so this refactor provably cannot change the client. Verified: ruff check+format clean, ABI smoke 18/18, stdlib render smoke green, built artifacts unchanged. CLAUDE.md/README paths updated. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Two import-order fixes in reflex_fastcharts_app so CI lint stays green. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
Closes the everyday Plotly zoom-parity gap: - Modebar (top-right, faint until hover): zoom in/out, pan/box-zoom toggle, reset. Inline stroke SVGs (currentColor) — no external assets. - Box-zoom: a "zoom" drag-mode fits the view to the dragged rectangle, keyed off the modebar toggle; shift-drag still box-selects. - Center-anchored button zoom shares the deep-zoom precision floor (§16) so we never zoom past f32 resolution. Render smoke extended to assert the modebar renders, zoom-in shrinks the span, box-zoom fits the rectangle, and the drag-mode toggle flips the cursor. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HS5WXAwy3xr5YUH3RH2wzK
50_chartview.js was a 3486-line class holding every ChartView concern.
Decompose the cohesive subsystems into ordered parts that augment the
prototype (no bundler, so `Object.assign(ChartView.prototype, {...})` after
the class definition — `this.*` and hot-path method lookup are unchanged):
- 51_annotations.js annotation canvas overlay (markers/arrows/shapes/labels)
- 52_tooltip.js hover -> source-row resolution + tooltip DOM
- 53_interaction.js pointer/drag/wheel, selection, modebar, view animation
- 54_kernel.js kernel comm: view-requests, streaming append, drill (§16)
50_chartview.js -> 2332 lines (core: layout, GL, marks, draw, chrome, pick).
Render smoke passes unchanged (96.6% lit; tooltip/selection/zoom/drill/
annotation paths all green).
Also fixes a real, latent release blocker found while wiring the build: the
bundler split 60_entries.js on `// ---- exports ----` but that marker shared
its line with descriptive prose, so the prose leaked into index.js as a bare
`(everything below ...)` line — a syntax error in the ESM that widget.py
ships as anywidget's `_esm`. CI never caught it (smoke tests use the IIFE
standalone bundle). Move the prose to its own line and harden build.mjs to
reject any text trailing the marker, so this can't regress. index.js now
parses as valid ESM; new test_widget_bundle_is_valid_esm guards it.
Static-client tests now assert source invariants against the concatenated
js/src parts (split-agnostic) plus each built bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two chrome surfaces still pinned themeable properties inline, which beat a user's utility class on specificity — the audit's remaining customizability gaps: - Annotation labels (DOM) hardcoded font + color inline and carried no slot. Add the `annotation_label` slot, stamp `data-fc-slot`, move font/color defaults into the zero-specificity :where() stylesheet (--chart-annotation- text), and pin color inline only when the annotation explicitly sets one. - The plot cursor was set inline (canvas cssText + _setDragMode), so a `cursor-*` utility couldn't win. Drive it off a `data-fc-dragmode` attribute with defeatable :where() defaults (--chart-cursor/-pan); same for the modebar button's cursor:pointer. Every rendered DOM chrome element is now a defeatable slot. New docs/styling.md documents all 20 slots, the four styling mechanisms, the zero-specificity contract, every --chart-* token, and the canvas limitation (painted marks/shapes aren't CSS-addressable). Tests assert the new slot and that no themeable cursor/font stays inline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`ty check python` now passes clean. The four diagnostics were all the same numpy-stubs limitation: `.max()`/`.min()` mis-resolve their no-arg overload for the kernel f32 grid and the mask-indexed f64 column view, even though the value is a finite scalar at runtime. Documented per-line `# ty: ignore[invalid- argument-type]` at each site rather than reshaping correct code around a stub bug. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The marks themselves now take the customizations users reach for in CSS,
via props that speak CSS (both APIs, parity-guarded):
- fill="linear-gradient(...)" on area/bar/column/histogram: real CSS
syntax — direction keywords, 2–8 stops with % positions (CSS resolution
rules), currentColor = the mark's own resolved color, and premultiplied
interpolation so fades to transparent keep their hue. Mark-space by
default (t runs base->tip along each mark's value axis, so an area fades
from its curve to the baseline); plot-space opt-in via
{"gradient": ..., "space": "plot"}.
- corner_radius / stroke / stroke_width on the bar family: antialiased
rounded-rect + border SDF in the shared rect/bar fragment shader.
corner_radius=(tip, base) rounds only the value end (the classic
rounded-top bar), orientation- and negative-bar-aware via the same v_t
mark coordinate the gradients use. Radius/stroke/gradient at their
defaults compile to the exact old flat-quad output — the render smoke's
lit-pixel count is byte-identical.
- curve="smooth" on line/area: monotone cubic (Fritsch–Carlson — never
overshoots, affine-safe on offset-encoded columns), applied at build and
re-applied on every zoom-refined tier_update window, capped at 32k verts.
Purely visual: hover/tooltips keep reading the source rows.
- All mark colors — gradient stops and strokes included — resolve as live
CSS (var()/oklch()/named) and re-resolve on theme change.
Python validates everything fail-fast (linear-gradient parser with CSS
stop-position semantics in _validate.mark_fill); the client receives
normalized stops only. Headless-Chromium smoke gains 7 pixel probes:
gradient fade on both GPU programs, all-corner + tip-only rounding, stroke
color at the border, and smooth densification (65 GPU verts from 5 source
rows with _cpu intact). docs/styling.md gains the full mark-styling matrix
+ roadmap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The x/y axis baselines were stroked on the chrome canvas, which the DOM stacks *below* the WebGL marks canvas. A filled mark that reaches the baseline (bar, zero-based area) therefore painted over the axis line, so it only peeked between bars — the bars looked like they bled past the axis. Grid lines belong behind the data and stay on the chrome canvas; the axis baselines now render as crisp 1px rules in the labels overlay, which sits above the marks. Bars/areas terminate cleanly on a continuous baseline (verified: the axis-color line now shows on top of the bar's bottom rows). They honor the existing axis_color / axis_width style API and are rebuilt with the labels (static between throttled zoom frames — the plot rect doesn't move on zoom). Marks still fill to the canvas bottom, so the zero-gap baseline guarantee (bar/hist smoke) is unchanged; a new smoke probe asserts the baselines are overlay rules above the marks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dashboards need chrome-less charts that fill their box. Two small primitives: - Figure(padding=...) overrides the label-aware plot margins — a scalar or (top,right,bottom,left) px; 0 gives an edge-to-edge sparkline. Threaded through the composition Chart too; emitted as spec.padding and honored by the client's _layout. - tick_label_strategy="none" now hides every tick label (it previously just skipped collision layout and drew them all — the intuitive meaning was a no-op). Grid/axis colors set to transparent finish the chrome-less look. examples/dashboard/site_overview.py recreates an Ahrefs-style site overview: five metric sparklines (flat line, rising/gradient areas, spike, volatile, decline) built with padding=0 + curve="smooth" + linear-gradient fills, the bundle embedded once and each chart mounted into its card. Charts use width="100%" to fill their panel via ResizeObserver. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`opacity` was silently ignored when a mark used a gradient fill: the gradient branch replaced the color outright, dropping u_color.a. Multiply the (pre- multiplied) gradient sample by the mark opacity so one scalar fades every stop, including a fade-to-transparent, on both rect and area programs. Opacity is now uniform across the mark surface: `opacity` (fill/body, gradient included), `line_opacity` (area outline), and full CSS alpha on any color (`rgba()`, `#rrggbbaa`, `oklch(.. / %)`). The stroke keeps its own color alpha, so translucent-fill + solid-border is `bar(opacity=.3, stroke=.., stroke_width=2)`. docs/styling.md gains an Opacity section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…segment An area is one instanced quad per segment, and the gradient position was v_t = c.y — the corner's fraction *within its own quad* (0 at the baseline, 1 at that segment's top). Because each segment's top sits at a different height, adjacent quads disagreed on the gradient value at a shared screen row, leaving vertical seams down the fill (worst where the curve varies fast). Compute v_t from the fragment's clip-space Y instead (clip -1..1 -> 0..1), so the gradient runs continuously through the plot's vertical space across every segment — no seams, same "color at the curve, fade to the baseline" look. Bars keep their per-quad v_t (each bar is one isolated quad, so no seam). Verified before/after on a 60-point smooth area. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous fix stopped the streaks by anchoring the gradient to the plot box (clip Y), but that made saturation depend on absolute height — a tall peak reached the top of the ramp while a low stretch never did, so the fill read darker under the peaks. Normalize per column instead: 0 at the baseline, 1 exactly at the curve, so the fill is evenly saturated right under the line whatever its height. The trick that keeps it seam-free (where the original per-quad c.y streaked): carry the curve-top, baseline, and fragment Y as three separate varyings — each linear in x and continuous across segments — and divide them in the fragment. Dividing an interpolated ratio facets over the slanted-top quad; dividing separately- interpolated endpoints doesn't. Bars are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bring the README current with this cycle's work: the status line and API section now cover CSS/Tailwind mark styling (gradient fills, rounded/stroked bars, smooth curves, opacity, CSS-alpha colors) and edge-to-edge sparklines (padding=0 + none-hides-labels), with a runnable styling snippet. Add the examples/dashboard/site_overview.py Ahrefs-style dashboard and docs/styling.md to Example Apps, note mark styling in What Exists, and drop annotations from the roadmap backlog (already shipped).
scatter() gains symbol (circle/square/diamond/triangle/cross), stroke, and stroke_width. Each shape is an antialiased SDF in the point fragment shader (u_symbol picks the marker; the point sprite carries its device size so the px stroke width converts to sprite space), and the border is a true ring mixed over the fill — a stroke width with no color borders in the mark color, matching the rect family. u_symbol=0 with no stroke reduces to the old circle (render smoke unchanged). Threaded through both APIs (parity-guarded) and re-resolved on theme change; symbols compose with color/size channels and the selected/unselected states. docs/styling.md updated.
…cted color) mark_style already drove hover color/size/opacity and selected/unselected opacity; the one gap was selected/unselected *color* — the point shader only dimmed alpha. Add per-state recolor: the point carries its selection flag to the fragment shader, which tints toward u_selColor / u_unselColor (mix weight 0 = keep native color) when a selection is active, on the same u_selActive path as the existing dim. So a selection can both highlight the chosen points and gray out the rest in one pass. Verified end-to-end (left-half selection -> magenta, right -> faint gray). docs/styling.md gains an interaction-states section.
line()/area() gain dash=: a preset ("dashed"/"dotted"/"dashdot"/"solid") or an
explicit [on, off, …] px sequence (SVG/CSS convention). Dashes are measured in
screen-space arc length so they stay a constant on-screen size through zoom and
run continuously across every segment of a curve — a naive per-segment phase
would stipple dense polylines.
The line shader gains cumulative-length attributes (a_len0/a_len1) fed from a
per-frame CPU pass: for a dashed trace, _lineDash walks the drawn vertices,
sums device-px segment lengths for the current view, and uploads them; the
fragment walks the pattern array (feathered edges). O(vertices) per draw,
dashed traces only; solid lines set u_dashCount=0 and are pixel-unchanged
(render smoke stable). Smooth/decimated tiers keep the dash geometry current.
Threaded through both APIs (parity-guarded); docs/styling.md documents it.
plotly + matplotlib are used by scripts/bench_vs.py (three-way benchmark) and notebook side-by-sides but were installed ad hoc. Group them under a `bench` optional-dependencies extra, out of `dev` so the main test/CI install stays lean: `uv pip install -e ".[bench]"`.
Figure.to_svg() / Chart.to_svg() render the same decimated wire payload the browser client consumes into a standalone SVG — pure Python, no browser, no new dependencies. Because M4/density decimation runs first (at the export width), output is screen-bounded by construction: a 10M-point line exports in ~4 ms as a ~58 KB, resolution-independent file (measured), where the Chromium to_png path takes seconds. to_png stays as the pixel-exact raster fallback. The renderer (_svg.py) mirrors the client: layout margins/padding, the 30_ticks.js tick math (linear/log/time/calendar/category + formatters), the 10_colormaps.js LUT stops (test-guarded sync), and the full mark styling surface — gradient fills as <linearGradient>, dashes as stroke-dasharray, scatter symbols/strokes, rounded (tip, base) bars as arc paths, and smooth curves emitted as *exact* cubic Béziers of the monotone-cubic Hermite (the control points survive the affine axis map, so no polyline densification). Density/heatmap tiers embed as compact stdlib-encoded PNG rasters. Documented approximations: area mark-space gradients use the area bbox; var() colors fall back to the mark color (no DOM).
Kernel-less pages (to_html exports) used to stretch the density overview
texture on zoom — the client couldn't refine without a kernel round-trip.
Now the recorded §28 sample re-bins in a bundled Web Worker on view change:
the worker script travels inside the bundle and boots from a Blob URL (the
standalone CSP tightens from worker-src 'none' to blob: — own-source workers
only), binning happens entirely off the main thread with transferable
buffers, and the result applies through the LOD plumbing as a view-fitted
grid. The reduction is badged ("zoom re-binned from sample"), the full
kernel-binned overview is restored at the home view, the sample overlay —
the re-bin source — is never touched, and environments without workers fall
back to the old stretched-overview behavior.
The render smoke now runs every probe under the production CSP (string
test-guarded against export._STANDALONE_CSP) and proves the worker path
end-to-end in headless Chromium: zoom a kernel-less density chart -> the
worker returns a grid fitted to the zoom window, badge shown, home restore
intact. The smoke's hand-built density trace gains the sample real payloads
always carry.
This closes the deliberately-deferred Performance item (dossier Phase 1,
worker-side compute for standalone refinement).
Alek99
added a commit
that referenced
this pull request
Jul 9, 2026
Squash-rebase of codex/finance-charting-surface (old-repo PR #1) onto the restructured main — the original branch predated the figure.py mixin split, the chartview.js part split, the multi-axis system, and the mark-styling work, and contained a merge commit the repo ruleset forbids. What lands: - candlestick + ohlc chart kinds through both APIs (Figure.candlestick/ohlc, fc.candlestick(_chart)/fc.ohlc(_chart)), with OHLC columns on Trace, a shared-y-frame emitter (_payload.ship_at keeps all four price columns in one f32 offset frame), and low/high-driven y autorange. - finance.py: FinanceChart/FinanceLayer — overlays, studies, volume panes, drawings — plus 57_layers.js (LAYER_KINDS client registry), candle/OHLC WebGL programs, multi-pane plot layout (volume + oscillator strips), and a right-side price axis (Figure(y_side=...), finance area aliases fill_color/width/fill_opacity/baseline). - example app: candlestick editor + 1B-point live drilldown demo pages, build script, and generated assets. Merge notes: emitters live in _payload.py (the branch's stale figure.py copies were dropped); the pane split is grafted into the current padded/ compact _layout; candle tier updates ride the 54_kernel tier_update path; lazy candleProg matches the lazy-shader cache; grid floor uses the price pane height. Parity/registration guards extended (candlestick/ohlc in MARK_PAIRS, MARK_FACTORIES, sample marks, smoke asset lists, api-examples, dashboard snippets). Full gate: 877 tests, ruff/ty clean, render smoke green.
Alek99
force-pushed
the
codex/finance-charting-surface
branch
from
July 9, 2026 05:27
059ea08 to
233b79a
Compare
Squash-rebase of codex/finance-charting-surface (old-repo PR #1) onto the restructured main — the original branch predated the figure.py mixin split, the chartview.js part split, the multi-axis system, and the mark-styling work, and contained a merge commit the repo ruleset forbids. What lands: - candlestick + ohlc chart kinds through both APIs (Figure.candlestick/ohlc, fc.candlestick(_chart)/fc.ohlc(_chart)), with OHLC columns on Trace, a shared-y-frame emitter (_payload.ship_at keeps all four price columns in one f32 offset frame), and low/high-driven y autorange. - finance.py: FinanceChart/FinanceLayer — overlays, studies, volume panes, drawings — plus 57_layers.js (LAYER_KINDS client registry), candle/OHLC WebGL programs, multi-pane plot layout (volume + oscillator strips), and a right-side price axis (Figure(y_side=...), finance area aliases fill_color/width/fill_opacity/baseline). - example app: candlestick editor + 1B-point live drilldown demo pages, build script, and generated assets. Merge notes: emitters live in _payload.py (the branch's stale figure.py copies were dropped); the pane split is grafted into the current padded/ compact _layout; candle tier updates ride the 54_kernel tier_update path; lazy candleProg matches the lazy-shader cache; grid floor uses the price pane height. Parity/registration guards extended (candlestick/ohlc in MARK_PAIRS, MARK_FACTORIES, sample marks, smoke asset lists, api-examples, dashboard snippets). Full gate: 877 tests, ruff/ty clean, render smoke green.
Alek99
force-pushed
the
codex/finance-charting-surface
branch
from
July 9, 2026 05:28
233b79a to
0e85f21
Compare
Alek99
added a commit
that referenced
this pull request
Jul 9, 2026
Architecture-audit remediation (render path + small-data TTFR): - VAOs (audit #1, the biggest lever): every draw previously re-issued getAttribLocation + enable + pointer + divisor ~6-10x per trace per frame. Bind attribute slots at link time (ATTR_SLOTS in 40_gl.js) so they're compile-time constants, then build one VAO per (trace x draw-config) keyed by buffer ids + channel on/off state, rebuilt only when that signature changes. All draw/pick paths (_drawPoints, _drawLine, _drawArea, _drawRects, _drawBars, _drawHoverPoint, _renderPick) now bind a VAO; the dead _bindScalarAttr/_disableAttr helpers are removed. Drill siblings carry their own VAOs, freed in lodDropDrill + _destroyTraceResources. - Cached caps (audit #4): the density/heatmap draws did a per-frame gl.getParameter(MAX_VERTEX_ATTRIBS) + a disable-every-slot loop to scrub leftover attrib state. VAOs isolate that state per draw, so the driver round-trip and the loop are gone — both grid draws now bind a prebuilt quad VAO. - Lazy shader compile (audit #2): the 8 programs compiled eagerly at every chart init. They now compile on first use behind getters + a cache, so a simple line chart links one program, not seven, before first paint. - Bundle compaction (small-data #3): js/build.mjs gained a comment/whitespace compactor (template/string/regex-literal aware, line-structure preserving so ASI can't shift), shrinking the shipped bundles 193KB -> 154KB (20%) that every to_html() inlines and the widget parses on first paint. Deliberately not a renaming minifier: the client-security tests grep exact code lines in the built bundle, and mangling would need a real JS-parser dependency (violates the dependency-free build, §33). Compacted output is parse-validated via new Function() at build time so a compactor bug fails the build. The one comment-marker assertion in test_static_client_security moved to the source file (comments don't survive compaction; code markers still checked in the bundles). Verified: full headless render smoke green (all families + pick + box-select + LOD drill + context-loss restore) at the identical 96.571% lit fraction as before; pytest 810; bundles fresh.
Alek99
added a commit
that referenced
this pull request
Jul 9, 2026
Squash-rebase of codex/finance-charting-surface (old-repo PR #1) onto the restructured main — the original branch predated the figure.py mixin split, the chartview.js part split, the multi-axis system, and the mark-styling work, and contained a merge commit the repo ruleset forbids. What lands: - candlestick + ohlc chart kinds through both APIs (Figure.candlestick/ohlc, fc.candlestick(_chart)/fc.ohlc(_chart)), with OHLC columns on Trace, a shared-y-frame emitter (_payload.ship_at keeps all four price columns in one f32 offset frame), and low/high-driven y autorange. - finance.py: FinanceChart/FinanceLayer — overlays, studies, volume panes, drawings — plus 57_layers.js (LAYER_KINDS client registry), candle/OHLC WebGL programs, multi-pane plot layout (volume + oscillator strips), and a right-side price axis (Figure(y_side=...), finance area aliases fill_color/width/fill_opacity/baseline). - example app: candlestick editor + 1B-point live drilldown demo pages, build script, and generated assets. Merge notes: emitters live in _payload.py (the branch's stale figure.py copies were dropped); the pane split is grafted into the current padded/ compact _layout; candle tier updates ride the 54_kernel tier_update path; lazy candleProg matches the lazy-shader cache; grid floor uses the price pane height. Parity/registration guards extended (candlestick/ohlc in MARK_PAIRS, MARK_FACTORIES, sample marks, smoke asset lists, api-examples, dashboard snippets). Full gate: 877 tests, ruff/ty clean, render smoke green.
Alek99
added a commit
that referenced
this pull request
Jul 9, 2026
Squash-rebase of codex/finance-charting-surface (old-repo PR #1) onto the restructured main — the original branch predated the figure.py mixin split, the chartview.js part split, the multi-axis system, and the mark-styling work, and contained a merge commit the repo ruleset forbids. What lands: - candlestick + ohlc chart kinds through both APIs (Figure.candlestick/ohlc, fc.candlestick(_chart)/fc.ohlc(_chart)), with OHLC columns on Trace, a shared-y-frame emitter (_payload.ship_at keeps all four price columns in one f32 offset frame), and low/high-driven y autorange. - finance.py: FinanceChart/FinanceLayer — overlays, studies, volume panes, drawings — plus 57_layers.js (LAYER_KINDS client registry), candle/OHLC WebGL programs, multi-pane plot layout (volume + oscillator strips), and a right-side price axis (Figure(y_side=...), finance area aliases fill_color/width/fill_opacity/baseline). - example app: candlestick editor + 1B-point live drilldown demo pages, build script, and generated assets. Merge notes: emitters live in _payload.py (the branch's stale figure.py copies were dropped); the pane split is grafted into the current padded/ compact _layout; candle tier updates ride the 54_kernel tier_update path; lazy candleProg matches the lazy-shader cache; grid floor uses the price pane height. Parity/registration guards extended (candlestick/ohlc in MARK_PAIRS, MARK_FACTORIES, sample marks, smoke asset lists, api-examples, dashboard snippets). Full gate: 877 tests, ruff/ty clean, render smoke green.
Alek99
added a commit
that referenced
this pull request
Jul 9, 2026
Squash-rebase of codex/finance-charting-surface (old-repo PR #1) onto the restructured main — the original branch predated the figure.py mixin split, the chartview.js part split, the multi-axis system, and the mark-styling work, and contained a merge commit the repo ruleset forbids. What lands: - candlestick + ohlc chart kinds through both APIs (Figure.candlestick/ohlc, fc.candlestick(_chart)/fc.ohlc(_chart)), with OHLC columns on Trace, a shared-y-frame emitter (_payload.ship_at keeps all four price columns in one f32 offset frame), and low/high-driven y autorange. - finance.py: FinanceChart/FinanceLayer — overlays, studies, volume panes, drawings — plus 57_layers.js (LAYER_KINDS client registry), candle/OHLC WebGL programs, multi-pane plot layout (volume + oscillator strips), and a right-side price axis (Figure(y_side=...), finance area aliases fill_color/width/fill_opacity/baseline). - example app: candlestick editor + 1B-point live drilldown demo pages, build script, and generated assets. Merge notes: emitters live in _payload.py (the branch's stale figure.py copies were dropped); the pane split is grafted into the current padded/ compact _layout; candle tier updates ride the 54_kernel tier_update path; lazy candleProg matches the lazy-shader cache; grid floor uses the price pane height. Parity/registration guards extended (candlestick/ohlc in MARK_PAIRS, MARK_FACTORIES, sample marks, smoke asset lists, api-examples, dashboard snippets). Full gate: 877 tests, ruff/ty clean, render smoke green.
FarhanAliRaza
added a commit
that referenced
this pull request
Jul 15, 2026
- Legend now (re)freezes options and swatch scaling via _attach() when added to an axes, so a legend constructed against one axes but attached to another picks up the host's dpi/rcParams (review #1). - Legend.__init__ warns on handles/labels length mismatch and on handles without a legend entry instead of dropping them silently (review #4). - JS tracks every legend box (not just the primary) so _resize refreshes max-height on extra legends too (review #2). - JS line swatch uses ?? so an explicit lw=0 draws nothing instead of falling back to 1.5px (review #3). - Drop unreachable return in _best_legend_loc (review #5).
FarhanAliRaza
added a commit
that referenced
this pull request
Jul 15, 2026
* Improve pyplot legend and dash parity * Fix extra legend rendering regressions * Address review: legend attach recompute, resize tracking, lw=0 swatch - Legend now (re)freezes options and swatch scaling via _attach() when added to an axes, so a legend constructed against one axes but attached to another picks up the host's dpi/rcParams (review #1). - Legend.__init__ warns on handles/labels length mismatch and on handles without a legend entry instead of dropping them silently (review #4). - JS tracks every legend box (not just the primary) so _resize refreshes max-height on extra legends too (review #2). - JS line swatch uses ?? so an explicit lw=0 draws nothing instead of falling back to 1.5px (review #3). - Drop unreachable return in _best_legend_loc (review #5). --------- Co-authored-by: Alek <alek@reflex.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.