Add export format parity and a unified export API (ENG-10447)#115
Conversation
One format-selecting surface — to_image(format=...) and extension-inferred,
atomically-written write_image(path) — on charts, facet grids, and the
internal Figure, covering PNG, JPEG/JPG, WebP, SVG, and PDF alongside
interactive HTML, with to_png/to_svg/to_html kept as compatibility
conveniences.
Every image format exports browser-free by default, preserving the
architectural fast path:
- JPEG: new pure-numpy baseline encoder (_jpeg.py) — 4:4:4 sampling for
line-graphics fidelity, Annex-K tables with the libjpeg quality curve,
fully vectorized (~63 ms for 1800x1000).
- WebP: new bit-exact lossless VP8L encoder (_webp.py) with alpha,
histogram-built length-limited prefix codes.
- PDF: new vector backend (_pdf.py) converting xy's own SVG subset —
vector text via embedded Helvetica AFM metrics, axial-shading gradients
(alpha stops via luminosity soft masks), density/heatmap layers embedded
as bounded raster XObjects per the documented hybrid-vector policy,
byte-accurate xref, deterministic output. The subset is strictly
whitelisted so SVG-generator drift fails loudly.
Engine.auto selects deterministically: native for every format, Chromium
only when custom_css needs a real CSS engine. Engine.chromium adds
browser-fidelity JPEG/WebP (CDP captureScreenshot) and PDF (printToPDF) to
the existing PNG path, and SVG stays native-only. A shared background
policy ("auto" / CSS color / "transparent") spans raster, vector, and
browser output; JPEG rejects transparent instead of silently flattening,
and an explicit color now paints the same single backdrop in the
rasterizer and the SVG/PDF exporters.
xy.write_images(figures=..., files=...) batches mixed formats with
per-file extension inference, atomic writes, and one reused Chromium
session for every browser-resolved file.
xy.export_config() declares formats/filename/width/height/scale/
background/quality on the chart itself (no I/O at build time): it fills
Python export defaults (explicit arguments win) and rides the spec to
govern the modebar download menu, which now offers PNG, JPEG, WebP, SVG,
and CSV with the same filename/scale/background/quality semantics as the
Python exporters — including in kernel-free standalone HTML and Reflex.
formats=[] hides the menu.
Tests cover the encoder round-trips (Pillow oracles), PDF structure and
vector-text preservation, the format/engine/background/quality matrices,
batch and facet parity, declarative defaults, and back-compat; docs gain
a format/engine capability table, the background policy, and a Plotly
migration table.
Merging this PR will not alter performance
Comparing 
Footnotes
|
Alek99
left a comment
Alek99
left a comment
There was a problem hiding this comment.
I found seven actionable contract gaps in the new export paths. Existing CI and the full local suite pass, but the cases called out inline are not currently covered.
| "pyarrow>=15", | ||
| # Decode oracle for the native JPEG/WebP encoders, never a runtime | ||
| # dependency: the image-export tests importorskip it. | ||
| "pillow>=10", |
There was a problem hiding this comment.
[P1] Regenerate uv.lock after adding Pillow. The committed lockfile does not contain Pillow or include it in the xy dev extra, so uv sync --frozen --extra dev omits the decoder oracle and the new JPEG/WebP suites silently skip via pytest.importorskip. Please run uv lock and commit the result.
There was a problem hiding this comment.
Fixed in 49db00b — ran uv lock; Pillow now resolves in the lock (pillow v12.3.0) and is wired into the xy dev extra, so uv sync --frozen --extra dev installs the oracle and the JPEG/WebP suites run for real.
| # Export-time canvas override (unified export API `background=`): one | ||
| # backdrop rect behind the figure patch. "transparent"/"none" mean "no | ||
| # backdrop", which is already SVG's default — nothing to paint. | ||
| canvas_paint = spec.get("canvas_background") |
There was a problem hiding this comment.
[P1] The explicit export backdrop is painted underneath the figure and plot theme backgrounds, so background= is invisible whenever theme(background=...) or plot_background is set. I reproduced background="#112233" producing the theme's red margins and green plot in both native and Chromium PNGs; transparent remains opaque too. Explicit export backgrounds need to replace/suppress those theme paints across the native SVG/PDF/raster and browser paths.
There was a problem hiding this comment.
Fixed in 49db00b. Explicit background= now replaces the theme paints instead of layering under them: a shared _svg.apply_export_background (used by raster, SVG, and thereby PDF) drops the theme figure patch and turns --chart-bg transparent so the override composites exactly once; the browser paths inject the same override as !important CSS that beats the root's inline theme background and the --chart-bg token the client reads. Verified with your repro (theme red margins + green plot): native PNG/WebP/SVG and real-Chromium PNG all show the override color edge-to-edge, and transparent yields alpha-0 in both regions. Covered by test_explicit_background_replaces_theme_paints and test_browser_background_css_overrides_theme_tokens.
| paths = files | ||
| if figs is None or paths is None: | ||
| raise ValueError("write_images needs both figures and files") | ||
| figs = [f.figure() if callable(getattr(f, "figure", None)) else f for f in figs] |
There was a problem hiding this comment.
[P2] Converting chart objects to Figure here drops Chart._export_defaults. A chart configured for 123×77 at scale 1 exports at 123×77 directly, but write_images([chart], ...) produces 600×400 from the chart's 300×200 dimensions and the batch scale default. Please resolve per-chart declarative defaults before discarding the wrapper.
There was a problem hiding this comment.
Fixed in 49db00b — write_images keeps the chart wrapper long enough to resolve its _export_defaults per file (width/height/scale/background/quality, including the engine-aware quality rule) before compiling to a Figure; batch-level arguments still override. Your 123×77@1 case is pinned in test_write_images_resolves_chart_export_config_defaults.
| from . import _webp | ||
|
|
||
| return _webp.encode(canvas) | ||
| doc = self.to_html(custom_css=custom_css) |
There was a problem hiding this comment.
[P2] The validated facet background is never added to the HTML passed to Chromium; it is used only for the transparent CDP flag below. A real Chrome export with background="#112233" still had a white corner. Please build the document with the background override, as the single-chart path does, including PDF.
There was a problem hiding this comment.
Fixed in 49db00b — the facet browser branch now injects the validated background into the captured document via the same _background_css helper as the single-chart path (PDF included), and the native facet SVG passes the override into each panel so panel theme paints can't bury the grid backdrop. Verified against real Chromium (corner pixel = override color); pinned by test_facet_browser_background_reaches_document.
| non-lossy formats; a config "transparent" background is dropped for | ||
| JPEG) — only *explicit* arguments produce hard errors downstream.""" | ||
| config = self.figure().export_options or {} | ||
| if quality is None and fmt == "jpeg": |
There was a problem hiding this comment.
[P2] Declarative quality is copied only for JPEG. export_config(quality=37) followed by to_image("webp", engine=Engine.chromium) passes the generic default 90 to Chromium. Chromium WebP should inherit the configured quality while native lossless WebP continues ignoring it.
There was a problem hiding this comment.
Fixed in 49db00b — _export_defaults now takes a lossy_webp flag derived from the resolved engine, so export_config(quality=37) reaches Chromium WebP while native WebP stays lossless and ignores it. Pinned by test_export_config_quality_reaches_chromium_webp.
| const show = Boolean(open); | ||
| // export_config(formats=[]) leaves nothing to show: the grip stays a | ||
| // pure drag handle rather than opening an empty menu. | ||
| const show = Boolean(open) && exportMenuItems.length > 0; |
There was a problem hiding this comment.
[P2] This makes exportMenuItems empty for export_config(formats=[]) (and PDF/HTML-only lists), but the existing ArrowDown/ArrowUp handler still indexes that array and calls .focus() on undefined, causing a runtime TypeError. Please guard the keyboard handler when there are no client-side export items.
There was a problem hiding this comment.
Fixed in 49db00b — the grip's ArrowDown/ArrowUp handler returns early when there are no client-side export items, matching the pointer path's guard. Verified via CDP with export_config(formats=[]): no page errors, menu stays closed.
| plan.append((fig, path, fmt, "html", None, None)) | ||
| continue | ||
| resolved = _resolve_image_engine(engine, fmt, custom_css) | ||
| file_quality = ( |
There was a problem hiding this comment.
[P2] The batch contract says quality is ignored for non-lossy members, but a mixed native JPEG+WebP batch with quality=50 fails here because native WebP is routed through _validated_quality. Apply quality to JPEG and Chromium WebP only; native lossless WebP should not abort an otherwise valid mixed batch.
There was a problem hiding this comment.
Fixed in 49db00b — batch quality now applies to JPEG and Chromium-WebP members only; native lossless WebP no longer aborts a mixed batch (the value range itself is still validated once up front, so quality=500 fails fast). Pinned by the extended test_write_images_quality_ignored_by_non_lossy_batch_members.
…routing - Regenerate uv.lock so the Pillow dev-extra oracle actually installs and the JPEG/WebP suites stop skipping under --frozen syncs. - An explicit export background= now REPLACES the theme paints instead of being buried under them: apply_export_background (shared by raster, SVG, and thereby PDF) drops the theme figure patch and turns the plot token transparent so the override composites exactly once; the browser paths inject the same override as !important CSS that beats the chart root's inline theme background and the --chart-bg token the client reads. Fully-transparent paints are now skipped as no-op fills. Verified against real Chromium for both single charts and facet grids (which previously never received the override in their captured document). - write_images keeps chart wrappers long enough to resolve their export_config defaults (width/height/scale/background/quality) per file before compiling to figures; batch-level arguments still win. - Declarative quality now reaches Chromium's lossy WebP, not just JPEG, while native WebP stays lossless; in batches, quality applies to JPEG and Chromium WebP only, so a native mixed JPEG+WebP batch no longer aborts (value range still validated up front). - Guard the modebar grip's ArrowDown/ArrowUp handler when export_config leaves no client-side menu items (formats=[] or PDF/HTML-only), which previously threw focusing undefined; verified error-free via CDP. - Regression tests for each case: theme-replacement pixels/SVG, browser CSS injection (single + facet), batch config defaults, chromium-webp quality, and the mixed-batch quality contract.
The move in this PR promoted docs/engineering/ to spec/ and CLAUDE.md declares that tree the source of truth. An audit of all 24 documents against the code found the tree had drifted at PR #96: five behavior PRs (#103, #105, #106, #108, #115) had landed with no spec update, and ~100 claims contradicted shipped code. Accuracy: correct claims that no longer match the implementation, including the Rust ABI version (v31 -> v36), the "zero-crate cdylib" claim (png is a real dependency), the pyramid ABI function names, the panic-shield status, the R2/VAO claim in the renderer doc, the WASM/Web-Worker architecture in dossier §3/§8 (superseded by §32's C-ABI cdylib), the count-only tier rule, the matplotlib shim overhead figures (the cited guardrail says +60% at 10k, not +9%), and the Plotly Scattergl 10M figure that appeared as two different numbers from the same run. Coverage: add spec/api/export.md, spec/api/interaction.md and spec/design/wire-protocol.md for three surfaces that had no coverage at all, plus the config.py threshold table, full JS and Rust module inventories, and the hexbin centers-only wire contract that three renderers must expand identically. Structure: group 15 top-level files into api/, design/, matplotlib/, benchmarks/ and process/, and rewrite spec/README.md as a real index (it previously listed only the two SVG assets, leaving 21 of 22 docs unreachable). All 150 consumer references repointed, including the CI --emit-md target and the compat-matrix generator. Guard: verify_sdist.py pinned only 5 top-level spec files, so the sdist could silently lose spec/design/ or spec/assets/ and still pass; it now asserts markdown under every subdirectory and both SVGs.
…roup the tree (#122) * Move engineering docs to root spec directory * Resync spec/ with the implementation and group the tree The move in this PR promoted docs/engineering/ to spec/ and CLAUDE.md declares that tree the source of truth. An audit of all 24 documents against the code found the tree had drifted at PR #96: five behavior PRs (#103, #105, #106, #108, #115) had landed with no spec update, and ~100 claims contradicted shipped code. Accuracy: correct claims that no longer match the implementation, including the Rust ABI version (v31 -> v36), the "zero-crate cdylib" claim (png is a real dependency), the pyramid ABI function names, the panic-shield status, the R2/VAO claim in the renderer doc, the WASM/Web-Worker architecture in dossier §3/§8 (superseded by §32's C-ABI cdylib), the count-only tier rule, the matplotlib shim overhead figures (the cited guardrail says +60% at 10k, not +9%), and the Plotly Scattergl 10M figure that appeared as two different numbers from the same run. Coverage: add spec/api/export.md, spec/api/interaction.md and spec/design/wire-protocol.md for three surfaces that had no coverage at all, plus the config.py threshold table, full JS and Rust module inventories, and the hexbin centers-only wire contract that three renderers must expand identically. Structure: group 15 top-level files into api/, design/, matplotlib/, benchmarks/ and process/, and rewrite spec/README.md as a real index (it previously listed only the two SVG assets, leaving 21 of 22 docs unreachable). All 150 consumer references repointed, including the CI --emit-md target and the compat-matrix generator. Guard: verify_sdist.py pinned only 5 top-level spec files, so the sdist could silently lose spec/design/ or spec/assets/ and still pass; it now asserts markdown under every subdirectory and both SVGs.
1 participant
Closes ENG-10447.
Brings XY's export surface to practical format parity with Plotly — PNG, JPEG/JPG, WebP, SVG, PDF, and interactive HTML — behind one unified API, while preserving the browser-free native fast path for every format.
API
to_png/to_svg/to_htmlare unchanged compatibility conveniences.Format & engine matrix
_jpeg.py, ~63 ms @ 1800×1000)quality1–100_webp.py)quality)_pdf.py) converting XY's own SVG subsetprintToPDFto_htmlwrite_image("chart.html")routes thereengine="auto"is deterministic: native for everything, Chromium only whencustom_cssneeds a real CSS engine. Missing-browser errors nameXY_BROWSERand the supported executables. Batch export shares one Chromium session across all browser-resolved files.Background & quality policy
background="auto" | <css color> | "transparent"behaves identically across raster/vector/browser output; JPEG rejectstransparentinstead of silently flattening. An explicit color now paints the same single backdrop in the rasterizer and SVG (plot-rect default harmonized).qualityapplies to JPEG and Chromium-WebP; native WebP is always lossless by policy.Modebar / declarative / Reflex
export_configrides the spec: the modebar download menu is now config-driven (client-safe subset png/jpeg/webp/svg/csv, ordered;formats=[]hides it) with the same filename/scale/background/quality semantics as Python — verified working in kernel-free standalone HTML via headless CDP (menu items, configured filename, correct extensions, toBlob-fallback naming). Reflex charts inherit the same spec-driven config.Facets
FacetGrid/FacetChartgainto_image/write_imageover the same matrix: native raster composes panel renders; SVG/PDF compose vector panels; Chromium renders the HTML grid.Verification
test_jpeg.py(23),test_webp.py(19, bit-exact PIL round-trips incl. alpha),test_pdf_export.py(xref-offset parser, vector-text assertions, image XObjects, determinism),test_image_export.py(31, full matrix + engine/background/quality/batch/config/facet + fake-CDP chromium plumbing).render_smoke_nonumpy.py,abi_smoke.py, pre-commit hooks, ruff check/format,ty(no new diagnostics),cargo testall green. No Rust changes; JS bundle rebuilt vianode js/build.mjs.Follow-ups deliberately out of scope: EPS (dropped by modern Plotly/Kaleido too), golden-image CI corpus for the new formats, benchmark harness rows for JPEG/WebP/PDF export.