ENG-10167 Expand xy.pyplot shim: plot-family depth, scales, export chrome, and pinned Matplotlib 3.11 evidence - #37
Merged
Merged
Conversation
…rome, and pinned Matplotlib 3.11 evidence (#36) * Honor pyplot grid and legend options * feat(pyplot): pin Matplotlib 3.11 reference and audit compat contracts Replace the hard-coded plotting-surface assertion with a reviewed snapshot of the pinned upstream revision (bde111fb4e), kept in sync by scripts/sync_matplotlib_compat.py, which also generates the method-by-method compatibility matrix from compatibility.json and the executable corpus. A dedicated CI job checks the snapshot against a pinned Matplotlib checkout and runs the dual-engine corpus and optional-interop tests. Tighten the compatibility contract itself: formerly silent option discards now raise loudly (pie/quiver/barbs/contour/table/norm options and more), options the marks can honor are implemented instead of rejected (contour extent, plain Normalize, dashed stem/eventplot/ triplot, streamplot start_points/direction/widths, stepfilled hist), and streamplot always uses the shim's own bounded integrator so output no longer depends on whether Matplotlib is installed. Documented approximation levels, the accepted visual approximations, and a compatibility changelog; added contract tests covering rc chrome, color export, transforms, P3 options, and silent-drop regressions. * feat(pyplot): compare engines pixel-for-pixel and honor export chrome Turn the dual-engine evidence from crash-free execution into real comparison: all 54 corpus cases now render PNGs in both engines and compare normalized ink density and bounding-box geometry, semantic oracles extend to contours, triangulations, vector directions, masked scatter, RGBA images, and removable handles, and thresholds gain negative controls proving blank or wrong output fails. A mechanical scan of every public adapter rejects unexplained accepted-and-dropped options (explicit `compat-noop:` rationales required), and the reference CI job moves to the released matplotlib==3.11.0 wheel. On the export side, implement the common savefig options instead of rejecting them: bbox_inches="tight", transparent=, facecolor=, and PNG/SVG metadata= via standards-compliant text chunks. Static PNG export now honors the axes background token, and multi-panel HTML/SVG/PNG composition applies figure backgrounds and styled suptitles (size/weight/family/color/position) instead of dropping the kwargs. * feat(pyplot): stats-plot depth, nonlinear scales, and post-audit fixes Boxplot notches/bootstrap/user statistics, violin KDE bandwidths/ quantiles/sides, hexbin C aggregation, symlog/logit/asinh scales, secondary axes, and affine data transforms — with the divergences an adversarial cross-engine audit found in both this tranche and the previous one repaired rather than shipped silently: - scatter drops rows masked in x/y/s; fill_between(interpolate=True) draws single-point where-regions; imsave colormaps original values; set_cmap validates names and feeds imshow/scatter defaults; usermedians no longer shift notch CIs; boxplot sym works (empty string suppresses fliers); hexbin mincnt and C aggregation share one bin membership; violinplot survives constant data. - Loud rejections replace silent discards: bxp component linestyles, secondary-axis set_ticks extras, fraction-space transforms on data artists, html-format metadata; singular transforms fail at set_transform time. - Export: SVG/HTML honor savefig(facecolor=); single-chart SVG keeps the suptitle; suptitle y maps as a figure fraction; non-Latin-1 PNG metadata keys raise ValueError. - Scales: logit masks values at/outside (0,1); auto ticks refresh as data arrives and reset when the scale returns to linear; explicit ticks keep data-unit labels. - Docs state the real boundaries (rectangular hexbin geometry, HTML-only secondary axes, xy-owned tick locators, snapshot pin no longer CI-compared); the corrupted todo section is rewritten; the tight-bbox and negative-control tests can now actually fail. Excludes examples/pdsh notebooks (in-flight, currently failing ruff). * feat(pyplot): close the top PDSH benchmark gaps The Python Data Science Handbook ch. 4 benchmark (import-swap over 14 notebooks) scored 121/171 runnable cells; the failures clustered on a handful of missing surfaces rather than rendering gaps. This closes them: xy-owned tick locators/formatters resolved against live data limits, style.context with the stock sheets reduced to the supported rcParams subset, RdGy/jet engine tables, gci/clim with colorbars that track their mappable, GridSpec slice spans via explicit figure rectangles, and the pandas-facing Axes surface (get_figure, get_lines, shared-axes groupers, Period coordinates). Scatter vmin/vmax previously crashed at render; they now flow into a real color_domain on the engine's color channel. grid(linestyle= 'solid') injected an invalid style value and broke every later export of the axes. subplot()/axes() silently dropped keyword arguments. Benchmark after: 154/171 (90%); 147/154 (95%) excluding the out-of-scope 3-D notebook; zero savefig errors on passing cells. * docs(examples): ship the PDSH notebooks as a measured compat example Add the Python Data Science Handbook matplotlib-chapter code cells (MIT; the CC-BY-NC-ND prose is omitted) under examples/pdsh with only the import swapped to xy.pyplot, plus their data files and a measured scorecard: 154/171 runnable cells (90%) after the gap-closing pass in the previous commit, 147/154 (95%) excluding the out-of-scope 3-D notebook, with matplotlib 3.11 at 171/171 on the identical code. The notebooks keep upstream's code style, so they are exempted from the repo lint rules that would amount to restyling third-party code.
FarhanAliRaza
changed the title
Merging this PR will degrade performance by 13.6%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | test_first_payload_contour_core_2d |
5.3 ms | 6.1 ms | -13.6% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing matplotlib-shim (eaaca02) with main (7228f99)
Footnotes
-
1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩
An image-level audit rendered every PDSH ch4 cell under both engines (per-cell seeded RNG, matplotlib 3.11 reference) and found that cells which *run* mostly did not *look* like matplotlib: 39 major divergences across 105 pairs. This lands the audit docs, the fix rounds, and the re-measured result: 34 match / 38 minor / 8 major over the 84 graded pairs (majors 33 -> 8 on the same notebooks). Legends render explicit label lists with line/marker samples and honor frameon; plot(x, y_2d) advances the color cycle; colorbars carry real count domains and nice ticks; every colormap stop table is resampled exactly from matplotlib 3.11 (plasma's padded tail merged discrete bands; RdBu was a coolwarm alias) in Python and the JS client; discrete N-band colormaps quantize marks and colorbars; contours dash negatives at matplotlib's level counts; insets keep their rect beside default axes in PNG; sharex/sharey accepts all/col/row with per-group domains and inner-label hiding via a new labels-only "off" strategy; annotate places text at xytext and draws a real arrow; tick labels keep locator precision; log axes label decades 10^k; a bounded TeX-subset converter turns $...$ into unicode across all shim text; scatter s= keeps matplotlib's absolute area semantics; errorbar color=/ecolor= resolve independently (regression caught by re-grading). The native rasterizer bakes greek/super-subscript/math glyphs, takes UTF-8 text, and rotates y-axis titles; the text wire changed, so ABI_VERSION bumps 32 -> 33 and the abi_smoke heatmap expectation moves to the corrected NaN-transparent/zero-opaque semantics. Explicitly out of scope, by direction: no runtime fallback onto real matplotlib, ever (seaborn-internal drawing paths fail loudly at import in a matplotlib-free environment and stay soft evidence only). Remaining boundaries are enumerated in the compat changelog.
CI's check_python_floor.py requires every annotated file to import `from __future__ import annotations`; the new color-pipeline regression tests were missing it.
…ual-engine Visual parity round on the shim and renderers: - Directional triangles and diagonal x get distinct glyphs in SVG, native raster, and WebGL instead of collapsing into up-triangle/plus. - clabel places bounded, separated labels along each contour level. - hexbin uses true offset-grid tessellation (nearest-center metric, complete zero-count lattice, six data-space triangles per cell) so bin membership agrees across browser, PNG, and SVG. - Horizontal bar/step/stepfilled histograms put counts on x with touching horizontal bins. - Legend borderpad/labelspacing/fancybox/framealpha/shadow/title reach all renderers; subplots_adjust now raises NotImplementedError for material values, removing the last known silent option discard. - Automatic ticks add the 2.5 nice step with shared fixed precision, loc='best' picks the least-occupied corner, and a four-sided frame with independently controllable spines ships to browser/PNG/SVG. Each PDSH notebook now interleaves a Matplotlib 3.11 reference cell before every xy cell so outputs compare side by side in one kernel run. sync_dual_engine_notebooks.py regenerates the mirrored cells and a test gates freshness. The notebooks are excluded from ruff format (the two plotting aliases differ in length, so near-limit lines wrap in only one copy — the formatter and the sync check cannot both pass) and F811 is ignored there since paired cells redefine names by design.
Notebook frontends treat rich-HTML payloads as fragments even when they contain a full <html> document, so the standalone export's global CSS (html,body resets, font-family) could restyle the surrounding notebook. notebook_iframe() wraps the document in a sandboxed srcdoc iframe sized from the figure, giving the repr a real document boundary while keeping the self-contained offline display path. Both Figure._repr_html_ surfaces (engine and pyplot shim) and pyplot.show() in IPython now route through it.
…ity gaps
Matplotlib sizes chrome in points while the engine consumes CSS pixels;
apply the 96-dpi 4/3 factor uniformly to fonts, line widths, marker
diameters (edge width now widens the diameter as mpl does), spine
widths, and tick geometry instead of passing point values through.
Also close the remaining axis()/layout behavior gaps: full axis() mode
set (auto/equal/scaled/image/square, bool/keyword forms) starting from
margin-padded autoscale limits, tight domains that honor axes.{x,y}margin,
an AutoLocator-style tick-count heuristic from panel size, frame tick
marks in the render client, and a bbox_inches="tight"-style notebook
repr for single-axes figures.
… symbols Map ".", ",", "+", "x", and "d" to new dedicated symbols (point, pixel, plus_line, x_line, thin_diamond) instead of approximating them with circle/cross/diamond, and implement those symbols across all three renderers (WebGL shader, Rust raster SDF, SVG export). Triangles now use Matplotlib's normalized path (apex to full-width base) rather than the equilateral IQ SDF, unfilled plus/x markers derive their line width from markeredgewidth, and point/pixel keep Matplotlib's sizing semantics (half-size circle; fixed one-pixel rect). HTML legends render the trace's actual marker shape as the swatch instead of a color block.
Replace the analytic IQ pentagon SDF with a polygon SDF built from Matplotlib's Path.unit_regular_polygon(5), keeping the raster core, the WebGL marker shader, and the legend SVG path in lockstep so the pentagon marker matches Matplotlib's vertices instead of an approximation.
Adopt Matplotlib's scatter defaults: edgecolors="face" (each point's edge resolves to its own face color, including per-point LUT/palette colors), marker size that includes the edge with the stroke centered on the path boundary, source-over edge compositing, default opacity 1.0, and collection alpha applied to face and edge independently. Suppress the implicit continuous-color "value" legend for unlabeled collections. Keep the raster core, WebGL shader, and SVG emitter in lockstep.
…lpha Add a stroke_perimeter style option to the area mark so fill_between outlines both its boundaries (the closed polygon) instead of only the value curve, drawn at the collection's line width and alpha to match Matplotlib's PolyCollection edge. Thread it through the public area API, the marks core, and the raster/SVG/WebGL renderers in lockstep.
Resolve marching-squares saddle ambiguity with the bilinear asymptotic decider so contours join like contourpy instead of a fixed table. Densify the contourf field bilinearly before band snapping so boundaries cross between grid points rather than stair-stepping cells, drop the band outlines, and carry the discrete levels/boundaries through to the colorbar. Convert contour linewidths from points to CSS px, scale the negative-level dash preset by width, and render dashes on the GPU via a path-walked arc-length pattern. Rebuild the colorbar as a banded bar with ticks, label, reserved layout room, and Matplotlib-style tick selection that keeps zero labeled.
Match Matplotlib's adjustable='box': for aspect='equal', shrink the axes rectangle through padding instead of expanding the data domain, so an explicit extent stays pinned to the plot edges rather than floating in blank space. Fold colorbar room into that box computation and the notebook-inline footprint, and widen the vertical colorbar gap and reserved space consistently across the live view, SVG, and raster paths.
Raise the non-nearest imshow interpolation surface from 128² to a bounded 512², so smoothed images are at least display resolution in the notebook image box instead of each sample covering ~3x3 pixels. Nearest interpolation still keeps its source cells.
Marker sizes, edge widths, tick geometry, font sizes, and offset-points annotations were converted with a hardcoded 4/3 (96 dpi) factor, while Matplotlib's default figure DPI is 100 — every default chart rendered slightly small and figure(dpi=...) was ignored. Route all conversions through Axes._point_scale() so they follow the owning figure's DPI. Also carried along from the same parity pass: - area marks gain a line_color style channel so hist(histtype= "stepfilled", edgecolor=...) draws its outline (plumbed through components/marks, raster, SVG, and the JS client) - vertical colorbar labels use writing-mode:vertical-rl instead of a rotate(-90deg) transform, which inflated notebook scroll width - default colorbar ticks target 8 divisions so small decimal domains get a usable scale in raster/SVG exports The static/ JS edits are rebuilt from js/src via node js/build.mjs.
Matplotlib spells axis("equal") as set_aspect("equal",
adjustable="datalim"): it keeps the ordinary axes rectangle and expands
the narrower data dimension around its center so one x unit and one y
unit span the same number of pixels. The shim previously forced every
equal-aspect request through adjustable='box', which shrank the panel
instead and left tick 0 floating inside the frame.
Track the adjustable mode per-axes and branch at build time: 'datalim'
expands a data limit (surfaced through aspect_domains so the renderer's
range padding can't move it), while 'box' keeps the existing
rectangle-shrinking path used by scaled/image/square and by imshow. The
inline-bbox crop in _mplfig only applies to the box path now.
set_aspect() also accepts the adjustable/anchor/share keywords instead
of silently dropping them: adjustable is honored, anchor/share are
documented compat-noops, and unknown keywords raise TypeError.
This was referenced Jul 15, 2026
FarhanAliRaza
pushed a commit
that referenced
this pull request
Jul 15, 2026
Two regressions from #37 kept CI red on main and every open PR: Hexbin (benchmarks/test_codspeed_kernels.py::test_first_payload_hexbin_core_2d): _emit_hexbin expanded every cell into six fan triangles across seven f32 columns (168 B/cell — 3.17 MB at gridsize=128, 2x the raw input; #45 had loosened the benchmark bound to accept it). Hexagons all share one geometry, so ship centers plus one color value per cell (12 B/cell, 227 KB) and expand locally in each renderer: instanced triangles in the WebGL client (_buildHexbinMark), one polygon per cell in SVG, vectorized triangles in the raster exporter. Headless-Chromium readback is pixel-identical to the old wire shape. The benchmark bound is restored to grid-bounded centers. Perf guardrail (tests/pyplot/test_perf_guardrail.py, 10k case): pyplot's fixed per-build cost had grown to ~2x the declarative baseline locally. Removed per-build revalidation and O(n) temporaries: - styles.compile_axis_style / compile_mark_style and _validate.style_mapping now memoize per exact input dict — the same few rc-derived chrome dicts were re-parsed through the native CSS grammar on every build. - The rc chrome snapshot (_load_rc_chrome) is cached per (RcParams.version, dpi); RcParams now versions its mutations. - plot() gap/path detection drops an astype copy + diff + compare for the native is_sorted kernel and skips the isfinite mask entirely for clean data via a sum probe (non-finite anywhere poisons the sum). Local ratio at 10k: 2.03x -> ~1.6x; 100k: 1.26x. Full suite, ruff, ty, render smoke pass; hexbin WebGL/SVG/PNG verified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.