Skip to content

Fix statistical gallery semantics and defaults [mpl compatibility] - #335

Merged
Alek99 merged 7 commits into
mainfrom
agent/gallery-statistics-round2
Jul 27, 2026
Merged

Fix statistical gallery semantics and defaults [mpl compatibility]#335
Alek99 merged 7 commits into
mainfrom
agent/gallery-statistics-round2

Conversation

@sselvakumaran

@sselvakumaran sselvakumaran commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

This groups the Matplotlib statistical-chart fixes that share one compatibility surface:

  • correct weighted/stacked/density histogram normalization, including unequal bins and cumulative directions;
  • match Matplotlib's default non-detrended Welch behavior for psd, csd, cohere, and specgram;
  • render pyplot boxplots with Matplotlib's unfilled component geometry and handle cardinalities;
  • render pyplot violins with Gaussian KDE and one joined, seam-free body per group.

The opinionated native xy.box() and xy.violin() APIs remain unchanged; these defaults are contained in the Matplotlib shim.

Why

The previous histogram path normalized each input series independently before stacking, so the top envelope was not a probability density. The previous box/violin fast paths reused native opinionated marks, which changed box fills, flier placement, returned artist counts, and violin distribution shapes relative to Matplotlib. The spectral kernel also subtracted each segment mean even though Matplotlib's omitted/default detrend is detrend_none.

Visual comparisons

These are review-only images from immutable evidence commit a856621; they are not part of this PR's diff.

Weighted stacked density with unequal bins

Matplotlib / XY before / XY after

The before plot's density scale reaches ~0.85 because each dataset was normalized independently. The after envelope matches Matplotlib's ~0.35 scale and shape.

Matplotlib gallery: box plot vs. violin plot

Matplotlib / XY before / XY after

Matplotlib gallery: violin customization

Matplotlib / XY before / XY after

Verification

  • exact adapted gallery scripts completed on the combined branch:
    • statistics/histogram_multihist.py: 6 figures
    • statistics/boxplot_vs_violin.py: 1 figure
    • statistics/violinplot.py: 1 figure
  • combined focused integration run: 13 passed, 5 deselected
  • histogram/spectral commit before grouping: 802 passed, 65 skipped; 135 Rust tests
  • box/violin commit before grouping: 801 passed, 65 skipped; 35 focused compatibility tests
  • Ruff check, Ruff format check, and git diff --check pass

No browser capture was used for this static-rendering group.

Summary by CodeRabbit

  • Bug Fixes
    • Improved hist rendering to match Matplotlib for stacked density=True normalization (unequal bins, cumulative modes), dash/edge styling, and consistent filled vs outline stroke handling in vertical and horizontal orientations.
    • Refined boxplot/violinplot adapter output to align Matplotlib-like component structure, geometry, tick/median behavior, and joined violin body styling (including updated placeholder opacity).
    • Updated Welch-based spectra (psd/csd/cohere/specgram) to default to non-detrended output and fail loudly for unsupported detrending.
  • Documentation
    • Refreshed compatibility and design docs for the corrected histogram and spectral semantics.
  • Tests
    • Added regression coverage for histogram rendering, box/violin defaults, and spectral/statistical numeric results.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@sselvakumaran, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 23 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a77175d6-1de4-477a-88e4-34a6a3347126

📥 Commits

Reviewing files that changed from the base of the PR and between 1c3528b and 25e8b39.

📒 Files selected for processing (6)
  • python/xy/pyplot/_artists.py
  • python/xy/pyplot/_plot_types.py
  • spec/matplotlib/compat.md
  • spec/matplotlib/shim-todo.md
  • tests/pyplot/test_box_violin_default_compat.py
  • tests/pyplot/test_statistics_numeric_regressions.py
📝 Walkthrough

Walkthrough

The change aligns pyplot boxplot, violinplot, histogram, and Welch-based spectral behavior with Matplotlib compatibility expectations, updates native spectral processing to non-detrended defaults, and adds rendering and numeric regression coverage.

Changes

Matplotlib compatibility corrections

Layer / File(s) Summary
Boxplot and violin rendering
python/xy/pyplot/_plot_types.py, tests/pyplot/test_box_violin_default_compat.py, tests/pyplot/test_axes_charts.py, spec/matplotlib/compat-changelog.md
Boxplot and violinplot now use expanded component rendering, computed widths, KDE-based violin output, updated styling, and revised trace expectations.
Histogram computation and styling
python/xy/pyplot/_axes.py, tests/pyplot/test_gallery_hist_errorbar_compat.py, tests/pyplot/test_gallery_statistics_semantics.py, tests/pyplot/test_statistics_numeric_regressions.py, spec/matplotlib/compat-changelog.md, spec/matplotlib/compat.md
Histogram stacking, density normalization, cumulative handling, unequal-bin geometry, baseline connectors, and edge styling are updated and covered by numeric and rendering regressions.
Non-detrended spectral defaults
src/kernels.rs, python/xy/_native.py, spec/design/rust-engine.md, tests/pyplot/test_statistics_numeric_regressions.py
Welch and spectrogram processing no longer subtracts segment means; default numeric outputs and rejection of explicit detrending are tested and documented.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PyplotSpectralMethod
  participant NativeWelchAPI
  participant RustSpectralKernel
  PyplotSpectralMethod->>NativeWelchAPI: request default Welch or spectrogram output
  NativeWelchAPI->>RustSpectralKernel: process windowed samples without mean subtraction
  RustSpectralKernel-->>NativeWelchAPI: return spectral arrays
  NativeWelchAPI-->>PyplotSpectralMethod: return frequencies and spectra
Loading

Possibly related PRs

  • reflex-dev/xy#280: Both changes modify Matplotlib-compatible histogram bin handling and edge/dash rendering.
  • reflex-dev/xy#310: Both changes touch native integration, although the primary functionality differs.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the PR’s Matplotlib compatibility fixes for statistical gallery semantics and default behavior.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/gallery-statistics-round2

Comment @coderabbitai help to get the list of available commands.

@codspeed-hq

codspeed-hq Bot commented Jul 27, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 103 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing agent/gallery-statistics-round2 (25e8b39) with main (c1233a3)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@sselvakumaran

Copy link
Copy Markdown
Contributor Author

Added one more exact-gallery fix to this grouped statistics PR: axes_demo.py's dense inset histogram.

Root cause: pyplot gave default filled histogram patches black edges. In the 400-bin inset, ~1.39 px strokes completely overpainted ~0.32 px blue bars in PNG; the SVG retained 400 rectangles but incorrectly stroked every one. Matplotlib leaves filled patch edges transparent unless an edge is explicitly requested or patch.force_edgecolor is enabled.

Matplotlib / XY before / XY after

Verification on the combined PR branch:

  • exact axes_demo.py PNG and SVG rerendered;
  • blue-like inset pixels: 0 before -> 3,882 after (Matplotlib: 3,752);
  • SVG stroked bars: 400/400 before -> 0/400 after;
  • 23 focused histogram tests passed, 1 skipped;
  • Ruff and SVG validation passed.

The image lives only on the review-evidence branch and is not part of this PR diff.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
python/xy/pyplot/_plot_types.py (1)

3294-3344: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Optional: bind mean_segment/mean_point unconditionally or compute at the emit site.

Both names are conditionally assigned in each orientation branch and read ~15 lines later under a repeated showmeans and "mean" in item guard. It is correct today, but it reads as possibly-unbound and will trip type checkers/linters if the guards ever drift.

♻️ Suggested shape
-                if showmeans and "mean" in item:
-                    mean = float(item["mean"])
-                    if meanline:
-                        mean_segment = (center - half, mean, center + half, mean)
-                    else:
-                        mean_point = (center, mean)
+            # ... later, at the single emit site:
+            # mean = float(item["mean"])
+            # mean_segment = ((center - half, mean, center + half, mean)
+            #                 if orientation == "vertical"
+            #                 else (mean, center - half, mean, center + half))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/xy/pyplot/_plot_types.py` around lines 3294 - 3344, Ensure
mean_segment and mean_point are initialized unconditionally before the
orientation-specific branches, or compute them directly at their emission site,
so both names are definitely bound when later used. Preserve the existing
showmeans, "mean" in item, and meanline behavior in both branches.
python/xy/pyplot/_axes.py (1)

2040-2092: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Dense stacked/density/cumulative transform is well-tested but hard to follow.

The numeric semantics here (stack → normalize top envelope → optionally re-derive cumulative tops) are exercised by test_stacked_weighted_density_matches_matplotlib_311 and match the expected Matplotlib values, so I have no correctness concern. Given the flagged high complexity, consider extracting this pure-NumPy transform (inputs: counts_array, stacked, density, edges, cumulative) into a small helper so hist() itself stays readable and the transform becomes independently unit-testable.

♻️ Illustrative extraction sketch
def _stack_and_accumulate(
    counts_array: np.ndarray, edges: np.ndarray, *, stacked: bool, density: bool, cumulative: bool | int
) -> np.ndarray:
    if stacked:
        tops = np.cumsum(counts_array, axis=0)
        if density:
            with np.errstate(divide="ignore", invalid="ignore"):
                tops = (tops / np.diff(edges)) / tops[-1].sum()
        counts_array = np.diff(np.vstack((np.zeros((1, counts_array.shape[1])), tops)), axis=0)
    if cumulative:
        ...  # existing cumulative branch, unchanged
    return counts_array
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/xy/pyplot/_axes.py` around lines 2040 - 2092, Extract the stacked,
density-normalization, and cumulative NumPy transformation from hist() into a
small pure helper, such as _stack_and_accumulate, accepting counts_array, edges,
stacked, density, and cumulative. Preserve the existing operation order and
numeric behavior, including weighted stacked normalization and reverse
cumulative handling, then have hist() call the helper and convert its result to
counts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/xy/pyplot/_axes.py`:
- Around line 2190-2199: Update the horizontal stepfilled histogram entry
construction around the bar properties so dashed linestyles do not render as
solid strokes. When the existing dash setting is present, either apply the same
dashed-border fallback used by the general bar path or omit the stroke
properties; preserve the current solid-edge behavior when no dash is configured.

---

Nitpick comments:
In `@python/xy/pyplot/_axes.py`:
- Around line 2040-2092: Extract the stacked, density-normalization, and
cumulative NumPy transformation from hist() into a small pure helper, such as
_stack_and_accumulate, accepting counts_array, edges, stacked, density, and
cumulative. Preserve the existing operation order and numeric behavior,
including weighted stacked normalization and reverse cumulative handling, then
have hist() call the helper and convert its result to counts.

In `@python/xy/pyplot/_plot_types.py`:
- Around line 3294-3344: Ensure mean_segment and mean_point are initialized
unconditionally before the orientation-specific branches, or compute them
directly at their emission site, so both names are definitely bound when later
used. Preserve the existing showmeans, "mean" in item, and meanline behavior in
both branches.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c9c55306-2e63-4486-95fb-5dbdd82f9d6a

📥 Commits

Reviewing files that changed from the base of the PR and between a2ac10e and 2c9658a.

📒 Files selected for processing (11)
  • python/xy/_native.py
  • python/xy/pyplot/_axes.py
  • python/xy/pyplot/_plot_types.py
  • spec/design/rust-engine.md
  • spec/matplotlib/compat-changelog.md
  • spec/matplotlib/compat.md
  • src/kernels.rs
  • tests/pyplot/test_axes_charts.py
  • tests/pyplot/test_box_violin_default_compat.py
  • tests/pyplot/test_gallery_hist_errorbar_compat.py
  • tests/pyplot/test_statistics_numeric_regressions.py

Comment thread python/xy/pyplot/_axes.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/pyplot/test_gallery_hist_errorbar_compat.py`:
- Around line 97-125: Update
test_hist_step_outline_connects_to_baseline_in_both_orientations to unpack both
entries from horizontal_ax._entries, matching the connector and stairs entries
produced for horizontal histtype="step". Select the segments connector for the
coordinate assertions and preserve the existing endpoint checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 378ca0b5-9807-4ef8-aeee-03c36ab43ae5

📥 Commits

Reviewing files that changed from the base of the PR and between dee0b6a and 1c3528b.

📒 Files selected for processing (3)
  • python/xy/pyplot/_axes.py
  • spec/matplotlib/compat.md
  • tests/pyplot/test_gallery_hist_errorbar_compat.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • spec/matplotlib/compat.md
  • python/xy/pyplot/_axes.py

Comment thread tests/pyplot/test_gallery_hist_errorbar_compat.py
@Alek99
Alek99 merged commit 4c26d97 into main Jul 27, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants