feat(tooling): local style-variant experimentation script#6378
Merged
Conversation
scripts/style-experiment.py is a sandbox for A/B-testing plot style variants (canvas size, font sizes, palette, ...) on real spec/library combos. Mimics agentic/workflows/modules/regen/render.py exactly (uv run -P, MPLBACKEND=Agg, ANYPLOT_THEME, xvfb fallback) but writes all output under /tmp/anyplot-style-experiments/<ts>/ so nothing lands in the repo or GCS. Patches the implementation source in place and restores it via try/finally + atexit + signal handlers - required because some impls (highcharts) resolve Path(__file__).parents[3] for shared assets and break when copied to a temp dir. scripts/style-variants.yaml ships starter variants: baseline, smaller_canvas (4800x2700 -> 2400x1350), bigger_fonts (title 24->40pt, axis 20->32pt, tick 16->26pt, line 3->5), combo_smaller_canvas_modest_fonts, and palette_tableau. New variants are pure YAML - no script changes. Output is a static compare.html with a thumbnail-width toggle (160/320/640/1200 px) so the same render can be evaluated at mobile, grid, and detail display scales side-by-side. uv.lock is bumped 2.2.0 -> 2.3.0 to match pyproject.toml.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a local sandbox tool to render the same spec×library across multiple “style variants” (patch bundles) and themes, producing a side-by-side comparison report to help choose new visual defaults before updating production prompts.
Changes:
- Added
scripts/style-experiment.pyto apply source patches in place, render viauv run python -P, and generatemanifest.json+compare.htmlunder/tmp/.... - Added
scripts/style-variants.yamlwith starter style-variant patch sets (canvas sizing, font/line weights, palette swap). - Updated
uv.lockto align the lockfile version withpyproject.toml(2.3.0).
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
uv.lock |
Bumps the editable package version entry to 2.3.0. |
scripts/style-variants.yaml |
Defines YAML-driven style variants and per-library patch rules consumed by the experiment script. |
scripts/style-experiment.py |
New CLI tool to patch implementations, render variants/themes into /tmp, and generate a browsable comparison report. |
Comment on lines
+408
to
+416
| pre_dirty = subprocess.run( | ||
| ["git", "diff", "--name-only", "--", "plots/"], | ||
| capture_output=True, text=True, cwd=REPO_ROOT, | ||
| ).stdout.strip() | ||
| if pre_dirty: | ||
| print("[style-experiment] WARNING: plots/ has uncommitted changes before run:") | ||
| print(pre_dirty) | ||
| print("Patches will still be reverted via try/finally, but verify with `git diff` after.\n") | ||
|
|
Comment on lines
+148
to
+153
| atexit.register(_restore_all) | ||
| for _sig in (signal.SIGINT, signal.SIGTERM, signal.SIGHUP): | ||
| try: | ||
| signal.signal(_sig, _signal_handler) | ||
| except (ValueError, OSError): | ||
| pass |
Comment on lines
+337
to
+358
| html.append(f"<div class='group'><h2>{spec} — {lib}</h2>") | ||
| html.append("<table><thead><tr><th></th>") | ||
| for vn in variant_names: | ||
| html.append(f"<th>{vn}<small>{variant_desc[vn]}</small></th>") | ||
| html.append("</tr></thead><tbody>") | ||
| for theme in themes: | ||
| html.append(f"<tr><th>{theme}</th>") | ||
| for vn in variant_names: | ||
| rec = by_key.get((spec, lib, vn, theme)) | ||
| html.append(f"<td class='cell {theme}'>") | ||
| if rec is None: | ||
| html.append("<div class='err'>(no record)</div>") | ||
| elif rec.success and rec.png_path: | ||
| html.append(f"<a href='{rec.png_path}' target='_blank'>" | ||
| f"<img src='{rec.png_path}' loading='lazy'></a>") | ||
| html.append(f"<div class='label'>{rec.duration_sec}s" | ||
| + (f", {rec.patches_applied} patch(es)" if rec.patches_applied else "") | ||
| + "</div>") | ||
| else: | ||
| err = (rec.error or "unknown error")[:600] | ||
| html.append(f"<div class='err'>{err}</div>") | ||
| html.append("</td>") |
| import json | ||
| import os | ||
| import re | ||
| import shutil |
Comment on lines
+8
to
+9
| Outputs land under ``/tmp/anyplot-style-experiments/<timestamp>/`` by default | ||
| - never inside the repo, never on GCS. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
- Drop unused `shutil` import. - Make signal trapping cross-platform: build list via getattr so SIGHUP-absence on Windows doesn't raise AttributeError at import. - HTML-escape every interpolated text field in compare.html (spec, library, variant name/description, theme, error/log excerpts) via html.escape, and URL-quote png paths used in href/src. - Replace `git diff --name-only` pre/post checks with `git status --porcelain --untracked-files=all -- plots/` so artifacts written next to __file__ by misbehaving impls are also flagged. - Soften module docstring: the script directs cwd to /tmp and detects leakage via the git status check, but cannot prevent an impl from writing relative to __file__.
Comment on lines
+141
to
+144
| except Exception as exc: | ||
| print(f"[style-experiment] WARNING: failed to restore {path}: {exc}", | ||
| file=sys.stderr) | ||
| _pending_restores.pop(path, None) |
Comment on lines
+103
to
+105
| data = yaml.safe_load(path.read_text()) | ||
| catalog = data.get("variants") or {} | ||
| missing = [n for n in names if n not in catalog] |
3 tasks
MarkusNeusinger
added a commit
that referenced
this pull request
May 16, 2026
…6946) ## Summary - Rescues an orphan post-merge commit (`c8e6828`) from `claude/improve-plot-responsiveness-gHYM5`. PR #6378 was squash-merged on 2026-05-11; a Copilot follow-up fix was pushed to the (closed) branch ~1h later and never landed on main. - Verified missing on main: `isinstance(data, Mapping)` guard is absent, and `_pending_restores.pop` runs unconditionally instead of only on successful restore. ## Changes - **`scripts/style-experiment.py`** - `_restore_all`: skip the `_pending_restores.pop` when the disk write fails, so atexit / signal handlers / `patched_in_place` cleanup can retry. Previously a failed restore silently dropped the entry and could leave the repo patched. - `load_variants`: validate that the loaded YAML is a mapping and that the `variants` key (if present) is also a mapping. Empty/scalar YAML used to crash with a cryptic `AttributeError` on `data.get(...)`. ## Test plan - [x] `python3 -m py_compile scripts/style-experiment.py` passes - [ ] CI green - [ ] `claude/improve-plot-responsiveness-gHYM5` can be deleted after merge Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
scripts/style-experiment.py— a local sandbox for A/B-testing plot style variants (canvas size, fonts, palette, line weights, ...) on real spec/library combos before committing to a style change. Motivated by the observation that the current 4800×2700 canvas with title=24pt / axis=20pt / tick=16pt / line=3 looks crisp at detail view (1600 px) but shrinks to <8 px effective text at thumbnail / mobile display sizes.Rather than guess at the right new defaults, this script lets us render the same spec×library through several variants in one shot and compare side-by-side at multiple display widths.
How it works
agentic/workflows/modules/regen/render.pyexactly:uv run python -P <impl>,MPLBACKEND=Agg,ANYPLOT_THEME=<theme>, xvfb fallback for headless bokeh.try/finally+atexit+ SIGINT/SIGTERM/SIGHUP handlers. In-place is required because some impls (highcharts) resolvePath(__file__).parents[3]for shared assets and break if the file is copied to a temp dir./tmp/anyplot-style-experiments/<timestamp>/— never inside the repo, never on GCS.git diff --name-only plots/check guards against patches not being restored on hard kills.Starter variants (
scripts/style-variants.yaml)baselinesmaller_canvasbigger_fontscombo_smaller_canvas_modest_fontspalette_tableauNew variants are pure YAML — no script changes. Patches are
{find, replace}literal or{regex, replace}and can be keyed per-library or"*"for all.Usage
uv run python scripts/style-experiment.py \ --spec scatter-basic --spec dendrogram-basic \ --library matplotlib --library plotly \ --variant baseline --variant smaller_canvas --variant bigger_fonts \ --theme both --openThe generated
compare.htmlshows rows = (spec, library), columns = variants × {light, dark}, with a dropdown to toggle thumbnail width between 160 / 320 / 640 / 1200 px — the same render evaluated at mobile / grid / detail / large display scale.Verified
scatter-basic / matplotlib / [baseline, smaller_canvas] / light→ baseline produces 4766×2672, smaller_canvas produces 2375×1342 (exactly half), both PNGs render correctly,git statusclean onplots/after the run.uv.lockis bumped 2.2.0 → 2.3.0 to sync withpyproject.toml.Test plan
compare.htmland step through the thumbnail-width toggle to evaluate readability at mobile (160 px) and grid (320 px) scalesgit statusstays clean onplots/after every runprompts/default-style-guide.md+prompts/library/*.mdhttps://claude.ai/code/session_01LnqUDs4Fpo1d87qeHbvVx6
Generated by Claude Code