Skip to content

feat(highcharts): implement swarm-basic#5696

Merged
MarkusNeusinger merged 4 commits into
mainfrom
implementation/swarm-basic/highcharts
May 6, 2026
Merged

feat(highcharts): implement swarm-basic#5696
MarkusNeusinger merged 4 commits into
mainfrom
implementation/swarm-basic/highcharts

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot commented May 5, 2026

Implementation: swarm-basic - python/highcharts

Implements the python/highcharts version of swarm-basic.

File: plots/swarm-basic/implementations/python/highcharts.py

Parent Issue: #974


🤖 impl-generate workflow

MarkusNeusinger added a commit that referenced this pull request May 5, 2026
…5727)

## Summary
Knocks out 8 of the 10 Quick Wins surfaced by `/audit` on 2026-05-05 —
every change is small-effort, low-risk, and either fixes a verified CI
break or addresses a high-importance finding.

- **Critical (CI red)**: `yarn install` restored the missing
`react-force-graph-2d` install; `yarn type-check` now passes (was 7
errors). Lint failure (32 errors) is intentionally not fixed in this PR
— those need human triage of `react-hooks/set-state-in-effect`
violations.
- **High**: fire-and-forget `asyncio.create_task` in `api/analytics.py`
could be GC'd mid-flight; now held in module-level `_BACKGROUND_TASKS`
set with a discard callback.
- **Codemod**: added `UP007`/`UP045` to ruff lint, ran `--fix` → 64
`Optional[X]` → `X | None` modernizations across `api/`, `core/`,
`automation/`, `agentic/`, `alembic/versions/`, `tests/`. Both `ruff
check .` and `ruff format --check .` are green.
- **Docs/hygiene**: `CLAUDE.md` Serena tools now use canonical
`mcp__serena__*` prefix; `agentic-auditor.md` no longer references
nonexistent `agentic/scripts/`; `docs/development.md` Node 18+ → 20+;
deleted `temp/tac/` (foreign CLAUDE.md from unrelated repo, gitignored).
- **Audit artifacts**: persisted `agentic/audits/2026-05-05-all.md` +
refreshed `latest.md` (full Phase-3 synthesis from 15 specialist
auditors).

CodeQL alert #101 (clear-text storage of sensitive data, false-positive
on `plots/radar-basic/implementations/python/highcharts.py:126`) is
dismissed separately via `gh api PATCH` — not part of this code diff.

The remaining Quick Wins from the audit (Cloud SQL auth-network purge,
Secret Manager rotation policies, Stuck PR #5696 re-trigger) are
GCP/GitHub admin actions and don't belong in a code PR.

## Test plan
- [x] `uv run ruff check .` → All checks passed
- [x] `uv run ruff format --check .` → 134 files already formatted
- [x] `cd app && yarn type-check` → no errors (was 7)
- [ ] CI: Run Linting / Run Tests / Run Frontend Tests (waiting on PR
pipeline)
- [ ] Verify Plausible og:image events still fire after analytics.py
refactor (manual check on a deployed revision)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 6, 2026

AI Review - Attempt 1/3

Image Description

Light render (plot-light.png): Warm off-white cream background (#FAF8F1) — correct, not pure white. Bold dark title "swarm-basic · highcharts · anyplot.ai" is fully readable; subtitle "Employee Performance Scores by Department" visible in soft dark ink. Y-axis label "Performance Score" and x-axis label "Department" are clear. Tick labels (34–106 range) are well-spaced and legible. Four beeswarm columns: Engineering in #009E73 (green), Marketing in #D55E00 (orange), Sales in #0072B2 (blue), Operations in #CC79A7 (pink). Black diamond mean markers overlay each group. Category labels (Engineering, Marketing, Sales, Operations) are clearly readable at the bottom. All text is readable against the light background — no light-on-light issues.

Dark render (plot-dark.png): Warm near-black background (#1A1A17) — correct, not pure black. Title and all chrome text render in light off-white (#F0EFE8) — fully readable against the dark surface. No dark-on-dark failures anywhere. Data point colors are IDENTICAL to the light render: #009E73, #D55E00, #0072B2, #CC79A7 — only chrome flips. Mean markers correctly switch to white diamonds (NEUTRAL #E8E8E0 on dark theme). Subtle grid lines are visible in both themes. All axis labels, tick labels, and category labels are clearly legible on the dark background.

Score: 86/100

Category Score Max
Visual Quality 28 30
Design Excellence 12 20
Spec Compliance 15 15
Data Quality 15 15
Code Quality 10 10
Library Mastery 6 10
Total 86 100

Visual Quality (28/30)

  • VQ-01: Text Legibility (8/8) — All font sizes explicitly set (72px title, 48px axis labels, 36px ticks). Fully readable in both themes.
  • VQ-02: No Overlap (6/6) — No text overlap. All category labels, tick labels, and titles are well-spaced.
  • VQ-03: Element Visibility (4/6) — Markers (radius 14) are visible; white outline helps separate them. However, no alpha/opacity on markers means stacked points in denser columns (Operations: 40 pts) lose individual identity. Alpha ~0.8 would improve density legibility.
  • VQ-04: Color Accessibility (2/2) — Okabe-Ito palette is CVD-safe. Categories distinguishable by both hue and luminance.
  • VQ-05: Layout & Canvas (4/4) — Good canvas utilization at 4800×2700. Plot fills the canvas with appropriate margins.
  • VQ-06: Axis Labels & Title (2/2) — "Performance Score" and "Department" are descriptive.
  • VQ-07: Palette Compliance (2/2) — First series #009E73 ✓, Okabe-Ito positions 1–4 in canonical order ✓, adaptive neutral for mean markers ✓, correct theme backgrounds ✓.

Design Excellence (12/20)

  • DE-01: Aesthetic Sophistication (5/8) — Above configured defaults: custom Okabe-Ito colors, custom beeswarm algorithm, mean diamond markers, full theme-adaptive chrome. Falls short of "strong design" — no intentional typographic hierarchy beyond size, no creative focal-point element or visual emphasis that elevates it to publication-ready.
  • DE-02: Visual Refinement (4/6) — Subtle GRID rgba, tickWidth:0 removes tick marks, custom background, clean axis styling, explicit marginBottom. Highcharts scatter is already fairly clean by default, so explicit refinement is present but not deep.
  • DE-03: Data Storytelling (3/6) — Mean markers add analytical value. Bimodal Engineering/Sales distributions vs. uniform Operations is an interesting story in the data. However, no visual emphasis (color contrast variation, size scaling, or focal element) actively guides the viewer to notice this insight. Raw distributions are shown but not interpreted.

Spec Compliance (15/15)

  • SC-01: Plot Type (5/5) — Correct beeswarm/swarm plot with custom inline algorithm spreading points horizontally to reduce overlap.
  • SC-02: Required Features (4/4) — All spec features present: individual points with horizontal spread, categorical comparison, color-coded groups, mean markers for central tendency, consistent point sizes.
  • SC-03: Data Mapping (3/3) — Categories on x-axis, values on y-axis. All 4 groups with 30–45 observations each (130 total), range 40–100 fully shown.
  • SC-04: Title & Legend (3/3) — Title format swarm-basic · highcharts · anyplot.ai correct. Legend enabled with all 5 series labeled.

Data Quality (15/15)

  • DQ-01: Feature Coverage (6/6) — Shows all swarm aspects: bimodal distributions (Engineering/Sales), varying spread (Marketing wide, Operations tight), outliers, density variation across groups.
  • DQ-02: Realistic Context (5/5) — Employee performance scores by department — realistic, neutral business scenario with plausible department names and distributions.
  • DQ-03: Appropriate Scale (4/4) — Scores in 40–100 range with np.clip ensuring no impossible values. Bimodal structures are plausible for performance data.

Code Quality (10/10)

  • CQ-01: KISS Structure (3/3) — Clean linear structure: imports → theme tokens → data → beeswarm algorithm → chart setup → export.
  • CQ-02: Reproducibility (2/2) — np.random.seed(42) set.
  • CQ-03: Clean Imports (2/2) — All imports used.
  • CQ-04: Code Elegance (2/2) — Inline beeswarm algorithm is appropriately complex. Uses strict=True in zip(). No fake UI or over-engineering.
  • CQ-05: Output & API (1/1) — Saves plot-{THEME}.png and plot-{THEME}.html correctly.

Library Mastery (6/10)

  • LM-01: Idiomatic Usage (4/5) — Correct Highcharts Python API: Chart(container='container'), HighchartsOptions, ScatterSeries, chart.add_series(), to_js_literal(), inline JS embedding per library guide pattern.
  • LM-02: Distinctive Features (2/5) — Highcharts used primarily as a scatter renderer. Beeswarm algorithm is pure Python, not a Highcharts feature. HTML output and pointFormat tooltip are standard Highcharts usage. No use of Highcharts-specific capabilities (plot bands/lines for group boundaries, Highcharts annotations API, custom series types) that would clearly differentiate this from a generic scatter implementation.

Score Caps Applied

  • None

Strengths

  • Correct Okabe-Ito palette (positions 1–4) in canonical order with adaptive neutral for mean markers
  • Custom inline beeswarm algorithm correctly spreads points to reveal distribution shape per category
  • All text sizes explicitly set — fully readable in both light and dark themes with no dark-on-dark failures
  • Full theme-adaptive chrome: all tokens (PAGE_BG, ELEVATED_BG, INK, INK_SOFT, GRID, NEUTRAL) flip correctly
  • Mean diamond markers add analytical value by surfacing central tendency alongside individual points
  • Realistic, neutral business scenario showing bimodal distributions that demonstrate swarm plot strengths
  • Clean KISS code; seed(42) for reproducibility; HTML + PNG artifacts for both themes

Weaknesses

  • DE-01 (5/8): No distinctive visual design element — lacks intentional focal-point emphasis, creative hierarchy, or a visual "hook" beyond correct color and layout
  • DE-02 (4/6): Visual refinement is good but incomplete — plot border/frame not explicitly addressed; margins could be more generous
  • DE-03 (3/6): Interesting distributional story in the data (bimodal Engineering/Sales) goes unemphasized — no visual hierarchy guides the viewer to the key insight
  • LM-02 (2/5): Highcharts used generically as scatter renderer; beeswarm logic is pure Python; no Highcharts-specific capabilities (plot bands, annotations API, custom series config) leveraged
  • VQ-03 (4/6): No alpha/opacity on overlapping markers; denser swarm columns appear as solid blocks

Issues Found

  1. DE-03 LOW: The data contains an interesting story (Engineering and Sales have bimodal distributions suggesting two performance tiers; Operations is tightly clustered) but nothing in the visualization emphasizes this
    • Fix: Add color intensity or size variation to highlight outliers, OR add Highcharts plot bands behind each category group to visually separate them, OR use Highcharts annotations API to label the mean values inline
  2. LM-02 LOW: Highcharts unique features underutilized — the chart could be any scatter library
    • Fix: Add Highcharts plot bands (xAxis plotBands) to visually separate category groups, use Highcharts point-level click events, or use the annotations module to label mean values directly on the chart
  3. VQ-03 MINOR: Overlapping points in dense swarm columns lose individual identity
    • Fix: Add opacity: 0.85 to scatter series markers to make stacked points distinguishable

AI Feedback for Next Attempt

The implementation is technically solid (perfect spec compliance, data quality, and code quality) but needs Design Excellence improvements to reach 90+. Focus on three changes: (1) Add Highcharts plotBands behind each category column (shaded alternating backgrounds) to visually separate the groups and show Highcharts-specific capability — this raises both DE-02 and LM-02. (2) Emphasize the most interesting pattern in the data using a Highcharts annotation or by making Engineering the visual focal point (e.g., larger marker or a plot band annotation labeling "Bimodal Distribution"). (3) Add opacity: 0.85 to all swarm markers to improve stacked-point density readability. These three targeted changes should push DE from 12/20 to 16+/20 and LM-02 from 2 to 3+.

Verdict: REJECTED

@github-actions github-actions Bot added quality:86 Quality score 86/100 ai-rejected Quality not OK, triggers update labels May 6, 2026
@github-actions github-actions Bot added ai-attempt-1 First repair attempt and removed ai-rejected Quality not OK, triggers update labels May 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor Author

github-actions Bot commented May 6, 2026

🔧 Repair Attempt 1/4

Applied fixes based on AI review feedback.

Status: Repair completed, re-triggering review...


🤖 impl-repair

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 6, 2026

AI Review - Attempt 2/3

Image Description

Light render (plot-light.png): The plot renders on a warm off-white #FAF8F1 background with four subtle per-category column bands (green tint for Engineering, orange for Marketing, blue for Sales, pink for Operations) providing clear visual structure. The title "swarm-basic · highcharts · anyplot.ai" appears in bold dark text at 72px — clearly readable. The subtitle "Employee Performance Scores by Department — Engineering & Sales show bimodal distributions" is in a softer grey at 40px. Four series of scattered points use Okabe-Ito colors 1–4 in order (brand green, vermillion, blue, reddish-purple) with semi-transparent (0.80 alpha) circular markers and white stroke borders. Diamond mean markers appear in an adaptive neutral per category. "bimodal ↕" annotations in the category color label Engineering and Sales columns at the top. "High Performer" and "Developing" zone labels appear on the y-axis left side. A dashed overall mean reference line sits at ~74 with a right-side label. All text is clearly readable against the light background — no light-on-light issues detected.

Dark render (plot-dark.png): The background is #1A1A17 warm near-black. The title renders in #F0EFE8 (INK dark token) — bright and clearly readable. Subtitle and tick labels use #B8B7B0 (INK_SOFT dark) — all readable. Legend background is #242420 (ELEVATED_BG dark) with #B8B7B0 item text. Grid lines use rgba(240,239,232,0.10) — subtle. The "bimodal ↕" annotations remain in their respective Okabe-Ito category colors (#009E73, #0072B2), which read well on the dark surface. Zone band labels "High Performer" and "Developing" use #B8B7B0 — visible. Data point colors are identical to the light render — all Okabe-Ito positions 1–4 match exactly. Mean diamond markers in #E8E8E0 (adaptive neutral dark) are visible. No dark-on-dark failures detected.

Both paragraphs are required. A review that only describes one render is invalid.

Score: 88/100

Category Score Max
Visual Quality 28 30
Design Excellence 15 20
Spec Compliance 15 15
Data Quality 15 15
Code Quality 9 10
Library Mastery 7 10
Total 88 100

Visual Quality (28/30)

  • VQ-01: Text Legibility (8/8) — Title 72px, subtitle 40px, axis labels 48px, tick labels 36px — all explicitly set, fully readable in both themes
  • VQ-02: No Overlap (4/6) — Beeswarm jitter reduces overlap but some point crowding visible in Operations and Marketing columns
  • VQ-03: Element Visibility (6/6) — Markers radius 14 with 0.80 alpha and white stroke, well-adapted to data density (~30-40 per group)
  • VQ-04: Color Accessibility (2/2) — Okabe-Ito palette is CVD-safe; semi-transparent markers still clearly distinguishable
  • VQ-05: Layout & Canvas (4/4) — 4800×2700 with custom margins, data fills canvas well, balanced whitespace
  • VQ-06: Axis Labels & Title (2/2) — "Department" and "Performance Score" — descriptive; units not needed for a 0-100 scale
  • VQ-07: Palette Compliance (2/2) — First series #009E73 ✓, Okabe-Ito positions 1-4 in order ✓, bg #FAF8F1/#1A1A17 ✓, all chrome tokens theme-adaptive ✓

Design Excellence (15/20)

  • DE-01: Aesthetic Sophistication (6/8) — Clearly above defaults: per-category background zones, bimodal annotations, zone labels, mean reference line create an intentional information hierarchy
  • DE-02: Visual Refinement (4/6) — Subtle grid, background bands, good margins; no spine removal (limited in Highcharts scatter) prevents full minimalist refinement
  • DE-03: Data Storytelling (5/6) — Strong: bimodal annotations signal distribution shape immediately, zone labels contextualize scores, mean line enables cross-group comparison

Spec Compliance (15/15)

  • SC-01: Plot Type (5/5) — Correct beeswarm/swarm plot with custom jitter algorithm spreading points horizontally
  • SC-02: Required Features (4/4) — Points spread to avoid overlap ✓, consistent point sizes ✓, mean markers ✓, color distinguishes categories ✓
  • SC-03: Data Mapping (3/3) — Categories on x-axis, values on y-axis; all data visible within range
  • SC-04: Title & Legend (3/3) — Title format swarm-basic · highcharts · anyplot.ai ✓; legend shows all 4 categories + Mean series ✓

Data Quality (15/15)

  • DQ-01: Feature Coverage (6/6) — Bimodal (Engineering, Sales), wide normal (Marketing), narrow normal (Operations); includes outliers and high-performer clusters
  • DQ-02: Realistic Context (5/5) — Employee performance scores by department — real-world plausible, neutral topic
  • DQ-03: Appropriate Scale (4/4) — Scores 40–100; means ~70-77 are realistic for department performance averages

Code Quality (9/10)

  • CQ-01: KISS Structure (3/3) — Imports → Data → Plot → Save; no functions or classes
  • CQ-02: Reproducibility (2/2) — np.random.seed(42) set
  • CQ-03: Clean Imports (2/2) — All imports used
  • CQ-04: Code Elegance (1/2) — Swarm offset algorithm (lines 47-76) uses nested for/else/break patterns that are somewhat verbose; functionally correct but harder to read at a glance
  • CQ-05: Output & API (1/1) — Saves plot-{THEME}.png and plot-{THEME}.html correctly

Library Mastery (7/10)

  • LM-01: Idiomatic Usage (4/5) — Uses Highcharts Series classes, Chart container, plotBands/plotLines API correctly; swarm x-positioning computed in Python (library used as rendering layer for this part)
  • LM-02: Distinctive Features (3/5) — Uses Highcharts-distinctive plotBands (zone backgrounds with labels) and plotLines (reference line with label) — native Highcharts features not easily replicated in static libraries

Score Caps Applied

  • None

Strengths

  • Perfect dual-theme adaptation — all chrome tokens (INK, INK_SOFT, ELEVATED_BG, GRID) correctly applied to both renders with no dark-on-dark failures
  • Excellent data storytelling: bimodal annotations, High Performer/Developing zone labels, and mean reference dashed line guide the viewer immediately to key insights
  • Per-category background zone bands (x-axis plotBands) with subtle Okabe-Ito-tinted fill provide clear visual structure
  • Correct Okabe-Ito palette — first series #009E73, multi-series follows canonical positions 1-4
  • Mean diamond markers (adaptive neutral position 8) per category provide the statistical summary suggested in the spec

Weaknesses

  • Beeswarm jitter algorithm produces visible point crowding in denser groups (Operations and Marketing)
  • No spine/frame removal — Highcharts scatter keeps the default chart border; removing top/right frame would improve minimalism
  • CQ-04: beeswarm offset computation uses nested for/else/break loops that are somewhat verbose
  • LM-01: swarm x-offsets computed entirely in Python; library used only as rendering layer for this logic

Issues Found

  1. VQ-02 PARTIAL: Some point crowding in denser columns
    • Fix: Increase the jitter step (e.g. 0.15 instead of 0.12) or the proximity threshold (2.53.0) to spread points further apart
  2. LM PARTIAL: Swarm positioning fully manual
    • Fix: Use jitter plot option if available in the Highcharts version, or document the manual approach as a deliberate choice

AI Feedback for Next Attempt

The implementation is strong and APPROVED. If a further repair were needed: widen the beeswarm proximity threshold slightly to reduce point crowding, and consider setting chart.options.chart.plotBorderWidth = 0 to remove the outer chart frame for a cleaner look.

Verdict: APPROVED

@github-actions github-actions Bot added quality:88 Quality score: 88/100 ai-approved Quality OK, ready for merge and removed quality:86 Quality score 86/100 labels May 6, 2026
@MarkusNeusinger MarkusNeusinger merged commit efcb168 into main May 6, 2026
6 checks passed
@MarkusNeusinger MarkusNeusinger deleted the implementation/swarm-basic/highcharts branch May 6, 2026 22:07
MarkusNeusinger added a commit that referenced this pull request May 6, 2026
…view (#5819)

## Summary

Two minimal resilience patches for the dominant transient failures
observed in the daily-regen audit (2026-05-06):

1. **`impl-review.yml` — \"Extract PR info\" step** (5 failures in 24h).
Wrap `gh pr view` in a 3-attempt retry with exponential backoff. When
the GitHub API blips, the entire review job aborts and the PR ends up
unlabeled — blocking the review → repair → merge cascade. Concretely
caused 4 of the 14 stuck PRs we recovered today (#5696, #5789, #5796,
#5801 had no labels at all because review never made it past step 1).

2. **`impl-generate.yml` — \"Create library metadata file\" step** (6
failures in 24h). Wrap the final `git push origin \"\$BRANCH\"` in a
3-attempt retry that does `fetch + rebase` between attempts. The
dominant failure mode is racing against Claude's earlier push to the
same branch — when the metadata commit hits a non-fast-forward, the
whole generation aborts and the PR never opens.

Both fixes are inline bash retries — no new action dependency. Each step
still hard-fails after 3 attempts so persistent issues still surface (we
don't want to mask real bugs).

## Out of scope (deferred)

- **`daily-regen.yml` \"pick\" job clean exit**: the existing logic
already writes `count=0` when no specs are eligible, and downstream is
gated on `if: needs.pick.outputs.count != '0'`. The 2 reported
\"cancellations\" in the audit period look unrelated to the pick step
itself (likely scheduler-level events).
- **Auto-retry on `ai-review-failed`**: would require a more invasive
`if: failure()` job-level step. Holding until we see whether the
Extract-PR-info retry alone reduces the rate enough.

## Context

This branch is the Stage 5 follow-up to today's recovery work, which
manually shepherded 13 of 14 stuck PRs through the review/repair/merge
pipeline. The full investigation + recovery plan lives at
\`/home/tirao/.claude/plans/bitte-schaue-dir-alle-peppy-bunny.md\`
(local).

## Test plan

- [x] YAML-validate both edited workflows (passes)
- [x] No-op for the happy path: first attempt of each retry preserves
existing behavior exactly
- [ ] CI green on this PR
- [ ] Post-merge: watch the next 24h of impl-generate / impl-review run
conclusions; expect failure rate to drop from ~10% to near 0% on these
two steps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-approved Quality OK, ready for merge ai-attempt-1 First repair attempt quality:88 Quality score: 88/100

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant