Skip to content

Harden GL context-loss recovery and gate dashboard reliability in CI #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Alek99 merged 1 commit into from
Jul 9, 2026
Merged

Harden GL context-loss recovery and gate dashboard reliability in CI #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,14 @@ jobs:
CHROME=$(node -e "console.log(require('playwright').chromium.executablePath())")
.venv/bin/python scripts/interaction_stress_smoke.py "$CHROME"

- name: Browser dashboard reliability smoke (Chromium)
run: |
CHROME=$(node -e "console.log(require('playwright').chromium.executablePath())")
.venv/bin/python benchmarks/bench_dashboard.py \
--chart-counts 10,20,50 --chromium "$CHROME" --json dashboard-smoke.json
.venv/bin/python scripts/verify_benchmark_report.py \
dashboard-smoke.json --kind dashboard-browser

- name: Benchmark smoke (1e5/1e6 — §12 harness runs every phase)
run: .venv/bin/python scripts/bench.py --sizes 1e5,1e6

Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,13 @@ in the README).
- `LICENSE` (Apache-2.0), `CHANGELOG.md`, `SECURITY.md`, root `CONTRIBUTING.md`.

### Changed
- **Rendering hardening:** context loss now quiesces draw/animation/re-bin work,
invalidates pre-loss replies, retains streamed canonical payloads, reports
recovery state, and rebuilds without throwing an unhandled event error. The
dependency-free browser smoke forces three pixel-identical recovery cycles
and verifies interaction afterward. CI now hard-gates a loss-free 10-chart
dashboard, pins interaction/visual budget ceilings in the verifier, and
fails timing regressions beyond 4x while retaining the 2x advisory band.
- **Native PNG export compression** dropped from zlib level 9 to level 6: a
1M-point line export goes from ~298 ms to ~64 ms (reference hardware) for
~2.65% larger output. Regression tests pin the level for both truecolor
Expand Down
7 changes: 7 additions & 0 deletions docs/design/benchmark-methodology.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ reported).
prep, navigation readiness, JS heap, redraw-submission p95, per-chart context
loss/restoration events, and the stable loss-free chart-count ceiling. Partial
dashboards remain successful measurement rows rather than losing their metrics.
CI hard-gates the 10-chart row as loss-free/nonblank and applies deliberately
loose catastrophic budgets to its render, scroll, and redraw timings.
8. `install_import`: lower-bound distribution size plus opt-in fresh-venv total
site-packages, transitive distribution count, install time, and cold import.
9. `public_workflows`: `benchmarks/bench_workflows.py` tracks ingestion shapes,
Expand All @@ -138,3 +140,8 @@ reported).
`dashboard_20` scenario.
4. Reference-hardware runbook (`benchmarks/README`): exact pins + one-command
repro; publish both tiers on the next README refresh.

Timing regression policy is two-level: movement beyond 2x is advisory on shared
runners, while movement beyond 4x is a hard failure. Interaction and visual
budgets are capped in the report verifier so a benchmark change cannot silently
make its own gate easier.
14 changes: 9 additions & 5 deletions docs/design/renderer-architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,15 @@ Ordered by how much each compounds as kinds multiply.
adopt when a second state-toggling pass lands (e.g. scissored panels or
an additive-blend mark).
- **R4 — No GL context-loss handling.** ✅ **Done.**
`_initContextLossRecovery` listens for loss (preventDefault, halt RAF) and
restore (drop dead handles, re-run `_initGl` from the retained
screen-bounded payload, re-fire the view request to re-sync live tiers).
Smoke probe forces `WEBGL_lose_context` loss/restore and asserts the
rebuilt frame hashes pixel-identical (`ctxloss` flag).
`_initContextLossRecovery` listens for loss, prevents default eviction
handling, quiesces draw/animation/re-bin work, and increments the request
sequence so pre-loss kernel/worker replies cannot mutate the rebuilt state.
Streaming appends still replace the retained canonical payload while the
context is down. Restore drops dead handles, re-runs `_initGl` from that
payload, and re-fires the view request to re-sync live tiers; a failed
restore remains explicitly failed instead of throwing from the event
handler. The dependency-free smoke forces three `WEBGL_lose_context`
cycles, checks pixel-identical frames after each, and zooms after recovery.
- **R5 — Shader source conventions are informal.** ✅ **Done.** `build.mjs`
lints every shader at build time: `#version 300 es` first line, every FS
declares `precision highp float;`, every VS references a `u_*map` uniform
Expand Down
87 changes: 64 additions & 23 deletions examples/demo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,14 @@
" doc = re.sub(\n",
" r'<meta http-equiv=\"Content-Security-Policy\".*?>',\n",
" '<meta http-equiv=\"Content-Security-Policy\" content=\"default-src \\'self\\' '\n",
" '\\'unsafe-inline\\' \\'unsafe-eval\\' https://cdn.tailwindcss.com data: blob:; img-src * data:;\">',\n",
" doc, count=1, flags=re.S)\n",
" \"'unsafe-inline' 'unsafe-eval' https://cdn.tailwindcss.com data: blob:; img-src * data:;\\\">\",\n",
" doc,\n",
" count=1,\n",
" flags=re.S,\n",
" )\n",
" return doc.replace(\"</head>\", '<script src=\"https://cdn.tailwindcss.com\"></script></head>', 1)\n",
"\n",
"\n",
"def styled(chart, custom_css=None, tailwind=False, height=420):\n",
" \"\"\"Render a chart's self-contained HTML (optionally +Tailwind) inside an iframe.\"\"\"\n",
" with tempfile.NamedTemporaryFile(\"w+\", suffix=\".html\", delete=True) as f:\n",
Expand All @@ -256,15 +260,21 @@
" 'srcdoc=\"' + _html.escape(doc, quote=True) + '\"></iframe>'\n",
" )\n",
"\n",
"\n",
"# shared data for the styling variations\n",
"sx = rng.normal(0, 1, 4000)\n",
"sy = sx * 0.5 + rng.normal(0, 0.6, 4000)\n",
"sd = sx**2 + sy**2 # continuous -> colormap\n",
"\n",
"\n",
"def base_scatter(title, cmap):\n",
" return fc.scatter_chart(\n",
" fc.scatter(x=sx, y=sy, color=sd, colormap=cmap, opacity=0.75, size=5),\n",
" fc.x_axis(label=\"x\"), fc.y_axis(label=\"y\"),\n",
" title=title, width=\"100%\", height=360,\n",
" fc.x_axis(label=\"x\"),\n",
" fc.y_axis(label=\"y\"),\n",
" title=title,\n",
" width=\"100%\",\n",
" height=360,\n",
" )"
]
},
Expand Down Expand Up @@ -311,18 +321,35 @@
"tt = np.linspace(0, 12, 240)\n",
"wave = np.sin(tt) + 0.12 * np.cumsum(rng.normal(0, 0.1, 240))\n",
"area = fc.area_chart(\n",
" fc.area(x=tt, y=wave, color=\"#3b82f6\", curve=\"smooth\",\n",
" fill=\"linear-gradient(currentColor, transparent)\", line_width=2),\n",
" fc.area(\n",
" x=tt,\n",
" y=wave,\n",
" color=\"#3b82f6\",\n",
" curve=\"smooth\",\n",
" fill=\"linear-gradient(currentColor, transparent)\",\n",
" line_width=2,\n",
" ),\n",
" fc.line(x=tt, y=wave + 2.4, color=\"#ec4899\", dash=\"dashed\", width=2, curve=\"smooth\"),\n",
" fc.x_axis(label=\"t\"), fc.y_axis(label=\"value\"),\n",
" title=\"gradient area + dashed line (mark props)\", width=\"100%\", height=340,\n",
" fc.x_axis(label=\"t\"),\n",
" fc.y_axis(label=\"value\"),\n",
" title=\"gradient area + dashed line (mark props)\",\n",
" width=\"100%\",\n",
" height=340,\n",
")\n",
"bars = fc.bar_chart(\n",
" fc.bar(x=[\"Q1\", \"Q2\", \"Q3\", \"Q4\", \"Q5\", \"Q6\"], y=[5, 9, 4, 7, 6, 8],\n",
" corner_radius=(10, 0), stroke=\"#1e293b\", stroke_width=1.25,\n",
" fill=\"linear-gradient(to top, #6366f1, #a5b4fc)\"),\n",
" fc.x_axis(label=\"quarter\"), fc.y_axis(label=\"value\"),\n",
" title=\"rounded, stroked, gradient bars\", width=\"100%\", height=340,\n",
" fc.bar(\n",
" x=[\"Q1\", \"Q2\", \"Q3\", \"Q4\", \"Q5\", \"Q6\"],\n",
" y=[5, 9, 4, 7, 6, 8],\n",
" corner_radius=(10, 0),\n",
" stroke=\"#1e293b\",\n",
" stroke_width=1.25,\n",
" fill=\"linear-gradient(to top, #6366f1, #a5b4fc)\",\n",
" ),\n",
" fc.x_axis(label=\"quarter\"),\n",
" fc.y_axis(label=\"value\"),\n",
" title=\"rounded, stroked, gradient bars\",\n",
" width=\"100%\",\n",
" height=340,\n",
")\n",
"_css = \".fastcharts{--chart-text:#334155;font-family:ui-sans-serif,system-ui}\"\n",
"display(styled(area, custom_css=_css, height=320))\n",
Expand All @@ -339,9 +366,11 @@
"# Variation 3 — Tailwind utilities on chrome slots (class_names) + root class_name.\n",
"tw = fc.scatter_chart(\n",
" fc.scatter(x=sx, y=sy, color=sd, colormap=\"viridis\", opacity=0.8, size=5),\n",
" fc.x_axis(label=\"x\"), fc.y_axis(label=\"y\"),\n",
" fc.x_axis(label=\"x\"),\n",
" fc.y_axis(label=\"y\"),\n",
" title=\"Tailwind-styled chrome\",\n",
" width=\"100%\", height=360,\n",
" width=\"100%\",\n",
" height=360,\n",
" class_name=\"bg-slate-900 rounded-2xl p-3 ring-1 ring-indigo-500/30\",\n",
" class_names={\n",
" \"title\": \"text-indigo-300 font-semibold tracking-wide\",\n",
Expand All @@ -353,9 +382,12 @@
")\n",
"# tailwind=True injects the Play CDN for this standalone demo (needs network);\n",
"# a Reflex/Tailwind host already has Tailwind, so class_names apply with no CDN.\n",
"styled(tw, tailwind=True,\n",
" custom_css=\".fastcharts{--chart-text:#cbd5e1;--chart-grid:rgba(148,163,184,.18);--chart-bg:transparent}\",\n",
" height=380)"
"styled(\n",
" tw,\n",
" tailwind=True,\n",
" custom_css=\".fastcharts{--chart-text:#cbd5e1;--chart-grid:rgba(148,163,184,.18);--chart-bg:transparent}\",\n",
" height=380,\n",
")"
]
},
{
Expand Down Expand Up @@ -395,11 +427,20 @@
"mobile = desktop * (0.40 + 0.20 * rng.random(n))\n",
"\n",
"analytics = fc.area_chart(\n",
" fc.area(x=days, y=desktop, name=\"Desktop\", color=\"#880808\",\n",
" line_width=1.5, line_opacity=1,\n",
" fill={\"gradient\": \"linear-gradient(to bottom, #6E260E, #FF0000)\", \"space\": \"plot\"}),\n",
" fc.x_axis(label=None), fc.y_axis(label=None),\n",
" width=\"100%\", height=320, padding=[18, 20, 46, 20],\n",
" fc.area(\n",
" x=days,\n",
" y=desktop,\n",
" name=\"Desktop\",\n",
" color=\"#880808\",\n",
" line_width=1.5,\n",
" line_opacity=1,\n",
" fill={\"gradient\": \"linear-gradient(to bottom, #6E260E, #FF0000)\", \"space\": \"plot\"},\n",
" ),\n",
" fc.x_axis(label=None),\n",
" fc.y_axis(label=None),\n",
" width=\"100%\",\n",
" height=320,\n",
" padding=[18, 20, 46, 20],\n",
")\n",
"analytics_css = \"\"\"\n",
".fastcharts{ --chart-bg:#ffffff; --chart-text:#94a3b8; --chart-grid:rgba(15,23,42,.045);\n",
Expand Down
51 changes: 47 additions & 4 deletions js/src/50_chartview.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,11 @@ class ChartView {
// spec + payload by design (§18/§27).
this._payload = buffer;
this._glLost = false;
this._contextLossCount = 0;
this._contextRestoreCount = 0;
this._contextRecoveryError = null;
this._initGl(buffer);
this.root.dataset.fcContextState = "ready";
this._initContextLossRecovery();
this._initInteraction();
this._buildModebar(this.root); // after theme (icon color) + canvas (cursor)
Expand Down Expand Up @@ -407,25 +411,64 @@ class ChartView {
_initContextLossRecovery() {
this._listen(this.canvas, "webglcontextlost", (e) => {
e.preventDefault();
if (this._destroyed || this._glLost) return;
this._glLost = true;
this._contextLossCount += 1;
this._contextRecoveryError = null;
this.root.dataset.fcContextState = "lost";
// Quiesce every source of deferred GPU work, not only the draw RAF.
// Incrementing seq makes pre-loss kernel/worker replies stale, so they
// cannot populate the newly restored context with an old view.
this.seq += 1;
if (this._raf) cancelAnimationFrame(this._raf);
this._raf = null;
if (this._wheelZoomRaf) cancelAnimationFrame(this._wheelZoomRaf);
this._wheelZoomRaf = null;
this._pendingWheelZoom = null;
this._cancelViewAnimation();
clearTimeout(this._viewTimer);
this._viewTimer = null;
clearTimeout(this._rebinTimer);
this._rebinTimer = null;
this._viewRequestBurstStart = null;
this._dispatchChartEvent("context_lost", {
loss_count: this._contextLossCount,
});
});
this._listen(this.canvas, "webglcontextrestored", () => {
if (this._destroyed) return;
this._glLost = false;
// A failed recovery replaced the canvas with the error message; a later
// restore firing on the detached canvas must not resurrect GL state the
// user can no longer see.
if (this._destroyed || this._contextRecoveryError) return;
// Old handles died with the context — drop them without delete calls.
this._lutCache.clear();
this.pickFbo = null;
this.pickTex = null;
try {
this._initGl(this._payload);
} catch (err) {
this._glLost = true;
this._contextRecoveryError = err;
this.root.dataset.fcContextState = "failed";
try { this._destroyGlResources(); } catch (_cleanupErr) {}
this.gl = null;
this._dispatchChartEvent("context_restore_failed", {
loss_count: this._contextLossCount,
message: err instanceof Error ? err.message : String(err),
});
this.root.textContent = "fastcharts: WebGL2 context could not be restored.";
throw err;
return;
}
this._glLost = false;
this._contextRestoreCount += 1;
this._contextRecoveryError = null;
this.root.dataset.fcContextState = "ready";
this._scheduleViewRequest(this.view, { delay: 0 });
this.draw();
this._dispatchChartEvent("context_restored", {
loss_count: this._contextLossCount,
restore_count: this._contextRestoreCount,
});
});
}

Expand Down Expand Up @@ -1308,7 +1351,7 @@ class ChartView {
}

draw() {
if (this._destroyed) return;
if (this._destroyed || this._glLost || !this.gl) return;
if (this._raf) return;
this._raf = requestAnimationFrame(() => {
this._raf = null;
Expand Down
28 changes: 17 additions & 11 deletions js/src/54_kernel.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

Object.assign(ChartView.prototype, {
_scheduleViewRequest(viewOverride = this.view, opts = {}) {
if (this._destroyed) return;
if (this._destroyed || this._glLost) return;
if (!this.comm) {
// Kernel-less (standalone HTML): density traces refine via the bundled
// re-bin worker instead of a kernel round-trip.
Expand Down Expand Up @@ -74,7 +74,7 @@ Object.assign(ChartView.prototype, {
// request path, then the retained §28 sample re-bins in the bundled worker —
// off the main thread — and applies like a density_update.
_scheduleSampleRebin(viewOverride = this.view, opts = {}) {
if (this._destroyed || this._sampleRebinDisabled) return;
if (this._destroyed || this._glLost || this._sampleRebinDisabled) return;
const targets = (this.gpuTraces || []).filter(
(g) => g.tier === "density" && g.sampleOverlay && g.sampleOverlay._cpu
);
Expand Down Expand Up @@ -144,7 +144,7 @@ Object.assign(ChartView.prototype, {
},

_onRebinResult(msg) {
if (this._destroyed || !msg || msg.type !== "grid" || msg.seq !== this.seq) return;
if (this._destroyed || this._glLost || !msg || msg.type !== "grid" || msg.seq !== this.seq) return;
const g = this.gpuTraces.find((t) => t.trace.id === msg.trace && t.tier === "density");
if (!g) return;
const grid = new Float32Array(msg.grid);
Expand Down Expand Up @@ -201,14 +201,6 @@ Object.assign(ChartView.prototype, {
this.spec = spec;
this.axes = this._normalizeAxes(spec);
this._payload = blob;
const texSeen = new Set();
for (const id of msg.affected || []) {
const i = this.gpuTraces.findIndex((g) => g.trace.id === id);
const ts = spec.traces.find((t) => t.id === id);
if (i < 0 || !ts) continue;
this._destroyTraceResources(this.gpuTraces[i], texSeen);
this.gpuTraces[i] = this._buildTrace(blob, ts);
}
this.view0 = {
x0: spec.x_axis.range[0], x1: spec.x_axis.range[1],
y0: spec.y_axis.range[0], y1: spec.y_axis.range[1],
Expand All @@ -219,6 +211,19 @@ Object.assign(ChartView.prototype, {
const w = this.view.x1 - this.view.x0;
this.view = { ...this.view, x1: this.view0.x1, x0: this.view0.x1 - w };
}
// Append payloads are canonical state, so retain them even while the
// context is lost. The restore path rebuilds every affected GPU object
// from this latest payload; attempting partial uploads to a dead context
// would only create handles that must immediately be discarded.
if (this._glLost || !this.gl) return;
const texSeen = new Set();
for (const id of msg.affected || []) {
const i = this.gpuTraces.findIndex((g) => g.trace.id === id);
const ts = spec.traces.find((t) => t.id === id);
if (i < 0 || !ts) continue;
this._destroyTraceResources(this.gpuTraces[i], texSeen);
this.gpuTraces[i] = this._buildTrace(blob, ts);
}
this._pickable = this.gpuTraces.some(
(g) => markOf(g.trace.kind).pointPick && (g.tier !== "density" || g.drill));
if (this._pickable && !this.pickFbo) this._initPickTarget();
Expand All @@ -229,6 +234,7 @@ Object.assign(ChartView.prototype, {
_onKernelMsg(msg, buffers) {
if (this._destroyed) return;
if (!msg) return;
if (this._glLost && msg.type !== "append" && msg.type !== "pick_result") return;
if (msg.type === "tier_update") {
if (msg.seq !== this.seq) return;
for (const upd of msg.traces) {
Expand Down
Loading
Loading