Skip to content

Move density normalization into the shader#192

Closed
Alek99 wants to merge 1 commit into
mainfrom
agent/density-normalization-shader
Closed

Move density normalization into the shader#192
Alek99 wants to merge 1 commit into
mainfrom
agent/density-normalization-shader

Conversation

@Alek99

@Alek99 Alek99 commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

  • upload wire-native log-u8 density attachments directly to R8 textures
  • animate exposure with a u_normScale shader uniform instead of decoding, requantizing, allocating, and re-uploading the full grid every frame
  • preserve the old visual order by requantizing each of four fetched texels before manual bilinear interpolation; settled scale=1 frames retain hardware LINEAR sampling
  • make the standalone re-bin worker return log-u8 directly
  • keep cached/home density windows GPU-only: texture + metadata, with zero retained CPU grid bytes
  • reuse the pinned home texture instead of allocating/re-uploading it

Context recovery rebuilds home from the canonical payload, then the existing view-request path refines the current window.

Verification

  • complete suite with only the confirmed base marker deselected: 2212 passed, 66 skipped, 1 deselected in 75.77s
  • the deselected minified-bundle marker fails identically on untouched main and is corrected in draft PR Avoid full payload reships for restyles and unchanged columns #189
  • focused real-Chromium shader, worker, cache-eviction, drilldown-alignment, and context-loss probes: 16 passed
  • shader vs independently CPU-requantized reference: 0 pixel mismatches
  • manual four-texel sampling vs hardware LINEAR at scale 1: 0 pixel mismatches, max delta 0
  • 61-frame exposure lifecycle: 0 texImage2D calls
  • real context loss/fresh-canvas rebuild: pixel-identical
  • worker re-bin: exactly one upload, no retained encoded/f32 cache
  • ruff check .: passed
  • ruff format --check .: 343 files already formatted
  • tsc -p js/tsconfig.json --noEmit: passed
  • node js/build.mjs --check: shipped bundles fresh
  • git diff --check: passed

Local real-Chromium benchmark

500,000 source points, 824×488 density grid (402,112 cells), 120 exposure frames under SwiftShader:

Measurement Result
normalization texture uploads 0
retained CPU cache bytes 0 B
former f32 cache footprint 2,394,880 B
shader draw median / p95 / max 0.10 / 0.30 / 0.30 ms
removed CPU requantization loop 0.397 ms/frame
worker result bytes 402,112 B
animation completed yes

Closes #166

@codspeed-hq

codspeed-hq Bot commented Jul 22, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 97 untouched benchmarks
⏩ 1 skipped benchmark1


Comparing agent/density-normalization-shader (cba3b49) with main (2a7d898)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

@greptile-apps

greptile-apps Bot commented Jul 22, 2026

Copy link
Copy Markdown

Greptile Summary

This PR moves density normalization into the fragment shader and removes retained CPU density grids. The main changes are:

  • Upload log-u8 density data directly to R8 textures.
  • Animate exposure with a shader uniform instead of repeated texture uploads.
  • Return compact log-u8 grids from the re-bin worker.
  • Keep cached density windows as textures and metadata only.
  • Reuse the pinned home texture and rebuild it from the canonical payload after context loss.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.

T-Rex T-Rex Logs

What T-Rex did

  • The quantitative probe was run against the committed head standalone bundle in real Chromium/SwiftShader and produced the complete JSON payload.
  • Before/after Playwright recordings were generated from repository-generated standalone HTML, bundled JavaScript, styles, and a 90,000-point density chart, and the visuals were verified to be identical, with 1,556 colors and 29,965 non-white screenshot pixels, while browser console and page-error logs were empty, establishing the zero-upload lifecycle change as validated by the browser probe.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
js/src/40_gl.ts Adds shader-side texel requantization and manual bilinear sampling during exposure animation.
js/src/45_lod.ts Adds compact encoding and normalization helpers while changing density caches to texture-only ownership.
js/src/46_worker.ts Encodes worker re-bin output as log-u8 before transferring it to the main thread.
js/src/50_chartview.ts Uploads encoded density bytes directly and supplies the normalization scale during drawing.
js/src/54_kernel.ts Consumes compact worker results and restores the home view by reusing its pinned texture.
tests/test_density_shader_normalization.py Adds browser coverage for shader parity, upload-free animation, compact caching, and context recovery.
tests/test_density_texture_eviction.py Covers texture eviction and verifies that density cache entries do not retain CPU grids.
benchmarks/bench_density_normalization.py Adds a Chromium benchmark for draw time, texture uploads, worker output size, and retained cache bytes.

Reviews (1): Last reviewed commit: "Move density normalization into the shad..." | Re-trigger Greptile

@Alek99 Alek99 closed this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Density normalization should be shader-side: per-frame CPU re-tonemap, u8->f32->u8 round trip, and cached CPU grids

1 participant