[RFC] Single-source C++ patterns — in-browser WebAssembly compilation

[engmung]

Why this matters

Patternflow's long-term vision is simple: a pattern you preview in the browser is the same pattern that runs on the device. No translation, no drift, no surprises.

Today, patterns are authored in JavaScript, previewed in the web Pattern Lab, and then converted to C++ via AI prompts before being added to the ESP32-S3 firmware. This conversion step is the source of subtle inconsistencies — mod semantics, float precision, sin/cos timing — small things that compound into visible differences between the simulator and the real device.

The proposed direction is single-source C++: patterns are written once in C++, compiled to WebAssembly in the browser for live preview, and compiled with the standard toolchain (Arduino / PlatformIO / ESP-IDF) for the device. Same .cpp file, two compilers, identical behavior.

This RFC opens the discussion on how to compile C++ in the browser without a backend server, and what the pattern API should look like to make this practical.

---

Current architecture

JS pattern code
↓
Web Pattern Lab preview (new Function())
↓
"Copy C++ prompt" → AI conversion
↓
Manual paste into firmware

Pain points:

• AI translation introduces inconsistencies between web preview and firmware
• Two parallel APIs to maintain (JS-side and C++-side)
• Web heuristic cost analyzer (esp32CostAnalyzer.ts) operates on JS, not the code that actually ships to the device
• Contributors can't author firmware patterns directly from the web

---

Proposed architecture

C++ pattern code
↓
In-browser C++ → WASM compilation
↓
Web Pattern Lab preview (WASM module)
↓
Same .cpp file → ESP32-S3 firmware (no changes)

Key principles:

• One source of truth. The web preview and firmware run identical code.
• No backend. Compilation happens entirely in the browser. Static hosting only (Vercel).
• No real-time compilation required. A "Compile / Preview" button with a 1–2 second latency is acceptable — but actual responsiveness must be validated via POC.
• Stateless render-first. Patterns implement a small surface area (likely a render() function plus metadata), keeping LLM-generated code reliable and web↔device parity easy to guarantee.

---

Shared pattern API (draft)

The web and firmware would share a single header defining the pattern contract:

// pattern_api.h — shared between web and firmware

struct Knobs {
    float a, b, c, d;         // 0.0–1.0, accumulated by the system
    float da, db, dc, dd;     // per-frame delta (optional, for advanced use)
    bool tap_a, tap_b, tap_c, tap_d;     // short-press events
    bool hold_a, hold_b, hold_c, hold_d; // long-press state
};

struct Pixel { uint8_t r, g, b; };

struct KnobMeta {
    const char* label;
    float sensitivity;
    float default_v;
};

struct PatternMeta {
    const char* name;
    KnobMeta knob_a, knob_b, knob_c, knob_d;
};

// Pattern authors implement these two:
extern const PatternMeta PATTERN;
Pixel render(int x, int y, float t, const Knobs& k);

Plus a small math utility header (fastSin LUT, smoothstep, mix, vec2/vec3, etc.) shared by both targets to guarantee bit-identical behavior.

Open questions:

• Stateless render() only, or also setup() + update(dt, k) for stateful patterns (particles, trails)?
• How strict should the sandbox be? (No STL? No globals? No Arduino APIs?)

---

Display adapter

Patterns call a unified setPixel(x, y, r, g, b) API. The backend swaps:

• Web: writes into a JS-visible frame buffer → uploaded as a DataTexture to the three.js LED matrix preview
• Firmware: forwards to dma_display->drawPixelRGB888(...)

Pattern code stays identical; only the output backend differs.

---

Knob input

The firmware currently exposes knobDeltas[i]. The proposed shared API exposes absolute values as the primary channel, with delta and press events as auxiliary. The system (firmware core / web simulator) is responsible for:

• Accumulating encoder deltas into the absolute knobValues
• Applying per-knob sensitivity from PatternMeta
• Resetting to default_v on short-press

This way the pattern code stays clean (k.a), and the per-pattern semantics (sensitivity, default, label) live in metadata.

---

The core technical decision

How do we compile C++ to WASM inside the browser?

This is the single decision that determines whether the whole direction is feasible. Three options on the table:

A. wasm-clang family

Run Clang + LLD compiled to WASM directly in the browser.

• ✅ Real C++ compilation
• ✅ No server required (aligns with hosting constraints)
• ❌ Heavy compiler asset (~10–25MB compressed)
• ❌ sysroot / standard headers need to be packaged
• ❌ Toolchain version may lag
• ❓ Real-world compile time on Patternflow patterns is unmeasured

B. Wasmer's in-browser Clang

A more recent take on browser-based Clang execution.

• ✅ Active project, modern tooling
• ❌ Package size
• ❌ C-first examples; C++ pattern environment needs verification
• ❓ Integration effort needs a POC

C. Serverless compilation API

Send code to an edge function, compile server-side, return WASM.

• ✅ Easiest path technically; great caching potential (hash → cached WASM)
• ❌ Violates the no-server constraint (current priority is to avoid backend)
• Backup option only

D. (Worth considering) A narrower DSL / C++ subset compiler

If pattern code is small and bounded (one render() function), a custom mini-compiler targeting WASM might be lighter than full Clang.

• ✅ Tiny asset (<200KB), fast compile (<100ms)
• ✅ Could double as a static analyzer for ESP32 cost prediction
• ❌ Custom compiler maintenance burden
• ❌ Less expressive — may limit what contributors can write

---

POC scope — what we need to measure

Before locking in a direction, a minimal POC should answer:

• First compiler-load time (cold and cached)
• Compile time for trivial patterns
• Compile time for real patterns (pattern_origin.h, pattern_wave_saw.h-class, including sinLUT / distLUT)
• Compiled WASM bundle size
• Browser memory footprint
• Error message capture (stderr → Monaco line markers)
• Behavior parity: web WASM vs ESP32 output, pixel-by-pixel, for the same .cpp
• Safety: infinite loops / pathological patterns isolated via Web Worker with timeout

Target environment: desktop Chrome first, mobile Safari later.

---

What gets simplified if this works

• esp32CostAnalyzer.ts becomes a real analyzer (operates on actual shipping code, not JS heuristics)
• "Copy C++ prompt" can be retired
• compilePatternCode() (JS via new Function()) is replaced by a WASM runtime
• External contributors can write firmware-ready patterns directly from the browser
• LLM-generated patterns can be previewed and shipped without human translation

---

What this RFC is NOT proposing (yet)

• Migration to PlatformIO — orthogonal, can come later
• Removal of existing JS pattern infrastructure during transition — runs in parallel
• Specific UI changes to Pattern Lab beyond the editor/preview swap

---

How to contribute to this discussion

• Comment with experience if you've worked with wasm-clang, Wasmer, or similar in-browser toolchains
• Try a POC — even a "hello world C++ compiles in browser and renders a single pixel" is enormously valuable
• Challenge the API draft — especially around stateful patterns, knob semantics, or sandbox scope
• Suggest alternatives — option D above (custom DSL) is wide open

This is exploratory. No decision is locked. The goal is to gather signal before committing to a path.

---

cc: anyone interested in patterns, embedded graphics, WASM toolchains, or LLM-assisted creative coding.