perf(tts_rvq_e2e): hierarchical CLAM 256×256 for vocab tensors + docs + F32x16 rms_norm

[AdaWorldAPI]

Summary

Follow-up to #176 (merged). Three commits:

SHA	What
30df7b1	Docs — three chunked guides in docs/: RVQ_ENCODER_REPLICATION.md (347 L, runnable pipeline for any BF16 safetensors model), RVQ_K_LADDER_TUNING.md (175 L, shape→k decision rule + hierarchical CLAM 256×256 design), RVQ_ALTERNATIVES.md (207 L, codec-family comparison incl. Qwen3-VL adaptation, Jina 5-lane / DeepNSM / bgz-tensor palette)
5047618	Hierarchical CLAM 256×256 — build_hclam_256x256 + reconstruct_hclam, dispatched in load_weights when n_rows > 8192. Remediation for the text_embedding cos=0.054 collapse documented in #176. Tree quantisation (not residual): L1 coarse 256 clusters, L2 256 fine centroids per cluster, one L2 leaf per row. At [151936, 2048]: ~257 MB vs 620 MB BF16 → 2.4:1 at cos ≈ 1.
aea0642	rms_norm F32x8 → F32x16 + mul_add FMA — upgrade from AVX2 256-bit to AVX-512 512-bit lane width. On target-cpu=x86-64-v4, (vx * inv_v).mul_add(vw, zero_v) compiles to one VFMADD231PS per 16-float block vs two ops per 8-float block. 8-wide + scalar tails preserved. Inference-side only; encoder hot path (l2_dist_sq) was already 4×-unrolled F32x16 FMA from #176.

Relation to merged PR #176

#176 shipped the AVX-512 F32x16 FMA encoder + AMX TDPBF16PS polyfill and established the first end-to-end RVQ baseline. The follow-up comment there (#176#issuecomment-4245767939) documented the one remaining failure: model.text_embedding.weight [151936, 2048] at cos=0.0544, dragging codec token match to 80.4% and inverting the storage ratio to 1:1.24.

This PR fixes exactly that via the algorithmic fix from RVQ_K_LADDER_TUNING.md, plus rolls in the one SIMD opportunity I flagged in the audit (inference-side rms_norm width upgrade).

Test plan

• cargo build --release --example tts_rvq_e2e — clean
• cargo build --release --example amx_bf16_probe — clean (unchanged from perf(tts_rvq_e2e): AVX-512 F32x16 FMA + AMX polyfill probe; recover AudioNode bridge #176)
• Docs render (no emojis, cross-references valid)
• End-to-end run on Qwen3-TTS-0.6B: expect 478/478 tensors cos ≥ 0.99, codec token match ≥ 95%, storage ratio ≥ 2:1. Run launched in background; I'll post numbers as a PR comment.
• AMX validation on a ≥5.19 kernel (polyfill from perf(tts_rvq_e2e): AVX-512 F32x16 FMA + AMX polyfill probe; recover AudioNode bridge #176 is already merged in ndarray)

Design rationale (full text in docs/RVQ_K_LADDER_TUNING.md)

Progressive residual RVQ at k=[..., 4096] cannot reach cos ≈ 1 when k_final < n_rows / 4. At 151,936 rows with k=4096, coverage is 2.7%. Hierarchical CLAM sidesteps the residual-coverage problem by picking one L2 centroid per row (not summing residuals across levels), giving near-centroid reconstruction when average rows-per-leaf ≤ 3.

Forward compatibility

Nothing in this PR blocks the further switch to bgz-tensor::HhtlDTensor + SharedPaletteGroup + FisherZTable for lookup-grade 343:1 ratios. That's a separate session: replace f32 GEMM inference with HHTL cascade inference. This PR keeps the reconstruction-grade path valid for f32 GEMM.

https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj

[AdaWorldAPI]

HCLAM run finished: cos collapsed further, not improved

Exit 0. End-to-end numbers on Qwen3-TTS-0.6B:

[474] model.text_embedding.weight [151936×2048]
   HCLAM 256×256 path:  cos = 0.0046  (58.9s)
   vs RVQ prior run:     cos = 0.0544  (891.1s)

Compressed pass 2:  588.4 s total (was 1417 s with RVQ on that tensor — HCLAM is 15× faster)
Codebook:           4794.4 MB
Indices:               3.5 MB
Total:              4797.9 MB   vs  3657.2 MB original  →  1:1.31  (worse than 1:1.24)

Codec token match:  11/225 (4.9%)   — down from 80.4%
First 5 tokens c0:  RAW 324 324 324 324 324  /  RVQ 1284 1049 1024 1024 1155

Why HCLAM failed worse than RVQ here

My math in docs/RVQ_K_LADDER_TUNING.md § 3 assumed "~2.32 rows per L2 leaf = near 1:1 centroid-to-row". That's false for high-dim quasi-orthogonal rows like a 151K × 2048 BPE embedding.

• Cos between two distinct, unrelated vocab rows in 2048-d space is ≈ 0 (they point in nearly-orthogonal directions by training design).
• Picking the nearest existing row as the reconstruction → cos between target and its single-nearest-neighbor ≈ 0, not 1.
• Furthest-point sampling on near-uniform-direction rows doesn't produce tight clusters — it just partitions the sphere into Voronoi cells of roughly-equal "angular mass" with uncorrelated rows sharing cells by L2 distance accident.

Progressive residual RVQ at k=[…, 4096] failed because k_final < n_rows / 4, but at least the additive reconstruction could synthesize directions by summing codebook vectors. Single-centroid tree quantization cannot synthesize — it can only pick an existing row.

So the RVQ_K_LADDER_TUNING.md Section 3 claim (2.4:1 at cos ≈ 1 via hierarchical CLAM 256×256) is wrong for vocab embeddings. It would be correct for tensors whose rows lie on a low-dim manifold (attention / MLP projections), but those are already at cos=1.000 via the existing RVQ path — they don't need a fix.

Correct next action

The remaining valid options from the original RVQ_ALTERNATIVES.md / my pre-HCLAM analysis are:

1. Skip RVQ on n_rows > 8192 — lossless (keep BF16). Storage: 620 MB for that one tensor. All other 477 tensors still 2:1 or better. Pipeline reaches 100% codec token match immediately. 3-line change.
2. Wire bgz-tensor::HhtlDTensor + SharedPaletteGroup + FisherZTable — the existing 343:1 lookup-grade machinery. Cos ≈ 0.95 per row (doc's own number), but inference goes through the HHTL cascade / Fisher-z lookup, not f32 GEMM — so per-row cos isn't the failure metric. Bigger integration (separate session).
3. Progressive residual RVQ with an ADDITIONAL level at k ≈ n_rows/4 (e.g. [256, 512, 1024, 4096, 32768] on 151K rows). Storage adds 32768 × 2048 × 4B = 256 MB per extra level. Viable but adds weight, doesn't help ratio.

Recommendation: (1) now, (2) in a separate session

(1) is the fastest path to a working pipeline end-to-end. The session's value is a 100% codec token match reality check, not hitting a specific compression ratio. (2) is the real long-term answer; HCLAM was a dead end.

Docs correction

I need to update docs/RVQ_K_LADDER_TUNING.md Section 3 to flag "hierarchical CLAM 256×256 does NOT work for vocab embeddings — kept here as a refuted design". That's a doc-only follow-up PR.

I'll implement (1) now on a new commit, re-run, and post the 100%-token-match number as another comment.

https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj

---

Generated by Claude Code

[AdaWorldAPI]

Passthrough run finished — CORRECTNESS GATE PASSED

Exit 0. End-to-end on Qwen3-TTS-0.6B with the passthrough fix (3b084d1):

[474] model.text_embedding.weight [151936x2048]
  cos=1.0000  passthrough (n_rows>8192, BF16 622.3MB)  — 0s

All 477 other tensors: cos=1.0000 via existing RVQ path

Pass 2 total:        528.4s  (was 1417.1s with broken HCLAM, was 891s with broken RVQ)
Codec token match:   225/225 (100.0%)  ★ SUCCESS

First 5 tokens, codebook 0:
  RAW: 324 324 324 324 324
  RVQ: 324 324 324 324 324     ← exact match

What this confirms (correctness)

• The 33-layer Qwen3-TTS inference + codec head + 15 lm_heads + RVQ codebook round-trip preserves every generated codec token when the encoder's failure mode (vocab tensor at n_rows > 8192) is sidestepped.
• F32x16 FMA + AVX-512 + streaming two-pass + fused l2_dist_sq stack works at production scale on this model.
• The n_rows > 8192 dispatch is a clean gate; all 477 sub-8192 tensors compress at cos=1.000.

What this does NOT solve (storage)

Original weights:   3657.2 MB
RVQ compressed:     5096.7 MB    ← LARGER than original
Ratio:              1:1.39       ← net loss

Root cause: the per-tensor RVQ codebook is individually larger than the BF16 tensor itself for every MLP projection. Example — mlp.gate_proj.weight [3072, 1024]:

• BF16 original: 6.0 MB
• RVQ codebook at k=[256, 512, 1024, 4096]: ~64 MB (4 levels × 4096 centroids × 1024 × 4B, pessimistic upper bound; actual is less because levels 1-3 are smaller)

The custom progressive-residual RVQ in this example shipped reconstruction-grade cos=1 at f32 precision for inference, which is correct but not compression. Shipping compressed weights requires either:

1. bgz-tensor::HhtlDTensor + SharedPaletteGroup (existing code, 343:1 on this exact model per BGZ_HHTL_D.md) — requires switching from f32 GEMM inference to HHTL cascade lookup. Separate session.
2. Lance 4.0 IVF_RQ + multi-split (see docs/LANCE_UPGRADE_ROADMAP.md filed in f6ed834 on this branch) — native Lance index, ~2-6 week migration.
3. Smaller k-ladders + quantized codebook entries — quick fix. Store codebook as BF16 instead of f32, drop the k=4096 level from tensors below ~8192 rows. Order-of-magnitude improvement available cheaply.

Forward signal

Next session's cheapest win is option 3: drop codebook dtype to BF16, cut k-ladder final level for small tensors. Probably 3:1–5:1 storage ratio at still cos=1.000. No architectural change required.

Then option 1 (HHTL-D lookup-grade) gives the 343:1 story for actually shipping codebooks to Releases.

Commits summary on this branch (post-merge of #177)

SHA	What
3b084d1	passthrough BF16 for n_rows > 8192 (HCLAM was wrong — section 3 of k-ladder doc needs REFUTED stamp)
f6ed834	Lance 2 → 4/5 upgrade roadmap (161 LOC doc, next-session planning)

Both on claude/teleport-session-setup-wMZfb, ready for a follow-up PR to main.

https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj

---

Generated by Claude Code