test(kv): measure KL by context depth; INT4 depth-stable, KIVI degrades (#69)

[geisten]

Closes #69.

Follow-up from #65: quality was measured only at a fixed length. bench_kv_quality's score_mode now splits mean KL(fp32||mode) at the sequence midpoint (early vs late positions), so 'near-lossless' can be checked as context grows. A longer document via argv[2] drives the full long-context run.

Result (BitNet-2b4t, 314 tokens, early = first ~156, late = last ~157):

mode	KL early	KL late	trend
INT8	0.00190	0.00204	flat
INT4	0.01357	0.01194	flat
INT4+ROT	0.01042	0.00681	improves with depth
INT2	1.596	1.674	flat-ish
KIVI	0.12033	0.40269	3.3× worse with depth

Two findings:

1. INT4 / INT4+ROT do not degrade with context depth — late KL ≤ early KL. Rotation is per-position independent and INT4 quant is per-token, so there's no error accumulation. Confirms the near-lossless claim holds as context grows.
2. KIVI degrades markedly with depth (0.12 → 0.40): its per-channel 2-bit drain accumulates error across the 128-token groups. So the packed-INT4+ROT advantage over KIVI widens with context.

Caveat: this is a 314-token midpoint split, not a multi-thousand-token run — enough to show the trend (the KIVI drain fires ~token 128, so its depth-dependence is already visible), and the tooling now supports the full run via argv[2].

🤖 Generated with Claude Code