docs: the bitnet.cpp comparison recording, and three use-case demos - #214
Merged
Conversation
assets/versus-bitnetcpp.gif is the fairness-first version of the headline claim: geistlib and bitnet.cpp on the SAME 4 GB Pi 5, the SAME byte-identical i2_s GGUF, both greedy at 4 threads, each take starting from a thermally gated board with temperature, engine version and context visible in the frame — recorded sequentially (running both at once would split the cores and slow both) and composited side by side. This recording: 110 tokens in 7.1 s (15.5 tok/s end-to-end) vs bitnet.cpp's own "9.31 tokens per second" timing line. Baseline choice, documented in docs/DEMOS.md: microsoft/BitNet @ 404980e (June 2025), because today's main (0b341e5) mis-decodes the canonical i2_s GGUF on ARM into "????" — speed-only harnesses never notice, a video does. The June state generates text token-identical to geist's greedy output for the opening tokens, which is the iso-quality statement made visible. Getting there also surfaced that gcc-12 ICEs on the fork's quantize_i2_s (their clang requirement is real). Two use-case recordings, both honest by construction: the offline knowledge box runs the whole shell under `unshare -n` with the failing ping in frame (answers chosen from prompts the model verifiably gets right — the radiator question it got confidently wrong stayed on the cutting-room floor and the model-limits doc exists for exactly that reason); the writing assistant pipes a draft through batch mode. The embedded-appliance use case is a code box in DEMOS.md, not a video — there is no terminal footage of a product that doesn't exist yet. README: the comparison GIF sits above the scoreboard in "How fast?", one line links the demos page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
assets/versus-bitnetcpp.gif— geist vs bitnet.cpp on the same 4 GB Pi 5, byte-identical GGUF, both greedy/4 threads, thermally gated, versions+temps in frame; recorded sequentially, composited side by side. 15.5 tok/s vs 9.31 tok/s (its own timing line). MP4 twin delivered locally for social use.docs/DEMOS.md— the comparison methodology (incl. why the baseline is BitNet404980e: currentmainmis-decodes canonical i2_s on ARM into????— found while making this), the offline-box demo (unshare -n, failing ping in frame), the writing-assistant demo (batch mode), and the embedded-appliance code box.