gibblox + gobblytes

༼;´༎ຶ ۝ ༎ຶ༽つ ┳━┳ pl0x gib bl0x.

gibblox fetches blocks of data.

Maybe they're on another computer.

Maybe they're in an EroFS in an ISO9660.

Maybe they're in an ext4 in a GPT in an Android sparse image in an XZ stream.

gibblox doesn't give a flying fuck.

gibblox just gives blocks.

This project is primarily intended to serve smoo and fastboop. Maybe you have some other boring-ass use case that calls for some blocks. gibblox doesn't discriminate. gibblox has a vision for a perfectly just and fair block-ocracy. You will be assimilated. To gibblox you will always be just another block.

Quickstart

gibblox is primarily intended to be consumed as a library. It also offers a simple CLI:

# gibblox hasn't had a release yet, hence the funny version specifier
cargo install 'gibblox-cli@>=0.0.1-rc'

# alternatively, use binstall if your CPU needs a break and you don't mind
# me mining some cryptos on your machine (thanks for that, btw ❤️)
cargo binstall 'gibblox-cli@>=0.0.1-rc'

Use this CLI work with the pipeline schema, as well as testing the data path:

# create a pipeline that fetches a remote file
gibblox pipeline encode <<HERE > /tmp/pipeline.bin
http: https://boot.ipxe.org/ipxe.iso
HERE

# run that pipeline
gibblox /tmp/pipeline.bin > /tmp/ipxe.iso

Goals

• Provide read-only BlockReader + ByteReader abstractions, with explicit block/byte boundary adapters.
• Support HTTP Range reads on native and wasm targets.
• Offer a file-backed cache layer with in-flight coalescing.
• Keep core crates no_std + alloc where practical.

Crates

• gobblytes-core: core filesystem abstraction traits, OSTree wrapper, and test helpers.
• gobblytes-erofs: EROFS-backed filesystem adapter.
• gobblytes-fat: FAT32-backed filesystem adapter.
• gibblox-core: core traits and error types.
• gibblox-android-sparse: Android sparse image block reader.
• gibblox-mbr: MBR partition-backed block reader.
• gibblox-zip: ZIP file entry-backed block reader (stored and deflate entries).
• gibblox-casync: no_std casync index + reconstruction core.
• gibblox-casync-std: native index/chunk source + integrated chunk cache/store.
• gibblox-casync-web: wasm-only index/chunk source + CacheStorage-backed chunk store.
• gibblox-iso9660: ISO9660 file-backed block reader.
• gibblox-ext4: ext4 filesystem adapter and ext4 file-backed block reader.
• gibblox-http: HTTP Range-backed block reader (native + wasm).
• gibblox-web-file: browser File-backed block reader for wasm targets.
• gibblox-pipeline: product-agnostic source pipeline descriptor + validation + binary framing.
• gibblox-cache: cache layer and cache file format logic.
• gibblox-cache-store-std: XDG-friendly filesystem cache backend for native CLI/desktop apps.
• gibblox-cache-store-opfs: OPFS cache backend for wasm web apps.
• gibblox-cli: CLI utility for pipeline YAML/binary conversion and validation.

ext4: today vs tomorrow

Today:

• gibblox-ext4 works on native and wasm targets.
• The parser path is still sync under the hood; async APIs bridge through blocking reads.
• This is acceptable for now when gibblox runs in a worker thread, but long ext4 operations can still monopolize that worker.

Tomorrow:

• Move ext4 loading and file reads to a true async path end-to-end (no blocking bridge).
• Keep worker responsiveness by chunking/coop-yielding large ext4 operations.
• Push ext4 support toward a clean no_std + alloc profile.

Usage (native)

use gibblox_core::{BlockReader, ReadContext};
use gibblox_cache::CachedBlockReader;
use gibblox_cache_store_std::StdCacheOps;
use gibblox_http::HttpReader;

# async fn example() -> anyhow::Result<()> {
let source = HttpReader::new("https://example.com/rootfs.img".parse()?, 4096).await?;
let cache = StdCacheOps::open_default_for_reader(&source).await?;
let source = CachedBlockReader::new(source, cache).await?;
let mut buf = vec![0u8; 4096];
source.read_blocks(0, &mut buf, ReadContext::FOREGROUND).await?;
# Ok(())
# }

Tests

cargo test --workspace

Status

Early-stage: API and crate layout are still moving. Expect breaking changes.