gibblox + gobblytes

༼;´༎ຶ ۝ ༎ຶ༽つ ┳━┳ pl0x gib bl0x.

gibblox fetches blocks of data.

Maybe they're on another computer.

Maybe they're in an EroFS in an ISO9660.

Maybe they're in an ext4 in a GPT in an Android sparse image in an XZ stream.

gibblox doesn't give a flying fuck.

gibblox just gives blocks.

This project is primarily intended to serve smoo and fastboop. Maybe you have some other boring-ass use case that calls for some blocks. gibblox doesn't discriminate. gibblox has a vision for a perfectly just and fair block-ocracy. You will be assimilated. To gibblox you will always be just another block.

Quickstart

gibblox is primarily intended to be consumed as a library. It also offers a simple CLI:

# gibblox hasn't had a release yet, hence the funny version specifier
cargo install 'gibblox-cli@>=0.0.1-rc'

# alternatively, use binstall if your CPU needs a break and you don't mind
# me mining some cryptos on your machine (thanks for that, btw ❤️)
cargo binstall 'gibblox-cli@>=0.0.1-rc'

Use this CLI work with the pipeline schema, as well as testing the data path:

# create a pipeline that fetches a remote file
gibblox pipeline encode <<HERE > /tmp/pipeline.bin
http: https://boot.ipxe.org/ipxe.iso
HERE

# run that pipeline
gibblox /tmp/pipeline.bin > /tmp/ipxe.iso

Goals

• Provide read-only BlockReader + ByteReader abstractions, with explicit block/byte boundary adapters.
• Support HTTP Range reads on native and wasm targets.
• Offer a file-backed cache layer with in-flight coalescing.
• Keep core crates no_std + alloc where practical.

Crates

• gobblytes-core: core filesystem abstraction traits, OSTree wrapper, and test helpers.
• gobblytes-erofs: EROFS-backed filesystem adapter.
• gobblytes-fat: FAT32-backed filesystem adapter.
• gibblox-core: core traits and error types.
• gibblox-android-sparse: Android sparse image block reader.
• gibblox-mbr: MBR partition-backed block reader.
• gibblox-zip: ZIP file entry-backed block reader (stored and deflate entries).
• gibblox-tar: TAR file entry-backed byte reader.
• gibblox-casync: no_std casync index + reconstruction core.
• gibblox-casync-std: native index/chunk source + integrated chunk cache/store.
• gibblox-casync-web: wasm-only index/chunk source + CacheStorage-backed chunk store.
• gibblox-iso9660: ISO9660 file-backed block reader.
• gibblox-ext4: ext4 filesystem adapter and ext4 file-backed block reader.
• gibblox-http: HTTP Range-backed block reader (native + wasm).
• gibblox-web-file: browser File-backed block reader for wasm targets.
• gibblox-pipeline: product-agnostic source pipeline descriptor + validation + binary framing.
• gibblox-cache: cache layer and cache file format logic.
• gibblox-cache-store-std: XDG-friendly filesystem cache backend for native CLI/desktop apps.
• gibblox-cache-store-opfs: OPFS cache backend for wasm web apps.
• gibblox-cli: CLI utility for pipeline YAML/binary conversion and validation.

ext4

• gibblox-ext4 uses an async-first parser backend via ext4plus.
• ext4 load and read operations run without blocking bridges.
• The crate remains read-only at the gibblox/gobblytes integration surface.

Updating hadris dependencies

The FAT and MBR adapters constrain the complete hadris dependency family with exact versions. FAT/common/I/O/macros use 1.2.0; partition support remains on 1.1.2. The extra direct I/O, common, and macro dependencies are intentional: they constrain transitive versions in published library manifests, where this repository's Cargo.lock does not apply. All runtime dependencies retain default-features = false for the no_std + alloc path.

Update these pins and the workspace lockfile together. Run the FAT/MBR tests, Clippy, and just ci-consumers (also included in native/wasm CI). The latter builds FAT-only, MBR-only, and combined consumers in separate workspaces with fresh lockfiles, using the repository's Rust toolchain. This checks supported downstream resolution without inheriting the workspace lock or unrelated members' features. It uses local adapter sources and registry dependencies; release packaging remains a separate check.

In particular, hadris-fat 1.2.0 returns Cow<str> for entry names. A prior Clippy cleanup against a lockfile containing hadris-fat 1.0.1 removed the necessary .as_ref() conversions and broke fresh consumers. Keep the locked and downstream graphs aligned when applying dependency-related lint fixes.

Usage (native)

use gibblox_core::{BlockReader, ReadContext};
use gibblox_cache::CachedBlockReader;
use gibblox_cache_store_std::StdCacheOps;
use gibblox_http::HttpReader;

# async fn example() -> anyhow::Result<()> {
let source = HttpReader::new("https://example.com/rootfs.img".parse()?, 4096).await?;
let cache = StdCacheOps::open_default_for_reader(&source).await?;
let source = CachedBlockReader::new(source, cache).await?;
let mut buf = vec![0u8; 4096];
source.read_blocks(0, &mut buf, ReadContext::FOREGROUND).await?;
# Ok(())
# }

Tests

cargo test --workspace

Status

Early-stage: API and crate layout are still moving. Expect breaking changes.