Skip to content

Add CPU image decoders with measured per-format backends - #12

Closed
MilkClouds wants to merge 5 commits into
mainfrom
feat/image-decoders
Closed

MilkClouds wants to merge 5 commits into
mainfrom
feat/image-decoders

Conversation

@MilkClouds

@MilkClouds MilkClouds commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Image decoding now has a TorchCodec-style CPU API without a Pillow or PyTorch runtime dependency. Images use dedicated codec backends selected through pixel comparisons and release-build measurements instead of routing through FFmpeg.

  • Add decode_image, decode_jpeg/png/webp/gif/avif/heic, and ImageReadMode, returning NumPy CHW/NCHW arrays. Support mode/dtype conversion, orientation, alpha, animations, JPEG lists/CMYK, PNG 8/16-bit samples, and high-depth AVIF/HEIC.
  • Use Rust png for PNG; libjpeg-turbo for JPEG; libwebp/libwebpdemux for WebP; giflib for GIF; and libavif/dav1d/libyuv for AVIF. Wheel builds bundle these dependencies with their licenses. HEIC continues to use optional system libheif. Audio/video retain FFmpeg.
  • Add textured/progressive/subsampled image regressions, PNG palette/transparency/Adam7 and mode/dtype parity, malformed-input checks, AVIF alpha/sequences, and Pillow-free wheel runtime tests. Animated PNG remains explicitly unsupported.
  • Include a reproducible comparison against TorchCodec, Pillow, image and libjpeg-turbo-rs. Across 54 measured RGB inputs, the selected implementation matched TorchCodec pixels exactly; JPEG/WebP latency was roughly comparable and PNG was about 1.7x faster. This is a limited x86 workload, not a general performance or full-format parity guarantee. Experimental crates are isolated from production dependencies.

Validation:

  • pytest --compare: 444 passed locally.
  • Ruff lint/format, Cargo fmt and Clippy with -D warnings passed.
  • Source-built image dependencies and a repaired Linux wheel were validated locally. A fresh uv environment with only NumPy and the wheel decoded image/animation, audio and video fixtures with no Pillow, TorchCodec, PyTorch or PyAV.
  • Local repaired wheel: 13.28 MiB download / 33.09 MiB unpacked, within existing size limits. Local ABI tag is manylinux_2_34; portable manylinux2014 remains the existing release CI build target.

See docs/image-backends.md for versions, methodology, raw reports, alternatives and limitations. Python runtime dependencies remain NumPy only. This PR does not add average_rate or change the released version.

@MilkClouds MilkClouds changed the title Add TorchCodec-style CPU image decoding Add CPU image decoders with measured per-format backends Oct 2, 2026
@MilkClouds

Copy link
Copy Markdown
Collaborator Author

Superseded by a smaller Python-level OpenCV image adapter. The replacement PR will retain the comparison findings without adding native image builds or vendored codec sources.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant