diff --git a/README.md b/README.md index ea07525f..91573c62 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,24 @@ print(y) Browse available kernels at [huggingface.co/kernels](https://huggingface.co/kernels). +## 🎯 Hardware Support + +The `kernel-builder` component of the project supports a number of hardware in +different tiers: + +| Hardware | Kernels Support | Kernel-Builder Support | Kernels Validated in CI | Tier | +| ---------- | --------------- | ---------------------- | ----------------------- | ---- | +| CUDA | ✓ | ✓ | ✓ | 1 | +| CPU | ✓ | ✓ | ✓ | 1 | +| ROCm | ✓ | ✓ | ✗ | 2 | +| XPU | ✓ | ✓ | ✗ | 2 | +| Metal | ✓ | ✓ | ✗ | 2 | +| TPU | ✓ | ✓ | ✗ | 2 | +| Huawei NPU | ✓ | ✗ | ✗ | 3 | +| Neuron | ✓ | x | x | 3 | + +**Warning:** Neuron and TPU support are experimental and packages are not public yet. + ## 📚 Documentation Read the [documentation of kernels and kernel-builder](https://huggingface.co/docs/kernels/). diff --git a/nix-builder/README.md b/nix-builder/README.md deleted file mode 100644 index e59b7119..00000000 --- a/nix-builder/README.md +++ /dev/null @@ -1,73 +0,0 @@ -# kernel-builder - -The kernel-builder is a build system for creating Hub-compatible compute kernels. It handles the complexity of building kernels that are: - -- Portable: kernels can be loaded from paths outside `PYTHONPATH`. -- Unique: multiple versions of the same kernel can be loaded in the same Python process. -- Compatible: kernels support all recent versions of Python and the different PyTorch build configurations (various CUDA versions and C++ ABIs). - -**Note:** Torch 2.10 builds are still based on PyTorch release candidates. -Typically the ABI does not break during release candidates. If it does, -you have to recompile your kernels with the final 2.10.0 release. - -[Join us on GitHub Discussions](https://github.com/huggingface/kernels/discussions) -for questions and discussions! - -This repo contains a Nix package that can be used to build custom machine learning kernels for PyTorch. The kernels are built using the [PyTorch C++ Frontend](https://pytorch.org/cppdocs/frontend.html) and can be loaded from the Hub with the [kernels](https://github.com/huggingface/kernels) -Python package. - -This builder is a core component of the larger kernel build/distribution system. - -## 🚀 Quick Start - -We recommend using [Nix](https://nixos.org/download.html) to build kernels. To speed up builds, first enable the Hugging Face binary cache: - -```bash -# Install cachix and configure the cache -cachix use huggingface - -# Or run once without installing cachix -nix run nixpkgs#cachix -- use huggingface -``` - -Then quick start a build with: - -```bash -cd examples/relu -nix run .#build-and-copy \ - --max-jobs 2 \ - --cores 8 \ - -L -``` - -Where `--max-jobs` specifies the number of build variant that should be built concurrently and `--cores` the number of CPU cores that should be used per build variant. - -The compiled kernel will then be available in the local `build/` directory. - -## 🎯 Hardware Support - -| Hardware | Kernels Support | Kernel-Builder Support | Kernels Validated in CI | Tier | -| ---------- | --------------- | ---------------------- | ----------------------- | ---- | -| CUDA | ✓ | ✓ | ✓ | 1 | -| ROCm | ✓ | ✓ | ✗ | 2 | -| XPU | ✓ | ✓ | ✗ | 2 | -| Metal | ✓ | ✓ | ✗ | 2 | -| Huawei NPU | ✓ | ✗ | ✗ | 3 | -| Neuron | ✓ | x | x | 3 | -| TPU | ✓ | ✗ | ✗ | 3 | - -**Warning:** Neuron and TPU support are experimental and packages are not public yet. - -# 📚 Documentation - -- [Writing Hub kernels](../docs/source/builder/writing-kernels.md) -- [Building kernels with Nix](../docs/source/builder/nix.md) -- Framework-specific notes: - - [Metal](../docs/source/builder/metal.md) -- [Local kernel development](../docs/source/builder/local-dev.md) (IDE integration) -- [Kernel security](../docs/source/builder/security.md) -- [Why Nix?](../docs/source/builder/why-nix.md) - -## Credits - -The generated CMake build files are based on the vLLM build infrastructure.