kernel-builder: Python script to generate symbols in `__all__` by sayakpaul · Pull Request #878 · huggingface/kernels

sayakpaul · 2026-10-06T06:52:47Z

Part of #876.

As we decided to have a post-hoc build approach, this is the first PR in the series to ship a kernels describe-api.

This PR adds a Python script inside nix-builder (with tests) that generates symbols.json after loading the built kernel through get_local_kernel() and then inspecting it. Example symbols.json:

Unfold

{
  "schema_version": 1,
  "module": "_liger_kernels_cuda_symbols_demo",
  "functions": [
    {
      "name": "LigerForCausalLMLoss",
      "qualname": "LigerForCausalLMLoss",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "function",
      "doc": null,
      "signature": {
        "parameters": [
          {
            "name": "logits",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "None"
          },
          {
            "name": "labels",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "torch.Tensor"
          },
          {
            "name": "vocab_size",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "int"
          },
          {
            "name": "num_items_in_batch",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "None",
            "annotation": "Optional[int]"
          },
          {
            "name": "ignore_index",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "-100",
            "annotation": "int"
          },
          {
            "name": "shift_labels",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "None",
            "annotation": "Optional[torch.Tensor]"
          },
          {
            "name": "hidden_states",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "None",
            "annotation": "torch.Tensor | None"
          },
          {
            "name": "lm_head_weight",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "None",
            "annotation": "torch.Tensor | None"
          },
          {
            "name": "lm_head_bias",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "None",
            "annotation": "torch.Tensor | None"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": null
          }
        ],
        "return_annotation": null
      }
    },
    {
      "name": "liger_rotary_pos_emb",
      "qualname": "liger_rotary_pos_emb",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "function",
      "doc": "Apply standard rotary positional embedding to ``q`` and ``k``.",
      "signature": {
        "parameters": [
          {
            "name": "q",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "torch.Tensor"
          },
          {
            "name": "k",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "torch.Tensor"
          },
          {
            "name": "cos",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "torch.Tensor"
          },
          {
            "name": "sin",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": null,
            "annotation": "torch.Tensor"
          },
          {
            "name": "unsqueeze_dim",
            "kind": "POSITIONAL_OR_KEYWORD",
            "default": "1",
            "annotation": "int"
          }
        ],
        "return_annotation": "tuple[torch.Tensor, torch.Tensor]"
      }
    }
  ],
  "layers": [
    {
      "name": "LigerRMSNorm",
      "qualname": "LigerRMSNorm",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerLinear",
      "qualname": "LigerLinear",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerSwiGLUMLP",
      "qualname": "LigerSwiGLUMLP",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerGEGLUMLP",
      "qualname": "LigerGEGLUMLP",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerTiledSwiGLUMLP",
      "qualname": "LigerTiledSwiGLUMLP",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerTiledGEGLUMLP",
      "qualname": "LigerTiledGEGLUMLP",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "liger_rotary_pos_emb_layer",
      "qualname": "liger_rotary_pos_emb_layer",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    },
    {
      "name": "LigerForCausalLMLossLayer",
      "qualname": "LigerForCausalLMLossLayer",
      "module": "_liger_kernels_cuda_symbols_demo.layers",
      "kind": "class",
      "doc": "Base class for all neural network modules.\n\nYour models should also subclass this class.\n\nModules can also contain other Modules, allowing them to be nested in\na tree structure. You can assign the submodules as regular attributes::\n\n    import torch.nn as nn\n    import torch.nn.functional as F\n\n\n    class Model(nn.Module):\n        def __init__(self) -> None:\n            super().__init__()\n            self.conv1 = nn.Conv2d(1, 20, 5)\n            self.conv2 = nn.Conv2d(20, 20, 5)\n\n        def forward(self, x):\n            x = F.relu(self.conv1(x))\n            return F.relu(self.conv2(x))\n\nSubmodules assigned in this way will be registered, and will also have their\nparameters converted when you call :meth:`to`, etc.\n\n.. note::\n    As per the example above, an ``__init__()`` call to the parent class\n    must be made before assignment on the child.\n\n:ivar training: Boolean represents whether this module is in training or\n                evaluation mode.\n:vartype training: bool",
      "signature": {
        "parameters": [
          {
            "name": "args",
            "kind": "VAR_POSITIONAL",
            "default": null,
            "annotation": "Any"
          },
          {
            "name": "kwargs",
            "kind": "VAR_KEYWORD",
            "default": null,
            "annotation": "Any"
          }
        ],
        "return_annotation": "None"
      },
      "attributes": {
        "has_backward": null,
        "can_torch_compile": true
      }
    }
  ]
}

HuggingFaceDocBuilderDev · 2026-10-06T06:56:47Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-10-06T07:23:40Z

Coverage report — `kernels/`

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.4% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	14	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	130	14	89%	53, 59-60, 63-64, 102, 165-170, 199, 219
src/kernels/archs.py	56	1	98%	94
src/kernels/backends.py	213	62	71%	42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 226, 230, 234-254, 262, 285-305
src/kernels/compat.py	9	1	89%	5
src/kernels/deps.py	70	1	99%	56
src/kernels/hf_hub.py	63	2	97%	21, 23
src/kernels/importer.py	44	5	89%	80, 84, 87, 101-102
src/kernels/install.py	21	7	67%	76-100
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	85	6	93%	90, 115, 191, 311, 338, 368
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	82	8	90%	259, 297, 305-306, 312, 316, 332-334
src/kernels/layer/layer.py	309	22	93%	211, 258, 285, 416, 437, 444-447, 453-454, 556-557, 578, 586, 597, 637, 641, 654, 708, 738, 813
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py	71	2	97%	338, 378
src/kernels/locking.py	89	64	28%	35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py	58	6	90%	59-60, 64-65, 101, 104
src/kernels/resolver.py	156	2	99%	220, 226
src/kernels/status.py	50	2	96%	25, 79
src/kernels/validate.py	88	5	94%	9, 100, 167, 190-191
src/kernels/variants.py	278	17	94%	65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 455-461
src/kernels/verify.py	127	6	95%	46, 202-204, 318-319
TOTAL	2339	294	87%

Updated by the Test kernels workflow on commit 170bc87649dea7d7814b9ed537dae6f8da6bea4d.

danieldk

Cool! Added some small comments.

danieldk

Nice!!!!

feat: implement kernels describe.

1928456

add machine-readability support.

317ed94

sayakpaul marked this pull request as ready for review October 7, 2026 05:53

Merge branch 'main' into feat-kernels-describe

26ce524

sayakpaul changed the title ~~[wip] feat: implement kernels describe.~~ feat: implement kernels describe. Oct 7, 2026

sayakpaul added 2 commits October 7, 2026 14:54

rename api to describe-api.

0e9cdd5

Merge branch 'main' into feat-kernels-describe

2dd5971

sayakpaul changed the title ~~feat: implement kernels describe.~~ feat: implement kernels describe-api Oct 7, 2026

sayakpaul marked this pull request as draft October 7, 2026 12:15

sayakpaul added 2 commits October 8, 2026 11:32

Merge branch 'main' into feat-kernels-describe

170bc87

kernel-builder: first stab at symbol generation python script.

843e4e9

sayakpaul changed the title ~~feat: implement kernels describe-api~~ kernel-builder: Python script to generate symbols in __all__ Oct 8, 2026

sayakpaul added 2 commits October 8, 2026 08:41

kernel-builder: remove unneeded tests.

55c573d

simplify tests.

ed17a22

sayakpaul marked this pull request as ready for review October 8, 2026 10:28

danieldk reviewed Oct 8, 2026

View reviewed changes

Comment thread nix-builder/pkgs/generate-symbols/src/generate_symbols.py

Comment thread nix-builder/pkgs/generate-symbols/generate_symbols.py Outdated

Comment thread nix-builder/pkgs/generate-symbols/test_generate_symbols.py Outdated

sayakpaul added 5 commits October 8, 2026 12:12

kernel-builder: move script and adjust paths.

904b52b

remove importlib fallback.

21f99a7

kernel-builder: test for no export.

d313f02

kernel-builder: turn symbol generation into a proper python project

70a1f8c

Merge branch 'main' into feat-kernels-describe

2251cd7

sayakpaul requested review from danieldk and a balanced review from Copilot and removed request for Copilot October 8, 2026 12:42

Copilot stopped reviewing on behalf of sayakpaul due to an error October 8, 2026 12:43
An unexpected error occurred. For more details, see the detailed logs in GitHub Actions.

danieldk approved these changes Oct 8, 2026

View reviewed changes

sayakpaul merged commit 2a5f9d1 into main Oct 8, 2026
47 of 48 checks passed

sayakpaul deleted the feat-kernels-describe branch October 9, 2026 02:24

Navigation Menu

Repository navigation

kernel-builder: Python script to generate symbols in `all` - #878

kernel-builder: Python script to generate symbols in `all`#878
sayakpaul merged 14 commits into
mainfrom
feat-kernels-describe

sayakpaul commented Oct 6, 2026 •

edited

Loading

Uh oh!

HuggingFaceDocBuilderDev commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

danieldk left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danieldk left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sayakpaul commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report — kernels/

Uh oh!

danieldk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danieldk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sayakpaul commented Oct 6, 2026 •

edited

Loading

github-actions Bot commented Oct 6, 2026 •

edited

Loading

Coverage report — `kernels/`