Add quantized_div op (#21294) by 3l1 · Pull Request #21294 · pytorch/executorch

3l1 · 2026-07-23T21:36:50Z

Summary:

Adds a quantized_div operator to the ExecuTorch Cortex-M (CMSIS-NN)
backend, giving int8 and int16 elementwise division alongside the
existing quantized_add/quantized_mul ops.

CMSIS-NN has no integer elementwise-division primitive, so the quotient
of the zero-point-corrected operands is evaluated in float and rescaled
by the effective scale scale_a / (scale_b * scale_out). That scale is
folded AoT into output_multiplier/output_shift (so the schema matches
quantized_mul) and reconstructed to a float in the kernel the same way
op_softmax reconstructs its input scale. A zero denominator maps to a 0
quotient in both the kernel and the reference.

int8 and int16 activations are both supported. The kernel dispatches on
the output dtype (Char/Short) into a templated loop, and the
composite reference clamps to the dtype range. A new
INT16_PER_TENSOR_CONFIG (symmetric int16 activations) is added,
CortexMDivCheck accepts int8 or int16, and CortexMQuantizer takes an
optional per_tensor_config so callers can opt into int16. The op schema
and the lowering pass are dtype-transparent and unchanged.

Reviewed By: rascani

Differential Revision: D113440315

pytorch-bot · 2026-07-23T21:36:54Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21294

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2141c1e with merge base b20f16a ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

meta-codesync · 2026-07-23T21:37:00Z

@3l1 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113440315.

github-actions · 2026-07-23T21:37:47Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so — following the Arm reference path in bolt's `flatbuffers_ops` backend — the quotient of the zero-point-corrected int8 operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. Division is not commutative, so the operand-swap trick that lets `quantized_mul` support channel broadcasting does not apply here; the op and its quantizer pattern check (`CortexMDivCheck`) require identically shaped int8 per-tensor inputs and reject any broadcasting. Wiring: `operators.py` (schema + reference impl), `op_quantized_div.cpp` (kernel), `operators.yaml`/`targets.bzl`/`CMakeLists.txt` (registration and build), `aten_to_cortex_m_pass.py` (`aten.div.Tensor` lowering), and the quantizer support dict. All changes are mirrored across the fbcode and xplat trees. Review order: start with `op_quantized_div.cpp` and the `quantized_div` block in `operators.py` (the arithmetic), then the pass and quantizer wiring, then the registration/build files and the test. Authored with Claude Code. Differential Revision: D113440315

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so the quotient of the zero-point-corrected int8 operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. Division is not commutative, so the operand-swap trick that lets `quantized_mul` support channel broadcasting does not apply here; the op and its quantizer pattern check (`CortexMDivCheck`) require identically shaped int8 per-tensor inputs and reject any broadcasting. Wiring: `operators.py` (schema + reference impl), `op_quantized_div.cpp` (kernel), `operators.yaml`/`targets.bzl`/`CMakeLists.txt` (registration and build), `aten_to_cortex_m_pass.py` (`aten.div.Tensor` lowering), and the quantizer support dict. All changes are mirrored across the fbcode and xplat trees. This also wires the Cortex-M op tests under `backends/cortex_m/test/ops/` into Buck for the first time — they previously ran only via raw `pytest`. `test/TARGETS` now generates one `python_pytest` target per op test file, plus the `tester` and `conftest` `python_library` wrappers the tests import. These targets live in `TARGETS` (fbcode-only), not `targets.bzl`/`BUCK`, so they never enter the OSS-parsed build graph; open-source keeps running these files via `pytest`. Authored with Claude Code. Differential Revision: D113440315

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so — following the Arm reference path in bolt's `flatbuffers_ops` backend — the quotient of the zero-point-corrected int8 operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. Differential Revision: D113440315

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so the quotient of the zero-point-corrected int8 operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. Differential Revision: D113440315

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 and int16 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so the quotient of the zero-point-corrected operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. int8 and int16 activations are both supported. The kernel dispatches on the output dtype (`Char`/`Short`) into a templated loop, and the composite reference clamps to the dtype range. A new `INT16_PER_TENSOR_CONFIG` (symmetric int16 activations) is added, `CortexMDivCheck` accepts int8 or int16, and `CortexMQuantizer` takes an optional `per_tensor_config` so callers can opt into int16. The op schema and the lowering pass are dtype-transparent and unchanged. Differential Revision: D113440315

rascani · 2026-07-24T22:42:45Z

+        : 0.0f;
+
+    int32_t result =
+        static_cast<int32_t>(std::round(quotient * effective_scale)) + out_zp;


If we're not worried about performance here, we might want to keep this and the clamp in float to avoid any risk of exceeding int32 bounds.

Summary: Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 and int16 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so the quotient of the zero-point-corrected operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. int8 and int16 activations are both supported. The kernel dispatches on the output dtype (`Char`/`Short`) into a templated loop, and the composite reference clamps to the dtype range. A new `INT16_PER_TENSOR_CONFIG` (symmetric int16 activations) is added, `CortexMDivCheck` accepts int8 or int16, and `CortexMQuantizer` takes an optional `per_tensor_config` so callers can opt into int16. The op schema and the lowering pass are dtype-transparent and unchanged. Reviewed By: rascani Differential Revision: D113440315

Summary: Pull Request resolved: #21294 Adds a `quantized_div` operator to the ExecuTorch Cortex-M (CMSIS-NN) backend, giving int8 and int16 elementwise division alongside the existing `quantized_add`/`quantized_mul` ops. CMSIS-NN has no integer elementwise-division primitive, so the quotient of the zero-point-corrected operands is evaluated in float and rescaled by the effective scale `scale_a / (scale_b * scale_out)`. That scale is folded AoT into `output_multiplier`/`output_shift` (so the schema matches `quantized_mul`) and reconstructed to a float in the kernel the same way `op_softmax` reconstructs its input scale. A zero denominator maps to a 0 quotient in both the kernel and the reference. int8 and int16 activations are both supported. The kernel dispatches on the output dtype (`Char`/`Short`) into a templated loop, and the composite reference clamps to the dtype range. A new `INT16_PER_TENSOR_CONFIG` (symmetric int16 activations) is added, `CortexMDivCheck` accepts int8 or int16, and `CortexMQuantizer` takes an optional `per_tensor_config` so callers can opt into int16. The op schema and the lowering pass are dtype-transparent and unchanged. Reviewed By: rascani Differential Revision: D113440315

Erik-Lundell

I think you should revisit not supporting broadcasting, looks good otherwise.

Erik-Lundell · 2026-07-27T11:29:09Z

+
+    const float quotient = (denominator != 0)
+        ? static_cast<float>(numerator) / static_cast<float>(denominator)
+        : 0.0f;


Can you motivate the choice of 0 here?

Erik-Lundell · 2026-07-27T11:32:43Z

+    output_scale = node.meta["output_qparams"][0].scale
+    output_zero_point = node.meta["output_qparams"][0].zp
+
+    output_mult, output_shift = quantize_multiplier_aot(


Why do this an then reverse in the kernel? Is this just to mirror other operators?

Erik-Lundell · 2026-07-27T11:41:39Z

+    return out;
+  }
+
+  // Division is not commutative, so channel broadcasting (which relies on


This is not a mathematical blocker right? I don't see a good reason for not supporting broadcasting, the loop just needs to be a bit smarter about how it selects indices.

Erik-Lundell · 2026-07-27T11:51:55Z

 class CortexMQuantizer(ComposableQuantizer):

-    def __init__(self) -> None:
+    def __init__(self, per_tensor_config: Optional[QuantizationConfig] = None) -> None:


Changes public API, but looks reasonable to me. Can you add a proper docstring?

Erik-Lundell · 2026-07-27T11:52:58Z


-    def __init__(self) -> None:
+    def __init__(self, per_tensor_config: Optional[QuantizationConfig] = None) -> None:
+        # Per-tensor activation config used for the "global" (non-conv) ops such


Hm, I have not heard of "global" ops. What does that mean?

Erik-Lundell · 2026-07-27T11:54:26Z

+
+# Divisors are kept strictly positive so the quantized denominator never lands
+# on its zero point (which the kernel maps to a 0 quotient).
+test_cases = {


Can we get a test case with a very small denominator?

3l1 requested review from kirklandsign, larryliu0820 and rascani as code owners July 23, 2026 21:36

3l1 had a problem deploying to cadence July 23, 2026 21:36 — with GitHub Actions Error

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 23, 2026

meta-codesync Bot added the meta-exported label Jul 23, 2026

meta-codesync Bot changed the title ~~Add quantized_div op~~ Add quantized_div op (#21294) Jul 23, 2026

meta-codesync Bot force-pushed the export-D113440315 branch from ffc40e1 to 7a8d40f Compare July 23, 2026 21:39

meta-codesync Bot had a problem deploying to cadence July 23, 2026 21:39 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 7a8d40f to 984afe1 Compare July 23, 2026 21:44

meta-codesync Bot had a problem deploying to cadence July 23, 2026 21:45 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 984afe1 to 083a996 Compare July 23, 2026 21:51

meta-codesync Bot had a problem deploying to cadence July 23, 2026 21:52 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 083a996 to 2c27357 Compare July 23, 2026 21:53

meta-codesync Bot had a problem deploying to cadence July 23, 2026 21:54 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 2c27357 to 6ee96cf Compare July 23, 2026 21:55

meta-codesync Bot had a problem deploying to cadence July 23, 2026 21:55 Error

meta-codesync Bot had a problem deploying to cadence July 24, 2026 00:52 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 527ef5d to 8b5f571 Compare July 24, 2026 00:57

meta-codesync Bot temporarily deployed to cadence July 24, 2026 00:58 Inactive

meta-codesync Bot temporarily deployed to cadence July 24, 2026 01:22 Inactive

rascani requested review from AdrianLundell and Erik-Lundell July 24, 2026 16:12

rascani approved these changes Jul 24, 2026

View reviewed changes

meta-codesync Bot force-pushed the export-D113440315 branch from 8b5f571 to dfd1c6f Compare July 24, 2026 22:59

meta-codesync Bot had a problem deploying to cadence July 24, 2026 22:59 Error

meta-codesync Bot force-pushed the export-D113440315 branch from dfd1c6f to 0df6fd1 Compare July 24, 2026 23:00

meta-codesync Bot temporarily deployed to cadence July 24, 2026 23:01 Inactive

meta-codesync Bot temporarily deployed to cadence July 24, 2026 23:27 Inactive

meta-codesync Bot force-pushed the export-D113440315 branch from 0df6fd1 to 74ca6ee Compare July 25, 2026 00:29

meta-codesync Bot had a problem deploying to cadence July 25, 2026 00:29 Error

meta-codesync Bot force-pushed the export-D113440315 branch from 74ca6ee to ee5f86d Compare July 25, 2026 00:30

meta-codesync Bot had a problem deploying to cadence July 25, 2026 00:31 Error

Erik-Lundell requested changes Jul 27, 2026

View reviewed changes

Uh oh!

Conversation

3l1 commented Jul 23, 2026 • edited by meta-codesync Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21294

✅ No Failures

Uh oh!

meta-codesync Bot commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

This PR needs a release notes: label

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Erik-Lundell left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

3l1 commented Jul 23, 2026 •

edited by meta-codesync Bot

Loading

pytorch-bot Bot commented Jul 23, 2026 •

edited

Loading

This PR needs a `release notes:` label