Skip to content

WIP cl_khr_cooperative_matrix - #1533

Open
kpet wants to merge 1 commit into
mainfrom
cl-khr-cooperative-matrix
Open

kpet wants to merge 1 commit into
mainfrom
cl-khr-cooperative-matrix

Conversation

@kpet

@kpet kpet commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

No description provided.

Change-Id: I902b6ac47788dfb5083463e3ab02a23985e1ba52
@kpet

kpet commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author

This PR currently captures the working draft for the coming cl_khr_cooperative_matrix extension. The OpenCL working group is still making minor changes to the specification. We decided to continue doing this in public so we could engage with a wider set of stakeholders. Feedback welcome!

Comment thread env/extensions.asciidoc
{clGetDeviceCooperativeMatrixInfoKHR} device query, otherwise
the behavior of the program is undefined.

For *OpTypeCooperativeMatrixMulAddKHR*, the operands must match a supported

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For *OpTypeCooperativeMatrixMulAddKHR*, the operands must match a supported
For *OpCooperativeMatrixMulAddKHR*, the operands must match a supported

Comment on lines +2477 to +2480
* {CL_DEVICE_COOPERATIVE_MATRIX_COMPONENT_TYPE_SINT8_KHR_anchor} corresponds to SPIR-V `OpTypeInt 8 0`.
* {CL_DEVICE_COOPERATIVE_MATRIX_COMPONENT_TYPE_SINT16_KHR_anchor} corresponds to SPIR-V `OpTypeInt 16 0`.
* {CL_DEVICE_COOPERATIVE_MATRIX_COMPONENT_TYPE_SINT32_KHR_anchor} corresponds to SPIR-V `OpTypeInt 32 0`.
* {CL_DEVICE_COOPERATIVE_MATRIX_COMPONENT_TYPE_SINT64_KHR_anchor} corresponds to SPIR-V `OpTypeInt 64 0`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be OpTypeInt xx 1 to indicate signedness?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not, as the OpenCL SPIR-V Environment Specification Section 4 "Validation Rules" says:

For all OpTypeInt integer type-declaration instructions:

  • Signedness must be 0, indicating no signedness semantics.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks!

Comment thread xml/cl.xml
Comment on lines +8105 to +8107
<type name="cl_device_cooperative_matrix_variant_khr"/>
<type name="cl_device_cooperative_matrix_component_type_khr"/>
<type name="cl_device_cooperative_matrix_info_khr"/>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...variant_khr should go after ...component_type_khr as the former (a struct type) contains elements of the latter, and the wrong order breaks the generated header. I'd suggest putting the struct type after the cl_uint types, like so:

Suggested change
<type name="cl_device_cooperative_matrix_variant_khr"/>
<type name="cl_device_cooperative_matrix_component_type_khr"/>
<type name="cl_device_cooperative_matrix_info_khr"/>
<type name="cl_device_cooperative_matrix_component_type_khr"/>
<type name="cl_device_cooperative_matrix_info_khr"/>
<type name="cl_device_cooperative_matrix_variant_khr"/>

@anun333

anun333 commented Oct 15, 2026

Copy link
Copy Markdown

Feedback on floating-point accumulation, for this draft and the OpenCL C companion (#1627).

SPV_KHR_cooperative_matrix leaves two things to the client API: "The order of the operations is implementation-dependent" and "The internal precision of floating-point operations is defined by the client API". The environment section in this PR does not define the internal precision, and #1627 adds to section 7.4 that coop_mat_mulAdd "performs its operations in an implementation-dependent order and internal precision". So for a floating-point variant an application cannot tell what accuracy it gets, or whether two devices, or two runs on one device, give the same result. (Integer variants are already fully defined.)

A small suggestion while cl_device_cooperative_matrix_variant_khr is still a draft: add a bitfield member that reports the accumulation behaviour of each variant, for example:

  • CL_DEVICE_COOPERATIVE_MATRIX_ACCUMULATION_EXACT_KHR: each element of the result is the exact value of the dot product plus the element of C, rounded once to the result type (round to nearest, ties to even). Results are then the same on every device that reports it, whatever order the hardware uses internally.
  • CL_DEVICE_COOPERATIVE_MATRIX_ACCUMULATION_REPEATABLE_KHR: the same inputs give the same result on this device on every execution.

A device that sets neither flag loses nothing, and the query needs no hardware. Libraries can use it to choose an algorithm or state an error bound, and applications that need repeatable results (regression tests, safety cases) can check for it instead of finding out by testing. RISC-V's matrix extension draft (Zvvm, in architecture review) takes a similar line: each implementation must disclose its accumulation parameters. Adding a member now avoids a separate query once the struct has shipped.

AI tool usage: written with the assistance of Claude Code; the quotations are from the specifications and drafts linked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants