Repository navigation
Conversation
`clfunction` passed a closure to `GPUCompiler.cached_lookup` to link the kernel on a cache miss. The closure's type includes the kernel's function and argument types, so every new kernel compiled it again, together with the `methodinstance` call inlined into it. Use a callable struct holding these types as values instead, which is compiled once. This saves about 7 ms of host compilation for every new kernel (PoCL, Julia 1.13).
The first `show` of a `CLArray` compiled for about 400 ms, and on the REPL's `TTYTerminal` for about 950 ms, while the first garbage collection that frees an array compiled `free` for 30 ms. The workload doesn't use a device, so it can't do either; precompile them explicitly, displaying arrays in USM, the preferred memory back-end where it is supported. With PoCL and Julia 1.13, the first display of a `CLVector` takes 34 ms instead of 419 ms on an `IOBuffer`, and 46 ms instead of 969 ms on a `TTYTerminal`. The image grows from 9.6 to 13.1 MB, without a measurable change in load time.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #599 +/- ##
==========================================
+ Coverage 90.71% 90.75% +0.04%
==========================================
Files 18 18
Lines 1799 1807 +8
==========================================
+ Hits 1632 1640 +8
Misses 167 167 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two first-use latency fixes, found while working on the same in Metal.jl.
Displaying a
CLArrayfor the first time compiled for about 400 ms, and on the REPL'sTTYTerminalfor about 950 ms, while the first garbage collection that frees an array compiledfreefor 30 ms. The workload doesn't use a device, so these are now precompiled explicitly, displaying arrays in USM (the preferred memory back-end where it is supported). With PoCL and Julia 1.13, the first display of aCLVectortakes 34 instead of 419 ms on anIOBufferand 46 instead of 969 ms on aTTYTerminal. The image grows from 9.6 to 13.1 MB, without a measurable change in load time.clfunctionpassed a closure toGPUCompiler.cached_lookupto link a kernel on a cache miss. Its type includes the kernel's function and argument types, so every new kernel compiled it again; a callable struct holding these as values is compiled once, saving about 7 ms of host compilation per new kernel.