Repository navigation
Explicit Runge-Kutta solvers: an opt-in scan_kind="unrolled", without the stage loop - #793
Open
charleswilmot wants to merge 1 commit into
Open
charleswilmot wants to merge 1 commit into
charleswilmot wants to merge 1 commit into
Conversation
On GPU, every check of a while loop's predicate is a device-to-host copy that the host waits for, and the stage loop of `AbstractRungeKutta.step` makes one per stage: 7 a Tsit5 step, 14 a Dopri8 step. With `scan_kind="unrolled"`, the stages of an explicit tableau are traced one after another instead. Each stage is kept as its own array, and the stages are stacked once, at the end, for the error estimate and the dense output: written into the buffer, as the stage loop does, they made the step 1.6x slower on GPU, since XLA rebuilds the whole buffer at every write once the indices are static. An FSAL method's first stage is a `lax.cond`, so that no step evaluates the vector field once more. Opt-in, since the vector field is then traced once per stage, which costs compile time. Implicit tableaus are refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #792 .
What.
AbstractRungeKutta.stepevaluates its stages in aneqxi.while_loop, the stage loop. On a GPU,every check of a while loop's predicate is a host round trip: XLA copies the predicate to the host, and the
host waits for it. A Tsit5 step makes 7 of them, a Dopri8 step 14. This PR adds an opt-in for explicit tableaus:
Its step evaluates the stages one after another, with static stage indices, and no stage loop:
the dense output. Writing them into the
[stages, ...]buffer instead, as the stage loop does, made theunrolled step 1.6x slower on the GPU (a
lax.scanofstep, 6 states: 38.4 ms against 23.7 ms). Withstatic indices, XLA rebuilds the whole buffer at every write.
lax.cond. So no stepevaluates the vector field once more, and the profiler trace shows no host round trip for it.
scan_kind=None, keeps the stage loop. Nothing changes for anyone who does not ask for it.scan_kind="unrolled"raises, in__check_init__.DirectAdjointandForwardModereplace aNonescan kind, but not this one, andforward-mode autodiff goes straight through.
Results agree with the stage loop's to rounding, 4e-12 relative: the increments are summed in a different order.
An adaptive solve of a sensitive problem can therefore accept or reject a step differently and take a different
step sequence. Both results are then within the tolerance: in a batch of 1000 below, one solve in six did so, and
the largest error against a reference at rtol 1e-12 was the same for both, 2.2e-4.
Measured: batches of small systems. A small system belongs on a GPU only in a batch. Here it is the issue's
chain of 3 masses (6 states): Tsit5,
jax.vmapofdiffeqsolveover B initial states, 1000 constant steps.Times are in ms, for the whole batch:
B = 1 is a
jax.vmapover a batch of one. An unbatcheddiffeqsolvetakes 118.0 → 35.1 ms on the GPU, and0.31 → 0.18 ms on the CPU.
stages on both. From B = 1000 on, unrolled is 1.7–2.1x faster on the GPU. Today's GPU stage loop is slower than
the unrolled CPU until B = 10 000.
7.2x at 100 000.
PIDController(rtol=atol=1e-6)the picture is the same. The GPU beats the CPU from B = 1000 in bothvariants, and there unrolled is 1.6–1.8x faster on the GPU. On the CPU, unrolled is 2.8–3.1x faster for
B = 100–10 000.
Measured: host round trips per step, counted in a
jax.profilertrace:diffeqsolvelax.scanofstepThe one left in
diffeqsolveis its own loop over the steps.Measured: single solves of various vector fields. Tsit5 and Dopri8, with constant steps and with
PIDController; solves, andjax.gradthrough the defaultRecursiveCheckpointAdjoint. The speed-up ranges oversolver and stepping:
jax.gradjax.gradThe cost is compile time. The vector field is traced once per stage instead of once. Across the cases above,
compile time grows 0.8–3.2x.
benchmarks/compile_times.py(Dopri8, with a vector field built to be expensive totrace) gives 0.8–1.9x with
eqxi.noinline, and 4.3–13.5x without it. That cost, and single CPU solves up to17 % slower at mid sizes (the chain, 60 states: 1.51 → 1.77 ms), are why this is opt-in.
Tests (
test/test_solver.py, 22 new):scan_kind="unrolled"against the stage loop, for every explicit Runge-Kutta solver (Heun, Midpoint, Ralston,Bosh3, Tsit5, Dopri5, Dopri8). Each with constant steps and with adaptive steps across a jump, so that FSAL
re-evaluates its first stage partway through. Checked: the saved values, through dense output, and the gradients.
ForwardMode,DirectAdjoint,BacksolveAdjoint;vmap; an SDE (Heun on aMultiTerm).whilein its jaxpr; an implicit solver given"unrolled"raises.diffrax's whole suite passes, and passes again with every explicit Runge-Kutta step forced to
scan_kind="unrolled". The forcing is done by a local pytest plugin, which is not part of this PR.jax / jaxlib 0.12.0.1 (
jax[cuda12]), equinox 0.13.8, Python 3.14. NVIDIA RTX 4060 Laptop GPU, driver 595.99;CPU: Intel i5-12500H, pinned to its four performance cores. Medians of 11 to 14 calls, compile excluded.