fix(cuda): compile-time gate for green_ctx, device-aware PTX target for pre-Turing GPUs#12
Open
EliasVahlberg wants to merge 3 commits into
Open
fix(cuda): compile-time gate for green_ctx, device-aware PTX target for pre-Turing GPUs#12EliasVahlberg wants to merge 3 commits into
EliasVahlberg wants to merge 3 commits into
Conversation
green_ctx uses CUgreenCtx and its FFI functions, which cudarc only exposes behind the cuda-12040 through cuda-13020 feature flags (matching NVIDIA's own introduction of the Green Context API in CUDA 12.4). The hopper module declared 'pub mod green_ctx;' unconditionally, so ringkernel-cuda failed to compile at all with --features cuda on any CUDA Toolkit below 12.4 - regardless of GPU architecture. Confirmed on CUDA 12.0.140 (GTX 1080, Pascal): cargo build -p ringkernel --example basic_hello_kernel --features cuda failed with 14 unresolved-symbol errors before this fix. The module's own doc comment already claimed Hopper features 'gracefully fall back on older architectures' - true at runtime via check_hopper_support(), but there was no matching compile-time gate. This adds one, matching cudarc's own feature list exactly, plus a [lints.rust] check-cfg allowance so the transitively-inherited cudarc feature names don't trigger unexpected_cfgs warnings. Added 5 new #[ignore]-gated hardware tests (pascal_compat_tests module, run with --features cuda -- --ignored pascal_compat) confirming check_hopper_support, supports_cluster_launch, is_dsmem_available, and query_max_cluster_size all degrade gracefully (Err/false/1, not a panic) on real Pascal hardware now that the module compiles in. Also confirms supports_cooperative_groups (CC 6.0+ floor) correctly stays available, verifying the gating distinguishes per-feature floors rather than treating 'not Hopper' as 'nothing works'. All 5 passed on a GTX 1080. Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
RING_KERNEL_PTX_TEMPLATE was hardcoded to '.target sm_75', with a comment claiming this was safe as a 'lowest common denominator' because 'PTX is forward-compatible, so sm_75 PTX runs on sm_89/sm_90/sm_100 and newer'. That forward-compatibility claim is correct in direction, but the lowest-common-denominator framing is backwards: PTX built for sm_75 can run on sm_75 and newer, never older. This produced CUDA_ERROR_INVALID_PTX on a GTX 1080 (Pascal, sm_61) and would affect any pre-Turing GPU (Pascal, Maxwell), despite the runtime's own supports_cooperative_groups() gating correctly reporting CC 6.0+ as the real floor for that feature. Replaces the single hardcoded const with ring_kernel_ptx_template_for(major, minor), which generates the PTX template with a .version/.target matched to the actual device's compute capability (queried via CudaDevice::compute_capability() at both call sites in runtime.rs - the cooperative/persistent-simulation launch path and the standard launch path). The old const is kept, marked #[deprecated], for API compatibility with any external callers that don't have a device handle available. Verified: cargo run -p ringkernel --example basic_hello_kernel --features cuda now completes the full persistent-actor lifecycle (launch, activate, status check, suspend, resume, terminate) on a GTX 1080, where it previously failed at kernel launch with CUDA_ERROR_INVALID_PTX. Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
ringkernel-cuda's own crate-level doc comment (rendered on docs.rs) and README.md (rendered on crates.io) both claimed 'Compute Capability 7.0+' as a blanket requirement. This contradicts the top-level project README's own 'Feature to minimum compute capability' table, which correctly states the core persistent-actor/cooperative-groups path floors at CC 6.0 (Pascal) - the same feature these two docs are describing. Before the two fixes earlier in this branch, CC 7.0+ was arguably an accurate practical statement (Pascal genuinely couldn't build or run this crate). After those fixes, it's simply incorrect - the core path now builds and runs correctly on Pascal (verified: this branch's basic_hello_kernel example completes its full lifecycle on a GTX 1080). Corrects both docs to CC 6.0+, with a pointer to the top-level README's detailed table for per-feature floors above that (Hopper-only features like clusters/DSMEM/TMA/Green Contexts still correctly require CC 9.0+ - unchanged, not part of this fix). Co-Authored-By: Claude Sonnet 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two real bugs blocked
ringkernel-cuda(--features cuda) from building/running on any pre-Turing GPU or CUDA Toolkit below 12.4, found and fixed while getting the persistent-actor path running on a GTX 1080 (Pascal, sm_61). A third commit aligns crate-level docs with the fix.Both bugs are toolkit-version / PTX-direction bugs, not GPU-architecture-specific in cause — they'd affect anyone on an older CUDA Toolkit or pre-Turing card, not just Pascal.
What's broken today (on
main)hopper::green_ctxis declared unconditionally (pub mod green_ctx;inhopper/mod.rs), but its FFI calls needcudarc'scuda-12040+ feature (matching NVIDIA's own introduction ofCUgreenCtxin CUDA 12.4). On CUDA Toolkit 12.0.140,cudarc's own version-detection correctly does not enable that feature — soringkernel-cudafails to compile at all with--features cuda, on any GPU. The module's doc comment already claims Hopper features "gracefully fall back on older architectures" — true at runtime viacheck_hopper_support(), but there was no matching compile-time gate.RING_KERNEL_PTX_TEMPLATEis hardcoded to.target sm_75, with a comment claiming this is a safe "lowest common denominator" because "PTX is forward-compatible, so sm_75 PTX runs on sm_89/sm_90/sm_100 and newer." The forward-compat direction claim is correct, but PTX built forsm_75cannot run on anything older — so this producesCUDA_ERROR_INVALID_PTXon any pre-Turing card. This also directly contradicts the "Feature to minimum compute capability" table in the top-level README, which correctly statesgrid.sync()/persistent kernels float at CC 6.0 (Pascal).What this PR does
86dd850: cfg-gatesgreen_ctxbehind the exact samecuda-12040..cuda-13020feature setcudarcitself uses, so the crate builds cleanly on any CUDA Toolkit. Adds a[lints.rust]check-cfg allowance for the transitively-inherited feature names. Adds 5#[ignore]-gated hardware tests (hopper::pascal_compat_tests, run with--features cuda -- --ignored pascal_compat) confirmingcheck_hopper_support,supports_cluster_launch,is_dsmem_available, andquery_max_cluster_sizeall degrade gracefully (not panic) on real Pascal hardware, and thatsupports_cooperative_groups(its own, lower CC 6.0 floor) correctly stays available — i.e. the gating distinguishes per-feature floors rather than treating "not Hopper" as "nothing works."7aa2e0e: replaces the single hardcodedRING_KERNEL_PTX_TEMPLATEconst withring_kernel_ptx_template_for(major, minor), generating the PTX with a.version/.targetmatched to the actual device's compute capability (queried viaCudaDevice::compute_capability()at both call sites inruntime.rs). The old const is kept, marked#[deprecated], for compatibility with any external callers.fa08c18:ringkernel-cuda's crate-level doc comment andREADME.mdboth claimed a blanketCompute Capability 7.0+floor, which — after the above fixes — no longer matches the top-level README's own table for this exact feature. Corrected to CC 6.0+, with a pointer to the detailed table for higher per-feature floors (Hopper-only features are unaffected/unchanged).Verification
All on a real GTX 1080 (Pascal, sm_61), CUDA Toolkit 12.0.140:
completes the full persistent-actor lifecycle (launch, activate, status check, suspend, resume, terminate) where it previously failed — first at compile time (14 unresolved-symbol errors from
green_ctx), then at kernel launch (CUDA_ERROR_INVALID_PTX) once bug 1 was worked around.5 passed, 0 failed — confirms Hopper-only feature checks degrade gracefully rather than panicking, and that lower-floor features (cooperative groups) remain correctly available.
Also checked against the project's CI gates locally:
cargo fmt --all -- --check— cleancargo clippy -p ringkernel-cuda -p ringkernel -p ringkernel-core --lib --bins -- -D warnings— cleancargo doc -p ringkernel-cuda --no-deps— clean, no broken intra-doc linksclippy/MSRV-1.75 checks hit two pre-existing issues (ringkernel-accnet/ringkernel-procintGUI crates failing anf32: From<f64>lint under a newer rustc; acpufeaturestransitive dep needingedition2024under Cargo 1.75) — confirmed both exist identically on unmodifiedmain, unrelated to this PR.Notes