Skip to content

Dev/cuda/optimize - #42

Merged
Ra4ster merged 10 commits into
mainfrom
dev/CUDA/Optimize
Oct 31, 2026
Merged

Dev/cuda/optimize#42
Ra4ster merged 10 commits into
mainfrom
dev/CUDA/Optimize

Conversation

@Ra4ster

@Ra4ster Ra4ster commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator

Description

Optimized CUDA implementations and logged results.

Closes #

Type of change

  • Bug fix
  • New feature
  • Performance improvement
  • Documentation
  • Build/tooling
  • Breaking change (existing C++ or Python API behavior changes)

Testing

  • python build.py Release builds and DeepityTests passes
  • python build.py Debug builds and DeepityTests passes (required for anything correctness-sensitive)
  • pyright passes (if you touched bindings/, deepity/, examples/, or experiments/)

Platform(s) and compiler(s) tested:

  • Fedora Linux 44 w/ LLVM
  • RHEL on OSC Ascend w/ g++

Performance impact

~6s improvement from the previous MNIST run (69s).

Checklist

  • I've read CONTRIBUTING.md, including the portability rules and numerical gotchas relevant to this change.
  • I've added a CHANGELOG.md entry under [Unreleased] (for any user-facing change).
  • I've updated relevant docs (README, Doxygen comments) if behavior or the public API changed.
  • New/changed code follows the project's .clang-format style.

Notes for reviewers

Most of the additions are done to FullPCN. The other classes are not up to date with it quite yet.

Ra4ster and others added 10 commits October 30, 2026 17:51
@Ra4ster
- examples/mnist_cuda.py — Deepity SimplePCN with device="cuda", arch…
e924ff2 
…itecture/LR/stabilizer identical to mnist.py

- examples/test_ffnn_deep_cuda.py — PyTorch backprop control on torch.device("cuda"), raises clearly if no GPU is visible
- experiments/benchmark_jpc_mnist_cuda.py — jpc/JAX forced onto JAX_PLATFORMS=cuda, raises if jax.default_backend() isn't "gpu"
@Ra4ster
missing explicit template instantiation
fc88a02
@Ra4ster
Added results.'
c111024
@Ra4ster
Added comparison results
44cfce0
@Ra4ster
Commit of the OSC dir.
352c0c2
@Ra4ster
Merge branch 'dev/CUDA/Optimize' of github.com:Deepity-HQ/Deepity int…
8a121d5 
…o dev/CUDA/Optimize
@Ra4ster
** CUDA API Summary (cuda_api_sum):
59e036b 
 Time (%)  Total Time (ns)  Num Calls    Avg (ns)       Med (ns)      Min (ns)     Max (ns)    StdDev (ns)                   Name
 --------  ---------------  ---------  -------------  -------------  -----------  -----------  -----------  ---------------------------------------
     64.6    1,733,369,154      5,043      343,717.9        2,364.0        1,533   21,174,677    986,693.2  cudaStreamSynchronize
      9.9      265,595,694          1  265,595,694.0  265,595,694.0  265,595,694  265,595,694          0.0  cudaStreamCreate
      5.6      151,519,151     27,589        5,492.0        3,547.0          721   27,352,064    179,824.8  cudaLaunchKernel
      4.6      124,249,021        720      172,568.1      142,718.0      139,513    9,506,382    423,826.5  cudaGraphLaunch_v10000
      4.2      113,630,995      3,087       36,809.5       38,011.0        6,672   12,003,204    280,527.5  cudaMemcpy
      2.2       58,845,742         11    5,349,612.9    3,255,056.0       60,023   28,152,810  7,908,946.6  cuLibraryLoadData
      1.8       47,348,518      4,583       10,331.3        8,506.0        1,112       68,229      4,458.5  cudaMemcpyAsync
      1.6       42,611,275      5,561        7,662.5        4,269.0          541    8,114,567    128,139.3  cudaMemsetAsync
      1.5       39,721,496     50,346          789.0          110.0           80   26,267,055    120,600.6  cuStreamGetCaptureInfo_v2
      1.1       30,558,749      6,633        4,607.1        3,777.0        1,052    3,922,260     48,172.9  cudaLaunchKernelExC_v11060
      1.0       27,816,205      6,636        4,191.7        3,627.0          581    2,108,202     25,929.5  cuLaunchKernel
      0.4        9,791,097      8,731        1,121.4          972.0          771       42,499        780.3  cudaEventRecord
      0.3        8,464,074          1    8,464,074.0    8,464,074.0    8,464,074    8,464,074          0.0  cudaGraphInstantiate_v12000
      0.2        6,338,474      6,636          955.2          381.0          271    3,373,028     41,418.2  cuKernelGetFunction
      0.2        5,607,683         63       89,010.8          962.0          721    5,425,314    683,196.0  cudaMallocAsync_v11020
      0.2        5,561,098         23      241,786.9          661.0          240    4,308,085    898,096.3  cuLibraryGetKernel
      0.2        4,420,065     27,589          160.2          140.0           91       37,961        309.3  cuKernelGetName
      0.1        3,003,082          1    3,003,082.0    3,003,082.0    3,003,082    3,003,082          0.0  cudaGraphExecDestroy_v10000
      0.1        1,946,278         13      149,713.7       34,014.0        3,256      912,775    254,915.5  cudaFree
      0.1        1,356,499          1    1,356,499.0    1,356,499.0    1,356,499    1,356,499          0.0  cudaHostAlloc
      0.0          984,048         11       89,458.9      133,691.0        4,118      210,776     81,757.7  cudaMalloc
      0.0          653,269      1,337          488.6          180.0           90      368,873     10,084.7  cuGetProcAddress_v2
      0.0          593,364          1      593,364.0      593,364.0      593,364      593,364          0.0  cudaFreeHost
      0.0          399,080          1      399,080.0      399,080.0      399,080      399,080          0.0  cudaGraphDestroy_v10000
      0.0          290,766         19       15,303.5          360.0          310      277,692     63,546.0  cudaEventCreateWithFlags
      0.0          280,959         16       17,559.9        3,462.0        2,926      196,469     48,227.8  cudaStreamCreateWithFlags
      0.0          111,790         17        6,575.9        6,272.0        3,857       13,225      2,053.6  cudaStreamDestroy
      0.0           94,769         63        1,504.3          842.0          651        4,378        966.4  cudaFreeAsync_v11020
      0.0           61,045          1       61,045.0       61,045.0       61,045       61,045          0.0  cudaStreamBeginCapture_v10000
      0.0           25,668          1       25,668.0       25,668.0       25,668       25,668          0.0  cudaStreamEndCapture_v10000
      0.0           18,767          4        4,691.8        3,271.5        1,664       10,560      3,985.5  cudaDeviceSynchronize
      0.0           14,478          5        2,895.6        3,016.0        2,394        3,337        390.2  cuInit
      0.0            9,920         19          522.1          371.0          271        1,433        316.3  cudaEventDestroy
      0.0            8,135          2        4,067.5        4,067.5        3,156        4,979      1,289.1  cudaEventQuery
      0.0            3,797          4          949.3          165.5          130        3,336      1,591.3  cuModuleGetLoadingMode
      0.0            1,524          3          508.0          341.0          321          862        306.7  cudaGetDriverEntryPointByVersion_v12050
      0.0              321          1          321.0          321.0          321          321          0.0  cudaGetDeviceProperties_v12000

Fixes:
- include/deepity/networks/SimplePCNetwork.h — added kPredictGraphSlot, predictGraphCaptured, capturedPredictInferenceSteps.
- src/networks/SimplePCNetwork.cpp — PredictWithProjection now captures/replays its own graph on GPU, mirroring FullPCNetwork::Predict's proven pattern (no UpdateWeights(), no mu-cache invalidation needed — that was specific to the post-weight-update energy read in the training path, not applicable here).
- examples/mnist_cuda.py — the per-epoch accuracy check was manually reimplementing reset_state/clamp_input/project_forward/settling-loop in Python instead of just calling net.predict_with_projection(...), so it never could have benefited from graph capture regardless of the C++ fix. Switched it to call predict_with_projection directly. Also removed a redundant reset_state()/clamp_input() pair immediately before the final-eval's existing predict_with_projection call — dead work, since that function already does both internally.
@Ra4ster
Merge branch 'dev/CUDA/Optimize' of https://github.com/Deepity-HQ/Dee…
39789d2 
…pity into dev/CUDA/Optimize
@Ra4ster
- logs/mnist_cuda_graph_capture_investigation.txt — the full narrativ…
6ef67a8 
…e: baseline nsys profile → root cause (missing graph slot for PredictWithProjection) → the fix → re-profile confirming it → the energy-computation diagnostic and its result → conclusion on the remaining PyTorch gap being architectural, not a bug.

- README.md — the existing CUDA-columns bullet now links to that log for anyone who wants the full trail, without bloating the table itself.
- logs/mnist_cuda_run.txt / the comparison table — already updated last turn to the current 66.8s number.
@Ra4ster
- SimplePCNetwork: Predict() (plain, zero-init) now graph-captures on…
a956dde 
… its own slot (kPlainPredictGraphSlot=2), on top of PredictWithProjection()'s existing fix.

- ConvPCNetwork, DiscriminativePCNetwork, SimpleConvPCNetwork: both Predict() and PredictWithProjection() now graph-capture, each on its own dedicated slot.
- FullConvPCNetwork, GaussSeidelPCNetwork: their one Predict() method (no separate "with projection" variant exists for these two) now graph-captures, mirroring FullPCNetwork::Predict's existing EPC/cross-entropy handling where applicable.
- Every edit also switched the inner settling loop from the default CalculateState()/CalculateState(true) to CalculateState(false) — necessary, not cosmetic: computing energy mid-loop forces a synchronous scalar readback, which is illegal during CUDA stream capture and would have made EndGraphCapture() fail. This bug was latent before (these methods never ran on GPU at all until now).
@Ra4ster Ra4ster self-assigned this Oct 31, 2026
@Ra4ster Ra4ster added the cpp-core The core C++ engine label Oct 31, 2026
@Ra4ster
Ra4ster merged commit 44efaa8 into main Oct 31, 2026
10 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Main Feature Board Oct 31, 2026
@Ra4ster
Ra4ster deleted the dev/CUDA/Optimize branch October 31, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@Ra4ster Ra4ster

Labels

cpp-core The core C++ engine

Projects

Main Feature Board
Status: Done

Milestone

v0.1.0-alpha: ImageNet Pipeline & Docs

Development

Successfully merging this pull request may close these issues.

1 participant

@Ra4ster