[keshavv27]

Description

• Remove NvTensorRTRTXExecutionProvider source files/ references in operator tests/macros from the ORT repo-
• For perftest, remove cudart dependency to enable use of CPU build with TRT RTX plugin EP.
• AppendExecutionProvider with legacy EP reports deprecation error with the new plugin EP path

Motivation and Context

https://github.com/NVIDIA/TensorRT-RTX-EP-ABI plugin EP is the preferred path for TRT RTX EP.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Removes the legacy provider-bridge NvTensorRTRTX EP and shifts perftest support to externally registered NVIDIA plugin EPs.

Changes:

• Removes legacy EP implementation, build flags, bindings, packaging, and tests.
• Adds plugin NVIDIA device detection and external sync-stream handling to perftest.
• Removes perftest’s CUDA runtime dependency for CPU-only plugin builds.

Reviewed changes

Copilot reviewed 74 out of 76 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tools/ci_build/gen_def.py	Removes legacy provider exports.
tools/ci_build/build.py	Removes NvTensorRTRTX build/package handling.
tools/ci_build/build_args.py	Removes legacy build arguments.
setup.py	Removes TRT RTX wheel packaging.
onnxruntime/test/util/include/default_providers.h	Removes default provider declaration.
onnxruntime/test/util/default_providers.cc	Removes default provider factory.
onnxruntime/test/unittest_util/base_tester.cc	Removes legacy EP test routing.
onnxruntime/test/shared_lib/test_session_options.cc	Removes provider-interface test.
onnxruntime/test/python/onnxruntime_test_python_nv_tensorrt_rtx_ep_tests.py	Deletes Python EP tests.
onnxruntime/test/providers/tensorrt/tensorrt_basic_test.cc	Removes NvRTX-specific context test.
onnxruntime/test/providers/nv_tensorrt_rtx/test_nv_trt_rtx_ep_util.h	Deletes test utilities.
onnxruntime/test/providers/nv_tensorrt_rtx/nv_options_test.cc	Deletes option tests.
onnxruntime/test/providers/nv_tensorrt_rtx/nv_ep_context_test.cc	Deletes context-model tests.
onnxruntime/test/providers/nv_tensorrt_rtx/nv_basic_test.cc	Deletes provider tests.
onnxruntime/test/providers/nv_tensorrt_rtx/nv_basic_ort_interop_test.cc	Deletes interop tests.
onnxruntime/test/providers/cpu/nn/pool_op_test.cc	Removes legacy EP exclusions.
onnxruntime/test/providers/cpu/nn/deform_conv_op_test.cc	Removes legacy EP exclusions.
onnxruntime/test/perftest/utils.h	Adds optional plugin sync stream.
onnxruntime/test/perftest/ort_test_session.h	Stores environment and plugin stream.
onnxruntime/test/perftest/ort_test_session.cc	Adds NVIDIA plugin I/O binding support.
onnxruntime/test/perftest/common_utils.cc	Passes streams through plugin options.
onnxruntime/test/perftest/command_args_parser.cc	Removes legacy provider CLI option.
onnxruntime/test/common/cuda_op_test_utils.cc	Removes legacy CUDA gating.
onnxruntime/test/autoep/library/example_plugin_ep/ep_custom_op.h	Generalizes plugin documentation.
onnxruntime/python/onnxruntime_pybind_state.cc	Removes legacy Python registration.
onnxruntime/python/onnxruntime_pybind_state_common.h	Removes provider includes/interfaces.
onnxruntime/python/onnxruntime_pybind_schema.cc	Removes provider schema factory.
onnxruntime/python/onnxruntime_inference_collection.py	Removes legacy fallback handling.
onnxruntime/core/session/provider_registration.cc	Removes legacy provider registration.
onnxruntime/core/session/provider_bridge_ort.cc	Removes provider-bridge integration.
onnxruntime/core/session/inference_session.cc	Removes provider-specific fusion enablement.
onnxruntime/core/providers/webgpu/webgpu_provider_factory.cc	Removes obsolete provider reference.
onnxruntime/core/providers/shared_library/provider_api.h	Removes legacy provider identifier.
onnxruntime/core/providers/provider_factory_creators.h	Removes legacy factory include.
onnxruntime/core/providers/nv_tensorrt_rtx/version_script.lds	Deletes export script.
onnxruntime/core/providers/nv_tensorrt_rtx/symbols.txt	Deletes symbol metadata.
onnxruntime/core/providers/nv_tensorrt_rtx/symbols.def	Deletes Windows exports.
onnxruntime/core/providers/nv_tensorrt_rtx/onnx_ctx_model_helper.h	Deletes context helper API.
onnxruntime/core/providers/nv_tensorrt_rtx/onnx_ctx_model_helper.cc	Deletes context helper implementation.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_scoped_context.h	Deletes CUDA context helper.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_provider_factory.h	Deletes provider interface.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_provider_factory_creator.h	Deletes factory creator.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_platform_utils.h	Deletes platform utilities.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_includes.h	Deletes TensorRT include wrapper.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_file_utils.h	Deletes file utilities.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_utils.h	Deletes provider utilities.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_info.h	Deletes provider configuration.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_info.cc	Deletes option parsing.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_helper.cc	Deletes graph helpers.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.h	Deletes custom-op API.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc	Deletes custom-op implementation.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_data_transfer.h	Deletes data-transfer API.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_data_transfer.cc	Deletes CUDA transfers.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_cuda_call.cc	Deletes CUDA error handling.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_allocator.h	Deletes allocator API.
onnxruntime/core/providers/nv_tensorrt_rtx/nv_allocator.cc	Deletes allocator implementation.
onnxruntime/core/providers/nv_tensorrt_rtx/exported_symbols.lst	Deletes Apple exports.
onnxruntime/core/providers/get_execution_providers.cc	Removes provider availability entry.
java/build.gradle	Removes legacy build definition.
include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h	Removes obsolete provider note.
include/onnxruntime/core/session/onnxruntime_c_api.h	Removes legacy public option type.
include/onnxruntime/core/providers/nv_tensorrt_rtx/nv_provider_options.h	Deletes public provider options.
include/onnxruntime/core/graph/constants.h	Removes provider constant.
docs/CUDA_cuDNN_Optional_Design.md	Removes obsolete design references.
cmake/onnxruntime_unittests.cmake	Removes legacy test/link wiring.
cmake/onnxruntime_python.cmake	Removes Python package copying.
cmake/onnxruntime_providers.cmake	Removes provider CMake inclusion.
cmake/onnxruntime_providers_nv.cmake	Deletes provider build definition.
cmake/onnxruntime_framework.cmake	Removes legacy CUDA dependency condition.
cmake/CMakeLists.txt	Removes legacy CMake options and flags.

Suppressed comments (2)

onnxruntime/test/perftest/ort_test_session.cc:130

• When -g is used with registered plugin EPs but none of the selected EPs is an NVIDIA GPU, ext_stream_ remains empty, yet this passes its address. AppendPluginExecutionProviders then calls GetHandle(), and SyncStream_GetHandle dereferences the null underlying OrtSyncStream, causing a crash. Pass the pointer only when stream creation succeeded.
  onnxruntime/test/perftest/ort_test_session.cc:125
• This chooses the first NVIDIA GPU device in the entire environment, not the device(s) selected later by --select_ep_devices/--filter_ep_devices. On a multi-GPU EP, selecting GPU 1 can therefore create a GPU 0 stream and pass it to the GPU 1 EP instance (and multiple NVIDIA plugin EPs can receive a stream owned by a different EP), leading to invalid cross-device/EP stream usage. Create streams from the exact selected OrtEpDevice set and either maintain one per EP/device or reject unsupported multi-device combinations.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Plugin stream selection lacks the required NVIDIA-device guard, and the Go C API documentation retains an invalid removed-provider reference.

Review effort: Balanced
Findings: 2 · 1

Open (3)

• Restrict user stream options to NVIDIA GPU plugins · New
• This enables CUDA allocation based on any NVIDIA device exposed by a selected EP name, without…
• Update provider documentation to remove stale namespace references · New

Resolved since last review (2)

• The Go binding uses its vendored go/onnxruntime/onnxruntime_c_api.h, which still declares…
• The new perftest path explicitly enables CUDA I/O binding for NVIDIA plugin EPs, but this updated…

[keshavv27]

adrastogi Could you help in reviewing this? thanks

[adrastogi]

devang-ml Scott McKay (@skottmckay) is there any compatibility guidance or process from the ORT team as it relates to deprecation / removal of in-tree EPs? The NVIDIA folks want to remove their provider bridge EP code, and I wanted to double-check if there's any lifecycle process to follow (e.g., any broad comms that need to go out to community mailing lists, doc updates, etc.) or if it's ok to just remove the code.

[Copilot]

Copilot review overview

🟡 Changes recommended

The promised NVIDIA plugin sync-stream configuration is not implemented in the perftest path.

Review effort: Balanced
Findings: 2 · 1

Open (3)

• Configure external compute stream before appending TensorRT RTX EP · New
• This enables CUDA allocation based on any NVIDIA device exposed by a selected EP name, without…
• Document automatic fusion enablement for the TensorRT RTX plugin · New

Resolved since last review (2)

• Restrict user stream options to NVIDIA GPU plugins
• Update provider documentation to remove stale namespace references

[keshavv27]

adrastogi Could you help in reviewing this again and merging if the compatibility changes are fine?

[Tianlei Wu (tianleiwu)]

Overall this is a clean removal of the legacy provider-bridge NvTensorRTRTXExecutionProvider and nice addition of autoep graphics interop tests.

A few follow-up items noted inline (for plugin EP -g I/O binding in onnxruntime_perf_test, Python InferenceSession provider fallback/validation when NvTensorRTRTXExecutionProvider is registered as a plugin EP, and unused vulkan_headers in cmake/deps.txt), plus one cleanup in an untouched header:

• In onnxruntime/core/providers/cuda/shared_inc/cuda_call.h (lines 19 and 30), CU_CALL and CU_CALL_THROW were added in #26948 for the legacy NV EP and relied on the CudaErrString<CUresult> specialization and CudaCall<CUresult, ...> explicit template instantiations in the deleted onnxruntime/core/providers/nv_tensorrt_rtx/nv_cuda_call.cc. Neither onnxruntime/core/providers/cuda/cuda_call.cc nor onnxruntime/core/providers/tensorrt/cuda_call.cc instantiates CudaCall<CUresult, ...>, leaving those macros without a backing definition.

[Chi Lo (chilo-ms)]

Thanks for the update. Following two issues might need to address as well.

1. [P2] Plugin I/O binding still lacks a guaranteed completion barrier — ort_test_session.cc:137–138. The previous finding is unchanged:  -g  enables WebGPU device-output binding, but WebGPU’s run-end flush only submits work and its missing  OrtEp::Sync  makes  SynchronizeOutputs()  a no-op. Without profiling or another completion barrier, reported latency can exclude GPU execution. Restrict this path to plugins with guaranteed synchronization, or wait for GPU completion before recording  total_timing .

2. [P3] Restore the compatibility typedef in the Go header too — go\onnxruntime\onnxruntime_c_api.h:308–310. The update restores  OrtNvTensorRtRtxProviderOptions  in the canonical header for source compatibility, but the vendored Go copy still deletes it. Since  go\onnxruntime\cshim.h  includes that copy, consumers naming this type remain source-broken. Restore the declaration with the same deprecation note.

[keshavv27]

Thanks for the update. Following two issues might need to address as well.

1. [P2] Plugin I/O binding still lacks a guaranteed completion barrier — ort_test_session.cc:137–138. The previous finding is unchanged:  -g  enables WebGPU device-output binding, but WebGPU’s run-end flush only submits work and its missing  OrtEp::Sync  makes  SynchronizeOutputs()  a no-op. Without profiling or another completion barrier, reported latency can exclude GPU execution. Restrict this path to plugins with guaranteed synchronization, or wait for GPU completion before recording  total_timing .
2. [P3] Restore the compatibility typedef in the Go header too — go\onnxruntime\onnxruntime_c_api.h:308–310. The update restores  OrtNvTensorRtRtxProviderOptions  in the canonical header for source compatibility, but the vendored Go copy still deletes it. Since  go\onnxruntime\cshim.h  includes that copy, consumers naming this type remain source-broken. Restore the declaration with the same deprecation note.

Chi Lo (@chilo-ms) Addressed the 2nd issue in latest commit, thank you.

For the first issue, The missing completion barrier is in the WebGPU EP and doesn't depend on -g. The plugin -g path intentionally does not set disable_synchronize_execution_providers, so Run() still calls OnRunEnd(sync_stream=true) and stream cleanup with sync, exactly as the non--g path does. SynchronizeOutputs() is an extra wait on top, through OrtEp::Sync when the EP implements it.

WebGPU skips both: OnRunEnd ignores sync_stream and only flushes, its sync stream's OnSessionRunEnd is a no-op, and it doesn't implement OrtEp::Sync. So WebGPU timings stop at submission whether or not -g is used. The NV TensorRT RTX plugin EP doesn't implement OrtEp::Sync but synchronizes its stream at run end when sync_stream is set, so timings include device execution with or without -g.

The fix belongs in the WebGPU EP: honour sync_stream in OnRunEnd (e.g. via FlushAndWaitChecked), and/or implement OrtEp::Sync

[Tianlei Wu (tianleiwu)]

Thanks for the cleanup. The removal is thorough: at this head git grep for the removed symbols only finds the intentional leftovers (the NvTensorRtRtxProviderOptions typedef, kNvTensorRTRTXExecutionProvider, and the DQ → MatMulNBits EP-name check), the branch merges cleanly with current main, and the earlier review threads are addressed.

One item to fix before merge: Android CI / AndroidBinarySizeCheckJob_MinimalBaseline is the only failing check, and it is caused by the new deprecation branch in provider_registration.cc (numbers and a suggested fix inline). Compiling that branch out of minimal builds, together with its test, should turn it green and makes this PR a net size reduction for that build.

On the WebGPU -g timing discussion above (not a new finding): I agree the fix belongs in the WebGPU EP (honour sync_stream in OnRunEnd and/or implement OrtEp::Sync) and don't see it as a blocker for this PR. One nuance for that thread: without -g, the output copy to the host (BufferManager::Download, which blocks on MapAsync) is already an implicit completion barrier, so it is the -g IO-binding path that removes it. Until the WebGPU change lands, -g timings for that EP are submit-only, so it is worth tracking as a follow-up.

[Tianlei Wu (tianleiwu)]

This new branch is what pushes Android CI / AndroidBinarySizeCheckJob_MinimalBaseline over its gate (the only failing check on this head).

• The job reports Sections total=1607810 against BINARY_SIZE_THRESHOLD_IN_BYTES=1607680 (android.yml L82), i.e. 130 B over. Recent main runs of the same job measure 1,607,426–1,607,458 B, so this PR adds roughly +350–380 B.

• provider_registration.cc is compiled into minimal builds (only provider_bridge_ort.cc is excluded), and the provider-bridge NV EP never existed there. The ~270 B of message literals plus the std::string concatenation and two strcmps are pure growth for that build.

• A translation-unit A/B build of this file (x86-64 g++, -Os -fno-exceptions -fno-rtti -DORT_MINIMAL_BUILD -DORT_NO_EXCEPTIONS, flags taken from the minimal-build onnxruntime_session target; approximate, the arm64 CI number is authoritative), allocated size vs the merge base:

  Variant	Δ size
  this PR as-is	+428 B
  deprecation branch compiled out	−85 B
  short static message, no std::string	+147 B

  So the removal on its own is a net shrink, and this branch is what tips the gate.

Suggested fix: compile the branch, and the test that exercises it, out of minimal builds. Those builds then report Unknown provider name 'NvTensorRtRtx' …, which is accurate since they never carried this EP.

#if !defined(ORT_MINIMAL_BUILD)
  // The provider-bridge NvTensorRtRtx EP is deprecated. Report the plugin EP flow with the deprecation error
  if (strcmp(provider_name, "NvTensorRtRtx") == 0 || strcmp(provider_name, kNvTensorRTRTXExecutionProvider) == 0) {
    // ... existing CreateStatus(...) ...
  }
#endif  // !defined(ORT_MINIMAL_BUILD)

and the same #if !defined(ORT_MINIMAL_BUILD) / #endif around CApiTest.session_options_append_removed_nv_tensorrt_rtx_ep in test_session_options.cc.

Alternatives, if you want the message in minimal builds: shrink it to a short static literal (about +147 B, which should fit but leaves little margin against main's ~220 B of headroom), or bump the threshold in android.yml (precedent: #33164).

[keshavv27]

Added minimal build guard to the deprecation branch and its related test, please check the pipelines. Thanks!