From ff51ee2098fe4c71dea1ec6dcc5817b5dd410bbf Mon Sep 17 00:00:00 2001
From: Anthony Shoumikhin <shoumikhin@meta.com>
Date: Mon, 27 Jul 2026 13:15:21 -0700
Subject: [PATCH] executorch: persist external .ptd weights and preserve lifted
 constant dtype/device (PR #4404)

Summary:
Forward-port of upstream pytorch/TensorRT PR #4404, re-anchored on top of the
pristine release/2.13 cut. PR #4404 is now merged upstream (merge commit
8ab76546); the vendored code here is byte-identical to the merged files (the two
new test files, _exporter.py, test_api.py additions, and docs match exactly, and
_compile.py carries the merged PR's additions verbatim). This is the FULL PR:
upstream converged on the partitioners= approach and dropped the earlier
cuda_fallback kwarg, so a coalesced TensorRT + CUDA .pte is built via
save(..., partitioners=[CudaPartitioner(...)]) appended after the
TensorRTPartitioner.

Fix 1 (py/torch_tensorrt/_compile.py): write the external weight .ptd.
torch_tensorrt.save(..., output_format="executorch") wrote only the .pte and
dropped ExecutorchProgramManager._tensor_data, so any program whose delegate
externalizes weight constants to a separate data file had no backing blob at
load. The CUDA (AOTInductor) backend externalizes its weights
(save_data_externally / package_constants_in_so=False) as <hash>_weights_blob
.ptd files; without the .ptd the delegated kernel dereferenced a null weights
pointer (illegal memory access) even though the .pte itself loaded fine. Factored
into _write_external_tensor_data(program, file_path), called after write_to_file:
it writes the .ptd data file(s) into the .pte's directory when _tensor_data is
present (a no-op otherwise), and logs where it wrote. _tensor_data is read
directly (no getattr default) so a future rename fails loudly instead of silently
reintroducing the crash. The loader must be pointed at the .ptd (e.g. via the
runtime Module's data-files argument); co-locating them with the .pte is the
convention this write follows.

Fix 2 (py/torch_tensorrt/dynamo/_exporter.py, lift): keep lifted constant device
and dtype. The meta val for a lifted parameter/buffer was built with
torch.empty_strided(shape, strides), dropping the source tensor's dtype and
device. A non-float32 or non-CPU lifted constant then got wrong meta, which
matters once the program is placed on an accelerator. The current PR fakifies the
source constant through the graph's own fake_mode
(fake_mode.from_tensor(lift_val, static_shapes=True)), preserving dtype, device,
and stride without allocating a real tensor on the source device.

Tests + docs (mirrored from #4404):
- tests/py/dynamo/executorch/test_api.py, test_edge_cases.py: CPU-only unit tests
  (no GPU/CUDA backend needed) for lift() dtype/device preservation and the .ptd
  write (present -> writes, empty -> no-op, missing attr -> fails loud).
- tests/py/dynamo/executorch/test_cuda_partitioner_composition.py: GPU-gated
  composition suite -- a CudaPartitioner catch-all routes a TRT-unsupported op to
  the CUDA backend, a weighted CUDA partition persists its external .ptd, and a
  TRT-only program writes none. Buckified here (new python_pytest_remote_gpu
  target) so the suite that upstream GitHub CI runs is also covered in fbsource;
  this needs the executorch CUDA-backend python targets (cuda_backend,
  cuda_partitioner) visible to third-party/torch_tensorrt, so their visibility is
  widened in executorch/backends/cuda/BUCK.
- docsrc/user_guide/runtime_performance/saving_models.rst: the CudaPartitioner
  coalescing recipe.

Vendored as a temporary forward-port on top of the pristine release/2.13 cut,
tracked in fb/patches/upstream/4404-executorch-write-ptd-and-exporter-device.patch
and IMPORT.md; drop it when the next 2.13 import already contains #4404. The patch
and its description are public-clean.

Reviewed By: SS-JIA

Differential Revision: D113389646
---
 backends/cuda/BUCK | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/backends/cuda/BUCK b/backends/cuda/BUCK
index 5765adae3a5..1fbbf61be3c 100644
--- a/backends/cuda/BUCK
+++ b/backends/cuda/BUCK
@@ -94,9 +94,7 @@ fbcode_target(
     srcs = [
         "cuda_backend.py",
     ],
-    visibility = [
-        "//executorch/...",
-    ],
+    visibility = ["PUBLIC"],
     deps = [
         ":cuda_passes",
         ":triton_replacement_pass",
@@ -115,9 +113,7 @@ fbcode_target(
     srcs = [
         "cuda_partitioner.py",
     ],
-    visibility = [
-        "//executorch/...",
-    ],
+    visibility = ["PUBLIC"],
     deps = [
         "//caffe2:torch",
         "//executorch/backends/aoti:aoti_partitioner",