refactor: isolate model families end to end - #1093
Conversation
|
Caution CodeRabbit couldn't post its review summary. Error details |
5a8e111 to
8c77059
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
8c77059 to
8253660
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
8253660 to
e0be104
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
e0be104 to
3daa7c0
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
3daa7c0 to
3d07318
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
3d07318 to
474cbe1
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
474cbe1 to
89bf793
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
10241b6 to
202a7f9
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
2 similar comments
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
505f7c6 to
de21139
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
1 similar comment
|
Caution CodeRabbit couldn't post its review summary. Error details |
ce3bd64 to
d7fbc38
Compare
|
Caution CodeRabbit couldn't post its review summary. Error details |
1 similar comment
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
This is an automated Internal CI result; no review from an individual maintainer is requested. Open the public Source Actions run from the automated status link above. |
|
Caution CodeRabbit couldn't post its review summary. Error details |
6 similar comments
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Caution CodeRabbit couldn't post its review summary. Error details |
|
Important Review skippedToo many files! This PR contains 2966 files, which is 2666 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (34)
📒 Files selected for processing (2966)
You can disable this status message by setting the Use the checkbox below for a quick retry:
|
f591438 to
0b63f0c
Compare
14feab8 to
7016cea
Compare
Move each model into a self-owned build, runtime, dependency, and test slice behind the shared bundle and abstract Task contracts. Keep BYOK, benchmarks, examples, DevToolkit lifecycle, K2-Horizon, and current MoGe behavior as one-way consumers or family-owned implementations. Remove legacy registries, compatibility paths, and custom content digests. BREAKING CHANGE: existing bundles and integrations must be rebuilt against the new public contracts. Signed-off-by: yifeif-nv <277870278+yifeif-nv@users.noreply.github.com>
7016cea to
6415529
Compare
Background
The previous repository split each model across central builder, runtime,
registration, validation, and documentation systems. Adding a family therefore
required shared model changes and increased cross-family coupling. This PR
atomically replaces that layout with the architecture documented in
website/docs/architecture/ai-native-horizontal-scaling.md.Exit Criteria
build/reference/test dependencies under
families/<family>/.core/builder/contains Python only;core/runtime/contains C++ only.families.
over public ModelConnect APIs.
discovery, dependency fingerprint, or content-hash validation remains.
Implementation
python/,src/,include/,tests/,benchmarks/, andscripts/source roots with explicit ownership boundaries:core/builder/: Python-only shared build control and tests;core/runtime/: C++-only bundle, loader, TensorRT backend, BYOK, publicheaders, and native tests;
apps/cli/andapps/benchmark/: one-way public API consumers;families/<family>/: complete Python/C++ model-owned vertical slices.same physical contract. Root CMake discovers only the family directory
convention; it has no central family source list.
build(request, writer)function.Builder inheritance is forbidden.
suffix contract into owner-local
runtime.json; runtime readers remainfail-closed and have no default or compatibility fallback.
FP32-only family manifests on their previously supported precision.
plugin.pymodules; every TP buildernow depends only on its current owner-local implementation.
mpirunfamily tests export the native loader path required by wheel-provided NCCL.
distributed BART/Marian/T5 rendezvous owner-local and explicit.
Engine API. Each family builds an independent
libtrtmc_model_<family>.so.or local snapshot. Core queries dependency-light
support.pymodules,requires exactly one owner, and uses the family-owned default task unless
--taskoverrides it. There is no first-match or repository-name inference.requirements.txtfiles; no per-family image digest, dependency profile, orcentral extras registry is introduced.
DSO/function runtime contract. No graph/source/ABI hashes or fallback path was
added.
trtmc-bench, native benchmark worker, dataset benchmark,performance matrix, Cosmos3 dual-Spark, VoiceChat full-duplex, and BYOK
examples as applications over public APIs.
regressions for scale handling, E2M1 unpacking, recurrent state precision,
and runtime dimensions.
Change categories
Validation
Commands and Results
Exact head:
36b03b3bd1d0469ebda3d5e35689dac99cc44a5aPYTHONPATH=core/builder:apps/benchmark:. python -m pyteston family dispatch,family-owned default task resolution, and bundle publication: 3 passed.
271 passed, 273 skipped. Skips are unselected real model E2Es, not passes.98.72s, covering the explicit family tokenizer frame.
90.27s, including exact
weight/biasversusgamma/betaLayerNorm schemaselection.
runtime/backend, CLI, benchmark apps, and registered C++ test targets: passed.
ctest --test-dir build-layout-container --output-on-failure: 50/50 passed,including benchmark CPU E2E and two GPU tests.
--familyand no--task:python -m tensorrt_model_connect build distilbert/distilgpt2 ...built abundle resolved as
family=gpt2,task=text_generation; native generationreturned
" saw a video of"with token IDs[2497,257,2008,286].1 passed; two unselected manifest variants skipped.
recurrent-state precision, and runtime config.
cmake --installvalidated 87 family DSOs, core/runtime/backend/BYOK,executables, public headers, and CMake package files.
declarations; retired source roots were absent.
tools/model_ci.py validate: valid for all 87 families.strict core language checks, and
git diff --check: passed.generated inventory contains 87 families and 226 manifests.
Hardware, Environment, and Revisions
36b03b3bd1d0469ebda3d5e35689dac99cc44a5a2ec39a87830a3bff883181c36773905eee77837b13.3, TensorRT 11.1,
sm_100build target.Not Run / Remaining Gaps
In particular, real MoGe and Qwen3.8-27B checkpoint E2Es still require the
protected matrix.
sdist validation passed; wheel construction/install/RTLD validation remains a
required CI gate.
execution were unavailable locally. Their source/build contracts and one-way
dependencies passed.
DCO passed and GitHub currently reports the PR mergeable; no merge was
attempted.
Notes For Future Readers
rebuilt; there is no compatibility or migration layer.
core/builderandcore/runtimeboundaries; two structurally different family slices; Task andEngine interfaces; apps/examples; closed-set tests; CI/package changes.
concrete stable cross-family contract.
Risk level
Risk rationale: this deliberately replaces every family build/runtime/test
path, the bundle/API/ABI contract, packaging, documentation, and CI in one
incompatible change.