Integrating Husky for Pre-Commit Hooks & Exploring Language Bindings

[kanhaiya-dct]

Hello team! 👋

Following up on the recent PR feedback regarding premerge CI checks and high-level architectural improvements, I wanted to open this discussion to align on two potential enhancements for the project's roadmap:

1. Using Husky (or pre-commit) for Local Developer Workflows

Currently, developers might push code that fails the premerge/required checks (e.g., formatting issues with clang-format or failing unit tests). Catching these errors in the CI pipeline wastes time and resources.

Proposal:
We can integrate Husky (or the Python-based pre-commit framework) to run these checks locally before a commit is even created.

Benefits:

• Instant Feedback: clang-format or ruff (for Python) runs automatically on git commit, ensuring no bad formatting makes it to the PR.
• Save CI Time: Reduces the load on GitHub Actions by catching basic compilation or linting errors locally.
• Consistency: Enforces the same coding standards across all contributors' machines effortlessly.

2. Exploring New Language Bindings at Runtime

As mentioned in the previous PR, there is an opportunity to expand the ecosystem of TensorRT-Model-Connect. While the core is written in highly optimized C++ with Python bindings, creating runtime bindings for other popular languages could be highly beneficial.

Potential Languages:

• JavaScript / Node.js: Enables web developers to integrate TRTMC directly into their backend services.
• Rust: Offers safety and performance for systems engineers who want to interface with TRTMC without dealing with raw C/C++.
• Go: Great for microservices and cloud-native deployments.

Benefits to the Community:

• Opens up TRTMC to a massive pool of developers who don't primarily write C++ or Python.
• Different bindings can be dynamically loaded at runtime based on the user's environment.

I would love to hear the maintainers' and the community's thoughts on this! Are there specific languages you feel we should prioritize? And would pre-commit (Python) or Husky (JS) fit better into the current repository structure for git hooks?

Looking forward to the discussion!

[yifeif-nv]

Yeah, so it looks like there's actually two threads here.

The first one is that we definitely need to have a pre-commit hook to help the community do the basic sanity check before submitting their PR. This is an immediate actionable item, and I will create an issue to get this addressed soon.

[kanhaiya-dct]

That sounds great! I'm really glad to hear that the pre-commit hook idea is moving forward. It will definitely save everyone a lot of time and keep the codebase clean. Let me know if you'd like me to help implement it or review the setup once the issue is created.

[yifeif-nv]

Yes, the issue is already created, and I'm currently working on a PR to enable that and updating the contribution guide. #968

[yifeif-nv]

Thanks for raising this point. This definitely helps a lot for the community contributors.

[kanhaiya-dct]

Yes, the issue is already created, and I'm currently working on a PR to enable that and updating the contribution guide. #968

Sure

[kanhaiya-dct]

Yes, the issue is already created, and I'm currently working on a PR to enable that and updating the contribution guide. #968

Actually, you set up both a pre-commit hook and a pre-push hook.

In the pre-commit hook, all formatting checks are verified.
In the pre-push hook, you run the build/formatter checks so that the CLI works properly without any failures

[yifeif-nv]

And for language binding, I think this is also a very interesting direction. One of the benefits that TRT and Model-Connect provides is that we can provide embedded inference in other applications.

Are there any recommendations or other repos that has such interesting cases? Any pointer will be appriciated

[yifeif-nv]

Yeah, definitely sounds like an interesting direction. Let me pull in more people to brainstorm together. And yes, some kind of prototype is definitely welcome.

We just need to make sure this is actually scalable across all the models, because at the moment, all the model families have slightly different APIs. TRT-Model-Connect is designed in a way to be fully isolated between each model, so when we're using the AI to generate all the code, the error does not cascade.

[kanhaiya-dct]

Yeah, definitely sounds like an interesting direction. Let me pull in more people to brainstorm together. And yes, some kind of prototype is definitely welcome.

We just need to make sure this is actually scalable across all the models, because at the moment, all the model families have slightly different APIs. TRT-Model-Connect is designed in a way to be fully isolated between each model, so when we're using the AI to generate all the code, the error does not cascade.

That completely makes sense! Scalability and isolation are crucial here.

To prevent a maintenance nightmare where bindings have to be updated for every single model family, we shouldn't bind directly to the internal C++ classes of each model. Instead, we can build the language bindings strictly on top of the existing C ABI (src/cabi/api/trtmc_c.cpp).

Since C is the universal interface, tools like Rust's bindgen or Node.js N-API / node-addon-api can automatically consume the trtmc_c headers.

This gives us the best of both worlds:

1. Model Isolation is preserved: TRT-Model-Connect continues to isolate models internally.
2. Unified Binding Surface: The Rust/JS layer only ever interacts with trtmc_c.cpp. If a new model is added, the language bindings don't even need to know about it, as long as the C-ABI handles the routing.

If this architectural approach sounds aligned with the team's vision, I’d love to try putting together a small prototype using Rust (via cxx/bindgen) or Node.js targeting this C ABI. Let me know which language the team feels would be the most impactful starting point!

[yifeif-nv]

Yeah, looks like a lot of the AI applications are probably using some kind of web server, so I would say Node.js is a better fit. And yes, if you have time to do a quick prototype, I'm happy to take a look and discuss it.

[kanhaiya-dct]

Yeah, looks like a lot of the AI applications are probably using some kind of web server, so I would say Node.js is a better fit. And yes, if you have time to do a quick prototype, I'm happy to take a look and discuss it.

Thanks for the feedback! I completely agree—given how heavily AI applications rely on web services right now, Node.js bindings make a lot of sense as the first priority.

I'll start putting together a quick prototype for the Node.js integration to demonstrate how the bindings could work at runtime. I'll share it here once I have a basic proof-of-concept ready so we can discuss the design and architecture further.

Thanks again for the support!

[kanhaiya-dct]

Yeah, looks like a lot of the AI applications are probably using some kind of web server, so I would say Node.js is a better fit. And yes, if you have time to do a quick prototype, I'm happy to take a look and discuss it.

Hi, @yifeif-nv

I’ve completed the prototype. Could you please review and verify it when you get a chance?

PR: #1031
Issue: #1030

Thank you!