Issues

• Plan routed coding-model providers and gateway readiness for the internal pilot

  nebari-dev/llm-serving-pack#200 · Filter by author trentoliphant opened on Sep 30, 2026

• Bound and validate PassthroughModel spec.provider.credentialSecretName

  priority: low 🌱type: enhancement 💅🏼
  nebari-dev/llm-serving-pack#199 · Filter by author dcmcand opened on Sep 28, 2026

• Add user-configurable service-level concurrency control for self-hosted models

  nebari-dev/llm-serving-pack#198 · Filter by author ns-mkusper opened on Sep 27, 2026
  

• Add opt-in native CPU KV-cache offload with explicit memory budgeting

  nebari-dev/llm-serving-pack#196 · Filter by author ns-mkusper opened on Sep 27, 2026

• Enforce per-client token rate limits on served models

  nebari-dev/llm-serving-pack#195 · Filter by author ns-mkusper opened on Sep 24, 2026

• Delete a model's routes and policies when the model is deleted

  nebari-dev/llm-serving-pack#194 · Filter by author ns-mkusper opened on Sep 23, 2026

• Advertise per-model context window on /v1/models

  needs: discussion 💬priority: low 🌱type: enhancement 💅🏼
  nebari-dev/llm-serving-pack#193 · Filter by author ns-mkusper opened on Sep 22, 2026

• Report each model's hosting origin on /v1/models

  nebari-dev/llm-serving-pack#191 · Filter by author ns-mkusper opened on Sep 22, 2026

• Add nodeSelector, tolerations and imagePullSecrets for the EPP under spec.advanced.inferencePool

  needs: discussion 💬
  nebari-dev/llm-serving-pack#176 · Filter by author tylerpotts opened on Aug 24, 2026

• Move operator to aigateway.envoyproxy.io/v1beta1 and update examples to Envoy Gateway v1.8.3 / AI Gateway v1.1.0

  type: deprecation ⚰️
  nebari-dev/llm-serving-pack#175 · Filter by author tylerpotts opened on Aug 24, 2026
  2

• Default EPP image ghcr.io/llm-d/llm-d-inference-scheduler:v0.8.0 no longer exists

  type: bug 🐛
  nebari-dev/llm-serving-pack#174 · Filter by author tylerpotts opened on Aug 24, 2026

• Add Playwright for E2E and Accessibility Testing

  priority: medium ⚡UI/UX/DX

  MFilter by Size: M

  MediumFilter by Priority: Medium
  Task

  nebari-dev/llm-serving-pack#152 · Filter by author jbouder opened on Aug 5, 2026
  1

• Emit privacy-safe OpenTelemetry traces from AI Gateway and vLLM

  nebari-dev/llm-serving-pack#148 · Filter by author ns-mkusper opened on Jul 27, 2026

• Internal JWT provider does not bind an audience by default

  priority: high 🔥securitytype: enhancement 💅🏼
  nebari-dev/llm-serving-pack#136 · Filter by author dcmcand opened on Jul 14, 2026

• Pin model-downloader init container to an immutable digest

  priority: medium ⚡securitytype: enhancement 💅🏼
  nebari-dev/llm-serving-pack#135 · Filter by author dcmcand opened on Jul 14, 2026

• LLMModel create/edit grants arbitrary container execution with Secret access

  priority: high 🔥securitytype: enhancement 💅🏼
  nebari-dev/llm-serving-pack#134 · Filter by author dcmcand opened on Jul 14, 2026

• Served models on tainted GPU nodes can't get Longhorn storage

  nebari-dev/llm-serving-pack#124 · Filter by author dcmcand opened on Jun 30, 2026

• vLLM container crashloops on llm-d-cuda:v0.7.0 (no command set, NVIDIA wrapper entrypoint)

  nebari-dev/llm-serving-pack#123 · Filter by author dcmcand opened on Jun 30, 2026

• Per-model API-key scoping broken when one user has keys for multiple models (colliding client IDs)

  nebari-dev/llm-serving-pack#122 · Filter by author dcmcand opened on Jun 30, 2026

• EPP Role RBAC: operator ClusterRole missing inference.networking.x-k8s.io grants (served models fail on Helm install)

  nebari-dev/llm-serving-pack#121 · Filter by author dcmcand opened on Jun 30, 2026

• PassthroughModel reports Ready even when the upstream credential Secret is missing/misnamed

  priority: medium ⚡type: bug 🐛
  nebari-dev/llm-serving-pack#110 · Filter by author dcmcand opened on Jun 23, 2026
  1

• Converge LLMModel reconciler onto the PassthroughModel gateway-apply failure handling

  priority: medium ⚡type: maintenance 🛠
  nebari-dev/llm-serving-pack#109 · Filter by author dcmcand opened on Jun 23, 2026

• Design & Add a Dedicated Landing-Page Icon for the LLM Model Serving Pack

  good first issuepriority: low 🌱type: enhancement 💅🏼UI/UX/DX

  XSFilter by Size: XS

  LowFilter by Priority: Low
  Feature

  nebari-dev/llm-serving-pack#106 · Filter by author jbouder opened on Jun 18, 2026

• [DOC] - Create placeholder documentation

  area: documentation 📖priority: medium ⚡type: maintenance 🛠
  nebari-dev/llm-serving-pack#104 · Filter by author andrewfulton9 opened on Jun 17, 2026

• Support isolated LLM Gateway for large request buffer policies

  priority: medium ⚡type: enhancement 💅🏼

  MediumFilter by Priority: Medium
  Feature

  nebari-dev/llm-serving-pack#87 · Filter by author ns-mkusper opened on May 18, 2026