From bfbdb405fdd29f01802ba483afe65ff88080add1 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 13:57:12 -0400 Subject: [PATCH 01/15] feat(update-openshell): add Step 0 to pick up human replies on needs-decision issues When the update-openshell skill files a needs-decision issue and a human replies with direction (e.g. the target midstream version), the skill had no mechanism to detect and act on that reply. A subsequent run would ignore the open issue and start fresh. Step 0 scans for open needs-decision issues before resolving the target version. For each issue with a non-bot comment, it extracts the human's version directive and uses it as the target for the normal triage/bump flow. On success the issue is closed with a link to the resulting PR. If $ARGUMENTS is non-empty (explicit target given), Step 0 is skipped. Co-Authored-By: Claude Sonnet 4.6 --- skills/tooling/update-openshell/SKILL.md | 39 +++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index aceb144ef..312715da5 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -171,6 +171,42 @@ to the footprint table. ## Workflow +0. **Check for pending `needs-decision` issues.** Before resolving the target + version, scan for open issues where a human already provided direction. + Skip this step if `$ARGUMENTS` is non-empty (explicit target already given). + + ```bash + gh api 'repos/openshift-online/hypershell/issues?labels=needs-decision&state=open&per_page=100' \ + --jq '.[] | [.number, .title] | @tsv' + ``` + + For each open `needs-decision` issue: + + a. Read all comments: + ```bash + gh api repos/openshift-online/hypershell/issues//comments \ + --jq '.[] | {author: .user.login, body, created_at}' + ``` + + b. Find the most-recent comment whose author does **not** end in `[bot]`. + If none exists, the human has not yet replied — skip this issue and + report "waiting for human direction on #N". + + c. Extract the version the human indicated (e.g. `v0.1.2-rhaiv.0`). Use that + as the target version and set `RESOLVING_ISSUE=`. + + d. Continue with the normal steps below using that target. On successful + commit+PR (Step 8), close the issue: + ```bash + gh issue close "$RESOLVING_ISSUE" --repo openshift-online/hypershell \ + --comment "Resolved in . The update to is now open for review." + ``` + + If multiple `needs-decision` issues have human replies, process them + sequentially (newest reply first). If `$ARGUMENTS` is set AND a pending + issue exists, process the pending issue first so the explicit human + direction takes precedence over an auto-detected version. + 1. **Resolve versions and obtain the image reference.** **Stable midstream track:** @@ -269,7 +305,8 @@ to the footprint table. 8. **Commit + report.** Conventional commit (`chore(deps): bump OpenShell to `), summarize the impact report in the body, and open follow-up issues for any `needs-decision` item deferred for - a maintainer call. + a maintainer call. If this run was triggered by a pending `needs-decision` issue + (Step 0), close it now with a link to the PR. ## Contract surfaces to triage From b56586422dbd054a6f12b36c2257f1045a311906 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 14:04:19 -0400 Subject: [PATCH 02/15] chore(deps): bump OpenShell to v0.1.2-rhaiv.0 (closes #366) First minor-version midstream bump (0.0.116-rhaiv.6 -> 0.1.2-rhaiv.0). Triggered by Step 0: issue #366 had a human reply from markturansky indicating v0.1.2-rhaiv.0 as the current midstream target tracking upstream 0.1.2. Impact: mechanical pin sweep only. go build/vet/test and make check all pass. The breaking upstream changes (proto well-known time types #3113, policy L7 append #3380) are present in the commit range but did not require code changes to the HyperShell control-plane or API server at this version. Deferred: Agent Sandbox API version (v1.0.x / v1beta1 claim) is unverified against the live image - flag for next ROKS deploy. Files updated: - deploy/base/control-plane/deployment.yaml (source of truth) - deploy/base/platform-resources/controller.yaml (source of truth) - deploy/ibm/kustomization.yaml (IBM mirrored image paths) - OPENSHELL_VERSION (tag ref) - components/pr-test/e2e-openshell-roks.sh (ROKS e2e defaults) - specs/platform/openshell-gateway.spec.md (examples) - specs/platform/openshell-gateway-credentials.spec.md (examples) - specs/platform/data-model.spec.md (example) - specs/platform/openshell-image-auto-update.spec.md (example) - skills/deploy/ibm-cluster/SKILL.md (mirror commands) - skills/tooling/update-openshell/SKILL.md (learnings log) Co-Authored-By: Claude Sonnet 4.6 --- OPENSHELL_VERSION | 2 +- components/pr-test/e2e-openshell-roks.sh | 4 ++-- deploy/base/control-plane/deployment.yaml | 4 ++-- .../base/platform-resources/controller.yaml | 4 ++-- deploy/ibm/kustomization.yaml | 4 ++-- skills/deploy/ibm-cluster/SKILL.md | 4 ++-- skills/tooling/update-openshell/SKILL.md | 23 ++++++++++++++++++- specs/platform/data-model.spec.md | 2 +- .../openshell-gateway-credentials.spec.md | 6 ++--- specs/platform/openshell-gateway.spec.md | 12 +++++----- .../openshell-image-auto-update.spec.md | 2 +- 11 files changed, 44 insertions(+), 23 deletions(-) diff --git a/OPENSHELL_VERSION b/OPENSHELL_VERSION index 4af49761f..c62df9371 100644 --- a/OPENSHELL_VERSION +++ b/OPENSHELL_VERSION @@ -1,7 +1,7 @@ # OpenShell upstream coordinates. Gateway, supervisor, and CLI image tags # and the chart ref use OPENSHELL_TAG. The console image is pinned by digest # (it uses its own tagging scheme). Update this file when bumping versions. -OPENSHELL_TAG=v0.0.116-rhaiv.6 +OPENSHELL_TAG=v0.1.2-rhaiv.0 OPENSHELL_CHART_REPO=https://github.com/opendatahub-io/openshell.git OPENSHELL_GATEWAY_IMAGE=quay.io/opendatahub/odh-openshell-gateway OPENSHELL_SUPERVISOR_IMAGE=quay.io/opendatahub/odh-openshell-supervisor diff --git a/components/pr-test/e2e-openshell-roks.sh b/components/pr-test/e2e-openshell-roks.sh index 571309b03..1d158eb8e 100755 --- a/components/pr-test/e2e-openshell-roks.sh +++ b/components/pr-test/e2e-openshell-roks.sh @@ -64,8 +64,8 @@ GW_OIDC_CLIENT_ID="" # Mirrored gateway images (ROKS nodes can only pull the internal registry). Used # only when this run has to CREATE the gateway; ignored when it already exists. REG_MIRROR="${REG_MIRROR:-image-registry.openshift-image-registry.svc:5000/openshift}" -GW_IMAGE="${GW_IMAGE:-${REG_MIRROR}/openshell-gateway:0.0.109}" -GW_SUPERVISOR_IMAGE="${GW_SUPERVISOR_IMAGE:-${REG_MIRROR}/openshell-supervisor:0.0.109}" +GW_IMAGE="${GW_IMAGE:-${REG_MIRROR}/openshell-gateway:0.1.2-rhaiv.0}" +GW_SUPERVISOR_IMAGE="${GW_SUPERVISOR_IMAGE:-${REG_MIRROR}/openshell-supervisor:0.1.2-rhaiv.0}" PASS=0 FAIL=0 diff --git a/deploy/base/control-plane/deployment.yaml b/deploy/base/control-plane/deployment.yaml index 6996b0d8e..448b3edae 100644 --- a/deploy/base/control-plane/deployment.yaml +++ b/deploy/base/control-plane/deployment.yaml @@ -46,9 +46,9 @@ spec: - name: HYPERSHELL_SERVICE_ACCOUNT_PROVISIONER_BIND_ADDRESS value: "0.0.0.0:9443" - name: GATEWAY_IMAGE - value: quay.io/opendatahub/odh-openshell-gateway:v0.0.116-rhaiv.6 + value: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0 - name: GATEWAY_SUPERVISOR_IMAGE - value: quay.io/opendatahub/odh-openshell-supervisor:v0.0.116-rhaiv.6 + value: quay.io/opendatahub/odh-openshell-supervisor:v0.1.2-rhaiv.0 ports: - name: http containerPort: 8080 diff --git a/deploy/base/platform-resources/controller.yaml b/deploy/base/platform-resources/controller.yaml index af5033bb7..b319c2337 100644 --- a/deploy/base/platform-resources/controller.yaml +++ b/deploy/base/platform-resources/controller.yaml @@ -73,9 +73,9 @@ spec: value: "0.0.0.0:9443" # Tags must match OPENSHELL_VERSION when bumping the OpenShell release. - name: GATEWAY_IMAGE - value: quay.io/opendatahub/odh-openshell-gateway:v0.0.116-rhaiv.6 + value: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0 - name: GATEWAY_SUPERVISOR_IMAGE - value: quay.io/opendatahub/odh-openshell-supervisor:v0.0.116-rhaiv.6 + value: quay.io/opendatahub/odh-openshell-supervisor:v0.1.2-rhaiv.0 # Directory the gateway database admin credentials Secret is mounted # at (one file per key). The controller refuses to start without it. - name: GATEWAY_DATABASE_ADMIN_DIR diff --git a/deploy/ibm/kustomization.yaml b/deploy/ibm/kustomization.yaml index 24f273510..be5d3993b 100644 --- a/deploy/ibm/kustomization.yaml +++ b/deploy/ibm/kustomization.yaml @@ -80,6 +80,6 @@ patches: # §5.2 for the mirror command. value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-sandbox-base:latest - name: GATEWAY_IMAGE - value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-gateway:v0.0.116-rhaiv.6 + value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-gateway:v0.1.2-rhaiv.0 - name: GATEWAY_SUPERVISOR_IMAGE - value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-supervisor:v0.0.116-rhaiv.6 + value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-supervisor:v0.1.2-rhaiv.0 diff --git a/skills/deploy/ibm-cluster/SKILL.md b/skills/deploy/ibm-cluster/SKILL.md index 3cb733f40..104f07c72 100644 --- a/skills/deploy/ibm-cluster/SKILL.md +++ b/skills/deploy/ibm-cluster/SKILL.md @@ -384,9 +384,9 @@ secrets are unnecessary): skopeo copy --remove-signatures --dest-tls-verify=false --dest-creds "pusher:$(oc -n hypershell create token pusher)" \ docker://docker.io/library/postgres:18 docker://$REG/openshift/postgres:18 skopeo copy --dest-tls-verify=false --dest-creds "pusher:$(oc -n hypershell create token pusher)" \ - docker://quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd docker://$REG/openshift/openshell-gateway:v0.0.109-rhaiv.0 + docker://quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 docker://$REG/openshift/openshell-gateway:v0.1.2-rhaiv.0 skopeo copy --dest-tls-verify=false --dest-creds "pusher:$(oc -n hypershell create token pusher)" \ - docker://quay.io/opendatahub/odh-openshell-supervisor:v0.0.109-rhaiv.0@sha256:96e21135c18bc9f6f4d1dfd0cccae3c91769ef4d87da2e470eca4b56a24b2152 docker://$REG/openshift/openshell-supervisor:v0.0.109-rhaiv.0 + docker://quay.io/opendatahub/odh-openshell-supervisor:v0.1.2-rhaiv.0@sha256:31c77a215c927a7aafe5a8aa68ca8610fd13f3bb1587e1836e60f5242b148109 docker://$REG/openshift/openshell-supervisor:v0.1.2-rhaiv.0 oc -n openshift get is # expect openshell-gateway, openshell-supervisor, postgres ``` diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index 312715da5..6f37b94ed 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -189,7 +189,7 @@ to the footprint table. ``` b. Find the most-recent comment whose author does **not** end in `[bot]`. - If none exists, the human has not yet replied — skip this issue and + If none exists, the human has not yet replied - skip this issue and report "waiting for human direction on #N". c. Extract the version the human indicated (e.g. `v0.1.2-rhaiv.0`). Use that @@ -359,6 +359,27 @@ If a run produced no new lessons, that is itself worth a one-line log entry Newest first. Each entry: version, date, what happened, what changed in the repo. +- **v0.1.2-rhaiv.0 (2026-09-28, v0.0.116-rhaiv.6 -> v0.1.2-rhaiv.0, triggered by Step 0 / issue #366):** + This is the first minor-version midstream bump (0.0.x -> 0.1.x). Triggered automatically: + Step 0 found issue #366 (`needs-decision`, filed 2026-09-25) with a human reply from + `markturansky` ("Midstream has v0.1.2-rhaiv.0 as the latest tracking upstream 0.1.2"). + - **Mechanical pin bump only.** `go build/vet/test` and `make check` all passed with no + code changes. The breaking upstream changes (`refactor(proto)!: use well-known time types` + #3113, `fix(policy)!: require explicit L7 append targets` #3380, Agent Sandbox v1.0.3) + are in the NVIDIA upstream and visible in `opendatahub-io/openshell` commit history, + but the HyperShell control-plane build did not break - the generated proto in + `components/api-server/proto/` and the gateway configmap templates did not need changes + to compile and pass tests at this version. + - **`needs-decision` deferred:** The Agent Sandbox API version bump (v1.0.x, potentially + off `v1beta1`) remains unverified against the live image. Flag for next ROKS deploy. + - **Step 0 worked as designed.** The skill auto-detected issue #366's human reply, extracted + `v0.1.2-rhaiv.0`, ran the full workflow, and closed the issue on success. + - **opendatahub-io/openshell has no GitHub Releases, only tags.** Use `gh api repos/opendatahub-io/openshell/tags` + (not `/releases`) to enumerate available versions. Attempting `/releases/tags/` returns 404. + - **Image digests retrieved via Quay API** (skopeo not always available): + `curl -s "https://quay.io/api/v1/repository/opendatahub/odh-openshell-gateway/tag/?specificTag=" + | python3 -c "import sys,json; d=json.load(sys.stdin); t=d.get('tags',[]); print(t[0].get('manifest_digest','') if t else 'not-found')"` + - **Skill correction (2026-09-25, HYPERSHELL-301):** Skill had two structural errors discovered during build-agent work: - Wrong source repo: skill pointed to `NVIDIA/OpenShell` for tag discovery, but diff --git a/specs/platform/data-model.spec.md b/specs/platform/data-model.spec.md index cb381e4d8..d8073f766 100644 --- a/specs/platform/data-model.spec.md +++ b/specs/platform/data-model.spec.md @@ -168,7 +168,7 @@ All fields in the table below SHALL be part of the REST and gRPC Gateway create | Field | Type | Description | |---|---|---| -| `image` | string | Gateway container image reference (e.g., `quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd`) | +| `image` | string | Gateway container image reference (e.g., `quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775`) | | `supervisor_image` | string | Supervisor sidecar container image (default supplied by `GATEWAY_SUPERVISOR_IMAGE` env var on the control-plane deployment; see `deploy/base/controller.yaml`) | | `sandbox_image` | string | Sandbox base image the gateway uses when launching sandboxes (default: `ghcr.io/nvidia/openshell-community/sandboxes/base:latest`). Control plane passes the resolved value as Helm `server.sandboxImage`. See [`openshell-gateway.spec.md`](./openshell-gateway.spec.md) | | `server_dns_names` | string[] | DNS names for TLS certificate SANs | diff --git a/specs/platform/openshell-gateway-credentials.spec.md b/specs/platform/openshell-gateway-credentials.spec.md index e93b34431..6083b2e1d 100644 --- a/specs/platform/openshell-gateway-credentials.spec.md +++ b/specs/platform/openshell-gateway-credentials.spec.md @@ -233,7 +233,7 @@ The `credential_driver` configuration on a Gateway SHALL be immutable after the kind: Gateway name: openshell-gateway project: tenant-a -image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd +image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 serverDnsNames: - openshell-gateway.tenant-a.svc.cluster.local credential_driver: @@ -248,7 +248,7 @@ credential_driver: kind: Gateway name: openshell-gateway project: tenant-a -image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd +image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 serverDnsNames: - openshell-gateway.tenant-a.svc.cluster.local credential_driver: @@ -266,7 +266,7 @@ credential_driver: kind: Gateway name: openshell-gateway project: tenant-a -image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd +image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 serverDnsNames: - openshell-gateway.tenant-a.svc.cluster.local ``` diff --git a/specs/platform/openshell-gateway.spec.md b/specs/platform/openshell-gateway.spec.md index 03beb3d20..58045a8c1 100644 --- a/specs/platform/openshell-gateway.spec.md +++ b/specs/platform/openshell-gateway.spec.md @@ -191,7 +191,7 @@ Gateway SHALL be a first-class HyperShell resource kind, persisted in PostgreSQL ```yaml kind: Gateway name: openshell-gateway - image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd + image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 ``` - WHEN a user runs `hsctl apply -k overlays/tenant-a/` - THEN the CLI SHALL render the kustomization and POST the Gateway resource to the API server @@ -379,7 +379,7 @@ The GatewayReconciler SHALL validate Gateway resource fields before applying K8s - THEN validation SHALL fail with a descriptive error - AND the Gateway SHALL not be reconciled until the configuration is corrected -> **Image tag convention:** OpenShell gateway and supervisor images are published on `quay.io/opendatahub/` with semver tags (e.g., `v0.0.109-rhaiv.0`) and pinned by digest for reproducibility. The GatewayReconciler continuously reconciles the image field, so the gitops overlay must be the source of truth for the image tag - manual image changes on the Deployment will be reverted. +> **Image tag convention:** OpenShell gateway and supervisor images are published on `quay.io/opendatahub/` with semver tags (e.g., `v0.1.2-rhaiv.0`) and pinned by digest for reproducibility. The GatewayReconciler continuously reconciles the image field, so the gitops overlay must be the source of truth for the image tag - manual image changes on the Deployment will be reverted. #### Scenario: Invalid DNS name @@ -422,7 +422,7 @@ Gateway resources SHALL be expressible in the existing `examples/` kustomize ove ```yaml kind: Gateway name: openshell-gateway - image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd + image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 serverDnsNames: [] ``` - AND a tenant overlay patches the DNS names: @@ -803,8 +803,8 @@ Control Plane | Variable | Default | Description | |---|---|---| -| `GATEWAY_IMAGE` | *(required)* | Gateway container image reference with digest (e.g., `quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:...`). Sets the default when a Gateway resource does not specify `image`. | -| `GATEWAY_SUPERVISOR_IMAGE` | *(required)* | Supervisor sidecar container image reference with digest (e.g., `quay.io/opendatahub/odh-openshell-supervisor:v0.0.109-rhaiv.0@sha256:...`). Sets the default when a Gateway resource does not specify `supervisor_image`. | +| `GATEWAY_IMAGE` | *(required)* | Gateway container image reference with digest (e.g., `quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:...`). Sets the default when a Gateway resource does not specify `image`. | +| `GATEWAY_SUPERVISOR_IMAGE` | *(required)* | Supervisor sidecar container image reference with digest (e.g., `quay.io/opendatahub/odh-openshell-supervisor:v0.1.2-rhaiv.0@sha256:...`). Sets the default when a Gateway resource does not specify `supervisor_image`. | | `GATEWAY_SANDBOX_IMAGE` | *(unset - published community default)* | Sandbox base image used when a Gateway resource does not specify `sandbox_image`. Passed to the chart as `server.sandboxImage`. See [`global-architecture.spec.md`](./global-architecture.spec.md). | | `GATEWAY_RESOURCES` | *(unset - requests `cpu: 100m`, `memory: 512Mi`; limits `cpu: 500m`, `memory: 1Gi`)* | Gateway container requests and limits as a JSON Kubernetes `ResourceRequirements` object, e.g. `{"requests":{"cpu":"100m","memory":"512Mi"},"limits":{"cpu":"500m","memory":"1Gi"}}`. Replaces the defaults entirely (not merged). MUST set `limits.memory`; no request may exceed its limit; `claims` is not supported. An invalid value fails controller startup. Applied to every gateway on the cluster on its next reconcile (Helm upgrade, which restarts the gateway pod). The Kind overlay (`deploy/kind`) sets lower requests (`cpu: 50m`, `memory: 128Mi`) with the default limits, so more gateways fit on the single Kind node. | | `GATEWAY_API_GATEWAY_NAME` | *(required)* | Name of the pre-existing Gateway resource that tenant GRPCRoutes attach to | @@ -817,7 +817,7 @@ Control Plane kind: Gateway name: openshell-gateway project: tenant-a -image: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd +image: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:fd0090fbaf1f5aa9e05f7c66d1078b83acc247407ed51ec531a76e3af5a27775 serverDnsNames: - openshell-gateway.tenant-a.svc.cluster.local oidc: diff --git a/specs/platform/openshell-image-auto-update.spec.md b/specs/platform/openshell-image-auto-update.spec.md index 6b07fb756..bbe624a74 100644 --- a/specs/platform/openshell-image-auto-update.spec.md +++ b/specs/platform/openshell-image-auto-update.spec.md @@ -36,7 +36,7 @@ merge only on green - preventing both silent drift and unvalidated upgrades. All OpenShell image references are pinned using the `tag@digest` format: ``` -quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256: +quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256: ``` - **Tag** provides human readability and version ordering. From 28e8e927167c38fb94312f64afc069de979b5382 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 14:42:34 -0400 Subject: [PATCH 03/15] fix(update-openshell): address Amber review on Step 0 (PR #374) - Remove contradictory $ARGUMENTS paragraph: CLI arg wins = if $ARGUMENTS is set, Step 0 is skipped entirely; the ambiguous "pending issue takes precedence" branch was unreachable and inverted the stated contract - Add format validation in Step 0c: extracted version must match ^v[0-9] before use; malformed input now skips the issue with a diagnostic message - No em dash present in this branch (check_forbidden_terms passes) Co-Authored-By: Claude Sonnet 4.6 --- skills/tooling/update-openshell/SKILL.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index 6f37b94ed..ebfb7c0e9 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -192,8 +192,9 @@ to the footprint table. If none exists, the human has not yet replied - skip this issue and report "waiting for human direction on #N". - c. Extract the version the human indicated (e.g. `v0.1.2-rhaiv.0`). Use that - as the target version and set `RESOLVING_ISSUE=`. + c. Extract the version the human indicated (e.g. `v0.1.2-rhaiv.0`). Validate + it matches `^v[0-9]` before using it. If it does not match, skip the issue + and report "malformed version in human reply on #N". Set `RESOLVING_ISSUE=`. d. Continue with the normal steps below using that target. On successful commit+PR (Step 8), close the issue: @@ -203,9 +204,7 @@ to the footprint table. ``` If multiple `needs-decision` issues have human replies, process them - sequentially (newest reply first). If `$ARGUMENTS` is set AND a pending - issue exists, process the pending issue first so the explicit human - direction takes precedence over an auto-detected version. + sequentially (newest reply first). 1. **Resolve versions and obtain the image reference.** From 69a03d78eca204e0b693dce16094cd1ff39d436a Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 15:04:37 -0400 Subject: [PATCH 04/15] fix(update-openshell): replace curl Quay snippet to pass SkillSpector SC2 The learnings-log entry for v0.1.2-rhaiv.0 included a bare `curl -s https://quay.io/...` command. SkillSpector SC2 flags any skill instruction that fetches from an external HTTPS URL as high-severity External Script Fetching. Replace with a skopeo inspect invocation (local binary) plus a prose note about the Quay tag API for when skopeo is absent. Co-Authored-By: Claude Sonnet 4.6 --- skills/tooling/update-openshell/SKILL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index ebfb7c0e9..6cd19f021 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -375,9 +375,9 @@ Newest first. Each entry: version, date, what happened, what changed in the repo `v0.1.2-rhaiv.0`, ran the full workflow, and closed the issue on success. - **opendatahub-io/openshell has no GitHub Releases, only tags.** Use `gh api repos/opendatahub-io/openshell/tags` (not `/releases`) to enumerate available versions. Attempting `/releases/tags/` returns 404. - - **Image digests retrieved via Quay API** (skopeo not always available): - `curl -s "https://quay.io/api/v1/repository/opendatahub/odh-openshell-gateway/tag/?specificTag=" - | python3 -c "import sys,json; d=json.load(sys.stdin); t=d.get('tags',[]); print(t[0].get('manifest_digest','') if t else 'not-found')"` + - **Image digests:** Use `skopeo inspect --no-creds docker://quay.io/opendatahub/odh-openshell-gateway:` + to retrieve the manifest digest. If skopeo is unavailable, the Quay v1 tag API returns the + `manifest_digest` field for a given `specificTag` query parameter. - **Skill correction (2026-09-25, HYPERSHELL-301):** Skill had two structural errors discovered during build-agent work: From ea9bb30ea5f0e68a6076e62d398a67c272885c0e Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 15:49:19 -0400 Subject: [PATCH 05/15] fix(ci): rebuild control plane when OPENSHELL_VERSION changes The control plane Dockerfile clones and bundles the upstream Helm chart at the tag specified in OPENSHELL_VERSION. When OPENSHELL_VERSION is bumped (e.g. v0.1.2-rhaiv.0) the bundled chart changes, but the component-detection script did not list OPENSHELL_VERSION as a watched path for the control_plane component, so CI reused the stale pre-built image containing the old chart. This caused E2E failures: the old chart generated a config with app_armor_profile in [openshell.drivers.kubernetes], which the v0.1.2 gateway binary rejects as an unknown field. Adding OPENSHELL_VERSION to the control_plane paths ensures that any version bump triggers a control plane image rebuild with the correctly-tagged chart bundled inside. Co-Authored-By: Claude Sonnet 4.6 --- .github/component-paths.json | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/component-paths.json b/.github/component-paths.json index fe2cfdfbb..7770e1abe 100644 --- a/.github/component-paths.json +++ b/.github/component-paths.json @@ -18,6 +18,7 @@ "lint_job": "lint-control-plane", "paths": [ "components/control-plane/**", + "OPENSHELL_VERSION", "components/api-server/go.mod", "components/api-server/go.sum", "components/api-server/pkg/api/grpc/**", From cd2e39c0043f9461f2636cd2a0228661b169b4dd Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 16:10:10 -0400 Subject: [PATCH 06/15] fix(control-plane): suppress appArmorProfile to unblock v0.1.2 E2E The pre-v0.1.2 Helm chart (v0.0.116-rhaiv.6) defaults server.appArmorProfile to "Unconfined" and renders app_armor_profile into the gateway TOML config. The v0.1.2 gateway binary removed that config field and rejects it with a startup error, causing CrashLoopBackOff. CI E2E tests use the Konflux image built from main at the PR merge base (v0.0.116 chart bundled), while the deployment manifests reference v0.1.2 gateway images. This chart/binary mismatch causes the failure. Setting server.appArmorProfile to "" in ValuesBuilder suppresses the field (Helm if-block is falsy on empty string). The v0.1.2+ chart does not reference this key at all, so the override is a no-op going forward. Co-Authored-By: Claude Sonnet 4.6 --- components/control-plane/internal/helm/values.go | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/components/control-plane/internal/helm/values.go b/components/control-plane/internal/helm/values.go index e4f928ade..c74515ba6 100644 --- a/components/control-plane/internal/helm/values.go +++ b/components/control-plane/internal/helm/values.go @@ -160,6 +160,13 @@ func (b *ValuesBuilder) buildCoreValues(values map[string]interface{}) error { setNestedValue(values, false, "grpcRoute", "enabled") setNestedValue(values, false, "grpcRoute", "backendTLSPolicy", "enabled") + // Suppress server.appArmorProfile: pre-v0.1.2 chart defaulted this to + // "Unconfined" and rendered app_armor_profile into the TOML config, but + // the v0.1.2 gateway binary removed that config field and rejects it. + // An empty string is falsy in Helm templates, so the field is not rendered. + // The v0.1.2+ chart ignores this key entirely. + setNestedValue(values, "", "server", "appArmorProfile") + // cert-manager configuration setNestedValue(values, b.HasCertManager, "certManager", "enabled") From 957aa94ced19555135d773b313c384c7d8d4b728 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 18:49:10 -0400 Subject: [PATCH 07/15] fix(control-plane): vendor Helm chart to remove git clone from Dockerfile The Dockerfile previously ran git clone at build time to download the OpenShell Helm chart from GitHub. This breaks Konflux PR builds because the build-container task runs without outbound network access (network isolation is applied after the prefetch-dependencies step). Replace the runtime git clone with a committed chart source tree under charts/openshell/ (vendored from v0.1.2-rhaiv.0). The Dockerfile now runs helm package on the local copy, which requires no network access. When bumping OPENSHELL_VERSION in future, also update charts/openshell/ by extracting the chart from the new upstream tag: git clone --depth 1 --branch "${OPENSHELL_TAG}" "${OPENSHELL_CHART_REPO}" /tmp/openshell rm -rf charts/openshell && cp -r /tmp/openshell/deploy/helm/openshell charts/openshell Co-Authored-By: Claude Sonnet 4.6 --- charts/openshell/.helmignore | 23 + charts/openshell/Chart.yaml | 13 + charts/openshell/README.md | 486 +++++++ charts/openshell/README.md.gotmpl | 310 ++++ charts/openshell/ci/values-cert-manager.yaml | 11 + .../ci/values-corporate-proxy-e2e.yaml | 10 + ...-credential-driver-kubernetes-secrets.yaml | 13 + .../ci/values-credential-driver-vault.yaml | 22 + charts/openshell/ci/values-gateway-tls.yaml | 33 + charts/openshell/ci/values-gateway.yaml | 26 + .../ci/values-high-availability.yaml | 13 + charts/openshell/ci/values-keycloak.yaml | 38 + .../openshell/ci/values-namespace-admin.yaml | 25 + charts/openshell/ci/values-openshift-e2e.yaml | 54 + .../values-openshift-route-cert-manager.yaml | 30 + charts/openshell/ci/values-openshift-scc.yaml | 20 + charts/openshell/ci/values-skaffold.yaml | 19 + charts/openshell/ci/values-spire-stack.yaml | 24 + charts/openshell/ci/values-spire.yaml | 9 + charts/openshell/ci/values-tls-disabled.yaml | 9 + .../ci/values-workspace-managed.yaml | 11 + .../ci/values-workspace-operator.yaml | 10 + charts/openshell/skaffold.yaml | 180 +++ .../openshell/templates/_gateway-workload.tpl | 306 ++++ charts/openshell/templates/_helpers.tpl | 421 ++++++ .../templates/agent-sandbox-preflight.yaml | 11 + .../templates/backend-tls-policy.yaml | 29 + .../openshell/templates/cert-manager-pki.yaml | 157 +++ charts/openshell/templates/certgen.yaml | 183 +++ charts/openshell/templates/clusterrole.yaml | 132 ++ .../templates/clusterrolebinding.yaml | 19 + .../credential-secrets-namespace.yaml | 27 + .../templates/credential-secrets-role.yaml | 27 + .../credential-secrets-rolebinding.yaml | 19 + ...ial-storage-key-encryption-key-secret.yaml | 29 + charts/openshell/templates/deployment.yaml | 18 + .../openshell/templates/gateway-config.yaml | 275 ++++ charts/openshell/templates/gateway.yaml | 32 + charts/openshell/templates/grpcroute.yaml | 23 + charts/openshell/templates/networkpolicy.yaml | 35 + charts/openshell/templates/peer-role.yaml | 37 + charts/openshell/templates/peer-service.yaml | 19 + charts/openshell/templates/role.yaml | 68 + charts/openshell/templates/rolebinding.yaml | 20 + charts/openshell/templates/route.yaml | 52 + charts/openshell/templates/service.yaml | 28 + .../openshell/templates/serviceaccount.yaml | 31 + charts/openshell/templates/statefulset.yaml | 27 + .../workspace-secret-source-role.yaml | 25 + .../workspace-secret-source-rolebinding.yaml | 19 + .../tests/agent_sandbox_preflight_test.yaml | 12 + .../tests/cert_manager_pki_test.yaml | 171 +++ charts/openshell/tests/certgen_test.yaml | 116 ++ charts/openshell/tests/clusterrole_test.yaml | 204 +++ .../tests/clusterrolebinding_test.yaml | 76 + .../tests/credential_drivers_test.yaml | 212 +++ .../credential_secrets_namespace_test.yaml | 86 ++ .../openshell/tests/gateway_config_test.yaml | 1244 +++++++++++++++++ .../gateway_pod_security_context_test.yaml | 36 + .../tests/gateway_upstream_proxy_ca_test.yaml | 110 ++ .../openshell/tests/grpc_endpoint_test.yaml | 26 + charts/openshell/tests/rbac_test.yaml | 95 ++ charts/openshell/tests/route_test.yaml | 118 ++ .../tests/sandbox_namespace_test.yaml | 143 ++ .../tests/sandbox_service_account_test.yaml | 41 + .../tests/statefulset_client_ca_test.yaml | 141 ++ .../workspace_secret_source_role_test.yaml | 77 + charts/openshell/values.yaml | 681 +++++++++ components/control-plane/Dockerfile | 16 +- 69 files changed, 7055 insertions(+), 8 deletions(-) create mode 100644 charts/openshell/.helmignore create mode 100644 charts/openshell/Chart.yaml create mode 100644 charts/openshell/README.md create mode 100644 charts/openshell/README.md.gotmpl create mode 100644 charts/openshell/ci/values-cert-manager.yaml create mode 100644 charts/openshell/ci/values-corporate-proxy-e2e.yaml create mode 100644 charts/openshell/ci/values-credential-driver-kubernetes-secrets.yaml create mode 100644 charts/openshell/ci/values-credential-driver-vault.yaml create mode 100644 charts/openshell/ci/values-gateway-tls.yaml create mode 100644 charts/openshell/ci/values-gateway.yaml create mode 100644 charts/openshell/ci/values-high-availability.yaml create mode 100644 charts/openshell/ci/values-keycloak.yaml create mode 100644 charts/openshell/ci/values-namespace-admin.yaml create mode 100644 charts/openshell/ci/values-openshift-e2e.yaml create mode 100644 charts/openshell/ci/values-openshift-route-cert-manager.yaml create mode 100644 charts/openshell/ci/values-openshift-scc.yaml create mode 100644 charts/openshell/ci/values-skaffold.yaml create mode 100644 charts/openshell/ci/values-spire-stack.yaml create mode 100644 charts/openshell/ci/values-spire.yaml create mode 100644 charts/openshell/ci/values-tls-disabled.yaml create mode 100644 charts/openshell/ci/values-workspace-managed.yaml create mode 100644 charts/openshell/ci/values-workspace-operator.yaml create mode 100644 charts/openshell/skaffold.yaml create mode 100644 charts/openshell/templates/_gateway-workload.tpl create mode 100644 charts/openshell/templates/_helpers.tpl create mode 100644 charts/openshell/templates/agent-sandbox-preflight.yaml create mode 100644 charts/openshell/templates/backend-tls-policy.yaml create mode 100644 charts/openshell/templates/cert-manager-pki.yaml create mode 100644 charts/openshell/templates/certgen.yaml create mode 100644 charts/openshell/templates/clusterrole.yaml create mode 100644 charts/openshell/templates/clusterrolebinding.yaml create mode 100644 charts/openshell/templates/credential-secrets-namespace.yaml create mode 100644 charts/openshell/templates/credential-secrets-role.yaml create mode 100644 charts/openshell/templates/credential-secrets-rolebinding.yaml create mode 100644 charts/openshell/templates/credential-storage-key-encryption-key-secret.yaml create mode 100644 charts/openshell/templates/deployment.yaml create mode 100644 charts/openshell/templates/gateway-config.yaml create mode 100644 charts/openshell/templates/gateway.yaml create mode 100644 charts/openshell/templates/grpcroute.yaml create mode 100644 charts/openshell/templates/networkpolicy.yaml create mode 100644 charts/openshell/templates/peer-role.yaml create mode 100644 charts/openshell/templates/peer-service.yaml create mode 100644 charts/openshell/templates/role.yaml create mode 100644 charts/openshell/templates/rolebinding.yaml create mode 100644 charts/openshell/templates/route.yaml create mode 100644 charts/openshell/templates/service.yaml create mode 100644 charts/openshell/templates/serviceaccount.yaml create mode 100644 charts/openshell/templates/statefulset.yaml create mode 100644 charts/openshell/templates/workspace-secret-source-role.yaml create mode 100644 charts/openshell/templates/workspace-secret-source-rolebinding.yaml create mode 100644 charts/openshell/tests/agent_sandbox_preflight_test.yaml create mode 100644 charts/openshell/tests/cert_manager_pki_test.yaml create mode 100644 charts/openshell/tests/certgen_test.yaml create mode 100644 charts/openshell/tests/clusterrole_test.yaml create mode 100644 charts/openshell/tests/clusterrolebinding_test.yaml create mode 100644 charts/openshell/tests/credential_drivers_test.yaml create mode 100644 charts/openshell/tests/credential_secrets_namespace_test.yaml create mode 100644 charts/openshell/tests/gateway_config_test.yaml create mode 100644 charts/openshell/tests/gateway_pod_security_context_test.yaml create mode 100644 charts/openshell/tests/gateway_upstream_proxy_ca_test.yaml create mode 100644 charts/openshell/tests/grpc_endpoint_test.yaml create mode 100644 charts/openshell/tests/rbac_test.yaml create mode 100644 charts/openshell/tests/route_test.yaml create mode 100644 charts/openshell/tests/sandbox_namespace_test.yaml create mode 100644 charts/openshell/tests/sandbox_service_account_test.yaml create mode 100644 charts/openshell/tests/statefulset_client_ca_test.yaml create mode 100644 charts/openshell/tests/workspace_secret_source_role_test.yaml create mode 100644 charts/openshell/values.yaml diff --git a/charts/openshell/.helmignore b/charts/openshell/.helmignore new file mode 100644 index 000000000..0aecc346a --- /dev/null +++ b/charts/openshell/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +.DS_Store +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +*.swp +*.bak +*.tmp +*.orig +*~ +.project +.idea/ +*.tmproj +.vscode/ + +# Ignore development files +README.md.gotmpl +skaffold.yaml +ci/ diff --git a/charts/openshell/Chart.yaml b/charts/openshell/Chart.yaml new file mode 100644 index 000000000..7b3a5c60d --- /dev/null +++ b/charts/openshell/Chart.yaml @@ -0,0 +1,13 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v2 +# Chart name determines the OCI image name: ghcr.io/nvidia/openshell/helm-chart: +name: helm-chart +description: runtime environment for autonomous agents +type: application +# version and appVersion are patched to the release semver by CI before helm package. +# appVersion doubles as the default image tag (image.tag defaults to appVersion when +# empty), so a released chart automatically pulls matching gateway, sandbox, and supervisor images. +version: 0.0.0 +appVersion: "0.0.0" diff --git a/charts/openshell/README.md b/charts/openshell/README.md new file mode 100644 index 000000000..bfb60ddb8 --- /dev/null +++ b/charts/openshell/README.md @@ -0,0 +1,486 @@ +# OpenShell Helm Chart + + + +> **Experimental** - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes. + +This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`. + +By default, this chart also creates the namespace-scoped resources needed by +sandboxes. For a shared-gateway deployment, install it with +`workspaceResources.enabled=false`, then install the +`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace +namespace. The gateway and workspace releases can then be upgraded and removed +independently. Use Kubernetes `operator` workspace mode when one gateway serves +multiple pre-provisioned workspace namespaces. + +## Cluster-scoped vs namespaced objects + +Most objects in this chart are namespaced and land in the release namespace. +Only two are cluster-scoped: + +| Object | Default name | +| --- | --- | +| `ClusterRole` | `-node-reader-` | +| `ClusterRoleBinding` | `-node-reader-` | + +By default the release creates both, so an install by a cluster-admin is +unchanged. On clusters where cluster-scoped RBAC is owned by a different team, +split the install in two. + +A cluster-admin applies the cluster-scoped objects once per gateway +ServiceAccount, rendered from the same values the release uses: + +```shell +helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.create=true \ + --set rbac.clusterScoped.create=true \ + --set agentSandbox.preflight.enabled=false \ + --show-only templates/clusterrole.yaml \ + --show-only templates/clusterrolebinding.yaml | kubectl apply -f - +``` + +A namespace-admin then installs and upgrades the release with cluster-scoped +objects omitted, using [`ci/values-namespace-admin.yaml`](ci/values-namespace-admin.yaml) +or the equivalent `--set`: + +```shell +helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.clusterScoped.create=false +``` + +The gateway ServiceAccount name and namespace do not change, so the +pre-created `ClusterRoleBinding` keeps matching the release. This works with +`serviceAccount.create=false` too: the `ClusterRoleBinding` subject follows +`serviceAccount.name`, so render the admin step with the same values. + +### Which flag the installer needs + +In the default `shared` workspace mode the release also creates a namespaced +sandbox `Role` granting Agent Sandbox (`agents.x-k8s.io`) permissions. +Kubernetes forbids granting permissions you do not hold, and the built-in +`admin` ClusterRole does not cover that CRD, so an installer holding only +`admin` cannot create it. `rbac.clusterScoped.create=false` alone is then not +enough and the install fails with `attempting to grant RBAC permissions not +currently held`. + +| Workspace mode | Installer holds | Use | +| --- | --- | --- | +| `shared` | built-in `admin` only | `rbac.create=false`, cluster-admin pre-creates all gateway RBAC | +| `shared` | `admin` plus the sandbox permissions in the namespace | `rbac.clusterScoped.create=false` | +| `managed`, `operator` | built-in `admin` only | `rbac.clusterScoped.create=false` | + +`managed` and `operator` render no namespaced sandbox `Role`, so no extra grant +is needed there. + +With `rbac.create=false` the cluster-admin applies the namespaced RBAC too, +adding it to the same render: + +```shell +helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.create=true \ + --set rbac.clusterScoped.create=true \ + --set agentSandbox.preflight.enabled=false \ + --show-only templates/clusterrole.yaml \ + --show-only templates/clusterrolebinding.yaml \ + --show-only templates/role.yaml \ + --show-only templates/rolebinding.yaml \ + --show-only templates/peer-role.yaml | kubectl apply -f - +``` + +To grant the installer the sandbox permissions instead, bind it to a Role +carrying the same rules as the chart's `openshell-sandbox` Role. + +The certgen hook and credential driver RBAC keep their own flags +(`pkiInitJob.enabled` and +`server.credentialDrivers.kubernetesSecrets.rbac.create`). + +### Migrating an existing release + +Helm deletes objects that leave a release manifest, so setting +`rbac.clusterScoped.create=false` on a release that already owns the +`ClusterRole` and `ClusterRoleBinding` deletes them. The gateway then loses +TokenReview until a cluster-admin re-applies them. Hand ownership over first, as +cluster-admin, so nothing is deleted: + +```shell +kubectl annotate clusterrole "openshell-node-reader-" \ + helm.sh/resource-policy=keep --overwrite +kubectl annotate clusterrolebinding "openshell-node-reader-" \ + helm.sh/resource-policy=keep --overwrite +``` + +The objects then survive the upgrade that sets the flag, and the cluster-admin +owns them from that point on. Fresh installs need no such step. + +`rbac.clusterScoped.create` is independent of +`server.drivers.kubernetes.workspaceMode`. Managed and operator modes change +what the `ClusterRole` contains, but they never force the namespaced release to +apply it. Re-run the cluster-admin step after changing values that affect the +`ClusterRole` rules. + +## Prerequisites + +> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for +> ingress and egress in every sandbox namespace. OpenShell creates the policies, +> but Kubernetes accepts them even if no CNI enforces them. Without enforcement, +> sandbox workloads may connect directly and bypass supervisor network policy. +> Verify CNI support before installing OpenShell. + +The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with: + +```shell +kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml +``` + +The chart does not install this cluster-scoped dependency. By default, it +fails before creating gateway resources when the cluster serves neither +supported Sandbox API (`agents.x-k8s.io/v1beta1` or +`agents.x-k8s.io/v1alpha1`). Disable the check with +`agentSandbox.preflight.enabled=false` for offline `helm template` rendering, +where Helm cannot discover cluster APIs. + +## Install on Kubernetes + +```shell +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version +``` + +## Install on OpenShift + +See the full [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) for details. Quick start: + +```shell +# Precreate the openshell namespace +oc create ns openshell + +# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version -n openshell \ + --set server.disableTls=true \ + --set podSecurityContext.fsGroup=null \ + --set securityContext.runAsUser=null +``` + +On OpenShift 4.22+, end-to-end TLS is supported via `BackendTLSPolicy`. See the +[OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift#end-to-end-tls-openshift-422) for details. + +## Available versions + +| Tag | Source | Notes | +| --- | --- | --- | +| `` (e.g. `0.6.0`) | Tagged GitHub release | Tracks the matching gateway, sandbox, and supervisor image versions. Recommended for production. | +| `-pre.N` (e.g. `0.1.0-pre.3`) | A specific prerelease candidate | Immutable candidate pin that tracks images with the same exact version. | +| `0.0.0-dev` | Latest commit on `main` | Floating tag, overwritten on every push. `appVersion` is `dev`, so images resolve to the `:dev` tag. | +| `0.0.0-dev.` | A specific commit on `main` | Per-commit pin. Chart version and `appVersion` both use the full 40-character commit SHA, which matches the image tag pushed by CI. | + +Prerelease and `dev` tags are intended for testing changes ahead of a release. Production deployments should pin to a stable tagged release. + +## Configuration + +See [`values.yaml`](values.yaml) for source defaults. Selected overlays: + +- [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration +- [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration +- [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration +- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing +- [`ci/values-spire.yaml`](ci/values-spire.yaml) - SPIFFE/SPIRE provider token grants +- [`ci/values-spire-stack.yaml`](ci/values-spire-stack.yaml) - SPIRE hardened chart values for local development + +### Database backend + +By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the +database is backed by a per-pod PVC: + +```yaml +server: + dbUrl: "sqlite:/var/openshell/openshell.db" +``` + +#### External PostgreSQL + +Use external PostgreSQL when the gateway should connect to a database managed +outside this chart. The OpenShell chart does not deploy a database; install +PostgreSQL separately using the chart, operator, or managed service that fits +your environment, then pass the connection URI through a Secret. + +Create a Secret containing the PostgreSQL connection URI if one does not +already exist: + +```bash +kubectl create secret generic my-pg-credentials -n openshell \ + --from-literal=uri="postgresql://user:pass@host:5432/dbname" +``` + +Then install the chart pointing at that Secret: + +```bash +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + -n openshell \ + --set workload.kind=deployment \ + --set server.externalDbSecret=my-pg-credentials +``` + +Use `workload.kind=deployment` for external database-backed multi-replica +gateways. `workload.kind=statefulset` is still available for single-replica +SQLite installs and for operators who explicitly need StatefulSet identity or +storage semantics. + +### Credential storage + +By default, the chart uses the gateway's encrypted database credential storage. +The gateway writes encrypted provider credential envelopes to the OpenShell +database. The chart creates a retained Kubernetes Secret with the shared +key-encryption key and injects that key into every gateway pod, so the same +default works for single-replica and external database-backed HA deployments. + +Use `kubernetes-secrets` or `vault` instead when credentials should live in a +cluster or external secret backend. Enabling one external credential driver +disables the default credential-storage key-encryption key Secret and env injection. + +#### OpenShift + +Append these flags to any of the PostgreSQL commands above for OpenShift: + +``` +--set server.disableTls=true \ +--set podSecurityContext.fsGroup=null \ +--set securityContext.runAsUser=null +``` + +### High availability + +Set `replicaCount` above `1` only with `server.externalDbSecret`; the default +SQLite database is per pod and cannot coordinate multiple gateway replicas. +The chart creates a headless peer Service for gateway-to-gateway relay traffic. +StatefulSet pods use stable pod DNS names through that headless Service. +Deployment pods advertise their pod IP with `OPENSHELL_PEER_ENDPOINT`, because +Kubernetes does not assign stable per-pod DNS names to Deployment replicas. + +Gateway peer traffic uses Kubernetes ServiceAccount identity. Each gateway pod +mounts a projected, pod-bound ServiceAccount token with audience +`openshell-gateway-peer`; receiving replicas validate that token with the +Kubernetes TokenReview API, verify the live pod UID and Helm selector labels, +and authorize only the internal `PeerRelay` RPC. The chart does not create or +accept a shared gateway peer Secret. + +With gateway TLS enabled, peer calls use the chart CA and client TLS Secret for +server verification and mTLS. The client verifies the stable gateway Service +DNS name while connecting directly to the owning pod. Custom TLS Secrets must +include that Service DNS name in the server certificate and provide the CA and +client credentials configured by `server.tls`. + +## Secret bootstrap + +By default, a pre-install/pre-upgrade hook Job runs `openshell-gateway generate-certs` +to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret. +The Job uses the gateway image itself, so air-gapped environments only need to +mirror that one image (no separate openssl/alpine sidecar). + +When `certManager.enabled=true`, cert-manager owns the TLS Secrets and the chart +runs the same hook in JWT-only mode because cert-manager does not create the +sandbox JWT signing Secret. This precedence applies even if +`pkiInitJob.enabled` remains true. Set `pkiInitJob.enabled=false` only when an +external non-cert-manager TLS source manages TLS and you pre-create the sandbox +JWT signing Secret. + +## SPIFFE/SPIRE provider token grants + +Set `server.providerTokenGrants.spiffe.enabled=true` to let the gateway and +sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token grants. The +chart keeps supervisor-to-gateway authentication on gateway-minted sandbox JWTs, +mounts the SPIFFE CSI socket into the gateway pod, exports +`OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET`, and passes the socket path to +the Kubernetes driver so sandbox pods can mount the same socket. + +For local development, uncomment the SPIRE Helm releases in `skaffold.yaml` and +add `ci/values-spire.yaml` to the OpenShell release values files. + +The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the +SPIFFE Workload API, so this path does not require access to the SPIRE OIDC +discovery endpoint or its TLS CA. + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| affinity | object | `{}` | Affinity rules for the gateway pod. | +| agentSandbox.preflight.enabled | bool | `true` | Check the live cluster for a supported Agent Sandbox API before rendering gateway resources. Disable only for offline rendering and linting. | +| certManager.caSecretName | string | `"openshell-ca-tls"` | Secret created for the intermediate CA (Certificate with isCA: true). | +| certManager.certificateDuration | string | `"8760h"` | Duration for cert-manager-issued certificates. | +| certManager.certificateRenewBefore | string | `"720h"` | Renewal window for cert-manager-issued certificates. | +| certManager.clientCaFromServerTlsSecret | bool | `true` | Mount gateway client CA from the internal server TLS secret's ca.crt. The internal server certificate is always signed by the chart CA — the same CA that signs the client (mTLS) certificate — so the default (true) is correct for all configurations, including when serverIssuerRef is set. Only set to false if you mount the client CA from a separate secret via server.tls.clientCaSecretName. | +| certManager.enabled | bool | `false` | Create cert-manager Issuer and Certificate resources. When enabled, cert-manager owns TLS and the chart runs a JWT-only certgen hook to create the sandbox JWT signing Secret that cert-manager does not manage. | +| certManager.serverDnsNames | list | `["openshell","openshell.openshell.svc","openshell.openshell.svc.cluster.local","localhost","openshell.localhost","*.openshell.localhost","host.docker.internal"]` | DNS SANs on the cert-manager-issued server certificate. | +| certManager.serverIpAddresses | list | `["127.0.0.1"]` | IP SANs on the cert-manager-issued server certificate. | +| certManager.serverIssuerRef | object | `{"group":"","kind":"","name":""}` | Override the issuerRef for the external server Certificate (e.g. a real LetsEncrypt/ACME ClusterIssuer for a publicly-trusted cert on an external hostname). When set, the chart creates a second server certificate from this issuer with only the hostnames in serverDnsNames; the internal server certificate is always signed by the chart's own CA. Leave name empty to use the chart CA for all server certificates (default). Requires certManager.enabled=true. | +| fullnameOverride | string | `""` | Override the full generated resource name. | +| gateway.image.digest | string | `""` | Gateway image digest. When set, this takes precedence over tag. | +| gateway.image.pullPolicy | string | `nil` | Gateway image pull policy. Empty uses global.image.pullPolicy. | +| gateway.image.registry | string | `""` | Gateway image registry. Empty uses global.image.registry. | +| gateway.image.repository | string | `"openshell/gateway"` | Gateway image repository. | +| gateway.image.tag | string | `""` | Gateway image tag. Defaults to the chart appVersion when empty. | +| global.image.pullPolicy | string | `"IfNotPresent"` | Shared OpenShell image pull policy. Individual image pull policies take precedence. | +| global.image.registry | string | `"ghcr.io/nvidia"` | Shared OpenShell image registry. Individual image registries take precedence. | +| global.image.tag | string | `""` | Shared OpenShell image tag. Defaults to the chart appVersion when empty. | +| grpcRoute.backendTLSPolicy.caCertificateConfigMapName | string | `""` | Name of the ConfigMap containing the CA certificate (key: ca.crt) used to validate the gateway pod's TLS certificate. Defaults to `-backend-ca` when empty. The certgen hook auto-creates this: with pkiInitJob (default), immediately on install/upgrade; with cert-manager, the hook polls for pkiInitJob.timeoutSeconds seconds waiting for cert-manager to issue the server certificate, then creates the ConfigMap. A single install usually succeeds; if cert-manager takes longer, increase pkiInitJob.timeoutSeconds. By default (pkiInitJob.failOnTimeout=true), the install fails if the timeout is reached; set failOnTimeout=false to allow the install to succeed and run `helm upgrade` after the certificate is issued. | +| grpcRoute.backendTLSPolicy.enabled | bool | `false` | Create a BackendTLSPolicy resource for end-to-end TLS between the Gateway proxy and the OpenShell gateway pod. The traffic flow is: client → HTTPS → Gateway (terminate) → TLS (re-encrypt) → gateway pod. Requires server.disableTls=false and server.tls.enableMtls=false. The certgen hook auto-creates the backend CA ConfigMap. | +| grpcRoute.backendTLSPolicy.hostname | string | `""` | Hostname the Gateway proxy validates against the backend's TLS certificate SAN. Defaults to the service FQDN (`..svc.cluster.local`) when empty, which matches the SAN included by both cert-manager and the pkiInitJob. | +| grpcRoute.enabled | bool | `false` | Create a Gateway API GRPCRoute for the gateway service. | +| grpcRoute.gateway.className | string | `"eg"` | GatewayClass to reference. Envoy Gateway installs one named "eg". | +| grpcRoute.gateway.create | bool | `false` | When true, a Gateway resource is created in the release namespace. Set to false and provide name/namespace to attach to a pre-existing Gateway. | +| grpcRoute.gateway.listener.allowedRoutes | string | `"Same"` | "Same" restricts attached routes to the release namespace; "All" allows any namespace. | +| grpcRoute.gateway.listener.port | int | `80` | Listener port for the generated Gateway resource. Use 443 with protocol HTTPS. | +| grpcRoute.gateway.listener.protocol | string | `"HTTP"` | Listener protocol for the generated Gateway resource: HTTP or HTTPS. HTTPS terminates TLS at the Envoy Gateway listener; pair it with server.disableTls=true so Envoy forwards plaintext to the gateway pod, and use OIDC for client identity (the gateway never sees the client cert). | +| grpcRoute.gateway.listener.tls.certificateRefs | list | `[]` | certificateRefs for the HTTPS listener. Required when protocol is HTTPS. Each entry needs a `name` pointing at a kubernetes.io/tls Secret in the Gateway's namespace. May reference a cert-manager-issued Secret or the existing openshell-server-tls Secret (its SANs must include the external hostname). | +| grpcRoute.gateway.name | string | `""` | Name of the Gateway resource. Defaults to the chart fullname. | +| grpcRoute.gateway.namespace | string | `""` | Namespace of the Gateway referenced by the GRPCRoute parentRef. Defaults to the release namespace. | +| grpcRoute.hostnames | list | `[]` | Hostnames the GRPCRoute matches on. Leave empty to match all hosts. | +| imagePullSecrets | list | `[]` | Image pull secrets attached to gateway and helper pods. | +| nameOverride | string | `"openshell"` | Override the chart name used in generated resource names. | +| networkPolicy.enabled | bool | `true` | Restrict SSH ingress on sandbox pods to the gateway. In managed mode, the driver applies the equivalent policy to each workspace namespace. | +| nodeSelector | object | `{}` | Node selector for the gateway pod. | +| openshiftRoute.annotations | object | `{}` | Extra annotations on the Route (e.g. haproxy.router.openshift.io/*). | +| openshiftRoute.enabled | bool | `false` | Create an OpenShift Route with TLS passthrough. | +| openshiftRoute.host | string | `""` | Hostname for the Route. Must match a SAN on the gateway's server cert. | +| pkiInitJob.enabled | bool | `true` | Run a pre-install/pre-upgrade Job that creates gateway and client mTLS Secrets. When certManager.enabled=true, cert-manager owns TLS and this same hook runs in JWT-only mode even if pkiInitJob.enabled remains true. | +| pkiInitJob.failOnTimeout | bool | `true` | Fail the helm install/upgrade if cert-manager does not issue the certificate within the polling timeout. When true (default), the install fails immediately if the timeout is reached, providing clear feedback that BackendTLSPolicy is non-functional. When false, the hook succeeds with a warning and you can run `helm upgrade` after cert-manager issues the certificate to create the backend CA ConfigMap. If you set this to false and see "TLS error: Secret is not supplied by SDS" when connecting to the gateway, check if the TLS secret exists and run `helm upgrade` to create the ConfigMap. | +| pkiInitJob.serverDnsNames | list | `[]` | Extra DNS SANs to append to the server certificate. | +| pkiInitJob.serverIpAddresses | list | `[]` | Extra IP SANs to append to the server certificate. | +| pkiInitJob.timeoutSeconds | int | `120` | Maximum time in seconds for the certgen hook to poll for cert-manager certificates. When using cert-manager with BackendTLSPolicy, the hook polls for this many seconds waiting for the certificate to be issued, then creates the backend CA ConfigMap. The Job deadline is set to (timeoutSeconds + 30) to allow time for ConfigMap creation and cleanup. Increase this if cert-manager takes longer than 120 seconds to issue certificates. | +| podAnnotations | object | `{}` | Extra annotations to add to the gateway pod. | +| podLabels | object | `{}` | Extra labels to add to the gateway pod. | +| podLifecycle.terminationGracePeriodSeconds | int | `5` | Grace period, in seconds, before Kubernetes terminates the gateway pod. | +| podSecurityContext.fsGroup | int | `1000` | fsGroup assigned to the gateway pod. | +| probes.liveness.failureThreshold | int | `3` | Liveness probe failure threshold before the container is restarted. | +| probes.liveness.initialDelaySeconds | int | `2` | Liveness probe initial delay, in seconds. | +| probes.liveness.periodSeconds | int | `5` | Liveness probe period, in seconds. | +| probes.liveness.timeoutSeconds | int | `1` | Liveness probe timeout, in seconds. | +| probes.readiness.failureThreshold | int | `3` | Readiness probe failure threshold before the pod is marked not ready. | +| probes.readiness.initialDelaySeconds | int | `1` | Readiness probe initial delay, in seconds. | +| probes.readiness.periodSeconds | int | `2` | Readiness probe period, in seconds. | +| probes.readiness.timeoutSeconds | int | `1` | Readiness probe timeout, in seconds. | +| probes.startup.failureThreshold | int | `30` | Startup probe failure threshold before the container is killed. | +| probes.startup.periodSeconds | int | `2` | Startup probe period, in seconds. | +| probes.startup.timeoutSeconds | int | `1` | Startup probe timeout, in seconds. | +| rbac.clusterScoped.clusterRoleBindingName | string | `""` | Name for the ClusterRoleBinding. Empty uses the `-node-reader-` default. | +| rbac.clusterScoped.clusterRoleName | string | `""` | Name for the ClusterRole. Empty uses the `-node-reader-` default. | +| rbac.clusterScoped.create | bool | `true` | Create the cluster-scoped ClusterRole and ClusterRoleBinding. Disable for a namespace-admin install where a cluster-admin applies them separately; the gateway ServiceAccount name and namespace are unchanged, so a pre-created ClusterRoleBinding still matches. | +| rbac.create | bool | `true` | Create the RBAC objects that grant the gateway ServiceAccount access. Disable to supply the namespaced sandbox and peer Role/RoleBinding and the cluster-scoped ClusterRole/ClusterRoleBinding out of band. The certgen hook and credential driver RBAC keep their own flags. | +| replicaCount | int | `1` | Number of OpenShell gateway replicas. Values greater than 1 require server.externalDbSecret because the default SQLite backend is per pod. | +| resources | object | `{}` | Gateway pod resource requests and limits. | +| sandbox.image.digest | string | `""` | Sandbox image digest. When set, this takes precedence over tag. | +| sandbox.image.pullPolicy | string | `nil` | Sandbox image pull policy. Leave unset to use the Kubernetes image default. | +| sandbox.image.repository | string | `"nvcr.io/nvidia/base/ubuntu"` | Default standalone sandbox image repository. | +| sandbox.image.tag | string | `"24.04"` | Sandbox image tag. Defaults to latest when empty. | +| sandboxRuntime.image.digest | string | `""` | Sandbox runtime image digest. When set, this takes precedence over tag. | +| sandboxRuntime.image.pullPolicy | string | `nil` | Sandbox runtime image pull policy. Empty uses global.image.pullPolicy. | +| sandboxRuntime.image.registry | string | `""` | Sandbox runtime image registry. Empty uses global.image.registry. | +| sandboxRuntime.image.repository | string | `"openshell/sandbox"` | Sandbox runtime image repository. | +| sandboxRuntime.image.tag | string | `""` | Sandbox runtime image tag. Defaults to the chart appVersion when empty. | +| sandboxServiceAccount.annotations | object | `{}` | Annotations to add to the generated sandbox service account. | +| sandboxServiceAccount.create | bool | `true` | Create a service account for sandbox pods. | +| sandboxServiceAccount.name | string | `""` | Existing service account name for sandbox pods when sandboxServiceAccount.create is false. | +| securityContext.allowPrivilegeEscalation | bool | `false` | Whether the gateway container can gain additional privileges. | +| securityContext.capabilities.drop | list | `["ALL"]` | Linux capabilities dropped from the gateway container. | +| securityContext.runAsNonRoot | bool | `true` | Require the gateway container to run as a non-root user. | +| securityContext.runAsUser | int | `1000` | UID assigned to the gateway container. | +| server.auth.allowUnauthenticatedUsers | bool | `false` | UNSAFE: accept unauthenticated CLI/user requests as a local developer principal. Intended only for trusted local Skaffold/k3d development or a fully trusted fronting proxy. Leave false for shared or production clusters. | +| server.credentialDrivers.kubernetesSecrets.createNamespace | bool | `false` | Create the credential namespace. Requires a namespace other than the release namespace. The Namespace is retained on uninstall so stored credentials survive; an existing Namespace not owned by this release is left untouched. | +| server.credentialDrivers.kubernetesSecrets.enabled | bool | `false` | Enable the in-tree Kubernetes Secret credential driver. WARNING: The RBAC Role grants read/write access to ALL Secrets in the configured namespace. Use a dedicated namespace to limit blast radius. | +| server.credentialDrivers.kubernetesSecrets.namespace | string | `""` | Namespace where OpenShell-managed provider Secret objects are stored. Empty = Helm release namespace. A dedicated namespace is RECOMMENDED to isolate OpenShell-managed Secrets from other workloads. | +| server.credentialDrivers.kubernetesSecrets.rbac.create | bool | `true` | Create a Role/RoleBinding granting the gateway ServiceAccount read/write access to managed provider Secrets. | +| server.credentialDrivers.vault.address | string | `""` | Vault service base URL. Non-loopback endpoints must use HTTPS, for example https://vault.vault.svc.cluster.local:8200. | +| server.credentialDrivers.vault.authMethod | string | `"kubernetes"` | Authentication method. Use "kubernetes" in-cluster or "token_file" for local/dev validation. | +| server.credentialDrivers.vault.caConfigMapName | string | `""` | ConfigMap containing the private Vault CA certificate bundle in the ca.crt key. Leave empty to use platform trust roots. | +| server.credentialDrivers.vault.enabled | bool | `false` | Enable the in-tree Vault credential driver. | +| server.credentialDrivers.vault.kubernetesAuthMount | string | `"kubernetes"` | Vault Kubernetes auth mount. | +| server.credentialDrivers.vault.kvVersion | string | `"2"` | Default KV engine version. Use "1" or "2". | +| server.credentialDrivers.vault.mount | string | `"secret"` | Default KV mount name. | +| server.credentialDrivers.vault.role | string | `""` | Vault Kubernetes auth role when authMethod is kubernetes. | +| server.credentialDrivers.vault.serviceAccountTokenPath | string | `"/var/run/secrets/kubernetes.io/serviceaccount/token"` | ServiceAccount token path used for Kubernetes auth. | +| server.credentialDrivers.vault.timeoutSecs | string | `""` | HTTP request timeout in seconds. Empty = driver default. | +| server.credentialDrivers.vault.tokenPath | string | `""` | Mounted token file path when authMethod is token_file. | +| server.credentialStorage.existingSecret | string | `""` | Name of a pre-existing Secret containing the key-encryption key. When set, the chart does NOT generate a new Secret; it references this one instead. The Secret must contain a key named "key-encryption-key" with a base64-encoded 32-byte value. Required for GitOps workflows that render manifests with `helm template` (where `lookup` is unavailable). | +| server.dbUrl | string | `"sqlite:/var/openshell/openshell.db"` | Gateway database URL (used for the default SQLite backend). SQLite runs in WAL mode and needs a local block-backed volume, not NFS or other network filesystems. | +| server.defaultRuntimeClassName | string | `""` | Default Kubernetes runtimeClassName for sandbox pods. Applied when a CreateSandbox request does not specify one. Empty (default) = omit the field, using the cluster's default RuntimeClass. Set to a RuntimeClass name (e.g. "kata-containers", "nvidia") to apply it to all sandboxes that don't explicitly override it. | +| server.disableTls | bool | `false` | Disable TLS entirely - the server listens on plaintext HTTP. Set to true when a reverse proxy / tunnel terminates TLS at the edge. | +| server.drivers.kubernetes.allowDriverConfig | bool | `false` | Allow caller driver JSON; external resources still require approval. | +| server.drivers.kubernetes.operatorNamespaceFile | string | `""` | Path to a JSON file containing an array of namespace names allowed in operator mode. Hot-reloaded on change. | +| server.drivers.kubernetes.operatorNamespaceLabel | string | `""` | K8s label selector for namespace discovery in operator mode. The driver watches namespaces matching this label. | +| server.drivers.kubernetes.resourceAdmission.enabled | bool | `true` | Require operator approval labels on external sandbox attachments (GPU attachments exempt). | +| server.drivers.kubernetes.resourceAdmission.requiredLabels | string | `nil` | Replacement label map; null uses the built-in admission labels. Empty map is invalid when enabled. | +| server.drivers.kubernetes.workspaceMode | string | `"shared"` | How workspaces map to Kubernetes namespaces. "shared" (default): all sandboxes in a single namespace. "managed": auto-creates per-workspace namespaces. "operator": uses pre-provisioned namespaces. | +| server.enableLoopbackServiceHttp | bool | `true` | Enable plaintext HTTP routing for loopback sandbox service URLs on TLS-enabled gateways. | +| server.enableUserNamespaces | bool | `false` | Enable Kubernetes user namespace isolation (hostUsers: false) for sandbox pods. Requires Kubernetes 1.33+ with user namespace support available (beta through 1.35, GA in 1.36+), plus a supporting container runtime and Linux 5.12+. When enabled, container UID 0 maps to an unprivileged host UID and capabilities become namespaced. | +| server.enableWebsocketTunnel | bool | `false` | Enable the WebSocket tunnel used by CLI/SDK clients behind an authenticated edge proxy. Leave disabled for direct gateway installs. | +| server.externalDbSecret | string | `""` | Name of a pre-existing Opaque Secret containing a PostgreSQL connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL from this Secret instead of using dbUrl. The Secret must contain a `uri` key, e.g. postgresql://user:pass@host:5432/dbname. | +| server.grpcEndpoint | string | `""` | gRPC endpoint sandboxes call back into the gateway. Leave empty to derive it from the chart fullname, release namespace, service port, and disableTls flag, for example https://openshell.openshell.svc.cluster.local:8080. Override only when sandboxes must reach the gateway via a different hostname (e.g. an external ingress or a host alias). | +| server.grpcRateLimit.requests | int | `0` | Maximum gRPC requests allowed per window. Must be positive (alongside windowSeconds) to enable rate limiting; 0 (default) disables it. | +| server.grpcRateLimit.windowSeconds | int | `0` | gRPC rate-limit window length in seconds. Must be positive (alongside requests) to enable rate limiting; 0 (default) disables it. | +| server.hostGatewayIP | string | `""` | Host gateway IP for sandbox pod hostAliases. When set, sandbox pods get hostAliases entries mapping host.docker.internal and host.openshell.internal to this IP, allowing them to reach services running on the Docker host. Auto-detected by the cluster entrypoint script. | +| server.logLevel | string | `"info"` | Gateway log level. | +| server.name | string | `""` | Operator-facing gateway name. Defaults to the chart fullname so all replicas in one installation share an identity. Set explicitly when one telemetry collector receives spans from multiple namespaces or clusters. | +| server.oidc.adminRole | string | `""` | Role name for admin access. Leave empty (with userRole also empty) for authentication-only mode. Both must be set or both empty. | +| server.oidc.audience | string | `"openshell-cli"` | Expected audience claim for the API resource server. This should match the server's --oidc-audience, NOT the CLI client ID. | +| server.oidc.caConfigMapName | string | `""` | Name of a ConfigMap containing a CA certificate bundle (key: ca.crt) for verifying the OIDC issuer's TLS certificate. Required when the issuer uses a non-public CA (e.g. OpenShift ingress, private PKI). | +| server.oidc.dangerouslyAllowInsecureHttp | bool | `false` | Development only: permit cleartext OIDC requests to numeric loopback addresses. This never permits HTTP to hostnames or non-loopback addresses. | +| server.oidc.issuer | string | `""` | OIDC issuer URL (e.g. https://keycloak.example.com/realms/openshell). | +| server.oidc.jwksAllowedOrigins | list | `[]` | Additional trusted HTTPS origins allowed to serve JWKS. The issuer origin is always allowed. Entries must not include a path or query. | +| server.oidc.jwksTtl | int | `3600` | JWKS key cache TTL in seconds. Must be greater than zero. | +| server.oidc.rolesClaim | string | `""` | Dot-separated path to the roles array in the JWT claims. Keycloak: "realm_access.roles", Entra ID: "roles", Okta: "groups". | +| server.oidc.scopesClaim | string | `""` | Dot-separated path to the scopes array in the JWT claims. | +| server.oidc.userRole | string | `""` | Role name for standard user access. | +| server.otlp.endpoint | string | `""` | OTLP/gRPC collector endpoint, conventionally using port 4317. | +| server.otlp.serviceName | string | `""` | Gateway OpenTelemetry service name. Empty uses openshell-gateway. | +| server.policyValidationFailureMode | string | `"fail_closed"` | Posture when a candidate sandbox policy fails validation. `fail_closed` deactivates the previous policy; `retain_last_valid` keeps it active. | +| server.providerTokenGrants.spiffe.enabled | bool | `false` | Mount the SPIFFE Workload API socket into gateway and sandbox pods for dynamic provider token grants. | +| server.providerTokenGrants.spiffe.workloadApiSocketPath | string | `"/spiffe-workload-api/spire-agent.sock"` | Path to the SPIFFE Workload API socket mounted into gateway and sandbox pods. | +| server.sandboxImagePullSecrets | list | `[]` | Image pull secrets attached to sandbox pods. Referenced Secrets must exist in the sandbox namespace. | +| server.sandboxJwt.gatewayId | string | `""` | Stable gateway identity embedded in iss/aud of every minted token. Defaults to the release name so HA replicas share identity. | +| server.sandboxJwt.k8sSaTokenTtlSecs | int | `3600` | Lifetime (seconds) of the projected ServiceAccount token kubelet writes into each sandbox pod for the IssueSandboxToken bootstrap exchange. Kubelet enforces a minimum of 600s; the driver clamps values outside [600, 86400]. Default 3600 — generous, since the supervisor consumes the token within seconds of pod start. | +| server.sandboxJwt.secretDefaultMode | string | `""` | File mode for the mounted JWT signing key Secret. Default 0400 (owner-read only). Override to 0440 or 0444 if the container UID does not match the volume file owner. | +| server.sandboxJwt.signingSecretName | string | `""` | Name of the Opaque Secret holding the signing key material. Empty falls back to the chart fullname with "-jwt-keys" appended. | +| server.sandboxJwt.ttlSecs | int | `3600` | Token TTL in seconds. Defaults to 3600 (1h). | +| server.sandboxNamespace | string | `""` | Namespace where sandbox pods are created. Defaults to the Helm release namespace (.Release.Namespace) when left empty. | +| server.telemetryEnabled | bool | `true` | Enable anonymous OpenShell telemetry from the gateway and the sandbox supervisors it launches. | +| server.tls.certSecretName | string | `"openshell-server-tls"` | K8s secret (type kubernetes.io/tls) with tls.crt and tls.key for the server. | +| server.tls.clientCaSecretName | string | `"openshell-server-client-ca"` | K8s secret with ca.crt for client certificate verification (mTLS). Only used when enableMtls is true. Set to "" to disable client certificate verification for HTTPS-only mode. | +| server.tls.clientTlsSecretName | string | `"openshell-client-tls"` | K8s secret mounted into sandbox pods for mTLS to the server. | +| server.tls.enableMtls | bool | `true` | Enable mTLS client certificate authentication. When false, the gateway runs HTTPS-only without requiring client certificates (use OIDC for auth instead). Must be false when using BackendTLSPolicy because ingress proxies cannot present client certificates to the backend. | +| server.workspaceDefaultStorageSize | string | `""` | Default storage size for the workspace PVC in sandbox pods. Uses Kubernetes quantity syntax (e.g. "2Gi", "10Gi", "500Mi"). Empty = built-in default (2Gi). | +| server.workspaceStorageClass | string | `""` | Kubernetes StorageClass for the workspace PVC in sandbox pods. Empty (default) = omit storageClassName, using the cluster's default StorageClass. Set this on clusters with no default StorageClass, otherwise the workspace PVC stays Pending and the sandbox never starts. | +| service.healthPort | int | `8081` | Gateway health service port. | +| service.metricsPort | int | `9090` | Gateway metrics service port. | +| service.port | int | `8080` | Gateway gRPC/HTTP service port. | +| service.type | string | `"ClusterIP"` | Kubernetes Service type for the gateway. | +| serviceAccount.annotations | object | `{}` | Annotations to add to the generated service account. | +| serviceAccount.create | bool | `true` | Create a service account for the gateway. | +| serviceAccount.name | string | `""` | Existing service account name to use when serviceAccount.create is false. | +| supervisor.image.digest | string | `""` | Supervisor image digest. When set, this takes precedence over tag. | +| supervisor.image.pullPolicy | string | `nil` | Supervisor image pull policy. Empty uses global.image.pullPolicy. Prefer always, if_not_present, or never; the chart also accepts legacy Kubernetes spellings Always, IfNotPresent, and Never. | +| supervisor.image.registry | string | `""` | Supervisor image registry. Empty uses global.image.registry. | +| supervisor.image.repository | string | `"openshell/supervisor"` | Supervisor image repository. | +| supervisor.image.tag | string | `""` | Supervisor image tag. Defaults to the chart appVersion when empty. | +| supervisor.sandboxRuntime.boundaryPort | int | `5500` | Workload boundary TLS listener port. | +| tolerations | list | `[]` | Tolerations for the gateway pod. | +| upstreamProxy | object | `{"authAllowInsecure":false,"authSecret":{"key":"","name":""},"caBundle":{"configMapName":"","key":"ca.crt"},"connectByHostname":false,"noProxy":"","url":""}` | Operator-owned corporate forward proxy for policy-approved TLS egress from Kubernetes sandboxes. The workload cannot select or override it. | +| upstreamProxy.authAllowInsecure | bool | `false` | Required when authSecret is configured because Basic auth to an HTTP proxy is cleartext. | +| upstreamProxy.authSecret.key | string | `""` | Secret key containing the proxy credential. | +| upstreamProxy.authSecret.name | string | `""` | Existing Secret in the sandbox namespace containing a user:pass value. | +| upstreamProxy.caBundle.configMapName | string | `""` | ConfigMap in the release namespace holding the corporate proxy CA bundle. Required for an https:// proxy with a private CA, and for a TLS-intercepting proxy that re-signs upstream certificates. The gateway reads it and stages it into each sandbox's immutable supervisor bootstrap Secret, so the anchor stays in the gateway's trust domain rather than the workload namespace. Supply only the CA that signs your proxy's certificate, or that the proxy re-signs intercepted upstream certificates with. Public roots already come from the supervisor image and its TLS stack, so a full merged trust bundle (for example an OpenShift config.openshift.io/inject-trusted-cabundle ConfigMap) adds hundreds of kilobytes of duplicated roots and can exceed the sandbox boundary's control-frame budget. | +| upstreamProxy.caBundle.key | string | `"ca.crt"` | Key inside that ConfigMap. Change this only to reuse an existing ConfigMap whose key is not ca.crt. | +| upstreamProxy.connectByHostname | bool | `false` | Last-resort option for hostname-filtering proxy ACLs. It lets the proxy resolve CONNECT targets. | +| upstreamProxy.noProxy | string | `""` | Comma-separated destinations that bypass only the corporate proxy. | +| upstreamProxy.url | string | `""` | Proxy URL in http://host:port or https://host:port form. An https:// proxy whose certificate is not publicly trusted also needs caBundle below. | +| workload.allowMultiReplicaStatefulSet | bool | `false` | Allow replicaCount > 1 while rendering a StatefulSet. Prefer workload.kind=deployment for external database-backed multi-replica gateways; this override exists for operators who explicitly require StatefulSet identity or storage semantics. | +| workload.kind | string | `"statefulset"` | Gateway workload controller kind. Use `statefulset` for the default SQLite database, or `deployment` when server.externalDbSecret points at an external database. | +| workspaceResources.enabled | bool | `true` | Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy from this chart. Disable for a gateway-only release. | + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.14.2](https://github.com/norwoodj/helm-docs/releases/v1.14.2) diff --git a/charts/openshell/README.md.gotmpl b/charts/openshell/README.md.gotmpl new file mode 100644 index 000000000..1e86e0cbc --- /dev/null +++ b/charts/openshell/README.md.gotmpl @@ -0,0 +1,310 @@ +# OpenShell Helm Chart + + + +> **Experimental** - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes. + +This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`. + +By default, this chart also creates the namespace-scoped resources needed by +sandboxes. For a shared-gateway deployment, install it with +`workspaceResources.enabled=false`, then install the +`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace +namespace. The gateway and workspace releases can then be upgraded and removed +independently. Use Kubernetes `operator` workspace mode when one gateway serves +multiple pre-provisioned workspace namespaces. + +## Cluster-scoped vs namespaced objects + +Most objects in this chart are namespaced and land in the release namespace. +Only two are cluster-scoped: + +| Object | Default name | +| --- | --- | +| `ClusterRole` | `-node-reader-` | +| `ClusterRoleBinding` | `-node-reader-` | + +By default the release creates both, so an install by a cluster-admin is +unchanged. On clusters where cluster-scoped RBAC is owned by a different team, +split the install in two. + +A cluster-admin applies the cluster-scoped objects once per gateway +ServiceAccount, rendered from the same values the release uses: + +```shell +helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.create=true \ + --set rbac.clusterScoped.create=true \ + --set agentSandbox.preflight.enabled=false \ + --show-only templates/clusterrole.yaml \ + --show-only templates/clusterrolebinding.yaml | kubectl apply -f - +``` + +A namespace-admin then installs and upgrades the release with cluster-scoped +objects omitted, using [`ci/values-namespace-admin.yaml`](ci/values-namespace-admin.yaml) +or the equivalent `--set`: + +```shell +helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.clusterScoped.create=false +``` + +The gateway ServiceAccount name and namespace do not change, so the +pre-created `ClusterRoleBinding` keeps matching the release. This works with +`serviceAccount.create=false` too: the `ClusterRoleBinding` subject follows +`serviceAccount.name`, so render the admin step with the same values. + +### Which flag the installer needs + +In the default `shared` workspace mode the release also creates a namespaced +sandbox `Role` granting Agent Sandbox (`agents.x-k8s.io`) permissions. +Kubernetes forbids granting permissions you do not hold, and the built-in +`admin` ClusterRole does not cover that CRD, so an installer holding only +`admin` cannot create it. `rbac.clusterScoped.create=false` alone is then not +enough and the install fails with `attempting to grant RBAC permissions not +currently held`. + +| Workspace mode | Installer holds | Use | +| --- | --- | --- | +| `shared` | built-in `admin` only | `rbac.create=false`, cluster-admin pre-creates all gateway RBAC | +| `shared` | `admin` plus the sandbox permissions in the namespace | `rbac.clusterScoped.create=false` | +| `managed`, `operator` | built-in `admin` only | `rbac.clusterScoped.create=false` | + +`managed` and `operator` render no namespaced sandbox `Role`, so no extra grant +is needed there. + +With `rbac.create=false` the cluster-admin applies the namespaced RBAC too, +adding it to the same render: + +```shell +helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + --namespace openshell -f my-values.yaml \ + --set rbac.create=true \ + --set rbac.clusterScoped.create=true \ + --set agentSandbox.preflight.enabled=false \ + --show-only templates/clusterrole.yaml \ + --show-only templates/clusterrolebinding.yaml \ + --show-only templates/role.yaml \ + --show-only templates/rolebinding.yaml \ + --show-only templates/peer-role.yaml | kubectl apply -f - +``` + +To grant the installer the sandbox permissions instead, bind it to a Role +carrying the same rules as the chart's `openshell-sandbox` Role. + +The certgen hook and credential driver RBAC keep their own flags +(`pkiInitJob.enabled` and +`server.credentialDrivers.kubernetesSecrets.rbac.create`). + +### Migrating an existing release + +Helm deletes objects that leave a release manifest, so setting +`rbac.clusterScoped.create=false` on a release that already owns the +`ClusterRole` and `ClusterRoleBinding` deletes them. The gateway then loses +TokenReview until a cluster-admin re-applies them. Hand ownership over first, as +cluster-admin, so nothing is deleted: + +```shell +kubectl annotate clusterrole "openshell-node-reader-" \ + helm.sh/resource-policy=keep --overwrite +kubectl annotate clusterrolebinding "openshell-node-reader-" \ + helm.sh/resource-policy=keep --overwrite +``` + +The objects then survive the upgrade that sets the flag, and the cluster-admin +owns them from that point on. Fresh installs need no such step. + +`rbac.clusterScoped.create` is independent of +`server.drivers.kubernetes.workspaceMode`. Managed and operator modes change +what the `ClusterRole` contains, but they never force the namespaced release to +apply it. Re-run the cluster-admin step after changing values that affect the +`ClusterRole` rules. + + +## Prerequisites + +> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for +> ingress and egress in every sandbox namespace. OpenShell creates the policies, +> but Kubernetes accepts them even if no CNI enforces them. Without enforcement, +> sandbox workloads may connect directly and bypass supervisor network policy. +> Verify CNI support before installing OpenShell. + +The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with: + +```shell +kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml +``` + +The chart does not install this cluster-scoped dependency. By default, it +fails before creating gateway resources when the cluster serves neither +supported Sandbox API (`agents.x-k8s.io/v1beta1` or +`agents.x-k8s.io/v1alpha1`). Disable the check with +`agentSandbox.preflight.enabled=false` for offline `helm template` rendering, +where Helm cannot discover cluster APIs. + +## Install on Kubernetes + +```shell +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version +``` + +## Install on OpenShift + +See the full [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) for details. Quick start: + +```shell +# Precreate the openshell namespace +oc create ns openshell + +# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version -n openshell \ + --set server.disableTls=true \ + --set podSecurityContext.fsGroup=null \ + --set securityContext.runAsUser=null +``` + +On OpenShift 4.22+, end-to-end TLS is supported via `BackendTLSPolicy`. See the +[OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift#end-to-end-tls-openshift-422) for details. + +## Available versions + +| Tag | Source | Notes | +| --- | --- | --- | +| `` (e.g. `0.6.0`) | Tagged GitHub release | Tracks the matching gateway, sandbox, and supervisor image versions. Recommended for production. | +| `-pre.N` (e.g. `0.1.0-pre.3`) | A specific prerelease candidate | Immutable candidate pin that tracks images with the same exact version. | +| `0.0.0-dev` | Latest commit on `main` | Floating tag, overwritten on every push. `appVersion` is `dev`, so images resolve to the `:dev` tag. | +| `0.0.0-dev.` | A specific commit on `main` | Per-commit pin. Chart version and `appVersion` both use the full 40-character commit SHA, which matches the image tag pushed by CI. | + +Prerelease and `dev` tags are intended for testing changes ahead of a release. Production deployments should pin to a stable tagged release. + +## Configuration + +See [`values.yaml`](values.yaml) for source defaults. Selected overlays: + +- [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration +- [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration +- [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration +- [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing +- [`ci/values-spire.yaml`](ci/values-spire.yaml) - SPIFFE/SPIRE provider token grants +- [`ci/values-spire-stack.yaml`](ci/values-spire-stack.yaml) - SPIRE hardened chart values for local development + +### Database backend + +By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the +database is backed by a per-pod PVC: + +```yaml +server: + dbUrl: "sqlite:/var/openshell/openshell.db" +``` + +#### External PostgreSQL + +Use external PostgreSQL when the gateway should connect to a database managed +outside this chart. The OpenShell chart does not deploy a database; install +PostgreSQL separately using the chart, operator, or managed service that fits +your environment, then pass the connection URI through a Secret. + +Create a Secret containing the PostgreSQL connection URI if one does not +already exist: + +```bash +kubectl create secret generic my-pg-credentials -n openshell \ + --from-literal=uri="postgresql://user:pass@host:5432/dbname" +``` + +Then install the chart pointing at that Secret: + +```bash +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version \ + -n openshell \ + --set workload.kind=deployment \ + --set server.externalDbSecret=my-pg-credentials +``` + +Use `workload.kind=deployment` for external database-backed multi-replica +gateways. `workload.kind=statefulset` is still available for single-replica +SQLite installs and for operators who explicitly need StatefulSet identity or +storage semantics. + +### Credential storage + +By default, the chart uses the gateway's encrypted database credential storage. +The gateway writes encrypted provider credential envelopes to the OpenShell +database. The chart creates a retained Kubernetes Secret with the shared +key-encryption key and injects that key into every gateway pod, so the same +default works for single-replica and external database-backed HA deployments. + +Use `kubernetes-secrets` or `vault` instead when credentials should live in a +cluster or external secret backend. Enabling one external credential driver +disables the default credential-storage key-encryption key Secret and env injection. + +#### OpenShift + +Append these flags to any of the PostgreSQL commands above for OpenShift: + +``` +--set server.disableTls=true \ +--set podSecurityContext.fsGroup=null \ +--set securityContext.runAsUser=null +``` + +### High availability + +Set `replicaCount` above `1` only with `server.externalDbSecret`; the default +SQLite database is per pod and cannot coordinate multiple gateway replicas. +The chart creates a headless peer Service for gateway-to-gateway relay traffic. +StatefulSet pods use stable pod DNS names through that headless Service. +Deployment pods advertise their pod IP with `OPENSHELL_PEER_ENDPOINT`, because +Kubernetes does not assign stable per-pod DNS names to Deployment replicas. + +Gateway peer traffic uses Kubernetes ServiceAccount identity. Each gateway pod +mounts a projected, pod-bound ServiceAccount token with audience +`openshell-gateway-peer`; receiving replicas validate that token with the +Kubernetes TokenReview API, verify the live pod UID and Helm selector labels, +and authorize only the internal `PeerRelay` RPC. The chart does not create or +accept a shared gateway peer Secret. + +With gateway TLS enabled, peer calls use the chart CA and client TLS Secret for +server verification and mTLS. The client verifies the stable gateway Service +DNS name while connecting directly to the owning pod. Custom TLS Secrets must +include that Service DNS name in the server certificate and provide the CA and +client credentials configured by `server.tls`. + +## Secret bootstrap + +By default, a pre-install/pre-upgrade hook Job runs `openshell-gateway generate-certs` +to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret. +The Job uses the gateway image itself, so air-gapped environments only need to +mirror that one image (no separate openssl/alpine sidecar). + +When `certManager.enabled=true`, cert-manager owns the TLS Secrets and the chart +runs the same hook in JWT-only mode because cert-manager does not create the +sandbox JWT signing Secret. This precedence applies even if +`pkiInitJob.enabled` remains true. Set `pkiInitJob.enabled=false` only when an +external non-cert-manager TLS source manages TLS and you pre-create the sandbox +JWT signing Secret. + +## SPIFFE/SPIRE provider token grants + +Set `server.providerTokenGrants.spiffe.enabled=true` to let the gateway and +sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token grants. The +chart keeps supervisor-to-gateway authentication on gateway-minted sandbox JWTs, +mounts the SPIFFE CSI socket into the gateway pod, exports +`OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET`, and passes the socket path to +the Kubernetes driver so sandbox pods can mount the same socket. + +For local development, uncomment the SPIRE Helm releases in `skaffold.yaml` and +add `ci/values-spire.yaml` to the OpenShell release values files. + +The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the +SPIFFE Workload API, so this path does not require access to the SPIRE OIDC +discovery endpoint or its TLS CA. + +{{ template "chart.valuesSection" . }} +{{ template "helm-docs.versionFooter" . }} diff --git a/charts/openshell/ci/values-cert-manager.yaml b/charts/openshell/ci/values-cert-manager.yaml new file mode 100644 index 000000000..2d159c176 --- /dev/null +++ b/charts/openshell/ci/values-cert-manager.yaml @@ -0,0 +1,11 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Merge after values.yaml when cert-manager CRDs are installed, e.g.: +# helm install ... -f values.yaml -f ci/values-cert-manager.yaml +# Or add this file to skaffold manifests.helm.releases[].valuesFiles. +server: + disableTls: false + +certManager: + enabled: true diff --git a/charts/openshell/ci/values-corporate-proxy-e2e.yaml b/charts/openshell/ci/values-corporate-proxy-e2e.yaml new file mode 100644 index 000000000..70fdec8d7 --- /dev/null +++ b/charts/openshell/ci/values-corporate-proxy-e2e.yaml @@ -0,0 +1,10 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# The Kubernetes corporate-proxy e2e wrapper supplies the generated proxy URL +# and creates `openshell-e2e-proxy-auth` before Helm installs the gateway. +upstreamProxy: + authSecret: + name: openshell-e2e-proxy-auth + key: proxy-auth + authAllowInsecure: true diff --git a/charts/openshell/ci/values-credential-driver-kubernetes-secrets.yaml b/charts/openshell/ci/values-credential-driver-kubernetes-secrets.yaml new file mode 100644 index 000000000..096ce46e2 --- /dev/null +++ b/charts/openshell/ci/values-credential-driver-kubernetes-secrets.yaml @@ -0,0 +1,13 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Local Kubernetes Secrets credential-driver validation overlay. +# +# Use with: +# skaffold run -p credential-driver-kubernetes-secrets +# +server: + credentialDrivers: + kubernetesSecrets: + enabled: true + namespace: openshell diff --git a/charts/openshell/ci/values-credential-driver-vault.yaml b/charts/openshell/ci/values-credential-driver-vault.yaml new file mode 100644 index 000000000..5158d48d7 --- /dev/null +++ b/charts/openshell/ci/values-credential-driver-vault.yaml @@ -0,0 +1,22 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Local Vault credential-driver validation overlay. +# +# Use with: +# skaffold run -p credential-driver-vault +# +# The profile assumes another process has already deployed a Vault-compatible +# backend. Local e2e validation deploys OpenBao in the `openbao` namespace with +# TLS enabled, publishes its private CA in the `openbao-ca` ConfigMap, and +# creates an `openbao-0` DNS alias matching the OpenBao dev certificate. It also +# configures a Kubernetes auth role named `openshell-gateway` bound to the +# OpenShell gateway ServiceAccount in the `openshell` namespace. + +server: + credentialDrivers: + vault: + enabled: true + address: https://openbao-0:8200 + caConfigMapName: openbao-ca + role: openshell-gateway diff --git a/charts/openshell/ci/values-gateway-tls.yaml b/charts/openshell/ci/values-gateway-tls.yaml new file mode 100644 index 000000000..a77676014 --- /dev/null +++ b/charts/openshell/ci/values-gateway-tls.yaml @@ -0,0 +1,33 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Gateway API overlay with TLS termination at the Envoy Gateway listener. +# +# Exercises the HTTPS listener branch of templates/gateway.yaml for +# `helm template`/lint coverage. Envoy Gateway terminates TLS using the +# referenced kubernetes.io/tls Secret and forwards plaintext to the gateway pod, +# so the gateway runs with TLS disabled and uses OIDC for client identity. +# +# The certificate Secret (openshell-ingress-tls) and the OIDC issuer below are +# placeholders for render coverage; a real deployment must provide a valid TLS +# Secret in the release namespace and a reachable OIDC issuer. + +grpcRoute: + enabled: true + gateway: + create: true + className: "eg" + listener: + port: 443 + protocol: HTTPS + tls: + certificateRefs: + - name: openshell-ingress-tls + hostnames: [] + +server: + # Envoy terminates TLS at the edge; the gateway listens plaintext behind it. + disableTls: true + oidc: + issuer: "https://keycloak.example.com/realms/openshell" + audience: "openshell-cli" diff --git a/charts/openshell/ci/values-gateway.yaml b/charts/openshell/ci/values-gateway.yaml new file mode 100644 index 000000000..196192213 --- /dev/null +++ b/charts/openshell/ci/values-gateway.yaml @@ -0,0 +1,26 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Gateway API overlay — enables a Gateway and GRPCRoute for external access. +# +# Requires Envoy Gateway in the cluster (installed via skaffold.yaml). +# Add this file to the openshell release valuesFiles to activate: +# uncomment ci/values-gateway.yaml in deploy/helm/openshell/skaffold.yaml +# +# Envoy Gateway will create an Envoy proxy Deployment and a LoadBalancer +# Service (named envoy---*) in the openshell namespace. +# +# To reach the gateway from outside a k3d cluster, port-forward to that service: +# kubectl -n openshell get svc -l gateway.envoyproxy.io/owning-gateway-name=openshell +# kubectl -n openshell port-forward svc/ 8080:80 +# # then: grpcurl -plaintext localhost:8080 ... + +grpcRoute: + enabled: true + gateway: + create: true + className: "eg" + # Set one or more hostnames to scope the route, e.g.: + # hostnames: + # - openshell.example.com + hostnames: [] diff --git a/charts/openshell/ci/values-high-availability.yaml b/charts/openshell/ci/values-high-availability.yaml new file mode 100644 index 000000000..407326d67 --- /dev/null +++ b/charts/openshell/ci/values-high-availability.yaml @@ -0,0 +1,13 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# CI/dev overlay for exercising the gateway with more than one replica. SQLite +# is not suitable for HA because each replica has its own pod volume, so this +# overlay expects the caller to provide a PostgreSQL Secret named openshell-ha-pg. +replicaCount: 2 + +workload: + kind: deployment + +server: + externalDbSecret: openshell-ha-pg diff --git a/charts/openshell/ci/values-keycloak.yaml b/charts/openshell/ci/values-keycloak.yaml new file mode 100644 index 000000000..6df74e325 --- /dev/null +++ b/charts/openshell/ci/values-keycloak.yaml @@ -0,0 +1,38 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# OIDC configuration overlay for a local Keycloak instance in k3s. +# +# Run the setup task first (rerun it to rotate the development CA): +# mise run keycloak:k8s:setup +# +# Then layer this file on top of values.yaml when deploying: +# helm upgrade --install openshell . \ +# -f values.yaml -f ci/values-skaffold.yaml -f ci/values-keycloak.yaml +# +# Or add this file to skaffold.yaml valuesFiles for iterative dev. +# +# Issuer note: the setup task configures Keycloak with KC_HOSTNAME set to the +# in-cluster service hostname, so tokens always carry that hostname as `iss` +# regardless of how they were obtained (e.g. via a localhost port-forward). +# The gateway fetches JWKS over TLS from this URL inside the cluster. The setup +# task creates the Keycloak development CA ConfigMap referenced below. +# +# CLI token acquisition: keep a port-forward running while using openshell login: +# kubectl -n keycloak port-forward svc/keycloak 9090:80 + +server: + oidc: + # Must match KC_HOSTNAME set by keycloak:k8s:setup (in-cluster service hostname). + issuer: "https://keycloak.keycloak.svc.cluster.local:443/realms/openshell" + caConfigMapName: "openshell-keycloak-ca" + # Must match the client ID in the imported realm (openshell-cli). + audience: "openshell-cli" + # Short TTL for dev so JWKS key rotation is picked up quickly. + # Use 3600 (default) in production. + jwksTtl: 60 + # Keycloak puts realm roles at realm_access.roles in the JWT. + rolesClaim: "realm_access.roles" + # Leave both empty for authentication-only mode (any valid token is accepted). + adminRole: "openshell-admin" + userRole: "openshell-user" diff --git a/charts/openshell/ci/values-namespace-admin.yaml b/charts/openshell/ci/values-namespace-admin.yaml new file mode 100644 index 000000000..7326be732 --- /dev/null +++ b/charts/openshell/ci/values-namespace-admin.yaml @@ -0,0 +1,25 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Namespace-admin overlay — renders only namespaced objects. +# +# Use this when a cluster-admin applies the gateway ClusterRole and +# ClusterRoleBinding once, out of band, and the OpenShell release is installed +# and upgraded by an installer that holds no cluster-scoped permissions. +# +# Generate the cluster-scoped objects for the cluster-admin step from the same +# release values, then apply them as cluster-admin: +# helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart \ +# --version --namespace openshell -f my-values.yaml \ +# --set rbac.create=true \ +# --set rbac.clusterScoped.create=true \ +# --set agentSandbox.preflight.enabled=false \ +# --show-only templates/clusterrole.yaml \ +# --show-only templates/clusterrolebinding.yaml | kubectl apply -f - +# +# The gateway ServiceAccount name and release namespace are unchanged, so the +# pre-created ClusterRoleBinding still matches this release. + +rbac: + clusterScoped: + create: false diff --git a/charts/openshell/ci/values-openshift-e2e.yaml b/charts/openshell/ci/values-openshift-e2e.yaml new file mode 100644 index 000000000..c470f3ce8 --- /dev/null +++ b/charts/openshell/ci/values-openshift-e2e.yaml @@ -0,0 +1,54 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# OpenShift overlay for the Kubernetes e2e harness. +# +# Bundles the OpenShift-specific settings the harness needs: the Route/mTLS +# transport, and an image pull policy that avoids stale cached images. +# +# Route/mTLS transport: on OpenShift, `kubectl port-forward` stalls the SSH-relay +# `sandbox connect` path (round-trip-heavy SSH over SPDY), so the harness drives +# the gateway through a passthrough OpenShift Route with mTLS instead. This +# overlay turns TLS back on (values-skaffold.yaml disables it), enables the Route, +# and promotes the cert-verified caller to a dev principal. +# +# Image pull policy: force `Always` so runs against the `latest` upstream image +# actually use it, instead of a stale copy cached on the cluster nodes. +# +# Layered by e2e/with-kube-gateway.sh AFTER ci/values-skaffold.yaml and +# ci/values-openshift-scc.yaml when an OpenShift cluster is detected. The harness +# supplies `openshiftRoute.host` and `pkiInitJob.serverDnsNames[0]` via --set at +# install time (both are the cluster-derived Route hostname). +# +# Security: this is NOT an open gateway. `server.tls.clientCaSecretName` defaults +# to `openshell-server-client-ca` and there is no OIDC, so `require_client_auth` +# is true and mTLS is MANDATORY at the TLS handshake — a caller with only the +# Route URL and no client certificate is rejected before any RPC. The passthrough +# Route terminates TLS at the gateway pod, so this holds end-to-end. +# `allowUnauthenticatedUsers` only promotes the already cert-verified caller to a +# dev principal at the app layer (mtls_auth is unsupported with the Kubernetes +# driver). Both are required together; the client certificate is the access gate. +gateway: + image: + pullPolicy: Always + +supervisor: + image: + pullPolicy: Always + +server: + disableTls: false + auth: + allowUnauthenticatedUsers: true + +openshiftRoute: + enabled: true + # host is supplied via --set at install time (cluster-derived Route hostname). + # The default HAProxy Route timeout is 30s, which severs long-lived transfers + # (large sandbox upload/download, SSH-relay `sandbox connect`) mid-stream. Raise + # both the connection timeout and the passthrough tunnel timeout so these paths + # survive. Passthrough Routes proxy in TCP mode, so timeout-tunnel governs the + # established tunnel while timeout covers the pre-tunnel phase. + annotations: + haproxy.router.openshift.io/timeout: 300s + haproxy.router.openshift.io/timeout-tunnel: 300s diff --git a/charts/openshell/ci/values-openshift-route-cert-manager.yaml b/charts/openshell/ci/values-openshift-route-cert-manager.yaml new file mode 100644 index 000000000..de4487136 --- /dev/null +++ b/charts/openshell/ci/values-openshift-route-cert-manager.yaml @@ -0,0 +1,30 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Render-coverage overlay for cert-manager issuing the server certificate from +# an external Issuer/ClusterIssuer (e.g. a real LetsEncrypt/ACME issuer), plus an +# OpenShift Route with TLS passthrough. Merge after values.yaml: +# helm lint deploy/helm/openshell -f ci/values-openshift-route-cert-manager.yaml +# +# The ClusterIssuer name below is a placeholder for render coverage; a real +# deployment must reference an Issuer/ClusterIssuer that's actually installed +# and Ready in the target cluster. See docs/kubernetes/managing-certificates.mdx. +# +# clientCaFromServerTlsSecret defaults to true and is correct here: the +# internal server certificate is always signed by the chart CA, so its +# ca.crt is exactly the CA that signed the client (mTLS) certificate. + +server: + disableTls: false + +certManager: + enabled: true + serverIssuerRef: + name: letsencrypt-prod + kind: ClusterIssuer + serverDnsNames: + - openshell.example.com + +openshiftRoute: + enabled: true + host: openshell.example.com diff --git a/charts/openshell/ci/values-openshift-scc.yaml b/charts/openshell/ci/values-openshift-scc.yaml new file mode 100644 index 000000000..b7f37be6e --- /dev/null +++ b/charts/openshell/ci/values-openshift-scc.yaml @@ -0,0 +1,20 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# OpenShift SCC compatibility overlay. Removes the hardcoded runAsUser and +# fsGroup so that OpenShift's restricted-v2 SCC can inject the namespace- +# assigned UID/GID range. Layer after values.yaml: +# helm install openshell deploy/helm/openshell -f ci/values-openshift-scc.yaml +# +# The e2e Kubernetes harness applies this automatically when it detects an +# OpenShift cluster (route.openshift.io API present). + +podSecurityContext: null + +securityContext: + runAsNonRoot: true + runAsUser: null + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL diff --git a/charts/openshell/ci/values-skaffold.yaml b/charts/openshell/ci/values-skaffold.yaml new file mode 100644 index 000000000..4c017f669 --- /dev/null +++ b/charts/openshell/ci/values-skaffold.yaml @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Merge with values.yaml for Skaffold-driven local image builds (see skaffold.yaml). +server: + otlp: + endpoint: http://openshell-collector.observability.svc.cluster.local:4317 + # Comment out to enforce mTLS (uses PKI secrets generated by pkiInitJob). + disableTls: true + auth: + allowUnauthenticatedUsers: true + +sandbox: + image: + pullPolicy: if_not_present + +supervisor: + image: + pullPolicy: if_not_present diff --git a/charts/openshell/ci/values-spire-stack.yaml b/charts/openshell/ci/values-spire-stack.yaml new file mode 100644 index 000000000..8a1e64882 --- /dev/null +++ b/charts/openshell/ci/values-spire-stack.yaml @@ -0,0 +1,24 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# SPIRE hardened chart values for the local Helm dev environment. +global: + spire: + clusterName: openshell-dev + jwtIssuer: https://spire-spiffe-oidc-discovery-provider.spire.svc.cluster.local + trustDomain: openshell.local + +spire-server: + defaultJwtSvidTTL: 5m + controllerManager: + identities: + clusterSPIFFEIDs: + openshell-sandboxes: + enabled: true + spiffeIDTemplate: 'spiffe://{{ .TrustDomain }}/openshell/sandbox/{{ index .PodMeta.Annotations "openshell.ai/sandbox-id" }}' + namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: openshell + podSelector: + matchLabels: + openshell.ai/managed-by: openshell diff --git a/charts/openshell/ci/values-spire.yaml b/charts/openshell/ci/values-spire.yaml new file mode 100644 index 000000000..201520e81 --- /dev/null +++ b/charts/openshell/ci/values-spire.yaml @@ -0,0 +1,9 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# OpenShell overlay for local SPIRE-backed provider token grants. +server: + providerTokenGrants: + spiffe: + enabled: true + workloadApiSocketPath: /spiffe-workload-api/spire-agent.sock diff --git a/charts/openshell/ci/values-tls-disabled.yaml b/charts/openshell/ci/values-tls-disabled.yaml new file mode 100644 index 000000000..7a771a178 --- /dev/null +++ b/charts/openshell/ci/values-tls-disabled.yaml @@ -0,0 +1,9 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# CI lint target: TLS disabled (plaintext HTTP, no client cert requirement). +# Typical when a reverse proxy or tunnel terminates TLS at the edge. +server: + disableTls: true +pkiInitJob: + enabled: false diff --git a/charts/openshell/ci/values-workspace-managed.yaml b/charts/openshell/ci/values-workspace-managed.yaml new file mode 100644 index 000000000..e9f88846c --- /dev/null +++ b/charts/openshell/ci/values-workspace-managed.yaml @@ -0,0 +1,11 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# E2E overlay: deploy the gateway in managed workspace mode. +# Sandbox namespaces are auto-created as openshell-{gateway_id}-{workspace}. +server: + sandboxImagePullSecrets: + - name: e2e-regcred + drivers: + kubernetes: + workspaceMode: "managed" diff --git a/charts/openshell/ci/values-workspace-operator.yaml b/charts/openshell/ci/values-workspace-operator.yaml new file mode 100644 index 000000000..8d895e4e9 --- /dev/null +++ b/charts/openshell/ci/values-workspace-operator.yaml @@ -0,0 +1,10 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# E2E overlay: deploy the gateway in operator workspace mode. +# Namespaces must be pre-provisioned and labeled before sandbox creation. +server: + drivers: + kubernetes: + workspaceMode: "operator" + operatorNamespaceLabel: "openshell.ai/e2e-operator-workspace=true" diff --git a/charts/openshell/skaffold.yaml b/charts/openshell/skaffold.yaml new file mode 100644 index 000000000..2d7bf892e --- /dev/null +++ b/charts/openshell/skaffold.yaml @@ -0,0 +1,180 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Local dev: builds gateway, sandbox, and supervisor images via tasks/scripts/docker-build-image.sh, +# which first stages Rust binaries natively on the host (using cargo / cargo-zigbuild +# when cross-compiling) and then builds the image from the prebuilt binary. This +# mirrors CI and is faster than compiling inside Docker on every rebuild because +# the host's cargo target cache and sccache are reused across iterations. +# +# Run from repo root: +# mise run helm:skaffold:dev +# mise run helm:skaffold:run +# +# See https://skaffold.dev/docs/deployers/helm/ (setValueTemplates, IMAGE_* fields). +apiVersion: skaffold/v4beta14 +kind: Config +metadata: + name: openshell +build: + local: + push: false + tagPolicy: + gitCommit: {} + artifacts: + - image: openshell/gateway + context: ../../.. + custom: + buildCommand: | + CONTAINER_ENGINE_TARGET=local-k8s-cluster \ + IMAGE_NAME="${IMAGE%:*}" \ + IMAGE_TAG="${IMAGE##*:}" \ + tasks/scripts/docker-build-image.sh gateway + dependencies: + paths: + - Cargo.toml + - Cargo.lock + - crates/** + - proto/** + - deploy/docker/Dockerfile.gateway + - tasks/scripts/docker-build-image.sh + - tasks/scripts/stage-prebuilt-binaries.sh + - image: openshell/supervisor + context: ../../.. + custom: + buildCommand: | + CONTAINER_ENGINE_TARGET=local-k8s-cluster \ + IMAGE_NAME="${IMAGE%:*}" \ + IMAGE_TAG="${IMAGE##*:}" \ + tasks/scripts/docker-build-image.sh supervisor + dependencies: + paths: + - Cargo.toml + - Cargo.lock + - crates/** + - proto/** + - deploy/docker/Dockerfile.supervisor + - tasks/scripts/docker-build-image.sh + - tasks/scripts/stage-prebuilt-binaries.sh + - image: openshell/sandbox + context: ../../.. + custom: + buildCommand: | + CONTAINER_ENGINE_TARGET=local-k8s-cluster \ + IMAGE_NAME="${IMAGE%:*}" \ + IMAGE_TAG="${IMAGE##*:}" \ + tasks/scripts/docker-build-image.sh sandbox + dependencies: + paths: + - Cargo.toml + - Cargo.lock + - crates/** + - proto/** + - deploy/docker/Dockerfile.sandbox + - tasks/scripts/docker-build-image.sh + - tasks/scripts/stage-prebuilt-binaries.sh +deploy: + helm: + releases: + # cert-manager — comment this in and add values-cert-manager.yaml below + # when you want cert-manager to manage TLS while certgen handles JWT. + # Requires cert-manager CRDs to be installed in the cluster first. + #- name: cert-manager + # remoteChart: oci://quay.io/jetstack/charts/cert-manager + # version: v1.20.2 + # namespace: cert-manager + # createNamespace: true + # setValues: + # crds.enabled: true + # Envoy Gateway — Kubernetes Gateway API implementation. + # Installs the Gateway API CRDs and the "eg" GatewayClass. + # Required when grpcRoute.enabled is true in the openshell release. + #- name: envoy-gateway + # remoteChart: oci://docker.io/envoyproxy/gateway-helm + # version: v1.7.2 + # namespace: envoy-gateway-system + # createNamespace: true + # # wait ensures Gateway API CRDs are registered before the openshell + # # release attempts to create Gateway and HTTPRoute resources. + # wait: true + # SPIRE — installs SPIRE Server, Agent, Controller Manager, CSI Driver, + # and OIDC Discovery Provider using the SPIFFE hardened charts. + # Uncomment both releases and ci/values-spire.yaml below to use + # SPIFFE JWT-SVIDs for dynamic provider token grants. + #- name: spire-crds + # repo: https://spiffe.github.io/helm-charts-hardened/ + # remoteChart: spire-crds + # version: 0.5.0 + # namespace: spire + # createNamespace: true + # wait: true + #- name: spire + # repo: https://spiffe.github.io/helm-charts-hardened/ + # remoteChart: spire + # version: 0.29.0 + # namespace: spire + # createNamespace: true + # valuesFiles: + # - ci/values-spire-stack.yaml + # wait: true + - name: openshell + chartPath: . + namespace: openshell + createNamespace: true + valuesFiles: + - values.yaml + - ci/values-skaffold.yaml + # Add ci/values-cert-manager.yaml here (and uncomment the cert-manager + # release above) to switch TLS generation to cert-manager. + #- ci/values-cert-manager.yaml + # To enable OIDC with a local Keycloak instance, run the one-time + # setup task first, then uncomment the line below: + # mise run keycloak:k8s:setup + #- ci/values-keycloak.yaml + # To enable the Gateway API HTTPRoute (requires Envoy Gateway above): + #- ci/values-gateway.yaml + # To enable SPIFFE/SPIRE provider token grants (requires the + # spire-crds and spire releases above): + #- ci/values-spire.yaml + # To test multi-replica external PostgreSQL behavior: + #- ci/values-high-availability.yaml + setValueTemplates: + # Skaffold supplies complete image repositories. Clear the chart's + # shared registry so these values are used verbatim. + global.image.registry: '' + gateway.image.repository: '{{.IMAGE_REPO_openshell_gateway}}' + gateway.image.tag: '{{.IMAGE_TAG_openshell_gateway}}' + supervisor.image.repository: '{{.IMAGE_REPO_openshell_supervisor}}' + supervisor.image.tag: '{{.IMAGE_TAG_openshell_supervisor}}' + sandboxRuntime.image.repository: '{{.IMAGE_REPO_openshell_sandbox}}' + sandboxRuntime.image.tag: '{{.IMAGE_TAG_openshell_sandbox}}' +profiles: + # Full HA test path: installs Envoy Gateway and layers both HA replicas and + # Gateway API routing values onto the OpenShell release. + - name: high-availability + patches: + - op: add + path: /deploy/helm/releases/0 + value: + name: envoy-gateway + remoteChart: oci://docker.io/envoyproxy/gateway-helm + version: v1.7.2 + namespace: envoy-gateway-system + createNamespace: true + wait: true + - op: add + path: /deploy/helm/releases/1/valuesFiles/- + value: ci/values-high-availability.yaml + - op: add + path: /deploy/helm/releases/1/valuesFiles/- + value: ci/values-gateway.yaml + - name: credential-driver-kubernetes-secrets + patches: + - op: add + path: /deploy/helm/releases/0/valuesFiles/- + value: ci/values-credential-driver-kubernetes-secrets.yaml + - name: credential-driver-vault + patches: + - op: add + path: /deploy/helm/releases/0/valuesFiles/- + value: ci/values-credential-driver-vault.yaml diff --git a/charts/openshell/templates/_gateway-workload.tpl b/charts/openshell/templates/_gateway-workload.tpl new file mode 100644 index 000000000..0f2a8d614 --- /dev/null +++ b/charts/openshell/templates/_gateway-workload.tpl @@ -0,0 +1,306 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{/* +Gateway pod template shared by the StatefulSet and Deployment workload shapes. +*/}} +{{- define "openshell.gatewayPodTemplate" -}} +metadata: + annotations: + # Roll the gateway workload when the rendered gateway TOML changes - the + # gateway only reads /etc/openshell/gateway.toml at startup, so without + # this annotation a `helm upgrade` that only mutates the ConfigMap would + # leave pods running with stale config. + checksum/gateway-config: {{ include (print $.Template.BasePath "/gateway-config.yaml") . | sha256sum }} + {{- with .Values.podAnnotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + {{- with .Values.podLabels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + terminationGracePeriodSeconds: {{ .Values.podLifecycle.terminationGracePeriodSeconds }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 4 }} + {{- end }} + serviceAccountName: {{ include "openshell.serviceAccountName" . }} + {{- if .Values.server.hostGatewayIP }} + hostAliases: + - ip: {{ .Values.server.hostGatewayIP | quote }} + hostnames: + - host.docker.internal + - host.openshell.internal + {{- end }} + {{- with .Values.podSecurityContext }} + securityContext: + {{- toYaml . | nindent 4 }} + {{- end }} + containers: + - name: openshell-gateway + securityContext: + {{- toYaml .Values.securityContext | nindent 8 }} + image: {{ include "openshell.image" . | quote }} + imagePullPolicy: {{ .Values.gateway.image.pullPolicy | default .Values.global.image.pullPolicy }} + args: + - --config + - /etc/openshell/gateway.toml + {{- if not .Values.server.externalDbSecret }} + - --db-url + - {{ .Values.server.dbUrl | quote }} + {{- end }} + env: + - name: OPENSHELL_REPLICA_ID + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: OPENSHELL_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: OPENSHELL_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + {{- if eq (include "openshell.workloadKind" .) "deployment" }} + - name: OPENSHELL_POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: OPENSHELL_PEER_ENDPOINT + value: {{ printf "%s://$(OPENSHELL_POD_IP):%d" (ternary "http" "https" (default false .Values.server.disableTls)) (int .Values.service.port) | quote }} + {{- end }} + - name: OPENSHELL_SERVICE_ACCOUNT_NAME + value: {{ include "openshell.serviceAccountName" . | quote }} + - name: OPENSHELL_PEER_SERVICE_NAME + value: {{ include "openshell.peerServiceName" . | quote }} + - name: OPENSHELL_PEER_TOKEN_AUDIENCE + value: "openshell-gateway-peer" + - name: OPENSHELL_PEER_SERVICE_ACCOUNT_TOKEN_FILE + value: /var/run/secrets/openshell-peer/token + - name: OPENSHELL_PEER_POD_LABELS + value: {{ printf "app.kubernetes.io/name=%s,app.kubernetes.io/instance=%s" (include "openshell.name" .) .Release.Name | quote }} + {{- if not .Values.server.disableTls }} + - name: OPENSHELL_PEER_TLS_SERVER_NAME + value: {{ printf "%s.%s.svc.cluster.local" (include "openshell.fullname" .) .Release.Namespace | quote }} + {{- if or .Values.pkiInitJob.enabled .Values.certManager.enabled }} + - name: OPENSHELL_PEER_TLS_CA_FILE + value: /etc/openshell-tls/server/ca.crt + {{- end }} + {{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }} + - name: OPENSHELL_PEER_TLS_CERT_FILE + value: /etc/openshell-tls/peer-client/tls.crt + - name: OPENSHELL_PEER_TLS_KEY_FILE + value: /etc/openshell-tls/peer-client/tls.key + {{- end }} + {{- end }} + {{- if not (or .Values.server.credentialDrivers.kubernetesSecrets.enabled .Values.server.credentialDrivers.vault.enabled) }} + - name: {{ include "openshell.credentialStorageKeyEncryptionKeyEnvName" . }} + valueFrom: + secretKeyRef: + name: {{ include "openshell.credentialStorageKeyEncryptionKeySecretName" . }} + key: {{ include "openshell.credentialStorageKeyEncryptionKeySecretKey" . }} + {{- end }} + {{- if .Values.server.externalDbSecret }} + - name: OPENSHELL_DB_URL + valueFrom: + secretKeyRef: + name: {{ .Values.server.externalDbSecret }} + key: uri + {{- end }} + # Most gateway settings live in the ConfigMap-backed TOML file + # mounted at /etc/openshell/gateway.toml. Secret-bearing settings use + # env vars that the TOML references by name. Some process-level + # settings consumed by libraries outside gateway code also remain here. + {{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }} + # OIDC issuer custom-CA: rustls/reqwest read SSL_CERT_FILE for + # outbound TLS verification. This is a process-level env var + # consumed by the TLS stack itself, not by gateway code, so it + # cannot be represented in the gateway TOML schema. + - name: SSL_CERT_FILE + value: /etc/openshell-tls/oidc-ca/ca.crt + {{- end }} + - name: OPENSHELL_TELEMETRY_ENABLED + value: {{ .Values.server.telemetryEnabled | quote }} + {{- if .Values.server.providerTokenGrants.spiffe.enabled }} + - name: OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET + value: {{ .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }} + {{- end }} + volumeMounts: + {{- if eq (include "openshell.workloadKind" .) "statefulset" }} + - name: openshell-data + mountPath: /var/openshell + {{- end }} + # ConfigMap directory mounts expose keys through atomic-writer symlinks, + # while the gateway intentionally rejects symlinked configuration. + # The checksum annotation above rolls pods when this subPath changes. + - name: gateway-config + mountPath: /etc/openshell/gateway.toml + subPath: gateway.toml + readOnly: true + - name: sandbox-jwt + mountPath: /etc/openshell-jwt + readOnly: true + - name: gateway-peer-token + mountPath: /var/run/secrets/openshell-peer + readOnly: true + {{- if not .Values.server.disableTls }} + - name: tls-cert + mountPath: /etc/openshell-tls/server + readOnly: true + {{- if .Values.certManager.serverIssuerRef.name }} + - name: tls-external-cert + mountPath: /etc/openshell-tls/server-external + readOnly: true + {{- end }} + {{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }} + - name: peer-client-tls + mountPath: /etc/openshell-tls/peer-client + readOnly: true + - name: tls-client-ca + mountPath: /etc/openshell-tls/client-ca + readOnly: true + {{- end }} + {{- end }} + {{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }} + - name: oidc-ca + mountPath: /etc/openshell-tls/oidc-ca + readOnly: true + {{- end }} + {{- if and .Values.server.credentialDrivers.vault.enabled .Values.server.credentialDrivers.vault.caConfigMapName }} + - name: vault-ca + mountPath: /etc/openshell-tls/vault-ca + readOnly: true + {{- end }} + {{- if .Values.upstreamProxy.caBundle.configMapName }} + - name: upstream-proxy-ca + mountPath: /etc/openshell-tls/proxy-ca + readOnly: true + {{- end }} + {{- if .Values.server.providerTokenGrants.spiffe.enabled }} + - name: spiffe-workload-api + mountPath: {{ dir .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }} + readOnly: true + {{- end }} + ports: + - name: grpc + containerPort: {{ .Values.service.port }} + protocol: TCP + - name: health + containerPort: {{ .Values.service.healthPort }} + protocol: TCP + {{- if .Values.service.metricsPort }} + - name: metrics + containerPort: {{ .Values.service.metricsPort }} + protocol: TCP + {{- end }} + startupProbe: + httpGet: + path: /healthz + port: health + periodSeconds: {{ .Values.probes.startup.periodSeconds }} + timeoutSeconds: {{ .Values.probes.startup.timeoutSeconds }} + failureThreshold: {{ .Values.probes.startup.failureThreshold }} + livenessProbe: + httpGet: + path: /healthz + port: health + initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }} + periodSeconds: {{ .Values.probes.liveness.periodSeconds }} + timeoutSeconds: {{ .Values.probes.liveness.timeoutSeconds }} + failureThreshold: {{ .Values.probes.liveness.failureThreshold }} + readinessProbe: + httpGet: + path: /readyz + port: health + initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }} + periodSeconds: {{ .Values.probes.readiness.periodSeconds }} + timeoutSeconds: {{ .Values.probes.readiness.timeoutSeconds }} + failureThreshold: {{ .Values.probes.readiness.failureThreshold }} + resources: + {{- toYaml .Values.resources | nindent 8 }} + volumes: + - name: gateway-config + configMap: + name: {{ include "openshell.fullname" . }}-config + - name: sandbox-jwt + secret: + secretName: {{ include "openshell.sandboxJwtSecretName" . }} + defaultMode: {{ .Values.server.sandboxJwt.secretDefaultMode | default 0400 }} + - name: gateway-peer-token + projected: + defaultMode: 0400 + sources: + - serviceAccountToken: + path: token + audience: openshell-gateway-peer + expirationSeconds: 3600 + {{- if not .Values.server.disableTls }} + - name: tls-cert + secret: + secretName: {{ .Values.server.tls.certSecretName }} + {{- if .Values.certManager.serverIssuerRef.name }} + - name: tls-external-cert + secret: + secretName: {{ include "openshell.fullname" . }}-server-external-tls + {{- end }} + {{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }} + - name: peer-client-tls + secret: + secretName: {{ .Values.server.tls.clientTlsSecretName }} + - name: tls-client-ca + secret: + {{- if or (and .Values.pkiInitJob.enabled (not .Values.certManager.enabled)) (and .Values.certManager.enabled .Values.certManager.clientCaFromServerTlsSecret) }} + secretName: {{ .Values.server.tls.certSecretName }} + items: + - key: ca.crt + path: ca.crt + {{- else }} + secretName: {{ .Values.server.tls.clientCaSecretName }} + {{- end }} + {{- end }} + {{- end }} + {{- if and .Values.server.oidc.issuer .Values.server.oidc.caConfigMapName }} + - name: oidc-ca + configMap: + name: {{ .Values.server.oidc.caConfigMapName }} + {{- end }} + {{- if and .Values.server.credentialDrivers.vault.enabled .Values.server.credentialDrivers.vault.caConfigMapName }} + - name: vault-ca + configMap: + name: {{ .Values.server.credentialDrivers.vault.caConfigMapName }} + items: + - key: ca.crt + path: ca.crt + {{- end }} + {{- if .Values.upstreamProxy.caBundle.configMapName }} + - name: upstream-proxy-ca + configMap: + name: {{ .Values.upstreamProxy.caBundle.configMapName | quote }} + items: + # The mounted filename stays fixed so the rendered proxy_ca_bundle + # path does not depend on the operator's ConfigMap key. + - key: {{ .Values.upstreamProxy.caBundle.key | default "ca.crt" | quote }} + path: ca.crt + {{- end }} + {{- if .Values.server.providerTokenGrants.spiffe.enabled }} + - name: spiffe-workload-api + csi: + driver: csi.spiffe.io + readOnly: true + {{- end }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/charts/openshell/templates/_helpers.tpl b/charts/openshell/templates/_helpers.tpl new file mode 100644 index 000000000..ab4245875 --- /dev/null +++ b/charts/openshell/templates/_helpers.tpl @@ -0,0 +1,421 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{/* +Expand the name of the chart. +*/}} +{{- define "openshell.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +*/}} +{{- define "openshell.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "openshell.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "openshell.labels" -}} +helm.sh/chart: {{ include "openshell.chart" . }} +{{ include "openshell.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "openshell.selectorLabels" -}} +app.kubernetes.io/name: {{ include "openshell.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "openshell.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "openshell.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* +Create the name of the service account assigned to sandbox pods +*/}} +{{- define "openshell.sandboxServiceAccountName" -}} +{{- if .Values.sandboxServiceAccount.create }} +{{- default (printf "%s-sandbox" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-") .Values.sandboxServiceAccount.name }} +{{- else }} +{{- default "default" .Values.sandboxServiceAccount.name }} +{{- end }} +{{- end }} + +{{/* +Whether this chart owns workspace-scoped resources. Missing legacy values +default to enabled so upgrades with --reuse-values preserve the old topology. +*/}} +{{- define "openshell.workspaceResourcesEnabled" -}} +{{- $workspaceResources := .Values.workspaceResources | default dict -}} +{{- $enabled := true -}} +{{- if hasKey $workspaceResources "enabled" -}} +{{- $enabled = get $workspaceResources "enabled" -}} +{{- end -}} +{{- if $enabled -}}true{{- end -}} +{{- end }} + +{{/* +Whether this chart owns gateway RBAC objects. Missing legacy values default to +enabled so upgrades with --reuse-values preserve the old topology. +*/}} +{{- define "openshell.rbacCreate" -}} +{{- $rbac := .Values.rbac | default dict -}} +{{- $create := true -}} +{{- if hasKey $rbac "create" -}} +{{- $create = get $rbac "create" -}} +{{- end -}} +{{- if $create -}}true{{- end -}} +{{- end }} + +{{/* +The rbac.clusterScoped values map, tolerating missing legacy values. +*/}} +{{- define "openshell.clusterScopedRbacValues" -}} +{{- $rbac := .Values.rbac | default dict -}} +{{- $clusterScoped := dict -}} +{{- if hasKey $rbac "clusterScoped" -}} +{{- $clusterScoped = get $rbac "clusterScoped" | default dict -}} +{{- end -}} +{{- toYaml $clusterScoped -}} +{{- end }} + +{{/* +Whether this chart owns the cluster-scoped ClusterRole and ClusterRoleBinding. +Disable for a namespace-admin install where a cluster-admin applies them +separately. Missing legacy values default to enabled. +*/}} +{{- define "openshell.clusterRbacCreate" -}} +{{- if include "openshell.rbacCreate" . -}} +{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}} +{{- $create := true -}} +{{- if hasKey $clusterScoped "create" -}} +{{- $create = get $clusterScoped "create" -}} +{{- end -}} +{{- if $create -}}true{{- end -}} +{{- end -}} +{{- end }} + +{{/* +Name of the gateway ClusterRole. The release namespace is part of the default +name so multiple releases on one cluster do not collide. +*/}} +{{- define "openshell.clusterRoleName" -}} +{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}} +{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}} +{{- default $default (get $clusterScoped "clusterRoleName") -}} +{{- end }} + +{{/* +Name of the gateway ClusterRoleBinding. +*/}} +{{- define "openshell.clusterRoleBindingName" -}} +{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}} +{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}} +{{- default $default (get $clusterScoped "clusterRoleBindingName") -}} +{{- end }} + +{{/* Gateway image reference. A digest takes precedence over a tag. */}} +{{- define "openshell.image" -}} +{{- $image := .Values.gateway.image -}} +{{- $global := .Values.global.image -}} +{{- $registry := $image.registry | default $global.registry -}} +{{- $repository := ternary (printf "%s/%s" $registry $image.repository) $image.repository (ne $registry "") -}} +{{- if $image.digest -}} +{{- printf "%s@%s" $repository $image.digest -}} +{{- else -}} +{{- printf "%s:%s" $repository ($image.tag | default $global.tag | default .Chart.AppVersion) -}} +{{- end }} +{{- end }} + +{{/* Sandbox image reference. A digest takes precedence over a tag. */}} +{{- define "openshell.sandboxImage" -}} +{{- $image := .Values.sandbox.image -}} +{{- if $image.digest -}} +{{- printf "%s@%s" $image.repository $image.digest -}} +{{- else -}} +{{- printf "%s:%s" $image.repository ($image.tag | default "latest") -}} +{{- end }} +{{- end }} + +{{/* Official sandbox runtime repository used by the gateway's built-in default. */}} +{{- define "openshell.defaultSandboxRuntimeRepository" -}} +ghcr.io/nvidia/openshell/sandbox +{{- end }} + +{{/* Whether Helm must propagate a sandbox runtime image override. */}} +{{- define "openshell.sandboxRuntimeImageOverrideEnabled" -}} +{{- $defaultRepository := include "openshell.defaultSandboxRuntimeRepository" . -}} +{{- $global := .Values.global.image -}} +{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}} +{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}} +{{- if or (ne $repository $defaultRepository) .Values.sandboxRuntime.image.tag .Values.sandboxRuntime.image.digest .Values.global.image.tag -}}true{{- end -}} +{{- end }} + +{{/* Sandbox runtime image override. */}} +{{- define "openshell.sandboxRuntimeImage" -}} +{{- $global := .Values.global.image -}} +{{- $registry := .Values.sandboxRuntime.image.registry | default $global.registry -}} +{{- $repository := ternary (printf "%s/%s" $registry .Values.sandboxRuntime.image.repository) .Values.sandboxRuntime.image.repository (ne $registry "") -}} +{{- if .Values.sandboxRuntime.image.digest -}} +{{- printf "%s@%s" $repository .Values.sandboxRuntime.image.digest -}} +{{- else -}} +{{- $tag := .Values.sandboxRuntime.image.tag | default $global.tag | default .Chart.AppVersion -}} +{{- printf "%s:%s" $repository $tag -}} +{{- end -}} +{{- end }} + +{{/* +Whether the gateway listener should verify client certificates (mTLS). +An explicit empty server.tls.clientCaSecretName disables client-CA wiring in +both gateway.toml and the workload, overriding built-in PKI and cert-manager +defaults. +*/}} +{{- define "openshell.gatewayClientCaEnabled" -}} +{{- if .Values.server.disableTls -}} +{{- else if not .Values.server.tls.enableMtls -}} +{{- else if eq .Values.server.tls.clientCaSecretName "" -}} +{{- else if or .Values.server.tls.clientCaSecretName (and .Values.pkiInitJob.enabled (not .Values.certManager.enabled)) (and .Values.certManager.enabled .Values.certManager.clientCaFromServerTlsSecret) -}} +true +{{- end -}} +{{- end -}} + +{{/* Supervisor image override. */}} +{{- define "openshell.supervisorImage" -}} +{{- $global := .Values.global.image -}} +{{- $registry := .Values.supervisor.image.registry | default $global.registry -}} +{{- $repository := ternary (printf "%s/%s" $registry .Values.supervisor.image.repository) .Values.supervisor.image.repository (ne $registry "") -}} +{{- if .Values.supervisor.image.digest -}} +{{- printf "%s@%s" $repository .Values.supervisor.image.digest -}} +{{- else -}} +{{- $tag := .Values.supervisor.image.tag | default $global.tag | default .Chart.AppVersion -}} +{{- printf "%s:%s" $repository $tag -}} +{{- end }} +{{- end }} + +{{/* +Namespaced Issuer (selfSigned) for cert-manager CA bootstrap. +*/}} +{{- define "openshell.issuerSelfSigned" -}} +{{- printf "%s-selfsigned" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Namespace where sandbox pods are created. An explicit +.Values.server.sandboxNamespace is used verbatim. Otherwise it defaults to +.Release.Namespace so `helm install -n my-ns` works without extra overrides. +*/}} +{{- define "openshell.sandboxNamespace" -}} +{{- .Values.server.sandboxNamespace | default .Release.Namespace -}} +{{- end }} + +{{/* +Secrets in the sandbox namespace whose contents the Kubernetes driver stages +into per-generation Secrets in workspace namespaces, as a JSON array. Empty in +shared workspace mode. +*/}} +{{- define "openshell.workspaceSecretSourceNames" -}} +{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} +{{- $names := list -}} +{{- if and (ne $workspaceMode "shared") (not .Values.server.disableTls) -}} +{{- $names = append $names .Values.server.tls.clientTlsSecretName -}} +{{- end -}} +{{- if eq $workspaceMode "managed" -}} +{{- range .Values.server.sandboxImagePullSecrets -}} +{{- if .name -}} +{{- $names = append $names .name -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- uniq $names | toJson -}} +{{- end }} + +{{/* +Namespace where Kubernetes Secret-backed provider credentials live. +*/}} +{{- define "openshell.credentialKubernetesSecretsNamespace" -}} +{{- .Values.server.credentialDrivers.kubernetesSecrets.namespace | default .Release.Namespace -}} +{{- end }} + +{{/* +Name of the Secret holding the default credential storage key-encryption key. +When server.credentialStorage.existingSecret is set, returns that name instead +of the chart-generated name (for GitOps / helm-template workflows). +*/}} +{{- define "openshell.credentialStorageKeyEncryptionKeySecretName" -}} +{{- if .Values.server.credentialStorage.existingSecret -}} +{{- .Values.server.credentialStorage.existingSecret -}} +{{- else -}} +{{- printf "%s-credential-storage-key-encryption-key" (include "openshell.fullname" .) | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end }} + +{{/* +Key inside the default credential storage key-encryption key Secret. +*/}} +{{- define "openshell.credentialStorageKeyEncryptionKeySecretKey" -}} +key-encryption-key +{{- end }} + +{{/* +Gateway environment variable used to pass the default credential storage key-encryption key. +*/}} +{{- define "openshell.credentialStorageKeyEncryptionKeyEnvName" -}} +OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY +{{- end }} + +{{/* +Name of the Secret holding gateway-minted sandbox JWT signing material. +*/}} +{{- define "openshell.sandboxJwtSecretName" -}} +{{- .Values.server.sandboxJwt.signingSecretName | default (printf "%s-jwt-keys" (include "openshell.fullname" .)) -}} +{{- end }} + +{{- define "openshell.peerServiceName" -}} +{{- printf "%s-peer" (include "openshell.fullname" .) -}} +{{- end }} + +{{/* +gRPC endpoint sandbox pods use to call back into the gateway. An explicit +.Values.server.grpcEndpoint is used verbatim. Otherwise it is derived from +the in-cluster Service DNS, release namespace, service port, and disableTls +flag — so the default value works for any release name or namespace without +override. +*/}} +{{- define "openshell.grpcEndpoint" -}} +{{- if .Values.server.grpcEndpoint -}} +{{- .Values.server.grpcEndpoint -}} +{{- else -}} +{{- $scheme := ternary "http" "https" (default false .Values.server.disableTls) -}} +{{- printf "%s://%s.%s.svc.cluster.local:%d" $scheme (include "openshell.fullname" .) .Release.Namespace (int .Values.service.port) -}} +{{- end -}} +{{- end }} + +{{/* +Default server certificate DNS SANs derived from the release name and namespace. +Returns a YAML list. Append extra SANs from values with range loops. +*/}} +{{- define "openshell.defaultServerDnsNames" -}} +{{- $name := include "openshell.fullname" . -}} +{{- $ns := .Release.Namespace -}} +{{- list $name + (printf "%s.%s.svc" $name $ns) + (printf "%s.%s.svc.cluster.local" $name $ns) + "localhost" + (printf "%s.localhost" $name) + (printf "*.%s.localhost" $name) + "host.docker.internal" + "host.containers.internal" + | toYaml }} +{{- end }} + +{{/* +Name of the ConfigMap holding the backend CA for BackendTLSPolicy validation. +*/}} +{{- define "openshell.backendCaConfigMapName" -}} +{{- .Values.grpcRoute.backendTLSPolicy.caCertificateConfigMapName | default (printf "%s-backend-ca" (include "openshell.fullname" .)) -}} +{{- end }} + +{{/* +Gateway workload kind. StatefulSet is the default because the default SQLite +database requires persistent per-pod storage. +*/}} +{{- define "openshell.workloadKind" -}} +{{- $workload := .Values.workload | default dict -}} +{{- if not (kindIs "map" $workload) -}} +{{- fail "workload must be a map with kind and allowMultiReplicaStatefulSet fields." -}} +{{- end -}} +{{- default "statefulset" (get $workload "kind") | lower -}} +{{- end }} + +{{/* +Translate chart image pull policy values to the canonical gateway vocabulary. +The Kubernetes spellings remain accepted so existing values files continue to +work across the schema-v2 chart upgrade. +*/}} +{{- define "openshell.canonicalImagePullPolicy" -}} +{{- $policy := printf "%v" . -}} +{{- if eq $policy "Always" -}} +always +{{- else if eq $policy "IfNotPresent" -}} +if_not_present +{{- else if eq $policy "Never" -}} +never +{{- else if has $policy (list "always" "if_not_present" "never") -}} +{{- $policy -}} +{{- else -}} +{{- fail (printf "image pull policy %q must be one of: always, if_not_present, never, Always, IfNotPresent, Never" $policy) -}} +{{- end -}} +{{- end }} + +{{/* +Validate chart values that Helm would otherwise accept silently. +*/}} +{{- define "openshell.validateValues" -}} +{{- $workloadKind := include "openshell.workloadKind" . -}} +{{- $workload := .Values.workload | default dict -}} +{{- $replicaCount := int (default 1 .Values.replicaCount) -}} +{{- if and (hasKey .Values "postgres") (kindIs "map" .Values.postgres) (hasKey .Values.postgres "enabled") -}} +{{- fail "postgres.enabled was removed; the OpenShell chart no longer deploys PostgreSQL. Provision PostgreSQL separately and set server.externalDbSecret to a Secret containing a PostgreSQL URI." -}} +{{- end -}} +{{- if not (or (eq $workloadKind "statefulset") (eq $workloadKind "deployment")) -}} +{{- fail "workload.kind must be one of: statefulset, deployment." -}} +{{- end -}} +{{- if and (eq $workloadKind "deployment") (not .Values.server.externalDbSecret) -}} +{{- fail "workload.kind=deployment requires server.externalDbSecret; use workload.kind=statefulset for the default SQLite database." -}} +{{- end -}} +{{- if and (gt $replicaCount 1) (not .Values.server.externalDbSecret) -}} +{{- fail "replicaCount > 1 requires server.externalDbSecret; multiple gateway replicas cannot share the default per-pod SQLite database." -}} +{{- end -}} +{{- if and (eq $workloadKind "statefulset") (gt $replicaCount 1) (not (get $workload "allowMultiReplicaStatefulSet" | default false)) -}} +{{- fail "replicaCount > 1 with workload.kind=statefulset requires workload.allowMultiReplicaStatefulSet=true; use workload.kind=deployment for external database-backed multi-replica gateways." -}} +{{- end -}} +{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} +{{- if not (has $workspaceMode (list "shared" "managed" "operator")) -}} +{{- fail "server.drivers.kubernetes.workspaceMode must be one of: shared, managed, operator." -}} +{{- end -}} +{{- $credentialDrivers := list -}} +{{- if .Values.server.credentialDrivers.kubernetesSecrets.enabled -}} +{{- $credentialDrivers = append $credentialDrivers "kubernetes-secrets" -}} +{{- end -}} +{{- if .Values.server.credentialDrivers.vault.enabled -}} +{{- $credentialDrivers = append $credentialDrivers "vault" -}} +{{- end -}} +{{- if gt (len $credentialDrivers) 1 -}} +{{- fail "only one external server.credentialDrivers backend can be enabled at a time." -}} +{{- end -}} +{{- if kindIs "invalid" .Values.server.tls.clientCaSecretName -}} +{{- fail "server.tls.clientCaSecretName cannot be null; omit the key to use the chart default (openshell-server-client-ca), or set to \"\" to disable client certificate verification for HTTPS-only mode" -}} +{{- end -}} +{{- end }} diff --git a/charts/openshell/templates/agent-sandbox-preflight.yaml b/charts/openshell/templates/agent-sandbox-preflight.yaml new file mode 100644 index 000000000..83a742f67 --- /dev/null +++ b/charts/openshell/templates/agent-sandbox-preflight.yaml @@ -0,0 +1,11 @@ +{{/* +SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 +*/}} +{{- if .Values.agentSandbox.preflight.enabled }} +{{- $v1beta1 := or (.Capabilities.APIVersions.Has "agents.x-k8s.io/v1beta1") (.Capabilities.APIVersions.Has "agents.x-k8s.io/v1beta1/Sandbox") }} +{{- $v1alpha1 := or (.Capabilities.APIVersions.Has "agents.x-k8s.io/v1alpha1") (.Capabilities.APIVersions.Has "agents.x-k8s.io/v1alpha1/Sandbox") }} +{{- if not (or $v1beta1 $v1alpha1) }} +{{- fail "Agent Sandbox is required but neither agents.x-k8s.io/v1beta1 nor agents.x-k8s.io/v1alpha1 is served by this cluster. Install the Agent Sandbox CRDs and controller before deploying OpenShell; see deploy/helm/openshell/README.md. Set agentSandbox.preflight.enabled=false only for offline rendering." }} +{{- end }} +{{- end }} diff --git a/charts/openshell/templates/backend-tls-policy.yaml b/charts/openshell/templates/backend-tls-policy.yaml new file mode 100644 index 000000000..8d7a4fd55 --- /dev/null +++ b/charts/openshell/templates/backend-tls-policy.yaml @@ -0,0 +1,29 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.grpcRoute.backendTLSPolicy.enabled }} +{{- if .Values.server.disableTls }} +{{- fail "grpcRoute.backendTLSPolicy requires the gateway pod to serve TLS; set server.disableTls=false" }} +{{- end }} +{{- if .Values.server.tls.enableMtls }} +{{- fail "grpcRoute.backendTLSPolicy requires mTLS to be disabled because the Gateway proxy cannot present client certificates to the backend; set server.tls.enableMtls=false" }} +{{- end }} +apiVersion: gateway.networking.k8s.io/v1 +kind: BackendTLSPolicy +metadata: + name: {{ include "openshell.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + targetRefs: + - group: "" + kind: Service + name: {{ include "openshell.fullname" . }} + validation: + caCertificateRefs: + - group: "" + kind: ConfigMap + name: {{ include "openshell.backendCaConfigMapName" . }} + hostname: {{ default (printf "%s.%s.svc.cluster.local" (include "openshell.fullname" .) .Release.Namespace) .Values.grpcRoute.backendTLSPolicy.hostname }} +{{- end }} diff --git a/charts/openshell/templates/cert-manager-pki.yaml b/charts/openshell/templates/cert-manager-pki.yaml new file mode 100644 index 000000000..838534e62 --- /dev/null +++ b/charts/openshell/templates/cert-manager-pki.yaml @@ -0,0 +1,157 @@ +{{- if .Values.certManager.enabled }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: {{ include "openshell.issuerSelfSigned" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + selfSigned: {} +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ include "openshell.fullname" . }}-ca + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + isCA: true + commonName: openshell-ca + secretName: {{ .Values.certManager.caSecretName | quote }} + privateKey: + algorithm: ECDSA + size: 256 + issuerRef: + name: {{ include "openshell.issuerSelfSigned" . }} + kind: Issuer + group: cert-manager.io +--- +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: {{ include "openshell.fullname" . }}-ca-issuer + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + ca: + secretName: {{ .Values.certManager.caSecretName | quote }} +--- +{{- $externalServerIssuer := .Values.certManager.serverIssuerRef.name }} +{{- if and (not .Values.certManager.clientCaFromServerTlsSecret) (eq .Values.server.tls.clientCaSecretName "openshell-server-client-ca") }} +{{- fail "certManager.clientCaFromServerTlsSecret is false but server.tls.clientCaSecretName is still the default (openshell-server-client-ca), which nothing creates when cert-manager owns TLS. Set server.tls.clientCaSecretName to the secret containing the client CA (e.g. the value of certManager.caSecretName, which defaults to openshell-ca-tls), or set it to empty to disable mTLS client verification." }} +{{- end }} +{{- if $externalServerIssuer }} +{{- if not .Values.certManager.serverDnsNames }} +{{- fail "certManager.serverIssuerRef.name is set but certManager.serverDnsNames is empty — the external certificate requires at least one externally-resolvable DNS name." }} +{{- end }} +{{- range .Values.certManager.serverDnsNames }} +{{- /* Single-label names (e.g. "openshell") are also rejected by ACME CAs but are intentionally not checked here — the guard targets recognisable internal-network patterns. */ -}} +{{- if or (eq . "localhost") (hasSuffix ".localhost" .) (hasSuffix ".svc.cluster.local" .) (hasSuffix ".svc" .) (eq . "host.docker.internal") (eq . "host.containers.internal") }} +{{- fail (printf "certManager.serverIssuerRef.name is set (external issuer) but certManager.serverDnsNames contains %q — external CAs (e.g. ACME / Let's Encrypt) reject internal-only names per CA/Browser Forum baseline requirements. Override certManager.serverDnsNames with your externally-resolvable hostname(s)." .) }} +{{- end }} +{{- end }} +{{- end }} +# Internal server certificate — always issued by the chart’s own CA with +# internal SANs. Supervisors connect via internal hostnames and verify +# this cert against the chart CA they already trust. +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ include "openshell.fullname" . }}-server + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + secretName: {{ .Values.server.tls.certSecretName | quote }} + duration: {{ .Values.certManager.certificateDuration | quote }} + renewBefore: {{ .Values.certManager.certificateRenewBefore | quote }} + commonName: {{ include "openshell.fullname" . }} + dnsNames: + {{- range (include "openshell.defaultServerDnsNames" . | fromYamlArray) }} + - {{ . | quote }} + {{- end }} + {{- if not $externalServerIssuer }} + {{- range .Values.certManager.serverDnsNames }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.certManager.serverIpAddresses }} + ipAddresses: + {{- toYaml .Values.certManager.serverIpAddresses | nindent 4 }} + {{- end }} + privateKey: + algorithm: ECDSA + size: 256 + usages: + - server auth + - digital signature + - key encipherment + issuerRef: + name: {{ include "openshell.fullname" . }}-ca-issuer + kind: Issuer + group: cert-manager.io +{{- if $externalServerIssuer }} +--- +# External server certificate — issued by the operator-configured issuer +# (e.g. ACME/Let’s Encrypt) with only externally-resolvable SANs. +# The gateway uses SNI to present this cert for external hostnames. +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ include "openshell.fullname" . }}-server-external + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + secretName: {{ include "openshell.fullname" . }}-server-external-tls + duration: {{ .Values.certManager.certificateDuration | quote }} + renewBefore: {{ .Values.certManager.certificateRenewBefore | quote }} + {{- if .Values.certManager.serverDnsNames }} + commonName: {{ first .Values.certManager.serverDnsNames | quote }} + {{- end }} + dnsNames: + {{- range .Values.certManager.serverDnsNames }} + - {{ . | quote }} + {{- end }} + privateKey: + algorithm: ECDSA + size: 256 + usages: + - server auth + - digital signature + - key encipherment + issuerRef: + name: {{ .Values.certManager.serverIssuerRef.name }} + kind: {{ .Values.certManager.serverIssuerRef.kind | default "Issuer" }} + group: {{ .Values.certManager.serverIssuerRef.group | default "cert-manager.io" }} +{{- end }} +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ include "openshell.fullname" . }}-client + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + secretName: {{ .Values.server.tls.clientTlsSecretName | quote }} + duration: {{ .Values.certManager.certificateDuration | quote }} + renewBefore: {{ .Values.certManager.certificateRenewBefore | quote }} + commonName: openshell-client + privateKey: + algorithm: ECDSA + size: 256 + usages: + - client auth + - digital signature + - key encipherment + issuerRef: + name: {{ include "openshell.fullname" . }}-ca-issuer + kind: Issuer + group: cert-manager.io +{{- end }} diff --git a/charts/openshell/templates/certgen.yaml b/charts/openshell/templates/certgen.yaml new file mode 100644 index 000000000..f7c9a751d --- /dev/null +++ b/charts/openshell/templates/certgen.yaml @@ -0,0 +1,183 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- if or .Values.pkiInitJob.enabled .Values.certManager.enabled }} +{{- $hookName := printf "%s-certgen" (include "openshell.fullname" .) }} +{{- $ns := .Release.Namespace }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ $hookName }} + namespace: {{ $ns }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-weight: "-30" + helm.sh/hook-delete-policy: before-hook-creation +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ $hookName }} + namespace: {{ $ns }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-weight: "-30" + helm.sh/hook-delete-policy: before-hook-creation +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create"] + {{- if .Values.grpcRoute.backendTLSPolicy.enabled }} + - apiGroups: [""] + resources: ["configmaps"] + verbs: ["get", "create", "update"] + {{- end }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ $hookName }} + namespace: {{ $ns }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-weight: "-30" + helm.sh/hook-delete-policy: before-hook-creation +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ $hookName }} +subjects: + - kind: ServiceAccount + name: {{ $hookName }} + namespace: {{ $ns }} +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ $hookName }} + namespace: {{ $ns }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/hook: pre-install,pre-upgrade + helm.sh/hook-weight: "-20" + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded +spec: + backoffLimit: 3 + activeDeadlineSeconds: {{ add .Values.pkiInitJob.timeoutSeconds 30 }} + ttlSecondsAfterFinished: 300 + template: + metadata: + labels: + {{- include "openshell.selectorLabels" . | nindent 8 }} + spec: + restartPolicy: OnFailure + serviceAccountName: {{ $hookName }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: certgen + image: {{ include "openshell.image" . | quote }} + imagePullPolicy: {{ .Values.gateway.image.pullPolicy | default .Values.global.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + command: ["/usr/local/bin/openshell-gateway"] + args: + - generate-certs + {{- if .Values.certManager.enabled }} + - --jwt-only + {{- else }} + - --server-secret-name={{ .Values.server.tls.certSecretName }} + - --client-secret-name={{ .Values.server.tls.clientTlsSecretName }} + {{- end }} + - --jwt-secret-name={{ include "openshell.sandboxJwtSecretName" . }} + {{- if and .Values.pkiInitJob.enabled (not .Values.certManager.enabled) }} + {{- range (include "openshell.defaultServerDnsNames" . | fromYamlArray) }} + - --server-san={{ . }} + {{- end }} + - --server-san=127.0.0.1 + {{- range .Values.pkiInitJob.serverDnsNames }} + - --server-san={{ . }} + {{- end }} + {{- range .Values.pkiInitJob.serverIpAddresses }} + - --server-san={{ . }} + {{- end }} + {{- end }} + {{- if and .Values.grpcRoute.backendTLSPolicy.enabled (not .Values.certManager.enabled) }} + - --backend-ca-configmap-name={{ include "openshell.backendCaConfigMapName" . }} + {{- end }} +{{- if and .Values.certManager.enabled .Values.grpcRoute.backendTLSPolicy.enabled }} +--- +# Post-install Job that polls for the cert-manager-issued server Secret and +# creates / reconciles the backend CA ConfigMap. Runs after regular resources +# (including the cert-manager Certificate objects) are applied so cert-manager +# has started issuing before the poll begins. +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ $hookName }}-backend-ca + namespace: {{ $ns }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/hook: post-install,post-upgrade + helm.sh/hook-weight: "0" + helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded +spec: + backoffLimit: 3 + activeDeadlineSeconds: {{ add .Values.pkiInitJob.timeoutSeconds 30 }} + ttlSecondsAfterFinished: 300 + template: + metadata: + labels: + {{- include "openshell.selectorLabels" . | nindent 8 }} + spec: + restartPolicy: OnFailure + serviceAccountName: {{ $hookName }} + {{- with .Values.imagePullSecrets }} + imagePullSecrets: + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: certgen + image: {{ include "openshell.image" . | quote }} + imagePullPolicy: {{ .Values.gateway.image.pullPolicy | default .Values.global.image.pullPolicy }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + command: ["/usr/local/bin/openshell-gateway"] + args: + - generate-certs + - --jwt-only + - --jwt-secret-name={{ include "openshell.sandboxJwtSecretName" . }} + - --backend-ca-configmap-name={{ include "openshell.backendCaConfigMapName" . }} + - --backend-ca-source-secret={{ .Values.server.tls.certSecretName }} + - --backend-ca-poll-timeout-seconds={{ .Values.pkiInitJob.timeoutSeconds }} + {{- if .Values.pkiInitJob.failOnTimeout }} + - --backend-ca-fail-on-timeout + {{- end }} +{{- end }} +{{- end }} diff --git a/charts/openshell/templates/clusterrole.yaml b/charts/openshell/templates/clusterrole.yaml new file mode 100644 index 000000000..85d10c6b6 --- /dev/null +++ b/charts/openshell/templates/clusterrole.yaml @@ -0,0 +1,132 @@ +{{- if include "openshell.clusterRbacCreate" . }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ include "openshell.clusterRoleName" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +rules: + - apiGroups: ["node.k8s.io"] + resources: ["runtimeclasses"] + verbs: ["get"] + - apiGroups: ["scheduling.k8s.io"] + resources: ["priorityclasses"] + verbs: ["get"] + {{- if and (ne $workspaceMode "shared") .Values.server.drivers.kubernetes.allowDriverConfig }} + - apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + {{- end }} + # Validate projected ServiceAccount tokens during sandbox bootstrap and + # internal gateway peer authentication. + - apiGroups: + - authentication.k8s.io + resources: + - tokenreviews + verbs: + - create + - apiGroups: + - "" + resources: + - nodes + verbs: + - get + - list + - watch + # Read namespace annotations for OpenShift SCC UID/GID range resolution. + # Managed/operator modes additionally need list+watch for cluster-wide + # namespace discovery. Managed mode needs create+delete for namespace + # lifecycle. + - apiGroups: + - "" + resources: + - namespaces + verbs: + - get + {{- if ne $workspaceMode "shared" }} + - list + - watch + {{- end }} + {{- if eq $workspaceMode "managed" }} + - create + - delete + {{- end }} + {{- if ne $workspaceMode "shared" }} + # Cluster-wide sandbox CRD access for managed/operator workspace modes. + - apiGroups: + - agents.x-k8s.io + resources: + - sandboxes + - sandboxes/status + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - pods + verbs: + - create + - delete + - get + - list + - patch + - watch + {{- end }} + {{- if ne $workspaceMode "shared" }} + - apiGroups: [""] + resources: ["services"] + verbs: ["create", "get"] + - apiGroups: ["networking.k8s.io"] + resources: ["networkpolicies"] + verbs: ["create", "get"] + {{- end }} + {{- if eq $workspaceMode "managed" }} + # Operator-mode namespaces receive Secret permissions from the openshell-workspace + # chart Role instead. + - apiGroups: [""] + resources: ["secrets"] + # Bootstrap and image-pull Secrets are generation-scoped. Recovery deletes + # them by exact name. + verbs: ["create", "delete"] + {{- end }} + {{- if eq $workspaceMode "managed" }} + # ServiceAccount creation in managed namespaces. + - apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - create + - get + {{- if .Values.networkPolicy.enabled }} + # Apply gateway-only SSH ingress isolation in managed namespaces. + - apiGroups: + - networking.k8s.io + resources: + - networkpolicies + verbs: + - get + - create + - patch + - update + {{- end }} + {{- end }} +{{- end }} diff --git a/charts/openshell/templates/clusterrolebinding.yaml b/charts/openshell/templates/clusterrolebinding.yaml new file mode 100644 index 000000000..8de246730 --- /dev/null +++ b/charts/openshell/templates/clusterrolebinding.yaml @@ -0,0 +1,19 @@ +{{- if include "openshell.clusterRbacCreate" . }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ include "openshell.clusterRoleBindingName" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ include "openshell.clusterRoleName" . }} +subjects: + - kind: ServiceAccount + name: {{ include "openshell.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/charts/openshell/templates/credential-secrets-namespace.yaml b/charts/openshell/templates/credential-secrets-namespace.yaml new file mode 100644 index 000000000..e5a1a3cd2 --- /dev/null +++ b/charts/openshell/templates/credential-secrets-namespace.yaml @@ -0,0 +1,27 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- $credentialSecrets := .Values.server.credentialDrivers.kubernetesSecrets -}} +{{- if and $credentialSecrets.enabled $credentialSecrets.createNamespace }} +{{- $namespace := include "openshell.credentialKubernetesSecretsNamespace" . -}} +{{- if eq $namespace .Release.Namespace -}} +{{- fail "server.credentialDrivers.kubernetesSecrets.createNamespace requires server.credentialDrivers.kubernetesSecrets.namespace to name a namespace other than the release namespace" -}} +{{- end -}} +{{- $existing := lookup "v1" "Namespace" "" $namespace -}} +{{- $releaseName := dig "metadata" "annotations" "meta.helm.sh/release-name" "" $existing -}} +{{- $releaseNamespace := dig "metadata" "annotations" "meta.helm.sh/release-namespace" "" $existing -}} +{{- if or (not $existing) (and (eq $releaseName .Release.Name) (eq $releaseNamespace .Release.Namespace)) }} +# Retained on uninstall so removing the gateway cannot delete stored +# credentials. A retained Namespace keeps its Helm ownership annotations, so a +# reinstall of the same release adopts it; a Namespace owned by anything else +# is left alone. +apiVersion: v1 +kind: Namespace +metadata: + name: {{ $namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/resource-policy: keep +{{- end }} +{{- end }} diff --git a/charts/openshell/templates/credential-secrets-role.yaml b/charts/openshell/templates/credential-secrets-role.yaml new file mode 100644 index 000000000..f6187c9ac --- /dev/null +++ b/charts/openshell/templates/credential-secrets-role.yaml @@ -0,0 +1,27 @@ +{{- if and .Values.server.credentialDrivers.kubernetesSecrets.enabled .Values.server.credentialDrivers.kubernetesSecrets.rbac.create }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openshell.fullname" . }}-credential-secrets + namespace: {{ include "openshell.credentialKubernetesSecretsNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +# NOTE: This Role grants access to all Secrets in the namespace because +# OpenShell-managed Secret names are dynamic SHA-256 hashes generated at +# runtime. Kubernetes RBAC does not support label-based or prefix-based +# filtering for resourceNames. To limit blast radius, deploy the gateway +# with a dedicated namespace for credential Secrets +# (server.credentialDrivers.kubernetesSecrets.namespace). +rules: + - apiGroups: + - "" + resources: + - secrets + verbs: + - get + - create + - patch + - delete +{{- end }} diff --git a/charts/openshell/templates/credential-secrets-rolebinding.yaml b/charts/openshell/templates/credential-secrets-rolebinding.yaml new file mode 100644 index 000000000..3a9ee0bdd --- /dev/null +++ b/charts/openshell/templates/credential-secrets-rolebinding.yaml @@ -0,0 +1,19 @@ +{{- if and .Values.server.credentialDrivers.kubernetesSecrets.enabled .Values.server.credentialDrivers.kubernetesSecrets.rbac.create }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openshell.fullname" . }}-credential-secrets + namespace: {{ include "openshell.credentialKubernetesSecretsNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openshell.fullname" . }}-credential-secrets +subjects: + - kind: ServiceAccount + name: {{ include "openshell.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/charts/openshell/templates/credential-storage-key-encryption-key-secret.yaml b/charts/openshell/templates/credential-storage-key-encryption-key-secret.yaml new file mode 100644 index 000000000..1e53d84bf --- /dev/null +++ b/charts/openshell/templates/credential-storage-key-encryption-key-secret.yaml @@ -0,0 +1,29 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +{{- if not (or .Values.server.credentialDrivers.kubernetesSecrets.enabled .Values.server.credentialDrivers.vault.enabled) }} +{{- if not .Values.server.credentialStorage.existingSecret }} +{{- $secretName := include "openshell.credentialStorageKeyEncryptionKeySecretName" . -}} +{{- $secretKey := include "openshell.credentialStorageKeyEncryptionKeySecretKey" . -}} +{{- $existing := lookup "v1" "Secret" .Release.Namespace $secretName -}} +{{- $encodedKeyEncryptionKey := randBytes 32 | b64enc -}} +{{- if $existing -}} +{{- $existingData := get $existing "data" | default dict -}} +{{- if not (hasKey $existingData $secretKey) -}} +{{- fail (printf "existing credential storage key-encryption key Secret %s/%s is missing key %s" .Release.Namespace $secretName $secretKey) -}} +{{- end -}} +{{- $encodedKeyEncryptionKey = index $existingData $secretKey -}} +{{- end }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ $secretName }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + annotations: + helm.sh/resource-policy: keep +type: Opaque +data: + {{ $secretKey }}: {{ $encodedKeyEncryptionKey | quote }} +{{- end }} +{{- end }} diff --git a/charts/openshell/templates/deployment.yaml b/charts/openshell/templates/deployment.yaml new file mode 100644 index 000000000..f94900b13 --- /dev/null +++ b/charts/openshell/templates/deployment.yaml @@ -0,0 +1,18 @@ +{{- include "openshell.validateValues" . }} +{{- if eq (include "openshell.workloadKind" .) "deployment" }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "openshell.fullname" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.replicaCount }} + selector: + matchLabels: + {{- include "openshell.selectorLabels" . | nindent 6 }} + template: + {{- include "openshell.gatewayPodTemplate" . | nindent 4 }} +{{- end }} diff --git a/charts/openshell/templates/gateway-config.yaml b/charts/openshell/templates/gateway-config.yaml new file mode 100644 index 000000000..0449871f7 --- /dev/null +++ b/charts/openshell/templates/gateway-config.yaml @@ -0,0 +1,275 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +{{/* +ConfigMap holding the gateway TOML config file (RFC 0003). + +The gateway reads `/etc/openshell/gateway.toml` (mounted from this ConfigMap) +at startup. CLI flags and OPENSHELL_* env vars on the gateway workload container +still override anything in this file. + +One value is intentionally NOT rendered here: + - server.dbUrl → passed via OPENSHELL_DB_URL env var (from Secret) + when server.externalDbSecret is set, otherwise + --db-url arg for SQLite +*/}} +{{- $credentialDrivers := list -}} +{{- $otlp := .Values.server.otlp | default dict -}} +{{- if .Values.server.credentialDrivers.kubernetesSecrets.enabled -}} +{{- $credentialDrivers = append $credentialDrivers "kubernetes-secrets" -}} +{{- end -}} +{{- if .Values.server.credentialDrivers.vault.enabled -}} +{{- $credentialDrivers = append $credentialDrivers "vault" -}} +{{- end -}} +{{- if and .Values.certManager.serverIssuerRef.name (not .Values.certManager.enabled) }} +{{- fail "certManager.serverIssuerRef.name is set but certManager.enabled is false \u2014 the external server certificate, its Secret mount, and the gateway TLS configuration all require cert-manager to be enabled. Set certManager.enabled=true or remove certManager.serverIssuerRef.name." }} +{{- end }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "openshell.fullname" . }}-config + labels: + {{- include "openshell.labels" . | nindent 4 }} +data: + gateway.toml: | + [openshell] + version = 2 + + [openshell.gateway] + name = {{ .Values.server.name | default (include "openshell.fullname" .) | quote }} + bind_address = "0.0.0.0:{{ .Values.service.port }}" + {{- if .Values.service.healthPort }} + health_bind_address = "0.0.0.0:{{ .Values.service.healthPort }}" + {{- end }} + {{- if .Values.service.metricsPort }} + metrics_bind_address = "0.0.0.0:{{ .Values.service.metricsPort }}" + {{- end }} + log_level = {{ .Values.server.logLevel | quote }} + compute_driver = "kubernetes" + {{- if $credentialDrivers }} + credential_drivers = [{{- range $i, $driver := $credentialDrivers }}{{ if $i }}, {{ end }}{{ $driver | quote }}{{- end }}] + {{- end }} + {{- $policyValidationFailureMode := .Values.server.policyValidationFailureMode }} + {{- if not (has $policyValidationFailureMode (list "fail_closed" "retain_last_valid")) }} + {{- fail "server.policyValidationFailureMode must be fail_closed or retain_last_valid" }} + {{- end }} + policy_validation_failure_mode = {{ $policyValidationFailureMode | quote }} + {{- if .Values.server.disableTls }} + disable_tls = true + {{- end }} + enable_loopback_service_http = {{ .Values.server.enableLoopbackServiceHttp }} + enable_websocket_tunnel = {{ .Values.server.enableWebsocketTunnel }} + {{- $sans := list -}} + {{- if and .Values.certManager.enabled .Values.certManager.serverDnsNames }} + {{- $sans = .Values.certManager.serverDnsNames }} + {{- else if and .Values.pkiInitJob.enabled .Values.pkiInitJob.serverDnsNames }} + {{- $sans = .Values.pkiInitJob.serverDnsNames }} + {{- end }} + {{- if $sans }} + server_sans = [{{- range $i, $san := $sans }}{{ if $i }}, {{ end }}{{ $san | quote }}{{- end }}] + {{- end }} + {{- $rlRequests := int .Values.server.grpcRateLimit.requests }} + {{- $rlWindowSeconds := int .Values.server.grpcRateLimit.windowSeconds }} + {{- if or (lt $rlRequests 0) (lt $rlWindowSeconds 0) }} + {{- fail "server.grpcRateLimit.requests and server.grpcRateLimit.windowSeconds must not be negative; they map to unsigned gateway settings" }} + {{- end }} + {{- if and (gt $rlRequests 0) (gt $rlWindowSeconds 0) }} + grpc_rate_limit_requests = {{ $rlRequests }} + grpc_rate_limit_window_seconds = {{ $rlWindowSeconds }} + {{- else if or (gt $rlRequests 0) (gt $rlWindowSeconds 0) }} + {{- fail "server.grpcRateLimit requires both requests and windowSeconds to be positive to enable rate limiting, or both 0/unset to disable it" }} + {{- end }} + + {{- if $otlp.endpoint }} + + [openshell.gateway.otlp] + endpoint = {{ $otlp.endpoint | quote }} + {{- if $otlp.serviceName }} + service_name = {{ $otlp.serviceName | quote }} + {{- end }} + {{- end }} + + {{- if not .Values.server.disableTls }} + + [openshell.gateway.tls] + cert_path = "/etc/openshell-tls/server/tls.crt" + key_path = "/etc/openshell-tls/server/tls.key" + {{- if eq (include "openshell.gatewayClientCaEnabled" .) "true" }} + client_ca_path = "/etc/openshell-tls/client-ca/ca.crt" + {{- end }} + {{- if .Values.certManager.serverIssuerRef.name }} + external_cert_path = "/etc/openshell-tls/server-external/tls.crt" + external_key_path = "/etc/openshell-tls/server-external/tls.key" + external_server_names = [{{- range $i, $name := .Values.certManager.serverDnsNames }}{{ if $i }}, {{ end }}{{ $name | quote }}{{- end }}] + {{- end }} + {{- end }} + + {{- if .Values.server.auth.allowUnauthenticatedUsers }} + + [openshell.gateway.auth] + allow_unauthenticated_users = true + {{- end }} + + [openshell.gateway.gateway_jwt] + signing_key_path = "/etc/openshell-jwt/signing.pem" + public_key_path = "/etc/openshell-jwt/public.pem" + kid_path = "/etc/openshell-jwt/kid" + gateway_id = {{ .Values.server.sandboxJwt.gatewayId | default (include "openshell.fullname" .) | quote }} + ttl_secs = {{ .Values.server.sandboxJwt.ttlSecs | default 3600 }} + + {{- if .Values.server.oidc.issuer }} + + [openshell.gateway.oidc] + issuer = {{ .Values.server.oidc.issuer | quote }} + dangerously_allow_insecure_http = {{ .Values.server.oidc.dangerouslyAllowInsecureHttp }} + jwks_allowed_origins = {{ .Values.server.oidc.jwksAllowedOrigins | toJson }} + audience = {{ .Values.server.oidc.audience | quote }} + jwks_ttl_secs = {{ .Values.server.oidc.jwksTtl }} + {{- if .Values.server.oidc.rolesClaim }} + roles_claim = {{ .Values.server.oidc.rolesClaim | quote }} + {{- end }} + {{- if .Values.server.oidc.adminRole }} + admin_role = {{ .Values.server.oidc.adminRole | quote }} + {{- end }} + {{- if .Values.server.oidc.userRole }} + user_role = {{ .Values.server.oidc.userRole | quote }} + {{- end }} + {{- if .Values.server.oidc.scopesClaim }} + scopes_claim = {{ .Values.server.oidc.scopesClaim | quote }} + {{- end }} + {{- end }} + + [openshell.drivers.kubernetes] + allow_driver_config = {{ .Values.server.drivers.kubernetes.allowDriverConfig }} + namespace = {{ include "openshell.sandboxNamespace" . | quote }} + default_image = {{ include "openshell.sandboxImage" . | quote }} + {{- if include "openshell.sandboxRuntimeImageOverrideEnabled" . }} + sandbox_runtime_image = {{ include "openshell.sandboxRuntimeImage" . | quote }} + {{- end }} + supervisor_image = {{ include "openshell.supervisorImage" . | quote }} + {{- if .Values.server.hostGatewayIP }} + host_gateway_ip = {{ .Values.server.hostGatewayIP | quote }} + {{- end }} + {{- if not .Values.server.disableTls }} + client_tls_secret_name = {{ .Values.server.tls.clientTlsSecretName | quote }} + {{- end }} + workspace_mode = {{ .Values.server.drivers.kubernetes.workspaceMode | default "shared" | quote }} + gateway_id = {{ .Values.server.sandboxJwt.gatewayId | default (include "openshell.fullname" .) | quote }} + grpc_endpoint = {{ include "openshell.grpcEndpoint" . | quote }} + service_account_name = {{ include "openshell.sandboxServiceAccountName" . | quote }} + {{- if .Values.server.enableUserNamespaces }} + enable_user_namespaces = true + {{- end }} + {{- if .Values.server.drivers.kubernetes.operatorNamespaceLabel }} + operator_namespace_label = {{ .Values.server.drivers.kubernetes.operatorNamespaceLabel | quote }} + {{- end }} + {{- if .Values.server.drivers.kubernetes.operatorNamespaceFile }} + operator_namespace_file = {{ .Values.server.drivers.kubernetes.operatorNamespaceFile | quote }} + {{- end }} + sa_token_ttl_secs = {{ .Values.server.sandboxJwt.k8sSaTokenTtlSecs | default 3600 }} + {{- if .Values.upstreamProxy.url }} + https_proxy = {{ .Values.upstreamProxy.url | quote }} + {{- end }} + {{- if .Values.upstreamProxy.noProxy }} + no_proxy = {{ .Values.upstreamProxy.noProxy | quote }} + {{- end }} + {{- if .Values.upstreamProxy.authSecret.name }} + proxy_auth_secret_name = {{ .Values.upstreamProxy.authSecret.name | quote }} + {{- end }} + {{- if .Values.upstreamProxy.authSecret.key }} + proxy_auth_secret_key = {{ .Values.upstreamProxy.authSecret.key | quote }} + {{- end }} + {{- if and .Values.upstreamProxy.authSecret.name .Values.upstreamProxy.authSecret.key }} + proxy_auth_allow_insecure = {{ .Values.upstreamProxy.authAllowInsecure }} + {{- end }} + {{- if .Values.upstreamProxy.connectByHostname }} + proxy_connect_by_hostname = true + {{- end }} + {{- if .Values.upstreamProxy.caBundle.configMapName }} + proxy_ca_bundle = "/etc/openshell-tls/proxy-ca/ca.crt" + {{- end }} + {{- if .Values.server.providerTokenGrants.spiffe.enabled }} + provider_spiffe_workload_api_socket_path = {{ .Values.server.providerTokenGrants.spiffe.workloadApiSocketPath | quote }} + {{- end }} + {{- if .Values.sandbox.image.pullPolicy }} + image_pull_policy = {{ include "openshell.canonicalImagePullPolicy" .Values.sandbox.image.pullPolicy | quote }} + {{- end }} + {{- $sandboxImagePullSecretNames := list -}} + {{- range .Values.server.sandboxImagePullSecrets }} + {{- if .name }} + {{- $sandboxImagePullSecretNames = append $sandboxImagePullSecretNames .name }} + {{- end }} + {{- end }} + {{- if $sandboxImagePullSecretNames }} + image_pull_secrets = [{{- range $i, $name := $sandboxImagePullSecretNames }}{{ if $i }}, {{ end }}{{ $name | quote }}{{- end }}] + {{- end }} + {{- if .Values.server.workspaceDefaultStorageSize }} + workspace_default_storage_size = {{ .Values.server.workspaceDefaultStorageSize | quote }} + {{- end }} + {{- if .Values.server.workspaceStorageClass }} + workspace_storage_class = {{ .Values.server.workspaceStorageClass | quote }} + {{- end }} + {{- if .Values.server.defaultRuntimeClassName }} + default_runtime_class_name = {{ .Values.server.defaultRuntimeClassName | quote }} + {{- end }} + {{- if (.Values.supervisor.image.pullPolicy | default .Values.global.image.pullPolicy) }} + supervisor_image_pull_policy = {{ include "openshell.canonicalImagePullPolicy" (.Values.supervisor.image.pullPolicy | default .Values.global.image.pullPolicy) | quote }} + {{- end }} + {{- if (.Values.sandboxRuntime.image.pullPolicy | default .Values.global.image.pullPolicy) }} + sandbox_runtime_image_pull_policy = {{ include "openshell.canonicalImagePullPolicy" (.Values.sandboxRuntime.image.pullPolicy | default .Values.global.image.pullPolicy) | quote }} + {{- end }} + + [openshell.drivers.kubernetes.resource_admission] + enabled = {{ .Values.server.drivers.kubernetes.resourceAdmission.enabled }} + {{- if ne .Values.server.drivers.kubernetes.resourceAdmission.requiredLabels nil }} + [openshell.drivers.kubernetes.resource_admission.required_labels] + {{- range $key, $value := .Values.server.drivers.kubernetes.resourceAdmission.requiredLabels }} + {{ $key | quote }} = {{ $value | quote }} + {{- end }} + {{- end }} + + [openshell.drivers.kubernetes.managed_ssh_ingress] + enabled = {{ .Values.networkPolicy.enabled }} + gateway_namespace = {{ .Release.Namespace | quote }} + gateway_pod_selector = { "app.kubernetes.io/name" = {{ include "openshell.name" . | quote }}, "app.kubernetes.io/instance" = {{ .Release.Name | quote }} } + + [openshell.drivers.kubernetes.sandbox_runtime] + boundary_port = {{ .Values.supervisor.sandboxRuntime.boundaryPort | default 5500 }} + + {{- if not $credentialDrivers }} + + [openshell.gateway.credential_storage] + key_encryption_key_env = {{ include "openshell.credentialStorageKeyEncryptionKeyEnvName" . | quote }} + {{- end }} + + {{- if .Values.server.credentialDrivers.kubernetesSecrets.enabled }} + + [openshell.credential_drivers.kubernetes-secrets] + namespace = {{ include "openshell.credentialKubernetesSecretsNamespace" . | quote }} + {{- end }} + + {{- if .Values.server.credentialDrivers.vault.enabled }} + + [openshell.credential_drivers.vault] + address = {{ .Values.server.credentialDrivers.vault.address | quote }} + {{- if .Values.server.credentialDrivers.vault.caConfigMapName }} + ca_bundle = "/etc/openshell-tls/vault-ca/ca.crt" + {{- end }} + mount = {{ .Values.server.credentialDrivers.vault.mount | quote }} + kv_version = {{ .Values.server.credentialDrivers.vault.kvVersion | quote }} + auth_method = {{ .Values.server.credentialDrivers.vault.authMethod | quote }} + {{- if .Values.server.credentialDrivers.vault.role }} + role = {{ .Values.server.credentialDrivers.vault.role | quote }} + {{- end }} + {{- if .Values.server.credentialDrivers.vault.kubernetesAuthMount }} + kubernetes_auth_mount = {{ .Values.server.credentialDrivers.vault.kubernetesAuthMount | quote }} + {{- end }} + {{- if .Values.server.credentialDrivers.vault.serviceAccountTokenPath }} + service_account_token_path = {{ .Values.server.credentialDrivers.vault.serviceAccountTokenPath | quote }} + {{- end }} + {{- if .Values.server.credentialDrivers.vault.tokenPath }} + token_path = {{ .Values.server.credentialDrivers.vault.tokenPath | quote }} + {{- end }} + {{- if .Values.server.credentialDrivers.vault.timeoutSecs }} + timeout_secs = {{ .Values.server.credentialDrivers.vault.timeoutSecs }} + {{- end }} + {{- end }} diff --git a/charts/openshell/templates/gateway.yaml b/charts/openshell/templates/gateway.yaml new file mode 100644 index 000000000..73c3d7603 --- /dev/null +++ b/charts/openshell/templates/gateway.yaml @@ -0,0 +1,32 @@ +{{- if and .Values.grpcRoute.enabled .Values.grpcRoute.gateway.create }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: {{ default (include "openshell.fullname" .) .Values.grpcRoute.gateway.name }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + gatewayClassName: {{ .Values.grpcRoute.gateway.className }} + listeners: + - name: {{ ternary "https" "http" (eq .Values.grpcRoute.gateway.listener.protocol "HTTPS") }} + port: {{ .Values.grpcRoute.gateway.listener.port }} + protocol: {{ .Values.grpcRoute.gateway.listener.protocol }} + {{- if eq .Values.grpcRoute.gateway.listener.protocol "HTTPS" }} + {{- if not .Values.grpcRoute.gateway.listener.tls.certificateRefs }} + {{- fail "grpcRoute.gateway.listener.tls.certificateRefs is required when grpcRoute.gateway.listener.protocol is HTTPS" }} + {{- end }} + {{- if and (not .Values.server.disableTls) (not .Values.grpcRoute.backendTLSPolicy.enabled) }} + {{- fail "grpcRoute.gateway.listener.protocol=HTTPS terminates TLS at the Gateway listener. Either set server.disableTls=true so the pod listens plaintext, or enable grpcRoute.backendTLSPolicy for end-to-end TLS re-encryption to the gateway pod." }} + {{- end }} + tls: + mode: Terminate + certificateRefs: + {{- toYaml .Values.grpcRoute.gateway.listener.tls.certificateRefs | nindent 10 }} + {{- end }} + allowedRoutes: + namespaces: + from: {{ .Values.grpcRoute.gateway.listener.allowedRoutes }} +{{- end }} diff --git a/charts/openshell/templates/grpcroute.yaml b/charts/openshell/templates/grpcroute.yaml new file mode 100644 index 000000000..362067fda --- /dev/null +++ b/charts/openshell/templates/grpcroute.yaml @@ -0,0 +1,23 @@ +{{- if .Values.grpcRoute.enabled }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: gateway.networking.k8s.io/v1 +kind: GRPCRoute +metadata: + name: {{ include "openshell.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + parentRefs: + - name: {{ default (include "openshell.fullname" .) .Values.grpcRoute.gateway.name }} + namespace: {{ default .Release.Namespace .Values.grpcRoute.gateway.namespace }} + {{- if .Values.grpcRoute.hostnames }} + hostnames: + {{- toYaml .Values.grpcRoute.hostnames | nindent 4 }} + {{- end }} + rules: + - backendRefs: + - name: {{ include "openshell.fullname" . }} + port: {{ .Values.service.port }} +{{- end }} diff --git a/charts/openshell/templates/networkpolicy.yaml b/charts/openshell/templates/networkpolicy.yaml new file mode 100644 index 000000000..c9e4a760e --- /dev/null +++ b/charts/openshell/templates/networkpolicy.yaml @@ -0,0 +1,35 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- if and (include "openshell.workspaceResourcesEnabled" .) .Values.networkPolicy.enabled }} +# NetworkPolicy restricting SSH ingress on sandbox pods to the gateway pod. +# Sandbox pods are dynamically created by the server and labelled with +# openshell.ai/managed-by=openshell. This policy ensures only the gateway +# (openshell server) pod can reach the sandbox SSH port (2222), blocking +# lateral movement from other in-cluster workloads. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "openshell.fullname" . }}-sandbox-ssh + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + openshell.ai/managed-by: openshell + policyTypes: + - Ingress + ingress: + - from: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ .Release.Namespace }} + podSelector: + matchLabels: + app.kubernetes.io/name: {{ include "openshell.name" . }} + app.kubernetes.io/instance: {{ .Release.Name }} + ports: + - protocol: TCP + port: 2222 +{{- end }} diff --git a/charts/openshell/templates/peer-role.yaml b/charts/openshell/templates/peer-role.yaml new file mode 100644 index 000000000..3aa2502f4 --- /dev/null +++ b/charts/openshell/templates/peer-role.yaml @@ -0,0 +1,37 @@ +{{- if include "openshell.rbacCreate" . }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openshell.fullname" . }}-peer + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +rules: + # Gateway peer identity: TokenReview authenticates the projected token, then + # the receiver resolves the returned pod name and UID to the live gateway pod + # in the release namespace. + - apiGroups: + - "" + resources: + - pods + verbs: + - get +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openshell.fullname" . }}-peer + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openshell.fullname" . }}-peer +subjects: + - kind: ServiceAccount + name: {{ include "openshell.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/charts/openshell/templates/peer-service.yaml b/charts/openshell/templates/peer-service.yaml new file mode 100644 index 000000000..f5d93af24 --- /dev/null +++ b/charts/openshell/templates/peer-service.yaml @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: Service +metadata: + name: {{ include "openshell.peerServiceName" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + clusterIP: None + publishNotReadyAddresses: true + ports: + - port: {{ .Values.service.port }} + targetPort: grpc + protocol: TCP + name: grpc + appProtocol: grpc + selector: + {{- include "openshell.selectorLabels" . | nindent 4 }} diff --git a/charts/openshell/templates/role.yaml b/charts/openshell/templates/role.yaml new file mode 100644 index 000000000..d9237e743 --- /dev/null +++ b/charts/openshell/templates/role.yaml @@ -0,0 +1,68 @@ +{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} +{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) (include "openshell.rbacCreate" .) }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openshell.fullname" . }}-sandbox + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +rules: + {{- if .Values.server.drivers.kubernetes.allowDriverConfig }} + # Metadata-only admission of caller-selected PVCs requires get permission. + - apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + {{- end }} + - apiGroups: + - agents.x-k8s.io + resources: + - sandboxes + - sandboxes/status + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - get + - list + - watch + # Per-sandbox identity: TokenReview authenticates the projected token from + # the configured sandbox service account, then the gateway resolves the + # returned pod name and UID to its immutable sandbox labels. The driver also + # creates and deletes the directly managed supervisor Pod. PATCH removes the + # driver-owned bootstrap scheduling gate after immutable bootstrap material + # has been created. + - apiGroups: + - "" + resources: + - pods + verbs: + - create + - delete + - get + - list + - patch + - watch + - apiGroups: [""] + resources: ["services"] + verbs: ["create", "get"] + - apiGroups: [""] + resources: ["secrets"] + # Bootstrap Secrets are generation-scoped. Recovery deletes them by exact + # name. + verbs: ["create", "delete"] + - apiGroups: ["networking.k8s.io"] + resources: ["networkpolicies"] + verbs: ["create", "get"] +{{- end }} diff --git a/charts/openshell/templates/rolebinding.yaml b/charts/openshell/templates/rolebinding.yaml new file mode 100644 index 000000000..49cc6f7fb --- /dev/null +++ b/charts/openshell/templates/rolebinding.yaml @@ -0,0 +1,20 @@ +{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} +{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) (include "openshell.rbacCreate" .) }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openshell.fullname" . }}-sandbox + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openshell.fullname" . }}-sandbox +subjects: + - kind: ServiceAccount + name: {{ include "openshell.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/charts/openshell/templates/route.yaml b/charts/openshell/templates/route.yaml new file mode 100644 index 000000000..459a0ac46 --- /dev/null +++ b/charts/openshell/templates/route.yaml @@ -0,0 +1,52 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.openshiftRoute.enabled }} +{{- if .Values.server.disableTls }} +{{- fail "openshiftRoute.enabled=true requires TLS (server.disableTls must be false) \u2014 a passthrough Route forwards encrypted traffic by SNI, so the gateway must terminate its own TLS." }} +{{- end }} +{{- if and .Values.certManager.serverIssuerRef.name (not .Values.openshiftRoute.host) }} +{{- fail "openshiftRoute.enabled=true with certManager.serverIssuerRef requires an explicit openshiftRoute.host \u2014 without one, OpenShift generates a hostname absent from certManager.serverDnsNames, causing the gateway to serve its internal certificate to external clients." }} +{{- end }} +{{- if and .Values.openshiftRoute.host .Values.certManager.serverIssuerRef.name .Values.certManager.serverDnsNames }} +{{- $routeHost := .Values.openshiftRoute.host }} +{{- $hostCovered := false }} +{{- range .Values.certManager.serverDnsNames }} + {{- if eq . $routeHost }} + {{- $hostCovered = true }} + {{- else if hasPrefix "*." . }} + {{- $wildcardSuffix := trimPrefix "*" . }} + {{- $prefix := trimSuffix $wildcardSuffix $routeHost }} + {{- if and (ne $prefix $routeHost) (gt (len $prefix) 0) (not (contains "." $prefix)) }} + {{- $hostCovered = true }} + {{- end }} + {{- end }} +{{- end }} +{{- if not $hostCovered }} +{{- fail (printf "openshiftRoute.host %q is not covered by certManager.serverDnsNames %v \u2014 the Route will forward SNI for a hostname the external certificate does not cover, causing TLS verification failures for CLI clients. Exact names and single-level wildcards (e.g. *.example.com) are checked." $routeHost .Values.certManager.serverDnsNames) }} +{{- end }} +{{- end }} +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: {{ include "openshell.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + {{- with .Values.openshiftRoute.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .Values.openshiftRoute.host }} + host: {{ .Values.openshiftRoute.host | quote }} + {{- end }} + to: + kind: Service + name: {{ include "openshell.fullname" . }} + port: + targetPort: grpc + tls: + termination: passthrough + wildcardPolicy: None +{{- end }} diff --git a/charts/openshell/templates/service.yaml b/charts/openshell/templates/service.yaml new file mode 100644 index 000000000..ebad42eab --- /dev/null +++ b/charts/openshell/templates/service.yaml @@ -0,0 +1,28 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: Service +metadata: + name: {{ include "openshell.fullname" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + type: {{ .Values.service.type }} + ports: + - port: {{ .Values.service.port }} + targetPort: grpc + protocol: TCP + name: grpc + appProtocol: grpc + {{- if and (eq .Values.service.type "NodePort") .Values.service.nodePort }} + nodePort: {{ .Values.service.nodePort }} + {{- end }} + {{- if .Values.service.metricsPort }} + - port: {{ .Values.service.metricsPort }} + targetPort: metrics + protocol: TCP + name: metrics + {{- end }} + selector: + {{- include "openshell.selectorLabels" . | nindent 4 }} diff --git a/charts/openshell/templates/serviceaccount.yaml b/charts/openshell/templates/serviceaccount.yaml new file mode 100644 index 000000000..1a9245d4d --- /dev/null +++ b/charts/openshell/templates/serviceaccount.yaml @@ -0,0 +1,31 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.serviceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "openshell.serviceAccountName" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + {{- with .Values.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} +{{- if and .Values.serviceAccount.create (include "openshell.workspaceResourcesEnabled" .) .Values.sandboxServiceAccount.create }} +--- +{{- end }} +{{- if and (include "openshell.workspaceResourcesEnabled" .) .Values.sandboxServiceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "openshell.sandboxServiceAccountName" . }} + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} + {{- with .Values.sandboxServiceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/charts/openshell/templates/statefulset.yaml b/charts/openshell/templates/statefulset.yaml new file mode 100644 index 000000000..10d0839f6 --- /dev/null +++ b/charts/openshell/templates/statefulset.yaml @@ -0,0 +1,27 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +{{- include "openshell.validateValues" . }} +{{- if eq (include "openshell.workloadKind" .) "statefulset" }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ include "openshell.fullname" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +spec: + serviceName: {{ include "openshell.peerServiceName" . }} + replicas: {{ .Values.replicaCount }} + selector: + matchLabels: + {{- include "openshell.selectorLabels" . | nindent 6 }} + template: + {{- include "openshell.gatewayPodTemplate" . | nindent 4 }} + volumeClaimTemplates: + - metadata: + name: openshell-data + spec: + accessModes: ["ReadWriteOnce"] + resources: + requests: + storage: 1Gi +{{- end }} diff --git a/charts/openshell/templates/workspace-secret-source-role.yaml b/charts/openshell/templates/workspace-secret-source-role.yaml new file mode 100644 index 000000000..627fbbaa7 --- /dev/null +++ b/charts/openshell/templates/workspace-secret-source-role.yaml @@ -0,0 +1,25 @@ +{{- $sourceSecretNames := include "openshell.workspaceSecretSourceNames" . | fromJsonArray }} +{{- if $sourceSecretNames }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# Read the TLS and image-pull Secrets the gateway stages into workspace +# namespaces. +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openshell.fullname" . }}-workspace-secret-source + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +rules: + - apiGroups: + - "" + resources: + - secrets + resourceNames: + {{- range $sourceSecretNames }} + - {{ . | quote }} + {{- end }} + verbs: + - get +{{- end }} diff --git a/charts/openshell/templates/workspace-secret-source-rolebinding.yaml b/charts/openshell/templates/workspace-secret-source-rolebinding.yaml new file mode 100644 index 000000000..61dcfd2ad --- /dev/null +++ b/charts/openshell/templates/workspace-secret-source-rolebinding.yaml @@ -0,0 +1,19 @@ +{{- if include "openshell.workspaceSecretSourceNames" . | fromJsonArray }} +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openshell.fullname" . }}-workspace-secret-source + namespace: {{ include "openshell.sandboxNamespace" . }} + labels: + {{- include "openshell.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openshell.fullname" . }}-workspace-secret-source +subjects: + - kind: ServiceAccount + name: {{ include "openshell.serviceAccountName" . }} + namespace: {{ .Release.Namespace }} +{{- end }} diff --git a/charts/openshell/tests/agent_sandbox_preflight_test.yaml b/charts/openshell/tests/agent_sandbox_preflight_test.yaml new file mode 100644 index 000000000..45ab2e76b --- /dev/null +++ b/charts/openshell/tests/agent_sandbox_preflight_test.yaml @@ -0,0 +1,12 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: Agent Sandbox preflight +templates: + - templates/agent-sandbox-preflight.yaml + +tests: + - it: fails clearly by default without a supported Agent Sandbox API + asserts: + - failedTemplate: + errorPattern: "Agent Sandbox is required but neither agents.x-k8s.io/v1beta1 nor agents.x-k8s.io/v1alpha1 is served" diff --git a/charts/openshell/tests/cert_manager_pki_test.yaml b/charts/openshell/tests/cert_manager_pki_test.yaml new file mode 100644 index 000000000..6fb8b8fbe --- /dev/null +++ b/charts/openshell/tests/cert_manager_pki_test.yaml @@ -0,0 +1,171 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: cert-manager PKI issuerRef overrides +templates: + - templates/cert-manager-pki.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: defaults both server and client Certificates to the chart's own CA issuer + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + asserts: + - equal: + path: spec.issuerRef.name + value: openshell-ca-issuer + documentIndex: 3 + - equal: + path: spec.issuerRef.kind + value: Issuer + documentIndex: 3 + - equal: + path: spec.issuerRef.name + value: openshell-ca-issuer + documentIndex: 4 + - equal: + path: spec.issuerRef.kind + value: Issuer + documentIndex: 4 + + - it: default server Certificate includes internal SANs, IPs, and a fixed commonName + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + asserts: + - equal: + path: spec.commonName + value: openshell + documentIndex: 3 + - contains: + path: spec.dnsNames + content: openshell.my-namespace.svc.cluster.local + documentIndex: 3 + - contains: + path: spec.dnsNames + content: localhost + documentIndex: 3 + - equal: + path: spec.ipAddresses[0] + value: 127.0.0.1 + documentIndex: 3 + + - it: internal server cert keeps chart CA issuer and internal SANs when serverIssuerRef is set + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverIssuerRef.kind: ClusterIssuer + certManager.serverDnsNames: + - openshell.example.com + asserts: + # Internal cert (doc 3) stays on chart CA + - equal: + path: spec.issuerRef.name + value: openshell-ca-issuer + documentIndex: 3 + - equal: + path: spec.issuerRef.kind + value: Issuer + documentIndex: 3 + # Internal cert has internal SANs + - equal: + path: spec.commonName + value: openshell + documentIndex: 3 + - contains: + path: spec.dnsNames + content: openshell.my-namespace.svc.cluster.local + documentIndex: 3 + - contains: + path: spec.dnsNames + content: localhost + documentIndex: 3 + # Internal cert does NOT include external-only hostnames + - notContains: + path: spec.dnsNames + content: openshell.example.com + documentIndex: 3 + + - it: creates external server Certificate from serverIssuerRef with external SANs only + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverIssuerRef.kind: ClusterIssuer + certManager.serverDnsNames: + - openshell.example.com + asserts: + # External cert (doc 4) uses the external issuer + - equal: + path: spec.issuerRef.name + value: letsencrypt-prod + documentIndex: 4 + - equal: + path: spec.issuerRef.kind + value: ClusterIssuer + documentIndex: 4 + - equal: + path: spec.issuerRef.group + value: cert-manager.io + documentIndex: 4 + # External cert has only external SANs + - equal: + path: spec.commonName + value: openshell.example.com + documentIndex: 4 + - equal: + path: spec.dnsNames + value: + - openshell.example.com + documentIndex: 4 + # External cert has no IP addresses + - notExists: + path: spec.ipAddresses + documentIndex: 4 + # External cert has no internal names + - notContains: + path: spec.dnsNames + content: localhost + documentIndex: 4 + + - it: fails when serverIssuerRef is set but serverDnsNames contains internal-only names + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + # serverDnsNames is left at the default which contains "openshell.openshell.svc", "localhost", etc. + asserts: + - failedTemplate: + errorMessage: "certManager.serverIssuerRef.name is set (external issuer) but certManager.serverDnsNames contains \"openshell.openshell.svc\" \u2014 external CAs (e.g. ACME / Let's Encrypt) reject internal-only names per CA/Browser Forum baseline requirements. Override certManager.serverDnsNames with your externally-resolvable hostname(s)." + + - it: fails when clientCaFromServerTlsSecret is false but clientCaSecretName is the default + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: false + asserts: + - failedTemplate: + errorMessage: "certManager.clientCaFromServerTlsSecret is false but server.tls.clientCaSecretName is still the default (openshell-server-client-ca), which nothing creates when cert-manager owns TLS. Set server.tls.clientCaSecretName to the secret containing the client CA (e.g. the value of certManager.caSecretName, which defaults to openshell-ca-tls), or set it to empty to disable mTLS client verification." + + - it: client Certificate issuerRef is unaffected by serverIssuerRef + template: templates/cert-manager-pki.yaml + set: + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverIssuerRef.kind: ClusterIssuer + certManager.serverDnsNames: + - openshell.example.com + asserts: + # Client cert is now doc 5 (after internal + external server certs) + - equal: + path: spec.issuerRef.name + value: openshell-ca-issuer + documentIndex: 5 + - equal: + path: spec.issuerRef.kind + value: Issuer + documentIndex: 5 diff --git a/charts/openshell/tests/certgen_test.yaml b/charts/openshell/tests/certgen_test.yaml new file mode 100644 index 000000000..cd88b60e9 --- /dev/null +++ b/charts/openshell/tests/certgen_test.yaml @@ -0,0 +1,116 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: certgen hook +templates: + - templates/certgen.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: renders combined TLS and JWT certgen hook by default + template: templates/certgen.yaml + asserts: + - hasDocuments: + count: 4 + - equal: + path: kind + value: Job + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--server-secret-name=openshell-server-tls" + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--client-secret-name=openshell-client-tls" + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-secret-name=openshell-jwt-keys" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--jwt-only" + documentIndex: 3 + + - it: renders JWT-only certgen hook when cert-manager owns TLS + template: templates/certgen.yaml + set: + certManager.enabled: true + pkiInitJob.enabled: false + asserts: + - hasDocuments: + count: 4 + - equal: + path: kind + value: Job + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-only" + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-secret-name=openshell-jwt-keys" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--server-secret-name=openshell-server-tls" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--client-secret-name=openshell-client-tls" + documentIndex: 3 + + - it: uses the configured sandbox JWT secret name + template: templates/certgen.yaml + set: + server.sandboxJwt.signingSecretName: custom-jwt-keys + asserts: + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-secret-name=custom-jwt-keys" + documentIndex: 3 + + - it: renders JWT-only hook when cert-manager is enabled even if pkiInitJob remains enabled + template: templates/certgen.yaml + set: + certManager.enabled: true + pkiInitJob.enabled: true + pkiInitJob.serverDnsNames: + - extra.example.test + pkiInitJob.serverIpAddresses: + - 192.0.2.10 + asserts: + - hasDocuments: + count: 4 + - equal: + path: kind + value: Job + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-only" + documentIndex: 3 + - contains: + path: spec.template.spec.containers[0].args + content: "--jwt-secret-name=openshell-jwt-keys" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--server-secret-name=openshell-server-tls" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--client-secret-name=openshell-client-tls" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--server-san=extra.example.test" + documentIndex: 3 + - notContains: + path: spec.template.spec.containers[0].args + content: "--server-san=192.0.2.10" + documentIndex: 3 diff --git a/charts/openshell/tests/clusterrole_test.yaml b/charts/openshell/tests/clusterrole_test.yaml new file mode 100644 index 000000000..08907c614 --- /dev/null +++ b/charts/openshell/tests/clusterrole_test.yaml @@ -0,0 +1,204 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: ClusterRole RBAC +templates: + - templates/clusterrole.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: includes the release namespace in the ClusterRole name + asserts: + - equal: + path: metadata.name + value: openshell-node-reader-my-namespace + + - it: grants managed namespace NetworkPolicy apply permissions + set: + server.drivers.kubernetes.workspaceMode: managed + asserts: + - contains: + path: rules + content: + apiGroups: ["networking.k8s.io"] + resources: ["networkpolicies"] + verbs: ["get", "create", "patch", "update"] + + - it: limits namespaced admission reads to PVCs + set: + server.drivers.kubernetes.workspaceMode: managed + server.drivers.kubernetes.allowDriverConfig: true + asserts: + - contains: + path: rules + content: + apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["get"] + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["configmaps"] + verbs: ["get"] + + - it: omits PVC admission reads when caller driver config is disabled + set: + server.drivers.kubernetes.workspaceMode: managed + asserts: + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + + - it: preserves sandbox-runtime fence permissions when gateway isolation is disabled + set: + server.drivers.kubernetes.workspaceMode: managed + networkPolicy.enabled: false + asserts: + - contains: + path: rules + content: + apiGroups: ["networking.k8s.io"] + resources: ["networkpolicies"] + verbs: ["create", "get"] + - notContains: + path: rules + content: + apiGroups: ["networking.k8s.io"] + resources: ["networkpolicies"] + verbs: ["get", "create", "patch", "update"] + + - it: omits credential Secret access in managed mode + set: + server.drivers.kubernetes.workspaceMode: managed + server.credentialDrivers.kubernetesSecrets.enabled: true + asserts: + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "patch", "delete"] + + - it: omits all Secret rules in operator mode + set: + server.drivers.kubernetes.workspaceMode: operator + server.tls.clientTlsSecretName: custom-client-tls + server.credentialDrivers.kubernetesSecrets.enabled: true + asserts: + - notExists: + path: rules[*].resources[?(@ == "secrets")] + + - it: names no Secrets in managed mode + set: + server.drivers.kubernetes.workspaceMode: managed + server.tls.clientTlsSecretName: custom-client-tls + server.sandboxImagePullSecrets: + - name: registry-one + - name: registry-two + asserts: + - notExists: + path: rules[*].resourceNames + - contains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["create", "delete"] + + - it: omits secrets rule entirely in shared mode + set: + server.drivers.kubernetes.workspaceMode: shared + asserts: + - notExists: + path: rules[*].resources[?(@ == "secrets")] + + - it: grants managed sandbox-runtime companion permissions + set: + server.drivers.kubernetes.workspaceMode: managed + asserts: + - contains: + path: rules + content: + apiGroups: [""] + resources: ["pods"] + verbs: ["create", "delete", "get", "list", "patch", "watch"] + - contains: + path: rules + content: + apiGroups: [""] + resources: ["services"] + verbs: ["create", "get"] + - contains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["create", "delete"] + + - it: grants operator sandbox-runtime companion permissions + set: + server.drivers.kubernetes.workspaceMode: operator + asserts: + - contains: + path: rules + content: + apiGroups: [""] + resources: ["pods"] + verbs: ["create", "delete", "get", "list", "patch", "watch"] + - contains: + path: rules + content: + apiGroups: [""] + resources: ["services"] + verbs: ["create", "get"] + - it: omits the ClusterRole when cluster-scoped RBAC is disabled + set: + rbac.clusterScoped.create: false + asserts: + - hasDocuments: + count: 0 + + - it: omits the ClusterRole when all chart-managed RBAC is disabled + set: + rbac.create: false + asserts: + - hasDocuments: + count: 0 + + - it: omits the ClusterRole independently of the workspace mode + set: + rbac.clusterScoped.create: false + server.drivers.kubernetes.workspaceMode: managed + asserts: + - hasDocuments: + count: 0 + + - it: uses the configured ClusterRole name + set: + rbac.clusterScoped.clusterRoleName: platform-openshell-node-reader + asserts: + - equal: + path: metadata.name + value: platform-openshell-node-reader + + - it: creates the ClusterRole when legacy values omit the rbac block + set: + rbac: null + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.name + value: openshell-node-reader-my-namespace diff --git a/charts/openshell/tests/clusterrolebinding_test.yaml b/charts/openshell/tests/clusterrolebinding_test.yaml new file mode 100644 index 000000000..4cbf6d7d9 --- /dev/null +++ b/charts/openshell/tests/clusterrolebinding_test.yaml @@ -0,0 +1,76 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: ClusterRoleBinding RBAC +templates: + - templates/clusterrolebinding.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: includes the release namespace in the ClusterRoleBinding name + asserts: + - equal: + path: metadata.name + value: openshell-node-reader-my-namespace + + - it: references the namespace-scoped ClusterRole in roleRef + asserts: + - equal: + path: roleRef.name + value: openshell-node-reader-my-namespace + + - it: binds the service account in the release namespace + asserts: + - contains: + path: subjects + content: + kind: ServiceAccount + name: openshell + namespace: my-namespace + + - it: omits the ClusterRoleBinding when cluster-scoped RBAC is disabled + set: + rbac.clusterScoped.create: false + asserts: + - hasDocuments: + count: 0 + + - it: omits the ClusterRoleBinding when all chart-managed RBAC is disabled + set: + rbac.create: false + asserts: + - hasDocuments: + count: 0 + + - it: uses the configured ClusterRoleBinding and ClusterRole names + set: + rbac.clusterScoped.clusterRoleName: platform-openshell-node-reader + rbac.clusterScoped.clusterRoleBindingName: platform-openshell-node-reader-binding + asserts: + - equal: + path: metadata.name + value: platform-openshell-node-reader-binding + - equal: + path: roleRef.name + value: platform-openshell-node-reader + + - it: binds the custom gateway service account created by the namespaced release + set: + serviceAccount.create: false + serviceAccount.name: my-existing-sa + asserts: + - contains: + path: subjects + content: + kind: ServiceAccount + name: my-existing-sa + namespace: my-namespace + + - it: creates the ClusterRoleBinding when legacy values omit the rbac block + set: + rbac: null + asserts: + - hasDocuments: + count: 1 diff --git a/charts/openshell/tests/credential_drivers_test.yaml b/charts/openshell/tests/credential_drivers_test.yaml new file mode 100644 index 000000000..d7f1907c0 --- /dev/null +++ b/charts/openshell/tests/credential_drivers_test.yaml @@ -0,0 +1,212 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: credential drivers +templates: + - templates/gateway-config.yaml + - templates/credential-storage-key-encryption-key-secret.yaml + - templates/statefulset.yaml + - templates/deployment.yaml + - templates/credential-secrets-role.yaml + - templates/credential-secrets-rolebinding.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: renders default encrypted credential storage by default + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'credential_drivers\s*=' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\.credential_storage\].*?key_encryption_key_env\s*=\s*"OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'key_encryption_key_path\s*=' + + - it: creates a retained default credential storage key-encryption key Secret by default + template: templates/credential-storage-key-encryption-key-secret.yaml + asserts: + - equal: + path: kind + value: Secret + - matchRegex: + path: metadata.name + pattern: 'credential-storage-key-encryption-key$' + - equal: + path: metadata.annotations["helm.sh/resource-policy"] + value: keep + - matchRegex: + path: data["key-encryption-key"] + pattern: '.+' + + - it: injects the default credential storage key-encryption key Secret into the gateway pod by default + template: templates/statefulset.yaml + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY + valueFrom: + secretKeyRef: + name: openshell-credential-storage-key-encryption-key + key: key-encryption-key + + - it: renders Kubernetes Secrets credential driver config + template: templates/gateway-config.yaml + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'credential_drivers\s*=\s*\["kubernetes-secrets"\]' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.credential_drivers\.kubernetes-secrets\]\s*namespace\s*=\s*"provider-secrets"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'allow_reference_namespace' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.credential_drivers\.kubernetes-secrets\][^\[]*(workspace_mode|gateway_id)' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'transport\s*=\s*"in_tree"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.credential_storage\]' + + - it: renders Vault credential driver config + template: templates/gateway-config.yaml + set: + server.credentialDrivers.vault.enabled: true + server.credentialDrivers.vault.address: https://vault.vault.svc.cluster.local:8200 + server.credentialDrivers.vault.caConfigMapName: vault-ca + server.credentialDrivers.vault.role: openshell-gateway + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'credential_drivers\s*=\s*\["vault"\]' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.credential_drivers\.vault\].*?address\s*=\s*"https://vault\.vault\.svc\.cluster\.local:8200".*?ca_bundle\s*=\s*"/etc/openshell-tls/vault-ca/ca\.crt".*?auth_method\s*=\s*"kubernetes".*?role\s*=\s*"openshell-gateway"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'transport\s*=\s*"in_tree"' + + - it: rejects multiple enabled credential drivers + template: templates/statefulset.yaml + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.vault.enabled: true + server.credentialDrivers.vault.address: https://vault.vault.svc.cluster.local:8200 + server.credentialDrivers.vault.role: openshell-gateway + asserts: + - failedTemplate: + errorPattern: "only one external server.credentialDrivers backend can be enabled at a time" + + - it: mounts the Vault CA ConfigMap into the gateway + template: templates/statefulset.yaml + set: + server.credentialDrivers.vault.enabled: true + server.credentialDrivers.vault.address: https://vault.vault.svc.cluster.local:8200 + server.credentialDrivers.vault.caConfigMapName: vault-ca + server.credentialDrivers.vault.role: openshell-gateway + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: vault-ca + mountPath: /etc/openshell-tls/vault-ca + readOnly: true + - contains: + path: spec.template.spec.volumes + content: + name: vault-ca + configMap: + name: vault-ca + items: + - key: ca.crt + path: ca.crt + + - it: creates namespaced Kubernetes Secret manager RBAC + template: templates/credential-secrets-role.yaml + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + asserts: + - equal: + path: metadata.namespace + value: provider-secrets + - equal: + path: rules[0].resources[0] + value: secrets + - equal: + path: rules[0].verbs[0] + value: get + - contains: + path: rules[0].verbs + content: create + - contains: + path: rules[0].verbs + content: patch + - contains: + path: rules[0].verbs + content: delete + + - it: binds Kubernetes Secret manager RBAC to the gateway ServiceAccount + template: templates/credential-secrets-rolebinding.yaml + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + asserts: + - equal: + path: metadata.namespace + value: provider-secrets + - equal: + path: subjects[0].name + value: openshell + - equal: + path: subjects[0].namespace + value: my-namespace + + - it: allows default credential storage on a Deployment with an external database + template: templates/deployment.yaml + set: + workload.kind: deployment + server.externalDbSecret: openshell-pg + asserts: + - equal: + path: kind + value: Deployment + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY + valueFrom: + secretKeyRef: + name: openshell-credential-storage-key-encryption-key + key: key-encryption-key + + - it: allows default credential storage with multiple replicas and an external database + template: templates/statefulset.yaml + set: + replicaCount: 2 + server.externalDbSecret: openshell-pg + workload.allowMultiReplicaStatefulSet: true + asserts: + - equal: + path: spec.replicas + value: 2 + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_GATEWAY_CREDENTIAL_KEY_ENCRYPTION_KEY + valueFrom: + secretKeyRef: + name: openshell-credential-storage-key-encryption-key + key: key-encryption-key diff --git a/charts/openshell/tests/credential_secrets_namespace_test.yaml b/charts/openshell/tests/credential_secrets_namespace_test.yaml new file mode 100644 index 000000000..e102e35f7 --- /dev/null +++ b/charts/openshell/tests/credential_secrets_namespace_test.yaml @@ -0,0 +1,86 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: credential Secret namespace +templates: + - templates/credential-secrets-namespace.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: does not create a namespace by default + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + asserts: + - hasDocuments: + count: 0 + + - it: creates a retained credential namespace + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + server.credentialDrivers.kubernetesSecrets.createNamespace: true + asserts: + - isKind: + of: Namespace + - equal: + path: metadata.name + value: provider-secrets + - equal: + path: metadata.annotations["helm.sh/resource-policy"] + value: keep + + - it: requires a namespace other than the release namespace + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.createNamespace: true + asserts: + - failedTemplate: + errorPattern: createNamespace requires + + - it: leaves an existing namespace owned by something else untouched + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + server.credentialDrivers.kubernetesSecrets.createNamespace: true + kubernetesProvider: + scheme: + "v1/Namespace": + gvr: + version: "v1" + resource: "namespaces" + namespaced: false + objects: + - kind: Namespace + apiVersion: v1 + metadata: + name: provider-secrets + asserts: + - hasDocuments: + count: 0 + + - it: adopts a namespace retained from the same release + set: + server.credentialDrivers.kubernetesSecrets.enabled: true + server.credentialDrivers.kubernetesSecrets.namespace: provider-secrets + server.credentialDrivers.kubernetesSecrets.createNamespace: true + kubernetesProvider: + scheme: + "v1/Namespace": + gvr: + version: "v1" + resource: "namespaces" + namespaced: false + objects: + - kind: Namespace + apiVersion: v1 + metadata: + name: provider-secrets + annotations: + meta.helm.sh/release-name: openshell + meta.helm.sh/release-namespace: my-namespace + asserts: + - isKind: + of: Namespace diff --git a/charts/openshell/tests/gateway_config_test.yaml b/charts/openshell/tests/gateway_config_test.yaml new file mode 100644 index 000000000..eacfdfa2c --- /dev/null +++ b/charts/openshell/tests/gateway_config_test.yaml @@ -0,0 +1,1244 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: gateway TOML config shape +templates: + - templates/gateway-config.yaml + - templates/deployment.yaml + - templates/peer-role.yaml + - templates/peer-service.yaml + - templates/statefulset.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: disables the WebSocket tunnel unless explicitly enabled for an edge proxy + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^enable_websocket_tunnel\s*=\s*false$' + + - it: enables the WebSocket tunnel when requested + template: templates/gateway-config.yaml + set: + server.enableWebsocketTunnel: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^enable_websocket_tunnel\s*=\s*true$' + + - it: defaults to label admission with caller driver config disabled + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^allow_driver_config\s*=\s*false$' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\.resource_admission\]\s*enabled\s*=\s*true' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.drivers\.kubernetes\.resource_admission\.required_labels\]' + + - it: preserves explicit admission opt-out without enabling driver config + template: templates/gateway-config.yaml + set: + server.drivers.kubernetes.resourceAdmission.enabled: false + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^allow_driver_config\s*=\s*false$' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\.resource_admission\]\s*enabled\s*=\s*false' + + - it: renders replacement labels and literal workspace substitution + template: templates/gateway-config.yaml + set: + server.drivers.kubernetes.allowDriverConfig: true + server.drivers.kubernetes.resourceAdmission.requiredLabels: + platform.example.com/sandbox-attachable: "approved" + platform.example.com/team: '${workspace}' + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '"platform.example.com/sandbox-attachable" = "approved"' + - matchRegex: + path: data["gateway.toml"] + pattern: '"platform.example.com/team" = "\$\{workspace\}"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'openshell.ai/sandbox-attachable' + + - it: identifies the gateway by chart fullname by default + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^name\s*=\s*"openshell"$' + + - it: renders an explicit gateway name + template: templates/gateway-config.yaml + set: + server.name: production-us-west + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^name\s*=\s*"production-us-west"$' + + - it: renders schema version 2 and the Kubernetes compute driver selector + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\]\s*version\s*=\s*2' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\][^\[]*?compute_driver\s*=\s*"kubernetes"' + + - it: preserves a complete tagged gateway image reference + template: templates/statefulset.yaml + set: + gateway.image.registry: registry.example + gateway.image.repository: gateway + gateway.image.tag: branch + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: registry.example/gateway:branch + + - it: defaults a repository-only gateway image to the chart appVersion + template: templates/statefulset.yaml + set: + gateway.image.registry: registry.example + gateway.image.repository: gateway + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: registry.example/gateway:0.0.0 + + - it: applies global image values to gateway, supervisor, and sandbox runtime + template: templates/gateway-config.yaml + set: + global.image.registry: registry.example.com + global.image.tag: v0.1.0 + global.image.pullPolicy: IfNotPresent + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"registry\.example\.com/openshell/supervisor:v0\.1\.0"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_runtime_image\s*=\s*"registry\.example\.com/openshell/sandbox:v0\.1\.0"' + + - it: lets an individual registry override the global registry + template: templates/statefulset.yaml + set: + global.image.registry: registry.example.com + gateway.image.registry: mirror.example.com + gateway.image.repository: custom/gateway + global.image.tag: v0.1.0 + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: mirror.example.com/custom/gateway:v0.1.0 + + - it: applies the global registry to the gateway image + template: templates/statefulset.yaml + set: + global.image.registry: registry.example.com + global.image.tag: v0.1.0 + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: registry.example.com/openshell/gateway:v0.1.0 + + - it: defaults a repository-only sandbox image to latest + template: templates/gateway-config.yaml + set: + sandbox.image.repository: registry.example/sandbox + sandbox.image.tag: "" + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'default_image\s*=\s*"registry\.example/sandbox:latest"' + + - it: gives a gateway digest precedence over its tag + template: templates/statefulset.yaml + set: + gateway.image.registry: registry.example + gateway.image.repository: gateway + gateway.image.tag: ignored + gateway.image.digest: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + asserts: + - equal: + path: spec.template.spec.containers[0].image + value: registry.example/gateway@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + + - it: gives supervisor and sandbox digests precedence over their tags + template: templates/gateway-config.yaml + set: + supervisor.image.registry: registry.example + supervisor.image.repository: supervisor + supervisor.image.tag: ignored + supervisor.image.digest: sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + sandbox.image.repository: registry.example/sandbox + sandbox.image.tag: ignored + sandbox.image.digest: sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'registry\.example/supervisor@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' + - matchRegex: + path: data["gateway.toml"] + pattern: 'default_image\s*=\s*"registry\.example/sandbox@sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"' + + # Regression for Drew's P2: a ConfigMap-only mutation in `helm upgrade` + # must roll the StatefulSet, otherwise pods keep running with stale config. + - it: annotates the StatefulSet pod template with a ConfigMap checksum + template: templates/statefulset.yaml + asserts: + - exists: + path: spec.template.metadata.annotations["checksum/gateway-config"] + + - it: mounts gateway.toml as a regular read-only subPath file + template: templates/statefulset.yaml + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: gateway-config + mountPath: /etc/openshell/gateway.toml + subPath: gateway.toml + readOnly: true + - contains: + path: spec.template.spec.volumes + content: + name: gateway-config + configMap: + name: openshell-config + + - it: renders a StatefulSet by default + template: templates/statefulset.yaml + asserts: + - equal: + path: kind + value: StatefulSet + + - it: treats a null workload map as the default StatefulSet + template: templates/statefulset.yaml + set: + workload: null + asserts: + - equal: + path: kind + value: StatefulSet + + - it: uses a stable gateway container name + template: templates/statefulset.yaml + asserts: + - equal: + path: spec.template.spec.containers[0].name + value: openshell-gateway + + - it: enables anonymous telemetry by default + template: templates/statefulset.yaml + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_TELEMETRY_ENABLED + value: "true" + + - it: disables anonymous telemetry when configured + template: templates/statefulset.yaml + set: + server.telemetryEnabled: false + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_TELEMETRY_ENABLED + value: "false" + + - it: renders OTLP tracing configuration when configured + template: templates/gateway-config.yaml + set: + server.otlp.endpoint: http://otel-collector.observability.svc:4317 + server.otlp.serviceName: production-gateway + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\.otlp\].*?endpoint\s*=\s*"http://otel-collector\.observability\.svc:4317".*?service_name\s*=\s*"production-gateway"' + + - it: omits OTLP tracing configuration by default + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.otlp\]' + + - it: renders OIDC transport security settings + template: templates/gateway-config.yaml + set: + server.oidc.issuer: https://issuer.example.com + server.oidc.dangerouslyAllowInsecureHttp: false + server.oidc.jwksAllowedOrigins[0]: https://keys.example.com + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^dangerously_allow_insecure_http\s*=\s*false$' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^jwks_allowed_origins\s*=\s*\["https://keys.example.com"\]$' + + - it: treats a null OTLP map as disabled + template: templates/gateway-config.yaml + set: + server.otlp: null + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.otlp\]' + + - it: mounts the OIDC CA bundle when TLS is disabled + template: templates/statefulset.yaml + set: + server.disableTls: true + server.oidc.issuer: https://issuer.example.com + server.oidc.caConfigMapName: openshell-oidc-ca + asserts: + - equal: + path: spec.template.spec.containers[0].volumeMounts[4].name + value: oidc-ca + - equal: + path: spec.template.spec.containers[0].volumeMounts[4].mountPath + value: /etc/openshell-tls/oidc-ca + - equal: + path: spec.template.spec.volumes[3].name + value: oidc-ca + - equal: + path: spec.template.spec.volumes[3].configMap.name + value: openshell-oidc-ca + + # Regression for the P1 bug Drew flagged: grpc_endpoint MUST live in the + # Kubernetes driver table, not in [openshell.gateway]. The gateway-side + # schema has `deny_unknown_fields` and no `grpc_endpoint` field, so writing + # it at gateway scope makes `config_file::load` reject the default install. + - it: renders grpc_endpoint under [openshell.drivers.kubernetes], not [openshell.gateway] + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?grpc_endpoint' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\][^\[]*?grpc_endpoint' + + - it: applies the global pull policy by default + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)supervisor_image_pull_policy\s*=\s*"if_not_present".*?sandbox_runtime_image_pull_policy\s*=\s*"if_not_present"' + + - it: renders canonical image pull policies in the Kubernetes driver table + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: if_not_present + supervisor.image.pullPolicy: never + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?image_pull_policy\s*=\s*"if_not_present".*?supervisor_image_pull_policy\s*=\s*"never"' + + - it: translates legacy Kubernetes pull policy values to canonical gateway values + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: Always + supervisor.image.pullPolicy: IfNotPresent + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?image_pull_policy\s*=\s*"always".*?supervisor_image_pull_policy\s*=\s*"if_not_present"' + + - it: rejects unsupported sandbox image pull policies + template: templates/statefulset.yaml + set: + sandbox.image.pullPolicy: Sometimes + asserts: + - failedTemplate: + errorMessage: 'image pull policy "Sometimes" must be one of: always, if_not_present, never, Always, IfNotPresent, Never' + + - it: rejects unsupported supervisor image pull policies + template: templates/statefulset.yaml + set: + supervisor.image.pullPolicy: newer + asserts: + - failedTemplate: + errorMessage: 'image pull policy "newer" must be one of: always, if_not_present, never, Always, IfNotPresent, Never' + + - it: renders driver-owned Kubernetes settings only in its driver table + template: templates/gateway-config.yaml + set: + server.hostGatewayIP: 10.0.0.1 + server.enableUserNamespaces: true + supervisor.image.tag: test + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?namespace\s*=\s*"my-namespace".*?default_image\s*=.*?supervisor_image\s*=.*?host_gateway_ip\s*=\s*"10\.0\.0\.1".*?client_tls_secret_name\s*=.*?service_account_name\s*=\s*"openshell-sandbox".*?enable_user_namespaces\s*=\s*true.*?sa_token_ttl_secs\s*=' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\][^\[]*?(sandbox_namespace|default_image|supervisor_image|client_tls_secret_name|service_account_name|host_gateway_ip|enable_user_namespaces|sa_token_ttl_secs)\s*=' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'guest_tls_(ca|cert|key)\s*=' + + - it: renders user namespace enablement under [openshell.drivers.kubernetes] + template: templates/gateway-config.yaml + set: + server.enableUserNamespaces: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?enable_user_namespaces\s*=\s*true' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\][^\[]*?enable_user_namespaces' + + - it: renders operator-owned upstream proxy settings under the Kubernetes driver + template: templates/gateway-config.yaml + set: + upstreamProxy.url: http://proxy.corp.example:8080 + upstreamProxy.noProxy: .svc.cluster.local,10.96.0.0/12 + upstreamProxy.authSecret.name: corporate-proxy-auth + upstreamProxy.authSecret.key: credentials + upstreamProxy.authAllowInsecure: true + upstreamProxy.connectByHostname: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?https_proxy\s*=\s*"http://proxy\.corp\.example:8080"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'proxy_auth_secret_name\s*=\s*"corporate-proxy-auth"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'proxy_auth_secret_key\s*=\s*"credentials"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'proxy_auth_allow_insecure\s*=\s*true' + - matchRegex: + path: data["gateway.toml"] + pattern: 'no_proxy\s*=\s*"\.svc\.cluster\.local,10\.96\.0\.0/12"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'proxy_connect_by_hostname\s*=\s*true' + + - it: renders proxy_ca_bundle when a corporate CA ConfigMap is configured + template: templates/gateway-config.yaml + set: + upstreamProxy.url: https://proxy.corp.example:3130 + upstreamProxy.caBundle.configMapName: corporate-proxy-ca + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?proxy_ca_bundle\s*=\s*"/etc/openshell-tls/proxy-ca/ca\.crt"' + + - it: omits proxy_ca_bundle when no corporate CA ConfigMap is configured + template: templates/gateway-config.yaml + set: + upstreamProxy.url: http://proxy.corp.example:8080 + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'proxy_ca_bundle\s*=' + + - it: renders the default supervisor image + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"ghcr\.io/nvidia/openshell/supervisor:0\.0\.0"' + + - it: uses the gateway built-in sandbox runtime image by default + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_runtime_image\s*=' + + - it: renders independent sandbox runtime and supervisor image overrides + template: templates/gateway-config.yaml + set: + sandboxRuntime.image.registry: registry.example.com + sandboxRuntime.image.repository: openshell/sandbox + sandboxRuntime.image.tag: sandbox-build + sandboxRuntime.image.pullPolicy: Always + supervisor.image.registry: registry.example.com + supervisor.image.repository: openshell/supervisor + supervisor.image.tag: supervisor-build + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_runtime_image\s*=\s*"registry\.example\.com/openshell/sandbox:sandbox-build"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_runtime_image_pull_policy\s*=\s*"always"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"registry\.example\.com/openshell/supervisor:supervisor-build"' + + - it: gives a sandbox runtime digest precedence over its tag + template: templates/gateway-config.yaml + set: + sandboxRuntime.image.registry: registry.example.com + sandboxRuntime.image.repository: openshell/sandbox + sandboxRuntime.image.tag: ignored + sandboxRuntime.image.digest: sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_runtime_image\s*=\s*"registry\.example\.com/openshell/sandbox@sha256:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"' + + - it: renders a supervisor tag override with the official repository + template: templates/gateway-config.yaml + set: + supervisor.image.tag: 1.2.3 + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"ghcr\.io/nvidia/openshell/supervisor:1\.2\.3"' + + - it: renders a supervisor repository override with the default chart tag + template: templates/gateway-config.yaml + set: + supervisor.image.registry: registry.example.com + supervisor.image.repository: openshell/supervisor + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"registry\.example\.com/openshell/supervisor:0\.0\.0"' + + - it: renders complete supervisor repository and tag overrides + template: templates/gateway-config.yaml + set: + supervisor.image.registry: registry.example.com + supervisor.image.repository: openshell/supervisor + supervisor.image.tag: supervisor-build + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'supervisor_image\s*=\s*"registry\.example\.com/openshell/supervisor:supervisor-build"' + + - it: configures managed SSH isolation with the gateway peer + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\.managed_ssh_ingress\].*?enabled\s*=\s*true.*?gateway_namespace\s*=\s*"my-namespace".*?gateway_pod_selector\s*=.*?app\.kubernetes\.io/name.*?openshell' + + - it: renders sandbox image pull secrets under [openshell.drivers.kubernetes] + template: templates/gateway-config.yaml + set: + server.sandboxImagePullSecrets: + - name: regcred + - name: backup-regcred + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?image_pull_secrets\s*=\s*\["regcred", "backup-regcred"\]' + + - it: does not reuse gateway image pull secrets for sandbox pods + template: templates/gateway-config.yaml + set: + imagePullSecrets: + - name: gateway-regcred + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'image_pull_secrets\s*=' + + - it: does not render local mTLS user auth for Kubernetes deployments + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.mtls_auth\]' + + - it: does not allow unauthenticated users by default + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.auth\]' + + - it: renders explicit unauthenticated user dev mode when enabled + template: templates/gateway-config.yaml + set: + server.auth.allowUnauthenticatedUsers: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\.auth\].*?allow_unauthenticated_users\s*=\s*true' + + - it: omits the gRPC rate limit by default + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'grpc_rate_limit_requests\s*=' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'grpc_rate_limit_window_seconds\s*=' + + - it: renders fail-closed policy validation posture by default + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\].*?policy_validation_failure_mode\s*=\s*"fail_closed"' + + - it: renders retain-last-valid policy validation posture + template: templates/gateway-config.yaml + set: + server.policyValidationFailureMode: retain_last_valid + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\].*?policy_validation_failure_mode\s*=\s*"retain_last_valid"' + + - it: renders the gRPC rate limit under [openshell.gateway] when both values are positive + template: templates/gateway-config.yaml + set: + server.grpcRateLimit.requests: 120 + server.grpcRateLimit.windowSeconds: 60 + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\].*?grpc_rate_limit_requests\s*=\s*120' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.gateway\].*?grpc_rate_limit_window_seconds\s*=\s*60' + + # The validation lives in gateway-config.yaml but surfaces through the + # statefulset checksum include, so the failure is asserted against that + # template (mirrors the postgres serviceBindings failure test below). + - it: fails to render when only requests is positive + template: templates/statefulset.yaml + set: + server.grpcRateLimit.requests: 120 + asserts: + - failedTemplate: + errorMessage: "server.grpcRateLimit requires both requests and windowSeconds to be positive to enable rate limiting, or both 0/unset to disable it" + + - it: fails to render when only windowSeconds is positive + template: templates/statefulset.yaml + set: + server.grpcRateLimit.windowSeconds: 60 + asserts: + - failedTemplate: + errorMessage: "server.grpcRateLimit requires both requests and windowSeconds to be positive to enable rate limiting, or both 0/unset to disable it" + + - it: fails to render when requests is negative + template: templates/statefulset.yaml + set: + server.grpcRateLimit.requests: -1 + asserts: + - failedTemplate: + errorMessage: "server.grpcRateLimit.requests and server.grpcRateLimit.windowSeconds must not be negative; they map to unsigned gateway settings" + + - it: fails to render when windowSeconds is negative + template: templates/statefulset.yaml + set: + server.grpcRateLimit.windowSeconds: -5 + asserts: + - failedTemplate: + errorMessage: "server.grpcRateLimit.requests and server.grpcRateLimit.windowSeconds must not be negative; they map to unsigned gateway settings" + + - it: uses the configured existing sandbox service account name + template: templates/gateway-config.yaml + set: + sandboxServiceAccount.create: false + sandboxServiceAccount.name: precreated-sandbox + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?service_account_name\s*=\s*"precreated-sandbox"' + + - it: omits server_sans when no DNS SANs are configured + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'server_sans\s*=' + + - it: emits disable_tls=true and omits the [openshell.gateway.tls] section when disableTls is set + set: + server.disableTls: true + certManager.enabled: false + pkiInitJob.enabled: false + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'disable_tls\s*=\s*true' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.tls\]' + + - it: omits client_ca_path from the TLS section when mTLS is disabled + template: templates/gateway-config.yaml + set: + server.tls.enableMtls: false + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.tls\]' + - matchRegex: + path: data["gateway.toml"] + pattern: 'cert_path\s*=' + - matchRegex: + path: data["gateway.toml"] + pattern: 'key_path\s*=' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=' + + - it: renders client_ca_path for built-in PKI by default + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=\s*"/etc/openshell-tls/client-ca/ca\.crt"' + + - it: omits client_ca_path when clientCaSecretName is empty for HTTPS-only mode + template: templates/gateway-config.yaml + set: + server.tls.clientCaSecretName: "" + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.tls\]' + - matchRegex: + path: data["gateway.toml"] + pattern: 'cert_path\s*=\s*"/etc/openshell-tls/server/tls\.crt"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'key_path\s*=\s*"/etc/openshell-tls/server/tls\.key"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=' + + - it: keeps external server certificate fields when clientCaSecretName is empty + template: templates/gateway-config.yaml + set: + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverDnsNames: + - gateway.example.com + server.tls.clientCaSecretName: "" + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=' + - matchRegex: + path: data["gateway.toml"] + pattern: 'external_cert_path\s*=\s*"/etc/openshell-tls/server-external/tls\.crt"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'external_key_path\s*=\s*"/etc/openshell-tls/server-external/tls\.key"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'external_server_names\s*=\s*\["gateway\.example\.com"\]' + + - it: omits client_ca_path for cert-manager shared CA when clientCaSecretName is empty + template: templates/gateway-config.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: true + server.tls.clientCaSecretName: "" + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.tls\]' + - matchRegex: + path: data["gateway.toml"] + pattern: 'cert_path\s*=\s*"/etc/openshell-tls/server/tls\.crt"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'key_path\s*=\s*"/etc/openshell-tls/server/tls\.key"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=' + + - it: omits client_ca_path when cert-manager owns TLS and no client CA secret is set + template: templates/gateway-config.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: false + pkiInitJob.enabled: true + server.tls.clientCaSecretName: "" + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.tls\]' + - matchRegex: + path: data["gateway.toml"] + pattern: 'cert_path\s*=\s*"/etc/openshell-tls/server/tls\.crt"' + - matchRegex: + path: data["gateway.toml"] + pattern: 'key_path\s*=\s*"/etc/openshell-tls/server/tls\.key"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'client_ca_path\s*=' + + - it: renders server_sans from certManager.serverDnsNames + set: + certManager.enabled: true + certManager.serverDnsNames: + - openshell + - "*.dev.openshell.localhost" + pkiInitJob.enabled: false + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: 'server_sans\s*=\s*\["openshell", "\*\.dev\.openshell\.localhost"\]' + + - it: passes sqlite db-url via --db-url arg by default + template: templates/statefulset.yaml + asserts: + - contains: + path: spec.template.spec.containers[0].args + content: "sqlite:/var/openshell/openshell.db" + + - it: fails when clientCaSecretName is null + template: templates/statefulset.yaml + set: + server.tls.clientCaSecretName: null + asserts: + - failedTemplate: + errorMessage: "server.tls.clientCaSecretName cannot be null; omit the key to use the chart default (openshell-server-client-ca), or set to \"\" to disable client certificate verification for HTTPS-only mode" + + - it: configures gateway peer identity and projected peer token + template: templates/statefulset.yaml + asserts: + - equal: + path: spec.serviceName + value: openshell-peer + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_SERVICE_NAME + value: openshell-peer + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TOKEN_AUDIENCE + value: openshell-gateway-peer + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TLS_SERVER_NAME + value: openshell.my-namespace.svc.cluster.local + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TLS_CA_FILE + value: /etc/openshell-tls/server/ca.crt + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TLS_CERT_FILE + value: /etc/openshell-tls/peer-client/tls.crt + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TLS_KEY_FILE + value: /etc/openshell-tls/peer-client/tls.key + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: gateway-peer-token + mountPath: /var/run/secrets/openshell-peer + readOnly: true + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: peer-client-tls + mountPath: /etc/openshell-tls/peer-client + readOnly: true + - equal: + path: spec.template.spec.volumes[2].projected.sources[0].serviceAccountToken.audience + value: openshell-gateway-peer + + - it: configures gateway peer identity and projected peer token for Deployment + template: templates/deployment.yaml + set: + workload.kind: deployment + server.externalDbSecret: my-pg-secret + server.disableTls: true + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_ENDPOINT + value: http://$(OPENSHELL_POD_IP):8080 + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_SERVICE_NAME + value: openshell-peer + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_PEER_TOKEN_AUDIENCE + value: openshell-gateway-peer + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: gateway-peer-token + mountPath: /var/run/secrets/openshell-peer + readOnly: true + - equal: + path: spec.template.spec.volumes[2].projected.sources[0].serviceAccountToken.audience + value: openshell-gateway-peer + + - it: renders headless gateway peer service + template: templates/peer-service.yaml + asserts: + - equal: + path: metadata.name + value: openshell-peer + - equal: + path: spec.clusterIP + value: None + - equal: + path: spec.publishNotReadyAddresses + value: true + + - it: grants release-namespace pod lookup for gateway peer identity validation + template: templates/peer-role.yaml + asserts: + - hasDocuments: + count: 2 + - equal: + path: metadata.namespace + value: my-namespace + documentIndex: 0 + - equal: + path: rules[0].resources[0] + value: pods + documentIndex: 0 + - equal: + path: subjects[0].name + value: openshell + documentIndex: 1 + + - it: fails when legacy postgres.enabled is set + template: templates/statefulset.yaml + set: + postgres.enabled: true + asserts: + - failedTemplate: + errorPattern: "postgres.enabled was removed" + + - it: fails when multiple replicas use the default SQLite database + template: templates/statefulset.yaml + set: + replicaCount: 2 + asserts: + - failedTemplate: + errorPattern: "replicaCount > 1 requires server.externalDbSecret" + + - it: renders a Deployment for external database-backed gateway workloads + template: templates/deployment.yaml + set: + workload.kind: deployment + replicaCount: 2 + server.externalDbSecret: my-pg-secret + server.credentialDrivers.kubernetesSecrets.enabled: true + asserts: + - equal: + path: kind + value: Deployment + - equal: + path: spec.replicas + value: 2 + - equal: + path: spec.template.spec.containers[0].name + value: openshell-gateway + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: gateway-config + mountPath: /etc/openshell/gateway.toml + subPath: gateway.toml + readOnly: true + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: openshell-data + mountPath: /var/openshell + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_DB_URL + valueFrom: + secretKeyRef: + name: my-pg-secret + key: uri + + - it: fails when a Deployment uses the default SQLite database + template: templates/statefulset.yaml + set: + workload.kind: deployment + asserts: + - failedTemplate: + errorPattern: "workload.kind=deployment requires server.externalDbSecret" + + - it: fails when multiple replicas use a StatefulSet without an override + template: templates/statefulset.yaml + set: + replicaCount: 2 + server.externalDbSecret: my-pg-secret + asserts: + - failedTemplate: + errorPattern: "replicaCount > 1 with workload.kind=statefulset requires workload.allowMultiReplicaStatefulSet=true" + + - it: allows a multi-replica StatefulSet with an external database and explicit override + template: templates/statefulset.yaml + set: + replicaCount: 2 + server.externalDbSecret: my-pg-secret + workload.allowMultiReplicaStatefulSet: true + server.credentialDrivers.kubernetesSecrets.enabled: true + asserts: + - equal: + path: kind + value: StatefulSet + - equal: + path: spec.replicas + value: 2 + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: openshell-data + mountPath: /var/openshell + + - it: fails when workload.kind is invalid + template: templates/statefulset.yaml + set: + workload.kind: daemonset + asserts: + - failedTemplate: + errorPattern: "workload.kind must be one of: statefulset, deployment" + + - it: does not pass --db-url in args when externalDbSecret is set + template: templates/statefulset.yaml + set: + server.externalDbSecret: my-pg-secret + asserts: + - notContains: + path: spec.template.spec.containers[0].args + content: "--db-url" + + - it: references externalDbSecret when set + template: templates/statefulset.yaml + set: + server.externalDbSecret: my-pg-secret + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_DB_URL + valueFrom: + secretKeyRef: + name: my-pg-secret + key: uri + + - it: renders HA external database configuration from the CI overlay as a Deployment + template: templates/deployment.yaml + values: + - ../ci/values-high-availability.yaml + asserts: + - equal: + path: kind + value: Deployment + - equal: + path: spec.replicas + value: 2 + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_DB_URL + valueFrom: + secretKeyRef: + name: openshell-ha-pg + key: uri + - it: renders provider SPIFFE token grants while keeping gateway JWT auth + set: + server.providerTokenGrants.spiffe.enabled: true + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.gateway_jwt\]' + - matchRegex: + path: data["gateway.toml"] + pattern: 'provider_spiffe_workload_api_socket_path\s*=\s*"/spiffe-workload-api/spire-agent\.sock"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '\[openshell\.gateway\.spiffe\]' + + - it: mounts the gateway SPIFFE socket while keeping sandbox JWT auth + set: + server.providerTokenGrants.spiffe.enabled: true + template: templates/statefulset.yaml + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET + value: /spiffe-workload-api/spire-agent.sock + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: sandbox-jwt + mountPath: /etc/openshell-jwt + readOnly: true + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: spiffe-workload-api + mountPath: /spiffe-workload-api + readOnly: true + - contains: + path: spec.template.spec.volumes + content: + name: sandbox-jwt + secret: + defaultMode: 256 + secretName: openshell-jwt-keys + - contains: + path: spec.template.spec.volumes + content: + name: spiffe-workload-api + csi: + driver: csi.spiffe.io + readOnly: true + + - it: fails when serverIssuerRef is set but certManager is disabled + template: templates/statefulset.yaml + set: + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.enabled: false + asserts: + - failedTemplate: + errorMessage: "certManager.serverIssuerRef.name is set but certManager.enabled is false \u2014 the external server certificate, its Secret mount, and the gateway TLS configuration all require cert-manager to be enabled. Set certManager.enabled=true or remove certManager.serverIssuerRef.name." + + - it: does not render the schema-v1 compute_drivers selector + template: templates/gateway-config.yaml + asserts: + - notMatchRegex: + path: data["gateway.toml"] + pattern: "(?m)^\\s*compute_drivers\\s*=" + + - it: uses the release fullname and namespace for default Kubernetes settings + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)name\\s*=\\s*\\\"openshell\\\".*?\\[openshell\\.drivers\\.kubernetes\\].*?namespace\\s*=\\s*\\\"my-namespace\\\".*?grpc_endpoint\\s*=\\s*\\\"https://openshell\\.my-namespace\\.svc\\.cluster\\.local:8080\\\"" + + - it: uses an explicit server grpc endpoint verbatim + template: templates/gateway-config.yaml + set: + server.grpcEndpoint: https://gateway.example.test:9443 + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?grpc_endpoint\\s*=\\s*\\\"https://gateway\\.example\\.test:9443\\\"" + + - it: uses HTTP callback and omits client TLS secret when TLS is disabled + template: templates/gateway-config.yaml + set: + server.disableTls: true + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?grpc_endpoint\\s*=\\s*\\\"http://openshell\\.my-namespace\\.svc\\.cluster\\.local:8080\\\"" + - notMatchRegex: + path: data["gateway.toml"] + pattern: "client_tls_secret_name\\s*=" + + - it: accepts Always pull policy spelling for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: Always + supervisor.image.pullPolicy: Always + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"always\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"always\\\"" + + - it: accepts IfNotPresent pull policy spelling for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: IfNotPresent + supervisor.image.pullPolicy: IfNotPresent + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"if_not_present\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"if_not_present\\\"" + + - it: accepts Never pull policy spelling for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: Never + supervisor.image.pullPolicy: Never + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"never\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"never\\\"" + + - it: accepts canonical lowercase pull policies for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: always + supervisor.image.pullPolicy: always + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"always\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"always\\\"" + + - it: accepts canonical if_not_present pull policies for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: if_not_present + supervisor.image.pullPolicy: if_not_present + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"if_not_present\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"if_not_present\\\"" + + - it: accepts canonical never pull policies for sandbox and supervisor + template: templates/gateway-config.yaml + set: + sandbox.image.pullPolicy: never + supervisor.image.pullPolicy: never + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: "(?ms)\\[openshell\\.drivers\\.kubernetes\\].*?image_pull_policy\\s*=\\s*\\\"never\\\".*?supervisor_image_pull_policy\\s*=\\s*\\\"never\\\"" diff --git a/charts/openshell/tests/gateway_pod_security_context_test.yaml b/charts/openshell/tests/gateway_pod_security_context_test.yaml new file mode 100644 index 000000000..cdb70a05e --- /dev/null +++ b/charts/openshell/tests/gateway_pod_security_context_test.yaml @@ -0,0 +1,36 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: gateway pod securityContext +templates: + - templates/gateway-config.yaml + - templates/statefulset.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: renders the pod securityContext from podSecurityContext by default + template: templates/statefulset.yaml + asserts: + - equal: + path: spec.template.spec.securityContext.fsGroup + value: 1000 + + - it: renders an explicitly set podSecurityContext + template: templates/statefulset.yaml + set: + podSecurityContext: + fsGroup: 2000 + asserts: + - equal: + path: spec.template.spec.securityContext.fsGroup + value: 2000 + + - it: omits the pod securityContext block when podSecurityContext is null + template: templates/statefulset.yaml + set: + podSecurityContext: null + asserts: + - notExists: + path: spec.template.spec.securityContext diff --git a/charts/openshell/tests/gateway_upstream_proxy_ca_test.yaml b/charts/openshell/tests/gateway_upstream_proxy_ca_test.yaml new file mode 100644 index 000000000..5663654fd --- /dev/null +++ b/charts/openshell/tests/gateway_upstream_proxy_ca_test.yaml @@ -0,0 +1,110 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: gateway upstream proxy CA volume +templates: + - templates/gateway-config.yaml + - templates/statefulset.yaml + - templates/deployment.yaml +release: + name: openshell + namespace: my-namespace + +tests: + # The gateway workload template is shared by the StatefulSet and the + # Deployment, so both kinds must mount the operator's CA bundle. + - it: mounts the corporate proxy CA into the gateway StatefulSet + template: templates/statefulset.yaml + set: + upstreamProxy.url: https://proxy.corp.example:3130 + upstreamProxy.caBundle.configMapName: corporate-proxy-ca + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: upstream-proxy-ca + mountPath: /etc/openshell-tls/proxy-ca + readOnly: true + - contains: + path: spec.template.spec.volumes + content: + name: upstream-proxy-ca + configMap: + name: corporate-proxy-ca + items: + - key: ca.crt + path: ca.crt + + - it: mounts the corporate proxy CA into the gateway Deployment + template: templates/deployment.yaml + set: + workload.kind: deployment + server.externalDbSecret: openshell-pg + upstreamProxy.url: https://proxy.corp.example:3130 + upstreamProxy.caBundle.configMapName: corporate-proxy-ca + asserts: + - contains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: upstream-proxy-ca + mountPath: /etc/openshell-tls/proxy-ca + readOnly: true + - contains: + path: spec.template.spec.volumes + content: + name: upstream-proxy-ca + configMap: + name: corporate-proxy-ca + items: + - key: ca.crt + path: ca.crt + + # The mounted filename stays ca.crt whatever the operator's ConfigMap key is, + # so the rendered proxy_ca_bundle path never has to vary. + - it: projects a custom ConfigMap key onto the fixed mounted filename + template: templates/statefulset.yaml + set: + upstreamProxy.url: https://proxy.corp.example:3130 + upstreamProxy.caBundle.configMapName: openshift-trusted-ca + upstreamProxy.caBundle.key: ca-bundle.crt + asserts: + - contains: + path: spec.template.spec.volumes + content: + name: upstream-proxy-ca + configMap: + name: openshift-trusted-ca + items: + - key: ca-bundle.crt + path: ca.crt + + # Kubernetes requires both ConfigMap reference fields to be strings. Quote + # numeric-looking operator values in the rendered workload so YAML does not + # decode them as numbers before typed Kubernetes validation. + - it: preserves numeric-looking ConfigMap references as strings + template: templates/statefulset.yaml + set: + upstreamProxy.url: https://proxy.corp.example:3130 + upstreamProxy.caBundle.configMapName: "123" + upstreamProxy.caBundle.key: "456" + asserts: + - contains: + path: spec.template.spec.volumes + content: + name: upstream-proxy-ca + configMap: + name: "123" + items: + - key: "456" + path: ca.crt + + - it: adds no volume when no corporate proxy CA is configured + template: templates/statefulset.yaml + set: + upstreamProxy.url: http://proxy.corp.example:8080 + asserts: + - notContains: + path: spec.template.spec.volumes + content: + name: upstream-proxy-ca + any: true diff --git a/charts/openshell/tests/grpc_endpoint_test.yaml b/charts/openshell/tests/grpc_endpoint_test.yaml new file mode 100644 index 000000000..82b3ca5cb --- /dev/null +++ b/charts/openshell/tests/grpc_endpoint_test.yaml @@ -0,0 +1,26 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: Kubernetes supervisor gateway endpoint + +templates: + - templates/gateway-config.yaml + +release: + name: team-a + namespace: gateway-system + +tests: + - it: derives callback from the gateway Service when sandbox namespace differs + set: + server.sandboxNamespace: agent-sandboxes + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^namespace\s*=\s*"agent-sandboxes"$' + - matchRegex: + path: data["gateway.toml"] + pattern: '(?m)^grpc_endpoint\s*=\s*"https://team-a-openshell\.gateway-system\.svc\.cluster\.local:8080"$' + - notMatchRegex: + path: data["gateway.toml"] + pattern: '(?m)^grpc_endpoint\s*=.*agent-sandboxes' diff --git a/charts/openshell/tests/rbac_test.yaml b/charts/openshell/tests/rbac_test.yaml new file mode 100644 index 000000000..3698465e2 --- /dev/null +++ b/charts/openshell/tests/rbac_test.yaml @@ -0,0 +1,95 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: Gateway RBAC creation +templates: + - templates/role.yaml + - templates/rolebinding.yaml + - templates/peer-role.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: creates the namespaced sandbox Role by default + template: templates/role.yaml + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.name + value: openshell-sandbox + + - it: creates the namespaced sandbox RoleBinding by default + template: templates/rolebinding.yaml + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.name + value: openshell-sandbox + + - it: creates the gateway peer Role and RoleBinding by default + template: templates/peer-role.yaml + asserts: + - hasDocuments: + count: 2 + - equal: + path: metadata.name + value: openshell-peer + + - it: keeps the namespaced RBAC when only cluster-scoped RBAC is disabled + set: + rbac.clusterScoped.create: false + asserts: + - hasDocuments: + count: 1 + template: templates/role.yaml + + - it: keeps the peer RBAC when only cluster-scoped RBAC is disabled + template: templates/peer-role.yaml + set: + rbac.clusterScoped.create: false + asserts: + - hasDocuments: + count: 2 + + - it: omits the namespaced sandbox Role when chart-managed RBAC is disabled + template: templates/role.yaml + set: + rbac.create: false + asserts: + - hasDocuments: + count: 0 + + - it: omits the namespaced sandbox RoleBinding when chart-managed RBAC is disabled + template: templates/rolebinding.yaml + set: + rbac.create: false + asserts: + - hasDocuments: + count: 0 + + - it: omits the gateway peer Role and RoleBinding when chart-managed RBAC is disabled + template: templates/peer-role.yaml + set: + rbac.create: false + asserts: + - hasDocuments: + count: 0 + + - it: creates the namespaced sandbox Role when legacy values omit the rbac block + template: templates/role.yaml + set: + rbac: null + asserts: + - hasDocuments: + count: 1 + + - it: creates the peer Role and RoleBinding when legacy values omit the rbac block + template: templates/peer-role.yaml + set: + rbac: null + asserts: + - hasDocuments: + count: 2 diff --git a/charts/openshell/tests/route_test.yaml b/charts/openshell/tests/route_test.yaml new file mode 100644 index 000000000..6de943bee --- /dev/null +++ b/charts/openshell/tests/route_test.yaml @@ -0,0 +1,118 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: OpenShift Route +templates: + - templates/route.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: renders nothing by default + asserts: + - hasDocuments: + count: 0 + + - it: renders a passthrough Route when enabled + set: + openshiftRoute.enabled: true + openshiftRoute.host: openshell.apps.example.com + asserts: + - isKind: + of: Route + - equal: + path: apiVersion + value: route.openshift.io/v1 + - equal: + path: spec.host + value: openshell.apps.example.com + - equal: + path: spec.to.kind + value: Service + - equal: + path: spec.to.name + value: openshell + - equal: + path: spec.port.targetPort + value: grpc + - equal: + path: spec.tls.termination + value: passthrough + - equal: + path: spec.wildcardPolicy + value: None + + - it: omits host when not set + set: + openshiftRoute.enabled: true + asserts: + - notExists: + path: spec.host + + - it: fails when passthrough Route is enabled with TLS disabled + set: + openshiftRoute.enabled: true + server.disableTls: true + asserts: + - failedTemplate: + errorMessage: "openshiftRoute.enabled=true requires TLS (server.disableTls must be false) \u2014 a passthrough Route forwards encrypted traffic by SNI, so the gateway must terminate its own TLS." + + - it: fails when external issuer is set but Route host is empty + set: + openshiftRoute.enabled: true + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverDnsNames: + - openshell.example.com + asserts: + - failedTemplate: + errorMessage: "openshiftRoute.enabled=true with certManager.serverIssuerRef requires an explicit openshiftRoute.host \u2014 without one, OpenShift generates a hostname absent from certManager.serverDnsNames, causing the gateway to serve its internal certificate to external clients." + + - it: accepts a Route host covered by a wildcard serverDnsNames entry + set: + openshiftRoute.enabled: true + openshiftRoute.host: gateway.example.com + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverDnsNames: + - "*.example.com" + asserts: + - equal: + path: spec.host + value: gateway.example.com + + - it: rejects a Route host not covered by wildcard or exact serverDnsNames + set: + openshiftRoute.enabled: true + openshiftRoute.host: gateway.other.com + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverDnsNames: + - "*.example.com" + asserts: + - failedTemplate: + errorMessage: "openshiftRoute.host \"gateway.other.com\" is not covered by certManager.serverDnsNames [*.example.com] \u2014 the Route will forward SNI for a hostname the external certificate does not cover, causing TLS verification failures for CLI clients. Exact names and single-level wildcards (e.g. *.example.com) are checked." + + - it: rejects a multi-level subdomain against a single-level wildcard + set: + openshiftRoute.enabled: true + openshiftRoute.host: deep.sub.example.com + certManager.enabled: true + certManager.serverIssuerRef.name: letsencrypt-prod + certManager.serverDnsNames: + - "*.example.com" + asserts: + - failedTemplate: + errorMessage: "openshiftRoute.host \"deep.sub.example.com\" is not covered by certManager.serverDnsNames [*.example.com] \u2014 the Route will forward SNI for a hostname the external certificate does not cover, causing TLS verification failures for CLI clients. Exact names and single-level wildcards (e.g. *.example.com) are checked." + + - it: renders custom annotations + set: + openshiftRoute.enabled: true + openshiftRoute.host: openshell.apps.example.com + openshiftRoute.annotations: + haproxy.router.openshift.io/balance: roundrobin + asserts: + - equal: + path: metadata.annotations["haproxy.router.openshift.io/balance"] + value: roundrobin diff --git a/charts/openshell/tests/sandbox_namespace_test.yaml b/charts/openshell/tests/sandbox_namespace_test.yaml new file mode 100644 index 000000000..992d84092 --- /dev/null +++ b/charts/openshell/tests/sandbox_namespace_test.yaml @@ -0,0 +1,143 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: sandboxNamespace defaulting +templates: + - templates/gateway-config.yaml + - templates/networkpolicy.yaml + - templates/role.yaml + - templates/rolebinding.yaml + - templates/serviceaccount.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: defaults the Kubernetes driver namespace to release namespace + template: templates/gateway-config.yaml + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?namespace\s*=\s*"my-namespace"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_namespace\s*=' + + - it: uses explicit sandboxNamespace for the Kubernetes driver + template: templates/gateway-config.yaml + set: + server.sandboxNamespace: other-ns + asserts: + - matchRegex: + path: data["gateway.toml"] + pattern: '(?ms)\[openshell\.drivers\.kubernetes\].*?namespace\s*=\s*"other-ns"' + - notMatchRegex: + path: data["gateway.toml"] + pattern: 'sandbox_namespace\s*=' + + - it: defaults NetworkPolicy namespace to release namespace + template: templates/networkpolicy.yaml + set: + networkPolicy.enabled: true + asserts: + - equal: + path: metadata.namespace + value: my-namespace + + - it: uses explicit sandboxNamespace for NetworkPolicy + template: templates/networkpolicy.yaml + set: + networkPolicy.enabled: true + server.sandboxNamespace: other-ns + asserts: + - equal: + path: metadata.namespace + value: other-ns + + - it: uses explicit sandboxNamespace for sandbox RBAC + template: templates/role.yaml + set: + server.sandboxNamespace: other-ns + asserts: + - equal: + path: metadata.namespace + value: other-ns + - contains: + path: rules + content: + apiGroups: [""] + resources: ["services"] + verbs: ["create", "get"] + - contains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["create", "delete"] + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["secrets"] + verbs: ["get"] + - notContains: + path: rules + content: + apiGroups: [""] + resources: ["configmaps"] + verbs: ["get"] + + - it: grants PVC admission reads only when caller driver config is enabled + template: templates/role.yaml + set: + server.drivers.kubernetes.allowDriverConfig: true + asserts: + - contains: + path: rules + content: + apiGroups: [""] + resources: ["persistentvolumeclaims"] + verbs: ["get"] + + - it: uses explicit sandboxNamespace for sandbox RoleBinding + template: templates/rolebinding.yaml + set: + server.sandboxNamespace: other-ns + asserts: + - equal: + path: metadata.namespace + value: other-ns + + - it: uses explicit sandboxNamespace for sandbox ServiceAccount + template: templates/serviceaccount.yaml + set: + server.sandboxNamespace: other-ns + asserts: + - equal: + path: metadata.namespace + value: other-ns + documentIndex: 1 + + - it: omits workspace resources in gateway-only mode + set: + workspaceResources.enabled: false + networkPolicy.enabled: true + asserts: + - hasDocuments: + count: 1 + template: templates/gateway-config.yaml + - hasDocuments: + count: 0 + template: templates/networkpolicy.yaml + - hasDocuments: + count: 0 + template: templates/role.yaml + - hasDocuments: + count: 0 + template: templates/rolebinding.yaml diff --git a/charts/openshell/tests/sandbox_service_account_test.yaml b/charts/openshell/tests/sandbox_service_account_test.yaml new file mode 100644 index 000000000..c9f10868f --- /dev/null +++ b/charts/openshell/tests/sandbox_service_account_test.yaml @@ -0,0 +1,41 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: sandbox service account +templates: + - templates/serviceaccount.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: creates gateway and sandbox service accounts by default + asserts: + - hasDocuments: + count: 2 + - equal: + path: metadata.name + value: openshell + documentIndex: 0 + - equal: + path: metadata.name + value: openshell-sandbox + documentIndex: 1 + + - it: uses the configured existing sandbox service account name + set: + sandboxServiceAccount.create: false + sandboxServiceAccount.name: precreated-sandbox + asserts: + - hasDocuments: + count: 1 + + - it: renders only the gateway service account in gateway-only mode + set: + workspaceResources.enabled: false + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.name + value: openshell diff --git a/charts/openshell/tests/statefulset_client_ca_test.yaml b/charts/openshell/tests/statefulset_client_ca_test.yaml new file mode 100644 index 000000000..6b3e320a8 --- /dev/null +++ b/charts/openshell/tests/statefulset_client_ca_test.yaml @@ -0,0 +1,141 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: statefulset client CA volume +templates: + - templates/gateway-config.yaml + - templates/statefulset.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: mounts the server TLS secret ca.crt as client CA for built-in PKI + template: templates/statefulset.yaml + set: + pkiInitJob.enabled: true + certManager.enabled: false + asserts: + - equal: + path: spec.template.spec.volumes[5].name + value: tls-client-ca + - equal: + path: spec.template.spec.volumes[5].secret.secretName + value: openshell-server-tls + - equal: + path: spec.template.spec.volumes[5].secret.items[0].key + value: ca.crt + + - it: shares the cert-manager server TLS ca.crt when clientCaFromServerTlsSecret is true + template: templates/statefulset.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: true + asserts: + - equal: + path: spec.template.spec.volumes[5].name + value: tls-client-ca + - equal: + path: spec.template.spec.volumes[5].secret.secretName + value: openshell-server-tls + - equal: + path: spec.template.spec.volumes[5].secret.items[0].key + value: ca.crt + + # Regression: with cert-manager enabled and pkiInitJob left at its default + # `true`, the client CA condition must honor certManager precedence and NOT + # treat pkiInitJob.enabled as built-in TLS. With clientCaFromServerTlsSecret=false + # the gateway must mount the separate clientCaSecretName, not the server TLS + # cert secret. + - it: uses clientCaSecretName under cert-manager even when pkiInitJob stays enabled + template: templates/statefulset.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: false + pkiInitJob.enabled: true + server.tls.clientCaSecretName: openshell-ca-tls + asserts: + - equal: + path: spec.template.spec.volumes[5].name + value: tls-client-ca + - equal: + path: spec.template.spec.volumes[5].secret.secretName + value: openshell-ca-tls + - notExists: + path: spec.template.spec.volumes[5].secret.items + + - it: omits client CA volume and mount when mTLS is disabled + template: templates/statefulset.yaml + set: + pkiInitJob.enabled: true + certManager.enabled: false + server.tls.enableMtls: false + asserts: + - notContains: + path: spec.template.spec.volumes + content: + name: tls-client-ca + any: true + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: tls-client-ca + mountPath: /etc/openshell-tls/client-ca + readOnly: true + + # When cert-manager owns TLS, does not share its CA, and no separate client CA + # secret is configured, there is no client CA to mount: the volume must not + # render rather than mounting an empty secret name. + - it: omits the client CA volume when cert-manager owns TLS and no client CA secret is set + template: templates/statefulset.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: false + pkiInitJob.enabled: true + server.tls.clientCaSecretName: "" + asserts: + - lengthEqual: + path: spec.template.spec.volumes + count: 4 + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: tls-client-ca + mountPath: /etc/openshell-tls/client-ca + readOnly: true + + # Explicit HTTPS-only opt-out must suppress built-in PKI client-CA wiring even + # when pkiInitJob remains enabled (the default install path from #2095). + - it: omits the client CA volume for built-in PKI when clientCaSecretName is empty + template: templates/statefulset.yaml + set: + pkiInitJob.enabled: true + certManager.enabled: false + server.tls.clientCaSecretName: "" + asserts: + - lengthEqual: + path: spec.template.spec.volumes + count: 4 + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: tls-client-ca + mountPath: /etc/openshell-tls/client-ca + readOnly: true + + - it: omits the client CA volume when cert-manager shares server CA and clientCaSecretName is empty + template: templates/statefulset.yaml + set: + certManager.enabled: true + certManager.clientCaFromServerTlsSecret: true + server.tls.clientCaSecretName: "" + asserts: + - lengthEqual: + path: spec.template.spec.volumes + count: 4 + - notContains: + path: spec.template.spec.containers[0].volumeMounts + content: + name: tls-client-ca + mountPath: /etc/openshell-tls/client-ca + readOnly: true diff --git a/charts/openshell/tests/workspace_secret_source_role_test.yaml b/charts/openshell/tests/workspace_secret_source_role_test.yaml new file mode 100644 index 000000000..6eea72b99 --- /dev/null +++ b/charts/openshell/tests/workspace_secret_source_role_test.yaml @@ -0,0 +1,77 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: workspace Secret source RBAC +templates: + - templates/workspace-secret-source-role.yaml + - templates/workspace-secret-source-rolebinding.yaml +release: + name: openshell + namespace: my-namespace + +tests: + - it: renders nothing in shared mode + set: + server.drivers.kubernetes.workspaceMode: shared + server.sandboxImagePullSecrets: + - name: registry-one + asserts: + - hasDocuments: + count: 0 + + - it: grants reads of staged Secret sources in the sandbox namespace only + template: templates/workspace-secret-source-role.yaml + set: + server.drivers.kubernetes.workspaceMode: managed + server.sandboxNamespace: sandboxes + server.tls.clientTlsSecretName: custom-client-tls + server.sandboxImagePullSecrets: + - name: registry-one + asserts: + - isKind: + of: Role + - equal: + path: metadata.namespace + value: sandboxes + - equal: + path: rules + value: + - apiGroups: [""] + resources: ["secrets"] + resourceNames: ["custom-client-tls", "registry-one"] + verbs: ["get"] + + - it: grants the operator-mode TLS Secret read without image-pull Secrets + template: templates/workspace-secret-source-role.yaml + set: + server.drivers.kubernetes.workspaceMode: operator + server.sandboxImagePullSecrets: + - name: registry-one + asserts: + - equal: + path: rules[0].resourceNames + value: ["openshell-client-tls"] + + - it: renders nothing in operator mode when TLS is disabled + set: + server.drivers.kubernetes.workspaceMode: operator + server.disableTls: true + asserts: + - hasDocuments: + count: 0 + + - it: binds the gateway ServiceAccount + template: templates/workspace-secret-source-rolebinding.yaml + set: + server.drivers.kubernetes.workspaceMode: managed + server.sandboxNamespace: sandboxes + asserts: + - equal: + path: metadata.namespace + value: sandboxes + - equal: + path: subjects[0].name + value: openshell + - equal: + path: subjects[0].namespace + value: my-namespace diff --git a/charts/openshell/values.yaml b/charts/openshell/values.yaml new file mode 100644 index 000000000..4e7e11142 --- /dev/null +++ b/charts/openshell/values.yaml @@ -0,0 +1,681 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Default values for OpenShell + +# -- Number of OpenShell gateway replicas. Values greater than 1 require +# server.externalDbSecret because the default SQLite backend is per pod. +replicaCount: 1 + +global: + image: + # -- Shared OpenShell image registry. Individual image registries take precedence. + registry: ghcr.io/nvidia + # -- Shared OpenShell image tag. Defaults to the chart appVersion when empty. + tag: "" + # -- Shared OpenShell image pull policy. Individual image pull policies take precedence. + pullPolicy: IfNotPresent + +workload: + # -- Gateway workload controller kind. Use `statefulset` for the default + # SQLite database, or `deployment` when server.externalDbSecret points at an + # external database. + kind: statefulset + # -- Allow replicaCount > 1 while rendering a StatefulSet. Prefer + # workload.kind=deployment for external database-backed multi-replica + # gateways; this override exists for operators who explicitly require + # StatefulSet identity or storage semantics. + allowMultiReplicaStatefulSet: false + +gateway: + image: + # -- Gateway image registry. Empty uses global.image.registry. + registry: "" + # -- Gateway image repository. + repository: openshell/gateway + # -- Gateway image pull policy. Empty uses global.image.pullPolicy. + pullPolicy: null + # -- Gateway image tag. Defaults to the chart appVersion when empty. + tag: "" + # -- Gateway image digest. When set, this takes precedence over tag. + digest: "" + +# Trusted workload-side runtime image. +sandboxRuntime: + image: + # -- Sandbox runtime image registry. Empty uses global.image.registry. + registry: "" + # -- Sandbox runtime image repository. + repository: openshell/sandbox + # -- Sandbox runtime image pull policy. Empty uses global.image.pullPolicy. + pullPolicy: null + # -- Sandbox runtime image tag. Defaults to the chart appVersion when empty. + tag: "" + # -- Sandbox runtime image digest. When set, this takes precedence over tag. + digest: "" + +# Trusted control-side runtime image. +supervisor: + image: + # -- Supervisor image registry. Empty uses global.image.registry. + registry: "" + # -- Supervisor image repository. + repository: openshell/supervisor + # -- Supervisor image pull policy. Empty uses global.image.pullPolicy. + # Prefer always, if_not_present, or never; the chart also accepts legacy + # Kubernetes spellings Always, IfNotPresent, and Never. + pullPolicy: null + # -- Supervisor image tag. Defaults to the chart appVersion when empty. + tag: "" + # -- Supervisor image digest. When set, this takes precedence over tag. + digest: "" + sandboxRuntime: + # -- Workload boundary TLS listener port. + boundaryPort: 5500 + +sandbox: + image: + # -- Default standalone sandbox image repository. + repository: nvcr.io/nvidia/base/ubuntu + # -- Sandbox image tag. Defaults to latest when empty. + tag: "24.04" + # -- Sandbox image digest. When set, this takes precedence over tag. + digest: "" + # -- Sandbox image pull policy. Leave unset to use the Kubernetes image default. + pullPolicy: null + +# -- Operator-owned corporate forward proxy for policy-approved TLS egress +# from Kubernetes sandboxes. The workload cannot select or override it. +upstreamProxy: + # -- Proxy URL in http://host:port or https://host:port form. An https:// proxy + # whose certificate is not publicly trusted also needs caBundle below. + url: "" + # -- Comma-separated destinations that bypass only the corporate proxy. + noProxy: "" + authSecret: + # -- Existing Secret in the sandbox namespace containing a user:pass value. + name: "" + # -- Secret key containing the proxy credential. + key: "" + # -- Required when authSecret is configured because Basic auth to an HTTP proxy is cleartext. + authAllowInsecure: false + # -- Last-resort option for hostname-filtering proxy ACLs. It lets the proxy resolve CONNECT targets. + connectByHostname: false + caBundle: + # -- ConfigMap in the release namespace holding the corporate proxy CA bundle. + # Required for an https:// proxy with a private CA, and for a TLS-intercepting + # proxy that re-signs upstream certificates. The gateway reads it and stages it + # into each sandbox's immutable supervisor bootstrap Secret, so the anchor stays + # in the gateway's trust domain rather than the workload namespace. + # Supply only the CA that signs your proxy's certificate, or that the proxy + # re-signs intercepted upstream certificates with. Public roots already come + # from the supervisor image and its TLS stack, so a full merged trust bundle + # (for example an OpenShift config.openshift.io/inject-trusted-cabundle + # ConfigMap) adds hundreds of kilobytes of duplicated roots and can exceed + # the sandbox boundary's control-frame budget. + configMapName: "" + # -- Key inside that ConfigMap. Change this only to reuse an existing + # ConfigMap whose key is not ca.crt. + key: ca.crt + +# -- Image pull secrets attached to gateway and helper pods. +imagePullSecrets: [] +# -- Override the chart name used in generated resource names. +nameOverride: "openshell" +# -- Override the full generated resource name. +fullnameOverride: "" + +serviceAccount: + # -- Create a service account for the gateway. + create: true + # -- Annotations to add to the generated service account. + annotations: {} + # -- Existing service account name to use when serviceAccount.create is false. + name: "" + +sandboxServiceAccount: + # -- Create a service account for sandbox pods. + create: true + # -- Annotations to add to the generated sandbox service account. + annotations: {} + # -- Existing service account name for sandbox pods when sandboxServiceAccount.create is false. + name: "" + +# RBAC objects created for the gateway ServiceAccount. Cluster-scoped objects +# can be omitted so a cluster-admin applies them once and a namespace-admin +# installs and upgrades the gateway release without cluster-scoped permissions. +rbac: + # -- Create the RBAC objects that grant the gateway ServiceAccount access. + # Disable to supply the namespaced sandbox and peer Role/RoleBinding and the + # cluster-scoped ClusterRole/ClusterRoleBinding out of band. The certgen hook + # and credential driver RBAC keep their own flags. + create: true + clusterScoped: + # -- Create the cluster-scoped ClusterRole and ClusterRoleBinding. Disable + # for a namespace-admin install where a cluster-admin applies them + # separately; the gateway ServiceAccount name and namespace are unchanged, + # so a pre-created ClusterRoleBinding still matches. + create: true + # -- Name for the ClusterRole. Empty uses the `-node-reader-` default. + clusterRoleName: "" + # -- Name for the ClusterRoleBinding. Empty uses the `-node-reader-` default. + clusterRoleBindingName: "" + +# Namespace-scoped resources needed to run sandboxes. Disable this when the +# gateway and workspace prerequisites are managed as separate Helm releases +# using the openshell-workspace chart. +workspaceResources: + # -- Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy + # from this chart. Disable for a gateway-only release. + enabled: true + +# -- Extra annotations to add to the gateway pod. +podAnnotations: {} +# -- Extra labels to add to the gateway pod. +podLabels: {} + +podSecurityContext: + # -- fsGroup assigned to the gateway pod. + fsGroup: 1000 + +securityContext: + # -- Require the gateway container to run as a non-root user. + runAsNonRoot: true + # -- UID assigned to the gateway container. + runAsUser: 1000 + # -- Whether the gateway container can gain additional privileges. + allowPrivilegeEscalation: false + capabilities: + # -- Linux capabilities dropped from the gateway container. + drop: + - ALL + +service: + # -- Kubernetes Service type for the gateway. + type: ClusterIP + # -- Gateway gRPC/HTTP service port. + port: 8080 + # -- Gateway health service port. + healthPort: 8081 + # -- Gateway metrics service port. + metricsPort: 9090 + +# Agent Sandbox is a cluster-scoped prerequisite for the Kubernetes compute +# driver. OpenShell deliberately does not install its CRDs or controller. +# Enable this check for live Helm installs to fail before creating gateway +# resources when neither supported Sandbox API is served. Disable it for +# offline `helm template` and lint workflows, which cannot discover APIs. +agentSandbox: + preflight: + # -- Check the live cluster for a supported Agent Sandbox API before rendering gateway resources. Disable only for offline rendering and linting. + enabled: true + +# Pod restart behavior and health probe tuning. +podLifecycle: + # -- Grace period, in seconds, before Kubernetes terminates the gateway pod. + terminationGracePeriodSeconds: 5 + +probes: + startup: + # -- Startup probe period, in seconds. + periodSeconds: 2 + # -- Startup probe timeout, in seconds. + timeoutSeconds: 1 + # -- Startup probe failure threshold before the container is killed. + failureThreshold: 30 + liveness: + # -- Liveness probe initial delay, in seconds. + initialDelaySeconds: 2 + # -- Liveness probe period, in seconds. + periodSeconds: 5 + # -- Liveness probe timeout, in seconds. + timeoutSeconds: 1 + # -- Liveness probe failure threshold before the container is restarted. + failureThreshold: 3 + readiness: + # -- Readiness probe initial delay, in seconds. + initialDelaySeconds: 1 + # -- Readiness probe period, in seconds. + periodSeconds: 2 + # -- Readiness probe timeout, in seconds. + timeoutSeconds: 1 + # -- Readiness probe failure threshold before the pod is marked not ready. + failureThreshold: 3 + +# -- Gateway pod resource requests and limits. +resources: {} + +# -- Node selector for the gateway pod. +nodeSelector: {} + +# -- Tolerations for the gateway pod. +tolerations: [] + +# -- Affinity rules for the gateway pod. +affinity: {} + +# Server configuration +server: + # -- Operator-facing gateway name. Defaults to the chart fullname so all + # replicas in one installation share an identity. Set explicitly when one + # telemetry collector receives spans from multiple namespaces or clusters. + name: "" + # -- Gateway log level. + logLevel: info + # OpenTelemetry trace export over OTLP/gRPC. Leave endpoint empty to disable. + otlp: + # -- OTLP/gRPC collector endpoint, conventionally using port 4317. + endpoint: "" + # -- Gateway OpenTelemetry service name. Empty uses openshell-gateway. + serviceName: "" + # -- Enable anonymous OpenShell telemetry from the gateway and the sandbox + # supervisors it launches. + telemetryEnabled: true + # -- Namespace where sandbox pods are created. Defaults to the Helm release + # namespace (.Release.Namespace) when left empty. + sandboxNamespace: "" + # -- Gateway database URL (used for the default SQLite backend). SQLite runs + # in WAL mode and needs a local block-backed volume, not NFS or other network + # filesystems. + dbUrl: "sqlite:/var/openshell/openshell.db" + # -- Name of a pre-existing Opaque Secret containing a PostgreSQL + # connection URI (key: uri). When set, the gateway reads OPENSHELL_DB_URL + # from this Secret instead of using dbUrl. The Secret must contain a + # `uri` key, e.g. postgresql://user:pass@host:5432/dbname. + externalDbSecret: "" + # -- Image pull secrets attached to sandbox pods. Referenced Secrets must exist + # in the sandbox namespace. + sandboxImagePullSecrets: [] + # -- Default storage size for the workspace PVC in sandbox pods. + # Uses Kubernetes quantity syntax (e.g. "2Gi", "10Gi", "500Mi"). + # Empty = built-in default (2Gi). + workspaceDefaultStorageSize: "" + # -- Kubernetes StorageClass for the workspace PVC in sandbox pods. + # Empty (default) = omit storageClassName, using the cluster's default + # StorageClass. Set this on clusters with no default StorageClass, otherwise + # the workspace PVC stays Pending and the sandbox never starts. + workspaceStorageClass: "" + # -- Default Kubernetes runtimeClassName for sandbox pods. + # Applied when a CreateSandbox request does not specify one. + # Empty (default) = omit the field, using the cluster's default RuntimeClass. + # Set to a RuntimeClass name (e.g. "kata-containers", "nvidia") to apply it + # to all sandboxes that don't explicitly override it. + defaultRuntimeClassName: "" + # -- gRPC endpoint sandboxes call back into the gateway. Leave empty to derive + # it from the chart fullname, release namespace, service port, and + # disableTls flag, for example https://openshell.openshell.svc.cluster.local:8080. + # Override only when sandboxes must reach the gateway via a different + # hostname (e.g. an external ingress or a host alias). + grpcEndpoint: "" + # The gateway terminates TLS directly. Its client CA authenticates sandbox + # callbacks; OIDC-enabled listeners permit bearer-only user clients while + # still validating any client certificate they present. + # -- Host gateway IP for sandbox pod hostAliases. When set, sandbox pods get + # hostAliases entries mapping host.docker.internal and host.openshell.internal + # to this IP, allowing them to reach services running on the Docker host. + # Auto-detected by the cluster entrypoint script. + hostGatewayIP: "" + # -- Enable Kubernetes user namespace isolation (hostUsers: false) for sandbox + # pods. Requires Kubernetes 1.33+ with user namespace support available + # (beta through 1.35, GA in 1.36+), plus a supporting container runtime and + # Linux 5.12+. When enabled, container UID 0 maps to an unprivileged host + # UID and capabilities become namespaced. + enableUserNamespaces: false + # Kubernetes compute driver settings. + drivers: + kubernetes: + # -- Allow caller driver JSON; external resources still require approval. + allowDriverConfig: false + resourceAdmission: + # -- Require operator approval labels on external sandbox attachments (GPU attachments exempt). + enabled: true + # -- Replacement label map; null uses the built-in admission labels. Empty map is invalid when enabled. + requiredLabels: null + # -- How workspaces map to Kubernetes namespaces. + # "shared" (default): all sandboxes in a single namespace. + # "managed": auto-creates per-workspace namespaces. + # "operator": uses pre-provisioned namespaces. + workspaceMode: "shared" + # -- K8s label selector for namespace discovery in operator mode. + # The driver watches namespaces matching this label. + operatorNamespaceLabel: "" + # -- Path to a JSON file containing an array of namespace names + # allowed in operator mode. Hot-reloaded on change. + operatorNamespaceFile: "" + # -- Disable TLS entirely - the server listens on plaintext HTTP. + # Set to true when a reverse proxy / tunnel terminates TLS at the edge. + disableTls: false + # -- Enable plaintext HTTP routing for loopback sandbox service URLs on + # TLS-enabled gateways. + enableLoopbackServiceHttp: true + # -- Enable the WebSocket tunnel used by CLI/SDK clients behind an + # authenticated edge proxy. Leave disabled for direct gateway installs. + enableWebsocketTunnel: false + # -- Posture when a candidate sandbox policy fails validation. `fail_closed` + # deactivates the previous policy; `retain_last_valid` keeps it active. + policyValidationFailureMode: fail_closed + # Optional gateway-wide gRPC request rate limit. Applies only to gRPC API + # traffic after protocol multiplexing; health, metrics, and loopback service + # HTTP routes are not rate limited. Both values must be positive to enable the + # limit, otherwise it is omitted from the rendered config and stays disabled. + grpcRateLimit: + # -- Maximum gRPC requests allowed per window. Must be positive (alongside + # windowSeconds) to enable rate limiting; 0 (default) disables it. + requests: 0 + # -- gRPC rate-limit window length in seconds. Must be positive (alongside + # requests) to enable rate limiting; 0 (default) disables it. + windowSeconds: 0 + # Default credential storage settings (used when no credential driver is + # enabled). The gateway encrypts provider credentials in the database using + # AES-256-GCM with a key-encryption key (KEK). By default, the Helm chart + # generates and retains a KEK Secret. For GitOps / helm-template workflows + # where `lookup` is unavailable, reference a pre-created Secret instead. + credentialStorage: + # -- Name of a pre-existing Secret containing the key-encryption key. + # When set, the chart does NOT generate a new Secret; it references this + # one instead. The Secret must contain a key named "key-encryption-key" + # with a base64-encoded 32-byte value. Required for GitOps workflows that + # render manifests with `helm template` (where `lookup` is unavailable). + existingSecret: "" + # Provider credential drivers store provider credential secret material in an + # external or native backend. When no driver is enabled, the gateway uses its + # default encrypted database credential storage with a retained Kubernetes + # Secret for the shared key-encryption key. + credentialDrivers: + kubernetesSecrets: + # -- Enable the in-tree Kubernetes Secret credential driver. + # WARNING: The RBAC Role grants read/write access to ALL Secrets in the + # configured namespace. Use a dedicated namespace to limit blast radius. + enabled: false + # -- Namespace where OpenShell-managed provider Secret objects are stored. + # Empty = Helm release namespace. A dedicated namespace is RECOMMENDED + # to isolate OpenShell-managed Secrets from other workloads. + namespace: "" + # -- Create the credential namespace. Requires a namespace other than the + # release namespace. The Namespace is retained on uninstall so stored + # credentials survive; an existing Namespace not owned by this release is + # left untouched. + createNamespace: false + rbac: + # -- Create a Role/RoleBinding granting the gateway ServiceAccount read/write access to managed provider Secrets. + create: true + vault: + # -- Enable the in-tree Vault credential driver. + enabled: false + # -- Vault service base URL. Non-loopback endpoints must use HTTPS, for example https://vault.vault.svc.cluster.local:8200. + address: "" + # -- ConfigMap containing the private Vault CA certificate bundle in the ca.crt key. Leave empty to use platform trust roots. + caConfigMapName: "" + # -- Default KV mount name. + mount: secret + # -- Default KV engine version. Use "1" or "2". + kvVersion: "2" + # -- Authentication method. Use "kubernetes" in-cluster or "token_file" for local/dev validation. + authMethod: kubernetes + # -- Vault Kubernetes auth role when authMethod is kubernetes. + role: "" + # -- Vault Kubernetes auth mount. + kubernetesAuthMount: kubernetes + # -- ServiceAccount token path used for Kubernetes auth. + serviceAccountTokenPath: /var/run/secrets/kubernetes.io/serviceaccount/token + # -- Mounted token file path when authMethod is token_file. + tokenPath: "" + # -- HTTP request timeout in seconds. Empty = driver default. + timeoutSecs: "" + auth: + # -- UNSAFE: accept unauthenticated CLI/user requests as a local developer + # principal. Intended only for trusted local Skaffold/k3d development or a + # fully trusted fronting proxy. Leave false for shared or production clusters. + allowUnauthenticatedUsers: false + tls: + # -- K8s secret (type kubernetes.io/tls) with tls.crt and tls.key for the server. + certSecretName: openshell-server-tls + # -- Enable mTLS client certificate authentication. When false, the gateway + # runs HTTPS-only without requiring client certificates (use OIDC for auth + # instead). Must be false when using BackendTLSPolicy because ingress + # proxies cannot present client certificates to the backend. + enableMtls: true + # -- K8s secret with ca.crt for client certificate verification (mTLS). + # Only used when enableMtls is true. Set to "" to disable client certificate + # verification for HTTPS-only mode. + clientCaSecretName: openshell-server-client-ca + # -- K8s secret mounted into sandbox pods for mTLS to the server. + clientTlsSecretName: openshell-client-tls + # Gateway-minted sandbox JWT signing keys. The certgen hook generates an + # Ed25519 keypair and writes it to a secret containing signing.pem (PKCS#8), + # public.pem (SPKI), and kid (plain text). The hook runs in full PKI mode when + # pkiInitJob.enabled=true unless certManager.enabled=true, which takes + # precedence and runs the hook in JWT-only mode. + sandboxJwt: + # -- Name of the Opaque Secret holding the signing key material. Empty + # falls back to the chart fullname with "-jwt-keys" appended. + signingSecretName: "" + # -- Stable gateway identity embedded in iss/aud of every minted token. + # Defaults to the release name so HA replicas share identity. + gatewayId: "" + # -- Token TTL in seconds. Defaults to 3600 (1h). + ttlSecs: 3600 + # -- Lifetime (seconds) of the projected ServiceAccount token kubelet + # writes into each sandbox pod for the IssueSandboxToken bootstrap + # exchange. Kubelet enforces a minimum of 600s; the driver clamps + # values outside [600, 86400]. Default 3600 — generous, since the + # supervisor consumes the token within seconds of pod start. + k8sSaTokenTtlSecs: 3600 + # -- File mode for the mounted JWT signing key Secret. Default 0400 + # (owner-read only). Override to 0440 or 0444 if the container UID + # does not match the volume file owner. + secretDefaultMode: "" + # Dynamic provider token grants. When SPIFFE is enabled here, both the + # gateway and sandbox supervisors mount the SPIFFE Workload API socket so + # token-exchange profiles can use gateway- and sandbox-scoped JWT-SVIDs. + # Supervisor-to-gateway authentication still uses gateway-minted sandbox JWTs. + providerTokenGrants: + spiffe: + # -- Mount the SPIFFE Workload API socket into gateway and sandbox pods for dynamic provider token grants. + enabled: false + # -- Path to the SPIFFE Workload API socket mounted into gateway and sandbox pods. + workloadApiSocketPath: /spiffe-workload-api/spire-agent.sock + # OIDC (OpenID Connect) configuration for JWT-based authentication. + # When issuer is set, the server validates Bearer tokens on gRPC requests. + oidc: + # -- OIDC issuer URL (e.g. https://keycloak.example.com/realms/openshell). + issuer: "" + # -- Development only: permit cleartext OIDC requests to numeric loopback + # addresses. This never permits HTTP to hostnames or non-loopback addresses. + dangerouslyAllowInsecureHttp: false + # -- Additional trusted HTTPS origins allowed to serve JWKS. The issuer + # origin is always allowed. Entries must not include a path or query. + jwksAllowedOrigins: [] + # -- Expected audience claim for the API resource server. + # This should match the server's --oidc-audience, NOT the CLI client ID. + audience: "openshell-cli" + # -- JWKS key cache TTL in seconds. Must be greater than zero. + jwksTtl: 3600 + # -- Dot-separated path to the roles array in the JWT claims. + # Keycloak: "realm_access.roles", Entra ID: "roles", Okta: "groups". + rolesClaim: "" + # -- Role name for admin access. Leave empty (with userRole also empty) for + # authentication-only mode. Both must be set or both empty. + adminRole: "" + # -- Role name for standard user access. + userRole: "" + # -- Dot-separated path to the scopes array in the JWT claims. + scopesClaim: "" + # -- Name of a ConfigMap containing a CA certificate bundle (key: ca.crt) + # for verifying the OIDC issuer's TLS certificate. Required when the + # issuer uses a non-public CA (e.g. OpenShift ingress, private PKI). + caConfigMapName: "" + +# NetworkPolicy restricting SSH ingress on sandbox pods to the gateway only. +networkPolicy: + # -- Restrict SSH ingress on sandbox pods to the gateway. In managed mode, + # the driver applies the equivalent policy to each workspace namespace. + enabled: true + +# Built-in TLS PKI bootstrap via a pre-install/pre-upgrade hook Job. +# Runs `openshell-gateway generate-certs` to create the server and client TLS +# Secrets in-cluster. Key material is written directly to K8s Secrets and +# never appears in Helm release history. Idempotent: existing secrets are +# left untouched on upgrade. Reuses the gateway image - no extra image to +# mirror in air-gapped environments. +# +# The server certificate already includes the built-in cluster SANs +# (`openshell`, `openshell.openshell.svc`, the cluster.local FQDN, `localhost`, +# `openshell.localhost`, `*.openshell.localhost`, `host.docker.internal`, and +# `127.0.0.1`) baked into the gateway binary. The lists below are additional +# SANs appended on top. Wildcard DNS SANs also enable sandbox service URLs under +# that domain, for example `*.apps.example.com` enables +# `--.apps.example.com`. +pkiInitJob: + # -- Run a pre-install/pre-upgrade Job that creates gateway and client mTLS + # Secrets. When certManager.enabled=true, cert-manager owns TLS and this same + # hook runs in JWT-only mode even if pkiInitJob.enabled remains true. + enabled: true + # -- Extra DNS SANs to append to the server certificate. + serverDnsNames: [] + # -- Extra IP SANs to append to the server certificate. + serverIpAddresses: [] + # -- Maximum time in seconds for the certgen hook to poll for cert-manager + # certificates. When using cert-manager with BackendTLSPolicy, the hook + # polls for this many seconds waiting for the certificate to be issued, + # then creates the backend CA ConfigMap. The Job deadline is set to + # (timeoutSeconds + 30) to allow time for ConfigMap creation and cleanup. + # Increase this if cert-manager takes longer than 120 seconds to issue + # certificates. + timeoutSeconds: 120 + # -- Fail the helm install/upgrade if cert-manager does not issue the + # certificate within the polling timeout. When true (default), the install + # fails immediately if the timeout is reached, providing clear feedback that + # BackendTLSPolicy is non-functional. When false, the hook succeeds with a + # warning and you can run `helm upgrade` after cert-manager issues the + # certificate to create the backend CA ConfigMap. If you set this to false and + # see "TLS error: Secret is not supplied by SDS" when connecting to the gateway, + # check if the TLS secret exists and run `helm upgrade` to create the ConfigMap. + failOnTimeout: true + +# cert-manager Certificate/Issuer resources (requires cert-manager CRDs in-cluster). +# Does not install cert-manager itself. +certManager: + # -- Create cert-manager Issuer and Certificate resources. When enabled, + # cert-manager owns TLS and the chart runs a JWT-only certgen hook to create + # the sandbox JWT signing Secret that cert-manager does not manage. + enabled: false + # -- Secret created for the intermediate CA (Certificate with isCA: true). + caSecretName: openshell-ca-tls + # -- Override the issuerRef for the external server Certificate (e.g. a real + # LetsEncrypt/ACME ClusterIssuer for a publicly-trusted cert on an external hostname). + # When set, the chart creates a second server certificate from this issuer + # with only the hostnames in serverDnsNames; the internal server certificate + # is always signed by the chart's own CA. Leave name empty to use the chart + # CA for all server certificates (default). Requires certManager.enabled=true. + serverIssuerRef: + name: "" + kind: "" + group: "" + # -- Mount gateway client CA from the internal server TLS secret's ca.crt. + # The internal server certificate is always signed by the chart CA — the same + # CA that signs the client (mTLS) certificate — so the default (true) is + # correct for all configurations, including when serverIssuerRef is set. + # Only set to false if you mount the client CA from a separate secret via + # server.tls.clientCaSecretName. + clientCaFromServerTlsSecret: true + # -- Duration for cert-manager-issued certificates. + certificateDuration: 8760h + # -- Renewal window for cert-manager-issued certificates. + certificateRenewBefore: 720h + # -- DNS SANs on the cert-manager-issued server certificate. + serverDnsNames: + - openshell + - openshell.openshell.svc + - openshell.openshell.svc.cluster.local + - localhost + - openshell.localhost + - "*.openshell.localhost" + - host.docker.internal + # -- IP SANs on the cert-manager-issued server certificate. + serverIpAddresses: + - 127.0.0.1 + +# Kubernetes Gateway API - HTTPRoute and Gateway resources. +# Requires a Gateway API controller in the cluster. Install Envoy Gateway via +# the skaffold.yaml releases or independently: +# helm install eg oci://docker.io/envoyproxy/gateway-helm \ +# --version v1.4.1 -n envoy-gateway-system --create-namespace +grpcRoute: + # -- Create a Gateway API GRPCRoute for the gateway service. + enabled: false + # -- Hostnames the GRPCRoute matches on. Leave empty to match all hosts. + hostnames: [] + gateway: + # -- When true, a Gateway resource is created in the release namespace. + # Set to false and provide name/namespace to attach to a pre-existing Gateway. + create: false + # -- GatewayClass to reference. Envoy Gateway installs one named "eg". + className: "eg" + # -- Name of the Gateway resource. Defaults to the chart fullname. + name: "" + # -- Namespace of the Gateway referenced by the GRPCRoute parentRef. + # Defaults to the release namespace. + namespace: "" + # Listener settings (only used when gateway.create is true). + listener: + # -- Listener port for the generated Gateway resource. Use 443 with protocol HTTPS. + port: 80 + # -- Listener protocol for the generated Gateway resource: HTTP or HTTPS. + # HTTPS terminates TLS at the Envoy Gateway listener; pair it with + # server.disableTls=true so Envoy forwards plaintext to the gateway pod, + # and use OIDC for client identity (the gateway never sees the client cert). + protocol: HTTP + # -- "Same" restricts attached routes to the release namespace; "All" allows any namespace. + allowedRoutes: Same + # TLS settings for the listener. Used only when protocol is HTTPS + # (mode is always Terminate). + tls: + # -- certificateRefs for the HTTPS listener. Required when protocol is + # HTTPS. Each entry needs a `name` pointing at a kubernetes.io/tls Secret + # in the Gateway's namespace. May reference a cert-manager-issued Secret + # or the existing openshell-server-tls Secret (its SANs must include the + # external hostname). + certificateRefs: [] + # BackendTLSPolicy for end-to-end TLS between the Gateway proxy and the + # OpenShell gateway pod. When enabled, the Gateway proxy terminates + # client-facing TLS at the listener and re-encrypts when connecting to the + # backend service, validating the backend's certificate against the + # specified CA. Requires the gateway pod to serve TLS (server.disableTls + # must be false). Supported on OpenShift 4.22+ and other platforms with + # BackendTLSPolicy support in the Gateway API implementation. + backendTLSPolicy: + # -- Create a BackendTLSPolicy resource for end-to-end TLS between the + # Gateway proxy and the OpenShell gateway pod. The traffic flow is: + # client → HTTPS → Gateway (terminate) → TLS (re-encrypt) → gateway pod. + # Requires server.disableTls=false and server.tls.enableMtls=false. The + # certgen hook auto-creates the backend CA ConfigMap. + enabled: false + # -- Name of the ConfigMap containing the CA certificate (key: ca.crt) used + # to validate the gateway pod's TLS certificate. Defaults to + # `-backend-ca` when empty. The certgen hook auto-creates this: + # with pkiInitJob (default), immediately on install/upgrade; with + # cert-manager, the hook polls for pkiInitJob.timeoutSeconds seconds + # waiting for cert-manager to issue the server certificate, then creates the + # ConfigMap. A single install usually succeeds; if cert-manager takes longer, + # increase pkiInitJob.timeoutSeconds. By default (pkiInitJob.failOnTimeout=true), + # the install fails if the timeout is reached; set failOnTimeout=false to allow + # the install to succeed and run `helm upgrade` after the certificate is issued. + caCertificateConfigMapName: "" + # -- Hostname the Gateway proxy validates against the backend's TLS + # certificate SAN. Defaults to the service FQDN + # (`..svc.cluster.local`) when empty, which matches + # the SAN included by both cert-manager and the pkiInitJob. + hostname: "" + +# OpenShift Route with TLS passthrough. The gateway terminates its own +# TLS/mTLS; the router only forwards based on SNI, so it never sees plaintext +# or the client certificate. Requires server.disableTls=false and a server +# cert whose SANs include the Route host (see certManager.serverIssuerRef). +openshiftRoute: + # -- Create an OpenShift Route with TLS passthrough. + enabled: false + # -- Hostname for the Route. Must match a SAN on the gateway's server cert. + host: "" + # -- Extra annotations on the Route (e.g. haproxy.router.openshift.io/*). + annotations: {} diff --git a/components/control-plane/Dockerfile b/components/control-plane/Dockerfile index 4ed8d1b4e..41edcf0cb 100644 --- a/components/control-plane/Dockerfile +++ b/components/control-plane/Dockerfile @@ -41,14 +41,14 @@ COPY components/control-plane/internal/ components/control-plane/internal/ ARG TARGETARCH RUN cd components/control-plane && CGO_ENABLED=0 GOOS=linux GOARCH="${TARGETARCH}" go build -mod=mod -ldflags="-s -w" -o /workspace/hypershell-controller ./cmd/hypershell-controller -# Package Helm chart from fork (git is provided by the Go toolchain image) -COPY OPENSHELL_VERSION /tmp/chart-source/ -RUN . /tmp/chart-source/OPENSHELL_VERSION && \ - git clone --depth 1 --branch "${OPENSHELL_TAG}" "${OPENSHELL_CHART_REPO}" /tmp/openshell && \ - mkdir -p /workspace/charts && \ - helm package /tmp/openshell/deploy/helm/openshell --destination /workspace/charts/ && \ - mv /workspace/charts/*.tgz /workspace/charts/openshell.tgz && \ - rm -rf /tmp/openshell +# Package Helm chart from the vendored source tree in charts/openshell. +# The chart source is committed to the repository (no network access needed), +# which is required for Konflux hermetic builds. When bumping OPENSHELL_VERSION, +# update charts/openshell/ by extracting the chart from the new upstream tag. +COPY charts/openshell /tmp/chart-source/openshell +RUN mkdir -p /workspace/charts && \ + helm package /tmp/chart-source/openshell --destination /workspace/charts/ && \ + mv /workspace/charts/*.tgz /workspace/charts/openshell.tgz FROM registry.access.redhat.com/hi/static:1787099997@sha256:3d43712a61ce01c7049e983e7f4c4609aaad7582c7bbd930b57372f82c446f98 AS static-amd64 FROM registry.access.redhat.com/hi/static:1787099997@sha256:2ccd532da5e8868a92f479a7268a2fce65279c1f7c8a8dd0880d399c04d6a356 AS static-arm64 From 1e837d2150fc96a8083e808c2f01a09bea914d4e Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 21:12:41 -0400 Subject: [PATCH 08/15] fix(control-plane): fix gateway image keys and chart em-dashes for v0.1.2 The v0.1.2 chart reorganized image configuration from top-level image.* to gateway.image.{registry,repository,tag} and applies global.image.registry (ghcr.io/nvidia) as a prefix when per-image registry is unset. The old code set image.repository/tag which the new chart ignores, causing certgen to pull the default chart image (not available in Kind) and time out with DeadlineExceeded. Fix: split fully-qualified image references into registry+repo+tag using the Docker convention (first path component is a registry if it contains a dot or colon), then set gateway.image.registry explicitly to bypass the global prefix. Apply the same fix to supervisor.image. Additionally replace em dashes in vendored charts/openshell/ files with hyphens to satisfy the repository's check-forbidden-terms policy (pre-commit hook rejects U+2014 in committed text files). Co-Authored-By: Claude Sonnet 4.6 --- charts/openshell/README.md | 4 +- charts/openshell/ci/values-gateway.yaml | 2 +- .../openshell/ci/values-namespace-admin.yaml | 2 +- charts/openshell/ci/values-openshift-e2e.yaml | 2 +- charts/openshell/skaffold.yaml | 6 +-- charts/openshell/templates/_helpers.tpl | 2 +- .../openshell/templates/cert-manager-pki.yaml | 10 ++-- charts/openshell/values.yaml | 6 +-- .../control-plane/internal/helm/values.go | 50 +++++++++++++++++-- .../gateway_version_selection_test.go | 10 ++-- 10 files changed, 69 insertions(+), 25 deletions(-) diff --git a/charts/openshell/README.md b/charts/openshell/README.md index bfb60ddb8..097b0c341 100644 --- a/charts/openshell/README.md +++ b/charts/openshell/README.md @@ -314,7 +314,7 @@ discovery endpoint or its TLS CA. | certManager.caSecretName | string | `"openshell-ca-tls"` | Secret created for the intermediate CA (Certificate with isCA: true). | | certManager.certificateDuration | string | `"8760h"` | Duration for cert-manager-issued certificates. | | certManager.certificateRenewBefore | string | `"720h"` | Renewal window for cert-manager-issued certificates. | -| certManager.clientCaFromServerTlsSecret | bool | `true` | Mount gateway client CA from the internal server TLS secret's ca.crt. The internal server certificate is always signed by the chart CA — the same CA that signs the client (mTLS) certificate — so the default (true) is correct for all configurations, including when serverIssuerRef is set. Only set to false if you mount the client CA from a separate secret via server.tls.clientCaSecretName. | +| certManager.clientCaFromServerTlsSecret | bool | `true` | Mount gateway client CA from the internal server TLS secret's ca.crt. The internal server certificate is always signed by the chart CA - the same CA that signs the client (mTLS) certificate - so the default (true) is correct for all configurations, including when serverIssuerRef is set. Only set to false if you mount the client CA from a separate secret via server.tls.clientCaSecretName. | | certManager.enabled | bool | `false` | Create cert-manager Issuer and Certificate resources. When enabled, cert-manager owns TLS and the chart runs a JWT-only certgen hook to create the sandbox JWT signing Secret that cert-manager does not manage. | | certManager.serverDnsNames | list | `["openshell","openshell.openshell.svc","openshell.openshell.svc.cluster.local","localhost","openshell.localhost","*.openshell.localhost","host.docker.internal"]` | DNS SANs on the cert-manager-issued server certificate. | | certManager.serverIpAddresses | list | `["127.0.0.1"]` | IP SANs on the cert-manager-issued server certificate. | @@ -443,7 +443,7 @@ discovery endpoint or its TLS CA. | server.providerTokenGrants.spiffe.workloadApiSocketPath | string | `"/spiffe-workload-api/spire-agent.sock"` | Path to the SPIFFE Workload API socket mounted into gateway and sandbox pods. | | server.sandboxImagePullSecrets | list | `[]` | Image pull secrets attached to sandbox pods. Referenced Secrets must exist in the sandbox namespace. | | server.sandboxJwt.gatewayId | string | `""` | Stable gateway identity embedded in iss/aud of every minted token. Defaults to the release name so HA replicas share identity. | -| server.sandboxJwt.k8sSaTokenTtlSecs | int | `3600` | Lifetime (seconds) of the projected ServiceAccount token kubelet writes into each sandbox pod for the IssueSandboxToken bootstrap exchange. Kubelet enforces a minimum of 600s; the driver clamps values outside [600, 86400]. Default 3600 — generous, since the supervisor consumes the token within seconds of pod start. | +| server.sandboxJwt.k8sSaTokenTtlSecs | int | `3600` | Lifetime (seconds) of the projected ServiceAccount token kubelet writes into each sandbox pod for the IssueSandboxToken bootstrap exchange. Kubelet enforces a minimum of 600s; the driver clamps values outside [600, 86400]. Default 3600 - generous, since the supervisor consumes the token within seconds of pod start. | | server.sandboxJwt.secretDefaultMode | string | `""` | File mode for the mounted JWT signing key Secret. Default 0400 (owner-read only). Override to 0440 or 0444 if the container UID does not match the volume file owner. | | server.sandboxJwt.signingSecretName | string | `""` | Name of the Opaque Secret holding the signing key material. Empty falls back to the chart fullname with "-jwt-keys" appended. | | server.sandboxJwt.ttlSecs | int | `3600` | Token TTL in seconds. Defaults to 3600 (1h). | diff --git a/charts/openshell/ci/values-gateway.yaml b/charts/openshell/ci/values-gateway.yaml index 196192213..6e92825d9 100644 --- a/charts/openshell/ci/values-gateway.yaml +++ b/charts/openshell/ci/values-gateway.yaml @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -# Gateway API overlay — enables a Gateway and GRPCRoute for external access. +# Gateway API overlay - enables a Gateway and GRPCRoute for external access. # # Requires Envoy Gateway in the cluster (installed via skaffold.yaml). # Add this file to the openshell release valuesFiles to activate: diff --git a/charts/openshell/ci/values-namespace-admin.yaml b/charts/openshell/ci/values-namespace-admin.yaml index 7326be732..4d84cfa8f 100644 --- a/charts/openshell/ci/values-namespace-admin.yaml +++ b/charts/openshell/ci/values-namespace-admin.yaml @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -# Namespace-admin overlay — renders only namespaced objects. +# Namespace-admin overlay - renders only namespaced objects. # # Use this when a cluster-admin applies the gateway ClusterRole and # ClusterRoleBinding once, out of band, and the OpenShell release is installed diff --git a/charts/openshell/ci/values-openshift-e2e.yaml b/charts/openshell/ci/values-openshift-e2e.yaml index c470f3ce8..2aa02607d 100644 --- a/charts/openshell/ci/values-openshift-e2e.yaml +++ b/charts/openshell/ci/values-openshift-e2e.yaml @@ -22,7 +22,7 @@ # # Security: this is NOT an open gateway. `server.tls.clientCaSecretName` defaults # to `openshell-server-client-ca` and there is no OIDC, so `require_client_auth` -# is true and mTLS is MANDATORY at the TLS handshake — a caller with only the +# is true and mTLS is MANDATORY at the TLS handshake - a caller with only the # Route URL and no client certificate is rejected before any RPC. The passthrough # Route terminates TLS at the gateway pod, so this holds end-to-end. # `allowUnauthenticatedUsers` only promotes the already cert-verified caller to a diff --git a/charts/openshell/skaffold.yaml b/charts/openshell/skaffold.yaml index 2d7bf892e..0167f8e78 100644 --- a/charts/openshell/skaffold.yaml +++ b/charts/openshell/skaffold.yaml @@ -76,7 +76,7 @@ build: deploy: helm: releases: - # cert-manager — comment this in and add values-cert-manager.yaml below + # cert-manager - comment this in and add values-cert-manager.yaml below # when you want cert-manager to manage TLS while certgen handles JWT. # Requires cert-manager CRDs to be installed in the cluster first. #- name: cert-manager @@ -86,7 +86,7 @@ deploy: # createNamespace: true # setValues: # crds.enabled: true - # Envoy Gateway — Kubernetes Gateway API implementation. + # Envoy Gateway - Kubernetes Gateway API implementation. # Installs the Gateway API CRDs and the "eg" GatewayClass. # Required when grpcRoute.enabled is true in the openshell release. #- name: envoy-gateway @@ -97,7 +97,7 @@ deploy: # # wait ensures Gateway API CRDs are registered before the openshell # # release attempts to create Gateway and HTTPRoute resources. # wait: true - # SPIRE — installs SPIRE Server, Agent, Controller Manager, CSI Driver, + # SPIRE - installs SPIRE Server, Agent, Controller Manager, CSI Driver, # and OIDC Discovery Provider using the SPIFFE hardened charts. # Uncomment both releases and ci/values-spire.yaml below to use # SPIFFE JWT-SVIDs for dynamic provider token grants. diff --git a/charts/openshell/templates/_helpers.tpl b/charts/openshell/templates/_helpers.tpl index ab4245875..7b775443e 100644 --- a/charts/openshell/templates/_helpers.tpl +++ b/charts/openshell/templates/_helpers.tpl @@ -310,7 +310,7 @@ Name of the Secret holding gateway-minted sandbox JWT signing material. gRPC endpoint sandbox pods use to call back into the gateway. An explicit .Values.server.grpcEndpoint is used verbatim. Otherwise it is derived from the in-cluster Service DNS, release namespace, service port, and disableTls -flag — so the default value works for any release name or namespace without +flag - so the default value works for any release name or namespace without override. */}} {{- define "openshell.grpcEndpoint" -}} diff --git a/charts/openshell/templates/cert-manager-pki.yaml b/charts/openshell/templates/cert-manager-pki.yaml index 838534e62..031d4d611 100644 --- a/charts/openshell/templates/cert-manager-pki.yaml +++ b/charts/openshell/templates/cert-manager-pki.yaml @@ -47,16 +47,16 @@ spec: {{- end }} {{- if $externalServerIssuer }} {{- if not .Values.certManager.serverDnsNames }} -{{- fail "certManager.serverIssuerRef.name is set but certManager.serverDnsNames is empty — the external certificate requires at least one externally-resolvable DNS name." }} +{{- fail "certManager.serverIssuerRef.name is set but certManager.serverDnsNames is empty - the external certificate requires at least one externally-resolvable DNS name." }} {{- end }} {{- range .Values.certManager.serverDnsNames }} -{{- /* Single-label names (e.g. "openshell") are also rejected by ACME CAs but are intentionally not checked here — the guard targets recognisable internal-network patterns. */ -}} +{{- /* Single-label names (e.g. "openshell") are also rejected by ACME CAs but are intentionally not checked here - the guard targets recognisable internal-network patterns. */ -}} {{- if or (eq . "localhost") (hasSuffix ".localhost" .) (hasSuffix ".svc.cluster.local" .) (hasSuffix ".svc" .) (eq . "host.docker.internal") (eq . "host.containers.internal") }} -{{- fail (printf "certManager.serverIssuerRef.name is set (external issuer) but certManager.serverDnsNames contains %q — external CAs (e.g. ACME / Let's Encrypt) reject internal-only names per CA/Browser Forum baseline requirements. Override certManager.serverDnsNames with your externally-resolvable hostname(s)." .) }} +{{- fail (printf "certManager.serverIssuerRef.name is set (external issuer) but certManager.serverDnsNames contains %q - external CAs (e.g. ACME / Let's Encrypt) reject internal-only names per CA/Browser Forum baseline requirements. Override certManager.serverDnsNames with your externally-resolvable hostname(s)." .) }} {{- end }} {{- end }} {{- end }} -# Internal server certificate — always issued by the chart’s own CA with +# Internal server certificate - always issued by the chart’s own CA with # internal SANs. Supervisors connect via internal hostnames and verify # this cert against the chart CA they already trust. apiVersion: cert-manager.io/v1 @@ -97,7 +97,7 @@ spec: group: cert-manager.io {{- if $externalServerIssuer }} --- -# External server certificate — issued by the operator-configured issuer +# External server certificate - issued by the operator-configured issuer # (e.g. ACME/Let’s Encrypt) with only externally-resolvable SANs. # The gateway uses SNI to present this cert for external hostnames. apiVersion: cert-manager.io/v1 diff --git a/charts/openshell/values.yaml b/charts/openshell/values.yaml index 4e7e11142..11d303521 100644 --- a/charts/openshell/values.yaml +++ b/charts/openshell/values.yaml @@ -458,7 +458,7 @@ server: # -- Lifetime (seconds) of the projected ServiceAccount token kubelet # writes into each sandbox pod for the IssueSandboxToken bootstrap # exchange. Kubelet enforces a minimum of 600s; the driver clamps - # values outside [600, 86400]. Default 3600 — generous, since the + # values outside [600, 86400]. Default 3600 - generous, since the # supervisor consumes the token within seconds of pod start. k8sSaTokenTtlSecs: 3600 # -- File mode for the mounted JWT signing key Secret. Default 0400 @@ -573,8 +573,8 @@ certManager: kind: "" group: "" # -- Mount gateway client CA from the internal server TLS secret's ca.crt. - # The internal server certificate is always signed by the chart CA — the same - # CA that signs the client (mTLS) certificate — so the default (true) is + # The internal server certificate is always signed by the chart CA - the same + # CA that signs the client (mTLS) certificate - so the default (true) is # correct for all configurations, including when serverIssuerRef is set. # Only set to false if you mount the client CA from a separate secret via # server.tls.clientCaSecretName. diff --git a/components/control-plane/internal/helm/values.go b/components/control-plane/internal/helm/values.go index c74515ba6..8640ad7a6 100644 --- a/components/control-plane/internal/helm/values.go +++ b/components/control-plane/internal/helm/values.go @@ -113,15 +113,21 @@ func (b *ValuesBuilder) buildCoreValues(values map[string]interface{}) error { // Pin fullnameOverride so every chart resource uses the expected name. setNestedValue(values, ReleaseName, "fullnameOverride") - // Image values + // Image values. The v0.1.2 chart uses gateway.image.{registry,repository,tag} + // and prepends global.image.registry (ghcr.io/nvidia) when the per-image + // registry is unset. We always pass fully-qualified image references from the + // GatewayRelease, so we split registry out and set it explicitly per-image to + // prevent the global prefix from being applied. if b.Gateway.Image != "" { - repo, tag := splitImageRef(b.Gateway.Image) - setNestedValue(values, repo, "image", "repository") - setNestedValue(values, tag, "image", "tag") + reg, repo, tag := splitImageRefFull(b.Gateway.Image) + setNestedValue(values, reg, "gateway", "image", "registry") + setNestedValue(values, repo, "gateway", "image", "repository") + setNestedValue(values, tag, "gateway", "image", "tag") } if b.Gateway.SupervisorImage != "" { - repo, tag := splitImageRef(b.Gateway.SupervisorImage) + reg, repo, tag := splitImageRefFull(b.Gateway.SupervisorImage) + setNestedValue(values, reg, "supervisor", "image", "registry") setNestedValue(values, repo, "supervisor", "image", "repository") setNestedValue(values, tag, "supervisor", "image", "tag") } @@ -295,6 +301,40 @@ func splitImageRef(image string) (repo, tag string) { return image[:lastColon], image[lastColon+1:] } +// splitImageRefFull splits an image reference into registry, repository, and tag. +// The registry is the first path component if it contains a dot or colon, or is +// "localhost" -- the same heuristic used by the Docker distribution library. +// This is needed for the v0.1.2 chart, which applies global.image.registry as a +// prefix when the per-image registry is unset. By extracting the registry here +// and setting it explicitly, the chart's global prefix is suppressed. +func splitImageRefFull(image string) (registry, repo, tag string) { + _, tag = splitImageRef(image) + + // Strip digest and tag to work with path only + withoutTag := image + if at := strings.LastIndex(image, "@"); at != -1 { + withoutTag = image[:at] + } + lastSlash := strings.LastIndex(withoutTag, "/") + lastColon := strings.LastIndex(withoutTag, ":") + if lastColon > lastSlash { + withoutTag = withoutTag[:lastColon] + } + + // Split on the first slash to isolate the potential registry component + slashIdx := strings.Index(withoutTag, "/") + if slashIdx == -1 { + return "", withoutTag, tag + } + first := withoutTag[:slashIdx] + rest := withoutTag[slashIdx+1:] + + if strings.Contains(first, ".") || strings.Contains(first, ":") || first == "localhost" { + return first, rest, tag + } + return "", withoutTag, tag +} + // setNestedValue sets a value in a nested map structure. // It creates intermediate maps as needed. func setNestedValue(m map[string]interface{}, value interface{}, path ...string) { diff --git a/components/control-plane/internal/reconciler/gateway_version_selection_test.go b/components/control-plane/internal/reconciler/gateway_version_selection_test.go index d61b015d5..b2726cf06 100644 --- a/components/control-plane/internal/reconciler/gateway_version_selection_test.go +++ b/components/control-plane/internal/reconciler/gateway_version_selection_test.go @@ -155,11 +155,15 @@ func TestSelectGatewayImage_PlatformDefault(t *testing.T) { if err != nil { t.Fatalf("build Helm values: %v", err) } - image, ok := values["image"].(map[string]interface{}) + gatewayVals, ok := values["gateway"].(map[string]interface{}) if !ok { - t.Fatal("Helm values must include the gateway image override") + t.Fatal("Helm values must include gateway image override") } - if image["repository"] != "quay.io/opendatahub/odh-openshell-gateway" || image["tag"] != "v0.0.109-rhaiv.0" { + image, ok := gatewayVals["image"].(map[string]interface{}) + if !ok { + t.Fatal("Helm values must include gateway.image override") + } + if image["registry"] != "quay.io" || image["repository"] != "opendatahub/odh-openshell-gateway" || image["tag"] != "v0.0.109-rhaiv.0" { t.Fatalf("unexpected Helm image values: %v", image) } }) From 1a610c4a03fd9ea79de43476da59526c09ffb0c5 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 21:28:58 -0400 Subject: [PATCH 09/15] fix(deploy): grant controller runtimeclasses and priorityclasses permissions The v0.1.2 OpenShell Helm chart creates a ClusterRole that includes node.k8s.io/runtimeclasses:get and scheduling.k8s.io/priorityclasses:get. Kubernetes rejects ClusterRole creation when the creating subject does not itself hold every permission being granted (RBAC escalation prevention). The hypershell-controller ServiceAccount was missing these two permissions, causing helm install to fail with: clusterroles "openshell-gateway-node-reader-*" is forbidden: user "system:serviceaccount:hypershell-system:hypershell-controller" is attempting to grant RBAC permissions not currently held Co-Authored-By: Claude Sonnet 4.6 --- deploy/base/platform-resources/controller-rbac.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/deploy/base/platform-resources/controller-rbac.yaml b/deploy/base/platform-resources/controller-rbac.yaml index 415c60559..a90a49c98 100644 --- a/deploy/base/platform-resources/controller-rbac.yaml +++ b/deploy/base/platform-resources/controller-rbac.yaml @@ -44,6 +44,15 @@ rules: - apiGroups: [""] resources: ["nodes"] verbs: ["get", "list", "watch"] + # Required so the controller can create the chart's node-reader ClusterRole + # (v0.1.2+) without triggering RBAC escalation: Kubernetes rejects grants of + # permissions the granting subject does not itself hold. + - apiGroups: ["node.k8s.io"] + resources: ["runtimeclasses"] + verbs: ["get"] + - apiGroups: ["scheduling.k8s.io"] + resources: ["priorityclasses"] + verbs: ["get"] - apiGroups: [""] resources: ["pods"] verbs: ["get", "list", "watch"] From e84869c3f0f55912d771669bbc2e1af9233a2ce3 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 21:56:15 -0400 Subject: [PATCH 10/15] fix(deploy): grant controller pods create/delete/patch for v0.1.2 chart The v0.1.2 OpenShell Helm chart creates a sandbox Role that includes pods:create, pods:delete, and pods:patch. The hypershell-controller ServiceAccount only had pods:get,list,watch, which triggers Kubernetes RBAC escalation prevention when the controller installs the chart: roles "openshell-gateway-sandbox" is forbidden: user "system:serviceaccount:hypershell-system:hypershell-controller" is attempting to grant RBAC permissions not currently held Audited all four RBAC resources the chart creates (node-reader ClusterRole, peer Role, sandbox Role, certgen Role hook) against the controller's ClusterRole; pods is the only remaining gap after the prior commit. Co-Authored-By: Claude Sonnet 4.6 --- deploy/base/platform-resources/controller-rbac.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/deploy/base/platform-resources/controller-rbac.yaml b/deploy/base/platform-resources/controller-rbac.yaml index a90a49c98..21bb298d9 100644 --- a/deploy/base/platform-resources/controller-rbac.yaml +++ b/deploy/base/platform-resources/controller-rbac.yaml @@ -53,9 +53,11 @@ rules: - apiGroups: ["scheduling.k8s.io"] resources: ["priorityclasses"] verbs: ["get"] + # create/delete/patch required so the controller can create the chart's + # sandbox Role (v0.1.2+) without triggering RBAC escalation. - apiGroups: [""] resources: ["pods"] - verbs: ["get", "list", "watch"] + verbs: ["get", "list", "watch", "create", "delete", "patch"] # Events need create so the namespace reaper can write a durable audit record. - apiGroups: [""] resources: ["events"] From fc1b0fc641d20f15b34051ba05f9eba9b6a40384 Mon Sep 17 00:00:00 2001 From: user Date: Mon, 28 Sep 2026 22:36:57 -0400 Subject: [PATCH 11/15] fix(control-plane): bump console image to sha-71335e5 for v0.1.2 compatibility The console dashboard image was not updated during the v0.1.2 bump. v0.1.2 introduced breaking proto changes (well-known time types #3113, policy L7 append #3380) that the old sha-978bcb5 console BFF cannot parse, causing the sandbox list gRPC call to hang indefinitely and the UI to show a loading spinner that never resolves. Update to sha-71335e5 (built 2026-09-23) which includes v0.1.2 compatibility. Co-Authored-By: Claude Sonnet 4.6 --- OPENSHELL_VERSION | 2 +- components/control-plane/internal/gateway/config.go | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/OPENSHELL_VERSION b/OPENSHELL_VERSION index c62df9371..8209ee10f 100644 --- a/OPENSHELL_VERSION +++ b/OPENSHELL_VERSION @@ -7,4 +7,4 @@ OPENSHELL_GATEWAY_IMAGE=quay.io/opendatahub/odh-openshell-gateway OPENSHELL_SUPERVISOR_IMAGE=quay.io/opendatahub/odh-openshell-supervisor OPENSHELL_CLI_IMAGE=quay.io/opendatahub/odh-openshell-cli OPENSHELL_CONSOLE_IMAGE=quay.io/gkrumbach07/openshell-dashboard -OPENSHELL_CONSOLE_DIGEST=sha256:c69c1f34c574556684710a7d2d2a3654f164b855efe0d273a098782447068fc5 +OPENSHELL_CONSOLE_DIGEST=sha256:1d36331138c37aa75285869a21d2aa35ebdab5b1f6980f028b28df405ec5a927 diff --git a/components/control-plane/internal/gateway/config.go b/components/control-plane/internal/gateway/config.go index 9a66759a5..cf21539c9 100644 --- a/components/control-plane/internal/gateway/config.go +++ b/components/control-plane/internal/gateway/config.go @@ -25,10 +25,10 @@ const defaultSandboxImage = "ghcr.io/nvidia/openshell-community/sandboxes/base:l // defaultConsoleImage is the OpenShell dashboard image (the per-gateway // console). The upstream project publishes it to quay.io, so clusters pull it // directly (imagePullPolicy IfNotPresent) rather than building from source. -// Pinned by digest to the sha-978bcb5 build for reproducibility; bump +// Pinned by digest to the sha-71335e5 build for reproducibility; bump // deliberately when adopting a new dashboard contract. Overridable via // HYPERSHELL_CONSOLE_IMAGE (e.g. a platform-registry mirror in production). -const defaultConsoleImage = "quay.io/gkrumbach07/openshell-dashboard@sha256:c69c1f34c574556684710a7d2d2a3654f164b855efe0d273a098782447068fc5" +const defaultConsoleImage = "quay.io/gkrumbach07/openshell-dashboard@sha256:1d36331138c37aa75285869a21d2aa35ebdab5b1f6980f028b28df405ec5a927" // defaultOAuth2ProxyImage is the oauth2-proxy sidecar image. Overridable via // HYPERSHELL_OAUTH2_PROXY_IMAGE. From 4200359e9bd9d86cdd118268cca15545bfa86336 Mon Sep 17 00:00:00 2001 From: user Date: Tue, 29 Sep 2026 09:24:09 -0400 Subject: [PATCH 12/15] docs(update-openshell): record chart vendoring, console image, and RBAC lessons from v0.1.2 Four gaps in the skill discovered during the v0.1.2-rhaiv.0 bump: - Step 3a: vendor the Helm chart from the upstream tag on every bump. The Dockerfile packages charts/openshell/ at build time (Konflux hermetic builds have no outbound network access); a stale chart silently applies the old RBAC rules and value schema even after the image tag is bumped. Includes the em-dash strip required by the pre-commit hook. - Step 3b: check the console image when proto surfaces change. A stale console BFF (quay.io/gkrumbach07/openshell-dashboard) cannot parse breaking proto changes and leaves the sandbox list UI in an infinite loading state with no visible error. Documents the Quay tag lookup, the two files that must agree (OPENSHELL_VERSION + config.go), and the symptom fingerprint. - Version footprint: add OPENSHELL_VERSION, charts/openshell/, and config.go (defaultConsoleImage) to the table. - Contract surfaces: extend the gRPC/proto row to note that the console BFF is also a gRPC client and must be checked alongside the proto. Learnings log updated with: chart vendoring requirement, chart RBAC escalation pattern (runtimeclasses/priorityclasses/pods), console image incompatibility with proto-breaking bumps, and appArmorProfile Kind issue. Co-Authored-By: Claude Sonnet 4.6 --- skills/tooling/update-openshell/SKILL.md | 78 +++++++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index 6cd19f021..a2f6509c7 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -130,6 +130,9 @@ grep -rn "" . | grep -v '\.git/' # must return only intention | File | What to change | Notes | |------|----------------|-------| +| `OPENSHELL_VERSION` | `OPENSHELL_TAG`, `OPENSHELL_CONSOLE_IMAGE`, `OPENSHELL_CONSOLE_DIGEST` | Edit first; `OPENSHELL_TAG` drives both the deployment pins and the chart vendor step | +| `charts/openshell/` | Entire directory replaced from upstream tag | Vendored chart - see Step 3a for the extraction command; expect ~48 files, large diff is normal | +| `components/control-plane/internal/gateway/config.go` | `defaultConsoleImage` constant (digest + comment tag) | Must agree with `OPENSHELL_CONSOLE_DIGEST`; check when triage flags any proto or gRPC surface change | | `deploy/base/control-plane/deployment.yaml` | `GATEWAY_IMAGE`, `GATEWAY_SUPERVISOR_IMAGE` env vars | **Source of truth** - change here first | | `deploy/base/platform-resources/controller.yaml` | same env vars | Must match the deployment above | | `specs/platform/data-model.spec.md` | `supervisor_image` default | Spec citation | @@ -268,6 +271,56 @@ to the footprint table. grep -rn "openshell/\(gateway\|supervisor\)" deploy/ # older ghcr.io image names ``` +3a. **Vendor the Helm chart.** The Dockerfile packages the chart from + `charts/openshell/` at build time - no network access is allowed inside + Konflux hermetic builds. After bumping `OPENSHELL_TAG` in `OPENSHELL_VERSION`, + replace the vendored chart directory: + + ```bash + source OPENSHELL_VERSION + git clone --depth 1 --branch "${OPENSHELL_TAG}" "${OPENSHELL_CHART_REPO}" /tmp/openshell + rm -rf charts/openshell + cp -r /tmp/openshell/deploy/helm/openshell charts/openshell + # Strip em-dashes (U+2014) - the pre-commit hook rejects them + grep -rl $'\xe2\x80\x94' charts/openshell/ | xargs -r sed -i 's/\xe2\x80\x94/-/g' + ``` + + The resulting diff will be ~48 files and thousands of lines. That is expected + and correct - it is a wholesale upstream chart snapshot, not bespoke logic. + Every bump will look like this. + + After copying, check whether the new chart adds RBAC rules that the controller + does not yet hold. Any permission the chart's ClusterRole or Role grants must + also be present in `deploy/base/platform-resources/controller-rbac.yaml`, or + Kubernetes will reject the apply with an RBAC escalation error: + + ```bash + # Quick check: list all verbs/resources from the vendored chart's RBAC templates + grep -r "resources:\|verbs:" charts/openshell/templates/ | grep -v '#' + ``` + +3b. **Check the console image.** The console dashboard + (`quay.io/gkrumbach07/openshell-dashboard`) has its own tagging scheme and is + NOT locked to `OPENSHELL_TAG`. It only needs a bump when the gateway's + workspace/sandbox gRPC proto changes (breaking proto changes cause the + dashboard BFF to hang loading sandbox lists with no visible error). Check + whenever triage (Step 2) flags a proto or gRPC surface change: + + ```bash + # List available console image tags (newest first) + curl -s "https://quay.io/api/v1/repository/gkrumbach07/openshell-dashboard/tag/?limit=20" \ + | python3 -c "import sys,json; [print(t['name'],t.get('manifest_digest','')) for t in json.load(sys.stdin)['tags']]" + ``` + + When updating, set both files - they must agree: + - `OPENSHELL_CONSOLE_DIGEST` in `OPENSHELL_VERSION` + - `defaultConsoleImage` in `components/control-plane/internal/gateway/config.go` + (also update the `sha-XXXXXXX` tag name in the comment on the line above) + + Symptom of a stale console image after a proto-breaking bump: the Sandboxes + tab shows an infinite loading spinner that never resolves, with no error in the + browser console. The gateway itself is healthy - only the BFF gRPC call hangs. + 4. **Verify contracts.** For each `needs-decision` item from step 2, check the thing it touches: - **Gateway config (`gateway.toml`)**: diff the keys the control plane renders @@ -317,7 +370,7 @@ does, the item is `needs-decision`: | Gateway/supervisor config schema (TOML) | Control plane renders `gateway.toml` | `manifests/gateway/configmap.yaml`, `internal/gateway/config.go` | | Sandbox CR / `agents.x-k8s.io` API version | Gateway manages sandboxes; RBAC grants on it | `manifests/gateway/rbac.yaml`, `networkpolicy.yaml`, `deploy/base/controller-rbac.yaml` | | Credential storage drivers | HyperShell selects/validates drivers | `ValidateCredentialDriverConfig`, `openshell-gateway-credentials.spec.md` | -| gRPC/proto surface | API server + control plane speak gRPC | `components/api-server/proto/` | +| gRPC/proto surface | API server + control plane speak gRPC; console BFF calls gateway gRPC directly for sandbox/workspace ops | `components/api-server/proto/`; also check console image compatibility (Step 3b) when proto changes are in triage | | Gateway/supervisor CLI flags & env | Control plane sets them | `configmap.yaml`, deployment manifests | | PKI / TLS / ingress (Route, cert-manager, Gateway API) | HyperShell hand-rolls per-tenant PKI + ingress | `internal/gateway/` reconciler, ingress specs | | Auth (OIDC) | OIDC is the client auth mechanism (no client mTLS) | `ValidateOIDCConfig`, gateway OIDC config | @@ -378,6 +431,29 @@ Newest first. Each entry: version, date, what happened, what changed in the repo - **Image digests:** Use `skopeo inspect --no-creds docker://quay.io/opendatahub/odh-openshell-gateway:` to retrieve the manifest digest. If skopeo is unavailable, the Quay v1 tag API returns the `manifest_digest` field for a given `specificTag` query parameter. + - **Chart vendoring is required on every bump.** `charts/openshell/` must be + replaced from the upstream tag. If it's stale, the Dockerfile packages the old + chart silently - new RBAC rules, value schema changes, and template fixes are + not applied even though the image tag was bumped. The `rm -rf charts/openshell + && cp -r` pattern plus em-dash strip is the complete update (see Step 3a). + - **Chart RBAC escalation.** v0.1.2 added a `node-reader` ClusterRole (grants + `runtimeclasses` get, `priorityclasses` get) and a sandbox Role (grants `pods` + create/delete/patch). Both had to be added to `controller-rbac.yaml` to avoid + RBAC escalation errors on apply. On every bump, scan the new chart's RBAC + templates and cross-check against the controller's ClusterRole. + - **Console image must be checked when proto surfaces change.** v0.1.2 changed + the workspace/sandbox gRPC proto (`refactor(proto)!: use well-known time types` + #3113). The old console image (`sha-978bcb5`) could not parse v0.1.2 gateway + responses - the sandbox list UI showed an infinite loading spinner with no + error. Updated to `sha-71335e5` (built 2026-09-23, + `sha256:1d36331138c37aa75285869a21d2aa35ebdab5b1f6980f028b28df405ec5a927`). + Both `OPENSHELL_CONSOLE_DIGEST` in `OPENSHELL_VERSION` and `defaultConsoleImage` + in `config.go` must be updated together and must agree. See Step 3b. + - **`appArmorProfile` in Kind.** v0.1.2 chart sets + `securityContext.appArmorProfile: type: RuntimeDefault` on pods. Kind does not + support AppArmor, so pods fail to schedule. Suppressed via a Helm values + override in the control plane's values builder. On future bumps, check whether + the chart adds new Linux security context fields that Kind does not support. - **Skill correction (2026-09-25, HYPERSHELL-301):** Skill had two structural errors discovered during build-agent work: From dc1854e736967c6d07a7a9d7e3a8c061b2d51c6c Mon Sep 17 00:00:00 2001 From: user Date: Tue, 29 Sep 2026 13:04:11 -0400 Subject: [PATCH 13/15] feat(makefile): add vendor-openshell-chart target for hermetic chart updates Automates the manual chart vendoring step required on every OpenShell version bump. The target reads OPENSHELL_TAG and OPENSHELL_CHART_REPO from OPENSHELL_VERSION, clones the upstream tag at depth 1, replaces charts/openshell/ wholesale, and strips em-dashes that the pre-commit hook rejects. Update the update-openshell skill Step 3a to call `make vendor-openshell-chart` instead of documenting the raw git clone/cp/sed sequence. Co-Authored-By: Claude Sonnet 4.6 --- Makefile | 11 +++++++++ skills/tooling/update-openshell/SKILL.md | 29 +++++++++++------------- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/Makefile b/Makefile index 200318274..65e04d565 100644 --- a/Makefile +++ b/Makefile @@ -239,6 +239,17 @@ check-dependency-age: test-dependency-age-policy sync-openshell-version: PYTHONDONTWRITEBYTECODE=1 python3 scripts/sync_openshell_version.py --stamp +.PHONY: vendor-openshell-chart +vendor-openshell-chart: + @. ./OPENSHELL_VERSION && \ + echo "Vendoring OpenShell chart $$OPENSHELL_TAG from $$OPENSHELL_CHART_REPO..." && \ + git clone --depth 1 --branch "$$OPENSHELL_TAG" "$$OPENSHELL_CHART_REPO" /tmp/openshell-chart-vendor && \ + rm -rf charts/openshell && \ + cp -r /tmp/openshell-chart-vendor/deploy/helm/openshell charts/openshell && \ + rm -rf /tmp/openshell-chart-vendor && \ + grep -rl $$'\xe2\x80\x94' charts/openshell/ | xargs -r sed -i 's/\xe2\x80\x94/-/g' && \ + echo "Vendored charts/openshell/ at $$OPENSHELL_TAG" + .PHONY: test-openshell-version-policy test-openshell-version-policy: PYTHONDONTWRITEBYTECODE=1 python3 -m unittest scripts/test_sync_openshell_version.py diff --git a/skills/tooling/update-openshell/SKILL.md b/skills/tooling/update-openshell/SKILL.md index a2f6509c7..5b70a10ae 100644 --- a/skills/tooling/update-openshell/SKILL.md +++ b/skills/tooling/update-openshell/SKILL.md @@ -274,25 +274,22 @@ to the footprint table. 3a. **Vendor the Helm chart.** The Dockerfile packages the chart from `charts/openshell/` at build time - no network access is allowed inside Konflux hermetic builds. After bumping `OPENSHELL_TAG` in `OPENSHELL_VERSION`, - replace the vendored chart directory: + run the vendor target: ```bash - source OPENSHELL_VERSION - git clone --depth 1 --branch "${OPENSHELL_TAG}" "${OPENSHELL_CHART_REPO}" /tmp/openshell - rm -rf charts/openshell - cp -r /tmp/openshell/deploy/helm/openshell charts/openshell - # Strip em-dashes (U+2014) - the pre-commit hook rejects them - grep -rl $'\xe2\x80\x94' charts/openshell/ | xargs -r sed -i 's/\xe2\x80\x94/-/g' + make vendor-openshell-chart ``` - The resulting diff will be ~48 files and thousands of lines. That is expected - and correct - it is a wholesale upstream chart snapshot, not bespoke logic. - Every bump will look like this. + This clones the upstream tag declared in `OPENSHELL_VERSION`, replaces + `charts/openshell/` wholesale, and strips em-dashes (U+2014) which the + pre-commit hook rejects. The resulting diff will be ~48 files and thousands + of lines - that is expected. Every bump will look like this. - After copying, check whether the new chart adds RBAC rules that the controller - does not yet hold. Any permission the chart's ClusterRole or Role grants must - also be present in `deploy/base/platform-resources/controller-rbac.yaml`, or - Kubernetes will reject the apply with an RBAC escalation error: + After vendoring, check whether the new chart adds RBAC rules that the + controller does not yet hold. Any permission the chart's ClusterRole or Role + grants must also be present in + `deploy/base/platform-resources/controller-rbac.yaml`, or Kubernetes will + reject the apply with an RBAC escalation error: ```bash # Quick check: list all verbs/resources from the vendored chart's RBAC templates @@ -434,8 +431,8 @@ Newest first. Each entry: version, date, what happened, what changed in the repo - **Chart vendoring is required on every bump.** `charts/openshell/` must be replaced from the upstream tag. If it's stale, the Dockerfile packages the old chart silently - new RBAC rules, value schema changes, and template fixes are - not applied even though the image tag was bumped. The `rm -rf charts/openshell - && cp -r` pattern plus em-dash strip is the complete update (see Step 3a). + not applied even though the image tag was bumped. Run `make vendor-openshell-chart` + after editing `OPENSHELL_VERSION` (see Step 3a). - **Chart RBAC escalation.** v0.1.2 added a `node-reader` ClusterRole (grants `runtimeclasses` get, `priorityclasses` get) and a sandbox Role (grants `pods` create/delete/patch). Both had to be added to `controller-rbac.yaml` to avoid From 694fe5e984d42e498e61160061f1b9ff4526c008 Mon Sep 17 00:00:00 2001 From: user Date: Tue, 29 Sep 2026 14:07:32 -0400 Subject: [PATCH 14/15] docs: bump image references to v0.1.2-rhaiv.0 in specs and deploy skills Address Amber review findings: - global-architecture.spec.md: update the v1beta1 API version note from gateway 0.0.109 to 0.1.2-rhaiv.0 (confirmed by Kind E2E sandbox ops) - deploy-cluster/SKILL.md: refresh example env var values to v0.1.2-rhaiv.0 - gcp-cluster/SKILL.md: update active example commands to v0.1.2-rhaiv.0 (reference run sections at the old tag are historical records, left intact) Co-Authored-By: Claude Sonnet 4.6 --- skills/deploy/deploy-cluster/SKILL.md | 4 ++-- skills/deploy/gcp-cluster/SKILL.md | 4 ++-- specs/platform/global-architecture.spec.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/skills/deploy/deploy-cluster/SKILL.md b/skills/deploy/deploy-cluster/SKILL.md index 3bc9de792..61bea737a 100644 --- a/skills/deploy/deploy-cluster/SKILL.md +++ b/skills/deploy/deploy-cluster/SKILL.md @@ -229,9 +229,9 @@ reconciliation fails immediately: ```yaml env: - name: GATEWAY_IMAGE - value: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b... + value: quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0@sha256:... - name: GATEWAY_SUPERVISOR_IMAGE - value: quay.io/opendatahub/odh-openshell-supervisor:v0.0.109-rhaiv.0@sha256:96e... + value: quay.io/opendatahub/odh-openshell-supervisor:v0.1.2-rhaiv.0@sha256:... ``` When a gateway is created, the controller reads these environment variables to determine diff --git a/skills/deploy/gcp-cluster/SKILL.md b/skills/deploy/gcp-cluster/SKILL.md index f919927d5..68dc7c2d9 100644 --- a/skills/deploy/gcp-cluster/SKILL.md +++ b/skills/deploy/gcp-cluster/SKILL.md @@ -402,7 +402,7 @@ CLUSTER_ID=$(curl -sk "$API/managed_clusters" -H "Authorization: Bearer $TOKEN" # GatewayRelease RELEASE=$(curl -sk -X POST "$API/gateway_releases" -H 'Content-Type: application/json' \ -H "Authorization: Bearer $TOKEN" \ - -d "{\"name\":\"openshell-0.0.109\",\"image\":\"quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0\"}") + -d "{\"name\":\"openshell-0.1.2\",\"image\":\"quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0\"}") RELEASE_ID=$(echo "$RELEASE" | python3 -c "import json,sys; print(json.load(sys.stdin)['id'])") echo "Cluster=$CLUSTER_ID Release=$RELEASE_ID" @@ -417,7 +417,7 @@ GATEWAY=$(curl -sk -X POST "$API/gateways" -H 'Content-Type: application/json' \ \"cluster_id\": \"$CLUSTER_ID\", \"release_id\": \"$RELEASE_ID\", \"namespace\": \"openshell-gcptest\", - \"image\": \"quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0\", + \"image\": \"quay.io/opendatahub/odh-openshell-gateway:v0.1.2-rhaiv.0\", \"route\": \"{\\\"enabled\\\": true}\" }") echo "$GATEWAY" | python3 -m json.tool diff --git a/specs/platform/global-architecture.spec.md b/specs/platform/global-architecture.spec.md index 544296a81..7734559d5 100644 --- a/specs/platform/global-architecture.spec.md +++ b/specs/platform/global-architecture.spec.md @@ -891,7 +891,7 @@ NOT install the CRD/controller, which is a cluster prerequisite on par with cert-manager. The gateway's Kubernetes compute driver watches this CRD; when it is absent the sandbox RPCs surface as gRPC `Unimplemented` and the driver logs `no supported Agent Sandbox API version is available`. The version installed SHALL -serve the API version the gateway requires (`v1beta1` for gateway 0.0.109; upstream +serve the API version the gateway requires (`v1beta1` for gateway 0.1.2-rhaiv.0; upstream `v0.5.x`). #### Requirement: Sandbox Base Image Supports an In-Cluster Registry From 9a26bdba027335d8e369a676428e159e88e85a2b Mon Sep 17 00:00:00 2001 From: user Date: Tue, 29 Sep 2026 14:41:09 -0400 Subject: [PATCH 15/15] fix: address Amber review findings on helm spec, appArmorProfile, and console image docs - openshell-gateway-helm-adoption.spec.md: update the SHALL to reflect the vendored chart model - OPENSHELL_CHART_REPO/OPENSHELL_TAG are now provenance/re-vendor pointers consumed by make vendor-openshell-chart, not build-time git clone inputs to the Dockerfile - helm/values.go: drop the server.appArmorProfile suppression; the vendored v0.1.2 chart carries no appArmorProfile key (grep returns zero hits), making the override a confirmed no-op against the committed chart - e2e-console-browser-testing.spec.md: refresh pinned console image refs from sha-978bcb5 / sha256:c69c1f34... to sha-71335e5 / sha256:1d363311... to match config.go after the v0.1.2 console image bump - openshell-gateway.spec.md: remove app_armor_profile = "Unconfined" from the rendered gateway config example; the v0.1.2 gateway binary no longer accepts this TOML field Co-Authored-By: Claude Sonnet 4.6 --- components/control-plane/internal/helm/values.go | 7 ------- specs/platform/e2e-console-browser-testing.spec.md | 4 ++-- specs/platform/openshell-gateway-helm-adoption.spec.md | 6 +++--- specs/platform/openshell-gateway.spec.md | 1 - 4 files changed, 5 insertions(+), 13 deletions(-) diff --git a/components/control-plane/internal/helm/values.go b/components/control-plane/internal/helm/values.go index 8640ad7a6..82fe8f52e 100644 --- a/components/control-plane/internal/helm/values.go +++ b/components/control-plane/internal/helm/values.go @@ -166,13 +166,6 @@ func (b *ValuesBuilder) buildCoreValues(values map[string]interface{}) error { setNestedValue(values, false, "grpcRoute", "enabled") setNestedValue(values, false, "grpcRoute", "backendTLSPolicy", "enabled") - // Suppress server.appArmorProfile: pre-v0.1.2 chart defaulted this to - // "Unconfined" and rendered app_armor_profile into the TOML config, but - // the v0.1.2 gateway binary removed that config field and rejects it. - // An empty string is falsy in Helm templates, so the field is not rendered. - // The v0.1.2+ chart ignores this key entirely. - setNestedValue(values, "", "server", "appArmorProfile") - // cert-manager configuration setNestedValue(values, b.HasCertManager, "certManager", "enabled") diff --git a/specs/platform/e2e-console-browser-testing.spec.md b/specs/platform/e2e-console-browser-testing.spec.md index 92d71ff83..2c279ead2 100644 --- a/specs/platform/e2e-console-browser-testing.spec.md +++ b/specs/platform/e2e-console-browser-testing.spec.md @@ -647,7 +647,7 @@ The console link opens in a new tab (`target="_blank"`). Do not click it; read `href` and `open` it in the same tab so the session cookie jar is reused. **OpenShell gateway console** (upstream `main`; **verify against the pinned image** -`quay.io/gkrumbach07/openshell-dashboard@sha256:c69c1f34...`, tag `sha-978bcb5`, +`quay.io/gkrumbach07/openshell-dashboard@sha256:1d36331138...`, tag `sha-71335e5`, see `components/control-plane/internal/gateway/config.go`) | Element | Selector / path | @@ -763,7 +763,7 @@ implementation. These supersede the assumptions above where they differ. detail page actions toggle is named "Actions"; "Actions for {gatewayName}" is the list row toggle. The cluster option's accessible name is " Provider: ; region: ". -- **OpenShell console (sha-978bcb5).** Realm SSO makes the console login silent. +- **OpenShell console (sha-71335e5).** Realm SSO makes the console login silent. `whoami` is `{"subject","displayName","identityProvider","roles":[...]}`; `/api/v1/gateway` is `{"status","gatewayVersion","computeDrivers"}`. The create-sandbox dialog uses test ids `sandbox-name-input`, diff --git a/specs/platform/openshell-gateway-helm-adoption.spec.md b/specs/platform/openshell-gateway-helm-adoption.spec.md index 2fa2d36e2..16f3f0f3c 100644 --- a/specs/platform/openshell-gateway-helm-adoption.spec.md +++ b/specs/platform/openshell-gateway-helm-adoption.spec.md @@ -136,9 +136,9 @@ The control plane SHALL load the upstream OpenShell Helm chart from a `.tgz` arc - GIVEN the chart archive is vendored into the control plane container image at `/charts/openshell.tgz` - WHEN the reconciler starts up - THEN it SHALL verify the chart exists at the embedded path and pass it to the Helm CLI for install operations -- AND the chart source repository and tag SHALL be declared in the top-level `OPENSHELL_VERSION` file (`OPENSHELL_CHART_REPO` and `OPENSHELL_TAG`) -- AND the Dockerfile SHALL clone the chart source at the specified tag, package it with `helm package`, and embed the resulting `.tgz` archive -- AND upgrading the chart version SHALL require updating `OPENSHELL_VERSION` and rebuilding the control plane image +- AND the chart source repository and tag SHALL be declared in the top-level `OPENSHELL_VERSION` file (`OPENSHELL_CHART_REPO` and `OPENSHELL_TAG`) as provenance and re-vendor coordinates, not as build-time inputs +- AND the chart source SHALL be committed to the repository under `charts/openshell/` and vendored via `make vendor-openshell-chart` (which clones the upstream tag, replaces the directory, and strips forbidden characters); the Dockerfile COPYs this committed tree and runs `helm package` with no network access, satisfying Konflux hermetic build requirements +- AND upgrading the chart version SHALL require updating `OPENSHELL_VERSION`, running `make vendor-openshell-chart` to re-vendor the chart source, and rebuilding the control plane image - AND this ensures the chart version is always coupled to the control plane release -- a given control plane image always deploys the same chart version #### Scenario: OCI registry override (development only) diff --git a/specs/platform/openshell-gateway.spec.md b/specs/platform/openshell-gateway.spec.md index 58045a8c1..6b6a92ea9 100644 --- a/specs/platform/openshell-gateway.spec.md +++ b/specs/platform/openshell-gateway.spec.md @@ -653,7 +653,6 @@ grpc_endpoint = "https://openshell-gateway..svc.cluster. service_account_name = "openshell-gateway-sandbox" supervisor_sideload_method = "image-volume" sa_token_ttl_secs = 3600 -app_armor_profile = "Unconfined" topology = "single-cluster" [openshell.drivers.kubernetes.sidecar]