Repository navigation
feat(update-openshell): bumps to openshell v0.1.2 #374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
bfbdb40
b565864
28e8e92
69a03d7
ea9bb30
cd2e39c
957aa94
1e837d2
1a610c4
e84869c
fc1b0fc
4200359
dc1854e
694fe5e
9a26bdb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,10 @@ | ||
| # OpenShell upstream coordinates. Gateway, supervisor, and CLI image tags | ||
| # and the chart ref use OPENSHELL_TAG. The console image is pinned by digest | ||
| # (it uses its own tagging scheme). Update this file when bumping versions. | ||
| OPENSHELL_TAG=v0.0.116-rhaiv.6 | ||
| OPENSHELL_TAG=v0.1.2-rhaiv.0 | ||
| OPENSHELL_CHART_REPO=https://github.com/opendatahub-io/openshell.git | ||
| OPENSHELL_GATEWAY_IMAGE=quay.io/opendatahub/odh-openshell-gateway | ||
| OPENSHELL_SUPERVISOR_IMAGE=quay.io/opendatahub/odh-openshell-supervisor | ||
| OPENSHELL_CLI_IMAGE=quay.io/opendatahub/odh-openshell-cli | ||
| OPENSHELL_CONSOLE_IMAGE=quay.io/gkrumbach07/openshell-dashboard | ||
| OPENSHELL_CONSOLE_DIGEST=sha256:c69c1f34c574556684710a7d2d2a3654f164b855efe0d273a098782447068fc5 | ||
| OPENSHELL_CONSOLE_DIGEST=sha256:1d36331138c37aa75285869a21d2aa35ebdab5b1f6980f028b28df405ec5a927 | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Minor] The v0.1.2 console bump to Separately (pre-existing, not introduced here): the console default still resolves to a personal registry |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Patterns to ignore when building packages. | ||
| .DS_Store | ||
| .git/ | ||
| .gitignore | ||
| .bzr/ | ||
| .bzrignore | ||
| .hg/ | ||
| .hgignore | ||
| .svn/ | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| *.orig | ||
| *~ | ||
| .project | ||
| .idea/ | ||
| *.tmproj | ||
| .vscode/ | ||
|
|
||
| # Ignore development files | ||
| README.md.gotmpl | ||
| skaffold.yaml | ||
| ci/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| apiVersion: v2 | ||
| # Chart name determines the OCI image name: ghcr.io/nvidia/openshell/helm-chart:<version> | ||
| name: helm-chart | ||
| description: runtime environment for autonomous agents | ||
| type: application | ||
| # version and appVersion are patched to the release semver by CI before helm package. | ||
| # appVersion doubles as the default image tag (image.tag defaults to appVersion when | ||
| # empty), so a released chart automatically pulls matching gateway, sandbox, and supervisor images. | ||
| version: 0.0.0 | ||
| appVersion: "0.0.0" |
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,310 @@ | ||
| # OpenShell Helm Chart | ||
|
|
||
| <!-- | ||
| This file is generated by helm-docs. | ||
| Edit README.md.gotmpl and values.yaml, then run `mise run helm:docs`. | ||
| --> | ||
|
|
||
| > **Experimental** - the Kubernetes deployment path is under active development. Expect rough edges and breaking changes. | ||
|
|
||
| This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`. | ||
|
|
||
| By default, this chart also creates the namespace-scoped resources needed by | ||
| sandboxes. For a shared-gateway deployment, install it with | ||
| `workspaceResources.enabled=false`, then install the | ||
| `deploy/helm/openshell-workspace` chart in every pre-provisioned workspace | ||
| namespace. The gateway and workspace releases can then be upgraded and removed | ||
| independently. Use Kubernetes `operator` workspace mode when one gateway serves | ||
| multiple pre-provisioned workspace namespaces. | ||
|
|
||
| ## Cluster-scoped vs namespaced objects | ||
|
|
||
| Most objects in this chart are namespaced and land in the release namespace. | ||
| Only two are cluster-scoped: | ||
|
|
||
| | Object | Default name | | ||
| | --- | --- | | ||
| | `ClusterRole` | `<fullname>-node-reader-<release namespace>` | | ||
| | `ClusterRoleBinding` | `<fullname>-node-reader-<release namespace>` | | ||
|
|
||
| By default the release creates both, so an install by a cluster-admin is | ||
| unchanged. On clusters where cluster-scoped RBAC is owned by a different team, | ||
| split the install in two. | ||
|
|
||
| A cluster-admin applies the cluster-scoped objects once per gateway | ||
| ServiceAccount, rendered from the same values the release uses: | ||
|
|
||
| ```shell | ||
| helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \ | ||
| --namespace openshell -f my-values.yaml \ | ||
| --set rbac.create=true \ | ||
| --set rbac.clusterScoped.create=true \ | ||
| --set agentSandbox.preflight.enabled=false \ | ||
| --show-only templates/clusterrole.yaml \ | ||
| --show-only templates/clusterrolebinding.yaml | kubectl apply -f - | ||
| ``` | ||
|
|
||
| A namespace-admin then installs and upgrades the release with cluster-scoped | ||
| objects omitted, using [`ci/values-namespace-admin.yaml`](ci/values-namespace-admin.yaml) | ||
| or the equivalent `--set`: | ||
|
|
||
| ```shell | ||
| helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \ | ||
| --namespace openshell -f my-values.yaml \ | ||
| --set rbac.clusterScoped.create=false | ||
| ``` | ||
|
|
||
| The gateway ServiceAccount name and namespace do not change, so the | ||
| pre-created `ClusterRoleBinding` keeps matching the release. This works with | ||
| `serviceAccount.create=false` too: the `ClusterRoleBinding` subject follows | ||
| `serviceAccount.name`, so render the admin step with the same values. | ||
|
|
||
| ### Which flag the installer needs | ||
|
|
||
| In the default `shared` workspace mode the release also creates a namespaced | ||
| sandbox `Role` granting Agent Sandbox (`agents.x-k8s.io`) permissions. | ||
| Kubernetes forbids granting permissions you do not hold, and the built-in | ||
| `admin` ClusterRole does not cover that CRD, so an installer holding only | ||
| `admin` cannot create it. `rbac.clusterScoped.create=false` alone is then not | ||
| enough and the install fails with `attempting to grant RBAC permissions not | ||
| currently held`. | ||
|
|
||
| | Workspace mode | Installer holds | Use | | ||
| | --- | --- | --- | | ||
| | `shared` | built-in `admin` only | `rbac.create=false`, cluster-admin pre-creates all gateway RBAC | | ||
| | `shared` | `admin` plus the sandbox permissions in the namespace | `rbac.clusterScoped.create=false` | | ||
| | `managed`, `operator` | built-in `admin` only | `rbac.clusterScoped.create=false` | | ||
|
|
||
| `managed` and `operator` render no namespaced sandbox `Role`, so no extra grant | ||
| is needed there. | ||
|
|
||
| With `rbac.create=false` the cluster-admin applies the namespaced RBAC too, | ||
| adding it to the same render: | ||
|
|
||
| ```shell | ||
| helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \ | ||
| --namespace openshell -f my-values.yaml \ | ||
| --set rbac.create=true \ | ||
| --set rbac.clusterScoped.create=true \ | ||
| --set agentSandbox.preflight.enabled=false \ | ||
| --show-only templates/clusterrole.yaml \ | ||
| --show-only templates/clusterrolebinding.yaml \ | ||
| --show-only templates/role.yaml \ | ||
| --show-only templates/rolebinding.yaml \ | ||
| --show-only templates/peer-role.yaml | kubectl apply -f - | ||
| ``` | ||
|
|
||
| To grant the installer the sandbox permissions instead, bind it to a Role | ||
| carrying the same rules as the chart's `openshell-sandbox` Role. | ||
|
|
||
| The certgen hook and credential driver RBAC keep their own flags | ||
| (`pkiInitJob.enabled` and | ||
| `server.credentialDrivers.kubernetesSecrets.rbac.create`). | ||
|
|
||
| ### Migrating an existing release | ||
|
|
||
| Helm deletes objects that leave a release manifest, so setting | ||
| `rbac.clusterScoped.create=false` on a release that already owns the | ||
| `ClusterRole` and `ClusterRoleBinding` deletes them. The gateway then loses | ||
| TokenReview until a cluster-admin re-applies them. Hand ownership over first, as | ||
| cluster-admin, so nothing is deleted: | ||
|
|
||
| ```shell | ||
| kubectl annotate clusterrole "openshell-node-reader-<namespace>" \ | ||
| helm.sh/resource-policy=keep --overwrite | ||
| kubectl annotate clusterrolebinding "openshell-node-reader-<namespace>" \ | ||
| helm.sh/resource-policy=keep --overwrite | ||
| ``` | ||
|
|
||
| The objects then survive the upgrade that sets the flag, and the cluster-admin | ||
| owns them from that point on. Fresh installs need no such step. | ||
|
|
||
| `rbac.clusterScoped.create` is independent of | ||
| `server.drivers.kubernetes.workspaceMode`. Managed and operator modes change | ||
| what the `ClusterRole` contains, but they never force the namespaced release to | ||
| apply it. Re-run the cluster-admin step after changing values that affect the | ||
| `ClusterRole` rules. | ||
|
|
||
|
|
||
| ## Prerequisites | ||
|
|
||
| > **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for | ||
| > ingress and egress in every sandbox namespace. OpenShell creates the policies, | ||
| > but Kubernetes accepts them even if no CNI enforces them. Without enforcement, | ||
| > sandbox workloads may connect directly and bypass supervisor network policy. | ||
| > Verify CNI support before installing OpenShell. | ||
|
|
||
| The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with: | ||
|
|
||
| ```shell | ||
| kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/sandbox.yaml | ||
| ``` | ||
|
|
||
| The chart does not install this cluster-scoped dependency. By default, it | ||
| fails before creating gateway resources when the cluster serves neither | ||
| supported Sandbox API (`agents.x-k8s.io/v1beta1` or | ||
| `agents.x-k8s.io/v1alpha1`). Disable the check with | ||
| `agentSandbox.preflight.enabled=false` for offline `helm template` rendering, | ||
| where Helm cannot discover cluster APIs. | ||
|
|
||
| ## Install on Kubernetes | ||
|
|
||
| ```shell | ||
| helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> | ||
| ``` | ||
|
|
||
| ## Install on OpenShift | ||
|
|
||
| See the full [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) for details. Quick start: | ||
|
|
||
| ```shell | ||
| # Precreate the openshell namespace | ||
| oc create ns openshell | ||
|
|
||
| # Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway | ||
| helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \ | ||
| --set server.disableTls=true \ | ||
| --set podSecurityContext.fsGroup=null \ | ||
| --set securityContext.runAsUser=null | ||
| ``` | ||
|
|
||
| On OpenShift 4.22+, end-to-end TLS is supported via `BackendTLSPolicy`. See the | ||
| [OpenShift install guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift#end-to-end-tls-openshift-422) for details. | ||
|
|
||
| ## Available versions | ||
|
|
||
| | Tag | Source | Notes | | ||
| | --- | --- | --- | | ||
| | `<semver>` (e.g. `0.6.0`) | Tagged GitHub release | Tracks the matching gateway, sandbox, and supervisor image versions. Recommended for production. | | ||
| | `<semver>-pre.N` (e.g. `0.1.0-pre.3`) | A specific prerelease candidate | Immutable candidate pin that tracks images with the same exact version. | | ||
| | `0.0.0-dev` | Latest commit on `main` | Floating tag, overwritten on every push. `appVersion` is `dev`, so images resolve to the `:dev` tag. | | ||
| | `0.0.0-dev.<commit-sha>` | A specific commit on `main` | Per-commit pin. Chart version and `appVersion` both use the full 40-character commit SHA, which matches the image tag pushed by CI. | | ||
|
|
||
| Prerelease and `dev` tags are intended for testing changes ahead of a release. Production deployments should pin to a stable tagged release. | ||
|
|
||
| ## Configuration | ||
|
|
||
| See [`values.yaml`](values.yaml) for source defaults. Selected overlays: | ||
|
|
||
| - [`ci/values-gateway.yaml`](ci/values-gateway.yaml) - gateway-only configuration | ||
| - [`ci/values-cert-manager.yaml`](ci/values-cert-manager.yaml) - cert-manager integration | ||
| - [`ci/values-keycloak.yaml`](ci/values-keycloak.yaml) - Keycloak OIDC integration | ||
| - [`ci/values-high-availability.yaml`](ci/values-high-availability.yaml) - CI overlay for multi-replica external PostgreSQL testing | ||
| - [`ci/values-spire.yaml`](ci/values-spire.yaml) - SPIFFE/SPIRE provider token grants | ||
| - [`ci/values-spire-stack.yaml`](ci/values-spire-stack.yaml) - SPIRE hardened chart values for local development | ||
|
|
||
| ### Database backend | ||
|
|
||
| By default, OpenShell uses SQLite and runs the gateway as a StatefulSet so the | ||
| database is backed by a per-pod PVC: | ||
|
|
||
| ```yaml | ||
| server: | ||
| dbUrl: "sqlite:/var/openshell/openshell.db" | ||
| ``` | ||
|
|
||
| #### External PostgreSQL | ||
|
|
||
| Use external PostgreSQL when the gateway should connect to a database managed | ||
| outside this chart. The OpenShell chart does not deploy a database; install | ||
| PostgreSQL separately using the chart, operator, or managed service that fits | ||
| your environment, then pass the connection URI through a Secret. | ||
|
|
||
| Create a Secret containing the PostgreSQL connection URI if one does not | ||
| already exist: | ||
|
|
||
| ```bash | ||
| kubectl create secret generic my-pg-credentials -n openshell \ | ||
| --from-literal=uri="postgresql://user:pass@host:5432/dbname" | ||
| ``` | ||
|
|
||
| Then install the chart pointing at that Secret: | ||
|
|
||
| ```bash | ||
| helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \ | ||
| -n openshell \ | ||
| --set workload.kind=deployment \ | ||
| --set server.externalDbSecret=my-pg-credentials | ||
| ``` | ||
|
|
||
| Use `workload.kind=deployment` for external database-backed multi-replica | ||
| gateways. `workload.kind=statefulset` is still available for single-replica | ||
| SQLite installs and for operators who explicitly need StatefulSet identity or | ||
| storage semantics. | ||
|
|
||
| ### Credential storage | ||
|
|
||
| By default, the chart uses the gateway's encrypted database credential storage. | ||
| The gateway writes encrypted provider credential envelopes to the OpenShell | ||
| database. The chart creates a retained Kubernetes Secret with the shared | ||
| key-encryption key and injects that key into every gateway pod, so the same | ||
| default works for single-replica and external database-backed HA deployments. | ||
|
|
||
| Use `kubernetes-secrets` or `vault` instead when credentials should live in a | ||
| cluster or external secret backend. Enabling one external credential driver | ||
| disables the default credential-storage key-encryption key Secret and env injection. | ||
|
|
||
| #### OpenShift | ||
|
|
||
| Append these flags to any of the PostgreSQL commands above for OpenShift: | ||
|
|
||
| ``` | ||
| --set server.disableTls=true \ | ||
| --set podSecurityContext.fsGroup=null \ | ||
| --set securityContext.runAsUser=null | ||
| ``` | ||
|
|
||
| ### High availability | ||
|
|
||
| Set `replicaCount` above `1` only with `server.externalDbSecret`; the default | ||
| SQLite database is per pod and cannot coordinate multiple gateway replicas. | ||
| The chart creates a headless peer Service for gateway-to-gateway relay traffic. | ||
| StatefulSet pods use stable pod DNS names through that headless Service. | ||
| Deployment pods advertise their pod IP with `OPENSHELL_PEER_ENDPOINT`, because | ||
| Kubernetes does not assign stable per-pod DNS names to Deployment replicas. | ||
|
|
||
| Gateway peer traffic uses Kubernetes ServiceAccount identity. Each gateway pod | ||
| mounts a projected, pod-bound ServiceAccount token with audience | ||
| `openshell-gateway-peer`; receiving replicas validate that token with the | ||
| Kubernetes TokenReview API, verify the live pod UID and Helm selector labels, | ||
| and authorize only the internal `PeerRelay` RPC. The chart does not create or | ||
| accept a shared gateway peer Secret. | ||
|
|
||
| With gateway TLS enabled, peer calls use the chart CA and client TLS Secret for | ||
| server verification and mTLS. The client verifies the stable gateway Service | ||
| DNS name while connecting directly to the owning pod. Custom TLS Secrets must | ||
| include that Service DNS name in the server certificate and provide the CA and | ||
| client credentials configured by `server.tls`. | ||
|
|
||
| ## Secret bootstrap | ||
|
|
||
| By default, a pre-install/pre-upgrade hook Job runs `openshell-gateway generate-certs` | ||
| to create the gateway's server/client mTLS Secrets and sandbox JWT signing Secret. | ||
| The Job uses the gateway image itself, so air-gapped environments only need to | ||
| mirror that one image (no separate openssl/alpine sidecar). | ||
|
|
||
| When `certManager.enabled=true`, cert-manager owns the TLS Secrets and the chart | ||
| runs the same hook in JWT-only mode because cert-manager does not create the | ||
| sandbox JWT signing Secret. This precedence applies even if | ||
| `pkiInitJob.enabled` remains true. Set `pkiInitJob.enabled=false` only when an | ||
| external non-cert-manager TLS source manages TLS and you pre-create the sandbox | ||
| JWT signing Secret. | ||
|
|
||
| ## SPIFFE/SPIRE provider token grants | ||
|
|
||
| Set `server.providerTokenGrants.spiffe.enabled=true` to let the gateway and | ||
| sandbox supervisors use SPIFFE JWT-SVIDs for dynamic provider token grants. The | ||
| chart keeps supervisor-to-gateway authentication on gateway-minted sandbox JWTs, | ||
| mounts the SPIFFE CSI socket into the gateway pod, exports | ||
| `OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET`, and passes the socket path to | ||
| the Kubernetes driver so sandbox pods can mount the same socket. | ||
|
|
||
| For local development, uncomment the SPIRE Helm releases in `skaffold.yaml` and | ||
| add `ci/values-spire.yaml` to the OpenShell release values files. | ||
|
|
||
| The gateway verifies supervisor JWT-SVIDs with JWT bundles fetched from the | ||
| SPIFFE Workload API, so this path does not require access to the SPIRE OIDC | ||
| discovery endpoint or its TLS CA. | ||
|
|
||
| {{ template "chart.valuesSection" . }} | ||
| {{ template "helm-docs.versionFooter" . }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # Merge after values.yaml when cert-manager CRDs are installed, e.g.: | ||
| # helm install ... -f values.yaml -f ci/values-cert-manager.yaml | ||
| # Or add this file to skaffold manifests.helm.releases[].valuesFiles. | ||
| server: | ||
| disableTls: false | ||
|
|
||
| certManager: | ||
| enabled: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Major] First minor-version bump (0.0.x -> 0.1.x): the default gateway/supervisor images move to v0.1.2-rhaiv.0, and the linked upstream changes include Agent Sandbox v1.0.3 plus breaking proto/policy changes. Because no Go code changed here, a green build does not validate runtime interop. The concrete risk: gateway 0.1.2 may require a different Agent Sandbox API version than the
v1beta1documented in specs/platform/global-architecture.spec.md (~L894, "v1beta1 for gateway 0.0.109"); if the installed CRD version no longer matches, sandbox RPCs surface as gRPCUnimplementedat runtime. The lessons-log entry in this PR defers this as unverified. Please confirm the required Agent Sandbox API version for 0.1.2 (ideally via the OpenShift e2e gate) and update that spec line rather than deferring it. Confidence: Medium.