OKDP Sandbox is a hands-on environment for deploying, testing, and exploring the OKDP ecosystem on a local Kubernetes cluster.

It deploys the platform foundations (identity, object storage, SQL, secrets, ingress) and the OKDP Control Plane on a local cluster. Data services (Spark jobs, notebooks, SQL querying, dashboards) are then instantiated per project through the Control Plane. See What is included for the full component list.

What is included in the sandbox?

The default OKDP sandbox deploys the platform foundations and the Control Plane:

• Keycloak for identity and access management
• CloudNativePG and a PostgreSQL instance for SQL storage
• External Secrets for secret management
• Spark Operator for Spark workloads
• cert-manager and ingress-nginx for TLS and routing
• OKDP Control Plane (server and UI) for platform management

SeaweedFS (S3 object storage, needed by the data services, replaceable by any S3-compatible backend such as RustFS) and Vault (secret backend) are optional components: see clusters/sandbox/optional.

Data services (Airflow, JupyterHub, Trino, Hive Metastore, Superset, Spark History Server) are not part of the sandbox deployment: they are instantiated per project through the Control Plane.

The three layers

The sandbox reads as three layers, each optional and building on the previous one:

1. The platform (clusters/sandbox/): what the Quick start below deploys.
2. The demo project (clusters/sandbox/project-demo/): a complete project instance with its PostgreSQL, Connections and data services, the same shape the Control Plane deploys from the service catalog. See its README.
3. Example workloads (clusters/sandbox/project-demo/examples/): the okdp-examples medallion lakehouse, seeded onto the demo project.

Repository scope

This repository owns the single-cluster sandbox deployment. It describes how to deploy the OKDP platform onto a local Kubernetes cluster and contains the deployment assets only:

• clusters/sandbox/flux/ : Flux bootstrap of the KuboCD controller (kubocd.yaml)
• clusters/sandbox/releases/ : KuboCD Release manifests (what gets installed, which package tag, which parameters)
• clusters/sandbox/contexts/ : the platform Context (in okdp-system, whose namespace is declared at the top of the file) and the kubocd-system one that keeps the platform Releases on hand-declared OIDC clients
• clusters/sandbox/contracts/ : the KuboCD ClusterContract files, applied before the contexts
• clusters/sandbox/optional/ : components the platform can run on but does not need, applied by hand only, see its README
• clusters/sandbox/project-demo/ : the demo project and its example workloads (layers 2 and 3), see its README
• docs/ : deployment guides (DNS, certificates)

The packages themselves (the KuboCD packages bundled as OCI artifacts) live in dedicated repositories, split by ownership, OKDP core versus the third-party dependencies OKDP does not own and are consumed here from the registry. This repository never builds packages, it only deploys published ones.

Concern	Owner
KuboCD packages for core services and control plane	OKDP/platform-packages
KuboCD packages for third-party and bootstrap dependencies	OKDP/sandbox-dependencies
Reusable utility Helm charts	OKDP/helm-charts-utilities
Notebooks, DAGs, and runnable examples	OKDP/okdp-examples
Control Plane web UI	OKDP/okdp-control-plane-ui
Control Plane backend server (API)	OKDP/okdp-control-plane-server
Single-cluster sandbox deployment (this repository)	OKDP/okdp-sandbox

Prerequisites

System requirements

• Minimum: 16 GB RAM and 4 CPUs
• Docker or Podman allocation: at least 8 GB RAM and 2 CPUs

Software dependencies

• Docker or a compatible container runtime
• Kind
• kubectl
• Flux CLI v2.7.5

Quick start

The sandbox deployment files live in this repository under clusters/sandbox/.

1. Clone this repository

git clone https://github.com/OKDP/okdp-sandbox.git
cd okdp-sandbox

2. Create Kubernetes Kind Cluster

Create a Kind cluster configuration file and deploy the cluster:

ℹ️ Why Kind?

Kind is a tool for running local Kubernetes clusters using Docker.
It’s ideal for development, testing, and sandbox reproducible environments.
Kind follows a manifest-first (infrastructure-as-code) approach, while Minikube is a command-line-first approach.

# Create cluster configuration
cat > /tmp/okdp-sandbox-config.yaml <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: okdp-sandbox
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30080
    hostPort: 80
  - containerPort: 30443
    hostPort: 443
  - containerPort: 30053
    hostPort: 30053
    protocol: UDP
EOF

# Create the cluster
kind create cluster --config /tmp/okdp-sandbox-config.yaml

PowerShell

# Create cluster configuration
@"
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: okdp-sandbox
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30080
    hostPort: 80
  - containerPort: 30443
    hostPort: 443
  - containerPort: 30053
    hostPort: 53
    protocol: UDP
"@ | Out-File -FilePath "$env:TEMP\okdp-sandbox-config.yaml" -Encoding UTF8

# Create the cluster
kind create cluster --config "$env:TEMP\okdp-sandbox-config.yaml"

3. Install Platform Components

Install Flux (GitOps engine)

ℹ️ Note

This step is only required for a fresh installation. If Flux is already installed and running, you do not need to install it again.
For upgrades, go directly to Deploy/Upgrade OKDP platform components.

ℹ️ What is Flux and how is it used here?

Flux is the GitOps controller that continuously reconciles your cluster state with what’s defined in Git.
The following command installs all Flux core components:

• source-controller: fetches sources such as Git repositories and Helm charts
• kustomize-controller: applies Kubernetes manifests using Kustomize
• helm-controller: manages Helm releases declaratively
• notification-controller: handles alerts and automation triggers

In this setup, Flux controllers manage resources locally and are not connected to a Git repository.
Manifests are applied manually with kubectl, so no Git access is required.

flux install

Configure proxy settings for Flux controllers (Optional):

If your environment requires a proxy to reach external sources (container registries), the following command sets the proxy configuration variables to all Flux controllers (source, kustomize, helm, notification):

[ -n "${https_proxy}${HTTPS_PROXY}" ] && kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux \
        HTTPS_PROXY="${HTTPS_PROXY:-${https_proxy}}" \
        HTTP_PROXY="${HTTP_PROXY:-${http_proxy}}" \
        NO_PROXY="${NO_PROXY:-${no_proxy}}"

PowerShell

if ($env:HTTPS_PROXY -or $env:https_proxy) {
    kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux `
        HTTPS_PROXY=($env:HTTPS_PROXY ?? $env:https_proxy) `
        HTTP_PROXY=($env:HTTP_PROXY ?? $env:http_proxy) `
        NO_PROXY=($env:NO_PROXY ?? $env:no_proxy)
}

Verify the proxy environment variables are correctly set for all Flux controllers (Optional):

kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux --list \
                                         | grep PROXY

💡 You may see the same variable (e.g., HTTPS_PROXY) repeated multiple times, one for each controller (source, kustomize, helm, notification).
This is expected and confirms that the variables were applied consistently.

💡 How to remove the Flux proxy configuration?

Use the following command if you want to remove the proxy configuration from Flux controllers:
After removing the proxy, Flux will no longer be able to pull images or manifests from external registries that require proxy access.

kubectl -n flux-system set env deploy -l app.kubernetes.io/part-of=flux \
   HTTPS_PROXY- \
   NO_PROXY-

Wait for Flux controllers to be ready

Ensures all Flux controllers (source-controller, kustomize-controller, helm-controller, notification-controller) are fully running before proceeding to the next step:

kubectl -n flux-system wait --for=condition=Available deploy \
  -l app.kubernetes.io/part-of=flux --timeout=300s

Install KuboCD (Flux extension)

ℹ️ What is KuboCD?

ℹ️ KuboCD is the continuous delivery layer built on top of Flux.
It manages platform components and applications declaratively, providing a higher-level CD abstraction for GitOps workflows.

kubectl apply -f clusters/sandbox/flux/kubocd.yaml

Wait for KuboCD to be installed before continuing:

1. Wait for Flux to finish installing KuboCD chart:

kubectl -n flux-system wait --for=condition=Ready helmrelease/kubocd-controller --timeout=300s

2. Wait for KuboCD CRDs to be registered:

kubectl wait --for=condition=Established --timeout=300s \
  crd/contexts.kubocd.kubotal.io \
  crd/releases.kubocd.kubotal.io \
  crd/configs.kubocd.kubotal.io \
  crd/clustercontracts.kubocd.kubotal.io \
  crd/connections.kubocd.kubotal.io

3. Wait for KuboCD controller to be up:

kubectl -n kubocd wait --for=condition=Available deploy/kubocd-ctrl-controller --timeout=300s

Install metrics-server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl patch deployment metrics-server -n kube-system --type=json \
  -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--kubelet-insecure-tls"},
       {"op": "replace", "path": "/spec/template/spec/containers/0/livenessProbe/timeoutSeconds", "value": 3},
       {"op": "replace", "path": "/spec/template/spec/containers/0/readinessProbe/timeoutSeconds", "value": 3}]'

The default probe timeout of 1s is too tight for a single-node kind cluster: under load the probes fail and the pod is restarted with a new IP, which the API server keeps NATing to the old one.

Wait for metrics-server to be ready:

1. Wait for the patched pod to roll out:

kubectl -n kube-system rollout status deploy/metrics-server --timeout=300s

2. Wait for the metrics API to be served:

kubectl wait --for=condition=Available apiservice/v1beta1.metrics.k8s.io --timeout=300s

3. Verify it's working:

kubectl top nodes

💡 kubectl top nodes may still report metrics not available yet for a few seconds after the API becomes available, until the first scrape completes. Re-run it if so.

Deploy/Upgrade OKDP platform components

💡 Upgrade note
To upgrade the OKDP platform components, run:

kubectl delete $(kubectl get release -n kubocd-system -o name) -n kubocd-system

This will delete all KuboCD Release resources in the kubocd-system namespace.

During upgrade command, you may see errors like:

Error from server (Forbidden): admission webhook "vrelease-v1alpha1.kb.io" denied the request: release cert-manager is protected
Error from server (Forbidden): admission webhook "vrelease-v1alpha1.kb.io" denied the request: release kubocd-webhooks is protected

These errors can be safely ignored. The affected releases are system-protected components managed by the platform and have a separate upgrade lifecycle.

Pull the latest updates locally before starting the upgrade.

git pull --rebase

Deploy/Upgrade the sandbox default context:

ℹ️ What is KuboCD Context?

KuboCD Context is a centralized, reusable, declarative and environment-aware configuration layer that provides user defined shared parameters (ingress suffixes, storage classes, certificate issuers, catalogs, and authentication settings, etc) to all the components, ensuring consistent deployment.

During deployment, KuboCD automatically resolves and injects these context variables into the target Kubernetes components across the cluster (cluster-wide), ensuring that every component is deployed with a consistent configuration.

During a Context update, changes are automatically propagated only to the affected components, which are then reconciled to align with the desired configuration.

For example, the Context enables defining different configurations for different environments:

• sandbox for experimentation
• dev for internal testing
• prod for stable production environments
• org (or global) for the organization-wide configuration that provides defaults to other environments.

Each environment can define, override or extend one or more contexts while preserving a unified, declarative deployment model.

kubectl apply -f clusters/sandbox/contracts/
kubectl apply --server-side -f clusters/sandbox/contexts/

Configure a custom ingress domain for OKDP Services (Optional)

💡 By default, the default Context uses okdp.sandbox as the ingress domain suffix.
This domain may be blocked if it does not comply with your organization’s allowed domain policy.

Use the following command to update the domain suffix to match your organization’s domain (replace <CUSTOM_DOMAIN> with your actual domain name):

kubectl -n okdp-system patch context platform --type=merge -p '{
  "spec": { "context": {
    "ingress": { "suffix": "<CUSTOM_DOMAIN>" },
    "oidc": {
      "issuerUri":   "https://keycloak.<CUSTOM_DOMAIN>/realms/master",
      "authUrl":     "https://keycloak.<CUSTOM_DOMAIN>/realms/master/protocol/openid-connect/auth",
      "tokenUrl":    "https://keycloak.<CUSTOM_DOMAIN>/realms/master/protocol/openid-connect/token",
      "jwksUri":     "https://keycloak.<CUSTOM_DOMAIN>/realms/master/protocol/openid-connect/certs",
      "userinfoUrl": "https://keycloak.<CUSTOM_DOMAIN>/realms/master/protocol/openid-connect/userinfo"
    }
  } }
}'

The OIDC endpoints carry the domain too. Patching the suffix alone moves the Keycloak route while the services keep validating tokens against the old host.

Configure proxy settings for OKDP Services (Optional)

💡 If your environment requires a proxy to reach external datasets (Superset examples, okdp examples, quay.io KuboCD packages), the following command sets the proxy configuration variables to the required OKDP services:

kubectl -n okdp-system patch context platform --type merge -p "$(cat <<EOF
spec:
  context:
    proxy:
      httpProxy: "${HTTP_PROXY:-${http_proxy}}"
      httpsProxy: "${HTTPS_PROXY:-${https_proxy}}"
      noProxy: "${NO_PROXY:-${no_proxy}}"
EOF
)"

PowerShell

kubectl -n okdp-system patch context platform --type merge -p @"
spec:
  context:
    proxy:
        httpProxy: "$($env:HTTP_PROXY ?? $env:http_proxy)"
        httpsProxy: "$($env:HTTPS_PROXY ?? $env:https_proxy)"
        noProxy: "$($env:NO_PROXY ?? $env:no_proxy)"
"@

Deploy/Upgrade OKDP components:

kubectl apply -f clusters/sandbox/releases/

ℹ️ Installing optional components:

The directory clusters/sandbox/optional is deliberately excluded from that apply. It contains components that are supported by the platform but are not required, including KubAuth, SeaweedFS, and HashiCorp Vault.
To install an optional component, see clusters/sandbox/optional/README.md for its purpose and configuration.

Verify and monitor release deployment status

Watch releases as they are deployed until all the components become ready.

kubectl get releases -A --watch

Or block until every release is ready instead of watching:

kubectl wait --for=jsonpath='{.status.phase}'=READY release --all -n kubocd-system --timeout=900s

4. DNS Setup

Enable access to OKDP services through DNS resolution for the okdp.sandbox or your custom domain <CUSTOM_DOMAIN>:

• Option 1 (Recommended): Local DNS server configuration (recommended, automatic for all services)
• Option 2: Manual /etc/hosts configuration (simple but requires manual updates)

📋 See dns-configuration.md for detailed setup instructions for your operating system.

5. SSL Certificate

For HTTPS access without warnings, two options:

Option 1: Install the CA certificate

The sandbox uses a local certificate authority.

To avoid browser warnings, export the generated CA certificate and import it into your system or browser trust store:

kubectl get secret default-issuer -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d > okdp-sandbox-ca.crt

PowerShell

# Import okdp-sandbox-ca.crt into your system's or browser's certificate store
kubectl get secret default-issuer -n cert-manager -o jsonpath='{.data.ca\.crt}' | ForEach-Object { [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_)) } | Out-File -FilePath "okdp-sandbox-ca.crt" -Encoding ASCII

Option 2: Ignore certificate warnings

• First, connect to Keycloak (https://keycloak.okdp.sandbox or https://keycloak.<CUSTOM_DOMAIN>) and accept the self-signed certificate in your browser.
• This step is mandatory for all OKDP services (UI, object storage, etc.) to communicate properly with Keycloak.

Quick Start Guide

1. Access OKDP UI: https://okdp-ui.okdp.sandbox or https://okdp-ui.<CUSTOM_DOMAIN>
2. Login credentials: Default authentication via Keycloak (login/password: adm/adm)
3. Deploy the demo project: Follow the demo project guide to provision the demo project using kubectl (namespace, storage, database, connections, and data services).

   ℹ️ Note: A project can also be provisioned directly through the OKDP UI instead of using kubectl.

4. Run the examples: Once the demo project is ready, follow OKDP examples guide to run the examples.

Cleanup

kind delete cluster --name okdp-sandbox
rm /tmp/okdp-sandbox-config.yaml

PowerShell

kind delete cluster --name okdp-sandbox
Remove-Item "$env:TEMP\okdp-sandbox-config.yaml" -Force

License

This project is licensed under the Apache License 2.0.

---

Built 🚀 for the OKDP Community