Repository navigation
Conversation
Draft. 4.0 hadn't shipped when this was written, targeted around 2026-10-28. Verified against releases/cluster/4.0 and the 4.0 docs PRs (macstadium-docs #390, #394), not against a released build. Covers the four items O3 KR4 names: emulator-config, --userdata, rbac, and the removed Intel surface. The skill had none of them; versions topped out at 3.6.4 and 3.7.0-alpha. Also fixes flags the emulator reference documented that the CLI has never accepted (--device, --cpu, --memory, and a --vm filter on list), carried over from the alpha. Noted so they don't come back. Uses orka3 throughout. The proposed orka3 to orka rename (OK-6390) is not implemented in the monorepo and is deliberately not reflected here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OK-6387 widened the platform validation regex to accept a dot and updated the CLI help text, which this reference mirrors. Matters as soon as Android 37 lands, since its SDK platform name carries a minor version. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There are no 3.7 VM Tools. 3.7.0-alpha was the emulator preview, released before the line was renamed 4.0, and VM Tools on 3.6 are 3.6.0. User Data needs the official 4.0 VM Tools. Raised by Spike on macstadium-docs#383. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restacks the 4.0 work on #20 so it carries #15, #16, and #20's corrected 3.x content. Conflicts resolved in favor of the corrected emulator flags (--device-profile, required --platform and --image-type, no --device, --cpu, --memory, or list --vm) while keeping the 4.0 additions. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…a flags - Emulator commands resolve the namespace like vm (flag, then ORKA_DEFAULT_NAMESPACE, then the orka context, then orka-default), per GetDefaultNamespace in releases/cluster/4.0. Not always orka-default. - The REST API accepts grpc on emulator configuration create and returns it in configuration responses (orka-apiserver androidemulatorconfig.go). It isn't CLI-only. - --device-profile was already in 3.7.0-alpha; 4.0 adds --config and --grpc. Fixed the SKILL.md and emulator reference wording. - Removed the CHANGELOG claim that the orka rename (OK-6390) isn't implemented. It landed in 4.0. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Verified against releases/cluster/4.0 (57f99689) and macstadium-docs #383, #390, #394, #417, #436. - Upgrading to 4.0: requires 3.6, Intel nodes block the upgrade at the pre-check, schedule with support - Removed vm resize --user/--password; vm and image reject unknown subcommands with a non-zero exit - API: removed routes return 404, cluster-info drops gpuPassthroughEnabled, Intel fields return 400, netBoost is ignored, webhook rejection messages, 3.6 objects can still be updated and deleted - Pod quota redesign (osx_node_orka_quota_job default 2, node quota 110, emulators uncapped); the 3.6 quota text is marked 3.6/alpha-only - Orka-managed PDBs and other control plane changes, 4.0 metrics - Disk-space deploy error, imagecache remove per-node 202 on --all/--tags, eviction fixes, same-name re-cache fixed in 4.0 - rbac --role admin not usable on MSDC-hosted clusters, rolebinding deprecation message and hidden help, remote-image deprecated - User Data: skipped phase, 3.7.0-alpha VM Tools skip, script visibility through Kubernetes read access - Emulator sizing and density, emulator config REST paths, config and grpc on deploy, grpcRelayPort; Play Store ADB as a 4.0 known issue Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… as orka3 OK-6390 landed in releases/cluster/4.0: the binary is orka, and the 4.0 package leaves orka3 installed as the 3.6 CLI. Follows the convention in macstadium-docs #402 and #406. - SKILL.md: new "Which CLI" section. orka is the 4.x CLI, orka3 stays the 3.6 CLI, both install side by side and share the config file and the orka kubeconfig context. No alias ships. Running the 4.0 CLI as orka3 warns on stderr only (verified in main.go). Optional symlink bridge and its caveats as #406 words them; no orka Homebrew cask yet; the legacy 2.x orka note. - Command examples, headings, and anchors across orka3-cli/ now use orka. Intel-only and removed commands and flags, the Intel to Apple silicon migration, the 3.6 AWS post-upgrade step, the 3.6 rb fallback, and the quoted GitLab doc stay orka3. - Workflow examples use orka rbac --role dev instead of the deprecated, hidden rolebinding (rb), with a 3.6 note. vm-config Intel flags are labeled removed in 4.0. Both found by the CLI drift check (skill PR #19) against the 4.0.0 CLI. - Shell completion filenames follow the new command name. - README: CLI usage text mentions both CLIs. The skill name, directory, frontmatter name, and repo are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Engineering tested it on a lab cluster: the grant covers custom-domain certificate uploads. The managed orka-admin service account is now the 3.6 hosted path only. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
|
||
| VM User Data (Orka 4.0+, Apple Silicon only): `orka vm deploy --userdata <script|base64>` runs a shell script inside the VM on first boot, as root, in the background, once per VM. Needs the Orka 4.0 orka-vm-tools in the image. Track it with the `USERDATA-PHASE` column in `orka vm list -o wide`; `skipped` means the VM already ran its script on an earlier boot. Images built with the 3.7.0-alpha VM Tools skip their User Data, so update them to the 4.0 VM Tools. Orka never returns the script in API responses, but anyone with Kubernetes read access to the VM's `VirtualMachineInstance` or pod can read it, so don't put long-lived secrets in it. See `references/commands/vm-commands.md`. | ||
|
|
||
| RBAC subjects (Orka 4.0+): `orka rbac add-subject|list-subjects|remove-subject --role admin|dev` replaces `orka rolebinding`, which still works but prints `Command "rolebinding" is deprecated` to stderr and no longer appears in `--help`. `--role admin` isn't usable by customer admins on MacStadium-hosted clusters (`--role dev` works); ask support for the managed `orka-admin` service account instead. See `references/commands/admin-commands.md`. |
There was a problem hiding this comment.
This says --role admin isn't usable on hosted clusters, but admin-commands.md says it is. In 4.0, OK-5675 lets the admin group patch the admin bindings on every cluster, so I believe admin-commands.md is right. I'd fix this line and the PR description.
There was a problem hiding this comment.
Fixed in bf31d3d: SKILL.md now says --role admin works on every 4.0 cluster, hosted included, matching admin-commands.md. Checked f646dc9aa0 (configure-orka-user-rbac grants the admin group patch/update on orka-admin, orka-dev, orka-traefik-admin and the two clusterrolebindings) and orka-go/pkg/rbac on releases/cluster/4.0. PR description updated too.
| - **Orka 4.0, any cluster, including MacStadium-hosted:** `orka rbac add-subject --role admin --serviceaccount <SA_NS>:<SA_NAME>`. This covers admin-only API calls such as custom TLS certificate uploads to `/api/v1/upload/cert`: before the grant the service account gets `403`, after it the upload succeeds (verified by engineering on a lab cluster, October 2026). Grants apply to the admin bindings that exist at the time, so re-run it after creating a namespace. | ||
| - **Orka 3.6, self-managed and Orka Anywhere on-prem:** you have cluster-level RBAC rights. Elevate a service account by editing the `orka-admin` RoleBinding directly with `kubectl`. Elevate a user by adding them to the admin OIDC group or editing the same RoleBinding. | ||
| - **Orka 3.6, MacStadium-hosted:** customer admins can't modify the `orka-admin` RoleBinding or anything else in the `default` namespace. This is expected. Ask MacStadium support to enable the managed `orka-admin` service account, after which any of your admins can mint a token with `orka3 sa token orka-admin`. No credentials are handed over. | ||
| - An admin service account's token can also create and delete service accounts, mint their tokens, and read or update registry credentials. Store it like any admin credential. If it's exposed, remove its admin role with `orka rbac remove-subject --role admin` (4.0), or delete and recreate the service account. |
There was a problem hiding this comment.
Removing the admin role doesn't revoke an exposed token, because tokens can't be revoked one at a time. remove-subject --role admin also strips the orka-dev access the service account had from creation. For an exposed token, I'd only recommend deleting the service account.
There was a problem hiding this comment.
Fixed in bf31d3d: admin-commands.md now says to delete the service account. Confirmed in orka-go/pkg/rbac RemoveAdminSubjects: it removes the subject from every binding that carries the admin OIDC group, and the orka-dev binding in orka-operator-managed.yaml.j2 carries that group, so the SA's orka-dev access goes too.
| - Output goes to `/var/log/orka/userdata.log` inside the VM. Keep credentials out of it, and note that anything printed there travels with an image saved or pushed from that VM. | ||
| - Changing User Data does not re-run the script on VMs that already ran it. Only newly deployed VMs pick up a new value. | ||
| - Check progress with `orka vm list -o wide` and the `USERDATA-PHASE` column (`userdataPhase` in API responses and CLI JSON): `pending`, `success`, `failed` (non-zero exit, check the log), `skipped` (the VM already ran the script on an earlier boot), `timeout` (only when `ORKA_USERDATA_TIMEOUT` is set). An empty value means the VM has no User Data or isn't running. Wait for any of the four terminal phases if automation needs a ready signal. | ||
| - A VM deployed from an image saved or pushed after its own User Data ran still runs its own script: the run-once record is tied to the deployment, not the image. Exception: images carrying the VM Tools from the 3.7.0-alpha emulator preview tie the record to the image, so VMs deployed from them skip their User Data. Update those images to the 4.0 VM Tools first. |
There was a problem hiding this comment.
There are no 3.7.0-alpha VM Tools; none were ever published. The sentinel fallback only matters for unreleased pre-4.0 builds, so I'd drop this exception here and in SKILL.md (line 59).
There was a problem hiding this comment.
Fixed in bf31d3d: dropped the exception in vm-commands.md, SKILL.md, and the CHANGELOG. Also checked that no orka-vm-tools/official/3.7.0-alpha pkg exists in the orka-tools bucket.
| ### Upgrading to Orka 4.0 | ||
|
|
||
| - **Orka 3.6 first.** Upgrading to 4.0.0 requires Orka 3.6. If the cluster runs an earlier version, upgrade to the latest 3.6 release first. | ||
| - **No Intel worker nodes.** Orka 4.0 runs on Apple silicon only. The upgrade checks for Intel worker nodes and stops before making any changes (the pre-check reports `Orka 4.0 does not support Intel worker nodes; remove them first:` followed by the node names). Remove the Intel nodes first, or keep the cluster on Orka 3.6, which MacStadium supports through March 2028. |
There was a problem hiding this comment.
The pre-check actually prints "Orka 4.0 does not support Intel worker nodes. Remove these nodes from the cluster first:".
There was a problem hiding this comment.
Fixed in bf31d3d: now quotes the orka-ansible/preflight.yml message. Note the Upgrade Service preflight (operator/internal/run/steps/preflight.go) uses a different string, 'Orka 4.0 does not support Intel worker nodes; remove them first: ', but since 4.0 upgrades are support-led the Ansible one is what applies.
|
|
||
| ## RBAC Commands (Rolebinding, deprecated in Orka 4.0) | ||
|
|
||
| Deprecated in Orka 4.0, in favor of `orka rbac ... --role dev`. They still work and behave the same, but on 4.0 they print `Command "rolebinding" is deprecated, use "orka rbac" with "--role dev" instead` to stderr and no longer appear in `--help`. Scripts that parse stderr will see the extra line. The `/api/v1/namespaces/{namespace}/rolebindings/orka-dev/subjects` routes are deprecated alongside them, replaced by `/api/v1/rbac`. |
There was a problem hiding this comment.
Each subcommand carries its own deprecation, so a script sees, for example, Command "add-subject" is deprecated, use "orka rbac add-subject --role dev --namespace TARGET_NAMESPACE" instead. The "rolebinding" message only appears for a bare orka rolebinding.
There was a problem hiding this comment.
Fixed in bf31d3d: admin-commands.md lists all three per-subcommand strings, copied from the Deprecated fields in orka-cli-v2/cmd/rolebinding/{add,list,remove}_subject.go, and says the "rolebinding" one is only for a bare orka rolebinding. SKILL.md quotes the add-subject example.
|
|
||
| Each emulator runs on the host next to the VMs, so it uses host memory and CPU on top of theirs. An emulator's CPU and memory come from its device profile; you can't set them at deploy time. | ||
|
|
||
| - **Plan for about 5 to 6 GB of host memory per emulator.** In MacStadium's capacity testing, a default `pixel_9` emulator (4 cores and about 2.5 GB of guest RAM) used about 5.7 GB of host memory. |
There was a problem hiding this comment.
Please sync this with the final macstadium/macstadium-docs#436 text: the pixel_9 wording fix, and "slow down first because of scheduling priority and software rendering", not memory.
There was a problem hiding this comment.
Synced in bf31d3d with #436 at 0179572: 'an emulator using the pixel_9 profile' and the 'slow down first when a node is busy' bullet (scheduling priority and software rendering). I took the priority and rendering wording from #436; I couldn't find where either is set in orka-engine or the virtual kubelet.
There was a problem hiding this comment.
No code sets that priority, which is why you couldn't find it. Engine and the Virtual Kubelet run as launchd daemons without a ProcessType, so macOS runs them at Utility priority, and the emulator inherits that because Engine starts it. VMs run in Apple's Virtualization service at normal priority, so under contention macOS favors VMs over emulators. That's still true on 4.0. It's a side effect of launchd defaults rather than a design choice, so it may change in a later release.
I can't point to a source for the software-rendering sentence, so please drop it here and in macstadium-docs#436.
| Android Emulators (Apple Silicon, 3.7+ early access): `orka3 emulator deploy|list|delete` runs an Android emulator on the same host node as a running macOS VM, connected over an ADB relay. `--vm`, `--platform`, and `--image-type` are required; there are no defaults. Not GA, not for production. Network isolation is node-level (any VM on the node can reach any emulator on it), NAT only, and ephemeral (no persistent AVD state). See `references/commands/emulator-commands.md`. | ||
| Android Emulators (Apple Silicon, GA in 4.0, early access in 3.7.0-alpha): `orka emulator deploy|list|delete` runs an Android emulator on the same host node as a running macOS VM, connected over an ADB relay. `--vm` is required, plus either `--platform` and `--image-type` or, on 4.0, `--config`; `--device-profile` is optional. Orka 4.0 adds `orka emulator-config` (alias `emc`) for reusable templates, plus `--config` and `--grpc` on deploy. There is no `--device`, `--cpu`, or `--memory` flag, and `emulator list` has no `--vm` filter. Network isolation is node-level (any VM on the node can reach any emulator on it), NAT only, and ephemeral (no persistent AVD state). See `references/commands/emulator-commands.md`. | ||
|
|
||
| Host node macOS upgrades (3.7.0-alpha+): Orka can upgrade Apple silicon host nodes to macOS Tahoe (26) in place. |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ols, Intel pre-check, rolebinding deprecations, emulator sizing - SKILL.md: --role admin works on every 4.0 cluster (OK-5675); drop the leftover host macOS line and the duplicate pod quota paragraph - admin-commands.md: delete an exposed service account; remove-subject --role admin also strips orka-dev (orka-go/pkg/rbac RemoveAdminSubjects) - Drop the 3.7.0-alpha VM Tools User Data exception (none published) - Intel pre-check message from orka-ansible/preflight.yml; 4.0 upgrade is support-led - Per-subcommand rolebinding deprecation strings from orka-cli-v2/cmd/rolebinding/*.go - Emulator sizing synced with macstadium-docs#436 (0179572) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… JSON exception - jenkins.md: add the Orka 4.0 section dropped from the 3.x PR (no internal ticket reference); netBoost defaulting per orka-operator/webhooks/defaulters/virtualmachineinstance_defaulter.go and IntelOnlySettings.Validate in virtualmachine_common_validator.go - node-commands.md: the Upgrade-Agent printcolumn isn't on the 4.0 OrkaNode - SKILL.md: name the upgrade JSON exception with orka naming Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
What changed
Rebuilds the Orka 4.0 update on top of #20, so it carries the corrected 3.x content from #15, #16, and #20 instead of reintroducing old errors (the merge kept #20's emulator flags:
--device-profile, required--platformand--image-type, no--device,--cpu,--memory, orlist --vm).orka.SKILL.mdhas a new "Which CLI" section near the top.orka3toorkarename landed in 4.0, so the "not implemented" note is gone. The REST API acceptsgrpcon emulator configuration create, so it isn't CLI-only. Emulator commands follow the usual namespace resolution, not alwaysorka-default.--device-profilewas already in 3.7.0-alpha.vm resize --user/--passwordare gone, and the Intel flags are gone from bothvm deployandvm-config create.vmandimagenow reject unknown subcommands with a non-zero exit.404,cluster-infodropsgpuPassthroughEnabled, and Intel-only fields return400.netBoostis accepted and ignored. Webhook rejection messages are included. 3.6 objects can still be updated and deleted.osx_node_orka_quota_jobdefault 2, node quota 110, emulators uncapped), with the 1.4.0 quota text marked 3.6 and alpha only. Also covers Orka-managed PDBs and other control plane changes, and the 4.0 operator metrics.Not enough disk space on node <NODE> to deploy the VMerror andimagecache remove --all/--tagsskipping uncached nodes. Covers the eviction fixes, and the same-name re-cache known issue is marked fixed in 4.0.rbac --role adminworks on every 4.0 cluster, MSDC-hosted included. Eachrolebindingsubcommand prints its own deprecation message and is hidden from--help. Workflow examples now userbac --role dev.remote-imageand the other NFS image commands are deprecated, with the exact stderr warning.skippedmeans. The script is readable by anyone with Kubernetes read access to the VMI or pod.adbcommands, and REST paths for emulator configurations. Also coversconfig/grpcon deploy andgrpcRelayPort. The Play Store ADB limitation is now a 4.0 known issue.2.0.0 (Orka 4.0)entry.Why
Customers' agents load this skill. On 4.0 the binary is
orka, the Intel surface is gone, and several 3.x answers (quota, rolebinding, emulator namespace) are wrong for 4.0. An agent following the old content would give 4.0 users commands that fail.How the dual CLI is handled
This follows macstadium-docs #402 and #406.
orkais the CLI for Orka 4.x.orka3stays the 3.6 CLI, and both can be installed side by side.orkakubeconfig context. Switching clusters still means runningconfig set --api-urland logging in again.orka3prints a warning on stderr only; stdout is unchanged.orka3fororkain the examples.SKILL.mdincludes the optionalorka3symlink bridge with #406's caveats. It also says not to suggest theorkaHomebrew cask yet (it still installs 2.4.0) and keeps #406's legacy 2.xorkanote.orka3: Intel and removed commands and flags, the Intel to Apple silicon migration, the 3.6 AWS post-upgrade step, the 3.6rbfallback, and a quoted GitLab doc.orka3-cli), directory, frontmattername, repo, URLs, image names likesonoma-90gb-orka3-arm, andorka3.pkgare unchanged.orka, and 57 lines intentionally left asorka3.Verified against
releases/cluster/4.0at57f99689(2026-10-22). I checked the CLI inpackages/orka-cli-v2(the Makefile builds and installsorka),orka-apiserver,orka-operator,orka-upgrade-service, andorka-ansible.scripts/cli-drift-check.pyfrom Add a CI check for CLI drift in the skill #19, not added to this branch), run with--binary-name orka --strict:v4.0.0-9ad87621): 0 errors, 0 warnings, 580 example commands.releases/cluster/4.0: same result.orka3CLI against the remainingorka3lines: 0 errors.Before this leaves draft
4.0 GA, around 2026-10-28.Done: 4.0 docs went live on 2026-10-27.Re-run the drift check against the released 4.0.0 CLI. In Add a CI check for CLI drift in the skill #19, bumpDone: 0 errors against the publishedCLI_VERSIONto4.0.0.v4.0.0-417ee1a5build, and Add a CI check for CLI drift in the skill #19 now defaults to4.0.0.🤖 Generated with Claude Code