[HWORKS-3047] Operator notes for the Kafka 4 and KRaft upgrade in 5.2 - #678
Merged
Merged
Conversation
A Cluster Administration page, in the style of the Airflow 3 operator notes, covering what the 5.2 upgrade does to the bundled Kafka (ZooKeeper to KRaft, Strimzi 1.2, Kafka 4.3.1), what to expect (the timeout budget, the broker replacements, the headroom, the irreversibility), the Helm and ArgoCD procedures, what to prepare (airgapped images, satellites first, 4.3/4.5 CRDs) and how to verify, with the chart README linked for the mechanics. Plus client compatibility: the broker side (Java clients from 2.1, librdkafka from 1.8.2) and the KIP-896 requirement on bring-your-own Kafka brokers, also added to the external Kafka page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
maismail
reviewed
Oct 2, 2026
maismail
left a comment
maismail
left a comment
Contributor
There was a problem hiding this comment.
Review for HWORKS-3047. The page covers everything asked for on logicalclocks/hopsworks-helm#2409, and its facts match the chart README at 43ecffa5: seven pod replacements, the topic command, the 3 x timeout budget. One thing to fix: the README links point at a private repository.
…ntence per line The chart repository is private, so the page no longer links to it: the 4.3/4.5 CRD step and the failure guidance are inlined, and the rest points at `helm show readme`. The ArgoCD step carries the terminal form next to the UI path, the "one upgrade" lead and the satellite-first bullet say why, cross-page links use heading IDs, and the prose is one sentence per line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
maismail
approved these changes
Oct 2, 2026
maismail
left a comment
maismail
left a comment
Contributor
There was a problem hiding this comment.
Approving: all four points are addressed at ad56fba.
Optional nit: the invalidate-cache curl hardcodes https://kubernetes.default.svc, which is right only when ArgoCD deploys to its own cluster. A <destination server, URL-encoded> placeholder, and <app> for the two argocd lines, would cover the rest.
Upstream Strimzi 1.2.0's Kafka CRD carries an empty properties map that the apiserver drops on store, so an ArgoCD Application on the chart never reaches Synced without an ignoreDifferences entry for that path. The operator page carries the block under the ArgoCD step, for clusters whose ArgoCD is not managed by Hopsworks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation for the Hopsworks 5.2 Kafka upgrade (logicalclocks/hopsworks-helm#2409), requested in that PR's review.
setup_installation/admin/kafka4.md, in the style of the Airflow 3 operator notes: what the upgrade does (ZooKeeper to KRaft, Strimzi 1.2, Kafka 4.3.1), what to expect (the timeout budget, the broker replacements, the headroom, nohelm rollbackand no--atomic), the Helm and ArgoCD procedures, what to prepare (airgapped images, satellites first, 4.3/4.5 CRDs), how to verify, and client compatibility (Java clients from 2.1, librdkafka from 1.8.2). The chart README is linked for the mechanics.setup_installation/on_prem/external_kafka_cluster.md: KIP-896 warning that bring-your-own Kafka brokers must run 2.1 or newer for 5.2's kafka-clients 4.3.1.🤖 Generated with Claude Code