Skip to content

fix(audit): gate log-shipper config on cert and warn on TLS handshake failures#2681

Closed
danilrwx wants to merge 2 commits into
mainfrom
fix/audit/gate-log-shipper-on-cert
Closed

fix(audit): gate log-shipper config on cert and warn on TLS handshake failures#2681
danilrwx wants to merge 2 commits into
mainfrom
fix/audit/gate-log-shipper-on-cert

Conversation

@danilrwx

@danilrwx danilrwx commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Description

Two robustness fixes for audit log delivery over the log-shipper.

  1. Audit's log-shipper resources (ClusterLogDestination and the ClusterLoggingConfigs) are no longer rendered until the audit TLS certificate has actually been generated. The certificate is produced asynchronously by a hook, so on startup these resources could be created with an empty CA/cert/key.

  2. When the audit receiver rejects a TLS connection, it now logs an actionable warning telling the operator to restart the log-shipper agent pods. Previously the handshake failure surfaced only at debug level, indistinguishable from normal connection churn.

Why do we need it, and what problem does it solve?

Audit events are delivered from the log-shipper agent to the audit receiver over a mutually-authenticated TLS socket. Two situations left audit silently broken:

  • On a fresh enablement, the log-shipper destination could be configured before the certificate existed, pointing it at an empty CA.
  • After a CA rotation, a hung vector reload keeps the agent retrying with a stale CA. Audit stops being delivered, but the only trace was a debug log the operator would never see — so there was no signal about what to do.

Now the destination is only configured once its certificate is ready, and a persistent stream of warnings on the receiver points the operator straight at the fix (restart the log-shipper agents).

Task 67136162.

What is the expected result?

  • With audit enabled but the certificate not yet generated, no audit log-shipper resources are created; they appear once the certificate is ready.
  • When the log-shipper connects with a mismatched CA, the audit receiver logs a warning naming the remedy instead of a silent debug line.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: audit
type: fix
summary: "Audit no longer configures its log-shipper destination before the TLS certificate is ready, and warns to restart the log-shipper agents when audit delivery fails on a TLS handshake."
impact_level: low

danilrwx added 2 commits July 15, 2026 10:59
The audit TLS certificate is generated asynchronously by a hook, but
audit.isEnabled only checked ModuleConfig and enabled modules. During
startup the ClusterLogDestination, cert secret and Deployment could
render with empty CA/cert/key. Require the certificate to be present
before rendering any audit resource.

Signed-off-by: Daniil Antoshin <daniil.antoshin@flant.com>
A hung vector reload after a CA rotation keeps the log-shipper agent
retrying the audit sink with a stale CA, which surfaced only as a
debug-level connection error indistinguishable from normal churn.
Perform the handshake explicitly and log an actionable warning telling
the operator to restart the log-shipper agent pods.

Signed-off-by: Daniil Antoshin <daniil.antoshin@flant.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant