Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bots/quoter-bot/helm/quoter-bot/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: >-
Morpho Midnight maker bot: validates Base and Midnight setup, bootstraps target lending
positions, and maintains two-sided rate ladders from one long-running container.
type: application
version: 0.3.0
version: 0.4.0
# Default image tag. The Docker Hub repository morphoorg/quoter publishes `latest` plus one
# immutable tag per release commit hash; pin a commit tag through `image.tag` for reproducible
# deployments.
Expand Down
34 changes: 34 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,39 @@ files over `--set` for the `config` block for the same reason.
- To keep the whole file out of Helm release storage, pre-create a Secret with the complete
configuration under the key `quoter-bot.yaml` and set `existingConfigSecret`.

## Runtime Secret from AWS Secrets Manager

`externalSecret.enabled` replaces the hand-applied runtime Secret with an
[External Secrets Operator](https://external-secrets.io) sync: the chart renders a namespaced
`SecretStore` (AWS Secrets Manager, authenticating as the chart-managed ServiceAccount through
`auth.jwt.serviceAccountRef`) and an `ExternalSecret` `<fullname>-runtime` that `dataFrom`
extracts a flat JSON object of environment variables from `externalSecret.remoteKey` (a secret
name or ARN, e.g. `quoter-bot/prd/1/runtime`). The rendered Secret is appended to `envFrom`
after any `envFrom` entries, so its keys override `config` like any other environment variable.

Prerequisites:

- External Secrets Operator installed in the cluster (`external-secrets.io/v1` CRDs).
- `serviceAccount.create: true` with an IRSA or EKS Pod Identity annotation; that IAM role
needs `secretsmanager:GetSecretValue`/`DescribeSecret` on `remoteKey` and `kms:Decrypt` on
the secret's key — no other Secrets Manager or KMS access.
- Optional [Stakater Reloader](https://github.com/stakater/Reloader): the StatefulSet is
annotated `secret.reloader.stakater.com/reload: <fullname>-runtime`, so a rotated value
restarts the pod automatically (environment variables are captured at container start).
Without Reloader, a rotation still syncs into the Secret on `refreshInterval` but the pod
needs a manual `kubectl rollout restart`.

```yaml
serviceAccount:
create: true
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::<account>:role/tools-quoter-bot-<env>-<chain>
externalSecret:
enabled: true
remoteKey: quoter-bot/prd/1/runtime
region: eu-west-3
```

## Parameters

### Image and workload
Expand Down Expand Up @@ -202,6 +235,7 @@ files over `--set` for the `config` block for the same reason.
| `existingConfigSecret` | `''` | Pre-created Secret with the full file under key `quoter-bot.yaml`; replaces the rendered Secret. |
| `env` | `[]` | Extra `EnvVar` objects; environment overrides YAML (signer secret, `BETTERSTACK_*`). `XDG_STATE_HOME` is reserved and filtered out. |
| `envFrom` | `[]` | Extra `EnvFromSource` objects for whole Secrets/ConfigMaps of overrides. |
| `externalSecret` | off | Fetch the runtime environment Secret from AWS Secrets Manager via External Secrets Operator — see below. Requires `serviceAccount.create: true` with an IRSA/Pod Identity annotation; the Secret becomes `<fullname>-runtime` and is appended to `envFrom`. |

### Persistence and security

Expand Down
6 changes: 6 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ WARNING: no `config` mapping and no `existingConfigSecret` were provided. The bo
environment-only mode and fails loudly unless `env`/`envFrom` supply every required variable.
{{- end }}

{{- if .Values.externalSecret.enabled }}
Runtime environment: ExternalSecret {{ include "quoter-bot.runtimeSecretName" . }} syncs
{{ .Values.externalSecret.remoteKey }} from AWS Secrets Manager every
{{ .Values.externalSecret.refreshInterval }}; Stakater Reloader restarts the pod on rotation.
{{- end }}

{{- if not .Values.persistence.enabled }}

WARNING: persistence is disabled. Durable offer-group ownership state is lost on every
Expand Down
10 changes: 10 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,13 @@ keeping the pin unique even when long release names share their truncated prefix
{{- define "quoter-bot.configYaml" -}}
{{- toYaml .Values.config }}
{{- end }}

{{/* Name of the ExternalSecret-owned runtime environment Secret. */}}
{{- define "quoter-bot.runtimeSecretName" -}}
{{- $fullname := include "quoter-bot.fullname" . }}
{{- if gt (len $fullname) 55 }}
{{- printf "%s-%s-runtime" ($fullname | trunc 46 | trimSuffix "-") (sha256sum $fullname | trunc 8) }}
{{- else }}
{{- printf "%s-runtime" $fullname }}
{{- end }}
{{- end }}
23 changes: 23 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/templates/externalsecret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{{- if .Values.externalSecret.enabled }}
# Extracts the whole Secrets Manager value as a flat JSON object of environment variables.
# creationPolicy Owner garbage-collects the rendered Secret with this ExternalSecret;
# deletionPolicy Retain keeps it (and the pod's env) if the ExternalSecret itself is removed.
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: {{ include "quoter-bot.runtimeSecretName" . | quote }}
labels:
{{- include "quoter-bot.labels" . | nindent 4 }}
spec:
refreshInterval: {{ .Values.externalSecret.refreshInterval }}
secretStoreRef:
name: {{ include "quoter-bot.fullname" . }}-aws
kind: SecretStore
target:
name: {{ include "quoter-bot.runtimeSecretName" . | quote }}
creationPolicy: Owner
deletionPolicy: Retain
Comment on lines +18 to +19

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use Orphan when the runtime Secret must survive deletion

If externalSecret.enabled is later disabled or the ExternalSecret is otherwise removed, creationPolicy: Owner places an owner reference on the generated Secret, so Kubernetes garbage collection deletes it with the ExternalSecret despite deletionPolicy: Retain. That deletion policy only governs what happens when the provider-side secret disappears. Consequently, a migration that disables this feature and references the supposedly retained runtime Secret through envFrom leaves the replacement pod unable to start; use creationPolicy: Orphan if retention on ExternalSecret removal is intended, or remove the retention claim.

Useful? React with 👍 / 👎.

dataFrom:
- extract:
key: {{ .Values.externalSecret.remoteKey | quote }}
{{- end }}
28 changes: 28 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/templates/secretstore.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{{- if .Values.externalSecret.enabled }}
{{- if not .Values.serviceAccount.create }}
{{- fail "externalSecret.enabled requires serviceAccount.create: the SecretStore authenticates as the chart-managed ServiceAccount (auth.jwt.serviceAccountRef)" }}
{{- end }}
{{- if not .Values.externalSecret.remoteKey }}
{{- fail "externalSecret.enabled requires externalSecret.remoteKey: the AWS Secrets Manager secret name or ARN" }}
{{- end }}
{{- if not .Values.externalSecret.region }}
{{- fail "externalSecret.enabled requires externalSecret.region: the AWS region of remoteKey" }}
{{- end }}
# Namespaced SecretStore authenticating as the chart-managed ServiceAccount; IRSA or EKS Pod
# Identity turns its projected token into the bot's IAM role.
apiVersion: external-secrets.io/v1
kind: SecretStore
metadata:
name: {{ include "quoter-bot.fullname" . }}-aws

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Long release names block secret sync

For a fullname over 59 characters, SecretStore adds -aws without shortening the name. Kubernetes rejects the resource, blocking installation of the runtime Secret.

Learn more

The fullname helper permits names up to 63 characters, while a Kubernetes SecretStore name must also fit within 63 characters. The same overlong value appears in secretStoreRef. The runtime Secret already uses a bounded, hashed name helper.

Example: With a 63-character fullname, appending -aws produces a 67-character SecretStore name. Kubernetes rejects it instead of creating the store.

Recommended fix: Add a bounded, hash-suffixed SecretStore name helper and use it for both the SecretStore metadata name and the ExternalSecret's secretStoreRef.name.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

labels:
{{- include "quoter-bot.labels" . | nindent 4 }}
spec:
provider:
aws:
service: SecretsManager
region: {{ .Values.externalSecret.region | quote }}
auth:
jwt:
serviceAccountRef:
name: {{ include "quoter-bot.serviceAccountName" . | quote }}
Comment on lines +24 to +27

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Pod Identity cannot sync runtime secrets

With EKS Pod Identity alone, auth.jwt.serviceAccountRef attempts web-identity authentication rather than using the bot pod's credentials. ESO runs in another pod, so the SecretStore cannot use the bot's Pod Identity association.

Learn more

A SecretStore runs in External Secrets Operator, not in the bot pod. JWT service-account authentication uses a requested service-account token for a web-identity exchange with AWS STS. An EKS Pod Identity association supplies container credentials to pods using the account, not a web-identity role ARN on the ServiceAccount. The chart advertises both IRSA and EKS Pod Identity as compatible identities in runtime Secret setup.

Example: An operator associates the chart's ServiceAccount with an IAM role using EKS Pod Identity and enables externalSecret. The bot can receive Pod Identity credentials, but ESO cannot use that association through auth.jwt; the runtime Secret never syncs and the bot remains unable to start with its required environment.

Recommended fix: Either restrict this SecretStore configuration and documentation to IRSA with a service-account role annotation, or implement an ESO-supported Pod Identity authentication mode and document the required identity of the ESO controller.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +24 to +27

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not advertise Pod Identity for JWT authentication

When this chart is deployed with EKS Pod Identity as the README permits, auth.jwt.serviceAccountRef does not use that association: ESO obtains a projected token for the referenced service account and performs the IRSA web-identity flow, which requires the IRSA role annotation and trust policy. Pod Identity instead supplies container credentials to the ESO controller pod and has no service-account role annotation, so the SecretStore cannot authenticate and the runtime Secret is never created. Restrict this mode to IRSA or provide a separate Pod Identity-compatible authentication path.

Useful? React with 👍 / 👎.

{{- end }}
11 changes: 10 additions & 1 deletion bots/quoter-bot/helm/quoter-bot/templates/statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ metadata:
name: {{ include "quoter-bot.fullname" . | quote }}
labels:
{{- include "quoter-bot.labels" . | nindent 4 }}
{{- if .Values.externalSecret.enabled }}
annotations:
# Annotation lives on the workload, not the pod template: Stakater Reloader rolls the
# StatefulSet itself when the ExternalSecret-owned Secret changes (env is captured at
# container start, so a rotated value must restart the pod to take effect).
secret.reloader.stakater.com/reload: {{ include "quoter-bot.runtimeSecretName" . | quote }}
{{- end }}
spec:
# The bot is a singleton writer: its nonce cursor, serialized mutation queue, and durable
# offer-group ownership state are per-instance, so two replicas against one maker on the same
Expand Down Expand Up @@ -195,7 +202,9 @@ spec:
{{- with $extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.envFrom }}
{{- $runtimeEnvFrom := ternary (list (dict "secretRef" (dict "name" (include "quoter-bot.runtimeSecretName" .)))) (list) .Values.externalSecret.enabled }}
{{- $envFrom := concat .Values.envFrom $runtimeEnvFrom }}
{{- with $envFrom }}
envFrom:
{{- toYaml . | nindent 12 }}
{{- end }}
Expand Down
16 changes: 16 additions & 0 deletions bots/quoter-bot/helm/quoter-bot/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,22 @@ envFrom: []
# - secretRef:
# name: quoter-bot-signer

# Deliver the runtime environment Secret from AWS Secrets Manager through External Secrets
# Operator (external-secrets.io/v1) instead of a hand-applied Secret. Requires
# serviceAccount.create: true with an IRSA/Pod Identity annotation: the SecretStore
# authenticates as that ServiceAccount (auth.jwt.serviceAccountRef), so the bot's own IAM role
# needs secretsmanager:GetSecretValue/DescribeSecret on `remoteKey` and kms:Decrypt on its key.
# The remote value is a flat JSON object of environment variables; every key becomes a Secret
# key of <fullname>-runtime, which the chart appends to envFrom. The StatefulSet is annotated
# `secret.reloader.stakater.com/reload: <fullname>-runtime`, so with Stakater Reloader installed a
# rotated value restarts the pod (environment variables are captured at container start).
externalSecret:
enabled: false
# Secrets Manager secret name or ARN, e.g. quoter-bot/prd/1/runtime.
remoteKey: ''
region: ''
refreshInterval: 5m

# Durable ownership state (bot-issued offer-group IDs) lives under this mount; the chart
# always sets XDG_STATE_HOME to `mountPath`. Losing it makes previously bot-issued groups
# unknown, which fails readiness until an operator invalidates or adopts them — keep
Expand Down
Loading