feat(notification): audit Flow Notification subscriptions fleet-wide (#600)

CLAUDE.md

@@ -761,6 +761,22 @@ kbagent schedule list [--project NAME ...] [--enabled-only] [--branch ID]
 kbagent schedule detail --project NAME --schedule-id ID [--branch ID]
 kbagent schedule find [--cron-window START-END] [--not-run-since DAYS] [--project NAME ...] [--branch ID]
 
+kbagent notification list [--project NAME ...] [--event NAME] [--component-id ID] [--config-id ID] [--branch ID]
+# notification (0.84.2+, #600): read-only fleet audit of Flow Notification subscriptions -- the
+#   Flow Builder Notifications tab (bell icon: Success / Error / Processing-delay cards). Backed by
+#   the Notification Service (`GET /project-subscriptions` on `notification.{stack}`, plain Storage
+#   token, no elevated scope). NOT in the flow's `configuration`, so `flow detail` / `config detail`
+#   cannot show these -- the in-flow `type: "notification"` TASK is a different mechanism and IS
+#   visible there. Event names are KEBAB-case (`job-failed`, `job-succeeded`,
+#   `job-succeeded-with-warning`, `job-processing-long`, + `phase-job-*`); `--event` goes to the API,
+#   `--component-id`/`--config-id`/`--branch` match client-side on the subscription's own filter
+#   fields (`job.component.id` / `job.configuration.id` / `branch.id`). The endpoint is NOT
+#   branch-scoped: without `--branch` dev-branch subscriptions come back alongside production, and
+#   `--branch` is NEVER inferred from the project's active branch (that would hide the production
+#   recipients an audit exists to check). A subscription with no config filter is the catch-all
+#   (scope `project-wide`); one pointing at a deleted config keeps its id with an empty
+#   `config_name` -- a finding, not an error. Create/delete are deliberately NOT exposed.
+
 kbagent context
 kbagent init [--from-global] [--project ALIAS ...]
 # `--project ALIAS` (repeatable) copies only the named project(s) from the global config and implies --from-global.

plugins/kbagent/skills/kbagent/SKILL.md

@@ -220,6 +220,7 @@ When working inside a git repository or project directory, run `kbagent init` (o
 | List cron schedules (keboola.scheduler configs) across projects | `kbagent schedule list` |
 | Show full detail for a single cron schedule | `kbagent schedule detail --project PROJECT --schedule-id SCHEDULE-ID` |
 | Audit schedules by cron window or job-freshness | `kbagent schedule find` |
+| List Flow Notification subscriptions (the Notifications tab) across projects | `kbagent notification list` |
 | List development branches from connected projects | `kbagent branch list` |
 | Create a new development branch and auto-activate it | `kbagent branch create --project PROJECT --name NAME` |
 | Set an existing development branch as active | `kbagent branch use --project PROJECT --branch BRANCH` |

plugins/kbagent/skills/kbagent/references/commands-reference.md

@@ -105,6 +105,9 @@ The `permissions` subcommands persist a write/destructive policy to config.json
 ## Billing (PAYG Credits) (since v0.84.2)
 - `billing credits [--project ALIAS ...]` -- read-only PAYG credit balance (`GET /credits` on `billing.{stack}`, plain Storage token). Fans out across all registered projects in parallel by default; `--project` (repeatable) narrows. Per-project failures degrade individually and are collected in `errors`, never abort the run. A project without the `pay-as-you-go` `owner.features` flag never calls the billing host (NXDOMAIN on some non-PAYG stacks) -- it gets an `error_code: PAYG_NOT_AVAILABLE` entry instead. `--json` emits `{"credits": [...], "errors": [...]}`. Rows carry the API's native unit (`consumed`/`remaining` credits) plus derived `*_minutes` fields (1 credit = 60 minutes, matching the Keboola UI). Gives the current balance only -- purchase history / Stripe invoice IDs are not reachable with a project token (issue #594 primary ask, still open; that data lives on `connection.{stack}` `/pay-as-you-go/billing/*`). See [billing-workflow.md](billing-workflow.md) for the full shape of the invoice-history gap and why it must not be worked around.
 
+## Flow Notifications (since v0.84.2)
+- `notification list [--project ALIAS ...] [--event NAME] [--component-id ID] [--config-id ID] [--branch ID]` -- fleet-wide audit of Flow Notification subscriptions, i.e. the Flow Builder **Notifications tab** (bell icon: Success / Error / Processing-delay cards). Backed by the Notification Service (`GET /project-subscriptions` on `notification.{stack}`) with a plain Storage token -- no `canManageTokens` or other elevated scope. **These are NOT in the flow's `configuration`**, so `flow detail` / `config detail` cannot show them; the in-flow `type: "notification"` task is a different mechanism and IS visible there. Multi-project fan-out in parallel by default; per-project failures land in `errors` and never abort the run. `--json` emits `{"subscriptions": [...], "errors": [...]}`; each row has `project_alias`, `subscription_id`, `event`, `scope` (`config` | `project-wide`), `component_id`, `config_id`, `config_name`, `branch_id`, `channel` (`email` | `webhook`), `address` (the email address OR the webhook URL), `expires_at`, and the raw `filters` list. Event names are **kebab-case**: `job-failed`, `job-succeeded`, `job-succeeded-with-warning`, `job-processing-long`, plus `phase-job-*` variants. `--event` is forwarded to the API; `--component-id`, `--config-id` and `--branch` match client-side against the subscription's own filter fields (`job.component.id`, `job.configuration.id`, `branch.id`). Read-only: create/delete subscriptions are deliberately not exposed. See [notification-workflow.md](notification-workflow.md).
+
 ## Feature Flags (since v0.48.0)
 Requires a **super-admin** Manage API token (same kind as `org setup`). Same default-deny token policy: interactive hidden prompt by default, or `--allow-env-manage-token` + `KBC_MANAGE_API_TOKEN` for CI. `--project ALIAS` resolves the stack URL (and, for project ops, the numeric `project_id`) from config -- the alias is the only handle you pass.
 - `feature list --project ALIAS` -- the stack-wide feature catalogue (`GET /manage/features`). Returns `{alias, stack_url, features: [{name, title, description, type, ...}]}`. Only `name` is a stable identifier; extra fields pass through unmodified.

plugins/kbagent/skills/kbagent/references/gotchas.md

@@ -3714,3 +3714,51 @@ mapping in favor of `changed_since: adaptive`, which tracks a
   assumption: here the empty state is the expensive, surprising path, and a
   seeded checkpoint is the conservative one. Do not assume "no state = safe
   default" when adaptive is involved.
+
+## `notification list`: the Notifications tab is a different mechanism from an in-flow notification task (since v0.84.2)
+
+`kbagent notification list` closes the one flow-notification surface no CLI
+could reach (#600). Everything below is behavior an agent gets wrong by
+default.
+
+- **Two unrelated mechanisms share the word "notification".** The Flow
+  Builder **Notifications tab** (bell icon: Success / Error /
+  Processing-delay cards) lives in the Notification Service, NOT in the
+  flow's `configuration` JSON -- `flow detail` and `config detail` cannot
+  see it, and never could. The in-flow **notification task** (a phase task
+  with `type: "notification"` and `recipients: [{channel, address}]`) lives
+  inside the configuration and IS visible through `flow detail`. Auditing
+  "who gets paged when this flow breaks" requires BOTH; reporting either one
+  alone silently under-reports.
+- **Event names are kebab-case.** `job-failed`, `job-succeeded`,
+  `job-succeeded-with-warning`, `job-processing-long`, plus the
+  `phase-job-*` variants. Not `jobFailed`. The service types `event` as a
+  free-form string rather than an enum, so kbagent does not restrict it --
+  an unknown value produces the API's own 400 rather than a client-side
+  rejection, and a newly shipped event type works immediately.
+- **The filter fields are dotted, not camelCase.** A subscription is bound
+  to a flow through `filters: [{field: "job.component.id", ...},
+  {field: "job.configuration.id", ...}]` -- not `component` /
+  `configurationId`. `--component-id` and `--config-id` match against those
+  client-side (only `--event` is served API-side).
+- **A subscription with no config filter is the catch-all, not a broken
+  row.** It fires for every job in the project and is reported as
+  `scope: "project-wide"` with an empty `config_id`. This is usually the
+  most important row in an audit -- do not filter it out as noise.
+- **A row whose `config_name` is empty but `config_id` is set points at a
+  deleted configuration.** That is a finding (a subscription paging someone
+  about a flow that no longer exists), never an error to retry.
+- **The endpoint is NOT branch-scoped.** It answers with every branch's
+  subscriptions at once; a dev-branch one carries a `branch.id` filter,
+  a production one carries none. `--branch` filters client-side and, unlike
+  every other branch-aware command, is **never** inferred from the
+  project's active branch -- inheriting it would hide exactly the
+  production recipients the audit is checking. A `--branch` value is
+  meaningful in one project only, so it requires exactly one `--project`
+  (exit 2 otherwise).
+- **Read-only by construction.** The service also exposes create/delete for
+  subscriptions -- changing who gets paged when production breaks -- and
+  kbagent deliberately exposes neither. Do not suggest kbagent can add or
+  remove a recipient; that is still a UI (or direct API) operation.
+- **Plain Storage token, no elevated scope.** The read path needs no
+  `canManageTokens` and no manage token.

plugins/kbagent/skills/kbagent/references/notification-workflow.md

@@ -0,0 +1,81 @@
+# Flow Notification audit workflow (`kbagent notification list`)
+
+Answers the fleet-wide question **"who gets paged when a production flow
+breaks, and are those recipients still valid?"** -- across every registered
+project, in one command.
+
+Available since **v0.84.2** (issue #600). Read-only.
+
+## Why this command exists
+
+Auditing flow notifications used to be only half-possible from a CLI. Three
+of the four surfaces were already reachable:
+
+| Surface | Where it lives | Reachable before v0.84.2 |
+|---|---|---|
+| Owner / contact emails in flow descriptions | flow `description` | yes (`flow list`, `config detail`) |
+| In-flow notification **task** (`type: "notification"`) | the flow's `configuration` JSON | yes (`flow detail`) |
+| Email-sending component configs (e.g. `kds-team.app-email-smtp-sender`) | component configs | yes (`config search`) |
+| **Notifications tab** (bell icon: Success / Error / Processing-delay) | **Notification Service** | **no -- UI only** |
+
+The last row is the one that actually pages a human when a production flow
+fails, and it was the only one that required opening each flow in the web UI
+by hand. On the 20-project / 276-flow fleet that prompted the issue, that was
+the entire cost of the audit.
+
+## The command
+
+```bash
+# Every subscription, every registered project
+kbagent notification list
+
+# One project, only failures
+kbagent notification list --project prod --event job-failed
+
+# Everything pointed at one specific flow
+kbagent notification list --project prod --component-id keboola.flow --config-id 9001
+
+# Machine-readable, for joining against your own inventory
+kbagent --json notification list > subscriptions.json
+```
+
+Rows carry `project_alias`, `subscription_id`, `event`, `scope`,
+`component_id`, `config_id`, `config_name`, `branch_id`, `channel`,
+`address`, `expires_at`, and the raw `filters` list.
+
+## Reading the output
+
+- **`scope: "project-wide"`** -- no config filter: the subscription fires for
+  every job in the project. The catch-all "tell me about any failure". Often
+  the most important row; never noise.
+- **`config_name` empty while `config_id` is set** -- the subscription points
+  at a configuration that no longer exists. A dangling recipient, i.e. a
+  finding.
+- **`branch_id` set** -- the subscription is filtered to a dev branch.
+  Production subscriptions carry no branch filter. The endpoint is not
+  branch-scoped, so both come back together unless you pass `--branch`.
+- **`channel: "webhook"`** -- `address` holds the webhook URL rather than an
+  email address; the two share a column because both answer "where does this
+  go".
+
+## A complete audit
+
+1. `kbagent --json notification list > tab.json` -- the Notifications tab.
+2. `kbagent --json flow list` + `flow detail` -- the in-flow notification
+   **tasks** (a different mechanism; see gotchas.md).
+3. `kbagent config search --query "@"` -- addresses hiding in descriptions
+   and in email-sender component configs.
+4. Join all three against your directory of valid addresses. Typical
+   findings: placeholder addresses that were never replaced, recipients who
+   have left, flows with no owner at all, and subscriptions surviving the
+   flow they watched.
+
+## What this command will not do
+
+- **It cannot add or remove a recipient.** The Notification Service has
+  create/delete endpoints; kbagent deliberately wraps neither, at every layer
+  (the HTTP dispatcher takes no method argument). Changing who gets paged
+  stays a deliberate UI/API action.
+- **It does not read the in-flow notification task.** That is `flow detail`.
+- **It needs no elevated token.** The read path works with the same plain
+  Storage token every other project command uses.

src/keboola_agent_cli/changelog.py

@@ -25,6 +25,45 @@
 # Ordered newest-first.  Each value is a list of brief one-line descriptions.
 CHANGELOG: dict[str, list[str]] = {
     "0.84.2": [
+        "New: `kbagent notification list` audits Flow Notification subscriptions across "
+        "the whole fleet (closes #600). `[--project ALIAS ...] [--event NAME] "
+        "[--component-id ID] [--config-id ID] [--branch ID]`. The Flow Builder "
+        "**Notifications tab** -- the bell icon's Success / Error / Processing-delay cards -- "
+        "is backed by the Notification Service, not by the flow's `configuration`, so "
+        "`flow detail` and `config detail` never could show it. It was the one notification "
+        "surface with no CLI path at all: on the 20-project / 276-flow fleet that prompted "
+        "the issue, every other surface (owner emails in descriptions, in-flow "
+        '`type: "notification"` tasks, email-sender component configs) was auditable in '
+        "minutes, while the recipients that actually page someone when production breaks "
+        "required opening each flow in the UI by hand. Reads `GET /project-subscriptions` on "
+        "the derived `notification.{stack}` host with a plain Storage token (no elevated "
+        "scope), fans out across every registered project in parallel, and collects "
+        "per-project failures in `errors` instead of aborting the run. `--json` emits "
+        "`{subscriptions: [...], errors: [...]}`; each row carries project_alias, "
+        "subscription_id, event, scope, component_id, config_id, config_name, branch_id, "
+        "channel, address, expires_at and the raw filters. Read-only by construction: the "
+        "service's create/delete endpoints change who gets paged when production breaks, and "
+        "the HTTP dispatcher takes no method argument, so no future caller can reach them "
+        "through it. New surfaces: `client/notifications.py`, "
+        "`services/notification_service.py`, `commands/notification.py`, "
+        "`GET /notifications` on `kbagent serve`, permission `notification.list = read`.",
+        "Note: three details of the notification contract are easy to guess wrong. "
+        "The implementation follows the service's own OpenAPI, not the guess. "
+        "Event names are KEBAB-case (`job-failed`, `job-succeeded-with-warning`, "
+        "`job-processing-long`, `phase-job-*`), and `event` is typed as a free-form string "
+        "rather than an enum -- so no client-side allow-list is imposed and a newly shipped "
+        "event type works the day Keboola ships it. Subscriptions bind to a flow through "
+        "dotted filter fields (`job.component.id`, `job.configuration.id`, `branch.id`), not "
+        "`component`/`configurationId`. A recipient is a discriminated union: `email` carries "
+        "`address`, `webhook` carries `url` -- both render in one `address` column with "
+        "`channel` alongside. Two shapes that would otherwise read as broken rows are "
+        "reported honestly instead: a subscription with no config filter is the catch-all "
+        "(`scope: project-wide`, empty config_id), and one pointing at a deleted "
+        "configuration keeps its id with an empty `config_name` -- a finding, not an error. "
+        "The endpoint is NOT branch-scoped: it answers with every branch's subscriptions, and "
+        "`--branch` filters client-side and is NEVER inferred from the project's active "
+        "branch, because inheriting it would silently hide the production recipients the "
+        "audit exists to check.",
         "New: `kbagent config clone` duplicates a configuration WHOLE (closes #587). "
         "`--project P --component-id C --config-id ID --name N [--target-project P2] "
         "[--set PATH=VALUE ...] [--secret PATH=VALUE ...] [--dry-run]`. Until now there was "

src/keboola_agent_cli/cli.py

@@ -26,6 +26,7 @@
 from .commands.job import job_app
 from .commands.kai import kai_app
 from .commands.lineage import lineage_app
+from .commands.notification import notification_app
 from .commands.org import org_app
 from .commands.permissions import permissions_app
 from .commands.project import project_app
@@ -69,6 +70,7 @@
 from .services.lineage_service import LineageService
 from .services.mcp_service import McpService
 from .services.member_service import MemberService
+from .services.notification_service import NotificationService
 from .services.org_service import OrgService
 from .services.project_service import ProjectService
 from .services.repo_validate_service import RepoValidateService
@@ -140,6 +142,7 @@
 _FLOWS = "Flows"
 app.add_typer(flow_app, name="flow", rich_help_panel=_FLOWS)
 app.add_typer(schedule_app, name="schedule", rich_help_panel=_FLOWS)
+app.add_typer(notification_app, name="notification", rich_help_panel=_FLOWS)
 
 # -- Development --
 _DEV = "Development"
@@ -346,6 +349,7 @@ def main(
     encrypt_service = EncryptService(config_store=config_store)
     flow_service = FlowService(config_store=config_store)
     schedule_service = ScheduleService(config_store=config_store)
+    notification_service = NotificationService(config_store=config_store)
     workspace_service = WorkspaceService(config_store=config_store)
     data_app_service = DataAppService(config_store=config_store)
     data_app_git_service = DataAppGitService(config_store=config_store)
@@ -406,6 +410,7 @@ def main(
     ctx.obj["encrypt_service"] = encrypt_service
     ctx.obj["flow_service"] = flow_service
     ctx.obj["schedule_service"] = schedule_service
+    ctx.obj["notification_service"] = notification_service
     ctx.obj["workspace_service"] = workspace_service
     ctx.obj["data_app_service"] = data_app_service
     ctx.obj["data_app_git_service"] = data_app_git_service

src/keboola_agent_cli/client/_client.py

@@ -17,6 +17,7 @@
 from .branches import _BranchesMixin
 from .configs import _ConfigsMixin
 from .misc import _MiscMixin
+from .notifications import _NotificationMixin
 from .query import _QueryMixin
 from .queue import _QueueMixin
 from .storage_files import _StorageFilesMixin
@@ -37,6 +38,7 @@ class KeboolaClient(
     _QueryMixin,
     _WorkspacesMixin,
     _BillingMixin,
+    _NotificationMixin,
     _MiscMixin,
     _CoreClient,
 ):