Skip to content

docs: add operator runbooks (outage, catalog, backup restore, admin revoke) - #87

Merged
tibroc merged 1 commit into
mainfrom
worktree-agent-aa8e1b062ed276a18
Aug 28, 2026
Merged

docs: add operator runbooks (outage, catalog, backup restore, admin revoke)#87
tibroc merged 1 commit into
mainfrom
worktree-agent-aa8e1b062ed276a18

Conversation

@tibroc

@tibroc tibroc commented Aug 28, 2026

Copy link
Copy Markdown
Member

Summary

Adds the four operator runbooks named in docs/04-development-plan.md §5, each grounded in the actual service-layer/handler/MCP code (not the spec's aspirational description):

  • docs/runbooks/outage-announcement.md
  • docs/runbooks/manage-service.md
  • docs/runbooks/restore-postgres.md
  • docs/runbooks/revoke-admin.md

Linked from a new README "Operations" section and from the docs/04 §5 "Runbooks:" bullet.

Gaps found while writing these (flagging per CLAUDE.md, not silently working around them)

  1. No MCP tool exists for announcements. The admin MCP server (cmd/mcp) only exposes service.*/category.*/search.insights plus the propose_*/change.confirm staged-write flow for catalog services. Announcements can only be created/edited/deleted via the admin web form / HTTP API (POST|PATCH|DELETE /api/admin/announcements*). The runbook documents the HTTP-API fallback for scripting this, but there's no staged propose→confirm path for announcements today.

  2. No reactivate/undelete path for a soft-deleted catalog service. DELETE /api/admin/services/{id} and service.propose_deletechange.confirm both set is_active = false; there is no HTTP endpoint, MCP tool, or form action to flip it back, and editing an inactive service (UpdateService) never touches is_active. The only way back is to re-create the service as a new row (new id — breaks any favorites/role-defaults referencing the old one) or hand-edit the DB.

  3. No automated backup job exists in the repo. Searched scripts/, .github/workflows/, deploy/, both compose files, and the Makefile for pg_dump/pg_basebackup/cron/backup automation — none exists. The restore runbook documents restoring from whatever pg_dump artifact the operator already has; it does not (and cannot) document a "tested restore" of an automated backup, because there is no automated backup to test. This is a real product gap against docs/04 §5's "regular Postgres backups + a tested restore" aspiration.

  4. No admin-facing way to force-end a specific user's wolke session(s). DeleteSessionsBySID/DeleteSessionsByOIDCSub exist in internal/store but are only ever called from the OIDC back-channel-logout handler — there's no HTTP/MCP admin action to invoke them directly. The revoke-admin runbook's "immediate lockout" path therefore either relies on the IdP having back-channel logout registered (documented in docs/oidc-keycloak.md) and triggering a logout there, or falls back to a direct DELETE FROM sessions ... WHERE oidc_sub = ... via psql. An admin-facing "kill this user's sessions" action would close this gap.

Test plan

  • Read the actual service/handler/MCP/auth code for announcements, catalog services, audit log, and back-channel logout before writing any step (no invented UI copy or field names).
  • N/A — documentation only, no code changes; nothing to run beyond markdown review.

Ground each runbook in the actual service-layer/handler/MCP code rather
than the spec's aspirational description: announcements have no MCP
tool at all (form/HTTP API only), catalog service soft-delete has no
reactivate path, session revocation for a compromised admin needs
either registered back-channel logout or a direct DB session delete
since there's no admin-facing "kill this user's sessions" endpoint,
and there's no backup automation in the repo yet so the restore
runbook documents restoring from a manually-taken pg_dump.

Link the four runbooks from README's new Operations section and from
the docs/04 §5 Runbooks bullet.
@tibroc
tibroc merged commit a4f041e into main Aug 28, 2026
5 checks passed
@tibroc
tibroc deleted the worktree-agent-aa8e1b062ed276a18 branch August 28, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant