docs: refresh AI handover for current repository state

AI_HANDOVER.md

@@ -2,99 +2,113 @@
 
 > Quickstart for any engineer or AI agent continuing this project **from GitHub alone**.
 > The detailed, living handover is [`docs/PROJECT_STATE.md`](docs/PROJECT_STATE.md) — read it
-> next. This file is the 60-second orientation and the guardrails.
+> next. This file provides the 60-second orientation and enduring architectural guardrails.
 
 ## What this is
 
-*Gambit* — an AGPL-3.0 open-source chess platform (feature parity with Lichess/Chess.com plus
-a first-class AI layer), built as an npm-workspaces monorepo of **strict-TypeScript,
-dependency-free domain packages** tested with the built-in `node --test` runner.
+*Gambit* — an AGPL-3.0-or-later open-source chess platform (targeting feature parity with
+Lichess and Chess.com, plus a first-class AI layer), built as an npm-workspaces monorepo
+(`packages/*`), with deployable services (`services/gateway`) wired alongside. It maintains
+a strict boundary between **strict-TypeScript, zero-external-dependency domain packages**
+(rules, game authority, tournaments, social, search, learning) and **infrastructure adapters**
+(`pg`, `ioredis`, `ws`, external AI SDKs) wired at application entry points. All package suites
+execute via the built-in `node --test` runner.
 
 ## Where things are
 
-- `docs/ARCHITECTURE.md` — the target system architecture (the design everything builds toward).
-- `docs/ROADMAP.md` — milestones (M1–M14) with explicit acceptance criteria; ✅/🚧/⬜ status.
-- `docs/PROJECT_STATE.md` — the **living handover**: what's done, how it's built, decisions,
-  deferrals, and the exact next step. Update it after every milestone.
-- `docs/DATABASE.md` + `docs/adr/*` — the approved data contract and Architecture Decision Records.
-- `docs/RUNNING.md` — the one-command local stack (`docker compose up`).
-- `docs/DEPLOYING.md` — the Kubernetes/Helm deployment flow (`deploy/helm/gambit`).
-- `packages/*` — the domain/service packages. `services/gateway` — the deployable realtime
-  gateway binary (infra adapters: Postgres event log, Redis pub/sub, `ws`).
-
-## Current status (2026-07-18)
-
-| Milestone | Package(s) | Status | Tests |
-|---|---|---|---|
-| M1 | `@chess-platform/core` | ✅ rules engine (perft-verified) | 16 |
-| M2 | `@chess-platform/game` | ✅ event-sourced game aggregate + clocks + threefold repetition | 25 |
-| M3 | `@chess-platform/realtime-gateway` | ✅ realtime WS edge + token auth + durable `EventLog` port + `PubSub` (in-memory & Redis) | 61 |
-| M4a | `@chess-platform/persistence` | ✅ durable event store + repositories + Glicko-2 | 20 (6 DB-gated) |
-| M4b | `@chess-platform/api` | ✅ stateless REST + identity (scrypt, rotating refresh, RBAC, password reset + email verification — ADR-0026) | 128 (3 DB-gated) |
-| M5 | `@chess-platform/engine` | ✅ provider-agnostic UCI engine bridge | 50 |
-| M6 | `@chess-platform/web` + `@chess-platform/e2e-harness` | ✅ playable frontend; Playwright full-game e2e + Lighthouse a11y ≥ 0.95 in CI | 260 + 4 |
-| M7 | `@chess-platform/ai-orchestrator` | ✅ AI routing/failover/caching + engine-grounded prompts | 117 (2 key-gated) |
-| M8 | `@chess-platform/ai-features` | ✅ 9 features (Move Explanation → Tournament Commentator) | 140 (16 key-gated) |
-| M9 | `@chess-platform/tournament` (+ api/realtime integration) | ✅ round-robin, Swiss, and Arena formats; persistence + REST API; durable game launcher; realtime result recording (production reporter hosted by the gateway behind `TOURNAMENT_REPORTER=1`, with optimistic-concurrency CAS on tournament saves — inc 13, ADR-0025); live broadcast (ADR-0014 → ADR-0025) | 49 |
-| M12 | api security & anti-cheat | 🚧 **increments 1–4 complete:** CORS + security headers (ADR-0011) · httpOnly refresh cookie (ADR-0012) · auth rate limiting w/ Postgres buckets (ADR-0013) · **anti-cheat inc 1–4:** analyzer (ADR-0029) + cross-game aggregation (ADR-0030) + engine adapter (ADR-0031) + service/repo (ADR-0032) | 31 (anti-cheat) |
-| M13 | `@chess-platform/api` + `services/gateway` | 🚧 **inc 1 complete:** dependency-free `Logger`/`Metrics` ports + `traceparent` correlation; structured JSON logs, Prometheus `/v1/metrics` & gateway `/metrics`, real readiness; PII/cardinality-safe (ADR-0028) | — |
-| **M14** | compose + `services/gateway` + `deploy/helm` | 🚧 **increments 1–4 complete:** local compose stack · durable game authority (write-through `EventLog` → Postgres, evict/rehydrate) · Redis pub/sub multi-node fanout (ADR-0008) · Helm chart + kubeconform CI gate (ADR-0009) · threefold-repetition fix (en-passant legality in repetition key) | 4 (Redis-gated) |
-
-**Whole repo: 960 total tests, 0 failures** (skips: 33 = 8 Postgres-gated + 21
-API-key-gated + 4 Redis-gated — run `npm run test:counts` for the live per-package
-breakdown). Strict TS, lint clean. **CI is active** (`.github/workflows/ci.yml`, 6 jobs:
-build+typecheck+test on Node 22/24, Postgres integration, M6 Playwright+Lighthouse acceptance,
-helm lint+kubeconform, gateway service).
-
-M10 and M11 (social/learning + GraphQL, search) are ⬜ planned; M13 observability inc 1 is
-done (metrics/tracing export is a later increment); M12's anti-cheat half is also still ahead —
-see the ROADMAP.
-
-## Build & test
+- `docs/ARCHITECTURE.md` — target system architecture and core design principles.
+- `docs/ROADMAP.md` — milestone definitions and acceptance criteria.
+- `docs/PROJECT_STATE.md` — the **authoritative living handover**: active increment progress
+  (including M15 hardening), architectural decisions, active defect tracking, and exact next steps.
+  Updated after every increment.
+- `docs/DATABASE.md` + `docs/adr/*` — approved data contracts and Architecture Decision Records.
+- `docs/RUNNING.md` — local development stack (`docker compose up`).
+- `docs/DEPLOYING.md` — Kubernetes / Helm deployment architecture (`deploy/helm/gambit`).
+- `packages/*` — domain packages and service libraries (root npm workspaces).
+- `services/gateway` — deployable realtime WebSocket gateway binary (wiring Postgres event log,
+  Redis pub/sub, and connection management; standalone service outside root workspaces).
+
+## Architecture & Milestone Overview
+
+Milestone progress spans foundational core engines through active production hardening:
+
+- **Foundations (M1–M9)**:
+  - Rules engine (`@chess-platform/core` with perft verification across all eight supported variants, including standard chess).
+  - Event-sourced game aggregate and deterministic clock model (`@chess-platform/game`).
+  - Realtime edge gateway with token authentication and pub/sub ports (`@chess-platform/realtime-gateway`).
+  - PostgreSQL persistence, event store, and Glicko-2 ratings (`@chess-platform/persistence`).
+  - Stateless REST API, identity, and RBAC (`@chess-platform/api`).
+  - Provider-agnostic UCI engine bridge (`@chess-platform/engine`, with pools for Stockfish and Fairy-Stockfish, persistent Postgres analysis cache seam, and priority scheduling; remote distributed engine workers remain deferred).
+  - Playable web client with Playwright e2e and accessibility standards (`@chess-platform/web`).
+  - Pluggable AI routing, caching, and evaluation features (`@chess-platform/ai-orchestrator`, `@chess-platform/ai-features`).
+  - Tournaments (round-robin, Swiss, Arena), game launcher, and live broadcast (`@chess-platform/tournament`).
+- **Feature & Platform Expansions (M10–M14)**:
+  - **M10 (Social & learning)**: Social graph, 1:1 messaging, teams/communities, forums, achievements, collaborative studies, interactive lessons, and read-only GraphQL read layer.
+  - **M11 (Search)**: Keyword search, full-text Postgres indexing, live game indexing worker, natural-language normalization, pgvector semantic search (HNSW cosine similarity), and hybrid search (RRF).
+  - **M12 (Security & anti-cheat)**: CORS, security headers, httpOnly cookies, durable rate limiting, pure-domain engine-correlation anti-cheat scoring, behavioral bot detection, gateway hosting, and STRIDE pen-test audit (closed).
+  - **M13 (Observability & SRE)**: Dependency-free structured logger, metrics (Prometheus `/v1/metrics`), W3C `traceparent` propagation, OTLP tracing pipeline with batch processor and self-instrumentation, alert rules, Grafana dashboards, runbooks, and SLO definitions (closed).
+  - **M14 (Deployment & scale)**: Local multi-service Compose stack, multi-node gateway scaling with Redis ownership and command forwarding (ADR-0010), Helm chart with kubeconform CI validation (ADR-0009), External Secrets Operator integration (ADR-0044), blue/green and canary delivery (ADR-0075), automated deploy gates (ADR-0076), load and chaos test baselines (ADR-0065, ADR-0077, ADR-0078, ADR-0111), and comprehensive web UI surfaces for account security (WebAuthn passkeys, session visibility/revocation, password recovery, email verification).
+- **Active Milestone — M15 (Productionization & Hardening)**:
+  - M15 is active production hardening, reliability, integration correctness, test/database isolation, and unresolved-defect closure.
+  - See [`docs/PROJECT_STATE.md`](docs/PROJECT_STATE.md) for the active increment log, current defects, and immediate next steps.
+
+## Test Verification & CI
+
+- **Suite execution**: Run `npm test` across all root workspaces via `node --test`.
+- **Live metrics**: `npm run test:counts` aggregates test and skip counts across package and service suites; consult [`docs/PROJECT_STATE.md`](docs/PROJECT_STATE.md) for current tooling caveats and measured validation status.
+- **Hermetic defaults**: Unit and domain suites run offline without external infrastructure. Integration suites gate on environment variables: `DATABASE_URL` (PostgreSQL persistence), `REDIS_URL` (gateway Redis integration and scaling), and provider API keys (live AI features).
+- **Continuous Integration (`.github/workflows`)**:
+  - Multi-version Node matrix (Node 22 / 24) with strict typechecking, linting, and ADR claim validation (`npm run check:adr-claims`).
+  - Path-filtered workflows for efficient change detection (`changes` job).
+  - Real PostgreSQL integration suite for schema migrations, persistence repositories, and API pg-security tests (`postgres-integration`).
+  - Real UCI engine analysis smoke testing with pinned Stockfish and Fairy-Stockfish binaries (`analysis-smoke`).
+  - End-to-end acceptance testing with Playwright browser tests and Lighthouse accessibility scoring (`m6-acceptance`).
+  - Production Docker container image build and template verification (`docker-images`).
+  - Helm chart linting and kubeconform schema validation against Kubernetes schemas (`helm`).
+  - Parity and hygiene guards: CI parity (`check:ci-parity`), variant parity (`check:variant-parity`), engine pin parity (`check:engine-pin-parity`), and observability drift (`check:observability`).
+  - Workflow triggers: on-demand deployment (`deploy.yml`) on workflow dispatch or published release, on-demand chaos engineering (`chaos.yml`) on workflow dispatch, and release container image publishing (`release.yml`) on version-tag pushes (`v*`).
+
+## Build & Run
 
 ```bash
-npm ci          # reproducible install (root package-lock.json is committed)
-npm run build   # dependency order: core → game → realtime-gateway → persistence → api → engine → web → e2e-harness → ai-*
-npm test        # all package suites via node --test
-npm run lint    # strict typecheck across packages
+npm ci               # reproducible install (root package-lock.json is committed)
+npm run build        # builds all root workspace packages in dependency order
+npm test             # executes workspace package test suites via node --test
+npm run lint         # strict typecheck across all root workspaces
+npm run test:counts  # aggregates test and skip counts across suites (see PROJECT_STATE caveats)
 ```
-Run **build before lint/test** on a fresh clone — downstream packages resolve upstream types
-from built `dist/`. Postgres-gated tests need `DATABASE_URL`; AI-adapter integration tests need
-provider API keys; everything else is hermetic. Local full stack: `docker compose up --build`
-(see RUNNING.md). Helm chart checks: `helm lint deploy/helm/gambit`,
-`helm template deploy/helm/gambit | kubeconform -strict -summary`,
-`bash scripts/helm-snapshot-test.sh`.
-
-## Working method (do not skip)
-
-Every milestone: **build to explicit acceptance criteria with tests → self-critique loop →
-multi-perspective review (distributed-systems, performance, security, chess-server maintainer)
-→ refactor → document → commit → push.** Advance only when clean — run the full
-`npm ci && npm run build && npm test && npm run lint` gate before reporting done.
-Architectural decisions that introduce a durable/shared contract get a **gate** (a design doc +
-ADR, approved before code) — see `DATABASE.md` (M4), `ENGINE_BRIDGE.md` (M5), ADR-0008/0009 (M14).
-
-## Guardrails
-
-- **Gateway horizontal scaling requires Redis.** Since M14 inc 5 (ADR-0010) the gateway is
-  safe to scale (single-owner authority + Redis command forwarding; the Helm chart defaults to
-  `gateway.replicas: 2`), but ONLY with `REDIS_URL` set — never scale beyond 1 replica without
-  Redis command routing.
-- Keep domain packages **dependency-free**; native/infra code (pg, ioredis, ws) enters only via
-  documented ports (`EventLog`, `PubSub`/`RedisLike`, `TokenVerifier`, `EngineTransport`) wired
-  in `services/gateway` or package `/pg`-style subpaths — never in domain code.
-- No placeholders, TODO-implementations, or temporary hacks — production quality only.
-- Keep GitHub authoritative: after each checkpoint update README/ROADMAP/PROJECT_STATE/this
-  file, then commit and push, so the next agent needs no conversation history.
-
-## Known tech debt (tracked, updated 2026-07-18)
-
-- **Identity hardening (M4 follow-up)** — WebAuthn/passkeys (table exists, flow doesn't). Password reset + email verification are complete. (Refresh-token storage moved to an httpOnly cookie in
-  M12 inc 2; auth rate limiting landed in M12 inc 3.)
-- **Tournament reporter refinements (ADR-0025)** — event-log catch-up for ended-broadcasts
-  missed before subscription; dedicated single-replica reporter Deployment. Arena
-  withdraw is permanent (pause/rejoin needs a domain decision + ADR).
-- **M14 remaining** — Terraform, CI/CD deploy gates (blue/green), load/chaos testing,
-  secrets management.
-
-Full details and the exact next step: `docs/PROJECT_STATE.md`.
+
+- **Build order**: Run **`npm run build` before `lint` or `test` on a fresh clone** — downstream packages resolve upstream types from built `dist/`.
+- **Local full stack**: `docker compose up --build` (see [`docs/RUNNING.md`](docs/RUNNING.md)).
+- **Helm chart validation**: `bash scripts/helm-snapshot-test.sh` (or lint with test secrets: `helm lint deploy/helm/gambit --set secrets.accessTokenSecret=test-only-access-token-secret-32-bytes-minimum --set secrets.postgresPassword=test-only-postgres-password --set config.nodeEnv=development --set email.provider=console`).
+
+## Working Method (Do Not Skip)
+
+Every increment: **build to explicit acceptance criteria with tests → self-critique loop → multi-perspective review (distributed-systems, performance, security, chess-server maintainer) → refactor → document → commit → push.**
+
+- **Validation gate**: Advance only when clean — run `npm run build && npm run lint && npm test` before reporting done.
+- **Architectural gates**: Decisions that introduce a durable or shared contract require an Architecture Decision Record (ADR) approved under `docs/adr/` before implementation.
+- **Documentation hygiene**: Update documentation (`README.md`, `docs/ROADMAP.md`, `docs/PROJECT_STATE.md`) at every milestone checkpoint.
+
+## Enduring Guardrails
+
+- **Gateway horizontal scaling requires Redis.** Since M14 inc 5 (ADR-0010) the gateway is safe to scale (single-owner authority + Redis command forwarding; Helm chart defaults to `gateway.replicas: 2`), but ONLY with `REDIS_URL` set — never scale beyond 1 replica without Redis command routing.
+- **Strict package boundaries and dependency-free domain core.** Keep domain packages (`core`, `game`, `tournament`, `social`, `messaging`, `community`, `achievements`, `studies`, `learning`, `search`, `anti-cheat`) strictly free of external runtime dependencies; native/infra code (`pg`, `ioredis`, `ws`, AI SDKs) enters only via documented ports (`EventLog`, `PubSub`/`RedisLike`, `TokenVerifier`, `EngineTransport`, repositories) wired in application bootstrap or package `/pg`-style subpaths — never in domain code. Pure domains must not read wall clocks directly (`Date.now()`); inject time/clock as a parameter.
+- **No placeholders or mock shortcuts.** No stubs, TODO implementations, or temporary hacks in production paths — production quality only.
+- **GitHub-authoritative workflow.** Keep GitHub authoritative: document changes in the repository, commit, and push so any engineer or AI agent continuing the work has full context without conversation history.
+
+## Tracked Tech Debt & Deferred Work
+
+Only actionable, genuinely open architectural debt is tracked here:
+
+- **Tournament reporter refinements (ADR-0025)**:
+  - Event-log catch-up for `EndedBroadcast`s missed between game end and first subscription.
+  - Dedicated single-replica reporter Deployment (currently hosted across gateway replicas with optimistic-concurrency CAS on tournament saves).
+  - Arena withdrawal is permanent by design (pause/rejoin requires an explicit domain decision and ADR).
+- **Scale & Infrastructure (M14 deferred)**:
+  - Terraform IaC for cloud provisioning.
+  - 100k-user cluster load testing (workstation-bounded k6 and chaos baselines are implemented).
+- **Active defects and investigations**:
+  - For active defects and investigations (including Node test runner Signature B diagnostic tracking, isolated fresh-database suite dependencies, and gateway test tooling caveats), consult [`docs/PROJECT_STATE.md`](docs/PROJECT_STATE.md) as the single authoritative source of truth.
+
+For full project state, increment details, and the immediate next task, see [`docs/PROJECT_STATE.md`](docs/PROJECT_STATE.md).