Skip to content

feat(telemetry): opt-in anonymous boot and learning-funnel events - #94

Merged
OthmaneZ05 merged 3 commits into
mainfrom
feat/opt-in-telemetry
Aug 25, 2026
Merged

feat(telemetry): opt-in anonymous boot and learning-funnel events#94
OthmaneZ05 merged 3 commits into
mainfrom
feat/opt-in-telemetry

Conversation

@OthmaneZ05

@OthmaneZ05 OthmaneZ05 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary of Changes

Adds strictly opt-in, anonymous product telemetry, sent as Plausible-format custom events to a build-time-configurable endpoint (VITE_TELEMETRY_ENDPOINT / VITE_TELEMETRY_DOMAIN; an empty endpoint hard-disables telemetry entirely). Eleven events, mirrored verbatim in the README's Telemetry table:

  • Boot funnel — where a first run most often dies (Docker Desktop not started, WSL2, socket permissions, image downloads): app_started (once per page load), then one Docker readiness probe reported as runtime_check_startedruntime_ready | runtime_failed { reason }. GET /health now returns a stable, non-identifying checks.docker.reason (socket_not_found, permission_denied, connection_refused, timeout, unknown); the client adds backend_unreachable and timeout (10 s cap on the probe). image_pull_failed { node } fires when a node can't be created because its image failed to download (backend IMAGE_NOT_FOUND). first_validator_run { roadmap, step } fires once per install, on the first validation ever run.
  • Learning funnel: roadmap_started, step_validated, step_failed, roadmap_completed, roadmap_abandoned.
  • Consent is tri-state (unset / accepted / declined, localStorage key torollo_telemetry_consent): anything other than an explicit accept means zero network requests — not even the boot health probe. A one-time, non-blocking consent card on the home screen asks the question; a header toggle makes the choice reversible at any time. Accepting mid-session fires the boot funnel right away.
  • Anonymous by construction: every event is a typed entry of TelemetryEvents — props are catalogue ids or closed enums only, plus the app version and a random locally-generated install id. No socket paths, no error messages, no project names. Revoking consent deletes the install id and the "first time" markers tied to it, so re-enabling starts a fresh identity.
  • Honest counting: step_failed only counts pedagogical failures (engine error results are not counted); roadmap_started only fires when the backend progress store is confirmed empty; roadmap_completed fires once, on the validation that completes the roadmap; roadmap_abandoned fires once per play-through on close/unmount/pagehide (with keepalive).
  • Small shared refactors: probeDockerHealth() extracted from useDockerHealth (with the timeout, so the briefing page's Docker check can no longer hang), readApiError() next to readErrorMessage() to surface the backend's error code.
  • New README Telemetry section documents the exact event list, per-event props, payload contents, and how to decline/revoke.

Types of Changes

  • New feature / node type addition
  • Bug fix (non-breaking change resolving an issue)
  • Refactoring / structural cleanup
  • Documentation update

Verification & Testing

Automated Checks

  • Run npm run lint successfully with no errors
  • Run npm run build successfully with no compilation errors
  • Run npm test successfully (all tests pass)

Backend 456 tests (new: classifyDaemonFailure matrix, /health reason contract). Frontend 340 tests, new suites: consent store (tri-state, install-id + milestone lifecycle, storage-disabled fallback), trackEvent (zero fetch when declined/unset, exact prop allowlist per event, URL routing, keepalive passthrough, never throws), claimMilestone, probeDockerHealth (ok / reason passthrough / unknown reason / backend unreachable / timeout), useBootTelemetry (nothing without consent — not even the probe; fires once per page load; fires on mid-session accept), useContainers (IMAGE_NOT_FOUNDimage_pull_failed, other codes don't), and the player hook wiring (first_validator_run once per install, not on a failed request, plus the five roadmap events with their negatives).

Manual Verification

E2E against the real backend + real Docker, with a local Plausible-shaped capture server and Playwright driving the UI, across three backend states (23/23 checks):

  • Docker running: consent unset → zero requests on the wire and no /health probe; accepting on the card → app_started + runtime_check_started + runtime_ready; consent already accepted → the same three on load, app_started exactly once (StrictMode); opening a roadmap and validating for real → first_validator_run (roadmap + step ids only) before the verdict event, a second Validate doesn't repeat it, the milestone is persisted, a reload replays the boot funnel but not the milestone.
  • Dead Docker socket (DOCKER_HOST pointing at a missing socket): /health answers 503 with reason: socket_not_found; the app reports runtime_failed { reason: 'socket_not_found' }.
  • No backend at all: runtime_failed { reason: 'backend_unreachable' }.
  • Payload audit over every captured event: names and prop sets match the README table exactly, URLs are the app root or a roadmap path, configured domain honoured, no PII / paths / raw error strings anywhere.
  • Learning-funnel events re-verified in the same runs (started, step failed, abandoned with the right step) on top of the earlier 20/20 run (real pass, completion, no-restart-on-resume, no-abandon-after-completion).
  • image_pull_failed is covered by tests on both sides of the contract (backend 502 IMAGE_NOT_FOUND on a failed pull; client code → event); a live repro would require removing a host image, so it was not run against real Docker.
  • Consent card and toggle verified visually in light and dark themes.

Checklist

  • My code follows the repository's code style and lint standards
  • I have updated the documentation or instructions if necessary
  • All unit and integration tests are passing

🤖 Generated with Claude Code

@OthmaneZ05
OthmaneZ05 requested a review from Derssa as a code owner August 20, 2026 15:32
@OthmaneZ05 OthmaneZ05 changed the title feat(telemetry): opt-in anonymous learning-funnel events feat(telemetry): opt-in anonymous boot and learning-funnel events Aug 25, 2026
@OthmaneZ05
OthmaneZ05 merged commit 003ac1a into main Aug 25, 2026
4 checks passed
@OthmaneZ05
OthmaneZ05 deleted the feat/opt-in-telemetry branch August 25, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant