Skip to content

feat(analytics): anonymous storefront telemetry for the shopper funnel - #47

Merged
PhilippTheServer merged 1 commit into
OpenTaberna:mainfrom
PhilippTheServer:pl/storefront-analytics
Aug 26, 2026
Merged

PhilippTheServer merged 1 commit into
OpenTaberna:mainfrom
PhilippTheServer:pl/storefront-analytics

Conversation

@PhilippTheServer

Copy link
Copy Markdown
Contributor

Closes #46 · S2 of the telemetry programme

The order funnel starts at order creation, so it answers "how many orders were paid" but not
"how many people looked and left". This adds the part before the order.

POST /v1/analytics/events            public, rate limited, opt-in
GET  /v1/admin/analytics/storefront  sessions → product views → carts → checkouts → paid

Privacy is structural, not a policy

The table has no column that could hold PII, and a test fails if one appears:

forbidden = {"ip", "ip_address", "remote_addr", "user_agent", "email",
             "customer_id", "keycloak_user_id", "user_id", "first_name", "last_name"}
assert not (columns & forbidden)

The request schema uses extra="forbid", so a client sending email gets a 422 rather
than having it quietly dropped — silently discarding it would let a frontend believe it was
collecting something it was not, and nobody would find out.

Query strings are stripped before storage. That is where personal data arrives by accident,
and removing it at the boundary means it cannot be stored even if sent:

/shop?utm_source=mail&email=someone@example.com  →  /shop

Because nothing identifies anyone and nothing is kept in the browser beyond a per-tab
session id, this needs no consent banner in the EU. That is the point of the shape — a
banner costs 40–60% of sessions to opt-outs, which would make the funnel it feeds mostly
fiction.

Off unless the operator opts in

STOREFRONT_ANALYTICS_ENABLED defaults to false; ingest returns 404 while it is. 404
rather than 403 so a deployment that has not opted in does not advertise the capability.

The admin endpoint still answers, reporting enabled: false with zeroes — "nobody visited"
and "we are not counting" otherwise look identical.

The last step is not taken on trust

paid is read from orders, not from the events. A browser reporting checkout_started
means a button was pressed; whether money arrived is knowable only from the orders table. A
client fabricating an order_id inflates one step and cannot touch the next — there is a
test for exactly that.

order_id is deliberately not a foreign key: the event records what a browser reported
and must survive the order being deleted, rather than vanishing with it and silently
improving the conversion rate.

Verification

$ uv run pytest tests/test_storefront_analytics_unit.py tests/test_storefront_analytics_integration.py -q
23 passed

$ uv run pytest -q
990 passed, 5 skipped, 12 warnings in 16.06s

$ ruff check src/app tests/ && ruff format --check src/app
All checks passed!

Verified end to end against the live stack:

enabled: True | page_views: 2
  Visited the shop         2  conv=1.0   lost=None
  Viewed a product         2  conv=1.0   lost=0
  Added to cart            1  conv=0.5   lost=1
  Started checkout         1  conv=0.5   lost=0
  Paid                     1  conv=0.5   lost=0
  product interest: TAB-RED-001  viewed 2  added 1  rate 0.5

ingest with analytics disabled  → 404
query string carrying an email  → stored as "/shop"
event dated 1999                → accepted: 6, rejected: 1

One bug caught in review

I first declared occurred_at/created_at without timezone=True, which produced
TIMESTAMP WITHOUT TIME ZONE columns while every other table uses timestamptz. Comparing
an aware Python datetime against them failed with a DBAPIError on the first real query.
Fixed to match the TimestampMixin convention.

Because create_all does not alter existing tables, a database that already created the
mistyped table needs DROP TABLE storefront_events; once — it holds no durable data at
this point. Fresh databases are unaffected.

The storefront client and the admin display land separately.

The order funnel begins at order creation, so it can say how many orders were
paid but not how many people looked and left. Browsing, product views and
abandoned carts leave no trace in the order tables, because nothing happened
there.

Adds a public ingest endpoint, a storefront_events table, and an admin endpoint
returning the full funnel: sessions, product views, carts, checkouts and paid
orders.

    POST /v1/analytics/events            public, rate limited, opt-in
    GET  /v1/admin/analytics/storefront  the shopper funnel

Privacy is structural rather than a policy someone remembers. The table has no
column that could identify a person, and a test fails if one ever appears. The
request schema forbids extra fields, so a client sending email or ip_address
gets a 422 rather than having it quietly dropped — silently discarding it would
let a frontend believe it was collecting something it was not. Query strings are
stripped before storage, because that is where personal data arrives by
accident: an email in a share link, a token in a redirect.

Nothing identifies anyone and nothing is stored in the browser beyond a per-tab
session id, so this needs no consent banner in the EU. That is the point of the
shape, not a happy accident — a banner costs 40-60% of sessions to opt-outs,
which would make the funnel it feeds mostly fiction.

Collection is off unless STOREFRONT_ANALYTICS_ENABLED is set, and the ingest
endpoint returns 404 while it is off, so a deployment that has not opted in does
not advertise a capability it is not offering. The admin endpoint still answers,
reporting enabled: false with zeroes, because "nobody visited" and "we are not
counting" otherwise look identical.

The endpoint is public, so it is rate limited, batch capped, closed-vocabulary
and length-bounded throughout. The worst an abusive client can do is add noise
to a report.

Browser timestamps are accepted within 24 hours of server time and discarded
outside it: clocks are wrong often enough that rejecting all skew would lose
real data, and trusting all of it would let anyone write into a period an
administrator has already reported on.

The paid step is read from orders rather than from the events, so a client
claiming a checkout it never paid for inflates one step and cannot touch the
next. order_id is not a foreign key: the event records what a browser reported
and must survive the order being deleted, rather than vanishing with it and
silently improving the conversion rate.

Closes #46

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YY1ekLLeFLkAU2kvdQ8Ey4
@PhilippTheServer
PhilippTheServer merged commit 5ae08d6 into OpenTaberna:main Aug 26, 2026
5 checks passed
PhilippTheServer added a commit to OpenTaberna/wiki that referenced this pull request Aug 26, 2026
The API gained a public ingest endpoint and an admin shopper funnel
(OpenTaberna/fastapi#47), and refreshing the snapshot made the drift check fail
with two undocumented paths.

Documents both, and the parts a reader cannot recover from the schema: that the
pre-order steps are a floor rather than a count while the paid step is exact,
why the ingest endpoint is public, what it refuses to store, and why none of it
needs a consent banner.

Also records why the endpoint returns 404 rather than 403 when collection is
off, and adds STOREFRONT_ANALYTICS_ENABLED to the configuration reference.

Closes #9


Claude-Session: https://claude.ai/code/session_01YY1ekLLeFLkAU2kvdQ8Ey4

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The funnel starts at order creation — everything a shopper does before that is invisible

1 participant