perf(agent-sessions): rank the page on the index, then aggregate only that page

[JeremyFunk]

Why

The Agent Sessions list still took 5–15s per page after ai_trace_index landed (#692), and every scroll waited that long again. Production telemetry (WarehouseQueryService.executeSql, query.context = listAiSessions, 2026-08-30 → 09-02):

read	p50	p90	killed at the 15s ceiling
listAiSessions (post-index, 1-day window)	7.6s	15.3s	8 of 31
listAiSessions (30-day window, the page's largest preset)	killed	killed	every time
aiSessionsFacets (index only)	0.5s	0.9s	0

The index fixed detection, but the list still ran the whole trace_detail_spans fan-out over every agent trace in the window (~1,800 traces / 96k spans a day) and paged with LIMIT/OFFSET afterwards, so each page paid the full aggregation.

What that fan-out actually costs was measured on production (db.duration_ms, cold parts, one org):

shape	time
5 trace ids, window spanning 32 partitions	>10s (killed)
3 trace ids, one partition (±1h)	0.56s
one page of 50 sessions, one partition	1.3–2.8s cold, ~0.9s warm
old list shape, one day (3 partitions with the ±1-day pad)	5.7–15s

trace_detail_spans sits on object storage, and a TraceId IN (…) seek there is priced by the partitions it probes, not by the rows it returns. The caller's window — padded by a day on each side — is what the old query pruned by.

What

The list is now two reads, the shape traceListQuery already uses:

1. aiSessionPageQuery — over ai_trace_index alone, across the caller's whole window: resolves each trace to its session key, ranks sessions by first agent span, applies the vendor/service filters (as per-trace HAVING countIf(...) > 0 existence tests) and LIMIT/OFFSET, and returns the page's session ids with the bounds of their agent spans. ~0.6s cold over 30 days.
2. aiSessionListQuery — the existing aggregation, restricted to that page's traces (TraceId IN over the index, filtered to the page's keys) and bounded by the page's agent-span extent ± 1h instead of the caller's window ± 1 day. The session key comes from the index via an INNER JOIN on the per-trace rows, so both stages resolve a trace identically (a re-derivation from spans could disagree and drop the row from the page it was ranked into).

The handler (ai-sessions.http.ts) runs them in sequence, short-circuits an empty page, derives fanOutStart/fanOutEnd from the page, returns rows in the page's order, and annotates the span with maple.ai.page_size / maple.ai.aggregated so a page/aggregation mismatch is visible.

FAN_OUT_PAD_SECONDS drops from a day to an hour, measured: across 4,920 agent traces (two days of production), the first span leads the first agent span by ≤0.1s and the last span trails the last agent span by ≤10 min. The deep-link window resolve (aiSessionWindowQuery/aiTraceWindowQuery) keeps its day of padding under a separate WINDOW_PAD_SECONDS — that path resolves one session and the detail page shows the spans themselves, so it is not worth clamping.

Stage 2 is bounded by the page alone: its own index reads run over fanOutStart..fanOutEnd too (a page trace's index rows all lie inside its session's bounds, so it resolves exactly as the page did), which means a 30-day list request scans the whole index once, not three times. An empty sessionIds is a QueryBuilderDefect rather than IN ().

Measured end to end against production with the exact compiled SQL (cold day, never touched during testing):

read	before	after
1-day window, page 2	5.5–15s	0.57s (page) + 1.8s (fan-out)
30-day window, offset 200, sessions with up to 36 traces	killed at 15s	0.6s (page) + 2.4s (fan-out)

One additive contract change: ListAiSessionsResponse.ranked (optional) — how many sessions the page ranked. data can fall short of it: ai_trace_index and trace_detail_spans are two materialized views written one after the other from the same traces insert, so the newest session can be ranked a moment before its spans are readable (the old single-read shape hid this by paging after the aggregation). The hook now pages on ranked (next offset = Σ ranked, last page when ranked < limit) instead of data.length, which would have ended the scroll on a short page. Ordering changes from "first span of any kind" to "first agent span" (sub-second difference, documented); the row's startTime still reports the true first span. Vendor/service filters are now per-trace existence tests each (HAVING countIf(...) > 0), matching what the facets count, rather than one row matching both.

Known limit

The fan-out window is the page's extent. For an org whose 50 newest sessions spread across many days (a few sessions a day, 30-day preset), the fan-out still probes every partition in that spread. Measured: 5 trace ids across 11 partitions (partly warm) took 0.84s; 5 ids across 32 cold partitions were killed at 10s. That org is no worse off than today (the old shape probed the same partitions and more), but it is not where the 2–3s number applies. The next step for it is either a per-trace rollup so the fan-out reads a narrow table, or chunking stage 2 per day-cluster — not in this PR.

Interaction with open PRs

• feat(agent-sessions): filter by model, agent, tool, environment, id, errors, cost and usage, and sort #738 (filters/sorts) adds session-level HAVING filters and non-startTime sorts (cost, tokens, duration, errors) to aiSessionListQuery. Those need the full-window fan-out before LIMIT — the exact shape this PR removes. Index-level filters (vendor, service, environment, model, agent, tool, id search) slot straight into aiSessionPageQuery; the session-level ones need either their aggregates materialised into ai_trace_index so stage 1 can rank on them, or an explicit full-scan fallback. Flagging rather than resolving here.
• perf(agent-sessions): project the span read down to the keys the mapper reads #739 (lean span projection) touches aiSessionSpansQuery / the spans handler only; merge should be mechanical.

Verification

• packages/query-engine-integrations unit tests + SQL baseline regenerated (diff reviewed).
• apps/api route test for the two-stage handler (bounds, session ids, page order, dropped session + ranked, empty page = one read).
• ClickHouse e2e (ai-trace-index-materialization.clickhouse.e2e.test.ts, real migrations, both stages end to end: multi-span and two-trace sessions, an out-of-window trace, sub-second bounds fed back into stage 2) and the SQL-catalog analyzer sweep.
• Adversarial review pass (findings folded in: the ranked contract, comment corrections, the low-volume limit stated).
• Exact compiled SQL of both stages run against production via run_sql (numbers above).

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

🍁 Maple PR preview

Warning

Preview cleanup could not be confirmed. The Alchemy teardown outcome was skipped.

Final commit 026df77 · View workflow run