[babaliauskas]

What this changes

This follows up #57 (unreleased) after its first sync against a real bucket: 52 captures, max_cases: 10.

• Download budget. With captures.max_cases set, sync's automatic fetch takes only each suite's newest 3 × max_cases captures inside max_age. "Newest" goes by the bucket's last-modified time; an unknown time counts as oldest, and ties go by key. Captures already local or promoted count toward the budget, so a re-sync with nothing new downloads nothing. Before this, sync downloaded every capture in the window even though it uses at most max_cases per suite. At thousands of captures a day, that meant thousands of downloads per sync. The summary names the cut: (skipped 2 beyond each suite's newest 30). --since replaces the age cutoff, not the budget. --repair still fetches what its rebuilds need.
• capture list fetches what sync would, applying the same window and budget. Before, it fetched the entire bucket. capture fetch is unchanged.
• Leading-turn fill. Once no whole conversation fits a suite's free slots, the slots left take the next conversation's leading turns. Before, a lone capture plus a 10-turn conversation under max_cases: 10 produced a 1-case suite.
• Missing earlier turns. Under the limits, a conversation turn that recorded no messages list of its own rebuilds its history from the turns before it. Such a turn is now eligible only when all of those earlier turns are local; previously it was promoted with a truncated history. Sync counts these as not eligible: N missing earlier turns and warns once per suite with the likely causes:
  • the fetch window or budget cut the earlier turns off;
  • they were promoted on another machine, or removed by capture clean;
  • the SDK's per-turn sample rate dropped them;
  • the app numbers turn_index from 1.
• Quieter output. With max_cases set, build warnings print only for captures that become cases, after the case file is written.

With neither limit set, nothing changes.

Why

The first real sync showed both problems: a fetch that grows with the bucket, and a capped suite left almost empty.

Test plan

• make ci: 3288 passed, 1 skipped (the existing evalshift-server skip), coverage 95.19%. ruff, ruff format, mypy --strict and pre-commit run --all-files are clean.
• New tests:
  • budget ranking: per suite; local and promoted captures count; the window takes no slots; forced ids, unknown times and ties;
  • --since with the budget;
  • capture list fetching within the limits;
  • leading-turn fill, and a stable re-sync;
  • missing earlier turns: the warning, max_age alone, turn_index None, and an earlier turn that is local but ineligible;
  • warnings only for written cases.
• Offline, on a copy of testApp in the state feat(capture): bounded capture suites (captures.max_age / captures.max_cases) #57's sync left it: converter went from 1 case to 10 (the lone case plus turns 0–8). Only those 9 cases printed warnings. A re-sync left every suite file byte-identical.
• Read-only sync from the real bucket into an empty folder: fetched 50 capture(s) … (skipped 2 beyond each suite's newest 30), giving converter 10, planner 10 and triage 9 cases.
• Independent review of the branch. All findings were fixed: the sampling and turn_index caveats, the capture clean wording, stale docs, and warning order.

Follow-up (not in this PR)

The SDK's EVALSHIFT_SAMPLE_RATE draws once per agent run, which means once per turn, so a sampled conversation keeps some turns and loses others. Drawing per conversation_id (hash-based) in the SDK would keep conversations whole.

🤖 Generated with Claude Code