Do not advance the source checkpoint when a batch is truncated (#1212)

[rollroyces]

Why

A custom source endpoint can return more than MAX_NEW_PER_SYNC = 200
items in a single response. sync_custom processes only the first 200
and the run still finishes with status = 'ok'. The next round uses
that successful run's started_at as its since lower bound; if the
remaining items have last_change before that timestamp, an endpoint
honoring since excludes them. Repeated incremental syncs therefore
cannot import them unless they change again.

The fix is the same shape as the existing failed sync path
(#794): if a successful run actually left a window uncovered, it must
not look like a success to the incremental cursor. The run still
records the partial work in created_docs / updated_docs so the
audit trail shows what was imported; the run's status is failed
and last_sync_error carries the reason, so the next round picks up
where this one left off and the remaining items stay reachable.

What changes

crates/utopia-server/src/ingest_sources.rs:

• SyncStats.truncated: bool — true when a sync function knows it
  capped the source's response (items.len() > MAX_NEW_PER_SYNC).
  Field default is false, so existing constructions through
  SyncStats::default() and ..SyncStats::default() keep working.

• sync_custom — sets stats.truncated = true when the endpoint
  returns more than 200 items, before the take(MAX_NEW_PER_SYNC).
  The overflow items are still dropped; the difference is that the
  truncation is now observable.

• sync_source — adds a third arm to the match outcome:
  Ok(stats) if stats.truncated. It writes the run as failed (so
  last_successful_sync_start doesn't advance), the source as
  failed (so the next scheduled run uses the previous successful
  since), the partial work is still in created_docs /
  updated_docs / last_sync_added, and a tracing::warn! tells the
  operator why.

What this proposal does not do

• No schema change. source_sync_runs.status already has 'failed'
  in its CHECK constraint, and sources.last_sync_status accepts
  'failed' too.
• No last_successful_sync_start change. #794 already
  established the invariant that only 'ok' runs advance it; this
  PR sits on top.
• No GitHub / Jira equivalent. The issue text says they have the
  same shape, but Jira already emits a tracing::warn! for
  truncation (ingest_sources.rs:893) and the maintainer team's
  preferred path is per-issue. Adding truncated propagation there
  is straightforward if you want it, but the issue's reproduction
  exercises custom, and the smaller surface is easier to review.
• No tests that need a real cluster. The new test (#1212
  integration) needs Postgres via UTOPIA_DATABASE_URL and skips
  cleanly without it, like the existing
  source_checkpoint_tests.rs.

Verification

• cargo check -p utopia-server --bin utopia-server did not finish
  in the local sandbox (the dev-deps cold rebuild chain on the
  external FUSE mount). Recommend running on a workstation with a
  normal filesystem before merge.

Tracking

• Closes Source syncs import only the first 200 items but advance the checkpoint past the rest #1212 once merged.

[WaylandYang]

Thank you for picking up #1212. Both CI failures are mechanical. cargo fmt wants the ResponseTemplate::new(200).set_body_json(...) chain in source_checkpoint_tests.rs on one line. The test fails with a foreign-key error because it calls start_run for the source before the sources row exists; insert the source first.

The bigger problem is that keeping the old since does not get past the cap. The next run gets the same response, takes the same first 200 items (now all Unchanged), and never reaches item 201. The source then reports failed on every run from then on: the window only grows while the checkpoint stays put, so it never recovers, and anything new that the endpoint sorts after position 200 is blocked too. Each of those runs also raises a SOURCE_SYNC_FAILED error alert, and after 50 of them the keep-50 pruning in finish_run deletes the last ok run, so since falls back to None. That is worse than today, where only part of one window is lost. The ADR draft in #1213 lists this exact approach as rejected for the same reason.

For custom, the fix discussed in #1212 is small and fits this PR: walk the whole response (it is already complete) and count only Created/Updated against MAX_NEW_PER_SYNC. Mark the run truncated only when you stop with items left. Each retry then moves forward, and the run ends ok once everything is in. Please also extend the test to sync twice and assert that the 201st item arrives and the second run is ok; as written, the test would pass with the stall. Two smaller things: sync_custom never sets stats.discovered, so the message reads "(0 > 200)"; and whether a capped batch should raise an error alert at all is still open in #1212.

GitHub and Jira are not covered (GitHub's reader stops at ten pages without saying so), and their design is still being settled in #1212 and #1213. Please scope this PR to custom and change "Closes #1212" to "Refs #1212". Running cargo fmt and this test locally with UTOPIA_DATABASE_URL set before pushing would have caught both CI failures.