fix(agent): treat a cron run's in-band model error as a failed run

[williamwa]

Problem

During an account-wide 402 Insufficient credits outage on the amiko/OpenRouter pool, every frequent cron schedule (e.g. feed-scan-comment) looked like it was looping — a rapid burst of assistant(stopReason=error) + agent_end events, each firing an extra introspection LLM call.

Root cause (there is no retry loop)

Traced the full path — nothing retries internally:

• The OpenAI SDK does not retry 402/404 (only 408/409/429/≥500).
• pi-agent-core does not retry stopReason==='error' — it ends the turn and resolves.
• The central scheduler ticks every 10s, so it can't produce sub-second bursts.

The real mechanism: a model error does not throw — the turn ends with stopReason==='error' and waitForSessionIdle resolves normally. So runScheduledJob recorded the empty turn as a success, and the schedule kept firing at its full cron cadence while every firing failed. The "burst" is many frequent schedules all erroring each tick during the shared-pool outage, each also spending an introspection call. Confirmed against session_events: 392 credit-exhaustion errors in one day, balance near-zero/negative, across many agents' feed-scan schedules.

Fix

• handleAgentEvent stamps session.lastTurnModelError (classified kind + raw message) at message_end when stopReason==='error'; cleared at turn start.
• A cron runScheduledJob reads it after the turn completes and, if set, throws ScheduledRunModelError. The scheduler's existing failure path then applies exponential backoff (30s→1h) and auto-recovers on the next success — no permanent pause (wrong for a transient shared-pool outage).
• Scoped to type==='cron': once firings never re-fire, so their completion semantics are unchanged.
• Classify 404 No endpoints found as unavailable so the user-facing notice isn't the generic fallback (the dead anthropic/claude-3.5-haiku fallback surfaced this).

What this does and does not do

• Does: stop frequent cron schedules from hammering every tick through an outage; surface failed runs (with a clear classified message) instead of silent successes; auto-recover.
• Does not: make a 402 succeed. The operational fix for the live outage is topping up the OpenRouter credit pool — this PR only stops the amplification.

Tests

• scheduled-turn.test.ts: a cron run whose turn ended in an in-band model error now rejects with ScheduledRunModelError (kind quota) and heals the dedicated session.queue; a once run on the same error does not throw and tears down normally.
• user-facing-error.test.ts: 404 No endpoints found classifies as unavailable.
• Full files pass (25/25 across the two suites); no new typecheck errors (the two pre-existing InternalStateStore.research errors are on main).

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 3 included reviews currently available. Your 43 included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: 1c5c3de8-9421-4253-be9c-9eaf0cb4b8d5

📥 Commits

Reviewing files that changed from the base of the PR and between 86d7a3b and 963e01e.

📒 Files selected for processing (5)

• apps/agent/src/agent-runner.ts
• apps/agent/src/agent-runner/types.ts
• apps/agent/src/agent-runner/user-facing-error.ts
• apps/agent/test/scheduled-turn.test.ts
• apps/agent/test/user-facing-error.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}