Add prompt-prefix caching (cache.checkpoint) support

[sroussey]

Summary

Implements prompt-prefix caching across the AI task framework and provider implementations. This feature allows tasks to eagerly warm provider-side caches (e.g., Anthropic's prompt caching, local KV state) and emit opaque checkpoint handles that downstream tasks can consume to avoid re-processing identical prefixes.

Key Changes

Core Framework

• New CacheCheckpointTask: Eagerly warms a prompt-prefix cache with system prompt, tools, and messages; outputs an opaque checkpoint handle. Lifecycle is tied to the run's ResourceScope — handles are auto-disposed at run end unless a shared scope is injected.
• New CheckpointRegistry: Main-thread registry mapping checkpoint IDs to CheckpointEntry records (provider, model key, prefix content). Supports register/retrieve/delete operations.
• New CheckpointPorts module: Shared input/output schema fragments and resolution logic for tasks that consume/emit checkpoints. Includes resolveCheckpointSession() to validate checkpoint IDs and provider/model compatibility before dispatch.
• AiSessionContext: Extended AiProviderRegistry with a new session context type passed to provider run functions, carrying sessionId, prefix, and parentId for checkpoint support.

Task Updates

• ToolCallingTask: Added checkpoint input ports (checkpoint, emitCheckpoint, keepParentCheckpoint) and output port. Consumes checkpoints to rewind sessions; emits new checkpoints after turns. Parent checkpoints are superseded (disposed) by default unless keepParent is set.
• TextGenerationTask: Added same checkpoint ports as ToolCallingTask for consistency.
• AiTask base class: Updated to pass session context (instead of bare sessionId) to provider run functions.

Provider Implementations

• Anthropic: buildAnthropicCheckpointParams() constructs minimal messages.create params that write prefixes to server-side prompt cache with cache_control: { type: "ephemeral" } on system block, last tool, and last message. applyAnthropicPrefixReplay() prepends cached prefix to new turns.
• HuggingFace Transformers: renderHftPrefixPrompt() and renderHftContinuationPrompt() render prefixes through the model's chat template to ensure tokenization matches warm-up. Prefix-rewind sessions reuse cached KV state.
• Llama.cpp: renderLlamaCppPrefixText() flattens prefix into preloadable text. Sessions track prefix state and rewind on checkpoint consumption.
• Chrome AI / WebBrowser: Updated to accept AiSessionContext instead of bare session ID.

Testing

• Comprehensive test suite (CacheCheckpointTask.test.ts) covering:
  • Checkpoint registration and retrieval
  • Warm-up execution and session lifecycle
  • Parent checkpoint extension and supersession
  • keepParent branching behavior
  • Provider mismatch validation
  • Checkpoint consumption in ToolCallingTask
  • Emitted checkpoint registration and parent disposal
• Provider-specific tests for Anthropic checkpoint params and HFT prefix rendering.

Notable Implementation Details

• Checkpoint IDs double as provider session IDs: The same handle is used for both registry lookup and provider session management, simplifying lifecycle tracking.
• Prefix content is immutable: CheckpointPrefix uses readonly properties; providers replay or re-encode it deterministically.
• Parent supersession is default: Chained checkpoints dispose their parent's session and registry entry unless keepParent: true, preventing resource leaks in linear workflows while supporting branching.
• ResourceScope integration: Checkpoint handles are auto-disposed when the run's ResourceScope completes, ensuring cleanup even if tasks don't explicitly delete them.
• Provider-agnostic schema: Checkpoint ports are defined in CheckpointPorts and mixed into task schemas, allowing any task to opt in without duplication.

https://claude.ai/code/session_01DsBvrmfhe1aY63cjztn1WW

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	62.07%	27418 / 44169
🔵	Statements	61.93%	28387 / 45833
🔵	Functions	62.91%	5200 / 8265
🔵	Branches	50.31%	13584 / 26999

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
providers/huggingface-transformers/src/ai/common/HFT_CacheCheckpoint.ts	4%	0%	0%	4%	46-87, 95-122
providers/huggingface-transformers/src/ai/common/HFT_CapabilitySets.ts	100%	100%	100%	100%
providers/huggingface-transformers/src/ai/common/HFT_Chat.ts	1.42%	0%	0%	1.44%	52-232, 243-247
providers/huggingface-transformers/src/ai/common/HFT_JobRunFns.ts	42.85%	0%	0%	42.85%	83-87
providers/huggingface-transformers/src/ai/common/HFT_TextGeneration.ts	1.44%	0%	0%	1.51%	32-185
providers/huggingface-transformers/src/ai/common/HFT_ToolCalling.ts	0.59%	0%	0%	0.63%	48-359, 371-523

Generated in workflow #2740 for commit 9bf3ee6 by the Vitest Coverage Report Action

[sroussey]

✅ CI is fully green on a85e7d2 — all 14 checks pass (build, all test suites, CodeQL, typecheck-budget).

Since the PR opened, the branch gained two commits:

• 1bf9999 — re-baselined scripts/typecheck-budget.json (packages/workglow grew 157 → 242 instantiations from the new checkpoint type exports; this was the one failing check).
• a85e7d2 — hardening from a full-branch adversarial code review (15 verified findings, 12 fixed):
  • HFT tool-calling / chat checkpoint consumption now renders the prefix into the fed prompt and attaches KV only under a startsWith parity guard (was positionally misaligned → corrupted output).
  • HFT tool-set fingerprint cache warms only the shared tools+system region (was snapshotting the first call's user turn, poisoning reuse).
  • llama-cpp checkpoint consumption takes sole ownership of the live session (was mutating the warm KV in place; keepParentCheckpoint aliased one sequence under two ids).
  • Capability gate: checkpoint/emitCheckpoint on a provider without cache.checkpoint support now fails loudly instead of silently dropping the prefix.
  • Emit accumulator upserts tool calls by id (parallel tool calls from OpenAI-shaped providers were collapsing to one); Anthropic warm-up no longer crashes on all-system prefix messages; HFT emit-only checkpoints now snapshot their KV.
  • Plus smaller cleanups (shared validation/merge helpers, stale session-id reset, doubled getJobInput gated off the hot path, disposeSession idempotency contract documented).

Known deferred (correctness unaffected): AiChatTask doesn't yet reuse a checkpoint's warmed KV on local providers (re-encodes the prefix; needs a cross-provider design pass), and chained-emit KV reuse on HFT relies on the concatenative-template assumption with re-encode fallback.

---

Generated by Claude Code