Skip to content

Fix Grok Agent Mode context counts and reported model windows - #1455

Draft
baron wants to merge 3 commits into
mainfrom
fix/1325-grok-context-figure
Draft

baron wants to merge 3 commits into
mainfrom
fix/1325-grok-context-figure

Conversation

@baron

@baron baron commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1325

Root cause and protocol evidence

Grok's ACP PromptResponse._meta.usage.inputTokens is turn-wide billing input, already including cache reads. RPCE treated that sum plus cached reads as context occupancy. Grok separately supplies current conversation state as _meta.totalTokens and the last model call's full prompt count as _meta.inputTokens. The latter survives compaction, so the current-state count is preferred; last-call input is only a compatibility fallback when totalTokens is absent. Its per-model _meta.totalContextTokens was also discarded, leaving the context ring on its 500,000-token fallback.

Verified against the latest public Grok source, commit 2bdd1d6a6369de0e8c68132ea4539e9abd9e14a8:

Issue comments, related origin/main history, and open/merged PR searches were checked before editing; no existing fix or duplicate PR was found.

Changes

  • Prefer Grok's current _meta.totalTokens, which is reseeded after compaction; only responses without that field may fall back to last-call _meta.inputTokens. Zero/invalid current-state claims never resurrect stale last-call input. Do not add cache reads or substitute billing totals.
  • Preserve existing billing values, other ACP providers' mappings, best-effort confidence, and oversight's exclusion of finalization estimates. A last-call prompt is not exact post-response occupancy.
  • Retain positive reported model capacities on base and effort-variant options, including dynamic catalog persistence. The controller first matches the response's _meta.modelId, then falls back to its session-local confirmed catalog selection: Grok can report an API routing slug rather than a catalog key, including effort-specific routing. The resolved capacity travels on message_stop.modelContextWindow, including when the composer selection is default. The ring renders that stored per-turn window; there is no unobserved global-registry lookup. Nil still means no new window claim. If Grok cannot resolve capacity, it now explicitly withdraws the old bound using the documented zero-window signal; the estimator clears that bound before validating the current count and never stores/displays zero as a real capacity.
  • Add synthetic ACP process tests exercising controller bootstrap/prompt responses through ingestNonCodexTurnFinalization, the default-selection ring, oversight windows, response-model changes, compaction with stale last-call input, missing/legacy/zero counts, unchanged non-Grok behavior, and persisted model/variant capacities. Fixtures include sessionId, promptId, modelId, and totalTokens; end-to-end tests obtain capacities from the protocol metadata instead of injecting the 500k fallback.

The usage baseline is explicitly synthetic, reconstructed from the issue's observed token counts and verified source protocol; it is not represented as a captured wire recording. No live Grok runs were used.

Validation

Passed:

  • Fail-first make dev-test FILTER=GrokBuildContextUsageTests: confirmed the original 253,360 vs 25,678 context-count failure and 500,000 vs reported 1,000,000/128,000 window failures before the production fix. Earlier test-harness import/launch-fixture setup errors were corrected before this behavioral red run.
  • make dev-test FILTER='GrokBuildContextUsageTests|GrokBuildACPAgentProviderTests|GrokBuildReasoningEffortAdvertisementTests|AgentRuntimeSidebarViewModelTests|AgentSessionLinkContextLoadTests': 95 tests, 0 failures, including all 5 Grok context tests after round-2 review.
  • make dev-swift-build PRODUCT=RepoPrompt
  • make dev-lint (repository-pinned SwiftFormat 0.61.1 and strict SwiftLint); only changed files were formatted, not the repository.
  • Mandatory contribution preflights: preflight.sh commit and preflight.sh push (secret scanning, guardrails, clean outgoing boundary).
  • Review-specific fail-first run reproduced default-selection window failures, stale 180,000 vs 20,000 context after compaction, and zero-total stale-input resurrection; final suite passed after corrections. An intermediate initializer argument-order compile error was fixed before the final green run.
  • Oracle static review of the review fixes: no actionable findings.

Not run: full root suite, provider-package suite (unchanged package), full pr-ready lane, release/package validation, live CE MCP smoke, or UI/provider runtime validation. No visible app launch/stop/relaunch was performed; the changed build was not activated.

Maintainer-guidance check

Confirmed root cause; upstream protocol remains the authority. The new persisted field is optional for older catalogs. No destructive migration or new background work; ring lookup is memory-only. Scope is limited to Grok context accounting and reported model capacity.

Draft for the requested opposing-model review; please keep in draft until that review is addressed.

Adversarial review follow-up

Addressed all five forwarded items: response/session-local window resolution for default; compaction-aware current totals; stored snapshot windows rather than global-registry reads; realistic synthetic controller-to-view-model fixtures; provider-neutral shared-estimator documentation. Kept the PR in draft.

Additional verified source: conversation replacement reseeds total tokens, turn finalization reads current total.

Round-2 routing-ID review

  • Verified that catalog keys and API routing slugs differ, and effort can rewrite the routing slug.
  • Prefer direct model-ID matches, then the controller's confirmed current catalog key; no global or speculative model inference. If capacity remains unresolved, explicitly clear the carried window instead of inheriting a different model's bound.
  • The synthetic process now handles session/set_model. The regression switches 128k → 1M using routing slug grok-large-reasoning-high, verifies the 210k current count is retained, and asserts the same session/ring/oversight on every turn, including the unresolved-capacity turn. An additional unchanged-count regression verifies that window-only withdrawal publishes.
  • Fail-first focused tests reproduced the inherited 128k bound and blank count. Final coordinated validation: 95 focused app tests passed, RepoPrompt product build passed, lint passed; commit/push safety preflights passed. Oracle static review found no actionable issues. No live provider/UI actions, full suite, provider-package suite, release/package validation, or full pr-ready lane were run for this follow-up.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Grok Build: Agent Mode context figure sums billed input across model calls and counts cache reads twice

1 participant