Bound the complete classifier pipeline with one shared deadline - #46
Merged
Merged
Conversation
…t-ranking-20261005
…t-ranking-20261005
…t-ranking-20261005
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A live three-perspective managed batch exceeded the inherited 15-second ranking deadline and returned retrieval_fallback despite successful Gemini key recovery. Allow one bounded 25-second classifier pipeline, shared across the safety gate, parallel viewpoint ranking and any fallback, instead of restarting a timeout for each stage. Caller cancellation remains terminal; the safety gate retains its shorter local timeout.
This gives slower valid completions time to finish while bounding the complete classifier pipeline. It changes neither token/provider budgets nor retry counts and preserves automatic routing. Existing fallback remains explicitly degraded after the shared deadline.
Validation: 79 focused tests pass, covering deadline reuse, in-flight expiration, caller cancellation, original safety behavior, perspective scoring and strict telemetry. The actual Workerd runtime also checks combined AbortSignals in CI. Full tests, package validation and deployment packaging run in exact-source CI because the shared Fleet disk cap blocks another local full run. Refs #30 and sass-maker/free-ai#95. The live gateway advanced independently to 38354865408e51d7ff92c20b8f2f20b18b251706 / version 603d83ab-dd84-479a-a5e3-668e98f42df0; that subsequent release only changes Replay Lab/provider automation diagnostics and retains the key-recovery fix.