Skip to content

Test API keys with the cheapest registry models; remove Gemini 2.5 from Requesty - #421

Merged
ProfSynapse merged 1 commit into
mainfrom
fix/validation-registry-models
Oct 15, 2026
Merged

ProfSynapse merged 1 commit into
mainfrom
fix/validation-registry-models

Conversation

@ProfSynapse

Copy link
Copy Markdown
Owner

Summary

This follows #420 and makes sure no key check can break again because a model was retired.

  • Providers with a model-free key check use it. This was done in Stop retired models from failing API key checks; lock off unavailable models #420: OpenRouter /key, plus the model lists for OpenAI, Anthropic, Google, Mistral, Groq and DeepSeek.
  • Providers without one (Perplexity, Requesty) now send the test message to the cheapest models in their own registry:
    • priced models come first, up to three attempts;
    • an error about the model moves on to the next one;
    • an error about the key stops immediately;
    • if every attempt is refused only as a model, the key is accepted.
  • Today's picks:
    • Perplexity: sonar → sonar-reasoning-pro → sonar-deep-research
    • Requesty: openai/gpt-5.4-nano → mistral/mistral-large-4 → moonshot/kimi-k2.7-code
  • Requesty's old hard-coded test model, openai/gpt-5-nano, was not in its registry at all.
  • Guard test: fails if ValidationService.ts names a model id again, either as model: '...' or as a /models/<id>:generateContent URL.
  • Removed Gemini 2.5 Flash and Gemini 2.5 Pro from Requesty. Google Cloud retires both on 2026-10-20, and new Gemini API projects already get "no longer available to new users".

Test plan

  • LLMValidationService.test.ts, 29 tests:
    • cheapest model first;
    • fallback to the next model;
    • every model gone → key accepted;
    • a key refusal stops at once;
    • ordering, deduplication and the attempt cap;
    • the guard test.
  • Full Jest: 6,669 passing; npm run build; check_model_registry.py requesty perplexity clean
  • Live with bad keys: Perplexity stopped after sonar and Requesty after openai/gpt-5.4-nano, each with the provider's own error. No valid keys were available to test a successful check.

🤖 Generated with Claude Code

…hardcoded one

Perplexity and Requesty cannot check a key without sending a message. They
now take the cheapest models from their own registry (priced models first,
up to three) and move to the next when one is refused as a model; a refusal
of the key stops at once. Requesty's old test model, openai/gpt-5-nano, was
not even in its registry.

A guard test fails if ValidationService names a model id again.

Remove Gemini 2.5 Flash and Gemini 2.5 Pro from Requesty: Google retires both
(Cloud 2026-10-20) and new accounts already get "no longer available".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ProfSynapse
ProfSynapse merged commit 25ea5e6 into main Oct 15, 2026
1 check passed
@ProfSynapse
ProfSynapse deleted the fix/validation-registry-models branch October 15, 2026 11:36
ProfSynapse added a commit that referenced this pull request Oct 15, 2026
Finalize the changelog for #420 and #421, and document in README Network Use
and the provider guide that opening a provider's settings fetches its model
list and that retired models are marked unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant