Skip to content

Stop retired models from failing API key checks; lock off unavailable models - #420

Merged
ProfSynapse merged 1 commit into
mainfrom
fix/openrouter-key-validation
Oct 15, 2026
Merged

ProfSynapse merged 1 commit into
mainfrom
fix/openrouter-key-validation

Conversation

@ProfSynapse

Copy link
Copy Markdown
Owner

Summary

Adding an OpenRouter key failed with "No endpoints found for liquid/lfm-2.5-1.2b-thinking:free": the key check sent a test message to a free model OpenRouter has removed. A new provider is only turned on after the check passes, so the key was saved but OpenRouter stayed off. It looked connected but didn't work.

Rule applied everywhere: a retired model never fails a provider. It only can't be turned on.

Key checks

Provider Before Now
OpenRouter chat to liquid/lfm-2.5-1.2b-thinking:free (404) GET /api/v1/key
OpenAI chat to gpt-5-nano GET /v1/models (403 = restricted key, accepted)
Anthropic chat to claude-haiku-4-5-20251001 GET /v1/models
Google gemini-2.5-flash:generateContent GET /v1beta/models
Mistral chat to mistral-tiny GET /v1/models
Groq chat to llama3-8b-8192 (retired, always failed) GET /openai/v1/models
DeepSeek chat to deepseek-v4-flash GET /models
Perplexity, Requesty chat test message still a chat test message, but a model error (400/403/404 about the model, not the key) accepts the key

Unavailable models

The provider dialog fetches the provider's model list and locks off chat models it no longer lists, labelled "No longer available from ". Saved toggles are not changed, so a model that comes back keeps its setting. Dots and hyphens compare equal, because OpenRouter serves anthropic/claude-haiku-4-5 but lists anthropic/claude-haiku-4.5. If the list can't be fetched or comes back empty, every model stays available.

Live today, this flags only OpenRouter's xiaomi/mimo-v2-omni and xiaomi/mimo-v2-pro (0 endpoints). Nothing is flagged for OpenAI, Anthropic or Google.

Test plan

  • New unit tests: LLMValidationService.test.ts (23), ProviderModelAvailability.test.ts (12). The validator tests fail on the old code (7 of the original 14).
  • Full Jest: 556 suites / 6,663 tests passing; npm run build passes
  • Live against real endpoints:
    • valid keys pass for OpenRouter, OpenAI, Anthropic and Google;
    • bogus keys are rejected for all nine providers, with the provider's own message.
  • In the running app (test vault), the OpenRouter dialog shows 58 models, with the two MiMo models locked off and labelled
  • Mistral, Groq, DeepSeek, Perplexity and Requesty valid-key path: no keys available. Their bad-key responses were verified live.

🤖 Generated with Claude Code

OpenRouter key validation sent a test message to
liquid/lfm-2.5-1.2b-thinking:free, which OpenRouter removed, so every
OpenRouter key failed with "No endpoints found" and the provider stayed off.

- OpenRouter checks the key with GET /api/v1/key.
- OpenAI, Anthropic, Google, Mistral, Groq and DeepSeek check the key by
  listing models; OpenAI accepts 403 (restricted key without model-read).
  This also fixes Groq, whose test model llama3-8b-8192 was retired.
- Perplexity and Requesty still send a test message, but an error about the
  test model no longer counts as a bad key.
- The provider settings lock off chat models the provider no longer lists
  ("No longer available from <provider>") without changing saved toggles.
  Dots and hyphens compare equal so aliases like
  anthropic/claude-haiku-4-5 are not flagged. If the list can't be fetched,
  every model stays available.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ProfSynapse
ProfSynapse merged commit e8e370a into main Oct 15, 2026
1 check passed
@ProfSynapse
ProfSynapse deleted the fix/openrouter-key-validation branch October 15, 2026 11:28
ProfSynapse added a commit that referenced this pull request Oct 15, 2026
Finalize the changelog for #420 and #421, and document in README Network Use
and the provider guide that opening a provider's settings fetches its model
list and that retired models are marked unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant