Skip to content

feat: add Groq (groq.com) as an AI assistant provider - #82

Merged
ObscureAintSecure merged 1 commit into
ObscureAintSecure:masterfrom
cagerat:feat/groq-provider
Aug 28, 2026
Merged

ObscureAintSecure merged 1 commit into
ObscureAintSecure:masterfrom
cagerat:feat/groq-provider

Conversation

@cagerat

@cagerat cagerat commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Closes #81

Adds Groq (groq.com / GroqCloud) as a selectable provider under Settings > AI Assistant, following the existing provider pattern.

Keeping it distinct from Grok (xAI)

The names collide constantly, so this is separated at four levels:

  • Uses the official groq SDK rather than Grok's OpenAI-compatible endpoint, so the two share no client construction path
  • Combo label reads "Groq (GroqCloud)" and sits after Mistral, not adjacent to "Grok (xAI)"
  • Module docstring cross-references grok_provider.py
  • test_groq_is_distinct_from_grok asserts a GroqProvider is never a GrokProvider, and a smoke check confirms the two keep separate keys and model lists

Model IDs come from the deprecation page, not the model list

Worth calling out, since it cost me a debugging round. Groq's /docs/models page still lists llama-3.3-70b-versatile and llama-3.1-8b-instant as production models. They are not. Groq announced deprecation on 2026-06-17 and shut both down on 2026-08-16 for free and developer tier:

404 - {'error': {'message': 'The model `llama-3.3-70b-versatile` does not exist
or you do not have access to it.', 'type': 'invalid_request_error',
'code': 'model_not_found'}}

Auth and endpoint are both fine, so it reads as an API key problem. /docs/deprecations is the accurate source.

Shipped list: openai/gpt-oss-120b (default), openai/gpt-oss-20b, qwen/qwen3.6-27b. The last is Groq's own recommended migration target for Llama 3.3 70B, but their model docs still tag it Preview, so it is offered and not defaulted. The model combo is setEditable(True) for every provider, so newer IDs can be typed in without a code change.

Conventions

Follows .claude/rules/ai-providers.md:

  • Explicit timeout=120.0 on the client, per the 120s convention
  • embed_model_id = "st:all-MiniLM-L6-v2", embeddings via the shared get_sentence_transformer cache
  • Per-provider key/model isolation through provider_settings
  • On-demand SDK install via PROVIDER_PACKAGES
  • pyproject.toml groq extra plus all-ai, requirements.txt, and a regenerated uv.lock (uv lock --check passes, resolves 206 packages, adds groq v1.6.0)

Tests

Full suite green: 392 passed.

Nothing in the suite previously asserted a provider's default model, which is how a decommissioned default nearly shipped. Defaults are now pinned against a DECOMMISSIONED_GROQ_MODELS set, covering the provider default and the provider_factory default separately, since they are written in two places and only the factory one applies when a config omits model.

Two things for you to decide

  1. This touches .claude/rules/ai-providers.md to document the stale-model-list gotcha. Happy to drop that hunk if you would rather own that file yourself.
  2. The groq version floor is >=0.11.0 to match anthropic>=0.40.0 and friends, but the SDK resolved to 1.6.0 and has since crossed a major. Say the word if you want >=1.0.0.

Not verified: no live Groq API call was made against a paid or enterprise-tier key, only against the free/developer tier where the 404 above reproduced.

Adds Groq / GroqCloud as a selectable provider under Settings > AI Assistant,
following the existing provider pattern.

Kept deliberately distinct from the existing Grok (xAI) provider:
- uses the official `groq` SDK rather than Grok's OpenAI-compatible endpoint,
  so the two share no client construction path
- combo label "Groq (GroqCloud)", positioned after Mistral rather than next to
  "Grok (xAI)"
- module docstring cross-references grok_provider.py
- regression test asserts a GroqProvider is never a GrokProvider

Model IDs come from Groq's deprecation page, not its model list. /docs/models
still tags llama-3.3-70b-versatile and llama-3.1-8b-instant as production, but
both were shut down on 2026-08-16 for free and developer tier and now return
404 model_not_found. Default is openai/gpt-oss-120b.

Follows the repo conventions in .claude/rules/ai-providers.md: explicit
timeout=120.0, embed_model_id set for the per-recording embedding cache,
per-provider key/model isolation, and on-demand SDK install via
PROVIDER_PACKAGES.

Because nothing previously asserted a provider's default model, both the
provider default and the provider_factory default are now pinned in tests
against a DECOMMISSIONED_GROQ_MODELS set.

Closes ObscureAintSecure#81
@ObscureAintSecure

Copy link
Copy Markdown
Owner

Reviewed and merging. This is a well-put-together PR.

What I checked beyond the diff itself: the six places on master that name a provider (provider_factory, the settings combo, both is_api tuples, the model-list branch, and PROVIDER_PACKAGES) are all covered. A missed call site is exactly how these go wrong, and there isn't one here.

The implementation matches grok_provider and mistral_provider line for line on the shared parts — embed_model_id, the get_sentence_transformer cache, explicit timeout=120.0. Nothing novel where novelty wasn't wanted.

The deprecation finding is the valuable part. Pinning the default against DECOMMISSIONED_GROQ_MODELS in two places, because the provider default and the factory default are written separately and only the factory one applies when a config omits model, is a sharper test than the situation strictly required. Thanks for writing up the 404-reads-as-an-auth-problem detail in the rules file too — that's the kind of thing that costs someone an afternoon.

Note this is the first PR on the repo actually verified by CI (added in #12 last week; your run needed approval as a first-time contributor). 392 passed.

I could not independently verify qwen/qwen3.6-27b without GroqCloud access, so I'm taking your testing on that. It's offered rather than defaulted and the combo is editable, so the blast radius is small either way.

@ObscureAintSecure
ObscureAintSecure merged commit 81f46f6 into ObscureAintSecure:master Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Groq (groq.com) as an AI assistant provider

2 participants