feat(node): Add Groq and Together AI integrations - #24454
Merged
Merged
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8053e20. Configure here.
Contributor
size-limit report 📦
|
mydea
added this pull request to stack #24458
September 17, 2026 11:50
mydea
marked this pull request as ready for review
September 17, 2026 12:26
mydea
requested review from
a team,
RulaKhaled,
andreiborza,
chargome,
isaacs and
s1gr1d
and removed request for
a team,
andreiborza and
isaacs
September 17, 2026 12:26
RulaKhaled
approved these changes
Sep 17, 2026
RulaKhaled
left a comment
RulaKhaled
left a comment
Collaborator
There was a problem hiding this comment.
lgtm, one thing to look at before merging this
| * attributes and streaming — is shared with the openai integration; only the provider name and origin | ||
| * differ. | ||
| */ | ||
| export function createOpenAiCompatibleIntegration(provider: OpenAiCompatibleProvider): IntegrationFn { |
Collaborator
There was a problem hiding this comment.
this IntegrationFn return kills the options type on groq/together, defineIntegration re exposes Parameters as the public signature so they end up as (...args: any[]).. means a typo recordOutputs silently keeps recording on
maybe drop the annotation and let it infer?
Instrument the `groq-sdk` and `together-ai` clients (chat completions and embeddings, streaming and non-streaming). Both speak the OpenAI wire format, so a shared `createOpenAiCompatibleIntegration` factory reuses the existing openai span/streaming/response logic and only overrides the provider name and origin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`@langchain/groq` drives `groq-sdk`, so a `ChatGroq` call was opening a Groq `gen_ai.chat` span beside the LangChain one. The factory already honors `_INTERNAL_shouldSkipAiProviderWrapping`, but LangChain never registered Groq in its provider-skip list — now it does. Also re-export `groqIntegration`/`togetherAIIntegration` from `@sentry/astro`, which curates its `@sentry/node` re-exports (it can't `export *`), fixing the node-exports consistency check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mydea
force-pushed
the
feat/groq-together-ai-integrations
branch
from
September 18, 2026 07:25
63bd2f5 to
6c5d44b
Compare
mydea
added a commit
that referenced
this pull request
Sep 21, 2026
…ter (#24456) Stacked on #24454. Adds an optional `gen-ai-libraries` e2e app that drives every gen-AI **library** we instrument — OpenAI, Anthropic, Mistral, Together and the Vercel AI SDK — against a real model through the single `E2E_OPENROUTER_API_KEY` the other AI e2e apps use, with one chat-query route and one forced-tool-call route each. `node-mistral` is folded in and removed; Mistral is now one of the covered libraries. ## Structure Shared, framework-agnostic handlers in `src/libraries.ts` are wired into two runtime entries, selected by a `RUNTIME` variant (`tests/constants.ts`), so both instrumentation paths get real-call coverage from one app: - **node** (default) — `src/entry.node.ts`, an express app with runtime channel injection preloaded via `node --import ./src/instrument.node.ts`. - **cloudflare** (`optionalVariants`, `RUNTIME=cloudflare`) — `src/entry.cloudflare.ts`, built with `vite` + `@sentry/cloudflare/vite` (build-time channel injection) and run on workerd via `wrangler dev` against the build's generated config (`dist/gen_ai_libraries/wrangler.json`). `playwright.config.ts` picks the start command and port per `RUNTIME`; the tests (`tests/chat.test.ts`, `tests/tools.test.ts`) are runtime-agnostic — they hit `baseURL` and assert the streamed gen_ai spans, parametrized over the library list in `tests/utils.ts`. ## Why these five Every call uses `openai/gpt-4o-mini`. OpenRouter serves an OpenAI-compatible `/api/v1/chat/completions` and an Anthropic-compatible `/api/v1/messages` ("Anthropic skin"), so all five SDKs reach it (Together/Mistral via lenient OpenAI-shape parsing, the Vercel AI SDK via `@openrouter/ai-sdk-provider`). The model is incidental; each SDK's own request/response code path is what's under test. Two instrumented libraries can't reach OpenRouter and are omitted with a code comment: **Google GenAI** (native Gemini `generateContent` format isn't served) and **Groq** (its SDK hardcodes a `/openai/v1` request path OpenRouter doesn't expose). Groq shares Together's OpenAI-compatible instrumentation code path — exercised here — and is covered by the node-integration-tests in the base PR. ## Notes - The node variant was verified end-to-end against real OpenRouter calls: all five libraries emit the expected gen_ai spans (`together_ai`, `anthropic`, `mistralai`, `openai`, plus the Vercel AI SDK's `gen_ai.generate_content`), including recorded tool calls. The Cloudflare variant runs the same suite in CI. - Folding in `node-mistral` drops its Mistral-specific stream-drain edge cases (tee/pipe, dataloader co-instrumentation) in favour of the uniform query+tool-call coverage across libraries. - The app is marked `optional`, so it only runs when the OpenRouter key is present. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
47 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds auto-instrumentation for the
groq-sdkandtogether-aiclients — chat completions and embeddings, streaming and non-streaming.Both are Stainless-generated SDKs that speak the OpenAI wire format, so rather than duplicating the openai integration twice, a shared
createOpenAiCompatibleIntegrationfactory reuses the existingai/openaispan-building, streaming and response-parsing logic and only overridesgen_ai.provider.nameand the span origin. Each provider is then a small orchestrion config + a one-call integration file, so the next OpenAI-compatible provider (Perplexity, Cerebras, …) is ~15 lines on the same rail.Groq is the meaningful one by volume (~3.4M npm downloads/mo); Together is included since it shares the exact layout at no extra cost.
gen_ai.request.max_tokensis intentionally not asserted — the shared openai request extractor doesn't capture it today (a separate, pre-existing gap), so these providers match its behaviour.