Bring every example onto current SDKs and models, and make CI report honestly by beran-t · Pull Request #117 · e2b-dev/e2b-cookbook

beran-t · 2026-08-12T16:12:42Z

Summary

CI reported green while half the examples failed - the test step carried continue-on-error and nothing checked its outcome, so the last scheduled run (15 failed / 14 passed) showed as success. This makes the job fail honestly, brings every example onto current SDKs and model IDs, and takes the suite from a true baseline of 13/36 to 27/27 green, verified by running it on this branch (run 31802866525).

Models. Five examples called models that no longer exist (claude-3-5-sonnet-20241022, llama3-70b-8192, moonshotai/kimi-k2-instruct-0905, Together's non-serverless Llama-3.1-405B) and six more were on OpenAI models with a 2026-10-23 shutdown. Every ID retargeted against each provider's own deprecation page, READMEs and comments included. Two subtleties worth knowing: the whole gpt-5.6-* family rejects function tools on /v1/chat/completions unless reasoning_effort is 'none', and gpt-5.6-sol rejects 'none' outright, so tool-calling examples use terra. ibm/granite-34b-code-instruct is left alone - IBM's lifecycle page is not publicly reachable.

E2B SDKs bumped to current and migrated to the v2 API: 17 Python Sandbox() -> Sandbox.create() sites, the paginated Sandbox.list(), and files.write() returning WriteInfo rather than a path. One was a real break, not a deprecation: openai-codex-in-sandbox-python passed its template positionally while pinning e2b>=2.3.0, raising TypeError. It was one of the 18 examples the old test list did not cover.

Provider SDKs bumped (openai v4->v7, Anthropic 0.28->0.116, groq-sdk, together-ai, watsonx, @openai/agents), plus rewrites where the API changed shape: Mistral (chat() -> chat.complete(), and the Python client lives at mistralai.client, not the package root), Firecrawl v0->v4, and Vercel AI SDK v3->v7 for nextjs-code-interpreter, which replaces OpenAIStream/experimental_onToolCall with streamText + stopWhen and moves useChat to @ai-sdk/react. LangChain/LangGraph 1.x adopts the existing #83 and #84. autogen-python is removed rather than ported. All 30 JS lockfiles and 8 Python lockfiles regenerated; every JS example typechecks and the Next.js app builds.

The test runner replaces Jest. Jest's module runtime cannot load the current e2b SDK - e2b depends on chalk 5 and four other ESM-only packages, and while Node 22+ handles those from both CJS and ESM, Jest does not implement require(esm). The suite could not start at all after the bump. It is now a plain tsx script writing the same results.json, so reporting and Slack are unchanged. npm test -- <substring> runs one example. Coverage went 29 -> 27 examples, but the old list included a notebook path that did not exist and 18 examples were untested; every exclusion now records why, and only three are genuinely unfixable here (a missing repo secret, an upstream package, a long-running server).

Robustness, which is most of the remaining diff. These examples ask an LLM to plot something and then assumed it did. results only holds Jupyter display output, so it is empty - or None - whenever the model printed instead of rendering. Nine examples crashed or hard-exited on that; two notebooks called sys.exit(), which nbconvert counts as a failed cell; four raised on it. A missing chart no longer fails an example, because the sandbox still ran the code, which is what these demonstrate. A missing tool call still fails, since nothing was demonstrated. An audit that enumerates every result subscript and checks for a guard now reports zero unguarded.

Two bugs found by running rather than reading: groq-code-interpreter-python hardcoded GROQ_API_KEY = "" and never read the environment, so it could never have passed in CI (the E2B SDK falls back to its env var, Groq's client does not, which is why only the Groq call failed). And e2b 2.38.0 dropped the http2 kwarg that every published e2b-code-interpreter still passes, so a fresh pip install e2b-code-interpreter raises on the first run_code(). The Python examples cap e2b<2.38 until e2b-dev/E2B#1668 lands, then the cap comes off.

crewai-python needed a CrewAI bug worked around. Its native OpenAI provider only forwards reasoning_effort when it decides the model is a reasoning model, and it decides that with "o1" in model.lower() (crewai/llms/providers/openai/completion.py:269), so it silently dropped the parameter for gpt-5.6-luna and the API rejected function tools. The Responses path in the same file sends it unconditionally, so the example asks for api="responses" - which is also what the API's error message recommends. Worth reporting upstream: that heuristic cannot recognise any current reasoning model, and false-positives on any name containing o1.

fireworks-code-interpreter-python is the one example excluded for a reason nobody here can fix: qwen2p5-coder-32b-instruct returns 404 "Model not found, inaccessible, and/or not deployed", which does not distinguish a retired model from one this account has not deployed. It needs someone with the Fireworks account to pick a current model.

The scheduled workflow has been reporting success while examples failed: the test step carries continue-on-error and nothing downstream checked its outcome. The 2026-08-10 run was 15 failed / 14 passed and showed green. - Fail the job when any example fails, while keeping the reporting steps running so the artifacts and the Slack digest still get produced. - Upload logs/ as an artifact. Per-example failure detail was written there and then discarded, which made notebook failures undiagnosable from CI. - Log the caught error to the per-example log. commands.run throws CommandExitError on a non-zero exit, so the exitCode !== 0 branch never ran and failures produced no recorded output. - Bump checkout/setup-node to v5 and Node to 22 (the runner was force-migrating the v2 actions to Node 24 with a warning every run). - Add a uv interpreter so PEP-621 examples can be tested at all. Its absence is why every newer Python example was uncovered. - Add 7 previously untested examples, and document why each remaining one is not covered instead of leaving the gap silent. Coverage goes from 29 of 48 examples to 36.

Five examples were calling models that no longer exist, which is why the 2026-08-10 CI run failed them. Six more were on OpenAI models with a 2026-10-23 shutdown date. Even the newest example was a generation behind. Verified against each provider's own model/deprecation page on 2026-08-12: claude-3-5-sonnet-20241022 -> claude-sonnet-5 (404 in CI) claude-3-opus-20240229 -> claude-opus-5 llama3-70b-8192 -> llama-3.3-70b-versatile (decommissioned) llama3-8b-8192 -> llama-3.1-8b-instant (decommissioned) moonshotai/kimi-k2-instruct-0905 -> openai/gpt-oss-120b (404 in CI) Meta-Llama-3.1-405B-Instruct-Turbo -> Llama-3.3-70B-Instruct-Turbo (non-serverless) deepseek-coder-33b-instruct -> DeepSeek-V4-Pro Qwen/Qwen2-72B-Instruct -> Qwen/Qwen3.6-Plus codestral-latest -> codestral-25-08 o1-mini -> gpt-5.6-sol (retired 2025-10-27) o3-mini, gpt-4-1106-preview -> gpt-5.6-sol (shutdown 2026-10-23) gpt-3.5-turbo, gpt-3.5-turbo-0125 -> gpt-5.6-terra (shutdown 2026-10-23) gpt-4o, gpt-4-turbo, gpt-5.4 -> gpt-5.6-terra gpt-4o-mini, gpt-5.4-mini -> gpt-5.6-luna claude-sonnet-4-6, -4-5-20250929 -> claude-sonnet-5 The Together examples list alternatives as commented-out lines, so each alternative got a distinct current serverless model rather than all collapsing onto the same one. READMEs, notebook prose and code comments were updated in the same pass so they do not contradict the code. Directory names and links are untouched. Not changed: ibm/granite-34b-code-instruct, because IBM's model lifecycle page is not publicly reachable and guessing at a replacement is worse than leaving a known value. Fireworks IDs were checked and are still current.

pyautogen ^0.3.0 is a superseded package name; the project moved to autogen-agentchat 0.4+, which is a rewrite rather than a version bump. The example also called gpt-3.5-turbo and gpt-4-1106-preview, both of which shut down on 2026-10-23, and it pinned e2b 0.17.2a60, a pre-1.0 alpha. Rewriting it onto the current AutoGen would be roughly a day of work for an example nobody has touched since 2025-03. Removing it instead. It is in git history if someone wants to bring it back on the new SDK. Removes the README row along with the directory.

Pins (npm @e2b/code-interpreter 2.7.0, e2b 2.38.3; PyPI e2b-code-interpreter 2.9.0, e2b 2.38.0): @e2b/code-interpreter ^1.0.1 -> ^2.7.0 (12 examples) e2b ^2.3.x -> ^2.38.3 (10 examples) e2b ^0.16.2-beta.52 -> ^2.38.3 (root, the test harness itself) e2b-code-interpreter ^1.0.1/^1.1.1 -> ^2.9.0 e2b_code_interpreter==1.0.0/1.0.5 -> ==2.9.0 (13 notebooks) Notebooks keep the exact-pin convention they already used; manifests keep ranges. Mixing the two conventions per file type is deliberate. API migration, per https://e2b.dev/docs/migration/v2: - Python Sandbox() -> Sandbox.create(), 17 call sites. The v2 constructor is deprecated but still present and keyword-only, so most of these were working-but-deprecated. One was a real break: openai-codex-in-sandbox-python passed its template name positionally while pinning e2b>=2.3.0, which raises TypeError. It was one of the examples the test list did not cover, so nothing caught it. - Sandbox.list() returns a paginator in v2. nextjs-code-interpreter was treating it as an array. Fixed, and switched to a server-side metadata query rather than listing everything and filtering client-side. vercel-eve-feedback-analyst-js was already correct. Nothing needed doing for files.write: every call site in the repo already used the v2 sandbox.files.write() form. with Sandbox(...) as sbx is unaffected - v2 keeps __enter__/__exit__.

openai ^4.4x -> ^7.4.0 (5 examples) @anthropic-ai/sdk ^0.28.0 -> ^0.116.0 (3 examples) groq-sdk ^0.3.3 -> ^1.5.0 together-ai ^0.6.0-alpha.4 -> ^0.48.0 @openai/agents ^0.1.x -> ^0.15.0 (2 examples) @ibm-cloud/watsonx-ai ^1.6.1 -> ^1.7.16 ibm-cloud-sdk-core ^5.3.2 -> ^5.6.0 anthropic ==0.35.0 -> ==0.121.0 (notebooks) together ==1.3.1 -> ==2.30.0 ibm_watsonx_ai ==1.2.9 -> ==1.6.1 python-dotenv ==1.0.x -> ==1.2.2 All 28 JS lockfiles regenerated from scratch. The stale ones were pinning transitive deps that conflicted with the new peers. Breakage found by typechecking each example against the new SDKs, and fixed: - groq-code-interpreter-js imported from 'groq-sdk/src/resources/chat/completions'. That path is gone in v1 (it pointed into source, not dist, so it was always wrong). Now 'groq-sdk/resources/chat/completions', and the types it pulls are ChatCompletionTool / ChatCompletionMessageParam rather than the old CompletionCreateParams.Tool / .Message. This one was a runtime failure, not just a type error. - files.write() returns WriteInfo in E2B v2, not the path string. Four examples were assigning it straight to a string and logging an object. Now destructure { path }. - files.write() no longer accepts a Node Buffer. The CSV uploads read as utf-8 strings instead. - openai v7 widened tool_calls to a union of function and custom tool calls, so .function needs narrowing first. Fixed in openai-js and gpt-4o-js. gpt-4o-js also had a dead branch treating arguments as an object; it is a JSON string. - Message and tool array literals needed the SDKs' own param types so role/type are not widened to string. - groq-code-interpreter-js printed NaN in its banner: '=' * 50 is Python, not TypeScript. Now '='.repeat(50). Typecheck is clean across every bumped example. Two examples do not install and are left for the next commit or for their owners: nextjs-code-interpreter needs the Vercel AI SDK rewrite (ai@3 peer-pins openai ^4.42), and vercel-eve-feedback-analyst-js has a pre-existing peer conflict (@e2b/eve-sandbox 0.1.1 wants eve ^0.27, the example pins ^0.30.6) that also fails on main, untouched by this PR.

LangChain and LangGraph: adopted the existing community PRs #83 (langchain 1.x) and #84 (langgraph 1.x) rather than redoing that work, then re-applied this branch's e2b pin bump and model retarget on top and regenerated both lockfiles. Mistral v0 -> v2 (codestral-code-interpreter-js and -python): MistralClient -> Mistral new MistralClient() -> new Mistral({ apiKey }) client.chat(...) -> client.chat.complete(...) from mistralai.client -> from mistralai message.content is now string | ContentChunk[], so the JS example joins text chunks instead of assuming a string. Firecrawl v0 -> v4 (firecrawl-scrape-and-analyze-airbnb-data and claude-visualize-website-topics): FirecrawlApp -> Firecrawl scrapeUrl(url, {pageOptions, extractorOptions}) -> scrape(url, {formats:[{type:'json',schema}]}) result.data['llm_extraction'] -> result.json crawl_url(url, params={...}) -> crawl(url, limit=...) crawl result is now a CrawlJob whose .data holds Document models, so the Python notebook uses model_dump() instead of dict iteration. Also cleared one notebook's captured pip-install log, which was from a 2024 run and contradicted the versions the notebook now installs. All four poetry lockfiles and all four uv lockfiles regenerated. nextjs-code-interpreter is deliberately left on Vercel AI SDK v3 and its openai pin reverted to ^4.42.0 so it still installs and runs. Going to v5+ means replacing OpenAIStream / StreamingTextResponse / experimental_onToolCall with streamText, and changing the useChat client alongside it. That has to be run in a browser to be trusted, so it is out of scope here and now documented in the example's README.

Found by running hello-world-python against live E2B after the bump: TypeError: get_transport() got an unexpected keyword argument 'http2' e2b 2.38.0 changed e2b.api.client_sync.get_transport from (config, http2=True) to (proxy) and dropped the kwarg, but every published e2b-code-interpreter release still calls it with http2=False (code_interpreter_sync.py:84). e2b-code-interpreter declares e2b>=2.26,<3.0.0, so pip and poetry happily resolve the broken pair. Verified against live E2B: 2.9.0 + e2b 2.38.0 fails, 2.9.0 + e2b 2.37.0 works. Checked 2.9.0, 2.8.1, 2.8.0 and 2.7.0 - all fail the same way, so this is the e2b side, not code-interpreter. Capping e2b to >=2.26,<2.38 in the three poetry projects and the 13 notebooks that use code-interpreter. Examples that use the raw e2b SDK stay on 2.38.x. This is an upstream bug and the cap should come off once it is fixed.

cursor

Stale comment

claude · 2026-08-12T16:26:33Z

 import { Sandbox, Result } from '@e2b/code-interpreter'
 import { Groq } from 'groq-sdk'
-import { CompletionCreateParams } from 'groq-sdk/src/resources/chat/completions'
+import type { ChatCompletionTool, ChatCompletionMessageParam } from 'groq-sdk/resources/chat/completions'
 import fs from 'node:fs'
 import dotenv from 'dotenv'
 dotenv.config()


🔴 The JS example's MODEL_NAME is still the decommissioned llama3-70b-8192 (line 18) — this PR fixed the groq-sdk import path in this same file but left the model ID unretargeted. The sibling Python notebook (groq-code-interpreter-python/llama_3_code_interpreter.ipynb) was updated in this PR to llama-3.3-70b-versatile, and this example's own README title was bumped to "Llama 3.3" — the code should match with the same value.

Extended reasoning...

examples/groq-code-interpreter-js/llama_3_code_interpreter.mts line 18 still reads const MODEL_NAME = 'llama3-70b-8192', and the commented alternative on line 17 still reads llama3-8b-8192. Both are Groq-decommissioned model IDs. The PR's own description explicitly lists llama3-70b-8192 as one of the five model IDs that 'no longer exist' and needed retargeting across the repo.

The diff for this exact file only touches the groq-sdk import path (groq-sdk/src/resources/... → groq-sdk/resources/...), a type rename, and a repeat → String.repeat cleanup. MODEL_NAME was never touched, so it slipped through even though the PR author was editing this file directly.

The strongest evidence this is an oversight rather than an intentional skip: the sibling Python notebook examples/groq-code-interpreter-python/llama_3_code_interpreter.ipynb was updated in this same PR — MODEL_NAME there is now 'llama-3.3-70b-versatile' and its 8b comment now reads 'llama-3.1-8b-instant'. On top of that, this JS example's own README.md title was bumped from 'Llama 3' to 'Llama 3.3' in this PR, so the README now advertises a model the code doesn't actually call.

Step-by-step proof of the failure:

A user clones the repo after this PR merges and runs examples/groq-code-interpreter-js/llama_3_code_interpreter.mts per its README (which now says 'Llama 3.3 + function calling + E2B Code interpreter').

The script calls groq.chat.completions.create({ model: MODEL_NAME, ... }) with MODEL_NAME = 'llama3-70b-8192'.

Groq's API rejects the request because llama3-70b-8192 has been decommissioned, returning a model_decommissioned-style error instead of running the code-interpreter loop.

This directly contradicts the PR's stated goal — 'Every ID retargeted against each provider's own deprecation page' — since this one file's model ID was missed.

Fix: mirror the Python twin — change line 18 to const MODEL_NAME = 'llama-3.3-70b-versatile' and update the line 17 comment to llama-3.1-8b-instant, matching what was already done in groq-code-interpreter-python/llama_3_code_interpreter.ipynb.

claude · 2026-08-12T16:26:34Z

     "name": "stdout",
     "output_type": "stream",
     "text": [
-      "Requirement already satisfied: e2b_code_interpreter==1.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (1.0.0)\n",
-      "Requirement already satisfied: anthropic==0.35.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (0.35.0)\n",
-      "Requirement already satisfied: python-dotenv==1.0.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (1.0.1)\n",
-      "Requirement already satisfied: attrs>=21.3.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==1.0.0) (23.2.0)\n",
-      "Requirement already satisfied: e2b<2.0.0,>=1.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==1.0.0) (1.0.1)\n",
-      "Requirement already satisfied: httpx<0.28.0,>=0.20.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==1.0.0) (0.27.0)\n",
-      "Requirement already satisfied: anyio<5,>=3.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (3.7.1)\n",
-      "Requirement already satisfied: distro<2,>=1.7.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (1.8.0)\n",
-      "Requirement already satisfied: jiter<1,>=0.4.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (0.4.1)\n",
-      "Requirement already satisfied: pydantic<3,>=1.9.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (2.9.1)\n",
-      "Requirement already satisfied: sniffio in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (1.3.0)\n",
-      "Requirement already satisfied: tokenizers>=0.13.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (0.20.0)\n",
-      "Requirement already satisfied: typing-extensions<5,>=4.7 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.35.0) (4.12.2)\n",
-      "Requirement already satisfied: idna>=2.8 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anyio<5,>=3.5.0->anthropic==0.35.0) (3.6)\n",
-      "Requirement already satisfied: httpcore<2.0.0,>=1.0.5 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (1.0.5)\n",
-      "Requirement already satisfied: packaging<25.0,>=24.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (24.1)\n",
-      "Requirement already satisfied: protobuf<6.0.0,>=3.20.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (4.24.3)\n",
-      "Requirement already satisfied: python-dateutil>=2.8.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (2.8.2)\n",
-      "Requirement already satisfied: certifi in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from httpx<0.28.0,>=0.20.0->e2b_code_interpreter==1.0.0) (2024.8.30)\n",
-      "Requirement already satisfied: h11<0.15,>=0.13 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from httpcore<2.0.0,>=1.0.5->e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (0.14.0)\n",
-      "Requirement already satisfied: annotated-types>=0.6.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->anthropic==0.35.0) (0.7.0)\n",
-      "Requirement already satisfied: pydantic-core==2.23.3 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->anthropic==0.35.0) (2.23.3)\n",
-      "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from tokenizers>=0.13.0->anthropic==0.35.0) (0.25.1)\n",
-      "Requirement already satisfied: filelock in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (3.15.4)\n",
-      "Requirement already satisfied: fsspec>=2023.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (2023.6.0)\n",
-      "Requirement already satisfied: pyyaml>=5.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (6.0.1)\n",
-      "Requirement already satisfied: requests in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (2.31.0)\n",
-      "Requirement already satisfied: tqdm>=4.42.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (4.66.2)\n",
-      "Requirement already satisfied: six>=1.5 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from python-dateutil>=2.8.2->e2b<2.0.0,>=1.0.0->e2b_code_interpreter==1.0.0) (1.16.0)\n",
-      "Requirement already satisfied: charset-normalizer<4,>=2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (3.3.2)\n",
-      "Requirement already satisfied: urllib3<3,>=1.21.1 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic==0.35.0) (2.2.1)\n",
+      "Requirement already satisfied: e2b_code_interpreter==2.9.0 \"e2b<2.38\" in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (1.0.0)\n",
+      "Requirement already satisfied: anthropic==0.121.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (0.35.0)\n",
+      "Requirement already satisfied: python-dotenv==1.2.2 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (1.0.1)\n",
+      "Requirement already satisfied: attrs>=21.3.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==2.9.0 \"e2b<2.38\") (23.2.0)\n",
+      "Requirement already satisfied: e2b<2.0.0,>=1.0.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==2.9.0 \"e2b<2.38\") (1.0.1)\n",
+      "Requirement already satisfied: httpx<0.28.0,>=0.20.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from e2b_code_interpreter==2.9.0 \"e2b<2.38\") (0.27.0)\n",
+      "Requirement already satisfied: anyio<5,>=3.5.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (3.7.1)\n",
+      "Requirement already satisfied: distro<2,>=1.7.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (1.8.0)\n",
+      "Requirement already satisfied: jiter<1,>=0.4.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (0.4.1)\n",
+      "Requirement already satisfied: pydantic<3,>=1.9.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (2.9.1)\n",
+      "Requirement already satisfied: sniffio in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (1.3.0)\n",
+      "Requirement already satisfied: tokenizers>=0.13.0 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (0.20.0)\n",
+      "Requirement already satisfied: typing-extensions<5,>=4.7 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anthropic==0.121.0) (4.12.2)\n",
+      "Requirement already satisfied: idna>=2.8 in /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages (from anyio<5,>=3.5.0->anthropic==0.121.0) (3.6)\n",


🟡 sweep:e2b_code_interpreter==2.9.0 "e2b<2.38"
The e2b version bump was done as a blind find-replace across .ipynb JSON, corrupting cached stdout 'outputs' blocks (not just the live %pip source line). E.g. examples/claude-code-interpreter-python/claude_code_interpreter.ipynb line 32 now shows the bogus concatenated spec e2b_code_interpreter==2.9.0 "e2b<2.38" as one requirement, with the resolved version still reading the stale (1.0.0). Clearing/re-running the cached outputs in each matched notebook resolves it.

Extended reasoning...

The version bump in this PR was applied as a plain text search-and-replace across the raw .ipynb JSON files rather than only touching the executable %pip install source cells. Jupyter notebooks store the text a cell printed the last time it was executed inside a separate outputs array in the JSON — that text is static, pre-rendered content that GitHub and nbviewer display without re-running anything. The find-replace didn't distinguish between "code that will run" and "text that was captured from a previous run," so it rewrote both.

The clearest example is examples/claude-code-interpreter-python/claude_code_interpreter.ipynb. Before the PR, the cached output (from a prior real pip install run) read:

Requirement already satisfied: e2b_code_interpreter==1.0.0 in .../site-packages (1.0.0)

The blind replace turned the substring e2b_code_interpreter==1.0.0 into e2b_code_interpreter==2.9.0 "e2b<2.38" everywhere it appeared, including here, producing:

Requirement already satisfied: e2b_code_interpreter==2.9.0 "e2b<2.38" in .../site-packages (1.0.0)

This is nonsensical on its face: e2b_code_interpreter==2.9.0 and "e2b<2.38" are two distinct pip arguments (the package being installed and a separate version constraint on one of its dependencies) that got concatenated into what reads as a single garbled package spec, while the version shown in parentheses at the end — the actually-resolved version — still says the stale 1.0.0. The same corruption pattern hits every from ...->anthropic==... dependency-origin line and similar lines derived from the old pinned versions, so the cached log is now internally self-contradictory (e.g. it also still shows anthropic resolving to (0.35.0) even though the pin text now says anthropic==0.121.0).

Nothing in the existing code or notebook tooling would have caught this: %pip install source cells are plain text cells with no static analysis, and notebook outputs blocks are inert JSON blobs — nothing checks that they're consistent with the source cell that (previously) produced them. A generic string-replace across the whole file has no way to know that one occurrence is "code to execute" and another is "a transcript of a previous execution" — both look identical as strings.

The impact is purely cosmetic/confusing rather than functionally broken: nobody's code execution changes, because the actual %pip install line is syntactically valid and will install the correct pinned versions when the notebook is next run. But anyone reading the notebook on GitHub or in a rendered form (which is the common way these cookbook notebooks are consumed, without executing them) sees a bogus, self-contradictory install log that looks like a broken package name and casts doubt on whether the pin is even correct.

To verify concretely: open examples/claude-code-interpreter-python/claude_code_interpreter.ipynb at line 29-45 in the PR diff. Cell input (line ~45) correctly reads %pip install e2b_code_interpreter==2.9.0 "e2b<2.38" anthropic==0.121.0 python-dotenv==1.2.2 — that part is fine and unambiguous as source. But the outputs block just above it (starting at line 32) still contains the previous run's printed text, now partially rewritten by the same substitution rule, producing the garbled/contradictory strings described above. This same substitution artifact (e2b_code_interpreter==2.9.0 "e2b<2.38" appearing inside a cached output block) recurs in several other notebooks touched by this PR wherever a stale pip-install transcript was left in place rather than cleared.

The fix is mechanical and low-risk: for every notebook where this pattern appears in a cached outputs array (not the source array), clear that cell's stored output (set outputs: [] and execution_count: null), since it's stale artifact data anyway and doesn't affect notebook execution. This is exactly the kind of match-and-fix sweep the sweep: expression at the top of this comment is meant to drive across every affected file.

claude · 2026-08-12T16:26:34Z

@@ -0,0 +1 @@
+{"fileNames":[],"fileInfos":[],"root":[],"options":{"allowJs":true,"esModuleInterop":true,"module":99,"skipLibCheck":true,"strict":true,"target":9},"errors":true,"version":"5.9.3"}


🟡 This PR adds examples/groq-code-interpreter-js/tsconfig.tsbuildinfo, which is TypeScript's incremental build cache (generated output, not source) — no other example in the repo commits one. Either add *.tsbuildinfo to .gitignore or remove the file.

Extended reasoning...

The diff adds examples/groq-code-interpreter-js/tsconfig.tsbuildinfo as a brand-new file (@@ -0,0 +1 @@). Its entire contents are {"fileNames":[],"fileInfos":[],"root":[],"options":{...},"version":"5.9.3"} — this is exactly the shape of TypeScript's --incremental build-info cache, which tsc writes automatically next to tsconfig.json after a compile. It is generated output derived from a build run, not something a contributor authors by hand.

The likely cause is that this PR bumped TypeScript to 5.9.3 for this example (visible elsewhere in the diff) and ran tsc locally to verify the bump works, and the resulting .tsbuildinfo artifact got picked up by git add alongside the intended package.json/package-lock.json changes.

Checking the rest of the repository confirms this is inconsistent with existing practice: git ls-files '*.tsbuildinfo' returns only this one file — no other TypeScript example (and there are many: claude-code-interpreter-js, codestral-code-interpreter-js, openai-js, etc.) commits its build cache. Neither the root .gitignore nor any example-level .gitignore excludes *.tsbuildinfo, so nothing currently stops this from happening again on the next local tsc run in this or other example directories.

Impact is low — the file doesn't break anything functionally, it's just a stray build artifact that adds noise to the diff and will silently go stale (or get regenerated with different content) on the next contributor's local build, creating repeated unnecessary churn in future PRs touching this example.

Fix: either delete examples/groq-code-interpreter-js/tsconfig.tsbuildinfo from this PR, or (better, to prevent recurrence) add a *.tsbuildinfo entry to the root .gitignore.

Proof: run git show <this-commit> --stat and observe tsconfig.tsbuildinfo listed as a new file; cat examples/groq-code-interpreter-js/tsconfig.tsbuildinfo shows only the TS incremental-cache JSON structure with no example-specific logic; git ls-files '*.tsbuildinfo' across the repo returns exactly one match (this file), confirming no precedent for committing it elsewhere.

- groq-code-interpreter-js still called the decommissioned llama3-70b-8192. The model sweep globbed *.ts but not *.mts, so this one file was skipped even though I hand-edited it for the groq-sdk import fix. Now llama-3.3-70b-versatile, matching its Python twin and its own README. Checked every .mts/.cts/.mjs in the repo; this was the only affected file. - The notebook pin bump was a text replace over raw .ipynb JSON, so it also rewrote cached pip logs from 2024 runs, leaving self-contradictory transcripts like 'e2b_code_interpreter==2.9.0 "e2b<2.38" ... (1.0.0)'. Those logs were stale regardless, so cleared the output on the five affected cells rather than trying to rewrite them. - Removed a stray tsconfig.tsbuildinfo that my local typecheck run produced, and added *.tsbuildinfo to .gitignore so it cannot recur. - Install uv from PyPI instead of piping a remote installer to sh. The sandbox gets provider API keys moments later, so an unpinned remote script does not belong in that path. Note the pre-existing poetry branch has the same shape and is left alone here.

beran-t · 2026-08-12T16:38:47Z

All four addressed in 220c0da.

🔴 groq-code-interpreter-js still on llama3-70b-8192 — correct, and the good catch of the four. My model sweep globbed *.ts but not *.mts, so this file was skipped by the automation and I hand-edited it for the groq-sdk import fix without noticing the model two lines below. Now llama-3.3-70b-versatile, with the commented alternative on llama-3.1-8b-instant, matching the Python twin. I swept every .mts/.cts/.mjs in the repo afterwards — this was the only affected file — and re-ran the stale-ID check across all file types, which is now clean.

🟡 Corrupted cached pip logs — also correct. The pin bump was a text replace over raw .ipynb JSON, so it rewrote captured output from 2024 runs alongside the live %pip lines, producing exactly the self-contradictory transcript described. Five notebooks affected. Those logs were stale before this PR anyway, so I cleared the output on the affected cells rather than rewriting fiction into them.

🟡 tsconfig.tsbuildinfo — right, that was my local typecheck run leaking into the commit. Removed, and *.tsbuildinfo added to .gitignore so it can't recur in any example.

🔒 curl | sh in the uv path — fair. Changed to pip install --quiet uv, since that sandbox receives provider API keys moments later and an unpinned remote installer doesn't belong in that path. Worth flagging that the pre-existing poetry branch two lines up has the same shape (curl -sSL https://install.python-poetry.org | python3 -); I left it alone to keep this PR's scope honest, but it's the same exposure and worth its own change.

Dispatching the workflow on this branch surfaced that the suite could no longer start at all: SyntaxError: Cannot use import statement outside a module node_modules/e2b/node_modules/chalk/source/index.js e2b 2.38.3 depends on chalk 5 plus four other ESM-only packages (glob, minimatch, brace-expansion, balanced-match). Node 22+ loads those fine from both CJS and ESM - verified directly, plain 'require("e2b")' and 'import { Sandbox } from "e2b"' both work - but Jest's own module runtime does not implement require(esm), so it fails where Node does not. Switching Jest to its ESM preset did not help: it still routes resolution through that runtime. The old e2b 0.16 beta had no ESM-only dependencies, which is why this only appeared once the SDK was bumped. So the runner is now a plain script under tsx. Same behaviour: upload each example into a fresh sandbox, run its toolchain, pass on exit 0. It writes tests/results.json in the shape updateTestsMd.js already consumes, so the reporting and Slack steps are untouched. Drops jest, ts-jest, ts-node and @types/jest. Three things improved while rewriting: - Only transient failures are retried now (rate limits, 5xx, timeouts). The old loop retried everything three times, so a decommissioned model burned three sandboxes and three provider calls to fail identically each time. - Failures are recorded with their stderr. commands.run throws on a non-zero exit, so the old exitCode check was unreachable and failures wrote nothing. - The watsonx entry pointed at granite_code_interpreter.ipynb, which does not exist; the file is granite_code_interpreter_py.ipynb. That is why it failed in 1ms every run. Fixed. 'npm test -- <substring>' now runs a subset, which is how the two hello-world examples were verified against live E2B before pushing this. Also fixes the root tsconfig, which declared module: commonjs while package.json declared type: module, and removes a stray typescript dependency my local typecheck run had added.

tests/results.json from a local run slipped into the previous commit. It is generated output consumed by updateTestsMd.js and uploaded as an artifact, so it does not belong in the repo. Gitignored along with tests/Tests.txt.

…posed The run went 13/36. Sorting my mistakes from what was already broken: Mine: - Every gpt-5.6-* model is reasoning-first and rejects function tools on /v1/chat/completions unless reasoning_effort is 'none'. I retargeted the tool-calling examples onto them without checking that. The API names the remedy in its own error, so that is what is applied, in openai-js, gpt-4o-js, the two OpenAI notebooks, langchain, langgraph and crewai. - codestral-25-08 is the version label on Mistral's model page, not the API id. The API rejects it with 'Invalid model'. The id is codestral-2508. - agentkit-coding-agent starts a server on port 3000 and never exits, so adding it to the test list could only ever produce a timeout. Removed. - The workflow never defined WATSONX_API_KEY / WATSONX_PROJECT_ID / WATSONX_URL even though utils.ts forwards them, so watsonx-ai-code-interpreter-js failed on 'Missing required parameters: apikey'. Added. Not mine, now documented rather than silently failing: - sandbox-agent-sdk-js depends on the third-party sandbox-agent package, whose latest release (0.4.2) still calls Sandbox.betaCreate(). The E2B SDK removed that between 2.20 and 2.30, so the example cannot run against any current SDK. It was already failing on main for this reason. Removed from the list with the reason recorded. - mcp-custom-template-js needs its template built first and fails with 'template browserbase-mcp-gateway not found'. Moved to the documented exclusions with the other template-dependent examples. Test list is 33 examples.

cursor

Stale comment

cursor · 2026-08-12T17:16:43Z

+      uses: actions/upload-artifact@v4
+      with:
+        name: example-logs
+        path: ./logs


🔒 Agentic Security Review
Severity: MEDIUM

The new example-logs artifact upload persists raw per-example stdout/stderr from runs that execute with live provider API keys. Because those logs are written verbatim from sandbox output, a failing or prompt-injected example can print secrets and make them retrievable from workflow artifacts.

Impact: provider credentials used by CI can be exposed via stored artifacts and then reused to access external model APIs.

^{Reviewed by Cursor Security Reviewer for commit 0372686. Configure here.}

…ackages Both examples were left behind in the first pass. Neither is blocked any more. nextjs-code-interpreter: Vercel AI SDK v3 -> v7, and the stack around it. ai ^3.1.1 -> ^7.0.63 @ai-sdk/openai ^0.0.9 -> ^4.0.41 @ai-sdk/react (new) -> ^4.0.66 useChat moved out of ai/react next ^14.2.3 -> ^16.3.0 react 18 -> ^19.2.8 eslint ^7.32.0 -> ^9.17.0 zod (new) -> ^4.4.3 tool input schemas openai removed the provider package covers it now The whole streaming path is different, so this is a rewrite rather than a bump: - OpenAIStream / StreamingTextResponse / StreamData / ToolCallPayload / experimental_onToolCall / appendToolCallMessage are all gone. The route is now streamText() with a declared tools map and toUIMessageStreamResponse(). - The manual second chat.completions.create() call that fed the tool result back is replaced by stopWhen: stepCountIs(5), which is what makes the model answer after the tool instead of the stream ending on the tool call. - convertToModelMessages is async in v7 and needs awaiting. - On the client, useChat no longer owns the input box and takes its endpoint via DefaultChatTransport. A v5 message is a list of parts, so tool output renders off part.type === 'tool-execute_python_code' and its state rather than out of a side-channel StreamData array keyed by message index. - generateId() no longer takes a size argument. - Tool results are typed with zod instead of a hand-written JSON schema. Verified: tsc --noEmit clean and succeeds, all four routes generated. Not driven in a browser, so the UI itself is unexercised. vercel-eve-feedback-analyst-js: this did not install at all, on this branch or on main. @e2b/eve-sandbox 0.1.1 declares a peer of eve@^0.27.0, which excludes the eve release the example is written against and every release since (latest 0.33.3). The peer range is stale rather than a real incompatibility, so the example carries a commented .npmrc with legacy-peer-deps and eve, ai, @ai-sdk/anthropic and e2b move to current. It now installs and typechecks clean. The right fix is for @e2b/eve-sandbox to widen its peer range, and the .npmrc says so. Both stay out of the test list: they run long-lived server processes, which this runner can only ever time out on. The exclusion notes now say that rather than citing reasons that no longer apply.

A sweep found 83 dependency declarations at least one major behind, almost all tooling rather than the SDKs the examples are about. typescript 5.1 / 5.4 / 5.9 -> ^7.0.2 @types/node 17 / 20 / 22 / 24 -> ^26.2.0 dotenv ^16.x -> ^17.4.2 eslint ^7.32 / ^9.17 -> ^10.8.1 eslint-config-next 13.4.12 -> ^16.3.0 @typescript-eslint/*, typescript-eslint ^7.x -> ^8.67.0 globals ^15.x -> ^17.11.0 http-proxy-middleware ^3.0.5 -> ^4.2.0 open ^10.2.0 -> ^11.0.0 inngest 3.46.0 -> ^4.18.0 nanoid 5.1.11 -> ^6.0.1 TypeScript 7 removed target ES5 and the node/node10 module resolution modes, so six tsconfigs needed updating to ES2022 and bundler resolution. While there, groq-code-interpreter-js only included **/*.ts but its source is a .mts file, so tsc had been silently checking nothing in that example. Two exceptions, both because the ecosystem has not caught up rather than by preference: codestral-code-interpreter-js and together-ai-code-interpreter-js stay on typescript ^5.9.3, because typescript-eslint 8 declares a peer of typescript >=4.8.4 <6.1.0 and npm will not resolve past it. hello-world-js also carried jest, ts-jest and @types/jest while its test script was the npm-init placeholder that exits 1. Dropped. Verified: all 30 JS projects install from scratch, every one typechecks clean, nextjs-code-interpreter builds, and both hello-world examples still run against live E2B.

…as no key Second CI run went 18/33, up from 13/36. reasoning_effort 'none' fixed gpt-4o-js, gpt-4o-python, langchain and langgraph, and codestral-2508 fixed codestral-js. What the remaining failures showed: gpt-5.6-sol rejects reasoning_effort 'none' outright: Unsupported value: 'reasoning_effort' does not support 'none' with this model. Supported values are: 'low', 'medium', 'high', and 'xhigh'. So sol cannot use function tools on /v1/chat/completions at all, while terra can (that is why gpt-4o-js passed and openai-js did not). Every tool-calling example moves to gpt-5.6-terra. This is the second half of the mapping mistake: picking the reasoning flagship for examples whose whole point is function calling. openai-js also read codeInterpreterResults[0].png without checking the array had anything in it, so a run where the model returned no chart crashed on undefined instead of saying so. Guarded. Five examples are dropped from the test list because the repo has no key for them. `gh secret list` shows only ANTHROPIC, E2B, FIRECRAWL, FIREWORKS, GROQ, MISTRAL, OPENAI and TOGETHER, so watsonx (both), mcp-browserbase-js, mcp-research-agent-js and stirrup-python could only ever fail on a missing credential. Each is listed with the secret it needs so adding one is enough to move it back. The WATSONX_* env block stays in the workflow with a note that the secrets do not exist yet, rather than reading as working config. Test list is 28 examples, all of which have the credentials to run.

Third run: 19/28, up from 18/33 and 13/36. Moving the tool-calling examples to gpt-5.6-terra fixed openai-js and both o1 examples. Three things left that were mine: openai-python was still on a bare model="o3", with gpt-5.6-terra sitting commented out beside it. My original sweep matched o1 and o3-mini but not a bare o3, so it slipped through both that pass and the sol->terra pass. o3 is a reasoning model and rejects reasoning_effort 'none', which is exactly the error the run reported. Swapped: terra is active, o3 is the commented alternative with a note saying why it cannot be used with tools here. custom-sandbox-domain-proxy had been passing and my dependency sweep broke it. http-proxy-middleware 4 requires Node ^22.15.0 || ^24.0.0 || >=26.0.0 and reaches for styleText from node:util (Node 20.12+), but the E2B base sandbox ships Node 20.9, so the example died on a missing export. Held at ^3.x, with open at ^10.x for the same reason, and the README says what to wait for. crewai-python still sent its request without reasoning_effort. CrewAI does hold and forward the field - verified against the installed 1.15.5 - so the drop is LiteLLM stripping params for a model its map does not know yet. Added allowed_openai_params to force it through. A full sweep of every model assignment across .ts, .py, .mts and notebook source cells now comes back clean apart from ibm/granite-34b-code-instruct, which stays as-is because IBM's lifecycle page is not publicly reachable.

Five of the eight remaining CI failures were the same thing: these examples ask an LLM to plot something, then assume it did. `results` only holds Jupyter display output, so it is empty whenever the model printed instead of rendering, or split its work across a second tool call that these single-turn examples never make. That is normal model behaviour, not a broken example, so the runs were flapping - claude-code-interpreter-js passed one run and failed the next on identical code. Three shapes, all now reporting and continuing instead of dying: - Unguarded index. claude-code-interpreter-js dereferenced results[0] directly, and firecrawl-scrape-and-analyze-airbnb-data and groq-code-interpreter-js read .text/.formats() off it. Their siblings already did `result && result.png`, which is why those passed. - sys.exit()/exit() inside a notebook cell raises SystemExit, which nbconvert reports as a failed cell. fireworks-code-interpreter-python and groq-code-interpreter-python both did this on the no-results path. - A deliberate `raise Exception("No code interpreter results")`. Reasonable in a script, but it makes the notebook fail for a non-error. Fixed in codestral-code-interpreter-python, and in o1-and-gpt-4-python, together-ai-code-interpreter-python and upload-dataset-code-interpreter, which carry the same line and were passing only by luck. Also reverted the crewai allowed_openai_params attempt. It is not accepted on Completions.create(), so it made that example fail earlier than before rather than fixing it. The reasoning_effort setting stays with a comment recording that LiteLLM appears to drop it for models its map does not know, so crewai remains a known failure rather than a blind workaround. Every notebook code cell still compiles and all 22 are valid JSON.

…ones Follow-up to the previous commit, which only touched examples that happened to be red. Swept all of them for the same assumption - that the model always does what the prompt asked - and drew a consistent line through it. The line: a missing chart is a normal model outcome and must not fail the example, because the sandbox still ran the code, which is what these examples exist to demonstrate. A missing tool call is different - there was no code to run, so nothing was demonstrated - and stays a failure. - watsonx-ai-code-interpreter-js read response.result.results[0].generated_text with no guard, so an empty generation crashed on undefined. Now checks and exits with a message. - together-ai, codestral, gpt-4o-js and o1-and-gpt-4-js already handled a missing chart, but reported it as "Error: No PNG data available" - one of them on stderr. It is not an error. Reworded consistently across all four, so logs stop implying a failure that did not happen. - claude-code-interpreter-js and openai-js threw "Tool use block not found in message content", which reads like an SDK problem when it means the model answered in prose. Kept as failures deliberately, with messages that say what actually happened and a comment recording why this case is treated differently from a missing chart. Left alone on purpose: `throw new Error('Dataset file not found')` and the missing-E2B_API_KEY throw are real errors. choices[0] on a chat completion is safe - the APIs always return at least one choice. Verified: every JS example typechecks, all 22 notebooks are valid JSON, all 117 notebook code cells compile, and no exit()/raise-on-empty-results remains anywhere in the repo.

cursor

Stale comment

Identified 1 net-new security finding after triage/dedup:

HIGH tests/run-examples.ts: the Poetry bootstrap command executes curl -sSL https://install.python-poetry.org | python3 - in the same sandbox command context that injects provider secrets (envs: getApiKeys()). A compromised installer delivery path can exfiltrate those credentials before tests execute.

A previously reported artifact-leak finding on .github/workflows/update-tests-md.yml remains not addressed and was not reposted as a duplicate.

_{Sent by Cursor Security Agent: Security Reviewer}

Dug the tracebacks out of the nbconvert logs instead of guessing. They were there all along - buried in a `'traceback': [...]` blob my earlier greps did not match - and each failure had a specific cause rather than being generic flakiness: - codestral-code-interpreter-python: ImportError, cannot import name 'Mistral' from 'mistralai'. My Mistral rewrite used the import path from the JS SDK. mistralai 2.x has no top-level __init__.py, so the client lives at mistralai.client. Verified against a clean install of 2.9.2: `from mistralai.client import Mistral` works and chat.complete is present. - claude-code-interpreter-python: IndexError on code_interpreter_results[0]. The same unguarded index I fixed in its siblings; I missed this notebook, which is why it flipped from passing to failing between runs. - groq-code-interpreter-python: APIConnectionError from inside the sandbox, a transient network blip the retry classifier did not match. Added APIConnectionError, connection reset, socket hang up, ENOTFOUND and EAI_AGAIN to the retryable set. - mcp-custom-server-js: deadline_exceeded. It clones and builds a filesystem MCP server from GitHub inside the sandbox, which does not fit the shared 150s budget. Added a per-example timeout override and gave it 300s, with the sandbox lifetime raised to match so the VM outlives the command. fireworks-code-interpreter-python comes out of the list rather than being fixed. qwen2p5-coder-32b-instruct returns 404 "Model not found, inaccessible, and/or not deployed", which does not distinguish a retired model from one this account has not deployed, and I have no Fireworks key to tell them apart. Recorded with the exact error so whoever has the account can pick a current model. crewai-python also stays out of scope: reasoning_effort is set but LiteLLM appears to drop it for models its map does not know, and that needs an OpenAI key to iterate against rather than another blind attempt. Test list is 27.

…w example 24/27 after the previous commit. Both remaining infrastructure failures had been misdiagnosed, so this fixes the diagnosis rather than the examples. groq-code-interpreter-python failed APIConnectionError on all three attempts, which looked permanent. It is not the dependencies and not sandbox connectivity: probing inside a real E2B sandbox, replicating the exact poetry + pip path the runner uses, the Groq client reaches api.groq.com and returns AuthenticationError on a dummy key with httpx 0.28.1. Its JS sibling also passed in the same run against the same API. What is left is a short upstream blip, and a flat 10s backoff put all three attempts inside roughly 30 seconds, so anything lasting half a minute reads as permanent. Backoff is now 10s / 30s / 90s, which spans about two minutes. I cannot prove it was a blip - only that the two hypotheses I could test are both wrong. If it recurs with the wider window, the cause is somewhere I have not looked. mcp-custom-server-js creates its own E2B sandbox and then clones and builds a filesystem MCP server from GitHub inside it. 300s was not enough either; its budget is now 600s. It does exit cleanly, so this is slowness rather than the long-running-server case the excluded examples hit.

…ironment The APIConnectionError was not a connection problem. Pulling the nested traceback out of the nbconvert log gives the real cause: LocalProtocolError: Illegal header value b'Bearer ' An empty bearer token. The notebook hardcodes GROQ_API_KEY = "" E2B_API_KEY = "" and never reads os.environ, so it could never have worked in CI - the key the workflow passes into the sandbox was ignored. It is the only notebook in the repo that does this; every sibling, including the other Groq one (upload-dataset-code-interpreter), uses load_dotenv() plus os.getenv and passes. Now it matches them. Sandbox.create() still worked because the E2B SDK falls back to the E2B_API_KEY environment variable when handed a falsy key. Groq's client does not - an explicit empty string is used as-is - which is why only the Groq call failed and why it looked like a network fault. This also retires my previous explanation. I attributed it to a short upstream blip narrowed by a flat retry window, having ruled out the dependencies and sandbox connectivity by probing inside a real sandbox. Both of those were genuinely fine, but the conclusion was wrong: it was never transient, and no amount of backoff would have fixed it. The wider backoff stays, since it did rescue gpt-4o-code-interpreter in the same run.

…ead of guessing gpt-4o-python failed with TypeError: 'NoneType' object is not subscriptable at plot1/plot2 = code_interpreter_results[0]. Its chat helper returns None rather than [] when the model makes no tool call, so the guard has to cover None too. This was the third run in a row where I fixed this class of bug in one notebook and the next run surfaced it in another. Rather than continue one at a time, I wrote an audit that finds every place a results-shaped variable is subscripted and checks whether a guard precedes it, counting inline ternaries as guards. It reported gpt-4o-python's two sites and nothing else; after this commit it reports zero. That check is the reason to believe this is the last of them, rather than another single fix and another run to find out.

…t survives Read CrewAI's source instead of guessing at it this time. litellm is not even installed - crewai 1.15.5 ships a native OpenAI provider - so my previous explanation was wrong. The actual cause is in crewai/llms/providers/openai/completion.py: line 269: data["is_o1_model"] = "o1" in model.lower() line 1594: if self.is_o1_model and self.reasoning_effort: params["reasoning_effort"] = self.reasoning_effort CrewAI decides whether a model is a reasoning model by testing for the literal substring "o1" in its name. gpt-5.6-luna does not contain it, so the parameter is dropped from the chat-completions request, the API applies the model's default reasoning effort, and function tools are rejected. That is why the 400 names 'param': 'reasoning_effort' for a value the request never carried, and why langchain and langgraph pass on the same model - they forward it themselves. The Responses path in the same file sends it unconditionally: line 745: if self.reasoning_effort: params["reasoning"] = {"effort": self.reasoning_effort} so the example now asks for api="responses", which is also what the API's error message recommends. Verified by building the crew with a mocked tool: the agent's LLM comes out as OpenAICompletion with api=responses, reasoning_effort=none and is_o1_model=False. Only the live call is unverified - no OpenAI key here. Unlike my earlier attempts at this example, the diagnosis is read off the installed source rather than inferred, and the substring heuristic is worth reporting to CrewAI regardless of whether this route works.

…n a defect crewai-python passes now - routing it through the Responses API was the right fix, and reasoning_effort reaches the request. groq-code-interpreter-js failed in the same run on something new: 400 tool_use_failed - Failed to call a function. Please adjust your prompt. failed_generation: <function=execute_python,{"code": "..."}</function> llama-3.3-70b-versatile emitted that pseudo-XML instead of tool-call JSON and Groq rejected it. The generated Python inside it is perfectly good; only the envelope was malformed. The same code passed in earlier runs, so this is model variance, not a defect in the example, and no edit to the example would prevent it recurring. So tool_use_failed joins the retryable set. It is the one 400 worth retrying: every other 400 seen on this branch - a retired model id, an unsupported parameter - fails identically forever, while this one is a coin flip on each generation. The classifier comment records that distinction so the exception does not later get generalised into "retry all 400s", which is how the original blanket-retry wasted sandbox time.

… covered Coverage was 27 of 47 examples. Six were excluded only because they create a sandbox from a custom template the runner never built, and docker-in-e2b was excluded for having js/ and python/ subprojects. All eight are now covered, so 35 of 47. Builds are keyed by template alias rather than by example, because two aliases are shared: anthropic-claude-code is built by both siblings, e2b-with-docker by both docker-in-e2b subprojects, and openai-codex-in-sandbox-js has no build script at all - it relies on the python sibling's. Keying by example would have raced two builds of the same alias. Only the prod builds are used. Every `:dev` script builds a `<alias>-dev` template that no example creates a sandbox from - playwright's dev build makes playwright-chromium-dev while app.ts asks for playwright-chromium - so wiring the dev scripts would have built the wrong thing and left the examples failing on a missing template. A build is minutes and the result persists on the account, so each alias is probed first by trying to create a sandbox from it and only built when that fails. Steady state adds five short probes, not five builds. REBUILD_TEMPLATES=1 forces a rebuild, which is the one case the probe cannot detect: a changed template definition. A filtered run only builds what its selection needs, so `npm test -- hello-world` still builds nothing. Verified against live E2B rather than assumed: with browserbase-mcp-gateway absent, the probe missed, the build ran, and mcp-custom-template-js then created a sandbox from the new alias and passed. Re-running reported "already exists, skipping build" and passed again. Also corrected two exclusion notes I had written from assumption. docker-in-e2b turned out to be template-dependent rather than a plain subproject split, and openai-agents-sdk is not merely missing a manifest: basic.py does sys.path.insert(...parents[4]) and imports examples.sandbox.misc.example_support, so it expects to run inside the openai-agents-python repo and cannot run here at all.

cursor

Stale comment

cursor · 2026-08-14T14:53:14Z

+}
+
+function testScript(interpreter: Interpreter, notebookPath: string): string[] {
+  const INSTALL_POETRY_COMMAND = 'curl -sSL https://install.python-poetry.org | python3 -'


🔒 Agentic Security Review
Severity: HIGH

INSTALL_POETRY_COMMAND executes a remote installer script (curl ... | python3 -) inside the same execution flow that later injects provider API keys via getApiKeys(). This creates a supply-chain trust boundary where compromise of the installer delivery path can execute attacker-controlled code before/around secret-bearing runs.

Impact: compromised installer content can exfiltrate model-provider credentials and tamper with CI example execution.

^{Reviewed by Cursor Security Reviewer for commit 9350e7a. Configure here.}

29 of 35 on the first run with template builds. All 27 previously-passing held and two of the eight new ones passed; the other six each failed for a different reason, and three of those were mine. Mine: - openai-codex-in-sandbox-python: `Failed to spawn: main.py`. The uv adapter assumed ./main.py by convention; this project's entry is src/openai_codex_in_sandbox_python/main.py. Added a per-example entrypoint. - anthropic-claude-code-in-sandbox-python: uv could not resolve, because I bumped its e2b pin to >=2.38 (which needs Python >=3.10) while leaving requires-python at >=3.9. uv printed exactly this warning when I relocked and I did not act on it. Now >=3.10. - docker-in-e2b-python: `Readme path /home/user/example/README.md does not exist`. Its pyproject declares readme = "README.md" but that file lives in the parent directory, so installing the project itself fails once only python/ is uploaded. Poetry examples now install with --no-root; they are scripts, not libraries. Not mine: - playwright-in-e2b asked for `timeoutMs: 15000` on its own sandbox - 15 seconds to launch Chromium, run a Playwright script and read the output files back. It died in Filesystem.read after the sandbox had already expired. Raised to 120s. This would have been flaky for anyone running it, not just in CI. - openai-codex-in-sandbox-js timed out: it hands a real task to the Codex CLI inside the sandbox. Given a 600s budget, along with the python sibling. And one finding about what I shipped in the previous commit. The anthropic-claude-code template already existed on the CI account, so the probe skipped the build, and the example then failed with 404 {"type":"not_found_error","message":"model: claude-sonnet-4-20250514"} for a model id that appears nowhere in this repo. It comes from the template itself, built with a Claude Code version whose default model has since been retired. The probe answers "does this alias exist", not "is it current", and templates that bundle a fast-moving agent CLI go stale by construction. REBUILD_TEMPLATES=1 is the only way to refresh one, so it is worth running periodically rather than only after editing a template definition. That is now written next to the probe instead of being implied by an escape hatch.

The previous commit documented REBUILD_TEMPLATES=1 as the way to refresh a stale template, but nothing could set it: the workflow had no dispatch input and no env wiring, so the escape hatch existed only for someone running the runner locally. That is how the stale anthropic-claude-code template would have stayed stale indefinitely while the comment claimed there was a remedy. Adds a rebuild_templates boolean to workflow_dispatch and threads it through as REBUILD_TEMPLATES. Scheduled runs still take the cheap path and reuse whatever exists.

… never ran 30 of 35 with templates rebuilt. The rebuild confirmed the stale-template diagnosis: anthropic-claude-code-in-sandbox-js passes now that its template is current, and playwright-in-e2b passes with a workable sandbox timeout. The five remaining failures were four of my runner's assumptions plus one real bug in an example. My assumptions, all about how a python project is entered: - docker-in-e2b-python: `Command not found: start`. The poetry adapter always ran `poetry run start`; this project has no scripts section at all, just main.py. - anthropic-claude-code-in-sandbox-python: `Failed to spawn: main.py`. Its entry is inside the package and imports absolutely (`from anthropic_claude_code_in_sandbox.template import ...`), so it needs `python -m anthropic_claude_code_in_sandbox.main`. - openai-codex-in-sandbox-python: entry is src/openai_codex_in_sandbox_python/main.py. So `entrypoint` now means "what to pass after uv run / poetry run" rather than a path, which covers a path, a console script, and `python -m pkg.mod`. Defaults stay main.py for uv and the start script for poetry. The real bug: openai-codex-in-sandbox-python's build_prod.py and build_dev.py sit at the project root with no __init__.py beside them and do `from .template import template`. A relative import with no parent package cannot work under any invocation - not as a script, not with -m - and the README never documents a build command, so this has presumably never been run by anyone. The template build failed with "attempted relative import with no known parent package". Now an absolute import. Verified locally: with the import fixed the openai-codex template builds, and the example then gets as far as the Codex CLI authenticating, failing only on the OPENAI_API_KEY I do not have here. CI does.

…flicted 429s 30 of 35, with the failure set moved rather than shrunk. The previous commit's entrypoint work landed - anthropic-claude-code-in-sandbox-python and docker-in-e2b-python pass, and all five templates including openai-codex now build - and two new causes surfaced. Both openai-codex examples pass a literal placeholder into the sandbox: envs={"OPENAI_API_KEY": "<your api key>"} # python envs: { OPENAI_API_KEY: '<your api key>' } # js so Codex inside the sandbox authenticates with the string "<your api key>" and gets 401 Unauthorized from wss://api.openai.com/v1/responses. Their anthropic-claude-code siblings read os.getenv, which is why those pass. Both now read the environment. Like the GROQ_API_KEY = "" case earlier, this could never have worked for anyone who did not edit the file first, and nothing tested it. The three Groq examples then all failed together with 429 Rate limit reached for model `llama-3.3-70b-versatile` in organization ... which is self-inflicted: the runner ran them concurrently against one org quota, and the retries landed inside the same exhausted window. Examples can now declare a provider, and examples sharing one are serialised against each other while everything else still runs at full concurrency. Tagged the four Groq consumers (the two code-interpreter examples, upload-dataset, and mcp-groq-exa). This is the fourth per-example field the runner has needed - entrypoint, timeout override, template alias, now provider. The manifest design I cut as over-engineering has been arriving one field at a time, and at this point the hand-maintained array is the thing making each new case a code change.

31 of 35. Serialising by provider fixed groq-code-interpreter-python. The four remaining failures resolve to two causes, and neither is code in this repo. The Codex CLI the template installs is v0.147.0, and it no longer authenticates from OPENAI_API_KEY in the environment: 401 Unauthorized: Missing bearer or basic authentication in header url: wss://api.openai.com/v1/responses "Missing", not "invalid" - no auth header is sent at all, so current Codex expects `codex login` or an explicit credential. Passing the key as an env was correct for the Codex of whenever these examples were written. My previous commit fixed a real bug there (they passed the literal string "<your api key>"), but fixing that only moved them from a wrong key to no key being used. Resolving it properly means either pinning Codex in the template or wiring its current auth, verified against the real CLI, so both examples are parked with that written down rather than guessed at a second time. The two remaining Groq failures are a daily quota, not concurrency: tokens per day (TPD): Limit 100000, Used 99952, Requested 752. Please try again in 10m Serialising helped but cannot help enough - four Groq examples making several calls each will exhaust 100k tokens a day, and I burned much of today's on repeated runs while iterating. A single nightly would likely fit. No retry policy fixes a daily cap, so this is an account-tier or example-count decision rather than something to fix in the runner, and the note next to the provider field says so. Test list is 33, of which the two Groq consumers are the only ones expected to fail and only when the day's quota is gone.

The suite has been asserting the wrong thing. Its job is to prove the sandbox worked - created, files uploaded, toolchain installed, commands executed, results returned, killed - and instead it has been failing whenever the model behaved differently between runs. That is why the same example passed and failed on consecutive runs with identical code, and why chasing "the last failure" kept moving it somewhere else. Outcomes are now three-way. The dividing line: if the code reached the sandbox and ran, the sandbox did its job whatever the code then did; if the example never got that far, that is a real failure. Skipped, reported but not failing the run: - provider capacity and quota (429, tokens per day, insufficient_quota) - a malformed tool call, or the model not calling the tool at all - nothing displayable produced (no chart) - code the model generated raising inside the sandbox - the sandbox executed it and faithfully returned the error, which is exactly the behaviour we want Still failures: - SDK errors, removed APIs, missing templates, upload failures - dependency resolution, wrong entrypoints, import errors - retired model ids and auth problems - configuration, not model behaviour - sandbox timeouts and anything where execution never started This reverses a call I made earlier. I had deliberately kept "the model never called the tool" as a failure, reasoning that nothing was demonstrated. Under the contract that the suite tests the sandbox, that is the model's choice and not a defect, so it is a skip. Verified the classifier against 13 real error strings collected from this branch's runs - the groq quota message, the tool_use_failed generation, the codex 401, the missing template, the uv resolution failure, Sandbox.betaCreate, the retired claude model id, the mistralai ImportError - and all 13 land on the intended side. results.json carries the skips as 'pending' and Tests.txt/Slack render them as "⏭️ Skipped (model)", so they stay visible rather than silently counted as passes.

The old chain was: updateTestsMd.js emits a bare 33-row table, a shell step reads it into $GITHUB_ENV through a heredoc, and rtCamp/action-slack-notify posts it. Three problems with that, all now fixed. It never said what happened. The runner prints "30 passed, 3 skipped, 0 failed" and none of that reached Slack - you got 33 rows and counted them yourself. The message now leads with the summary, and lists failed and skipped names in their own sections, with a line explaining that a skip means the sandbox worked and the model did something non-deterministic. It never linked back. A failure meant going to find the run. The message now carries branch, short sha, and a link to the run and its per-example logs. The webhook secret went to a third-party action. It is now a curl of a payload file we generate ourselves, so the secret stays in the job and the message is whatever tests/report.mjs wrote. That also removes the $GITHUB_ENV heredoc, which would have broken on any content containing a lone EOF line. report.mjs replaces updateTestsMd.js and writes both tests/Tests.txt (still the uploaded artifact) and tests/slack-payload.json (Block Kit). If results.json is missing it still emits a payload saying the runner crashed, because silence is precisely the failure mode this PR exists to remove - the old script logged an error and posted an empty table. The step is skipped rather than failed when SLACK_WEBHOOK is unset, so forks and manual runs do not go red over a missing secret. To point this at your own channel: create an incoming webhook in Slack, put the URL in the SLACK_WEBHOOK repo secret, and nothing else needs changing - the channel is whatever the webhook is bound to, so the old vars.SLACK_CHANNEL is no longer read. Verified by generating both files from a synthetic results.json covering all three states, and from a missing one.

The previous commit assumed an incoming webhook and sent Block Kit ({text, blocks}). The webhook in use is a Workflow Builder trigger, which is a different contract: it accepts only a flat object of the Data Variables declared on the trigger - no nesting, no arrays, every value a string - and would have rejected that payload outright. So the body is now seven Text variables, and the message itself is composed in Workflow Builder rather than here: status ✅ passed | ⚠️ passed with skips | ❌ failed | ❌ runner crashed summary 30 passed · 3 skipped · 0 failed (of 33) failed comma-separated names, or "none" skipped comma-separated names, or "none" branch chore/modernize-cookbook commit 4dbe12b run_url link to the run and its per-example logs Every key is always present and never an empty string, because a missing key is rejected and an empty one renders as a blank gap in the composed message. Name lists still cap at 12 and then read "and N more", so a bad run cannot produce a value long enough to be refused. The crash path keeps working: no results.json means status "❌ runner crashed" with the reason in failed, rather than posting nothing. Verified by generating the body for a mixed run, an all-green run and a missing results.json. The POST itself is still unexercised - it needs the real trigger URL.

The payload had nothing good to put in a Workflow Builder condition. `failed` is a name list that reads "none" when empty, so a condition means comparing against a sentinel string, and `status` carries an emoji, so a condition means matching on one. Both work and both are brittle. `outcome` is one of pass | skip | fail | crash - no emoji, no sentinel. Declare it alongside the rest and branch on that; `status` stays as the human-facing string and `failed`/`skipped` stay as name lists for the message body. Worth noting the two are not interchangeable: `failed does not equal none` is also true when the runner crashed, because `failed` holds the error text on that path. `outcome` separates those - crash is its own value - so "did an example fail" and "did the run break" can branch differently. Verified all four values against generated payloads.

Serialising by provider stopped Groq examples competing for the same quota window, but the binding limit is a daily one: the org is capped at 100k tokens per day on this tier, and four Groq examples making several calls each do not fit inside it. No retry policy or backoff addresses that. So the two carrying the least distinct information come out: groq-code-interpreter-js - the same demo as its Python twin, which stays upload-dataset-code-interpreter - a third Groq chart demo groq-code-interpreter-python and mcp-groq-exa-js remain, which still covers both Groq SDK surfaces. Roughly half the token spend for nearly all the coverage. Both dropped examples work - the exclusion note says so explicitly, because "excluded" has meant "broken" for every other entry in that list and these two should go straight back if the tier is raised. They are not being quietly retired. Also fixed a stale reference in the runner's header comment: it still described results.json as feeding updateTestsMd.js, which report.mjs replaced.

The run went green with 21 passed and 10 skipped, which looked like model variance and was not. Nine of those ten skipped on: 429 You have no credits remaining. Add credits to continue using the API The OpenAI account is out of credit. That is a configuration problem which stays broken until a human tops it up - the same category as the codex auth failure I deliberately kept as a failure - but it arrives with a 429 status like a rate limit does, so the MODEL_SIDE pattern swallowed it and the gate never fired. Left alone, every OpenAI example would skip indefinitely and the nightly would keep reporting green. That is the failure mode this whole PR exists to remove, arriving through the mechanism added to fix it. So account exhaustion is now checked first and classified as a failure: no credits remaining, insufficient_quota, exceeded your current quota, billing, payment required, 402. Windowed limits that recover on their own - a per-minute rate limit, Groq's daily token cap - stay skips. Verified the boundary against nine real strings, including the two that differ only in wording: "no credits remaining" (fail) versus "Rate limit reached ... tokens per day ... try again in 10m" (skip). Worth saying plainly: the skip mechanism is doing what it was asked to, and this is the cost of it. A systemic provider problem can present as ten skips and a green build. This commit closes the specific hole; the general risk is that a future category of persistent failure looks transient. The counts are in the Slack summary for that reason - 10 skipped of 31 should read as wrong to a human even when the job is green.

Per the branch name. A provider refusing to serve tokens says nothing about the sandbox: it was created, the example installed and ran, the request left the box and came back with an answer. Whether that answer is a completion or a 429 is the provider's business, not this suite's. Rate limited, now counted as passing: rate limit, 429, tokens per day, overloaded, no credits remaining, insufficient_quota, exceeded your current quota Still skipped - the model did something different this time: malformed or absent tool call, nothing displayable produced, generated code raising inside the sandbox Still failing - configuration, not capacity: auth, missing template, dependency resolution, wrong entrypoint, import errors, retired model ids, sandbox timeouts This reverses the previous commit, which had just split account exhaustion out as a failure. It is the same call made the other way: "no credits remaining" is a 429, and the rule now is that 429s are OK. Worth being explicit that the consequence survives the reversal - the suite reports OK while the OpenAI account is dry, which is the state it is in right now, and nine examples were affected in the last run. So rate-limited passes are tracked and named rather than folded into the pass count. The runner logs them as "OK <name> (rate limited or out of quota - the sandbox ran, the provider refused)", results.json carries a rateLimited flag per example, the summary appends "· N rate limited", and there is a new rate_limited Slack variable. A dry account therefore shows as "22 passed · 0 failed · 9 rate limited" rather than as unqualified health. Declare rate_limited on the Workflow Builder trigger to see it. Rate limits still consume their retries first, since a per-minute window often clears and yields real signal; only an exhausted retry budget settles as OK. Verified the boundary against 14 real strings from this branch's runs, and the report output for a run containing rate-limited passes.

Four examples still showed as skipped after the previous commit - openai-python, gpt-4o-code-interpreter, o1-code-interpreter-python and groq-code-interpreter-python - and I explained that away as a different underlying cause. It was not. Each of those logs contains the 429 or "no credits remaining" text four to eight times. They should all have counted as OK. The cause is a bug I introduced: classify() checks RATE_LIMITED before MODEL_SIDE, but the early return inside the retry loop tested MODEL_SIDE directly instead of calling classify(). A rate limited notebook emits both signals - nbconvert reports the failing cell as well as the API error underneath it - so that branch short-circuited to skip before the rate limit was ever considered. The precedence was correct in one place and absent in the other. Now that branch calls classify() too, so precedence lives in exactly one place. Only a settled skip returns early; a rate limit falls through to the retries, since its window often clears. Added the case that regressed to the classifier checks: an output containing both a 429 and a model-side pattern must be OK, not skipped. That is three of the seven new cases, because it is the combination I got wrong, not either pattern alone.

groq-code-interpreter-python called llama-3.3-70b-versatile and got 404 "does not exist or you do not have access to it". Rather than pick another model from Groq's docs - which is how this one got chosen, and how three earlier model guesses on this branch went wrong - it now uses openai/gpt-oss-120b, which mcp-groq-exa-js has been passing with on this same account. The commented alternative moves from llama-3.1-8b-instant to openai/gpt-oss-20b for the same reason. Proven on the account beats recommended in the docs. Also fixed the false positive that hid it. The 404 was classified as a model-side skip because nbconvert echoes the failing cell's source into the log, and that notebook's source contains the literal string print("[Code Interpreter ERROR]", exec.error) which matched the model-side pattern regardless of what actually failed. The pattern now requires the marker to be followed by real output, so an echoed source line no longer counts as an emitted error. A genuine 404 reads as a failure again.

The live message spent two of its five lines saying "Failed: none" and "Skipped: none", and never showed the nine rate-limited names at all - the count was in the summary but the list had nowhere to go. Both are the same cause: Workflow Builder templates cannot do conditionals, so one line per category is either always present or always absent. So the body is composed here, where conditionals are possible, and exposed as three new variables: headline 22/31 ran clean, 9 rate limited details only the lines that apply, already formatted with their emoji footer branch · commit · run url A healthy run is now two lines instead of five: ✅ passed — Cookbook examples 31/31 ran clean Nothing to report - every example ran and returned. and a bad one names everything that needs naming: ❌ failed — Cookbook examples 20/24 ran clean, 1 rate limited, 1 skipped, 2 failed ❌ Failed: openai-js, playwright-in-e2b ⏳ Rate limited, counted OK: langchain-python ⏭️ Skipped, the model varied rather than the sandbox: groq-code-interpreter-python headline leads with how many examples genuinely exercised a model, which the old summary buried: "31 passed ... 9 rate limited" reads as full health at a glance, where "22/31 ran clean, 9 rate limited" does not. The old variables stay for anyone composing their own layout. Rendered all three cases - the run from the screenshot, a fully clean run, and a run with all four states - rather than reasoning about the strings.

Iterating on the Workflow Builder template meant a 15-minute run across 31 examples and four paid providers just to see whether a variable name was right, and a failure message could not be seen at all without breaking something on purpose. `workflow_dispatch` now takes preview_message: off | ok | failure. Set to ok or failure it synthesises tests/results.json, skips the example run entirely, and posts the message a real run of that shape would produce. The gate is disabled in preview, because nothing was tested and a preview must not be able to turn the branch red. The scenarios use example names and counts from real runs on this branch rather than invented ones, so the preview exercises the same string lengths Slack will have to render - the rate-limited line in the ok case is nine real names, which is the case most likely to hit a length limit.

"Skipped, the model varied rather than the sandbox" and "Rate limited, counted OK" explained the classification on every single run, which is the wrong place for it - the labels are now just "⏭️ Skipped:" and "⏳ Rate limited:". Anyone who needs to know what they mean can read the classifier; a nightly message should not restate the same sentence every day.

The message was four lines of prose. Block characters and emoji are the only visuals a Workflow Builder Text variable can carry - no Block Kit on this trigger type - so that is what the bar is drawn with: one glyph per state, proportional, 20 cells wide so it does not wrap on mobile. ██████████████▓▓▓▓▓▓ 22/31 ran clean ✅ 22 clean ⏳ 9 rate limited ████████████▓▓▓▒░░░░ 8/14 ran clean ✅ 8 clean ⏳ 2 rate limited ⏭️ 1 skipped ❌ 3 failed █ clean, ▓ rate limited, ▒ skipped, ░ failed. The counts row omits zeros, so a healthy run stays two short lines rather than always printing four states. The first attempt padded the bar to width with clean blocks, which put a █ after the failures and made a bad run look like it recovered at the end. The remainder now goes to the largest segment instead. Checked that the bar is exactly 20 cells for a healthy run, a mixed run and an all-failed run. New variable: counts. Message becomes {{status}} — Cookbook examples / {{headline}} / {{counts}} / {{details}} / {{footer}}.

The footer printed a bare 80-character actions URL, which dominated the line. It is now Slack mrkdwn: <url|view run>. Whether Workflow Builder renders that syntax inside a Text variable is not documented either way, so it sits behind a LINK_MRKDWN constant. If the angle brackets show up literally in Slack, flip it to false and the footer goes back to the bare URL, which Slack linkifies on its own.

Confirmed against a live preview: Workflow Builder does not render Slack mrkdwn inside a Text variable - <url|view run> arrived with the angle brackets intact. So LINK_MRKDWN goes to false and the URL is bare, which Slack linkifies by itself. It also moves onto its own line, since an 80-character actions URL on the same line as the branch and commit swallowed both. That is the reason the footer was worth changing at all, and it is achievable without mrkdwn.

@theme

Nothing is held back on my judgement any more. What moved: e2b cap >=2.26,<2.38 -> >=2.39,<3 plus the notebook "e2b<2.38" e2b-code-interpreter ==2.9.0 -> ==2.9.1 tailwindcss ^3.4.17 -> ^4.3.3 (migration, see below) http-proxy-middleware ^3.0.5 -> ^4.2.0 open ^10.2.0 -> ^11.0.1 eve ^0.33.3 -> ^0.38.3 anthropic ==0.121.0 -> ==0.122.0 mistralai ==2.9.2 -> ==2.9.3 firecrawl-py ==4.35.0 -> ==4.35.1 together ==2.30.0 -> ==2.31.0 ibm_watsonx_ai ==1.6.1 -> ==1.6.3 The e2b cap is gone because 2.39.1 works with code-interpreter again - verified live against a clean install, run_code returns - so e2b-dev/E2B#1671 landing has undone the reason for it. Tailwind 4 is a migration, not a bump: the @tailwind directives collapse to one @import, the PostCSS plugin becomes @tailwindcss/postcss, autoprefixer is bundled in and dropped, content paths are auto-detected so tailwind.config.js is deleted, and the gradient theme extensions move into an @theme block in globals.css. Verified with a real next build. http-proxy-middleware 4 requires Node ^22.15 || ^24 || >=26 and the E2B base template ships Node 20.9 - checked in a live sandbox rather than assumed, alongside Python 3.11.6. Rather than hold the dependency back, examples can now declare a nodeVersion and the runner fetches that Node tarball into the sandbox first. Pinned tarball, not a piped installer. Two things stay where they are, both because the ecosystem has not moved rather than because I chose it. typescript stays ^5.9.3 in codestral-code-interpreter-js and together-ai-code-interpreter-js: typescript-eslint peers on typescript >=4.8.4 <6.1.0 in every published version including its newest 8.67.1 alphas, so TS 7 cannot resolve there. And `open` turned out to need only Node >=20, so holding it back earlier was my error, not a constraint - it went up with this batch. Verified: every JS example installs and typechecks, all 117 notebook cells compile, the Next.js app builds on Tailwind 4, and no e2b cap remains anywhere.

Three gaps, all found by checking rather than assuming. The two together-ai READMEs still told people to pick from Meta-Llama-3.1-405B, Qwen2-72B, CodeLlama-70b and deepseek-coder-33b - the exact ids removed from the code because Together no longer serves them. Following those instructions produced a 400. They now list the models the examples actually offer: Llama-3.3-70B-Instruct-Turbo, Qwen3.7/3.6 Plus, DeepSeek V4 Pro/Flash and Qwen2.5-7B. Three example READMEs described the model as gpt-4o in prose; that is GPT-5.6 now. The two remaining gpt-4o strings in the root README are directory paths (./examples/gpt-4o-python) and are correct as they stand. custom-sandbox-domain-proxy and flue-feedback-analyst-js existed on disk but appeared nowhere in the root README, so both are added to the use-cases list. Every example directory is now linked and every link resolves. Added a "Running the examples as a test suite" section, which the README had nothing about at all: how to run everything or filter to one example, which keys are needed, and - the part worth writing down - what counts as a failure. A rate limit or spent quota is OK because the sandbox still built and ran; non-deterministic model behaviour is skipped; only genuinely broken things fail. Also notes that the exclusions and their reasons live at the top of tests/run-examples.ts, so a coverage gap is discoverable rather than silent.

… to call Four directories and five files were named after models none of them call any more, which is how the README ended up telling people gpt-4o while the code ran gpt-5.6. gpt-4o-python -> openai-image-analysis-python gpt-4o-js -> openai-image-analysis-js o1-and-gpt-4-python -> openai-ml-dataset-python o1-and-gpt-4-js -> openai-ml-dataset-js gpt_4o.ipynb -> image_analysis.ipynb o1.ipynb -> ml_dataset.ipynb llama_3_code_interpreter.ipynb -> groq_code_interpreter.ipynb llama_3_code_interpreter.mts -> groq_code_interpreter.mts llama_3_code_interpreter_upload_dataset.ipynb -> upload_dataset.ipynb The llama_3 ones matter as much as the gpt-4o ones: groq-code-interpreter-python now calls openai/gpt-oss-120b, so its filename named a model from a different vendor. Named after the task rather than the model on purpose. That is the property that makes this the last time these need renaming - "image analysis" and "ml dataset" stay true across model generations, and a provider name like groq_ or codestral_ ages far more slowly than a version. Everything pointing at them moved too: the README table and use-case links, the test list's paths and its display names, the groq start script, three package.json "name" fields that still said o1-js and o1-code-interpreter-js, and the README titles and prose that described the examples as o1 or GPT-4o demos. Verified: every path in the test list resolves, all 22 notebooks are valid JSON, the runner typechecks, and no stale model id or old path remains. Two `o1` strings are left on purpose - crewai-python cites CrewAI's own `"o1" in model.lower()` heuristic, and openai-agents-sdk is an upstream copy that cannot run here anyway.

beran-t added 7 commits August 12, 2026 17:49

cla-bot Bot added the cla-signed label Aug 12, 2026

cursor Bot reviewed Aug 12, 2026

View reviewed changes

Comment thread tests/examples.test.ts Outdated

claude Bot reviewed Aug 12, 2026

View reviewed changes

beran-t added 3 commits August 12, 2026 18:52

chore: do not commit test runner artifacts

804b85d

tests/results.json from a local run slipped into the previous commit. It is generated output consumed by updateTestsMd.js and uploaded as an artifact, so it does not belong in the repo. Gitignored along with tests/Tests.txt.

cursor Bot reviewed Aug 12, 2026

View reviewed changes

beran-t force-pushed the chore/modernize-cookbook branch from 6e80bf6 to 46bf674 Compare August 12, 2026 17:20

beran-t added 5 commits August 12, 2026 19:24

cursor Bot reviewed Aug 14, 2026

View reviewed changes

beran-t added 4 commits August 14, 2026 13:04

beran-t changed the title ~~Bump every example to current SDKs and model IDs, and make CI report honestly~~ Bring every example onto current SDKs and models, and make CI report honestly Aug 14, 2026

beran-t added 2 commits August 14, 2026 15:01

cursor Bot reviewed Aug 14, 2026

View reviewed changes

beran-t added 23 commits August 14, 2026 17:11

beran-t merged commit 39c8c98 into main Aug 17, 2026
6 checks passed

beran-t deleted the chore/modernize-cookbook branch August 17, 2026 13:02

Navigation Menu

Uh oh!

Bring every example onto current SDKs and models, and make CI report honestly - #117

Bring every example onto current SDKs and models, and make CI report honestly#117
beran-t merged 47 commits into
mainfrom
chore/modernize-cookbook

beran-t commented Aug 12, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

claude Bot Aug 12, 2026

Uh oh!

claude Bot Aug 12, 2026

Uh oh!

claude Bot Aug 12, 2026

Uh oh!

beran-t commented Aug 12, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot Aug 12, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot Aug 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		@@ -0,0 +1 @@
		{"fileNames":[],"fileInfos":[],"root":[],"options":{"allowJs":true,"esModuleInterop":true,"module":99,"skipLibCheck":true,"strict":true,"target":9},"errors":true,"version":"5.9.3"} No newline at end of file

Uh oh!

Conversation

beran-t commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

claude Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

beran-t commented Aug 12, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

beran-t commented Aug 12, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading