feat: adding variations for several frameworks to test at least openai + anthropic by constantinius · Pull Request #139 · getsentry/testing-ai-sdk-integrations

constantinius · 2026-04-13T07:49:15Z

Closes https://linear.app/getsentry/issue/TET-2164/add-variations-for-providers-for-the-higher-level-frameworks

…i + anthropic

linear-code · 2026-04-13T07:49:22Z

TET-2164 Add variations for providers for the higher level frameworks

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Unpinned anthropic version alongside pinned LangChain packages
- This is a real bug because @langchain/anthropic@latest now requires @langchain/core ^1.x while these templates pin core to 0.3.49, so I pinned @langchain/anthropic to 0.3.0 in the three flagged configs.

Or push these changes by commenting:

@cursor push 02d6605d7d

Preview (02d6605d7d)

diff --git a/src/runner/templates/agents/cloudflare/langgraph/config.json b/src/runner/templates/agents/cloudflare/langgraph/config.json
--- a/src/runner/templates/agents/cloudflare/langgraph/config.json
+++ b/src/runner/templates/agents/cloudflare/langgraph/config.json
@@ -14,7 +14,7 @@
     },
     {
       "package": "@langchain/anthropic",
-      "version": "latest"
+      "version": "0.3.0"
     },
     {
       "package": "@langchain/core",

diff --git a/src/runner/templates/agents/node/langgraph/config.json b/src/runner/templates/agents/node/langgraph/config.json
--- a/src/runner/templates/agents/node/langgraph/config.json
+++ b/src/runner/templates/agents/node/langgraph/config.json
@@ -14,7 +14,7 @@
     },
     {
       "package": "@langchain/anthropic",
-      "version": "latest"
+      "version": "0.3.0"
     },
     {
       "package": "@langchain/core",

diff --git a/src/runner/templates/llm/node/langchain/config.json b/src/runner/templates/llm/node/langchain/config.json
--- a/src/runner/templates/llm/node/langchain/config.json
+++ b/src/runner/templates/llm/node/langchain/config.json
@@ -15,7 +15,7 @@
     },
     {
       "package": "@langchain/anthropic",
-      "version": "latest"
+      "version": "0.3.0"
     },
     {
       "package": "@langchain/core",

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e921377. Configure here.}

cursor · 2026-04-13T07:55:20Z

src/runner/templates/agents/node/langgraph/config.json

+    {
+      "package": "@langchain/anthropic",
+      "version": "latest"
+    },


Unpinned anthropic version alongside pinned LangChain packages

Medium Severity

@langchain/anthropic is set to "latest" while sibling packages like @langchain/openai and @langchain/core are pinned to specific 0.3.x versions. If a new major version of @langchain/anthropic is released with incompatible peer dependency requirements for @langchain/core, npm install would fail or produce a broken dependency tree, causing all Anthropic provider test variants to break. Pinning @langchain/anthropic to a compatible version (like other LangChain packages) avoids this risk.

Additional Locations (2)

src/runner/templates/llm/node/langchain/config.json#L15-L19

src/runner/templates/agents/cloudflare/langgraph/config.json#L14-L18

^{Reviewed by Cursor Bugbot for commit e921377. Configure here.}

github-actions · 2026-04-13T08:13:17Z

🟡 AI SDK Integration Test Results

Status: 1 test fixed, 164 still failing

Summary

Metric	main	PR	Change
Total Tests	667	847	+180
Passed	468	467	-1 ⚠️
Failed	193	164	-29 ✅

✅ Fixed

These tests were failing on main but are now passing:

browser/openai :: Multi-Turn LLM Test (streaming)

🆕 New Tests

Passing (80):

✅ nextjs/langchain :: Basic LLM Test (streaming, openai)
✅ nextjs/langchain :: Basic LLM Test (blocking, openai)
✅ nextjs/langchain :: Basic LLM Test (blocking, anthropic)
✅ nextjs/langchain :: Multi-Turn LLM Test (streaming, openai)
✅ nextjs/langchain :: Multi-Turn LLM Test (blocking, openai)
✅ nextjs/langchain :: Multi-Turn LLM Test (blocking, anthropic)
✅ nextjs/langchain :: Basic Error LLM Test (streaming, openai)
✅ nextjs/langchain :: Basic Error LLM Test (streaming, anthropic)
✅ nextjs/langchain :: Basic Error LLM Test (blocking, openai)
✅ nextjs/langchain :: Basic Error LLM Test (blocking, anthropic)
✅ nextjs/langchain :: Vision LLM Test (streaming, openai)
✅ nextjs/langchain :: Vision LLM Test (blocking, openai)
✅ nextjs/langchain :: Vision LLM Test (blocking, anthropic)
✅ nextjs/langchain :: Long Input LLM Test (streaming, openai)
✅ nextjs/langchain :: Long Input LLM Test (streaming, anthropic)
✅ nextjs/langchain :: Long Input LLM Test (blocking, openai)
✅ nextjs/langchain :: Long Input LLM Test (blocking, anthropic)
✅ nextjs/langchain :: Conversation ID LLM Test (streaming, openai)
✅ nextjs/langchain :: Conversation ID LLM Test (blocking, openai)
✅ nextjs/langchain :: Conversation ID LLM Test (blocking, anthropic)
✅ python/langchain :: Basic LLM Test (sync, streaming, openai)
✅ python/langchain :: Basic LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Basic LLM Test (sync, blocking, openai)
✅ python/langchain :: Basic LLM Test (async, streaming, openai)
✅ python/langchain :: Basic LLM Test (async, streaming, anthropic)
✅ python/langchain :: Basic LLM Test (async, blocking, openai)
✅ python/langchain :: Multi-Turn LLM Test (sync, streaming, openai)
✅ python/langchain :: Multi-Turn LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Multi-Turn LLM Test (sync, blocking, openai)
✅ python/langchain :: Multi-Turn LLM Test (async, streaming, openai)
✅ python/langchain :: Multi-Turn LLM Test (async, streaming, anthropic)
✅ python/langchain :: Multi-Turn LLM Test (async, blocking, openai)
✅ python/langchain :: Basic Error LLM Test (sync, streaming, openai)
✅ python/langchain :: Basic Error LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Basic Error LLM Test (sync, blocking, openai)
✅ python/langchain :: Basic Error LLM Test (sync, blocking, anthropic)
✅ python/langchain :: Basic Error LLM Test (async, streaming, openai)
✅ python/langchain :: Basic Error LLM Test (async, streaming, anthropic)
✅ python/langchain :: Basic Error LLM Test (async, blocking, openai)
✅ python/langchain :: Basic Error LLM Test (async, blocking, anthropic)
✅ python/langchain :: Vision LLM Test (sync, streaming, openai)
✅ python/langchain :: Vision LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Vision LLM Test (sync, blocking, openai)
✅ python/langchain :: Vision LLM Test (async, streaming, openai)
✅ python/langchain :: Vision LLM Test (async, streaming, anthropic)
✅ python/langchain :: Vision LLM Test (async, blocking, openai)
✅ python/langchain :: Long Input LLM Test (sync, streaming, openai)
✅ python/langchain :: Long Input LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Long Input LLM Test (sync, blocking, openai)
✅ python/langchain :: Long Input LLM Test (sync, blocking, anthropic)
✅ python/langchain :: Long Input LLM Test (async, streaming, openai)
✅ python/langchain :: Long Input LLM Test (async, streaming, anthropic)
✅ python/langchain :: Long Input LLM Test (async, blocking, openai)
✅ python/langchain :: Long Input LLM Test (async, blocking, anthropic)
✅ python/langchain :: Conversation ID LLM Test (sync, streaming, openai)
✅ python/langchain :: Conversation ID LLM Test (sync, streaming, anthropic)
✅ python/langchain :: Conversation ID LLM Test (sync, blocking, openai)
✅ python/langchain :: Conversation ID LLM Test (async, streaming, openai)
✅ python/langchain :: Conversation ID LLM Test (async, streaming, anthropic)
✅ python/langchain :: Conversation ID LLM Test (async, blocking, openai)
✅ python/litellm :: Basic LLM Test (sync, streaming, openai)
✅ python/litellm :: Basic LLM Test (sync, streaming, anthropic)
✅ python/litellm :: Basic LLM Test (sync, blocking, openai)
✅ python/litellm :: Basic LLM Test (sync, blocking, anthropic)
✅ python/litellm :: Multi-Turn LLM Test (sync, streaming, openai)
✅ python/litellm :: Multi-Turn LLM Test (sync, streaming, anthropic)
✅ python/litellm :: Multi-Turn LLM Test (sync, blocking, openai)
✅ python/litellm :: Multi-Turn LLM Test (sync, blocking, anthropic)
✅ python/litellm :: Vision LLM Test (sync, streaming, openai)
✅ python/litellm :: Vision LLM Test (sync, streaming, anthropic)
✅ python/litellm :: Vision LLM Test (sync, blocking, openai)
✅ python/litellm :: Vision LLM Test (sync, blocking, anthropic)
✅ python/litellm :: Long Input LLM Test (sync, streaming, openai)
✅ python/litellm :: Long Input LLM Test (sync, streaming, anthropic)
✅ python/litellm :: Long Input LLM Test (sync, blocking, openai)
✅ python/litellm :: Long Input LLM Test (sync, blocking, anthropic)
✅ python/litellm :: Conversation ID LLM Test (sync, streaming, openai)
✅ python/litellm :: Conversation ID LLM Test (sync, streaming, anthropic)
✅ python/litellm :: Conversation ID LLM Test (sync, blocking, openai)
✅ python/litellm :: Conversation ID LLM Test (sync, blocking, anthropic)

Failing (280):

❌ browser/langgraph :: Basic Agent Test (streaming, openai, graph)