fix(checks): tighten checkToolErrorSpan to require non-ok span status

[constantinius]

Simplify the tool error span check to focus on what matters: the span status field must be set and must not be "ok". Previously the check looked for a grab-bag of error indicators (data.error, data.exception, gen_ai.tool.error, tags.error) in addition to status, which was overly lenient — a span could pass by having any random error-like data key even if the status itself was wrong.

The new check requires that span.status is present and is not "ok" (e.g. "internal_error", "error"). This catches frameworks like Python LangGraph where the tool span status is unset despite the tool raising an exception.

Closes https://linear.app/getsentry/issue/TET-2091/test-mismatch-between-span-status-and-tool-call-results

[linear-code]

TET-2091 Test mismatch between span status and tool call results

[github-actions]

🔴 AI SDK Integration Test Results

Status: 3 regressions detected

Summary

Metric	main	PR	Change
Total Tests	667	667	—
Passed	465	466	+1 ✅
Failed	192	195	+3 ⚠️

🔴 Regressions

These tests were passing on main but are now failing:

browser/langchain :: Multi-Turn LLM Test (blocking)

Error: Browser test timed out (60s)

Browser test timed out (60s)

browser/openai :: Multi-Turn LLM Test (blocking)

Error: Browser test timed out (60s)

Browser test timed out (60s)

python/google-genai :: Conversation ID LLM Test (async, streaming)

Error: Test execution timed out (60s)

Test execution timed out (60s)

✅ Fixed

These tests were failing on main but are now passing:

• cloudflare/anthropic :: Vision LLM Test (blocking)
• cloudflare/anthropic :: Long Input LLM Test (blocking)
• cloudflare/google-genai :: Basic Error LLM Test (blocking)
• cloudflare/langchain :: Vision LLM Test (blocking)

Test Matrix

Agent Tests

SDK	Basic Agent Test	Conversation ID Agent Test	Long Input Agent Test	Tool Call Agent Test	Tool Error Agent Test	Vision Agent Test
browser/langgraph	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain
cloudflare/langgraph	❌	❌	❌	❌	❌	❌
cloudflare/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
nextjs/mastra	✅	❌	—	✅	✅	✅
nextjs/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
node/langgraph	❌	❌	❌	❌	❌	❌
node/manual	✅	✅	✅	✅	✅	✅
node/mastra	✅	❌	❌	❌	❌	❌
node/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
python/langgraph	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s
python/manual	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s
python/openai-agents	✅	✅	✅	✅	✅	❌
python/pydantic-ai	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single

Embedding Tests

SDK	Basic Embeddings Test
browser/google-genai	✅
browser/langchain	✅
browser/openai	✅
cloudflare/google-genai	✅
cloudflare/langchain	✅
cloudflare/openai	✅
cloudflare/vercel	✅
nextjs/google-genai	✅
nextjs/langchain	✅
nextjs/openai	✅
nextjs/vercel	✅
node/google-genai	✅
node/langchain	✅
node/openai	✅
node/vercel	✅
python/google-genai	✅a, blk ✅s, blk
python/langchain	❌a, blk ❌s, blk
python/litellm	❌a, blk ✅s, blk
python/manual	✅a, blk ✅s, blk
python/openai	✅a, blk ✅s, blk

LLM Tests

SDK	Basic Error LLM Test	Basic LLM Test	Conversation ID LLM Test	Long Input LLM Test	Multi-Turn LLM Test	Vision LLM Test
browser/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
browser/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
browser/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	❌📉blk ✅str	✅blk ✅str
browser/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	❌📉blk ✅str	✅blk ✅str
cloudflare/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅🔧blk ✅str	✅blk ✅str	✅🔧blk ✅str
cloudflare/google-genai	✅🔧blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
cloudflare/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅🔧blk ✅str
cloudflare/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/manual	—	✅	✅	✅	✅	✅
node/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
python/anthropic	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str
python/google-genai	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ❌📉a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ❌s, blk ❌s, str
python/langchain	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str
python/litellm	❌a, blk ❌a, str ❌s, blk ❌s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str
python/manual	—	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk
python/openai	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	❌a, blk ❌a, str ❌s, blk ❌s, str

MCP Tests

SDK	Basic MCP Tool Call Test	MCP Multiple Tool Calls Test	MCP Prompt Get Test	MCP Resource Read Test	MCP Tool Error Test
node/mcp	✅sse ✅io	✅sse ✅io	✅sse ✅io	✅sse ✅io	✅sse ✅io
python/fastmcp	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io
python/mcp	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | _str=streaming _blk=blocking _a=async _s=sync _io=stdio _sse=sse _hi=highlevel _lo=lowlevel

---

Generated by AI SDK Integration Tests