feat(openai): add variants for the responses API by constantinius · Pull Request #142 · getsentry/testing-ai-sdk-integrations

constantinius · 2026-04-13T14:10:34Z

Closes https://linear.app/getsentry/issue/TET-2162/openai-add-test-for-responses-api

linear-code · 2026-04-13T14:10:39Z

TET-2162 OpenAI: add test for responses API

github-actions · 2026-04-13T14:45:46Z

🔴 AI SDK Integration Test Results

Status: 3 regressions detected

Summary

Metric	main	PR	Change
Total Tests	667	739	+72
Passed	466	534	+68 ✅
Failed	194	199	+5 ⚠️

🔴 Regressions

These tests were passing on main but are now failing:

cloudflare/vercel :: Long Input Agent Test (blocking, function, anthropic)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span af459ebc: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span af459ebc: Attribute 'gen_ai.usage.output_tokens' must exist but is missing

browser/langchain :: Multi-Turn LLM Test (streaming)

Error: Browser test timed out (60s)

Browser test timed out (60s)

python/google-genai :: Conversation ID LLM Test (sync, blocking)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span bacf46da: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span bacf46da: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist
gen_ai.response.model is missing (optional but recommended)

✅ Fixed

These tests were failing on main but are now passing:

browser/google-genai :: Multi-Turn LLM Test (blocking)
cloudflare/langchain :: Vision LLM Test (streaming)

🆕 New Tests

Passing (128):

✅ browser/openai :: Basic LLM Test (streaming, chat)
✅ browser/openai :: Basic LLM Test (streaming, responses)
✅ browser/openai :: Basic LLM Test (blocking, chat)
✅ browser/openai :: Basic LLM Test (blocking, responses)
✅ browser/openai :: Multi-Turn LLM Test (streaming, chat)
✅ browser/openai :: Multi-Turn LLM Test (streaming, responses)
✅ browser/openai :: Multi-Turn LLM Test (blocking, chat)
✅ browser/openai :: Multi-Turn LLM Test (blocking, responses)
✅ browser/openai :: Basic Error LLM Test (streaming, chat)
✅ browser/openai :: Basic Error LLM Test (streaming, responses)
✅ browser/openai :: Basic Error LLM Test (blocking, chat)
✅ browser/openai :: Basic Error LLM Test (blocking, responses)
✅ browser/openai :: Vision LLM Test (streaming, chat)
✅ browser/openai :: Vision LLM Test (streaming, responses)
✅ browser/openai :: Vision LLM Test (blocking, chat)
✅ browser/openai :: Vision LLM Test (blocking, responses)
✅ browser/openai :: Long Input LLM Test (streaming, chat)
✅ browser/openai :: Long Input LLM Test (streaming, responses)
✅ browser/openai :: Long Input LLM Test (blocking, chat)
✅ browser/openai :: Long Input LLM Test (blocking, responses)
✅ browser/openai :: Conversation ID LLM Test (streaming, chat)
✅ browser/openai :: Conversation ID LLM Test (streaming, responses)
✅ browser/openai :: Conversation ID LLM Test (blocking, chat)
✅ browser/openai :: Conversation ID LLM Test (blocking, responses)
✅ cloudflare/openai :: Basic LLM Test (streaming, chat)
✅ cloudflare/openai :: Basic LLM Test (streaming, responses)
✅ cloudflare/openai :: Basic LLM Test (blocking, chat)
✅ cloudflare/openai :: Basic LLM Test (blocking, responses)
✅ cloudflare/openai :: Multi-Turn LLM Test (streaming, chat)
✅ cloudflare/openai :: Multi-Turn LLM Test (streaming, responses)
✅ cloudflare/openai :: Multi-Turn LLM Test (blocking, chat)
✅ cloudflare/openai :: Multi-Turn LLM Test (blocking, responses)
✅ cloudflare/openai :: Basic Error LLM Test (streaming, chat)
✅ cloudflare/openai :: Basic Error LLM Test (streaming, responses)
✅ cloudflare/openai :: Basic Error LLM Test (blocking, chat)
✅ cloudflare/openai :: Basic Error LLM Test (blocking, responses)
✅ cloudflare/openai :: Vision LLM Test (streaming, chat)
✅ cloudflare/openai :: Vision LLM Test (streaming, responses)
✅ cloudflare/openai :: Vision LLM Test (blocking, chat)
✅ cloudflare/openai :: Vision LLM Test (blocking, responses)
✅ cloudflare/openai :: Long Input LLM Test (streaming, chat)
✅ cloudflare/openai :: Long Input LLM Test (streaming, responses)
✅ cloudflare/openai :: Long Input LLM Test (blocking, chat)
✅ cloudflare/openai :: Long Input LLM Test (blocking, responses)
✅ cloudflare/openai :: Conversation ID LLM Test (streaming, chat)
✅ cloudflare/openai :: Conversation ID LLM Test (streaming, responses)
✅ cloudflare/openai :: Conversation ID LLM Test (blocking, chat)
✅ cloudflare/openai :: Conversation ID LLM Test (blocking, responses)
✅ nextjs/openai :: Basic LLM Test (streaming, chat)
✅ nextjs/openai :: Basic LLM Test (streaming, responses)
✅ nextjs/openai :: Basic LLM Test (blocking, chat)
✅ nextjs/openai :: Basic LLM Test (blocking, responses)
✅ nextjs/openai :: Multi-Turn LLM Test (streaming, chat)
✅ nextjs/openai :: Multi-Turn LLM Test (streaming, responses)
✅ nextjs/openai :: Multi-Turn LLM Test (blocking, chat)
✅ nextjs/openai :: Multi-Turn LLM Test (blocking, responses)
✅ nextjs/openai :: Basic Error LLM Test (streaming, chat)
✅ nextjs/openai :: Basic Error LLM Test (streaming, responses)
✅ nextjs/openai :: Basic Error LLM Test (blocking, chat)
✅ nextjs/openai :: Basic Error LLM Test (blocking, responses)
✅ nextjs/openai :: Vision LLM Test (streaming, chat)
✅ nextjs/openai :: Vision LLM Test (streaming, responses)
✅ nextjs/openai :: Vision LLM Test (blocking, chat)
✅ nextjs/openai :: Vision LLM Test (blocking, responses)
✅ nextjs/openai :: Long Input LLM Test (streaming, chat)
✅ nextjs/openai :: Long Input LLM Test (streaming, responses)
✅ nextjs/openai :: Long Input LLM Test (blocking, chat)
✅ nextjs/openai :: Long Input LLM Test (blocking, responses)
✅ nextjs/openai :: Conversation ID LLM Test (streaming, chat)
✅ nextjs/openai :: Conversation ID LLM Test (streaming, responses)
✅ nextjs/openai :: Conversation ID LLM Test (blocking, chat)
✅ nextjs/openai :: Conversation ID LLM Test (blocking, responses)
✅ node/openai :: Basic LLM Test (streaming, chat)
✅ node/openai :: Basic LLM Test (streaming, responses)
✅ node/openai :: Basic LLM Test (blocking, chat)
✅ node/openai :: Basic LLM Test (blocking, responses)
✅ node/openai :: Multi-Turn LLM Test (streaming, chat)
✅ node/openai :: Multi-Turn LLM Test (streaming, responses)
✅ node/openai :: Multi-Turn LLM Test (blocking, chat)
✅ node/openai :: Multi-Turn LLM Test (blocking, responses)
✅ node/openai :: Basic Error LLM Test (streaming, chat)
✅ node/openai :: Basic Error LLM Test (streaming, responses)
✅ node/openai :: Basic Error LLM Test (blocking, chat)
✅ node/openai :: Basic Error LLM Test (blocking, responses)
✅ node/openai :: Vision LLM Test (streaming, chat)
✅ node/openai :: Vision LLM Test (streaming, responses)
✅ node/openai :: Vision LLM Test (blocking, chat)
✅ node/openai :: Vision LLM Test (blocking, responses)
✅ node/openai :: Long Input LLM Test (streaming, chat)
✅ node/openai :: Long Input LLM Test (streaming, responses)
✅ node/openai :: Long Input LLM Test (blocking, chat)
✅ node/openai :: Long Input LLM Test (blocking, responses)
✅ node/openai :: Conversation ID LLM Test (streaming, chat)
✅ node/openai :: Conversation ID LLM Test (streaming, responses)
✅ node/openai :: Conversation ID LLM Test (blocking, chat)
✅ node/openai :: Conversation ID LLM Test (blocking, responses)
✅ python/openai :: Basic LLM Test (sync, streaming, responses)
✅ python/openai :: Basic LLM Test (sync, blocking, chat)
✅ python/openai :: Basic LLM Test (sync, blocking, responses)
✅ python/openai :: Basic LLM Test (async, streaming, responses)
✅ python/openai :: Basic LLM Test (async, blocking, chat)
✅ python/openai :: Basic LLM Test (async, blocking, responses)
✅ python/openai :: Multi-Turn LLM Test (sync, streaming, responses)
✅ python/openai :: Multi-Turn LLM Test (sync, blocking, chat)
✅ python/openai :: Multi-Turn LLM Test (sync, blocking, responses)
✅ python/openai :: Multi-Turn LLM Test (async, streaming, responses)
✅ python/openai :: Multi-Turn LLM Test (async, blocking, chat)
✅ python/openai :: Multi-Turn LLM Test (async, blocking, responses)
✅ python/openai :: Basic Error LLM Test (sync, streaming, chat)
✅ python/openai :: Basic Error LLM Test (sync, streaming, responses)
✅ python/openai :: Basic Error LLM Test (sync, blocking, chat)
✅ python/openai :: Basic Error LLM Test (sync, blocking, responses)
✅ python/openai :: Basic Error LLM Test (async, streaming, chat)
✅ python/openai :: Basic Error LLM Test (async, streaming, responses)
✅ python/openai :: Basic Error LLM Test (async, blocking, chat)
✅ python/openai :: Basic Error LLM Test (async, blocking, responses)
✅ python/openai :: Long Input LLM Test (sync, streaming, responses)
✅ python/openai :: Long Input LLM Test (sync, blocking, chat)
✅ python/openai :: Long Input LLM Test (sync, blocking, responses)
✅ python/openai :: Long Input LLM Test (async, streaming, responses)
✅ python/openai :: Long Input LLM Test (async, blocking, chat)
✅ python/openai :: Long Input LLM Test (async, blocking, responses)
✅ python/openai :: Conversation ID LLM Test (sync, streaming, responses)
✅ python/openai :: Conversation ID LLM Test (sync, blocking, chat)
✅ python/openai :: Conversation ID LLM Test (sync, blocking, responses)
✅ python/openai :: Conversation ID LLM Test (async, streaming, responses)
✅ python/openai :: Conversation ID LLM Test (async, blocking, chat)
✅ python/openai :: Conversation ID LLM Test (async, blocking, responses)

Failing (16):

❌ python/openai :: Basic LLM Test (sync, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span b4ec8fb6: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span b4ec8fb6: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

❌ python/openai :: Basic LLM Test (async, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span a4eee807: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span a4eee807: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

❌ python/openai :: Multi-Turn LLM Test (sync, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span a2149175: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span a2149175: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span b55e3bcb: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span b55e3bcb: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 807ad1b7: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 807ad1b7: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

❌ python/openai :: Multi-Turn LLM Test (async, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 89e3163e: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 89e3163e: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 8f401220: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 8f401220: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span b41e8463: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span b41e8463: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

❌ python/openai :: Vision LLM Test (sync, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span a90ae985: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span a90ae985: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (sync, streaming, responses)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (sync, blocking, chat)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (sync, blocking, responses)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (async, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 8f654129: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 8f654129: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (async, streaming, responses)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (async, blocking, chat)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Vision LLM Test (async, blocking, responses)

Error: 1 check(s) failed:

1 check(s) failed:
Messages should not contain raw base64 data (should be redacted)
Messages should contain '[Blob substitute]' marker indicating binary content was redacted

❌ python/openai :: Long Input LLM Test (sync, streaming, chat)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 901e8436: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 901e8436: Attribute 'gen_ai.usage.output_tokens' must exist but is missing

❌ python/openai :: Long Input LLM Test (async, streaming, chat)

Error: 1 check(s) failed:

1 check(s) failed:
Attribute validation failed:
  Span 9fd18d22: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 9fd18d22: Attribute 'gen_ai.usage.output_tokens' must exist but is missing

❌ python/openai :: Conversation ID LLM Test (sync, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 8e662ff3: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 8e662ff3: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span ba9f6152: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span ba9f6152: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span a79ca025: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span a79ca025: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 90cda1b6: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 90cda1b6: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

❌ python/openai :: Conversation ID LLM Test (async, streaming, chat)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 97c19f4d: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 97c19f4d: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 8f8d09e4: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 8f8d09e4: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 97b13bbb: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 97b13bbb: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
  Span 9817e525: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 9817e525: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist

🗑️ Removed Tests

These tests existed on main but are not in the PR:

browser/openai :: Basic LLM Test (streaming)
browser/openai :: Basic LLM Test (blocking)
browser/openai :: Multi-Turn LLM Test (streaming)
browser/openai :: Multi-Turn LLM Test (blocking)
browser/openai :: Basic Error LLM Test (streaming)
browser/openai :: Basic Error LLM Test (blocking)
browser/openai :: Vision LLM Test (streaming)
browser/openai :: Vision LLM Test (blocking)
browser/openai :: Long Input LLM Test (streaming)
browser/openai :: Long Input LLM Test (blocking)
browser/openai :: Conversation ID LLM Test (streaming)
browser/openai :: Conversation ID LLM Test (blocking)
cloudflare/openai :: Basic LLM Test (streaming)
cloudflare/openai :: Basic LLM Test (blocking)
cloudflare/openai :: Multi-Turn LLM Test (streaming)
cloudflare/openai :: Multi-Turn LLM Test (blocking)
cloudflare/openai :: Basic Error LLM Test (streaming)
cloudflare/openai :: Basic Error LLM Test (blocking)
cloudflare/openai :: Vision LLM Test (streaming)
cloudflare/openai :: Vision LLM Test (blocking)
cloudflare/openai :: Long Input LLM Test (streaming)
cloudflare/openai :: Long Input LLM Test (blocking)
cloudflare/openai :: Conversation ID LLM Test (streaming)
cloudflare/openai :: Conversation ID LLM Test (blocking)
nextjs/openai :: Basic LLM Test (streaming)
nextjs/openai :: Basic LLM Test (blocking)
nextjs/openai :: Multi-Turn LLM Test (streaming)
nextjs/openai :: Multi-Turn LLM Test (blocking)
nextjs/openai :: Basic Error LLM Test (streaming)
nextjs/openai :: Basic Error LLM Test (blocking)
nextjs/openai :: Vision LLM Test (streaming)
nextjs/openai :: Vision LLM Test (blocking)
nextjs/openai :: Long Input LLM Test (streaming)
nextjs/openai :: Long Input LLM Test (blocking)
nextjs/openai :: Conversation ID LLM Test (streaming)
nextjs/openai :: Conversation ID LLM Test (blocking)
node/openai :: Basic LLM Test (streaming)
node/openai :: Basic LLM Test (blocking)
node/openai :: Multi-Turn LLM Test (streaming)
node/openai :: Multi-Turn LLM Test (blocking)
node/openai :: Basic Error LLM Test (streaming)
node/openai :: Basic Error LLM Test (blocking)
node/openai :: Vision LLM Test (streaming)
node/openai :: Vision LLM Test (blocking)
node/openai :: Long Input LLM Test (streaming)
node/openai :: Long Input LLM Test (blocking)
node/openai :: Conversation ID LLM Test (streaming)
node/openai :: Conversation ID LLM Test (blocking)
python/openai :: Basic LLM Test (sync, streaming)
python/openai :: Basic LLM Test (sync, blocking)
python/openai :: Basic LLM Test (async, streaming)
python/openai :: Basic LLM Test (async, blocking)
python/openai :: Multi-Turn LLM Test (sync, streaming)
python/openai :: Multi-Turn LLM Test (sync, blocking)
python/openai :: Multi-Turn LLM Test (async, streaming)
python/openai :: Multi-Turn LLM Test (async, blocking)
python/openai :: Basic Error LLM Test (sync, streaming)
python/openai :: Basic Error LLM Test (sync, blocking)
python/openai :: Basic Error LLM Test (async, streaming)
python/openai :: Basic Error LLM Test (async, blocking)
python/openai :: Vision LLM Test (sync, streaming)
python/openai :: Vision LLM Test (sync, blocking)
python/openai :: Vision LLM Test (async, streaming)
python/openai :: Vision LLM Test (async, blocking)
python/openai :: Long Input LLM Test (sync, streaming)
python/openai :: Long Input LLM Test (sync, blocking)
python/openai :: Long Input LLM Test (async, streaming)
python/openai :: Long Input LLM Test (async, blocking)
python/openai :: Conversation ID LLM Test (sync, streaming)
python/openai :: Conversation ID LLM Test (sync, blocking)
python/openai :: Conversation ID LLM Test (async, streaming)
python/openai :: Conversation ID LLM Test (async, blocking)

Test Matrix

Agent Tests

SDK	Basic Agent Test	Conversation ID Agent Test	Long Input Agent Test	Tool Call Agent Test	Tool Error Agent Test	Vision Agent Test
browser/langgraph	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}	❌_{blk, combined} ❌_{blk, compiled} ❌_{blk, custom-state} ❌_{blk, graph} ❌_{blk, langchain} ❌_{str, combined} ❌_{str, compiled} ❌_{str, custom-state} ❌_{str, graph} ❌_{str, langchain}
cloudflare/langgraph	❌	❌	❌	❌	❌	❌
cloudflare/vercel	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ❌📉_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}
nextjs/mastra	✅	❌	—	✅	✅	✅
nextjs/vercel	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}
node/langgraph	❌	❌	❌	❌	❌	❌
node/manual	✅	✅	✅	✅	✅	✅
node/mastra	✅	❌	❌	❌	❌	❌
node/vercel	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}	❌_{blk, class, anthropic} ❌_{blk, class, openai} ✅_{blk, function, anthropic} ✅_{blk, function, openai} ❌_{str, class, anthropic} ❌_{str, class, openai} ✅_{str, function, anthropic} ✅_{str, function, openai}
python/langgraph	❌_a ❌_s	❌_a ❌_s	❌_a ❌_s	❌_a ❌_s	❌_a ❌_s	❌_a ❌_s
python/manual	✅_a ✅_s	✅_a ✅_s	✅_a ✅_s	✅_a ✅_s	✅_a ✅_s	✅_a ✅_s
python/openai-agents	✅	✅	✅	✅	✅	❌
python/pydantic-ai	✅_{a, fallback} ✅_{a, single}	✅_{a, fallback} ✅_{a, single}	✅_{a, fallback} ✅_{a, single}	✅_{a, fallback} ✅_{a, single}	✅_{a, fallback} ✅_{a, single}	✅_{a, fallback} ✅_{a, single}

Embedding Tests

SDK	Basic Embeddings Test
browser/google-genai	✅
browser/langchain	✅
browser/openai	✅
cloudflare/google-genai	✅
cloudflare/langchain	✅
cloudflare/openai	✅
cloudflare/vercel	✅
nextjs/google-genai	✅
nextjs/langchain	✅
nextjs/openai	✅
nextjs/vercel	✅
node/google-genai	✅
node/langchain	✅
node/openai	✅
node/vercel	✅
python/google-genai	✅_{a, blk} ✅_{s, blk}
python/langchain	❌_{a, blk} ❌_{s, blk}
python/litellm	❌_{a, blk} ✅_{s, blk}
python/manual	✅_{a, blk} ✅_{s, blk}
python/openai	✅_{a, blk} ✅_{s, blk}

LLM Tests

SDK	Basic Error LLM Test	Basic LLM Test	Conversation ID LLM Test	Long Input LLM Test	Multi-Turn LLM Test	Vision LLM Test
browser/anthropic	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
browser/google-genai	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅🔧_blk ✅_str	✅_blk ✅_str
browser/langchain	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ❌📉_str	✅_blk ✅_str
browser/openai	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}
cloudflare/anthropic	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
cloudflare/google-genai	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
cloudflare/langchain	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅🔧_str
cloudflare/openai	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}
nextjs/anthropic	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
nextjs/google-genai	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
nextjs/langchain	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
nextjs/openai	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}
node/anthropic	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
node/google-genai	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
node/langchain	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str	✅_blk ✅_str
node/manual	—	✅	✅	✅	✅	✅
node/openai	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}	🗑️_blk ✅🆕_{blk, chat} ✅🆕_{blk, responses} 🗑️_str ✅🆕_{str, chat} ✅🆕_{str, responses}
python/anthropic	✅_{a, blk} ❌_{a, str} ✅_{s, blk} ❌_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}
python/google-genai	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ❌📉_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	❌_{a, blk} ❌_{a, str} ❌_{s, blk} ❌_{s, str}
python/langchain	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}	✅_{a, blk} ✅_{a, str} ✅_{s, blk} ✅_{s, str}
python/litellm	❌_{a, blk} ❌_{a, str} ❌_{s, blk} ❌_{s, str}	❌_{a, blk} ❌_{a, str} ✅_{s, blk} ✅_{s, str}	❌_{a, blk} ❌_{a, str} ✅_{s, blk} ✅_{s, str}	❌_{a, blk} ❌_{a, str} ✅_{s, blk} ✅_{s, str}	❌_{a, blk} ❌_{a, str} ✅_{s, blk} ✅_{s, str}	❌_{a, blk} ❌_{a, str} ✅_{s, blk} ✅_{s, str}
python/manual	—	✅_{a, blk} ✅_{s, blk}	✅_{a, blk} ✅_{s, blk}	✅_{a, blk} ✅_{s, blk}	✅_{a, blk} ✅_{s, blk}	✅_{a, blk} ✅_{s, blk}
python/openai	🗑️_{a, blk} ✅🆕_{a, blk, chat} ✅🆕_{a, blk, responses} 🗑️_{a, str} ✅🆕_{a, str, chat} ✅🆕_{a, str, responses} 🗑️_{s, blk} ✅🆕_{s, blk, chat} ✅🆕_{s, blk, responses} 🗑️_{s, str} ✅🆕_{s, str, chat} ✅🆕_{s, str, responses}	🗑️_{a, blk} ✅🆕_{a, blk, chat} ✅🆕_{a, blk, responses} 🗑️_{a, str} ❌🆕_{a, str, chat} ✅🆕_{a, str, responses} 🗑️_{s, blk} ✅🆕_{s, blk, chat} ✅🆕_{s, blk, responses} 🗑️_{s, str} ❌🆕_{s, str, chat} ✅🆕_{s, str, responses}	🗑️_{a, blk} ✅🆕_{a, blk, chat} ✅🆕_{a, blk, responses} 🗑️_{a, str} ❌🆕_{a, str, chat} ✅🆕_{a, str, responses} 🗑️_{s, blk} ✅🆕_{s, blk, chat} ✅🆕_{s, blk, responses} 🗑️_{s, str} ❌🆕_{s, str, chat} ✅🆕_{s, str, responses}	🗑️_{a, blk} ✅🆕_{a, blk, chat} ✅🆕_{a, blk, responses} 🗑️_{a, str} ❌🆕_{a, str, chat} ✅🆕_{a, str, responses} 🗑️_{s, blk} ✅🆕_{s, blk, chat} ✅🆕_{s, blk, responses} 🗑️_{s, str} ❌🆕_{s, str, chat} ✅🆕_{s, str, responses}	🗑️_{a, blk} ✅🆕_{a, blk, chat} ✅🆕_{a, blk, responses} 🗑️_{a, str} ❌🆕_{a, str, chat} ✅🆕_{a, str, responses} 🗑️_{s, blk} ✅🆕_{s, blk, chat} ✅🆕_{s, blk, responses} 🗑️_{s, str} ❌🆕_{s, str, chat} ✅🆕_{s, str, responses}	🗑️_{a, blk} ❌🆕_{a, blk, chat} ❌🆕_{a, blk, responses} 🗑️_{a, str} ❌🆕_{a, str, chat} ❌🆕_{a, str, responses} 🗑️_{s, blk} ❌🆕_{s, blk, chat} ❌🆕_{s, blk, responses} 🗑️_{s, str} ❌🆕_{s, str, chat} ❌🆕_{s, str, responses}

MCP Tests

SDK	Basic MCP Tool Call Test	MCP Multiple Tool Calls Test	MCP Prompt Get Test	MCP Resource Read Test	MCP Tool Error Test
node/mcp	✅_sse ✅_io	✅_sse ✅_io	✅_sse ✅_io	✅_sse ✅_io	✅_sse ✅_io
python/fastmcp	✅_{a, blk, sse} ✅_{a, blk, io}	✅_{a, blk, sse} ✅_{a, blk, io}	✅_{a, blk, sse} ✅_{a, blk, io}	✅_{a, blk, sse} ✅_{a, blk, io}	✅_{a, blk, sse} ✅_{a, blk, io}
python/mcp	✅_{a, blk, sse, hi} ✅_{a, blk, sse, lo} ✅_{a, blk, io, hi} ✅_{a, blk, io, lo}	✅_{a, blk, sse, hi} ✅_{a, blk, sse, lo} ✅_{a, blk, io, hi} ✅_{a, blk, io, lo}	✅_{a, blk, sse, hi} ✅_{a, blk, sse, lo} ✅_{a, blk, io, hi} ✅_{a, blk, io, lo}	✅_{a, blk, sse, hi} ✅_{a, blk, sse, lo} ✅_{a, blk, io, hi} ✅_{a, blk, io, lo}	✅_{a, blk, sse, hi} ✅_{a, blk, sse, lo} ✅_{a, blk, io, hi} ✅_{a, blk, io, lo}

Generated by AI SDK Integration Tests

feat(openai): add variants for the responses API

88e249b

constantinius requested review from alexander-alderman-webb and nicohrubec April 13, 2026 14:10

fix: detection of what is constituted as a chat span

bea498e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(openai): add variants for the responses API#142

feat(openai): add variants for the responses API#142
constantinius wants to merge 2 commits intomainfrom
constantinius/feat/opeani/test-responses-api

constantinius commented Apr 13, 2026

Uh oh!

linear-code bot commented Apr 13, 2026

Uh oh!

github-actions bot commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant