Skip to content

Add maxTokens support to WorkersAi chat and handle non-string responses#459

Merged
threepointone merged 2 commits intocloudflare:mainfrom
TimoWilhelm:main
Mar 25, 2026
Merged

Add maxTokens support to WorkersAi chat and handle non-string responses#459
threepointone merged 2 commits intocloudflare:mainfrom
TimoWilhelm:main

Conversation

@TimoWilhelm
Copy link
Copy Markdown
Contributor

  • Extract maxTokens from options in chatStream and pass as max_tokens to both streaming and non-streaming run. - Update createWorkersAiBindingFetch to stringify object responses when building assistant messages instead of defaulting to empty string.

Extract maxTokens from options in chatStream and pass as max_tokens to both streaming and non-streaming binding.run calls. Update createWorkersAiBindingFetch to stringify object responses when building assistant messages instead of defaulting to empty string.
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 23, 2026

🦋 Changeset detected

Latest commit: a375d3f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@cloudflare/tanstack-ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Mar 23, 2026

Open in StackBlitz

npx https://pkg.pr.new/cloudflare/ai/ai-gateway-provider@459
npx https://pkg.pr.new/cloudflare/ai/@cloudflare/tanstack-ai@459
npx https://pkg.pr.new/cloudflare/ai/workers-ai-provider@459

commit: a375d3f

Add support for forwarding maxTokens from TextOptions to the Workers AI binding as max_tokens (streaming and non-streaming paths). Update createWorkersAiBindingFetch to stringify object responses from the binding when building assistant messages and default to an empty string for non-string/non-object or null responses. Add tests to verify object stringification, null/non-object handling, and max_tokens forwarding, and include a changeset entry documenting the patch.
@threepointone
Copy link
Copy Markdown
Collaborator

Thanks for the PR! I've pushed a few additions on top of your commit:

Formatting fix: Broke the long ternary in create-fetcher.ts (line 379) into a multi-line expression for readability.

Changeset: Added .changeset/workers-ai-max-tokens.md (patch bump for @cloudflare/tanstack-ai).

Tests (6 new, all passing):

In workers-ai-adapter.test.ts:

  • should forward maxTokens to the binding as max_tokens — verifies maxTokens: 256 becomes max_tokens: 256 on the binding call
  • should not send max_tokens when maxTokens is not set — verifies no spurious max_tokens when omitted

In binding-fetch.test.ts:

  • should stringify object responses in non-streaming mode — verifies object response gets JSON.stringify'd
  • should default to empty string for non-string non-object responses — verifies numeric responses fall through to ""
  • should handle null response field gracefully — verifies null doesn't crash
  • should pass max_tokens to binding when provided — verifies the binding fetch layer forwards max_tokens

Note: workers-ai-provider already handles both maxTokens and non-string responses correctly, so no changes needed there.

@threepointone threepointone merged commit c0eabd6 into cloudflare:main Mar 25, 2026
3 checks passed
@github-actions github-actions bot mentioned this pull request Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants