feat: translate response_format to Anthropic output_config for structured output#5640
Open
03-CiprianoG wants to merge 1 commit intoHelicone:mainfrom
Open
feat: translate response_format to Anthropic output_config for structured output#564003-CiprianoG wants to merge 1 commit intoHelicone:mainfrom
03-CiprianoG wants to merge 1 commit intoHelicone:mainfrom
Conversation
…ured output
The toAnthropic() request translator now maps OpenAI's
response_format: { type: "json_schema", json_schema: { schema, name } }
to Anthropic's native output_config: { format: { type: "json_schema", json_schema: { schema, name } } }.
Previously, response_format was ignored during translation, causing
Anthropic to return prose instead of structured JSON. This broke
Output.object(), generateObject(), and any structured output workflow
when using Anthropic models through the AI Gateway.
Fixes Helicone#5639
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
|
@03-CiprianoG is attempting to deploy a commit to the Helicone Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5639
Problem
When using the AI Gateway with Anthropic models,
response_format: { type: "json_schema" }is passed through untranslated. Anthropic ignores this OpenAI-specific parameter, so the model returns prose instead of JSON — breakingOutput.object(),generateObject(), and any structured output workflow.Solution
Map OpenAI's
response_formatto Anthropic's nativeoutput_config.formatintoAnthropic().Translation
Note:
strictis omitted — it's OpenAI-specific and not part of Anthropic's API.Changes
packages/llm-mapper/transform/types/anthropic.ts: Addedoutput_configandAnthropicOutputConfigtype toAnthropicRequestBodypackages/llm-mapper/transform/providers/openai/request/toAnthropic.ts: Addedresponse_format→output_configtranslation intoAnthropic()Testing
The fix can be verified by sending a request with
response_format: { type: "json_schema", json_schema: { schema: { type: "object", properties: { name: { type: "string" } } } } }to an Anthropic model through the gateway. The model should now return valid JSON matching the schema instead of prose.