Summary

Bedrock's Converse API supports native structured JSON output via outputConfig.textFormat (launched February 2025 for Claude 3.5+, Llama, Mistral, and others). However, the gateway currently places response_format into additionalModelRequestFields on the base BedrockConverseChatCompleteConfig, where Bedrock silently ignores it — structured output never actually takes effect for non-Anthropic models.

This PR adds proper mapping from OpenAI's response_format: { type: "json_schema", ... } to Bedrock's native Converse API outputConfig.textFormat parameter on the base config.

Note: The 2.0.0 branch already adds response_format handling for the Anthropic variant via additionalModelRequestFields.output_config (Anthropic's proprietary path). That override takes precedence for Anthropic models. This PR complements that by adding Converse-native outputConfig support for all other Bedrock models (Meta, Cohere, AI21, Amazon, Mistral, etc.) via the base config.

What changed

• src/providers/bedrock/utils.ts

  • Removed response_format from transformAdditionalModelRequestFields() (was a no-op — Bedrock ignores unknown keys in additionalModelRequestFields)
  • Added transformOutputConfig() function that maps:

    // OpenAI format (input)
    { type: "json_schema", json_schema: { schema: {...}, name: "N", description: "D", strict: true } }
    
    // Bedrock Converse format (output)  
    { textFormat: { type: "json_schema", structure: { jsonSchema: { schema: "...", name: "N", description: "D" } } } }
    

  • Handles schema stringification: OpenAI sends schema as a JSON object, Bedrock expects a JSON string
  • Passes through name (defaults to "response") and description (omitted when not provided)
  • Guards against missing schema to prevent malformed requests
  • strict is intentionally dropped — Bedrock has no equivalent

• src/providers/bedrock/chatComplete.ts

  • Added response_format → outputConfig entry to BedrockConverseChatCompleteConfig
  • Imported transformOutputConfig from utils

• src/providers/bedrock/utils.test.ts (new)

  • 14 unit tests covering: json_schema mapping, schema stringification (object and string), name defaults, description pass-through and omission, missing schema guard, json_object/text/absent response_format, and verification that response_format is no longer in additionalModelRequestFields

How it interacts with the Anthropic variant

The BedrockConverseAnthropicChatCompleteConfig in 2.0.0 already overrides response_format to route through transformAnthropicAdditionalModelRequestFields, which maps it to additionalModelRequestFields.output_config. Since the spread + override pattern means the Anthropic-specific entry takes precedence, Anthropic models are unaffected by this change. This PR only affects the base Converse config used by non-Anthropic models.

Design decisions

1. Only json_schema type is mapped — Bedrock Converse does not support json_object. For json_object and text types, transformOutputConfig returns undefined so no outputConfig is set (same behavior as before).

2. strict is intentionally dropped — OpenAI's strict field has no Bedrock equivalent. Documented in the JSDoc.

3. No model filtering — Rather than maintaining a list of which Bedrock models support structured output, we let Bedrock return a 400 ValidationException for unsupported models. This matches how other unsupported features (e.g., tool use on models without it) are handled.

4. No streaming changes needed — The same chatComplete config is used for both Converse and ConverseStream API calls, so structured output works for both.

5. Scoped to Bedrock only — All changes are within src/providers/bedrock/. Zero impact on any other provider.

Context

We discovered this issue while building a Terraform PR review agent that uses Portkey to route structured output calls to Bedrock (Claude Sonnet via Converse API). The model returned correct decisions but empty arrays in the structured response — because without outputConfig, Bedrock has no schema to enforce and the model was free to minimize output. Switching to tool-use (tools + tool_choice) as a workaround confirmed the model was capable; the gateway translation was the gap.

Bedrock API Reference

• Converse API — outputConfig parameter
• OutputConfig — textFormat structure
• TextFormat — json_schema type with JsonSchemaFormat
• JsonSchemaDefinition — schema (string), name, description

Test plan

• 14 unit tests pass (npx jest src/providers/bedrock/utils.test.ts)
• Prettier + lint-staged checks pass
• Rollup build succeeds
• Rebased onto 2.0.0 — no conflicts
• Integration test with Bedrock Converse + non-Anthropic model using response_format: { type: "json_schema" }