feat: cloudflare ai gateway support #6966
Conversation
typpo
requested review from
a team,
MrFlounder,
mldangelo and
swarnap
as code owners
Security Review ✅No critical issues found. 🟡 Minor Observations (3 items)
Last updated: 2026-01-10 | Reviewing: 0d307bd |
![promptfoo-scanner[bot]](https://avatars.githubusercontent.com/in/2172945?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 All Clear
I reviewed this Cloudflare AI Gateway integration PR for LLM security vulnerabilities. The code implements a proxy layer for routing LLM API calls through Cloudflare's gateway infrastructure. API keys and authentication tokens are properly handled in HTTP headers following standard practices, and the implementation follows secure patterns appropriate for an LLM testing framework.
Minimum severity threshold for this scan: 🟡 Medium | Learn more
📝 WalkthroughWalkthroughThis pull request introduces a new Cloudflare AI Gateway provider integration. It adds a provider module (src/providers/cloudflare-gateway.ts) with two classes—CloudflareGatewayOpenAiProvider and CloudflareGatewayAnthropicProvider—and a factory function to instantiate them. The provider routes API calls for multiple underlying providers (OpenAI, Anthropic, Groq, and others) through Cloudflare's AI Gateway. A registry entry is added to enable provider discovery. Documentation includes a comprehensive provider guide, an example README, a sample configuration file with test cases, and extensive unit tests covering provider creation, URL construction, header management, error handling, and environment variable resolution. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~40 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Fix all issues with AI agents
In @examples/cloudflare-gateway/promptfooconfig.yaml:
- Line 1: Add the required top-level schema reference to promptfooconfig.yaml by
inserting the $schema key as the first line of the file and set it to the
official Promptfoo schema URI (the Promptfoo config schema), ensuring the schema
reference is present so the file validates against the expected config schema.
In @site/docs/providers/cloudflare-gateway.md:
- Around line 1-4: Add a sidebar_position field to the front matter of the
Cloudflare AI Gateway docs page by updating the YAML block at the top (the
existing --- ... ---). Insert a line like "sidebar_position: <number>" with an
appropriate integer to control ordering (e.g., 10) immediately after the
description or sidebar_label so the page includes the required front-matter
field.
- Line 3: The front-matter "description" value in cloudflare-gateway.md is ~180
characters and must be shortened to 150–160 characters for SEO; replace the
existing description string in the document's front-matter (the description
field) with a concise 150–160 character version (you can use the suggested
shorter text from the review) so it stays within the target length.
In @src/providers/cloudflare-gateway.ts:
- Around line 28-92: Update the Azure OpenAI API version from
"2024-02-15-preview" to the GA version "2024-10-21" in both places: the JSDoc
for CloudflareGatewayConfig.apiVersion and the code location where a default API
version is assigned (update the default value used when reading config, e.g.,
where apiVersion is initialized/merged). Locate the CloudflareGatewayConfig
interface (apiVersion JSDoc) and the code that references/sets config.apiVersion
(the default assignment used when constructing the provider) and replace the
string to "2024-10-21" in both spots.
- Around line 255-323: The constructor currently sets apiKeyEnvar for Azure
which causes the parent OpenAiChatCompletionProvider to add an Authorization:
Bearer header in addition to the manual headers['api-key'] and breaks Azure
auth; change the logic in the constructor to NOT set apiKeyEnvar when
underlyingProvider === 'azure-openai' and instead only place the Azure key into
headers (use providerOptions.config?.apiKey or
getEnvString('AZURE_OPENAI_API_KEY') as you already do) so the config passed to
super(...) omits apiKeyEnvar for Azure while headers still include 'api-key'.
🧹 Nitpick comments (3)
examples/cloudflare-gateway/promptfooconfig.yaml (1)
31-31: Consider using a specific Anthropic model identifier.While
claude-3-5-haiku-latestworks, using a specific date-based model identifier (e.g.,claude-3-5-haiku-20241022) is more reproducible and follows the pattern shown in documentation examples.site/docs/providers/cloudflare-gateway.md (1)
20-22: Add language identifiers to fenced code blocks.The static analysis tool flags these fenced code blocks as missing language specifiers. While they contain text/URL patterns rather than code, adding
textas the language identifier improves clarity and resolves the linter warnings.🔧 Add language identifiers
-``` +```text cloudflare-gateway:{provider}:{model}Apply similar changes to the code blocks at lines 215 and 221. </details> Also applies to: 215-217, 221-223 </blockquote></details> <details> <summary>src/providers/cloudflare-gateway.ts (1)</summary><blockquote> `350-366`: **Type-unsafe access to `top_k` parameter.** Line 356 accesses `top_k` using a type assertion because it's not defined in `CloudflareGatewayConfig`. While this works, it bypasses TypeScript's type safety. <details> <summary>♻️ Consider adding top_k to CloudflareGatewayConfig</summary> If `top_k` is a supported parameter for Anthropic through the gateway, consider adding it to the interface: ```diff export interface CloudflareGatewayConfig extends OpenAiCompletionOptions { /** Cloudflare account ID */ accountId?: string; // ... other fields ... + /** Top-k sampling parameter (Anthropic) */ + top_k?: number; }This would provide better type safety and IDE autocomplete support.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
examples/cloudflare-gateway/README.mdexamples/cloudflare-gateway/promptfooconfig.yamlsite/docs/providers/cloudflare-gateway.mdsrc/providers/cloudflare-gateway.tssrc/providers/registry.tstest/providers/cloudflare-gateway.test.ts
🧰 Additional context used
📓 Path-based instructions (11)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use TypeScript with strict type checking
Files:
src/providers/registry.tstest/providers/cloudflare-gateway.test.tssrc/providers/cloudflare-gateway.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{ts,tsx,js,jsx}: Follow consistent import order (Biome handles sorting)
Use consistent curly braces for all control statements
Preferconstoverlet; avoidvar
Use object shorthand syntax whenever possible
Useasync/awaitfor asynchronous code
Use consistent error handling with proper type checks
Avoid re-exporting from files; import directly from the source module
Use the logger with object context (auto-sanitized)
Files:
src/providers/registry.tstest/providers/cloudflare-gateway.test.tssrc/providers/cloudflare-gateway.ts
src/providers/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Document provider configurations following examples in existing code
Files:
src/providers/registry.tssrc/providers/cloudflare-gateway.ts
src/providers/**/*.ts
📄 CodeRabbit inference engine (src/providers/AGENTS.md)
src/providers/**/*.ts: Each provider must implement theApiProviderinterface fromsrc/types/providers.ts
Provider must transform prompts into provider-specific API format and return normalizedProviderResponse
If provider allocates resources (Python workers, connections, child processes), implement acleanup()method and register withproviderRegistry
ALWAYS set thecached: trueflag when returning a cached response in provider implementations
Use logger with object context for logging in providers (auto-sanitized)
Files:
src/providers/registry.tssrc/providers/cloudflare-gateway.ts
examples/**/README.md
📄 CodeRabbit inference engine (examples/AGENTS.md)
examples/**/README.md: Each example directory must include aREADME.mdfile starting with the heading# folder-name (Human Readable Name)
Document all required environment variables in the example README
Files:
examples/cloudflare-gateway/README.md
site/docs/**/*.md
📄 CodeRabbit inference engine (site/AGENTS.md)
site/docs/**/*.md: Don't modify heading text in documentation (often externally linked)
Avoid embellishment words like 'sophisticated' in documentation
Include front matter with title (under 60 chars), description (150-160 chars), and sidebar_position on all documentation pages
Addtitle="filename.yaml"attribute only to code blocks containing complete, runnable files
Don't add titles to code block fragments (only complete, runnable files)
Use// highlight-next-linedirective to emphasize specific lines in code blocks
Never remove existing highlight directives from code blocks
Use admonition syntax (:::note, :::warning, :::danger) with empty lines around content
Files:
site/docs/providers/cloudflare-gateway.md
site/**/*.{md,ts,tsx,js,json}
📄 CodeRabbit inference engine (site/AGENTS.md)
site/**/*.{md,ts,tsx,js,json}: Use 'eval' not 'evaluation' in commands and documentation
Use 'Promptfoo' when referring to the company or product, 'promptfoo' when referring to the CLI command or in code
Files:
site/docs/providers/cloudflare-gateway.md
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.test.{ts,tsx}: Use Vitest for all tests (bothtest/andsrc/app/)
Test both success and error cases for all functionality
Files:
test/providers/cloudflare-gateway.test.ts
test/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Backend tests in
test/should use Vitest with globals enabled (describe,it,expectavailable without imports)
test/**/*.test.{ts,tsx}: NEVER use.only()or.skip()in committed code
ALWAYS clean up mocks inafterEachusingvi.resetAllMocks()to prevent test pollution
All tests require explicit imports from vitest:import { afterEach, describe, expect, it, vi } from 'vitest'
Use Vitest's mocking utilities (vi.mock,vi.fn,vi.spyOn). Prefer shallow mocking over deep mocking. Mock external dependencies but not the code being tested.
Forvi.hoisted()mocks or mocks withmockReturnValue(), callmockReset()inbeforeEachto ensure test isolation when tests run in random order
Usevi.clearAllMocks()to clear call history, but usemockReset()for full isolation including mock implementations
For database tests, use in-memory instances or proper test fixtures
Files:
test/providers/cloudflare-gateway.test.ts
test/providers/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (test/AGENTS.md)
Every provider test must cover: (1) success case, (2) error cases (4xx, 5xx, rate limits), (3) configuration validation, (4) token usage tracking
Files:
test/providers/cloudflare-gateway.test.ts
examples/**/promptfooconfig.yaml
📄 CodeRabbit inference engine (examples/AGENTS.md)
examples/**/promptfooconfig.yaml: Test example configurations with local build usingnpm run local -- eval -c examples/<example-name>/promptfooconfig.yamlinstead of the published version
Each example directory must include apromptfooconfig.yamlwith schema referencehttps://promptfoo.dev/config-schema.json
Use YAML field order in promptfooconfig.yaml: description, env, prompts, providers, defaultTest, scenarios, tests
Use current model identifiers for OpenAI, Anthropic, and Google providers from the documentation insite/docs/providers/
Usefile://prefix for external file references in promptfooconfig.yaml
Files:
examples/cloudflare-gateway/promptfooconfig.yaml
🧠 Learnings (21)
📓 Common learnings
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: When adding a provider, implement `ApiProvider` interface, add tests in `test/providers/`, add docs in `site/docs/providers/`, and add example in `examples/`
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: examples/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:22.578Z
Learning: Applies to examples/**/promptfooconfig.yaml : Use current model identifiers for OpenAI, Anthropic, and Google providers from the documentation in `site/docs/providers/`
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/**/*.ts : Each provider must implement the `ApiProvider` interface from `src/types/providers.ts`
Applied to files:
src/providers/registry.tssrc/providers/cloudflare-gateway.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to src/providers/**/*.{ts,tsx} : Document provider configurations following examples in existing code
Applied to files:
src/providers/registry.ts
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/test/providers/**/*.test.{ts,tsx} : Provider tests must cover success cases, error cases, rate limits, timeouts, and invalid configs
Applied to files:
src/providers/registry.tstest/providers/cloudflare-gateway.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/providers/**/*.test.{ts,tsx} : Every provider test must cover: (1) success case, (2) error cases (4xx, 5xx, rate limits), (3) configuration validation, (4) token usage tracking
Applied to files:
src/providers/registry.tstest/providers/cloudflare-gateway.test.ts
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/README.md : Include a comprehensive README.md that explains the purpose, prerequisites, instructions, and expected outputs for the example
Applied to files:
examples/cloudflare-gateway/README.md
📚 Learning: 2025-07-18T17:25:38.445Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.445Z
Learning: Applies to examples/*/README.md : Document any required API keys or credentials in the README
Applied to files:
examples/cloudflare-gateway/README.md
📚 Learning: 2025-12-09T06:08:22.578Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: examples/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:22.578Z
Learning: Applies to examples/**/promptfooconfig.yaml : Use current model identifiers for OpenAI, Anthropic, and Google providers from the documentation in `site/docs/providers/`
Applied to files:
examples/cloudflare-gateway/README.mdsite/docs/providers/cloudflare-gateway.mdtest/providers/cloudflare-gateway.test.tsexamples/cloudflare-gateway/promptfooconfig.yamlsrc/providers/cloudflare-gateway.ts
📚 Learning: 2025-11-29T00:24:11.501Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/CLAUDE.md:0-0
Timestamp: 2025-11-29T00:24:11.501Z
Learning: Applies to src/providers/**/AGENTS.md : Maintain comprehensive AGENTS.md documentation file with agent descriptions, capabilities, and usage examples
Applied to files:
site/docs/providers/cloudflare-gateway.md
📚 Learning: 2025-07-18T17:25:38.445Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.445Z
Learning: Applies to examples/*/promptfooconfig*.yaml : Include a mix of providers when comparing model performance in configuration files
Applied to files:
site/docs/providers/cloudflare-gateway.mdexamples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/promptfooconfig*.yaml : For OpenAI, prefer models like 'openai:o3-mini' and 'openai:gpt-4o-mini' in configuration files
Applied to files:
site/docs/providers/cloudflare-gateway.mdexamples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/test/providers/**/*.test.{ts,tsx} : Tests must NEVER make real API calls - mock all HTTP requests using `vi.mock`
Applied to files:
test/providers/cloudflare-gateway.test.ts
📚 Learning: 2025-12-20T02:05:23.936Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:23.936Z
Learning: Applies to src/redteam/test/redteam/**/*.{ts,js} : Add tests for new plugins in the test/redteam/ directory following the reference pattern in src/redteam/plugins/pii.ts
Applied to files:
test/providers/cloudflare-gateway.test.ts
📚 Learning: 2025-12-09T06:08:22.578Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: examples/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:22.578Z
Learning: Applies to examples/**/promptfooconfig.yaml : Use YAML field order in promptfooconfig.yaml: description, env, prompts, providers, defaultTest, scenarios, tests
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/promptfooconfig*.yaml : For trivial test cases in configuration, make them quirky and fun to increase engagement
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/promptfooconfig*.yaml : Follow the specific field order in all configuration files: description, env (optional), prompts, providers, defaultTest (optional), scenarios (optional), tests
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/promptfooconfig*.yaml : Include a working promptfooconfig.yaml (or equivalent) file in each example
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-12-09T06:08:22.578Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: examples/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:22.578Z
Learning: Applies to examples/**/promptfooconfig.yaml : Each example directory must include a `promptfooconfig.yaml` with schema reference `https://promptfoo.dev/config-schema.json`
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.445Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.445Z
Learning: Applies to examples/*/promptfooconfig*.yaml : Format configuration files consistently
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.444Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.444Z
Learning: Applies to examples/*/promptfooconfig*.yaml : For Anthropic, prefer models like 'anthropic:claude-3-7-sonnet-20250219' in configuration files
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
📚 Learning: 2025-07-18T17:25:38.445Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/examples.mdc:0-0
Timestamp: 2025-07-18T17:25:38.445Z
Learning: Applies to examples/*/promptfooconfig*.yaml : When demonstrating specialized capabilities (vision, audio, etc.), use models that support those features in configuration files
Applied to files:
examples/cloudflare-gateway/promptfooconfig.yaml
🧬 Code graph analysis (3)
src/providers/registry.ts (3)
src/types/providers.ts (1)
ProviderOptions(42-51)src/types/index.ts (1)
LoadApiProviderContext(1275-1279)src/providers/cloudflare-gateway.ts (1)
createCloudflareGatewayProvider(446-481)
test/providers/cloudflare-gateway.test.ts (1)
src/providers/cloudflare-gateway.ts (4)
CloudflareGatewayConfig(34-55)createCloudflareGatewayProvider(446-481)CloudflareGatewayOpenAiProvider(252-345)CloudflareGatewayAnthropicProvider(373-427)
src/providers/cloudflare-gateway.ts (4)
src/providers/openai/types.ts (1)
OpenAiCompletionOptions(112-185)src/envars.ts (2)
getEnvString(417-433)EnvVarKey(407-407)src/providers/anthropic/types.ts (1)
AnthropicMessageOptions(41-60)src/types/providers.ts (1)
ApiProvider(94-112)
🪛 markdownlint-cli2 (0.18.1)
site/docs/providers/cloudflare-gateway.md
20-20: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
215-215: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
221-221: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
- GitHub Check: Test on Node 22.x and ubuntu-latest
- GitHub Check: Test on Node 20.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 24.x and windows-latest (shard 3/3)
- GitHub Check: Test on Node 24.x and windows-latest (shard 1/3)
- GitHub Check: Test on Node 22.x and windows-latest (shard 1/3)
- GitHub Check: Test on Node 20.x and macOS-latest
- GitHub Check: Test on Node 22.x and windows-latest (shard 3/3)
- GitHub Check: Test on Node 20.x and windows-latest (shard 3/3)
- GitHub Check: Test on Node 22.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 20.x and windows-latest (shard 1/3)
- GitHub Check: Test on Node 22.x and macOS-latest
- GitHub Check: Test on Node 24.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 24.x and ubuntu-latest
- GitHub Check: Test on Node 20.x and ubuntu-latest
- GitHub Check: Run Smoke Tests
- GitHub Check: Share Test
- GitHub Check: Redteam (Staging API)
- GitHub Check: Build on Node 24.x
- GitHub Check: Style Check
- GitHub Check: Build on Node 22.x
- GitHub Check: Build Docs
- GitHub Check: Redteam (Production API)
- GitHub Check: Build on Node 20.x
- GitHub Check: webui tests
- GitHub Check: security-scan
- GitHub Check: claude-review
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (9)
examples/cloudflare-gateway/README.md (1)
1-48: LGTM! Comprehensive example documentation.The README follows all guidelines:
- Proper heading format
- Documents all required environment variables
- Clear prerequisites, setup instructions, and usage examples
- Explains the benefits of using Cloudflare AI Gateway
- Links to full documentation
src/providers/registry.ts (1)
490-503: LGTM! Provider registry integration is correct.The new Cloudflare Gateway provider factory follows the established pattern:
- Correct path matching using
startsWith('cloudflare-gateway:')- Dynamic import for lazy loading
- Proper environment injection from context
- Consistent with other gateway providers
test/providers/cloudflare-gateway.test.ts (1)
1-773: Excellent test coverage! Consider adding more error scenarios.The test suite is comprehensive and well-structured, covering:
- Provider creation and factory logic
- Configuration validation
- Token usage tracking
- URL construction for various providers (OpenAI, Anthropic, Groq, Azure, Workers AI)
- Environment variable handling
- JSON serialization without exposing API keys
However, the error handling section (lines 375-402) only covers 4xx errors. According to coding guidelines, provider tests must also cover 5xx errors and rate limits.
🧪 Suggested additional error test cases
it('should handle API error responses (5xx)', async () => { const provider = new CloudflareGatewayOpenAiProvider('openai', 'gpt-4o', { config: minimumConfig, }); const mockResponse = { ...defaultMockResponse, status: 503, statusText: 'Service Unavailable', ok: false, text: vi.fn().mockResolvedValue(JSON.stringify({ error: { message: 'Service temporarily unavailable', type: 'server_error', }, })), }; mockFetch.mockResolvedValue(mockResponse); const result = await provider.callApi('Test prompt'); expect(result.error).toBeDefined(); expect(result.error).toContain('503'); }); it('should handle rate limit errors (429)', async () => { const provider = new CloudflareGatewayOpenAiProvider('openai', 'gpt-4o', { config: minimumConfig, }); const mockResponse = { ...defaultMockResponse, status: 429, statusText: 'Too Many Requests', ok: false, text: vi.fn().mockResolvedValue(JSON.stringify({ error: { message: 'Rate limit exceeded', type: 'rate_limit_error', }, })), }; mockFetch.mockResolvedValue(mockResponse); const result = await provider.callApi('Test prompt'); expect(result.error).toBeDefined(); expect(result.error).toContain('429'); });Based on coding guidelines.
src/providers/cloudflare-gateway.ts (6)
1-27: LGTM! Well-documented provider integration.The file header clearly explains the purpose, usage patterns, and gateway URL format. The imports are clean and follow TypeScript conventions.
182-213: LGTM! Proper URL construction with provider-specific handling.The function correctly handles different provider URL formats (standard, Azure OpenAI, Workers AI) with appropriate validation using invariants.
325-345: LGTM! Proper provider identification and secure JSON serialization.The methods correctly implement the provider interface. Setting
apiKey: undefinedintoJSON()(line 341) is good security practice to prevent accidental API key exposure in logs or outputs.
373-427: LGTM! Clean Anthropic provider implementation.The Anthropic provider follows the same pattern as the OpenAI provider, with appropriate config passthrough and security measures. The implementation is straightforward since Anthropic doesn't require special handling like Azure OpenAI does.
446-481: LGTM! Robust factory function with clear validation.The factory function correctly parses provider paths, validates inputs with helpful error messages, and routes to the appropriate provider class. The use of
slice(2).join(':')(line 462) properly handles model names that contain colons.
272-272: No type safety issue exists here.
apiKeyEnvaris properly defined inOpenAiSharedOptions, which is composed intoOpenAiCompletionOptionsas an intersection type. SinceCloudflareGatewayConfigextendsOpenAiCompletionOptions, accessingproviderOptions.config?.apiKeyEnvaron line 272 is fully type-safe and valid.Likely an incorrect or invalid review comment.
| @@ -0,0 +1,74 @@ | |||
| # Cloudflare AI Gateway Example | |||
There was a problem hiding this comment.
Add the required schema reference.
Each example promptfooconfig.yaml must include a schema reference at the top of the file.
📋 Add schema reference
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+
# Cloudflare AI Gateway ExampleBased on coding guidelines.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Cloudflare AI Gateway Example | |
| # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json | |
| # Cloudflare AI Gateway Example |
🤖 Prompt for AI Agents
In @examples/cloudflare-gateway/promptfooconfig.yaml at line 1, Add the required
top-level schema reference to promptfooconfig.yaml by inserting the $schema key
as the first line of the file and set it to the official Promptfoo schema URI
(the Promptfoo config schema), ensuring the schema reference is present so the
file validates against the expected config schema.
| --- | ||
| sidebar_label: Cloudflare AI Gateway | ||
| description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, analytics, and cost tracking with support for OpenAI, Anthropic, Groq, and other providers. | ||
| --- |
There was a problem hiding this comment.
Add sidebar_position to front matter.
All documentation pages require a sidebar_position field in the front matter to control ordering in the navigation.
📝 Add sidebar_position
---
sidebar_label: Cloudflare AI Gateway
+sidebar_position: [appropriate_number]
description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, analytics, and cost tracking with support for OpenAI, Anthropic, Groq, and other providers.
---Note: Choose an appropriate position number based on where this provider should appear in the sidebar.
Based on coding guidelines.
Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In @site/docs/providers/cloudflare-gateway.md around lines 1 - 4, Add a
sidebar_position field to the front matter of the Cloudflare AI Gateway docs
page by updating the YAML block at the top (the existing --- ... ---). Insert a
line like "sidebar_position: <number>" with an appropriate integer to control
ordering (e.g., 10) immediately after the description or sidebar_label so the
page includes the required front-matter field.
| @@ -0,0 +1,248 @@ | |||
| --- | |||
| sidebar_label: Cloudflare AI Gateway | |||
| description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, analytics, and cost tracking with support for OpenAI, Anthropic, Groq, and other providers. | |||
There was a problem hiding this comment.
Shorten the description to 150-160 characters.
The description field should be 150-160 characters for optimal SEO and display. The current description is approximately 180 characters.
✂️ Suggested shorter description
-description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, analytics, and cost tracking with support for OpenAI, Anthropic, Groq, and other providers.
+description: Route AI requests through Cloudflare AI Gateway for caching, rate limiting, and analytics with support for OpenAI, Anthropic, Groq, and more.This brings it to approximately 155 characters.
Based on coding guidelines.
🤖 Prompt for AI Agents
In @site/docs/providers/cloudflare-gateway.md at line 3, The front-matter
"description" value in cloudflare-gateway.md is ~180 characters and must be
shortened to 150–160 characters for SEO; replace the existing description string
in the document's front-matter (the description field) with a concise 150–160
character version (you can use the suggested shorter text from the review) so it
stays within the target length.
| // Cloudflare AI Gateway base URL | ||
| const CLOUDFLARE_GATEWAY_BASE_URL = 'https://gateway.ai.cloudflare.com/v1'; | ||
|
|
||
| /** | ||
| * Configuration for Cloudflare AI Gateway providers | ||
| */ | ||
| export interface CloudflareGatewayConfig extends OpenAiCompletionOptions { | ||
| /** Cloudflare account ID */ | ||
| accountId?: string; | ||
| /** Environment variable name for account ID (default: CLOUDFLARE_ACCOUNT_ID) */ | ||
| accountIdEnvar?: string; | ||
| /** AI Gateway ID */ | ||
| gatewayId?: string; | ||
| /** Environment variable name for gateway ID (default: CLOUDFLARE_GATEWAY_ID) */ | ||
| gatewayIdEnvar?: string; | ||
| /** Optional Cloudflare AI Gateway authentication token for authenticated gateways */ | ||
| cfAigToken?: string; | ||
| /** Environment variable name for cf-aig-authorization token (default: CF_AIG_TOKEN) */ | ||
| cfAigTokenEnvar?: string; | ||
|
|
||
| // Azure OpenAI specific options | ||
| /** Azure OpenAI resource name (required for azure-openai provider) */ | ||
| resourceName?: string; | ||
| /** Azure OpenAI deployment name (required for azure-openai provider) */ | ||
| deploymentName?: string; | ||
| /** Azure OpenAI API version (default: 2024-02-15-preview) */ | ||
| apiVersion?: string; | ||
| } | ||
|
|
||
| export interface CloudflareGatewayProviderOptions extends ProviderOptions { | ||
| config?: CloudflareGatewayConfig; | ||
| } | ||
|
|
||
| /** | ||
| * Provider configuration for routing through Cloudflare AI Gateway | ||
| */ | ||
| interface GatewayProviderConfig { | ||
| /** Default environment variable for the provider's API key */ | ||
| apiKeyEnvar: string; | ||
| } | ||
|
|
||
| /** | ||
| * Supported provider configurations for Cloudflare AI Gateway | ||
| * | ||
| * Note: Some providers have special URL requirements: | ||
| * - azure-openai: Requires resourceName and deploymentName in config | ||
| * - workers-ai: Model name is appended to URL path | ||
| * | ||
| * AWS Bedrock is NOT supported because it requires AWS request signing | ||
| * which is incompatible with the gateway proxy approach. | ||
| */ | ||
| const PROVIDER_CONFIGS: Record<string, GatewayProviderConfig> = { | ||
| openai: { apiKeyEnvar: 'OPENAI_API_KEY' }, | ||
| anthropic: { apiKeyEnvar: 'ANTHROPIC_API_KEY' }, | ||
| groq: { apiKeyEnvar: 'GROQ_API_KEY' }, | ||
| 'perplexity-ai': { apiKeyEnvar: 'PERPLEXITY_API_KEY' }, | ||
| 'google-ai-studio': { apiKeyEnvar: 'GOOGLE_API_KEY' }, | ||
| mistral: { apiKeyEnvar: 'MISTRAL_API_KEY' }, | ||
| cohere: { apiKeyEnvar: 'COHERE_API_KEY' }, | ||
| 'azure-openai': { apiKeyEnvar: 'AZURE_OPENAI_API_KEY' }, | ||
| 'workers-ai': { apiKeyEnvar: 'CLOUDFLARE_API_KEY' }, | ||
| huggingface: { apiKeyEnvar: 'HUGGINGFACE_API_KEY' }, | ||
| replicate: { apiKeyEnvar: 'REPLICATE_API_KEY' }, | ||
| grok: { apiKeyEnvar: 'XAI_API_KEY' }, | ||
| }; |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Azure OpenAI API latest stable version January 2026
💡 Result:
As of January 8, 2026:
- Data-plane (inference) GA: api-version 2024-10-21. [1]
- Control-plane (resource/deployment management) GA: api-version 2025-06-01. [1][2]
Note: Microsoft also offers a v1 "next‑generation" opt‑in model (v1) for ongoing access to the newest features (no monthly api-version updates). [2]
Sources:
[1] Microsoft Learn — Azure OpenAI REST API preview reference / API specs.
[2] Microsoft Learn — Azure OpenAI API version lifecycle (v1 opt‑in details).
🏁 Script executed:
# First, let's find and examine the cloudflare-gateway.ts file to see the actual default value
fd cloudflare-gateway.tsRepository: promptfoo/promptfoo
Length of output: 98
🏁 Script executed:
# Read the cloudflare-gateway.ts file
cat -n src/providers/cloudflare-gateway.ts | head -320 | tail -30Repository: promptfoo/promptfoo
Length of output: 1282
🏁 Script executed:
# Also search for references to the API version in the file
rg "2024-02-15-preview|apiVersion" src/providers/cloudflare-gateway.ts -nRepository: promptfoo/promptfoo
Length of output: 369
Update the Azure OpenAI API version default to the current GA version.
The default API version 2024-02-15-preview (lines 53 and 298) is outdated. As of January 2026, the current stable Azure OpenAI data-plane API version is 2024-10-21. Update both the JSDoc comment on line 53 and the default value on line 298 to use this version.
🤖 Prompt for AI Agents
In @src/providers/cloudflare-gateway.ts around lines 28 - 92, Update the Azure
OpenAI API version from "2024-02-15-preview" to the GA version "2024-10-21" in
both places: the JSDoc for CloudflareGatewayConfig.apiVersion and the code
location where a default API version is assigned (update the default value used
when reading config, e.g., where apiVersion is initialized/merged). Locate the
CloudflareGatewayConfig interface (apiVersion JSDoc) and the code that
references/sets config.apiVersion (the default assignment used when constructing
the provider) and replace the string to "2024-10-21" in both spots.
| function getAccountId(config?: CloudflareGatewayConfig, env?: EnvOverrides): string { | ||
| // Check explicit config value first | ||
| if (config?.accountId) { | ||
| return config.accountId; | ||
| } | ||
|
|
||
| // Check custom environment variable if specified | ||
| if (config?.accountIdEnvar) { | ||
| const customValue = | ||
| getEnvString(config.accountIdEnvar as EnvVarKey) || | ||
| env?.[config.accountIdEnvar as keyof EnvOverrides]; | ||
| if (customValue) { | ||
| return customValue as string; | ||
| } | ||
| logger.warn( | ||
| `[CloudflareGateway] Custom account ID environment variable '${config.accountIdEnvar}' is not set. Falling back to CLOUDFLARE_ACCOUNT_ID.`, | ||
| ); | ||
| } | ||
|
|
||
| // Fall back to default environment variable | ||
| const accountIdCandidate = env?.CLOUDFLARE_ACCOUNT_ID || getEnvString('CLOUDFLARE_ACCOUNT_ID'); | ||
|
|
||
| invariant( | ||
| accountIdCandidate, | ||
| 'Cloudflare account ID required. Supply it via config accountId or accountIdEnvar, or the CLOUDFLARE_ACCOUNT_ID environment variable', | ||
| ); | ||
|
|
||
| return accountIdCandidate; | ||
| } | ||
|
|
||
| /** | ||
| * Get the Cloudflare AI Gateway ID from config or environment | ||
| */ | ||
| function getGatewayId(config?: CloudflareGatewayConfig, env?: EnvOverrides): string { | ||
| // Check explicit config value first | ||
| if (config?.gatewayId) { | ||
| return config.gatewayId; | ||
| } | ||
|
|
||
| // Check custom environment variable if specified | ||
| if (config?.gatewayIdEnvar) { | ||
| const customValue = | ||
| getEnvString(config.gatewayIdEnvar as EnvVarKey) || | ||
| env?.[config.gatewayIdEnvar as keyof EnvOverrides]; | ||
| if (customValue) { | ||
| return customValue as string; | ||
| } | ||
| logger.warn( | ||
| `[CloudflareGateway] Custom gateway ID environment variable '${config.gatewayIdEnvar}' is not set. Falling back to CLOUDFLARE_GATEWAY_ID.`, | ||
| ); | ||
| } | ||
|
|
||
| // Fall back to default environment variable | ||
| const gatewayIdCandidate = env?.CLOUDFLARE_GATEWAY_ID || getEnvString('CLOUDFLARE_GATEWAY_ID'); | ||
|
|
||
| invariant( | ||
| gatewayIdCandidate, | ||
| 'Cloudflare AI Gateway ID required. Supply it via config gatewayId or gatewayIdEnvar, or the CLOUDFLARE_GATEWAY_ID environment variable', | ||
| ); | ||
|
|
||
| return gatewayIdCandidate; | ||
| } | ||
|
|
||
| /** | ||
| * Get the optional Cloudflare AI Gateway authentication token | ||
| */ | ||
| function getCfAigToken(config?: CloudflareGatewayConfig, env?: EnvOverrides): string | undefined { | ||
| return ( | ||
| config?.cfAigToken || | ||
| (config?.cfAigTokenEnvar | ||
| ? getEnvString(config.cfAigTokenEnvar as EnvVarKey) || | ||
| env?.[config.cfAigTokenEnvar as keyof EnvOverrides] | ||
| : undefined) || | ||
| env?.CF_AIG_TOKEN || | ||
| getEnvString('CF_AIG_TOKEN') | ||
| ); | ||
| } |
There was a problem hiding this comment.
Type safety concern with custom environment variable names.
Lines 106, 139, and 167 cast custom environment variable names to EnvVarKey using type assertions. If a user provides an arbitrary string as accountIdEnvar, gatewayIdEnvar, or cfAigTokenEnvar, it may not be a valid EnvVarKey, causing potential issues with getEnvString.
🔍 Consider defensive handling of custom env var names
The current code assumes custom env var names are valid. Consider either:
- Documenting that custom env var names should be pre-registered in the env system, or
- Using a more permissive type when calling getEnvString with custom names
This is a minor edge case since the fallback logic and warning messages handle missing values gracefully.
| constructor( | ||
| underlyingProvider: string, | ||
| modelName: string, | ||
| providerOptions: CloudflareGatewayProviderOptions, | ||
| ) { | ||
| const accountId = getAccountId(providerOptions.config, providerOptions.env); | ||
| const gatewayId = getGatewayId(providerOptions.config, providerOptions.env); | ||
| const gatewayUrl = buildGatewayUrl( | ||
| accountId, | ||
| gatewayId, | ||
| underlyingProvider, | ||
| providerOptions.config, | ||
| modelName, | ||
| ); | ||
| const passthrough = getPassthroughConfig(providerOptions.config); | ||
|
|
||
| const providerConfig = PROVIDER_CONFIGS[underlyingProvider]; | ||
| const apiKeyEnvar = providerOptions.config?.apiKeyEnvar || providerConfig?.apiKeyEnvar; | ||
|
|
||
| // Build headers, adding cf-aig-authorization if token is provided | ||
| const cfAigToken = getCfAigToken(providerOptions.config, providerOptions.env); | ||
| const headers: Record<string, string> = { | ||
| ...(providerOptions.config?.headers || {}), | ||
| }; | ||
| if (cfAigToken) { | ||
| headers['cf-aig-authorization'] = `Bearer ${cfAigToken}`; | ||
| } | ||
|
|
||
| // For Azure OpenAI, add api-key header instead of Authorization | ||
| if (underlyingProvider === 'azure-openai') { | ||
| const azureApiKey = providerOptions.config?.apiKey || getEnvString('AZURE_OPENAI_API_KEY'); | ||
| invariant( | ||
| azureApiKey, | ||
| 'Azure OpenAI API key is required. Set the AZURE_OPENAI_API_KEY environment variable or add apiKey to the provider config.', | ||
| ); | ||
| headers['api-key'] = azureApiKey; | ||
| } | ||
|
|
||
| // Build the final API base URL | ||
| let finalGatewayUrl = gatewayUrl; | ||
|
|
||
| // For Azure OpenAI, append the api-version query parameter | ||
| if (underlyingProvider === 'azure-openai') { | ||
| const apiVersion = providerOptions.config?.apiVersion || '2024-02-15-preview'; | ||
| // The endpoint path will be appended by the OpenAI SDK, so we just set the base | ||
| // Azure format: {baseUrl}/chat/completions?api-version={version} | ||
| finalGatewayUrl = `${gatewayUrl}?api-version=${apiVersion}`; | ||
| } | ||
|
|
||
| const config: OpenAiCompletionOptions = { | ||
| ...passthrough, | ||
| apiKeyEnvar, | ||
| apiBaseUrl: finalGatewayUrl, | ||
| headers: Object.keys(headers).length > 0 ? headers : undefined, | ||
| }; | ||
|
|
||
| super(modelName, { | ||
| ...providerOptions, | ||
| config, | ||
| }); | ||
|
|
||
| this.underlyingProvider = underlyingProvider; | ||
|
|
||
| logger.debug(`[CloudflareGateway] Configured ${underlyingProvider}:${modelName}`, { | ||
| gatewayUrl: finalGatewayUrl, | ||
| hasApiKey: !!providerOptions.config?.apiKey, | ||
| hasCfAigToken: !!cfAigToken, | ||
| }); | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Find OpenAI provider files
fd -t f 'openai' src/providers --type f -e ts -e tsxRepository: promptfoo/promptfoo
Length of output: 45
🏁 Script executed:
# Check OpenAiChatCompletionProvider class structure
rg -n 'class OpenAiChatCompletionProvider' src/providers -A 20 -B 2 --type=tsRepository: promptfoo/promptfoo
Length of output: 1682
🏁 Script executed:
# Find how apiKeyEnvar is used in OpenAI provider
rg -n 'apiKeyEnvar' src/providers/openai --type=ts -B 3 -A 3Repository: promptfoo/promptfoo
Length of output: 2568
🏁 Script executed:
# Check how headers are constructed and Authorization header
rg -n 'Authorization|api-key' src/providers/openai --type=ts -B 2 -A 2Repository: promptfoo/promptfoo
Length of output: 4739
🏁 Script executed:
# Check OpenAiCompletionOptions type definition
rg -n 'interface OpenAiCompletionOptions|type OpenAiCompletionOptions' src --type=ts -A 15Repository: promptfoo/promptfoo
Length of output: 1216
Fix Azure OpenAI authentication header conflict.
For Azure OpenAI, the code manually adds the api-key header but also sets apiKeyEnvar in the config. The parent OpenAiChatCompletionProvider will add an Authorization: Bearer header whenever getApiKey() returns a value (which it will, since apiKeyEnvar is set). Azure OpenAI only uses the api-key header; sending both headers will cause authentication to fail.
Do not set apiKeyEnvar for Azure OpenAI in the Cloudflare Gateway config. Instead, retrieve and set the API key directly in the custom headers without passing apiKeyEnvar to the parent class.
🤖 Prompt for AI Agents
In @src/providers/cloudflare-gateway.ts around lines 255 - 323, The constructor
currently sets apiKeyEnvar for Azure which causes the parent
OpenAiChatCompletionProvider to add an Authorization: Bearer header in addition
to the manual headers['api-key'] and breaks Azure auth; change the logic in the
constructor to NOT set apiKeyEnvar when underlyingProvider === 'azure-openai'
and instead only place the Azure key into headers (use
providerOptions.config?.apiKey or getEnvString('AZURE_OPENAI_API_KEY') as you
already do) so the config passed to super(...) omits apiKeyEnvar for Azure while
headers still include 'api-key'.
…version - Add BYOK (Bring Your Own Keys) documentation explaining how to use Cloudflare's stored API keys - Add Authenticated Gateways section with cfAigToken configuration - Document Anthropic BYOK limitation (SDK requires API key) - Update Azure OpenAI default API version from 2024-02-15-preview to 2024-12-01-preview - Improve example config with schema reference and cfAigToken examples Co-Authored-By: Claude Opus 4.5 <[email protected]>
No description provided.