chore(lint): add Biome rules for promise and async error detection #7006
Conversation
Enable new Biome lint rules to catch common promise/async errors: - noAsyncPromiseExecutor: prevents async Promise executors - noThenProperty: prevents accidental thenable objects - noDebugger: catches leftover debugger statements - useErrorMessage: requires error messages (warning level) Also fix all violations of useIterableCallbackReturn rule by converting expression arrow functions to block bodies in forEach callbacks. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
mldangelo
requested review from
a team,
JustinBeckwith,
MrFlounder,
faizanminhas,
swarnap,
typpo,
will-holley and
yash2998chhabria
as code owners
|
⏩ PR identified as a refactor (d780fbb) View output ↗ View check history
|
Security Review ✅No critical issues found. This PR adds Biome linting rules for promise/async error detection and fixes violations across the codebase. Summary of changes:
🟡 Minor Observations (2 items)
Last updated: 2026-01-10 20:58:00 UTC | Reviewing: d780fbb |
![promptfoo-scanner[bot]](https://avatars.githubusercontent.com/in/2172945?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 All Clear
This PR introduces linting configuration changes and code style refactoring. No LLM-related code paths were modified, and no LLM security vulnerabilities were identified.
Minimum severity threshold for this scan: 🟡 Medium | Learn more
| transcript: string; | ||
| }; | ||
| }> { | ||
| return new Promise(async (resolve, reject) => { |
There was a problem hiding this comment.
that is absolutely cursed
📝 WalkthroughWalkthroughThe pull request applies a widespread refactoring across approximately 40 files, primarily converting single-line arrow function bodies to explicit multi-line blocks with braces in forEach callbacks. In parallel, it introduces functional changes: updates biome.json to add linting rules (correctness, suspicious, performance, a11y, and nursery sections), adds Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
src/providers/google/util.ts (1)
970-970: Error message added for lint compliance.The new error message satisfies the
useErrorMessageBiome rule. However, note that this specific error is caught immediately on line 972 and replaced with a generic message, so the "Unrecognized function call format" text won't reach end users.💡 Optional: Preserve specific error context
If you want the specific error message to be more informative, consider capturing and including it in the outer catch block:
} catch { - throw new Error( - `Google did not return a valid-looking function call: ${JSON.stringify(output)}`, - ); + throw new Error( + `Google did not return a valid-looking function call: ${JSON.stringify(output)}. Ensure the output matches either the Live format (with 'toolCall'), Vertex/AIS format (array with 'functionCall'), or is valid JSON.`, + ); }Alternatively, capture the error to preserve the inner message:
- } catch { + } catch (err) { throw new Error( - `Google did not return a valid-looking function call: ${JSON.stringify(output)}`, + `Google did not return a valid-looking function call: ${JSON.stringify(output)}. ${err instanceof Error ? err.message : ''}`, ); }src/testCase/synthesis.ts (1)
58-68: OK, but consider a typed filter instead of.filter(Boolean)(optional). Returningnullis fine, but.filter(Boolean)also drops empty strings (and doesn’t always narrow types cleanly).Proposed tweak
- }) - .filter(Boolean) + }) + .filter((x): x is string => typeof x === 'string' && x.length > 0) .sort()src/app/src/pages/eval/components/ResultsTable.test.tsx (1)
51-54: Consider using theimportOriginalparameter for more idiomatic typing and consistency with Vitest best practices.The async
vi.mockfactory withawait vi.importActualworks correctly in Vitest 4.0.14. However, the recommended pattern is to use theimportOriginalparameter passed to the factory function, which provides better TypeScript type inference:Suggested refactor
vi.mock('react-router-dom', async (importOriginal) => { const actual = await importOriginal<typeof import('react-router-dom')>(); return { ...actual, useNavigate: vi.fn(() => vi.fn()), }; });This approach aligns with Vitest's recommended pattern for partial mocking and improves IDE refactoring support.
src/server/server.ts (1)
325-341: Consider adding error handling for the promise chain.The refactor correctly uses
voidto satisfy thenoFloatingPromisesrule. However, ifEval.findById()orEval.latest()rejects, the error will be unhandled since there's no.catch()on the promise chain.🔧 Add error handling to prevent unhandled rejections
const watcher = setupSignalWatcher(() => { // Try to get the specific eval that was updated from the signal file const signalEvalId = readSignalEvalId(); const evalPromise = signalEvalId ? Eval.findById(signalEvalId) : Eval.latest(); void evalPromise.then(async (updatedEval) => { const results = await updatedEval?.getResultsCount(); if (results && results > 0) { logger.debug( `Emitting update for eval: ${updatedEval?.config?.description || updatedEval?.id || 'unknown'}`, ); io.emit('update', updatedEval); allPrompts = null; } + }).catch((error) => { + logger.warn(`Failed to fetch eval for signal update: ${error instanceof Error ? error.message : error}`); }); });src/logger.ts (1)
184-194: Consider de-duping concurrent calls to prevent handler accumulation.
Thevoid initializeSourceMapSupport()calls insetLogLevel()(line 187) andcreateLogMethod()(line 290) can race—multiple concurrent calls proceed beforesourceMapSupportInitializedflips true. Whilesource-map-support's core formatter is protected by internal flags, its handler stacks (retrieveFile/retrieveSourceMap) accumulate on repeated calls. Use Promise-based deduplication to prevent redundant work and handler buildup.Proposed hardening (single in-flight init)
export let sourceMapSupportInitialized = false; +let sourceMapSupportInitPromise: Promise<void> | null = null; export async function initializeSourceMapSupport(): Promise<void> { - if (!sourceMapSupportInitialized) { - try { - const sourceMapSupport = await import('source-map-support'); - sourceMapSupport.install(); - sourceMapSupportInitialized = true; - } catch { - // Ignore errors. This happens in the production build, because source-map-support is a dev dependency. - } - } + if (sourceMapSupportInitialized) { + return; + } + if (sourceMapSupportInitPromise) { + return sourceMapSupportInitPromise; + } + + sourceMapSupportInitPromise = (async () => { + try { + const sourceMapSupport = await import('source-map-support'); + sourceMapSupport.install(); + sourceMapSupportInitialized = true; + } catch { + // Ignore errors. This happens in the production build, because source-map-support is a dev dependency. + } finally { + sourceMapSupportInitPromise = null; + } + })(); + + return sourceMapSupportInitPromise; }Also applies to: 282-295, 531-534
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (36)
biome.jsonscripts/validate-changelog.cjssite/src/components/Store/useFourthwall.tssrc/app/src/components/traces/TraceTimeline.tsxsrc/app/src/pages/eval/components/CustomMetricsDialog.tsxsrc/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/eval/components/store.test.tssrc/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/cacheMigration.tssrc/commands/eval.tssrc/commands/list.tssrc/commands/mcp/lib/performance.tssrc/logger.tssrc/onboarding.tssrc/providers/google/util.tssrc/providers/openai/realtime.tssrc/providers/xai/voice.tssrc/redteam/index.tssrc/server/server.tssrc/testCase/synthesis.tssrc/tracing/traceContext.tssrc/util/modelAuditCliParser.tssrc/util/templates.tssrc/validators/redteam.tstest/envars.test.tstest/providers/bedrock/converse.test.tstest/providers/openai/realtime.test.tstest/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tstest/redteam/providers/crescendo/index.test.tstest/redteam/providers/custom/index.test.tstest/util/functions/loadFunction.test.tstest/util/generation.test.ts
🧰 Additional context used
📓 Path-based instructions (15)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Use TypeScript with strict type checking
Files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/util/templates.tssrc/cacheMigration.tstest/providers/openai/realtime.test.tssrc/app/src/components/traces/TraceTimeline.tsxsrc/onboarding.tstest/util/generation.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tssrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/tracing/traceContext.tssrc/providers/openai/realtime.tssrc/util/modelAuditCliParser.tssrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/eval/components/store.test.tstest/redteam/providers/crescendo/index.test.tssrc/logger.tssrc/app/src/pages/eval/components/CustomMetricsDialog.tsxsrc/commands/mcp/lib/performance.tssrc/validators/redteam.tstest/envars.test.tssrc/server/server.tssrc/testCase/synthesis.tstest/redteam/providers/custom/index.test.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.tssrc/commands/eval.tssrc/providers/google/util.tssrc/providers/xai/voice.tstest/util/functions/loadFunction.test.tssrc/commands/list.tssite/src/components/Store/useFourthwall.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{ts,tsx,js,jsx}: Follow consistent import order (Biome handles sorting)
Use consistent curly braces for all control statements
Preferconstoverlet; avoidvar
Use object shorthand syntax whenever possible
Useasync/awaitfor asynchronous code
Use consistent error handling with proper type checks
Avoid re-exporting from files; import directly from the source module
Use the logger with object context (auto-sanitized)
Files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/util/templates.tssrc/cacheMigration.tstest/providers/openai/realtime.test.tssrc/app/src/components/traces/TraceTimeline.tsxsrc/onboarding.tstest/util/generation.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tssrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/tracing/traceContext.tssrc/providers/openai/realtime.tssrc/util/modelAuditCliParser.tssrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/eval/components/store.test.tstest/redteam/providers/crescendo/index.test.tssrc/logger.tssrc/app/src/pages/eval/components/CustomMetricsDialog.tsxsrc/commands/mcp/lib/performance.tssrc/validators/redteam.tstest/envars.test.tssrc/server/server.tssrc/testCase/synthesis.tstest/redteam/providers/custom/index.test.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.tssrc/commands/eval.tssrc/providers/google/util.tssrc/providers/xai/voice.tstest/util/functions/loadFunction.test.tssrc/commands/list.tssite/src/components/Store/useFourthwall.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.test.{ts,tsx}: Use Vitest for all tests (bothtest/andsrc/app/)
Test both success and error cases for all functionality
Files:
test/providers/bedrock/converse.test.tstest/providers/openai/realtime.test.tstest/util/generation.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tssrc/app/src/pages/eval/components/store.test.tstest/redteam/providers/crescendo/index.test.tstest/envars.test.tstest/redteam/providers/custom/index.test.tstest/util/functions/loadFunction.test.ts
test/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Backend tests in
test/should use Vitest with globals enabled (describe,it,expectavailable without imports)
test/**/*.test.{ts,tsx}: NEVER use.only()or.skip()in committed code
ALWAYS clean up mocks inafterEachusingvi.resetAllMocks()to prevent test pollution
All tests require explicit imports from vitest:import { afterEach, describe, expect, it, vi } from 'vitest'
Use Vitest's mocking utilities (vi.mock,vi.fn,vi.spyOn). Prefer shallow mocking over deep mocking. Mock external dependencies but not the code being tested.
Forvi.hoisted()mocks or mocks withmockReturnValue(), callmockReset()inbeforeEachto ensure test isolation when tests run in random order
Usevi.clearAllMocks()to clear call history, but usemockReset()for full isolation including mock implementations
For database tests, use in-memory instances or proper test fixtures
Files:
test/providers/bedrock/converse.test.tstest/providers/openai/realtime.test.tstest/util/generation.test.tstest/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tstest/redteam/providers/crescendo/index.test.tstest/envars.test.tstest/redteam/providers/custom/index.test.tstest/util/functions/loadFunction.test.ts
test/providers/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (test/AGENTS.md)
Every provider test must cover: (1) success case, (2) error cases (4xx, 5xx, rate limits), (3) configuration validation, (4) token usage tracking
Files:
test/providers/bedrock/converse.test.tstest/providers/openai/realtime.test.ts
src/app/**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (src/app/AGENTS.md)
NEVER use
fetch()directly. ALWAYS usecallApi()from '@app/utils/api' for all API calls to handle API base URL differences between dev and production
Files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/app/src/components/traces/TraceTimeline.tsxsrc/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/eval/components/store.test.tssrc/app/src/pages/eval/components/CustomMetricsDialog.tsxsrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.ts
src/app/**/*.{tsx,jsx}
📄 CodeRabbit inference engine (src/app/AGENTS.md)
src/app/**/*.{tsx,jsx}: Use React 19use()hook for reading promises and context
Use React 19 Actions withuseActionStateanduseFormStatusfor form handling
Use React 19useOptimistichook for optimistic UI updates
Use refs as props instead offorwardRefin React 19 components
Use React 19useTransitionfor async transitions to enable non-blocking updates
Avoid legacy React patterns: do not use class components, legacy context, or string refs
Import UI components from '@app/components/ui/' (Promptfoo's Design system based on Radix UI primitives) instead of creating custom components
Use Lucide React for all icon imports
Use Tailwind CSS for styling and apply thecn()utility from '@app/lib/utils' for conditional class composition
Use route constants from '@app/constants/routes' (EVAL_ROUTES, REDTEAM_ROUTES, ROUTES) for navigation instead of hardcoded strings
Use React Router v7 for routing
Files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/app/src/components/traces/TraceTimeline.tsxsrc/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/eval/components/CustomMetricsDialog.tsx
src/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (src/app/AGENTS.md)
Use TypeScript for all frontend code with strict typing
Files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/app/src/components/traces/TraceTimeline.tsxsrc/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxsrc/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsxsrc/app/src/pages/eval/components/store.test.tssrc/app/src/pages/eval/components/CustomMetricsDialog.tsxsrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.ts
src/app/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Frontend tests in
src/app/should use Vitest with explicit imports
Files:
src/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/eval/components/store.test.ts
src/app/**/*.test.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (src/app/AGENTS.md)
Use Vitest for testing with
vi.fn()for mocks andvi.mock()for module mocking
Files:
src/app/src/pages/eval/components/ResultsTable.test.tsxsrc/app/src/pages/eval/components/store.test.ts
src/providers/**/*.{ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Document provider configurations following examples in existing code
Files:
src/providers/openai/realtime.tssrc/providers/google/util.tssrc/providers/xai/voice.ts
src/providers/**/*.ts
📄 CodeRabbit inference engine (src/providers/AGENTS.md)
src/providers/**/*.ts: Each provider must implement theApiProviderinterface fromsrc/types/providers.ts
Provider must transform prompts into provider-specific API format and return normalizedProviderResponse
If provider allocates resources (Python workers, connections, child processes), implement acleanup()method and register withproviderRegistry
ALWAYS set thecached: trueflag when returning a cached response in provider implementations
Use logger with object context for logging in providers (auto-sanitized)
Files:
src/providers/openai/realtime.tssrc/providers/google/util.tssrc/providers/xai/voice.ts
src/commands/**/*.ts
📄 CodeRabbit inference engine (src/commands/AGENTS.md)
src/commands/**/*.ts: Register CLI commands with Commander insrc/main.ts
UsesetupEnv()for environment setup in CLI commands
Track command usage withtelemetry.record()
Useloggerfor all output messages, never useconsolefor logging in CLI commands
Setprocess.exitCode = 1on errors instead of usingprocess.exit()
Validate command inputs and arguments with Zod
Files:
src/commands/mcp/lib/performance.tssrc/commands/eval.tssrc/commands/list.ts
src/server/**/*.{ts,tsx}
📄 CodeRabbit inference engine (src/server/AGENTS.md)
src/server/**/*.{ts,tsx}: Validate requests with Zod schemas
Use proper HTTP status codes in Express responses
Wrap responses in { success, data/error } format
Handle errors with try-catch blocks
Import database access through db from '../database' and use Drizzle ORM query builders
Use logger with object context (auto-sanitized) as documented in docs/logging.md
Files:
src/server/server.ts
site/**/*.{md,ts,tsx,js,json}
📄 CodeRabbit inference engine (site/AGENTS.md)
site/**/*.{md,ts,tsx,js,json}: Use 'eval' not 'evaluation' in commands and documentation
Use 'Promptfoo' when referring to the company or product, 'promptfoo' when referring to the CLI command or in code
Files:
site/src/components/Store/useFourthwall.ts
🧠 Learnings (42)
📓 Common learnings
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/gh-cli-workflow.mdc:0-0
Timestamp: 2025-07-18T17:25:57.700Z
Learning: Avoid large, unrelated changes in a single PR
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : ALWAYS clean up mocks in `afterEach` using `vi.resetAllMocks()` to prevent test pollution
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/envars.test.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : For `vi.hoisted()` mocks or mocks with `mockReturnValue()`, call `mockReset()` in `beforeEach` to ensure test isolation when tests run in random order
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/envars.test.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use `vi.clearAllMocks()` to clear call history, but use `mockReset()` for full isolation including mock implementations
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/envars.test.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/smoke/**/*.test.{ts,tsx} : Smoke test temp output directories must be cleaned up in `afterAll()`
Applied to files:
test/providers/bedrock/converse.test.tstest/envars.test.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : All tests require explicit imports from vitest: `import { afterEach, describe, expect, it, vi } from 'vitest'`
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/envars.test.tssrc/testCase/synthesis.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : NEVER use `.only()` or `.skip()` in committed code
Applied to files:
test/providers/bedrock/converse.test.tstest/util/generation.test.tstest/redteam/constants/metadata.test.tstest/envars.test.tssrc/testCase/synthesis.tstest/redteam/providers/custom/index.test.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/test/providers/**/*.test.{ts,tsx} : Tests must NEVER make real API calls - mock all HTTP requests using `vi.mock`
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : Use Vitest's mocking utilities (`vi.mock`, `vi.fn`, `vi.spyOn`). Prefer shallow mocking over deep mocking. Mock external dependencies but not the code being tested.
Applied to files:
test/providers/bedrock/converse.test.tssrc/app/src/pages/eval/components/ResultsTable.test.tsxtest/util/functions/loadFunction.test.ts
📚 Learning: 2025-10-06T03:43:01.653Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-10-06T03:43:01.653Z
Learning: Applies to test/**/*.{ts,tsx,js,jsx} : Follow Jest best practices using describe and it blocks in tests
Applied to files:
test/providers/bedrock/converse.test.tstest/util/generation.test.tstest/envars.test.tssrc/testCase/synthesis.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/*.test.{ts,tsx} : For database tests, use in-memory instances or proper test fixtures
Applied to files:
test/providers/bedrock/converse.test.tstest/envars.test.tssrc/testCase/synthesis.ts
📚 Learning: 2025-10-06T03:43:01.653Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-10-06T03:43:01.653Z
Learning: Applies to {src,site}/**/*.{ts,tsx,js,jsx} : Use consistent curly braces for all control statements
Applied to files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/util/modelAuditCliParser.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to **/*.{ts,tsx,js,jsx} : Use consistent curly braces for all control statements
Applied to files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/util/modelAuditCliParser.ts
📚 Learning: 2025-11-29T00:24:17.021Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/CLAUDE.md:0-0
Timestamp: 2025-11-29T00:24:17.021Z
Learning: Applies to src/redteam/**/*agent*.{ts,tsx,js,jsx} : Maintain clear agent interface definitions and usage patterns
Applied to files:
src/app/src/pages/redteam/report/components/FrameworkCompliance.tsxsrc/app/src/pages/redteam/setup/components/strategies/StrategySection.tsxtest/redteam/providers/crescendo/index.test.tssrc/validators/redteam.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.ts
📚 Learning: 2025-10-06T03:43:01.653Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-10-06T03:43:01.653Z
Learning: Applies to {src,site}/**/*.{ts,tsx,js,jsx} : Prefer const over let and avoid var
Applied to files:
src/util/templates.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to **/*.{ts,tsx,js,jsx} : Prefer `const` over `let`; avoid `var`
Applied to files:
src/util/templates.ts
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/**/*.ts : Provider must transform prompts into provider-specific API format and return normalized `ProviderResponse`
Applied to files:
src/onboarding.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to src/providers/**/*.{ts,tsx} : Document provider configurations following examples in existing code
Applied to files:
src/onboarding.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to **/*.test.{ts,tsx} : Test both success and error cases for all functionality
Applied to files:
test/util/generation.test.tssrc/testCase/synthesis.tstest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.{tsx,jsx} : Use React Router v7 for routing
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsx
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.{tsx,jsx} : Use route constants from 'app/constants/routes' (EVAL_ROUTES, REDTEAM_ROUTES, ROUTES) for navigation instead of hardcoded strings
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsx
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.{tsx,jsx} : Use React 19 `useTransition` for async transitions to enable non-blocking updates
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsx
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.{tsx,jsx} : Use React 19 `useOptimistic` hook for optimistic UI updates
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsx
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.test.{ts,tsx,js,jsx} : Use Vitest for testing with `vi.fn()` for mocks and `vi.mock()` for module mocking
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsxtest/util/functions/loadFunction.test.ts
📚 Learning: 2026-01-06T19:12:06.738Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/app/AGENTS.md:0-0
Timestamp: 2026-01-06T19:12:06.738Z
Learning: Applies to src/app/**/*.{tsx,jsx} : Use React 19 `use()` hook for reading promises and context
Applied to files:
src/app/src/pages/eval/components/ResultsTable.test.tsx
📚 Learning: 2025-12-20T02:05:23.936Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:23.936Z
Learning: Applies to src/redteam/test/redteam/**/*.{ts,js} : Add tests for new plugins in the test/redteam/ directory following the reference pattern in src/redteam/plugins/pii.ts
Applied to files:
test/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tstest/redteam/providers/crescendo/index.test.tssrc/validators/redteam.tstest/redteam/providers/custom/index.test.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.ts
📚 Learning: 2025-12-20T02:05:23.936Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:23.936Z
Learning: Applies to src/redteam/plugins/**/*.ts : Implement `RedteamPluginObject` interface for new plugins and generate targeted test cases for specific vulnerabilities
Applied to files:
test/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tssrc/validators/redteam.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.ts
📚 Learning: 2025-12-20T02:05:23.936Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:23.936Z
Learning: Applies to src/redteam/plugins/**/*.ts : Include assertions defining failure conditions in plugin implementations
Applied to files:
test/redteam/constants/metadata.test.tstest/redteam/plugins/pluginDocumentation.test.tssrc/validators/redteam.tssrc/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/**/vitest*.config.ts : Test vitest configuration in `vitest.config.ts`, integration tests in `vitest.integration.config.ts`, and smoke tests in `vitest.smoke.config.ts`
Applied to files:
test/redteam/constants/metadata.test.tstest/envars.test.tssrc/testCase/synthesis.ts
📚 Learning: 2026-01-07T20:29:06.643Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/providers/AGENTS.md:0-0
Timestamp: 2026-01-07T20:29:06.643Z
Learning: Applies to src/providers/test/providers/**/*.test.{ts,tsx} : Provider tests must cover success cases, error cases, rate limits, timeouts, and invalid configs
Applied to files:
test/redteam/providers/crescendo/index.test.ts
📚 Learning: 2026-01-07T18:51:47.248Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: test/AGENTS.md:0-0
Timestamp: 2026-01-07T18:51:47.248Z
Learning: Applies to test/smoke/**/*.test.{ts,tsx} : Smoke tests verify the built CLI package (`dist/src/main.js`) works correctly end-to-end using `spawnSync`
Applied to files:
test/redteam/providers/crescendo/index.test.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to **/*.{ts,tsx,js,jsx} : Use the logger with object context (auto-sanitized)
Applied to files:
src/logger.ts
📚 Learning: 2025-12-09T06:08:55.096Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/commands/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:55.096Z
Learning: Applies to src/commands/**/*.ts : Validate command inputs and arguments with Zod
Applied to files:
src/validators/redteam.ts
📚 Learning: 2025-12-20T02:05:23.936Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:23.936Z
Learning: One plugin can be tested with multiple strategies for comprehensive coverage
Applied to files:
src/validators/redteam.tssrc/redteam/index.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to test/**/*.test.{ts,tsx} : Backend tests in `test/` should use Vitest with globals enabled (`describe`, `it`, `expect` available without imports)
Applied to files:
test/envars.test.tssrc/testCase/synthesis.ts
📚 Learning: 2025-12-09T06:08:36.114Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: site/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:36.114Z
Learning: Applies to site/**/*.{md,ts,tsx,js,json} : Use 'eval' not 'evaluation' in commands and documentation
Applied to files:
src/server/server.tssrc/commands/eval.tssrc/commands/list.ts
📚 Learning: 2025-07-18T17:25:57.700Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: .cursor/rules/gh-cli-workflow.mdc:0-0
Timestamp: 2025-07-18T17:25:57.700Z
Learning: Applies to **/*.{test,spec}.{js,ts,jsx,tsx} : Avoid disabling or skipping tests unless absolutely necessary and documented
Applied to files:
src/testCase/synthesis.ts
📚 Learning: 2025-12-20T02:05:28.355Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/plugins/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:28.355Z
Learning: Applies to src/redteam/plugins/**/redteam/**/graders.{ts,tsx} : Reference `harmful/graders.ts` as a model for implementing grader plugins
Applied to files:
src/app/src/pages/redteam/report/components/FrameworkComplianceUtils.tssrc/redteam/index.ts
📚 Learning: 2025-12-20T02:05:28.355Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/redteam/plugins/AGENTS.md:0-0
Timestamp: 2025-12-20T02:05:28.355Z
Learning: Applies to src/redteam/plugins/**/redteam/**/*.{ts,tsx} : Extend base classes RedteamPluginBase and RedteamGraderBase from `base.ts` when implementing redteam plugins and graders
Applied to files:
src/redteam/index.ts
📚 Learning: 2025-12-30T02:42:00.890Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-30T02:42:00.890Z
Learning: Applies to **/*.{ts,tsx,js,jsx} : Use consistent error handling with proper type checks
Applied to files:
biome.json
📚 Learning: 2025-12-09T06:08:55.096Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/commands/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:55.096Z
Learning: Applies to src/commands/**/*.ts : Use `logger` for all output messages, never use `console` for logging in CLI commands
Applied to files:
src/commands/eval.tssrc/commands/list.ts
📚 Learning: 2025-12-09T06:08:55.096Z
Learnt from: CR
Repo: promptfoo/promptfoo PR: 0
File: src/commands/AGENTS.md:0-0
Timestamp: 2025-12-09T06:08:55.096Z
Learning: Follow the standard CLI command structure demonstrated in `src/commands/eval.ts`: register with Commander, setup environment, track telemetry, use logger for output, and handle errors properly
Applied to files:
src/commands/eval.tssrc/commands/list.ts
🧬 Code graph analysis (8)
test/redteam/constants/metadata.test.ts (1)
src/redteam/constants/plugins.ts (6)
BASE_PLUGINS(279-285)ADDITIONAL_PLUGINS(288-355)BIAS_PLUGINS(229-229)PII_PLUGINS(227-227)MEDICAL_PLUGINS(231-238)FINANCIAL_PLUGINS(240-251)
src/util/modelAuditCliParser.ts (1)
src/providers/litellm.ts (1)
config(41-43)
src/validators/redteam.ts (3)
site/docs/_shared/data/strategies.ts (2)
strategies(14-370)Strategy(1-12)src/redteam/constants/strategies.ts (1)
Strategy(121-121)src/redteam/strategies/index.ts (1)
Strategy(39-39)
src/server/server.ts (2)
src/database/signal.ts (2)
setupSignalWatcher(70-89)readSignalEvalId(28-52)src/models/eval.ts (1)
Eval(235-1327)
test/redteam/providers/custom/index.test.ts (1)
examples/redteam-medical-agent/src/llm.js (1)
messages(81-85)
src/redteam/index.ts (1)
src/redteam/strategies/index.ts (1)
validateStrategies(339-378)
test/util/functions/loadFunction.test.ts (1)
src/util/functions/loadFunction.ts (1)
functionCache(9-9)
src/commands/list.ts (1)
src/__mocks__/database.ts (2)
evals(30-32)datasets(26-28)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)
- GitHub Check: Test on Node 24.x and windows-latest (shard 1/3)
- GitHub Check: Test on Node 24.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 20.x and windows-latest (shard 1/3)
- GitHub Check: Test on Node 22.x and macOS-latest
- GitHub Check: Test on Node 20.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 24.x and windows-latest (shard 3/3)
- GitHub Check: Test on Node 20.x and macOS-latest
- GitHub Check: Test on Node 22.x and windows-latest (shard 2/3)
- GitHub Check: Test on Node 22.x and ubuntu-latest
- GitHub Check: Test on Node 20.x and ubuntu-latest
- GitHub Check: Test on Node 22.x and windows-latest (shard 3/3)
- GitHub Check: Test on Node 24.x and ubuntu-latest
- GitHub Check: Test on Node 20.x and windows-latest (shard 3/3)
- GitHub Check: Redteam (Staging API)
- GitHub Check: Test on Node 22.x and windows-latest (shard 1/3)
- GitHub Check: webui tests
- GitHub Check: Share Test
- GitHub Check: Run Smoke Tests
- GitHub Check: Build Docs
- GitHub Check: Redteam (Production API)
- GitHub Check: Build on Node 20.x
- GitHub Check: Build on Node 24.x
- GitHub Check: Build on Node 22.x
- GitHub Check: security-scan
- GitHub Check: claude-review
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (40)
src/providers/xai/voice.ts (1)
343-343: LGTM! Correctly removes async from Promise executor.Removing
asyncfrom the Promise executor properly addresses thenoAsyncPromiseExecutorlint rule. The executor itself only performs synchronous setup (creating the WebSocket, setting up event handlers), while all async operations occur inside the event handler callbacks whereasyncis appropriate.site/src/components/Store/useFourthwall.ts (1)
369-371: LGTM! Lint-driven refactoring is correct.The conversion from an expression arrow function to a block body in this
forEachcallback aligns with the BiomeuseIterableCallbackReturnrule. The change is functionally identical and makes it explicit that the callback doesn't return a value.src/commands/mcp/lib/performance.ts (1)
110-110: LGTM! Correct use of fire-and-forget pattern.The addition of
voidprefixes on lines 110 and 145 correctly marks theseprocessQueue()calls as intentional fire-and-forget operations. This is appropriate because:
- Line 110: The promise returned by
add()resolves via the queued callback, not by awaitingprocessQueue(). The fire-and-forget trigger is intentional.- Line 145: The recursive continuation after batch processing is safe due to the
this.processingguard preventing concurrent execution.The forEach callback formatting changes (lines 132-138) align with the PR's standardization effort.
Also applies to: 132-138, 145-145
src/validators/redteam.ts (1)
441-443: Formatting change approved for Biome compliance.Both forEach callbacks are correctly converted from single-line arrow functions to explicit multi-line blocks, satisfying the
useIterableCallbackReturnrule requirement. The logic is preserved—strategySet.add()calls remain unchanged.Also applies to: 470-472
test/envars.test.ts (1)
309-314: Test formatting change approved.The beforeEach hook's forEach callback is correctly converted to block-body format while preserving test isolation. The multi-line format improves readability for this loop that clears CI environment variables before each test.
src/app/src/pages/redteam/report/components/FrameworkCsvExporter.tsx (1)
129-131: Formatting change approved.The forEach callback is correctly converted to block-body format while preserving the logic that adds each expanded plugin to the framework plugins set.
test/util/functions/loadFunction.test.ts (1)
55-61: MissingmockReset()calls for hoisted mocks inbeforeEach.While the formatting change (multi-line block body) is correct, the test setup needs improvement. The hoisted mocks (
mockResolveand the path mock at lines 19-46) should be explicitly reset inbeforeEachfor proper test isolation when tests run in random order. Currently, onlyvi.clearAllMocks()is called, which clears call history but doesn't reset mock implementations.🔧 Suggested fix to add mock resets
beforeEach(() => { vi.clearAllMocks(); + mockResolve.mockReset(); // Clear the function cache Object.keys(functionCache).forEach((key) => { delete functionCache[key]; }); });Based on learnings, for
vi.hoisted()mocks or mocks withmockReturnValue(), callmockReset()inbeforeEachto ensure test isolation when tests run in random order.test/redteam/providers/crescendo/index.test.ts (2)
106-108: Formatting refactor foruseIterableCallbackReturnrule.The forEach callback conversion to block body follows the Biome rule and maintains identical behavior. ✓
124-189: Test isolation: Reset hoisted mocks in allbeforeEachblocks.The main
beforeEachblock (lines 124-189) correctly resets hoisted mocks (mockGetGraderByIdat line 167,tryUnblockingat line 179,renderPromptat line 183-188). However, nestedbeforeEachblocks in otherdescribesuites (lines 1022, 1359, 1395, 1786, 2052, 2395) only callvi.clearAllMocks()without resetting the hoisted mocks. This can cause test pollution when tests run in random order.🔧 Example fix for nested beforeEach blocks
describe('continueAfterSuccess functionality', () => { beforeEach(() => { // Clear all mocks before each test vi.clearAllMocks(); + mockGetGraderById.mockReset(); + mockApplyRuntimeTransforms.mockReset(); + vi.mocked(tryUnblocking).mockReset(); + vi.mocked(evaluatorHelpers.renderPrompt).mockReset(); });Apply the same pattern to all other nested
beforeEachblocks.Based on learnings, for
vi.hoisted()mocks, callmockReset()inbeforeEachto ensure test isolation when tests run in random order.Also applies to: 1022-1025, 1359-1362, 1786-1822, 2052-2095
src/util/templates.ts (1)
95-104: Formatting refactor foruseIterableCallbackReturnrule.The forEach callback conversion to block body aligns with the Biome linting rules and preserves identical behavior. No functional changes. ✓
src/commands/eval.ts (1)
770-774: LGTM - Formatting change for Biome linting compliance.This change converts the forEach callback from a single-line expression to a multi-line block body. This is part of the broader PR effort to fix
useIterableCallbackReturnviolations across 30+ files by ensuring forEach callbacks use explicit block bodies rather than expression bodies.No behavioral change—the logger output remains identical.
src/providers/openai/realtime.ts (1)
1735-1737: LGTM - Formatting change for Biome linting compliance.Converts forEach callback to block body format, consistent with the Biome
useIterableCallbackReturnrule enforcement across the codebase. The timeout clearing logic remains unchanged.test/util/generation.test.ts (1)
80-82: LGTM - Formatting change for Biome linting compliance.Test assertion logic remains identical; only the callback syntax has changed from expression to block body format.
test/providers/bedrock/converse.test.ts (1)
1090-1092: LGTM - Formatting change for Biome linting compliance.Test cleanup logic remains unchanged; the mock restoration still executes correctly for each spy in the array.
src/onboarding.ts (1)
524-526: LGTM - Formatting change for Biome linting compliance.Warning messages are still logged correctly; only the callback syntax has been updated to use block body format per the Biome linting rules.
scripts/validate-changelog.cjs (1)
283-285: LGTM!Block body formatting for the
forEachcallback satisfies theuseIterableCallbackReturnlint rule. No functional change.src/app/src/pages/eval/components/store.test.ts (1)
45-47: LGTM!Block body formatting for the inner
forEachcallback aligns with theuseIterableCallbackReturnlint rule. No functional change to the test helper.test/redteam/constants/metadata.test.ts (1)
84-101: LGTM!Block body formatting for all six
forEachcallbacks satisfies theuseIterableCallbackReturnlint rule. The test logic adding plugins to the Set remains unchanged.test/redteam/providers/custom/index.test.ts (1)
100-102: LGTM!Block body formatting for the
forEachcallback aligns with theuseIterableCallbackReturnlint rule. Test setup logic remains unchanged.src/app/src/pages/eval/components/CustomMetricsDialog.tsx (1)
98-100: LGTM!Block body formatting for the inner
forEachcallback satisfies theuseIterableCallbackReturnlint rule. The memoized metric extraction logic remains unchanged.src/commands/list.ts (3)
25-27: LGTM!Block body formatting for the
forEachcallback satisfies theuseIterableCallbackReturnlint rule. Correctly usesloggerfor CLI output per coding guidelines.
78-80: LGTM!Consistent block body formatting matching the evals pattern above.
122-124: LGTM!Consistent block body formatting matching the evals and prompts patterns above.
src/util/modelAuditCliParser.ts (1)
166-168: LGTM!Block body formatting for the
forEachcallback satisfies theuseIterableCallbackReturnlint rule. The array argument parsing logic remains unchanged.src/app/src/pages/redteam/setup/components/strategies/StrategySection.tsx (1)
43-45: Brace-wrapping forEach callback is a no-op here (LGTM).test/providers/openai/realtime.test.ts (4)
716-718: Callback brace-wrap is a no-op (LGTM).
735-737: Callback brace-wrap is a no-op (LGTM).
757-759: Callback brace-wrap is a no-op (LGTM).
778-780: Callback brace-wrap is a no-op (LGTM).src/redteam/index.ts (2)
674-675: Good fix:await validateStrategiesprevents unhandled rejections / future async hazards.
785-787: Brace-wrapping forEach callback is a no-op here (LGTM).test/redteam/plugins/pluginDocumentation.test.ts (1)
46-48: LGTM!The forEach callbacks are correctly converted to block bodies to satisfy the
useIterableCallbackReturnlint rule. These are purely formatting changes with no behavioral impact.Also applies to: 54-56, 123-125, 143-145
biome.json (2)
262-264: Appropriate overrides for different contexts.Disabling
noFloatingPromisesin test files, scripts, examples, and frontend code is sensible since these contexts often have legitimate fire-and-forget patterns or different async handling requirements.Also applies to: 279-281, 289-291, 299-301, 354-356
145-170: Well-structured lint rule additions for promise/async safety.The new rules effectively target common promise/async pitfalls:
noAsyncPromiseExecutorprevents broken error handling in Promise constructorsnoFloatingPromisesandnoMisusedPromisescatch unhandled async operationsuseIterableCallbackReturnenforces explicit returns in array callbacksThe nursery rules (
noFloatingPromises,noMisusedPromises) remain experimental and subject to change in future Biome versions, but appropriately catch async issues now and align with the project's async/await standards.src/app/src/pages/redteam/report/components/FrameworkCompliance.tsx (1)
59-61: LGTM!The forEach callbacks are correctly converted to block bodies for lint compliance. No behavioral changes.
Also applies to: 127-129
src/cacheMigration.ts (1)
729-731: LGTM!The forEach callback is correctly converted to block body for lint compliance. Error logging behavior is unchanged.
src/app/src/pages/redteam/report/components/FrameworkComplianceUtils.ts (1)
36-38: LGTM!The forEach callback is correctly converted to block body for lint compliance. Plugin expansion logic is unchanged.
src/tracing/traceContext.ts (1)
106-108: LGTM!The forEach callback is correctly converted to block body for lint compliance. Span tree depth computation is unchanged.
src/app/src/components/traces/TraceTimeline.tsx (1)
122-131: Brace-wrappedforEachcallbacks look good (no behavior change).
This matches the new Biome style and keeps the depth traversal identical.src/logger.ts (1)
82-110:new Error('stack trace capture')is a good lint-driven tweak—double-check stack parsing assumptions.
Since you indexstack[3], ensure this still points at the intended frame across Node versions/build outputs.
JustinBeckwith
left a comment
JustinBeckwith
left a comment
There was a problem hiding this comment.
Clamp it down! Love it - just some questions and nits
src/server/server.ts
Outdated
| const evalPromise = signalEvalId ? Eval.findById(signalEvalId) : Eval.latest(); | ||
|
|
||
| const results = await updatedEval?.getResultsCount(); | ||
| void evalPromise.then(async (updatedEval) => { |
There was a problem hiding this comment.
This is an uncomfortable mix of promises and async await syntax. Can we standardize on async/await unless we really need to do this?
scripts/validate-changelog.cjs
Outdated
| console.log(result.message); | ||
| if (result.entries) { | ||
| result.entries.forEach((entry) => console.log(` ${entry}`)); | ||
| result.entries.forEach((entry) => { |
There was a problem hiding this comment.
Having this formatting sugar change along with actual behavioral changes is making this PR very hard to review
There was a problem hiding this comment.
It's actually more than that. This change prevents returns. I'll revert this because it does not matter as much.
There was a problem hiding this comment.
Ahhhh, what you're saying actually makes sense though. You're totally right.
|
|
||
| if (level === 'debug') { | ||
| initializeSourceMapSupport(); | ||
| void initializeSourceMapSupport(); |
There was a problem hiding this comment.
Admitting I've never seen void used like this - @claude what does void in front of an async function call do
There was a problem hiding this comment.
There was a problem hiding this comment.
Void means: yes, I know this is async. No, I don't want to await it. This is deliberate, not a mistake.
I'll fix the claude code permissions issue.
- Disable noBarrelFile and noReExportAll rules (too noisy) - Add void to fire-and-forget async calls in feedback, main, providers - Await addPrompts in evaluator - Make eval PATCH route async and await updateResult - Add void to logRequestResponse calls in monkeyPatchFetch - Add error messages to Error constructors in logger tests Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Revert all useIterableCallbackReturn-related forEach changes - Disable useIterableCallbackReturn rule (too noisy for stylistic preference) - Keep promise-related fixes (noFloatingPromises, noMisusedPromises, etc.) Co-Authored-By: Claude Opus 4.5 <[email protected]>
…patterns Refactors the signal watcher callback to use a named async function instead of mixing .then() with async/await, improving code clarity. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Reverts the GenAI span tracing additions to codex-sdk.ts. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Reverts the forEach block syntax back to single-line arrow function. Keeps the await validateStrategies fix. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Reverts forEach block syntax back to single-line arrow functions in: - src/tracing/traceContext.ts - src/util/modelAuditCliParser.ts - src/util/templates.ts - src/validators/redteam.ts Co-Authored-By: Claude Opus 4.5 <[email protected]>
JustinBeckwith
left a comment
JustinBeckwith
left a comment
There was a problem hiding this comment.
💌 this PR is awesome
Reverts forEach block syntax back to single-line arrow functions in: - test/envars.test.ts - test/providers/bedrock/converse.test.ts - test/providers/openai/realtime.test.ts - test/redteam/constants/metadata.test.ts - test/redteam/plugins/pluginDocumentation.test.ts - test/redteam/providers/crescendo/index.test.ts - test/redteam/providers/custom/index.test.ts - test/util/functions/loadFunction.test.ts - test/util/generation.test.ts Co-Authored-By: Claude Opus 4.5 <[email protected]>
Summary
New Rules Enabled
noAsyncPromiseExecutornew Promise(async ...)which breaks error handlingnoThenPropertynoDebuggeruseErrorMessagethrow new Error("...")Violations Fixed
forEachcallbacksvoidfor intentional fire-and-forget async callsvi.importActualspreadingasyncfrom Promise executor inxai/voice.tsTest plan
npm run lpasses on changed filesnpm run fpasses on changed files🤖 Generated with Claude Code