Skip to content

improvement(workflow-execution): perf improvements to passing workflow state + decrypted env vars #2119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
icecrasher321 merged 4 commits into from
Nov 26, 2025
Merged

improvement(workflow-execution): perf improvements to passing workflow state + decrypted env vars #2119

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ef7acd0
improvement(execution): load workflow state once instead of 2-3 times
icecrasher321 Nov 26, 2025
8026b22
decrypt only in get helper
icecrasher321 Nov 26, 2025
de56cc5
remove comments
icecrasher321 Nov 26, 2025
5d53633
remove comments
icecrasher321 Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
improvement(execution): load workflow state once instead of 2-3 times
  • Loading branch information
@icecrasher321
icecrasher321 committed Nov 26, 2025
commit ef7acd0730e9617ceefbbb1fc7ff83e93d8ec904
32 changes: 28 additions & 4 deletions apps/sim/app/api/workflows/[id]/execute/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,13 +348,30 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
workspaceId: workflow.workspaceId,
})

// Load workflow data ONCE and cache it for all execution paths
// This avoids duplicate DB calls in executeWorkflowCore
let cachedWorkflowData: {
blocks: Record<string, any>
edges: any[]
loops: Record<string, any>
parallels: Record<string, any>
} | null = null

let processedInput = input
try {
const workflowData = shouldUseDraftState
? await loadWorkflowFromNormalizedTables(workflowId)
: await loadDeployedWorkflowState(workflowId)

if (workflowData) {
// Cache the loaded data to avoid reloading in executeWorkflowCore
cachedWorkflowData = {
blocks: workflowData.blocks,
edges: workflowData.edges,
loops: workflowData.loops || {},
parallels: workflowData.parallels || {},
}

const serializedWorkflow = new Serializer().serializeWorkflow(
workflowData.blocks,
workflowData.edges,
Expand Down Expand Up @@ -402,6 +419,10 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
)
}

// Use user-provided workflowStateOverride, or fall back to our cached data
// This prevents executeWorkflowCore from making redundant DB calls
const effectiveWorkflowStateOverride = workflowStateOverride || cachedWorkflowData || undefined

if (!enableSSE) {
logger.info(`[${requestId}] Using non-SSE execution (direct JSON response)`)
try {
Expand All @@ -414,7 +435,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
triggerType,
useDraftState: shouldUseDraftState,
startTime: new Date().toISOString(),
workflowStateOverride,
workflowStateOverride: effectiveWorkflowStateOverride,
}

const snapshot = new ExecutionSnapshot(
Expand Down Expand Up @@ -479,8 +500,11 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
logger.info(`[${requestId}] Using SSE console log streaming (manual execution)`)
} else {
logger.info(`[${requestId}] Using streaming API response`)
const deployedData = await loadDeployedWorkflowState(workflowId)
const resolvedSelectedOutputs = resolveOutputIds(selectedOutputs, deployedData?.blocks || {})
// Use cached workflow data instead of reloading from DB
const resolvedSelectedOutputs = resolveOutputIds(
selectedOutputs,
cachedWorkflowData?.blocks || {}
)
const stream = await createStreamingResponse({
requestId,
workflow: {
Expand Down Expand Up @@ -677,7 +701,7 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id:
triggerType,
useDraftState: shouldUseDraftState,
startTime: new Date().toISOString(),
workflowStateOverride,
workflowStateOverride: effectiveWorkflowStateOverride,
}

const snapshot = new ExecutionSnapshot(
Expand Down