Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/dynamic-start-validation-core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@workflow/core": patch
---

Dynamic `start()` validation: cap `exportName` at 64 characters, detect `"use step"` only as a real directive, and name non-async, generator, and duplicate workflow declarations.
5 changes: 5 additions & 0 deletions .changeset/dynamic-start-validation-world-vercel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@workflow/world-vercel": patch
---

The dynamic execution-context budget error now says that step IDs and `exportName` count, not only aliases.
2 changes: 1 addition & 1 deletion docs/content/docs/v5/advanced/dynamic-workflows.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Supply a unique, deterministic approval token from the caller. The workflow must

`start()` validates the source before it writes anything, so a definition that could never run fails at the call site rather than on a queue delivery:

- It must declare `async function workflow(...)`. Pass `experimental_dynamic.exportName` to use a different name; export names may contain letters, digits, and `_`, and cannot start with a digit.
- It must declare `async function workflow(...)`. Pass `experimental_dynamic.exportName` to use a different name; export names may contain letters, digits, and `_`, cannot start with a digit, and are at most 64 characters.
- The function's first statement must be the `"use workflow"` directive.
- No `import` or `export`. Reach steps through `steps`, not through modules.
- JavaScript only — no TypeScript syntax, no npm dependencies, no bundling.
Expand Down
107 changes: 107 additions & 0 deletions packages/core/src/runtime/dynamic-workflow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { WorkflowRuntimeError } from '@workflow/errors';
import { describe, expect, it } from 'vitest';
import {
compileDynamicWorkflow,
DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH,
DYNAMIC_WORKFLOW_SOURCE_MAX_BYTES,
readDynamicWorkflowMetadata,
} from './dynamic-workflow.js';
Expand Down Expand Up @@ -489,6 +490,112 @@ async function workflow() {
).rejects.toThrow(/"\$" cannot appear in workflow queue names/);
});

it('rejects an export name over the length limit', async () => {
const exportName = 'w'.repeat(
DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH + 1
);
await expect(
compileDynamicWorkflow(
`async function ${exportName}() { "use workflow"; return 1; }`,
{ steps: STEPS, exportName }
)
).rejects.toThrow(
`Dynamic workflow exportName is ${exportName.length} characters, over the ${DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH}-character limit.`
);
});

it('accepts an export name at the length limit', async () => {
const exportName = 'w'.repeat(DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH);
await expect(
compileDynamicWorkflow(
`async function ${exportName}() { "use workflow"; return 1; }`,
{ steps: STEPS, exportName }
)
).resolves.toMatchObject({ metadata: { exportName } });
});

it.each([
['a string argument', 'await steps.fetchUser("use step");'],
['a single-quoted string', "const label = 'use step';"],
['a template literal', 'const label = `use step`;'],
['a string after the prologue', 'const x = 1;\n "use step";'],
])('accepts "use step" in %s', async (_label, statement) => {
const source = `async function workflow() {
"use workflow";
${statement}
return 1;
}`;
await expect(
compileDynamicWorkflow(source, { steps: STEPS })
).resolves.toMatchObject({ metadata: { version: 1 } });
});

it.each([
['an arrow function', 'const f = async () => { "use step"; return 1; };'],
['a class method', 'class C { async run() { "use step"; return 1; } }'],
[
'an object method',
"const o = { async run() { 'use step'; return 1; } };",
],
])('rejects a "use step" directive in %s', async (_label, statement) => {
const source = `async function workflow() {
"use workflow";
${statement}
return 1;
}`;
await expect(
compileDynamicWorkflow(source, { steps: STEPS })
).rejects.toThrow(/cannot declare "use step"/);
});

it('rejects a "use step" directive in the script prologue', async () => {
await expect(
compileDynamicWorkflow(`"use step";\n${SOURCE}`, { steps: STEPS })
).rejects.toThrow(/cannot declare "use step"/);
});

it('hints at async when a parse failure may be an await', async () => {
const source = `function workflow(input) {
"use workflow";
return await steps.fetchUser(input.id);
}`;
await expect(
compileDynamicWorkflow(source, { steps: STEPS })
).rejects.toThrow(
/not valid JavaScript: Unexpected token.*If the error is at an `await`, declare the function as `async function workflow\(\.\.\.\)`\./
);
});

it('validates a deeply nested expression without overflowing the stack', async () => {
const source = `function helper(x) { return x${'.a'.repeat(30_000)}; }
${SOURCE}`;
await expect(
compileDynamicWorkflow(source, { steps: STEPS })
).resolves.toMatchObject({ metadata: { version: 1 } });
});

it.each([
[
'declared twice',
`${SOURCE}\n${SOURCE}`,
/exactly once, but declares `function workflow` 2 times/,
],
[
'not async',
'function workflow() { "use workflow"; return 1; }',
/at top level, but `workflow` is not async/,
],
[
'a generator',
'async function* workflow() { "use workflow"; }',
/at top level, but `workflow` is a generator function/,
],
])('says what is wrong when the workflow function is %s', async (_label, source, message) => {
await expect(
compileDynamicWorkflow(source, { steps: STEPS })
).rejects.toThrow(message);
});

it('still accepts "$" in step aliases', async () => {
await expect(
compileDynamicWorkflow(
Expand Down
99 changes: 85 additions & 14 deletions packages/core/src/runtime/dynamic-workflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,15 @@ export const DYNAMIC_WORKFLOW_CODE_INLINE_MAX_BYTES = 24 * 1024;

const SAFE_DYNAMIC_IDENTIFIER = /^[a-zA-Z_$][a-zA-Z0-9_$]*$/;

/**
* The SDK's limit on `exportName`. The name becomes the last segment of the
* generated workflow id, which is also the queue topic name, and it counts
* against the run's execution-context budget. Queues cap topic names too
* (Vercel's at 256 characters, prefix included), so a long name buys nothing
* and fails late.
*/
export const DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH = 64;

/**
* The export name becomes the final segment of the generated workflow id,
* which is also the queue topic name. Queue names do not accept `$`, so this
Expand Down Expand Up @@ -235,16 +244,21 @@ async function sha256Hex(input: string): Promise<string> {
// Raise only after every supported Node and QuickJS runtime accepts the grammar.
const DYNAMIC_WORKFLOW_ECMA_VERSION = 2024;

function parseDynamicWorkflowSource(source: string): Program {
function parseDynamicWorkflowSource(
source: string,
exportName: string
): Program {
try {
return parse(source, {
ecmaVersion: DYNAMIC_WORKFLOW_ECMA_VERSION,
sourceType: 'script',
});
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
// A non-async workflow using `await` fails to parse with a bare
// "Unexpected token", which reads like one of the other two causes.
throw dynamicStartRefusal(
`Dynamic workflow source is not valid JavaScript: ${message}. The source is evaluated as-is, so it cannot contain TypeScript syntax or module declarations.`
`Dynamic workflow source is not valid JavaScript: ${message}. The source is evaluated as-is, so it cannot contain TypeScript syntax or module declarations. If the error is at an \`await\`, declare the function as \`async function ${exportName}(...)\`.`
);
}
}
Expand Down Expand Up @@ -280,17 +294,8 @@ function validateDynamicWorkflowSource(
);
}

const program = parseDynamicWorkflowSource(source);
const declarations = program.body.filter(
(node): node is FunctionDeclaration =>
node.type === 'FunctionDeclaration' && node.id.name === exportName
);
const declaration = declarations.length === 1 ? declarations[0] : undefined;
if (!declaration || !declaration.async || declaration.generator) {
throw dynamicStartRefusal(
`Dynamic workflow source must declare \`async function ${exportName}(...)\` at top level.`
);
}
const program = parseDynamicWorkflowSource(source, exportName);
const declaration = findWorkflowDeclaration(program, exportName);

const firstStatement = declaration.body.body[0] as
| ExpressionStatement
Expand All @@ -308,13 +313,74 @@ function validateDynamicWorkflowSource(
// would not split a step out: the function would simply run inside the
// workflow VM, as ordinary (and non-deterministic) workflow code. Steps
// come from `experimental_dynamic.steps` only.
if (/(?:"use step"|'use step')/.test(source)) {
if (hasDirective(program, 'use step')) {
throw dynamicStartRefusal(
'Dynamic workflow source cannot declare "use step" functions. Register the step with the deployment and expose it through `experimental_dynamic.steps` instead.'
);
}
}

/**
* The single top-level `async function <exportName>` declaration, or a
* refusal that says what is wrong with the one the source declares.
*/
function findWorkflowDeclaration(
program: Program,
exportName: string
): FunctionDeclaration {
const expected = `Dynamic workflow source must declare \`async function ${exportName}(...)\` at top level`;
const declarations = program.body.filter(
(node): node is FunctionDeclaration =>
node.type === 'FunctionDeclaration' && node.id.name === exportName
);
const [declaration] = declarations;
let problem = '';
if (declarations.length > 1) {
problem = ` exactly once, but declares \`function ${exportName}\` ${declarations.length} times`;
} else if (declaration?.generator) {
problem = `, but \`${exportName}\` is a generator function`;
} else if (declaration && !declaration.async) {
problem = `, but \`${exportName}\` is not async`;
}
if (!declaration || problem) {
throw dynamicStartRefusal(`${expected}${problem}.`);
}
return declaration;
}

/**
* Whether any directive prologue in the program (the script's own, or any
* function's) holds `directive`.
*
* Acorn marks prologue statements with `directive`, so a string that merely
* reads `"use step"` elsewhere, such as an argument or a template literal, is
* not one.
*/
function hasDirective(program: Program, directive: string): boolean {
// Iterative: a long member chain nests the AST far deeper than the call
// stack allows, well inside the source size limit.
const pending: object[] = [program];
while (pending.length > 0) {
const node = pending.pop() as { directive?: unknown };
if (node.directive === directive) return true;
for (const child of childNodes(node)) pending.push(child);
}
return false;
}

/** Child nodes of an acorn node, in no particular order. */
function childNodes(node: object): object[] {
const children: object[] = [];
for (const value of Object.values(node)) {
for (const child of Array.isArray(value) ? value : [value]) {
if (child && typeof child === 'object' && 'type' in child) {
children.push(child);
}
}
}
return children;
}

function resolveStepId(alias: string, value: unknown): string {
const stepId =
(value && typeof value === 'object') || typeof value === 'function'
Expand Down Expand Up @@ -368,6 +434,11 @@ export async function compileDynamicWorkflow(
options: DynamicWorkflowOptions
): Promise<CompiledDynamicWorkflow> {
const exportName = options.exportName ?? 'workflow';
if (exportName.length > DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH) {
throw dynamicStartRefusal(
`Dynamic workflow exportName is ${exportName.length} characters, over the ${DYNAMIC_WORKFLOW_EXPORT_NAME_MAX_LENGTH}-character limit. It becomes part of the workflow id and queue name.`
);
}
assertDynamicWorkflowIdentifier('exportName', exportName);
if (!SAFE_DYNAMIC_EXPORT_NAME.test(exportName)) {
throw dynamicStartRefusal(
Expand Down
2 changes: 1 addition & 1 deletion packages/world-vercel/src/execution-context.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ describe('validateRunExecutionContext', () => {
}
expect(WorkflowRuntimeError.is(error)).toBe(true);
expect((error as Error).message).toMatch(
/Dynamic workflow execution context is 2049 bytes.*2048-byte limit.*experimental_dynamic\.steps/
/Dynamic workflow execution context is 2049 bytes.*2048-byte limit.*step ID, which comes from the step's file path and function name.*experimental_dynamic\.steps/
);
});

Expand Down
2 changes: 1 addition & 1 deletion packages/world-vercel/src/execution-context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export function validateRunExecutionContext(
const bytes = new TextEncoder().encode(json).byteLength;
if (bytes > MAX_EXECUTION_CONTEXT_BYTES) {
throw new WorkflowRuntimeError(
`Dynamic workflow execution context is ${bytes} bytes, exceeding the ${MAX_EXECUTION_CONTEXT_BYTES}-byte limit, so no run was created. Bind fewer steps through \`experimental_dynamic.steps\` or use shorter aliases.`
`Dynamic workflow execution context is ${bytes} bytes, exceeding the ${MAX_EXECUTION_CONTEXT_BYTES}-byte limit, so no run was created. Each step binding counts its alias and its step ID, which comes from the step's file path and function name and is usually the longer of the two; the \`exportName\` counts too. Bind fewer steps through \`experimental_dynamic.steps\`, or shorten the aliases, the step files' paths or names, or the \`exportName\`.`
);
}
}
Loading