From de533629e24ab991fbd27de99d4570c6d4373482 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 06:28:19 -0400 Subject: [PATCH 01/15] =?UTF-8?q?=E2=9C=A8=20Add=20xmd=20prompt,=20the=20a?= =?UTF-8?q?uthorship=20command=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `xmd prompt ""` asks the configured ACP agent for a complete executable Markdown root, validates it without executing any of it, repairs its definite defects by asking again, shows a person the exact bytes, and runs the approved ones through the ordinary supplied-source path under the `` identity. It adds authorship around one normal document; it adds no second execution model, props model or journal. Authorship sits outside durability. The catalog, the fresh generator session, every repair and the review create no journal and replay nothing, and the generator's scope closes before the optional exclusive `--save` and before execution. A defect the draft authored is repairable and earns one of three turns; a defect the command line authored terminates without spending a turn on something the agent cannot fix. `xmd run` and `xmd prompt` now share one execution field set, one resolved Agent configuration, and one props source resolution. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 1 + packages/cli/src/agent-stack.ts | 174 +++++ packages/cli/src/cli.ts | 346 +++++++--- packages/cli/src/prompt-args.ts | 320 +++++++++ packages/cli/src/prompt.ts | 652 +++++++++++++++++++ packages/cli/src/props.ts | 63 +- packages/cli/src/syntax.ts | 23 +- packages/cli/tests/prompt-args.test.ts | 234 +++++++ packages/cli/tests/prompt-cli.test.ts | 458 +++++++++++++ packages/cli/tests/prompt.test.ts | 486 ++++++++++++++ packages/cli/tests/support/fake-acp.ts | 39 +- packages/cli/tests/support/prompt-harness.ts | 160 +++++ packages/cli/tests/syntax-cli.test.ts | 7 +- specs/acp-client-spec.md | 41 +- specs/executable-mdx-spec.md | 32 +- specs/prompt-command-spec.md | 343 ++++++++++ specs/root-document-props-spec.md | 53 +- 17 files changed, 3308 insertions(+), 124 deletions(-) create mode 100644 packages/cli/src/agent-stack.ts create mode 100644 packages/cli/src/prompt-args.ts create mode 100644 packages/cli/src/prompt.ts create mode 100644 packages/cli/tests/prompt-args.test.ts create mode 100644 packages/cli/tests/prompt-cli.test.ts create mode 100644 packages/cli/tests/prompt.test.ts create mode 100644 packages/cli/tests/support/prompt-harness.ts create mode 100644 specs/prompt-command-spec.md diff --git a/architecture.md b/architecture.md index 3ee295f3b..1529c6895 100644 --- a/architecture.md +++ b/architecture.md @@ -3541,6 +3541,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | +| `xmd prompt` | asks the configured agent for a complete executable Markdown root, validates it without executing any of it, repairs its definite defects by asking again at most three times per draft, shows a person the exact bytes, and executes the approved ones through the ordinary supplied-source path under the `` identity. Authorship sits outside durability: the catalog, the fresh generator session, every repair and the review create no journal, retain no durable identity and replay nothing, and the generator's scope closes before the optional exclusive save and before execution. What runs is one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A defect the draft authored is repairable; one the command line authored terminates without spending a turn on it | built on the #656 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/packages/cli/src/agent-stack.ts b/packages/cli/src/agent-stack.ts new file mode 100644 index 000000000..df1d99c12 --- /dev/null +++ b/packages/cli/src/agent-stack.ts @@ -0,0 +1,174 @@ +/** + * The one Agent configuration an invocation resolves, and the two things a + * host does with it (specs/acp-client-spec.md §Command-line configuration). + * + * `xmd run` and `xmd prompt` take the same Agent, permission and provider + * options, so they resolve them once, here, rather than each reading the flags + * again. What they do with the result differs, and deliberately: a run installs + * the registered provider into the Agent Api so a document may reach it, while + * prompt generation holds a provider handle of its own and installs no document + * components, no root provider and no foreground launcher — it needs turns, not + * authority. + * + * `agent-config.ts` stays the pure flag-to-permission mapping. This module is + * where the environment, the provider registry and the host's own machine + * session assembly enter. + */ + +import { + Agent, + installAgentComponents, + installPermissionMode, + registerAgentProvider, +} from "@executablemd/core"; +import type { AgentProviderFactory, PermissionMode } from "@executablemd/core"; +import { installForegroundLauncher, env as readEnv } from "@executablemd/runtime"; +import { createAcpxProvider, DEFAULT_AGENT_NAME, useAcpxProvider } from "@executablemd/acp"; +import type { AcpxProvider, AcpxProviderDependencies } from "@executablemd/acp"; +import { Err, Ok } from "effection"; +import type { Operation, Result } from "effection"; + +import { resolveAgentConfig } from "./agent-config.ts"; +import type { AgentFlags } from "./agent-config.ts"; +import type { MachineSessionAssembly } from "./session-coordinator.ts"; + +/** Everything one invocation settled about agents, resolved exactly once. */ +export interface AgentStack { + /** The provider name the caller selected, already known to be registered. */ + provider: string; + /** The agent every consumer defaults to, environment fallback applied. */ + defaultAgent: string; + permissionMode: PermissionMode; + /** What this host states about machine-wide agent sessions, if anything. */ + sessions?: MachineSessionAssembly; +} + +/** + * Read the command line, the environment and the host's assembly into one + * configuration. + * + * A failure comes back as a `Result` rather than as a printed line and an exit, + * so the same resolution serves a command that runs a document and one that + * generates one first. + */ +export function* resolveAgentStack( + flags: AgentFlags, + sessions: MachineSessionAssembly | undefined, +): Operation> { + const config = resolveAgentConfig(flags); + if ("error" in config) { + return Err(new Error(config.error)); + } + if (flags.agentProvider !== "acpx") { + return Err(new Error(`Unknown agent provider "${flags.agentProvider}"`)); + } + const defaultAgent = + config.defaultAgent ?? (yield* readEnv("DEFAULT_AGENT_NAME")) ?? DEFAULT_AGENT_NAME; + return Ok({ + provider: flags.agentProvider, + defaultAgent, + permissionMode: config.permissionMode, + ...(sessions === undefined ? {} : { sessions }), + }); +} + +/** + * What this host built, if it built anything. + * + * Each piece reaches the provider directly rather than through a context: who + * owns a session and which build it belongs to are security decisions, and ones + * a document could replace are not ones. The two advertised sets are stated by + * the host, not inherited. + */ +export function hostAcpDependencies( + sessions: MachineSessionAssembly | undefined, +): AcpxProviderDependencies { + if (sessions === undefined) { + return {}; + } + return { + ...(sessions.coordinator ? { coordinator: sessions.coordinator } : {}), + ...(sessions.routeStore ? { routeStore: sessions.routeStore } : {}), + ...(sessions.executableObserver ? { executableObserver: sessions.executableObserver } : {}), + advertiseNativeLaunch: sessions.advertiseNativeLaunch, + advertiseClientNativeAttachment: sessions.advertiseClientNativeAttachment, + }; +} + +/** + * Install the agent stack a document runs under: the registration, the + * components with the resolved root provider, the permission mode, and the + * terminal this command has to give away. + * + * Nothing starts an agent — the provider validates availability on first use. + */ +export function* installRunAgentStack(stack: AgentStack): Operation { + const acpx = createAcpxProvider(hostAcpDependencies(stack.sessions)); + yield* registerAgentProvider("acpx", acpx); + + // The trusted host selects its own root provider by name. Document-level + // selection goes through the installation protocol; this is the host saying + // what it configured, which no document is composing around. + const providers: Record = { acpx }; + const factory = providers[stack.provider]; + const { defaultAgent, permissionMode } = stack; + yield* installAgentComponents({ + defaultAgent, + permissionMode, + rootProvider: { factory, options: { defaultAgent, permissionMode } }, + }); + yield* installPermissionMode(permissionMode); + // `xmd run` is the one command that has a terminal to give away. Help, + // document inspection and `xmd test` install no launcher, so a document that + // reaches under any of them refuses instead of spawning. + yield* installForegroundLauncher(); +} + +/** + * A provider that can run turns, and nothing else. + * + * `xmd prompt` writes a document before one exists, so there is nothing to + * install components into and no launch for anyone to authorize. It takes the + * provider's operations directly and answers only the one Agent operation the + * provider itself performs — resolving the agent a session and a turn belong + * to. Everything else on the Agent Api stays at its base behavior, so a launch + * or a document prompt reaching this scope is refused rather than served. + * + * The permission policy is installed first, so it encloses the provider state's + * own scope and answers the native permission requests generation raises with + * the same `--approve-all`, `--approve-reads` or `--deny-all` choice a run uses. + * + * `instructions` become the fresh session's system prompt. They reach ACPX + * through `newSessionOptions`, which it applies when it creates a session and + * ignores when it reuses a record. + */ +export function* useGeneratorAgent( + stack: AgentStack, + instructions: string, + dependencies?: AcpxProviderDependencies, +): Operation { + yield* installPermissionMode(stack.permissionMode); + + let provider: AcpxProvider | undefined; + yield* Agent.around( + { + *agent([name], _next) { + if (provider === undefined) { + throw new Error("the generator agent provider is not established yet"); + } + return yield* provider.agent(name); + }, + }, + { at: "min" }, + ); + + provider = yield* useAcpxProvider( + { defaultAgent: stack.defaultAgent, permissionMode: stack.permissionMode }, + { + ...hostAcpDependencies(stack.sessions), + ...dependencies, + newSessionOptions: { systemPrompt: instructions }, + }, + ); + return provider; +} diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index ca4c45fe5..4811716a8 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -4,6 +4,7 @@ * Usage: * xmd run [options] * xmd [options] (run is the default command) + * xmd prompt "" [options] * xmd workflow start [options] * xmd workflow resume * xmd workflow status|history [--json] @@ -16,6 +17,7 @@ * * Examples: * xmd run packages/core/examples/hello-world.md + * xmd prompt "ask me for my age and write the result to a file" * xmd packages/core/examples/hello-world.md --verbose * xmd run packages/core/examples/hello-world.md --journal events.jsonl * xmd run README.md#Release/Publish @@ -61,8 +63,6 @@ import { inspectDocument, agentIdentityComponents, installAgentComponents, - installPermissionMode, - registerAgentProvider, retainedSource, rootSourcePath, useNormalizedOutput, @@ -71,14 +71,12 @@ import { import { executeInstalled } from "@executablemd/core/host"; import type { ExecutionInstallation } from "@executablemd/core/host"; import type { - AgentProviderFactory, DocumentTargetInfo, FileRootDocument, PropsSchema, RootDocumentSource, } from "@executablemd/core"; -import { command as hostCommand, env as readEnv } from "@executablemd/runtime"; -import { createAcpxProvider, DEFAULT_AGENT_NAME } from "@executablemd/acp"; +import { command as hostCommand } from "@executablemd/runtime"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { installTestingComponents, @@ -94,8 +92,7 @@ import { import { installWebComponents, installWebElicitation } from "@executablemd/web"; import { timebox } from "@effectionx/timebox"; import { timeout as runTimeout } from "@executablemd/runtime"; -import { installForegroundLauncher } from "@executablemd/runtime"; -import { resolveAgentConfig } from "./agent-config.ts"; +import { installRunAgentStack, resolveAgentStack } from "./agent-stack.ts"; import { TIMEOUT_FLAGS, resolveRunTimeouts } from "./timeouts.ts"; import type { RunTimeouts } from "./timeouts.ts"; import type { AgentFlags } from "./agent-config.ts"; @@ -108,9 +105,13 @@ import { describeError, extractPropsArgs, formatProperties, - resolveProps, + resolvePropsFromSources, } from "./props.ts"; import type { Binding, Extraction } from "./props.ts"; +import { namesPrompt, scanPromptArgs, SAVE_OPTION } from "./prompt-args.ts"; +import type { PromptScan } from "./prompt-args.ts"; +import { runPrompt } from "./prompt.ts"; +import type { PromptExecution } from "./prompt.ts"; import { componentSearchPath, resolveTestTarget } from "./test-target.ts"; import { renderSyntaxJson, renderSyntaxMarkdown, syntaxCatalog } from "./syntax.ts"; import { testingExecutionHost } from "./testing-host.ts"; @@ -156,19 +157,17 @@ const SECRET_DETECTION_FIELD = { ...field(z.boolean(), field.default(true)), }; -const runConfig = object({ - path: { - description: "markdown document to execute, optionally `#` and one target selector", - ...field(z.string().optional(), cli.argument()), - }, - // Declared so `xmd run --help` lists it with every other option. The value is - // lifted out of argv by readEvalFlags before parsing — see eval-source.ts — - // so this field is never the source of the document. - eval: { - description: "inline markdown document to execute, in place of a path", - aliases: ["-e"], - ...field(z.string().optional()), - }, +/** + * Everything a command that ends in a document execution configures. + * + * Declared once because `xmd run` and `xmd prompt` configure the same + * execution: the includes it resolves components through, what it writes and + * where, the agent stack it runs under, and the three deadlines. `run` adds the + * document, `prompt` adds the request and where to keep the source; nothing + * else differs, and a second copy of this list would be the two commands + * drifting apart one option at a time. + */ +const executionFields = { include: { description: "component search directory", ...field(z.array(z.string()), field.default(["components", "."]), field.array()), @@ -220,6 +219,43 @@ const runConfig = object({ ...field(z.boolean(), field.default(false)), }, secretDetection: SECRET_DETECTION_FIELD, +}; + +const runConfig = object({ + path: { + description: "markdown document to execute, optionally `#` and one target selector", + ...field(z.string().optional(), cli.argument()), + }, + // Declared so `xmd run --help` lists it with every other option. The value is + // lifted out of argv by readEvalFlags before parsing — see eval-source.ts — + // so this field is never the source of the document. + eval: { + description: "inline markdown document to execute, in place of a path", + aliases: ["-e"], + ...field(z.string().optional()), + }, + ...executionFields, +}); + +/** + * `xmd prompt` — the request, where to keep the source, and the execution the + * approved document runs under. + * + * The individual `--props-*` options a candidate declares are deliberately + * absent: they exist only once a document does, and the props phase binds them + * per candidate. The aggregate `--props` is absent for the reason `run`'s is — + * the parser coerces a separated value before any schema could judge it. + */ +const promptConfig = object({ + request: { + description: "what the agent should write a document to do", + ...field(z.string().optional(), cli.argument()), + }, + save: { + description: "write the approved document here before running it (path must not exist)", + ...field(z.string().optional()), + }, + ...executionFields, }); const testConfig = object({ @@ -284,6 +320,7 @@ const xmd = program({ config: commands( { run: runConfig, + prompt: promptConfig, test: testConfig, syntax: syntaxConfig, "test-agent": testAgentConfig, @@ -489,64 +526,13 @@ function* installAgentStack( flags: AgentFlags, sessions: MachineSessionAssembly | undefined, ): Operation { - const config = resolveAgentConfig(flags); - if ("error" in config) { - console.error(config.error); - yield* exit(1); - return; - } - - // What this host built, if it built anything. Each piece reaches the provider - // directly rather than through a context: who owns a session and which build - // it belongs to are security decisions, and ones a document could replace are - // not ones. A host that answers who owns a session also answers how it was - // constructed and which build accepted its identity, all from the same - // trusted root — an agent that names its own sessions needs all three, and - // refuses unless it has all three. - // - // The two advertised sets are stated by the host, not inherited: this is the - // ordinary `xmd run` profile saying what it has proven. - const acpx = createAcpxProvider( - sessions === undefined - ? undefined - : { - ...(sessions.coordinator ? { coordinator: sessions.coordinator } : {}), - ...(sessions.routeStore ? { routeStore: sessions.routeStore } : {}), - ...(sessions.executableObserver - ? { executableObserver: sessions.executableObserver } - : {}), - advertiseNativeLaunch: sessions.advertiseNativeLaunch, - advertiseClientNativeAttachment: sessions.advertiseClientNativeAttachment, - }, - ); - yield* registerAgentProvider("acpx", acpx); - const defaultAgent = - config.defaultAgent ?? (yield* readEnv("DEFAULT_AGENT_NAME")) ?? DEFAULT_AGENT_NAME; - - // The trusted host selects its own root provider by name. Document-level - // selection goes through the installation protocol; this is the host saying - // what it configured, which no document is composing around. - const providers: Record = { acpx }; - const factory = Object.hasOwn(providers, flags.agentProvider) - ? providers[flags.agentProvider] - : undefined; - if (!factory) { - console.error(`Unknown agent provider "${flags.agentProvider}"`); + const stack = yield* resolveAgentStack(flags, sessions); + if (!stack.ok) { + console.error(stack.error.message); yield* exit(1); return; } - - const permissionMode = config.permissionMode; - yield* installAgentComponents({ - defaultAgent, - permissionMode, - rootProvider: { factory, options: { defaultAgent, permissionMode } }, - }); - yield* installPermissionMode(permissionMode); - // `xmd run` is the one command that has a terminal to give away. Help, - // document inspection and `xmd test` install no launcher, so a document that - // reaches under any of them refuses instead of spawning. - yield* installForegroundLauncher(); + yield* installRunAgentStack(stack.value); } /** @@ -684,8 +670,8 @@ export function* installDocumentComponents(mode: DocumentMode, verbose: boolean) // run — and where the workflow's provider, installed further out, would lose // to it at the same `{ at: "min" }`. - // Agent flags are exclusive to `xmd run` — `xmd test` drives agents - // through the deterministic TestAgent stack instead. + // Agent flags belong to the two commands that end in a document execution — + // `xmd test` drives agents through the deterministic TestAgent stack instead. if (mode.agent) { yield* installAgentStack(mode.agent, mode.machineSessions); } @@ -895,6 +881,66 @@ function* runScopedDocument( } } +/** What an approved prompt document runs with, beyond the source and its props. */ +export interface PromptExecutionConfig { + include: string[]; + verbose: boolean; + journal: string | undefined; + raw: boolean; + secretDetection: boolean; + agentProvider: string; + defaultAgent: string | undefined; + approveAll: boolean; + approveReads: boolean; + denyAll: boolean; +} + +/** + * How `xmd prompt` runs the document a person approved: exactly as `xmd run` + * runs a supplied one. + * + * The generator's scope is already gone by the time this is called, so the + * executed program gets a fresh ordinary Agent provider and inherits neither + * the generator session nor its system prompt. The browser form is composed + * around the document for the same reason a run composes one: `xmd prompt` is a + * command a person is sitting in front of. + */ +export function promptExecutor( + config: PromptExecutionConfig, + sessions: MachineSessionAssembly | undefined, + installService: HostServiceInstaller, +): (approved: PromptExecution) => Operation> { + return (approved) => + scoped(function* (): Operation> { + announceSecretDetection(config.secretDetection); + yield* installWebElicitation(); + return yield* runScopedDocument( + { + root: approved.root, + include: config.include, + verbose: config.verbose, + journal: config.journal, + raw: config.raw, + secretDetection: config.secretDetection, + retainProcessOutput: keepsProcessOutput(config.journal), + }, + { + testing: false, + props: approved.props, + ...(sessions === undefined ? {} : { machineSessions: sessions }), + agent: { + agentProvider: config.agentProvider, + defaultAgent: config.defaultAgent, + approveAll: config.approveAll, + approveReads: config.approveReads, + denyAll: config.denyAll, + }, + }, + installService, + ); + }); +} + /** * A document-target failure as the command line reports it, or `undefined` * when this failure is not one. @@ -1167,6 +1213,15 @@ interface PropsPhase { * would lose exactly the token the separator was there to protect. */ workflow?: { action?: string; target?: string; argument?: string; value?: string }; + /** + * What fixed grammar established about an `xmd prompt` command line. + * + * Carried rather than re-derived downstream, for the same reason the workflow + * positionals are: the request may have been written after `--`, where the + * parser never sees it, and the generated property occurrences are the ones + * this scan classified. + */ + prompt?: PromptScan; root?: RootDocumentSource; bindings: Binding[]; extraction?: Extraction; @@ -1201,6 +1256,15 @@ interface PropsPhase { * of argv, so it needs no parse at all. */ function* preparePropsPhase(args: string[], evalFlags: EvalFlags): Operation { + // `xmd prompt` declares its own grammar and has no document to inspect: the + // schema its generated options come from is written by an agent that has not + // been asked anything yet. Everything below that reads a document, and every + // refusal that assumes one, is therefore skipped. + if (namesPrompt(args)) { + const scan = scanPromptArgs(args); + return { args: scan.fixed, bindings: [], prompt: scan }; + } + // `xmd workflow` reads its options from the head and its remaining positionals // from the tail. The parser only ever sees the head, so a dash-leading token // after `--` is never offered to it as an option; the grammar check below @@ -1264,7 +1328,9 @@ function* preparePropsPhase(args: string[], evalFlags: EvalFlags): Operation" --props-name Ada', + "", + ` ${AGGREGATE_OPTION} `, + " Set document properties as a JSON object", + ` Environment: ${AGGREGATE_ENV}`, + "", + ` ${SAVE_OPTION} `, + " Write the approved document there before running it. The path must not", + " exist; an existing one is left alone and the run stops.", +].join("\n"); + /** * Help for whichever command the arguments name. A command renders its * own help when `--help` is its first argument, so the flag removed @@ -1586,7 +1681,8 @@ function renderHelp(phase: PropsPhase): string { const help = xmd.parse({ args: [command, "--help"] }); const base = help.ok && help.value.config.help ? help.value.config.text : xmd.help({ args: [] }); - const epilogue = command === "run" ? RUN_SOURCE_HELP : ""; + const epilogue = + command === "run" ? RUN_SOURCE_HELP : command === "prompt" ? PROMPT_REQUEST_HELP : ""; const withSource = epilogue === "" ? base : `${base}\n\n${epilogue}`; if (!phase.root) { @@ -1628,23 +1724,11 @@ function* resolveRunProps( } try { - const individualEnv: { binding: Binding; value: string }[] = []; - for (const binding of phase.bindings) { - const value = yield* readEnv(binding.env); - if (value !== undefined) { - individualEnv.push({ binding, value }); - } - } - const aggregateEnv = yield* readEnv(AGGREGATE_ENV); - return { - value: resolveProps({ + value: yield* resolvePropsFromSources({ propsSchema: phase.propsSchema, bindings: phase.bindings, - individual: phase.extraction.individual, - aggregateCli: phase.extraction.aggregate, - aggregateEnv, - individualEnv, + extraction: phase.extraction, }), }; } catch (error) { @@ -1789,11 +1873,52 @@ function* dispatch( } break; } + case "prompt": { + const config = command.config; + const scan = propsPhase.prompt; + if (scan === undefined) { + console.error( + 'xmd prompt names the command first — write `xmd prompt "" [options]`', + ); + yield* exit(1); + break; + } + const exitCode = yield* runPrompt( + { + argv: helpRequest.args, + scan, + include: config.include, + ...(config.save === undefined ? {} : { save: config.save }), + agent: { + agentProvider: config.agentProvider, + defaultAgent: config.defaultAgent, + approveAll: config.approveAll, + approveReads: config.approveReads, + denyAll: config.denyAll, + }, + }, + { + ...(sessions === undefined ? {} : { sessions }), + catalog: syntaxCatalog, + // `` reaches a person through the browser form, and the + // review question is asked by the command rather than by a document. + // A host that answers installs a provider; one that does not installs + // none, and nothing downstream reads a profile to find out which. + installElicitation: installWebElicitation, + execute: promptExecutor(config, sessions, installService), + }, + ); + if (exitCode !== 0) { + yield* exit(exitCode); + } + break; + } case "test": { const strayTimeout = findTimeoutFlag(evalFlags.rest); if (strayTimeout) { console.error( - `unrecognized option for xmd test: ${strayTimeout} — timeout options are exclusive to xmd run`, + `unrecognized option for xmd test: ${strayTimeout} — timeout options are exclusive to ` + + "xmd run and xmd prompt", ); yield* exit(1); break; @@ -1801,7 +1926,8 @@ function* dispatch( const agentFlag = findAgentOnlyFlag(evalFlags.rest); if (agentFlag) { console.error( - `unrecognized option for xmd test: ${agentFlag} — agent options are exclusive to xmd run`, + `unrecognized option for xmd test: ${agentFlag} — agent options are exclusive to ` + + "xmd run and xmd prompt", ); yield* exit(1); break; @@ -1809,7 +1935,8 @@ function* dispatch( const propsFlag = findPropsFlag(evalFlags.rest); if (propsFlag) { console.error( - `unrecognized option for xmd test: ${propsFlag} — document properties are exclusive to xmd run`, + `unrecognized option for xmd test: ${propsFlag} — document properties are exclusive to ` + + "xmd run and xmd prompt", ); yield* exit(1); break; @@ -1848,7 +1975,8 @@ function* dispatch( const agentFlag = findAgentOnlyFlag(evalFlags.rest); if (agentFlag) { console.error( - `unrecognized option for xmd workflow: ${agentFlag} — agent options are exclusive to xmd run`, + `unrecognized option for xmd workflow: ${agentFlag} — agent options are exclusive to ` + + "xmd run and xmd prompt", ); yield* exit(1); break; @@ -1976,14 +2104,22 @@ export function* runXmd( } // Recognized before anything reads a document: a malformed duration is a // grammar failure, and a grammar failure never depends on what is on disk. - // Help, `--version`, and the other commands stay outside a run lifecycle, - // which is why the timeout options are read only for a run. + // Help, `--version`, and the commands that execute nothing stay outside a run + // lifecycle, which is why the timeout options are read only for the two that + // end in one. const provisional = xmd.parse({ args: helpRequest.args }); const selected = provisional.ok ? provisional.value.config : undefined; - const isRun = - !helpRequest.requested && selected !== undefined && !selected.help && selected.name === "run"; - - if (!isRun) { + // The two commands that end in a document execution. `xmd prompt`'s deadline + // encloses more than a run's — the catalog, the generator session, every + // repair, the human review, provider teardown, the save and the execution — + // because all of it is what the caller asked to be bounded. + const executes = + !helpRequest.requested && + selected !== undefined && + !selected.help && + (selected.name === "run" || selected.name === "prompt"); + + if (!executes) { return yield* dispatch(evalFlags, helpRequest, installService, workflowHost, sessions); } diff --git a/packages/cli/src/prompt-args.ts b/packages/cli/src/prompt-args.ts new file mode 100644 index 000000000..a9d49397e --- /dev/null +++ b/packages/cli/src/prompt-args.ts @@ -0,0 +1,320 @@ +/** + * `xmd prompt` argument grammar — everything decidable before a document exists + * (specs/prompt-command-spec.md). + * + * A pure function over argv. It reads no Context Apis, contacts no agent, and + * inspects nothing on disk, which is what lets the command refuse a malformed + * command line before it builds a catalog, opens a session, asks a person, or + * writes a file. + * + * The command line is read in two stages because only the first one is fixed. + * The request, the built-in options and the end-of-options separator are known + * from the grammar alone; whether `--props-name` takes a following token depends + * on what the *generated* document declares, and no document exists yet. So this + * stage classifies what it can, records each generated occurrence with the token + * it provisionally read, and leaves the rest to the candidate that supplies the + * schema. + */ + +import { AGGREGATE_OPTION } from "./props.ts"; +import type { Binding } from "./props.ts"; + +export const PROMPT_COMMAND = "prompt"; +export const SAVE_OPTION = "--save"; + +/** The built-in options that take a separated value. */ +const VALUE_OPTIONS: readonly string[] = [ + "--include", + "--journal", + "-j", + "--agent-provider", + "--default-agent", + "--timeout", + "--timeout-exec", + "--timeout-fetch", + SAVE_OPTION, +]; + +/** The built-in options that take none. */ +const SWITCH_OPTIONS: readonly string[] = [ + "--verbose", + "-V", + "--raw", + "--approve-all", + "--approve-reads", + "--deny-all", + "--secret-detection", + "--no-secret-detection", + "--help", + "-h", + "--version", +]; + +const VALUE = new Set(VALUE_OPTIONS); +const KNOWN = new Set([...VALUE_OPTIONS, ...SWITCH_OPTIONS, AGGREGATE_OPTION]); + +function optionName(token: string): string { + const equals = token.indexOf("="); + return equals === -1 ? token : token.slice(0, equals); +} + +function generatesProperty(name: string): boolean { + return name.startsWith("--props-") || name.startsWith("--no-props"); +} + +/** + * Whether this token is an option `xmd` itself defines. + * + * Used to decide what a generated property option may provisionally read as its + * value. A caller writing `--props-name --raw` means the switch, not a value of + * `--raw`; a value that really begins with `-` is written `--props-name=-value`. + */ +function isKnownOption(token: string): boolean { + if (!token.startsWith("-") || token === "-") { + return false; + } + const name = optionName(token); + return KNOWN.has(name) || generatesProperty(name); +} + +/** One `--props-*` token, as written. */ +export interface PropertyOccurrence { + /** The option name, without any `=value`. */ + option: string; + /** The value written with `=`, when it was written that way. */ + inline?: string; + /** + * The following token this scan read as the option's value. + * + * Provisional: whether the option takes one is the candidate's answer, and a + * candidate that declares it a switch turns this token into a second request. + */ + provisional?: string; +} + +/** What fixed grammar establishes about one `xmd prompt` command line. */ +export interface PromptScan { + /** The request, byte for byte, when exactly one was written. */ + request?: string; + /** + * The argv the built-in option parser sees. + * + * Generated property tokens and the aggregate `--props` are removed: the + * parser defines neither, and it coerces a separated value through `Number()` + * before any schema could judge it. Tokens after `--` are left out for the + * same reason the workflow command leaves them out — a dash-leading positional + * handed back to a parser is read as an option again. + */ + fixed: string[]; + /** Every generated property occurrence, in the order it was written. */ + occurrences: PropertyOccurrence[]; + /** Why fixed grammar refuses this command line. */ + error?: string; +} + +/** Whether these arguments select the `prompt` command. */ +export function namesPrompt(args: readonly string[]): boolean { + return args[0] === PROMPT_COMMAND; +} + +const ORDER_HELP = + "document properties follow the request, as in " + + '`xmd prompt "" --props-name `'; + +export function scanPromptArgs(args: readonly string[]): PromptScan { + const fixed: string[] = [PROMPT_COMMAND]; + const occurrences: PropertyOccurrence[] = []; + let request: string | undefined; + let extra: string | undefined; + let parsingOptions = true; + let index = 1; + + while (index < args.length) { + const token = args[index]; + + if (parsingOptions && token === "--") { + parsingOptions = false; + index += 1; + continue; + } + + if (parsingOptions && token.startsWith("-") && token !== "-") { + const equals = token.indexOf("="); + const name = optionName(token); + + if (name === AGGREGATE_OPTION) { + index += equals === -1 && args[index + 1] !== undefined ? 2 : 1; + continue; + } + + if (generatesProperty(name)) { + if (request === undefined) { + return { + fixed, + occurrences, + error: `unrecognized option: ${name} — ${ORDER_HELP}`, + }; + } + if (equals !== -1) { + occurrences.push({ option: name, inline: token.slice(equals + 1) }); + index += 1; + continue; + } + const next = args[index + 1]; + if (next === undefined || next === "--" || isKnownOption(next)) { + occurrences.push({ option: name }); + index += 1; + continue; + } + occurrences.push({ option: name, provisional: next }); + index += 2; + continue; + } + + fixed.push(token); + if (equals === -1 && VALUE.has(name) && args[index + 1] !== undefined) { + fixed.push(args[index + 1]); + index += 2; + continue; + } + index += 1; + continue; + } + + if (request === undefined) { + request = token; + // Kept out of the parser's argv when the separator carried it: a + // dash-leading request written after `--` is positional because of where + // it is, and handing it back would make it an option again. + if (parsingOptions) { + fixed.push(token); + } + } else if (extra === undefined) { + extra = token; + } + index += 1; + } + + if (extra !== undefined) { + return { + ...(request === undefined ? {} : { request }), + fixed, + occurrences, + error: + `unrecognized argument for xmd prompt: ${extra} — the command takes exactly one ` + + "request, and " + + ORDER_HELP, + }; + } + + if (request === undefined) { + return { + fixed, + occurrences, + error: 'xmd prompt requires one request — `xmd prompt ""`', + }; + } + + if (request.trim().length === 0) { + return { + request, + fixed, + occurrences, + error: "xmd prompt requires a request with at least one non-whitespace character", + }; + } + + return { request, fixed, occurrences }; +} + +/** + * How a supplied individual option is written: whether it takes a token, and + * whether repeating it accumulates. + * + * This is what a later candidate may not change. The comparison happens before + * any token is extracted, so a switch that became a value option cannot reach + * forward and consume the `--raw` written after it. + */ +export interface OptionSignature { + boolean: boolean; + array: boolean; +} + +export function signatureOf(binding: Binding): OptionSignature { + return { boolean: binding.boolean, array: binding.array }; +} + +function describeSignature(signature: OptionSignature): string { + if (signature.boolean) { + return "a bare switch"; + } + return signature.array ? "a repeated value option" : "a single-value option"; +} + +/** + * Whether the candidate still declares every supplied option the way the + * candidate that first bound it did. + * + * A removed option or a changed shape is the caller's command line meaning + * something else than it did, which no revision may do silently. + */ +export function signatureFailure( + frozen: ReadonlyMap, + bindings: readonly Binding[], +): string | undefined { + const current = new Map(bindings.map((binding) => [binding.option, signatureOf(binding)])); + for (const [option, signature] of frozen) { + const now = current.get(option); + if (now === undefined) { + return ( + `${option} was accepted by an earlier draft and this one declares no such property — ` + + "the command line no longer describes the document under review" + ); + } + if (now.boolean !== signature.boolean || now.array !== signature.array) { + return ( + `${option} was ${describeSignature(signature)} in an earlier draft and is ` + + `${describeSignature(now)} in this one — the command line no longer describes the ` + + "document under review" + ); + } + } + return undefined; +} + +/** + * The token a candidate's own arity turns into a second request. + * + * `--props-loud true` reads as an option and a value until a candidate declares + * `loud` a boolean; from then on `true` is a positional, and the command takes + * exactly one. + */ +export function strayPropertyValue( + occurrences: readonly PropertyOccurrence[], + bindings: readonly Binding[], +): string | undefined { + const byOption = new Map(bindings.map((binding) => [binding.option, binding])); + for (const occurrence of occurrences) { + const { provisional } = occurrence; + if (provisional === undefined) { + continue; + } + const binding = byOption.get(occurrence.option); + if (binding?.boolean === true) { + return ( + `unrecognized argument for xmd prompt: ${provisional} — ${occurrence.option} is a ` + + `switch, so this is a second request; write \`${occurrence.option}=${provisional}\` ` + + "to give it a value" + ); + } + } + return undefined; +} + +/** + * Whether this token is an option the invocation owns, and therefore one a + * generated property may not read as its value. + */ +export function isReservedOption(token: string): boolean { + return isKnownOption(token); +} diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts new file mode 100644 index 000000000..cda8bd3a0 --- /dev/null +++ b/packages/cli/src/prompt.ts @@ -0,0 +1,652 @@ +/** + * `xmd prompt` — ask the configured agent for a document, prove it is one, let + * a person decide, and then run it (specs/prompt-command-spec.md). + * + * The command adds authorship around one ordinary document; it adds no second + * execution model. Everything before approval happens with no journal, no + * durable identity and no execution: generation is a conversation, and a + * conversation is not a run. + * + * The lifecycle is one ordered ownership chain, and no phase after the first + * failure begins: + * + * ```text + * fixed CLI preflight + * -> structured syntax catalog + * -> fresh generator session + * -> candidate props + validation + * -> automatic repair, at most three turns + * -> live approval or human revision + * -> generator teardown + * -> optional exclusive save + * -> ordinary in-memory document execution + * ``` + * + * Two kinds of failure are told apart throughout, because they have different + * remedies. A *candidate* failure is something the agent wrote, so the agent is + * asked to write it again with the validation facts. A *caller* failure is + * something the command line or the environment said, so it terminates: teaching + * the agent about it would spend a repair turn on a defect the agent cannot fix. + */ + +import { Err, Ok, scoped, until } from "effection"; +import type { Operation, Result } from "effection"; +import { open } from "node:fs/promises"; +import type { FileHandle } from "node:fs/promises"; +import { randomUUID } from "node:crypto"; +import { resolve } from "node:path"; + +import { + agentIdentityComponents, + elicit, + inspectDocument, + retainedSource, + validateDocument, +} from "@executablemd/core"; +import type { + DocumentValidation, + DocumentValidationCode, + Json, + PropsSchema, + RootDocumentSource, + Session, + SyntaxCatalog, +} from "@executablemd/core"; +import type { AcpxProvider, AcpxProviderDependencies } from "@executablemd/acp"; +import { cwd } from "@executablemd/runtime"; + +import { resolveAgentStack, useGeneratorAgent } from "./agent-stack.ts"; +import type { AgentStack } from "./agent-stack.ts"; +import type { AgentFlags } from "./agent-config.ts"; +import type { MachineSessionAssembly } from "./session-coordinator.ts"; +import { + buildBindings, + describeError, + extractPropsArgs, + resolvePropsFromSources, +} from "./props.ts"; +import type { Binding, Extraction } from "./props.ts"; +import { renderSyntaxMarkdown, useRunProfileRegistry } from "./syntax.ts"; +import { + isReservedOption, + signatureFailure, + signatureOf, + strayPropertyValue, +} from "./prompt-args.ts"; +import type { OptionSignature, PromptScan } from "./prompt-args.ts"; + +/** + * The identity generated text runs under. + * + * Deliberate and fixed, the way `` is for an inline document: the bytes + * came from an agent, not from a file, and a source position reading + * `(:5:1)` says so. It affects positions and diagnostics only — + * components, includes and every relative filesystem operation still resolve + * from the contextual working directory. + */ +export const PROMPT_IDENTITY = ""; + +/** How many replacement drafts one candidate may be asked for automatically. */ +export const REPAIR_TURNS = 3; + +/** The approved bytes, and the props resolved under exactly those bytes. */ +export interface PromptExecution { + root: RootDocumentSource; + props: Record; +} + +/** What review approved: the exact candidate, and its own resolved props. */ +interface ApprovedPrompt { + source: string; + props: Record; +} + +/** What one `xmd prompt` invocation was asked to do. */ +export interface PromptCommand { + /** The argv this invocation holds, and the props source for every candidate. */ + argv: string[]; + /** What fixed grammar established about that argv. */ + scan: PromptScan; + include: string[]; + /** Where the approved source is written before it runs, when asked for. */ + save?: string; + agent: AgentFlags; +} + +/** What the host supplies. Every entry is a decision only a host can make. */ +export interface PromptDependencies { + /** What this host states about machine-wide agent sessions, if anything. */ + sessions?: MachineSessionAssembly; + /** What the generator provider is built on, beyond the host's own assembly. */ + acp?: AcpxProviderDependencies; + /** The run profile's complete structured vocabulary. */ + catalog(includes: readonly string[]): Operation; + /** Who answers the review question. */ + installElicitation(): Operation; + /** Run the approved document the way this host runs any supplied one. */ + execute(approved: PromptExecution): Operation>; +} + +/** The prompt-owned findings that are not core's to report. */ +interface PromptDiagnostic { + code: "generated-binding-collision" | "root-props-unreadable"; + message: string; +} + +/** Everything definite that is wrong with one candidate. */ +interface CandidateDefects { + /** Core's complete versioned answer, whenever core produced one. */ + validation?: DocumentValidation; + /** What this command found about the options the candidate generates. */ + prompt?: PromptDiagnostic; +} + +type CandidateOutcome = + | { kind: "valid"; props: Record } + | { kind: "repairable"; defects: CandidateDefects } + | { kind: "terminal"; error: Error }; + +/** A candidate offered for review, and what may be decided about it. */ +type ReviewSubject = + | { candidate: string; valid: true; props: Record } + | { candidate: string; valid: false; defects: CandidateDefects }; + +interface ReviewDecision { + decision: "approve" | "revise" | "abort"; + feedback?: string; +} + +/** The codes that say the root's own declaration could not be read. */ +const DECLARATION_CODES: ReadonlySet = new Set([ + "source-unreadable", + "source-invalid", + "target-invalid", + "frontmatter-invalid", + "props-declaration-invalid", + "returns-declaration-invalid", +]); + +const ABORTED = "xmd prompt: aborted at review — nothing was saved and nothing ran"; + +/** + * Run the command, and report the process status it earned. + * + * Every phase is behind a returned value rather than behind a flag another + * phase reads, so a refusal cannot be followed by the work it refused. + */ +export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Operation { + const { scan } = command; + if (scan.error !== undefined || scan.request === undefined) { + console.error(scan.error ?? 'xmd prompt requires one request — `xmd prompt ""`'); + return 1; + } + const request = scan.request; + + const stack = yield* resolveAgentStack(command.agent, deps.sessions); + if (!stack.ok) { + console.error(stack.error.message); + return 1; + } + + let instructions: string; + try { + instructions = generatorInstructions(yield* deps.catalog(command.include)); + } catch (error) { + console.error(describeError(error)); + return 1; + } + + // The generator lives and dies inside this scope. Leaving it is what closes + // the provider, so a teardown failure is raised here — before the save and + // the execution that would otherwise already have happened. + let approved: Result; + try { + approved = yield* scoped(function* (): Operation> { + yield* deps.installElicitation(); + // The same vocabulary the catalog just described. Validation and the + // catalog read one registry, so a component the generator was told about + // is one validation resolves. + yield* useRunProfileRegistry(); + const provider = yield* useGeneratorAgent(stack.value, instructions, deps.acp); + const session = yield* provider.session(generatorSessionName()); + return yield* author(command, provider, session, request); + }); + } catch (error) { + console.error(describeError(error)); + return 1; + } + + if (!approved.ok) { + console.error(approved.error.message); + return 1; + } + + const { source, props } = approved.value; + if (command.save !== undefined) { + const saved = yield* saveSource(command.save, source); + if (!saved.ok) { + console.error(saved.error.message); + return 1; + } + } + + const executed = yield* deps.execute({ + root: retainedSource(PROMPT_IDENTITY, source), + props, + }); + if (!executed.ok) { + console.error(executed.error.message); + return 1; + } + return 0; +} + +/** + * A logical session name nothing else can name. + * + * The conversation belongs to this invocation: a second `xmd prompt` places a + * different session rather than continuing this one, and a request nobody meant + * to repeat never arrives in a history it did not create. + */ +function generatorSessionName(): string { + return `xmd-prompt:${randomUUID()}`; +} + +/** + * What the generator session is told once, before it is asked anything. + * + * The catalog is the run profile's own, rendered by the same renderer + * `xmd syntax` uses — including its statement that a repository TypeScript + * component's contract was not read, which is the honest thing to tell a writer + * about a component nobody imported. + */ +export function generatorInstructions(catalog: SyntaxCatalog): string { + return [ + "You write complete executable Markdown root documents. Each user message is a", + "request for one document.", + "", + "Every answer is replacement document source and nothing else: no enclosing code", + "fence, no explanation, and no prose around it. What you return is written to a", + "file and executed exactly as you wrote it.", + "", + "A repair request carries the validation facts about the document you last sent.", + "Answer it with another complete replacement document, not a patch and not a", + "description of the change.", + "", + "The catalog below is the complete vocabulary available in the working directory", + "this command runs in and in its configured include paths. Nothing outside it is", + "available.", + "", + renderSyntaxMarkdown(catalog), + ].join("\n"); +} + +/** + * The authorship loop: draft, repair, review, revise. + * + * Each draft — the first one, and each answer to a human revision — starts its + * own repair budget. A human revision has no numeric bound: the person asking + * for changes is the one deciding when to stop. + */ +function* author( + command: PromptCommand, + provider: AcpxProvider, + session: Session, + request: string, +): Operation> { + const frozen = new Map(); + let draft = yield* runTurn(provider, session, request); + + while (true) { + if (!draft.ok) { + return draft; + } + let candidate = draft.value; + let budget = REPAIR_TURNS; + let subject: ReviewSubject | undefined; + + while (subject === undefined) { + const outcome = yield* evaluate(command, frozen, candidate); + if (outcome.kind === "terminal") { + return Err(outcome.error); + } + if (outcome.kind === "valid") { + subject = { candidate, valid: true, props: outcome.props }; + break; + } + if (budget === 0) { + subject = { candidate, valid: false, defects: outcome.defects }; + break; + } + budget -= 1; + const repaired = yield* runTurn(provider, session, repairRequest(outcome.defects)); + if (!repaired.ok) { + return repaired; + } + candidate = repaired.value; + } + + const decided = yield* review(subject); + if (!decided.ok) { + return decided; + } + const decision = decided.value; + if (decision.decision === "abort") { + return Err(new Error(ABORTED)); + } + if (decision.decision === "approve") { + if (!subject.valid) { + // The schema an exhausted candidate is offered has no approve value, so + // an answer carrying one came from somewhere the question did not. + return Err( + new Error( + "the review provider approved a document that does not validate, which the " + + "question did not offer — nothing was saved and nothing ran", + ), + ); + } + return Ok({ source: subject.candidate, props: subject.props }); + } + draft = yield* runTurn(provider, session, revisionRequest(decision.feedback ?? "")); + } +} + +/** + * One turn, consumed whole inside the provider's scope. + * + * Only a terminal `completed` produces a candidate. A failed, cancelled or + * unavailable turn is a generation failure however much text it emitted, and the + * partial text is discarded rather than presented: half a program is not one. + */ +function* runTurn( + provider: AcpxProvider, + session: Session, + content: string, +): Operation> { + return yield* scoped(function* (): Operation> { + try { + const subscription = yield* provider.promptStream(content, { session }); + let status: string | undefined; + let stopReason: string | undefined; + let failure: Error | undefined; + let next = yield* subscription.next(); + while (!next.done) { + const event = next.value; + if (event.type === "terminal") { + status = event.status; + stopReason = event.stopReason; + failure = event.error; + } + next = yield* subscription.next(); + } + if (status !== "completed") { + return Err(turnFailure(status, stopReason, failure)); + } + return Ok(next.value); + } catch (error) { + return Err(toError(error)); + } + }); +} + +function turnFailure( + status: string | undefined, + stopReason: string | undefined, + failure: Error | undefined, +): Error { + const detail = + failure?.message ?? (stopReason === undefined ? undefined : `stop reason ${stopReason}`); + const outcome = status ?? "produced no terminal event"; + return new Error( + `the agent did not complete the turn (${outcome})` + + (detail === undefined ? "" : `: ${detail}`) + + " — nothing was reviewed, saved or run", + ); +} + +/** + * Everything decidable about one candidate, in the order that keeps a caller's + * mistake from being taught to the agent. + */ +function* evaluate( + command: PromptCommand, + frozen: Map, + candidate: string, +): Operation { + const root = retainedSource(PROMPT_IDENTITY, candidate); + const includes = command.include; + const components = agentIdentityComponents(); + + // Whether the root declares itself readably. Inspection would raise on a + // malformed declaration, and recovering a code from an exception's prose is + // exactly what the structured answer exists to replace. + const declaration = yield* validateDocument({ ...root, includes, components }); + if (declaration.diagnostics.some((entry) => DECLARATION_CODES.has(entry.code))) { + return { kind: "repairable", defects: { validation: declaration } }; + } + + let propsSchema: PropsSchema; + try { + propsSchema = (yield* inspectDocument(root)).props; + } catch (error) { + return { + kind: "repairable", + defects: { prompt: { code: "root-props-unreadable", message: describeError(error) } }, + }; + } + + let bindings: Binding[]; + try { + bindings = buildBindings(propsSchema); + } catch (error) { + return { + kind: "repairable", + defects: { + prompt: { code: "generated-binding-collision", message: describeError(error) }, + }, + }; + } + + // Before a single token is extracted: an option that changed shape would + // otherwise reach forward and read the `--raw` written after it as its value. + const drift = signatureFailure(frozen, bindings); + if (drift !== undefined) { + return { kind: "terminal", error: new Error(drift) }; + } + const stray = strayPropertyValue(command.scan.occurrences, bindings); + if (stray !== undefined) { + return { kind: "terminal", error: new Error(stray) }; + } + + let extraction: Extraction; + let props: Record; + try { + extraction = extractPropsArgs(command.argv, bindings, { reserved: isReservedOption }); + props = yield* resolvePropsFromSources({ propsSchema, bindings, extraction }); + } catch (error) { + return { kind: "terminal", error: toError(error) }; + } + + for (const supplied of extraction.individual) { + frozen.set(supplied.binding.option, signatureOf(supplied.binding)); + } + + const validation = yield* validateDocument({ ...root, props, includes, components }); + if (validation.outcome === "invalid") { + return { kind: "repairable", defects: { validation } }; + } + return { kind: "valid", props }; +} + +/** + * The repair turn: the fixed instruction, and the facts, as data. + * + * Core's own versioned value travels whole. Nothing here renders a diagnostic + * into prose and nothing parses one back out, so what the agent reads is what + * validation concluded rather than a summary of it. + */ +function repairRequest(defects: CandidateDefects): string { + return [ + "The document you sent does not validate. Send a complete replacement document", + "that fixes every defect below. Reply with document source only.", + "", + fenced("json", JSON.stringify(defects, null, 2)), + ].join("\n"); +} + +function revisionRequest(feedback: string): string { + return [ + "A person reviewed the document and asked for this change:", + "", + feedback, + "", + "Send a complete replacement document. Reply with document source only.", + ].join("\n"); +} + +/** + * Ask a person, and read the one answer. + * + * The whole candidate is shown exactly as it will run, inside a fence chosen to + * be longer than any backtick run the candidate holds — arbitrary source cannot + * close a fence it cannot spell. Machinery stays out of what the person reads: + * the schema, the session and the repair budget are none of their business. + */ +function* review(subject: ReviewSubject): Operation> { + const decisions = subject.valid ? ["approve", "revise", "abort"] : ["revise", "abort"]; + const message = subject.valid + ? [ + "The agent wrote this document. Approve it to run it, ask for a change, or abort.", + "", + fenced("md", subject.candidate), + ].join("\n") + : [ + "The agent could not produce a document that validates. Ask for a change, or abort.", + "", + fenced("md", subject.candidate), + "", + "Validation reported these defects:", + "", + fenced("json", JSON.stringify(subject.defects, null, 2)), + ].join("\n"); + + try { + const answer = yield* elicit({ + message, + schema: reviewSchema(decisions), + label: "PromptReview", + }); + return readDecision(answer); + } catch (error) { + return Err(toError(error)); + } +} + +function reviewSchema(decisions: readonly string[]): Json { + return { + type: "object", + title: "Review the generated document", + properties: { + decision: { + type: "string", + title: "Decision", + enum: [...decisions], + }, + feedback: { + type: "string", + title: "Feedback", + description: "What to change. Required when asking for a revision.", + }, + }, + required: ["decision"], + additionalProperties: false, + if: { properties: { decision: { const: "revise" } }, required: ["decision"] }, + then: { properties: { feedback: { type: "string", minLength: 1 } }, required: ["feedback"] }, + }; +} + +/** Read the provider's answer rather than assert it. */ +function readDecision(answer: Json): Result { + if (typeof answer !== "object" || answer === null || Array.isArray(answer)) { + return Err(new Error("the review provider answered with something that is not a decision")); + } + const decision = answer.decision; + if (decision !== "approve" && decision !== "revise" && decision !== "abort") { + return Err(new Error("the review provider answered with no decision")); + } + const feedback = answer.feedback; + return Ok({ + decision, + ...(typeof feedback === "string" ? { feedback } : {}), + }); +} + +/** + * A fence long enough that the body cannot close it. Three backticks at least, + * and one more than the longest run the body holds. + */ +function fenced(language: string, body: string): string { + let longest = 2; + for (const run of body.match(/`+/g) ?? []) { + longest = Math.max(longest, run.length); + } + const delimiter = "`".repeat(longest + 1); + return [`${delimiter}${language}`, body, delimiter].join("\n"); +} + +/** + * Create the destination and write the approved bytes, or refuse. + * + * Exclusive creation, so an existing path is left exactly as it is and the run + * stops rather than replacing work somebody kept. There is no check-then-write: + * the open is the check. + */ +function* saveSource(path: string, source: string): Operation> { + const target = resolve(yield* cwd(), path); + let handle: FileHandle; + try { + handle = yield* until(open(target, "wx")); + } catch (error) { + const existing = + error instanceof Error && + (("code" in error && error.code === "EEXIST") || error.message.startsWith("EEXIST:")); + if (existing) { + return Err( + new Error( + `${target} already exists — choose another --save path; the approved document was ` + + "not written and nothing ran", + ), + ); + } + return Err(new Error(`could not create ${target}: ${describeError(error)}`)); + } + + const written = yield* writeAll(handle, source); + const closed = yield* closeHandle(handle, target); + if (!written.ok) { + return written; + } + return closed; +} + +function* writeAll(handle: FileHandle, source: string): Operation> { + try { + yield* until(handle.writeFile(source, "utf8")); + return Ok(undefined); + } catch (error) { + return Err(new Error(`could not write the approved document: ${describeError(error)}`)); + } +} + +function* closeHandle(handle: FileHandle, target: string): Operation> { + try { + yield* until(handle.close()); + return Ok(undefined); + } catch (error) { + return Err(new Error(`could not close ${target}: ${describeError(error)}`)); + } +} + +function toError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)); +} diff --git a/packages/cli/src/props.ts b/packages/cli/src/props.ts index 8cfb456de..8afb5e326 100644 --- a/packages/cli/src/props.ts +++ b/packages/cli/src/props.ts @@ -10,6 +10,8 @@ import type { StandardSchemaV1 } from "@standard-schema/spec"; import { createContext, field, object } from "configliere"; import type { Parser } from "configliere"; import type { Json } from "@executablemd/durable-streams"; +import { env as readEnv } from "@executablemd/runtime"; +import type { Operation } from "effection"; import { z } from "zod"; export interface Binding { @@ -523,6 +525,21 @@ export interface Extraction { rest: string[]; } +export interface ExtractOptions { + /** + * Whether a token is an option the invocation itself owns, and therefore one + * a separated property value may never be. + * + * `xmd run` supplies none: the document is already named, so the schema that + * decides a property's arity cannot change under the same command line. + * `xmd prompt` re-binds the same argv against every candidate, so a value + * option that swallowed `--raw` in one draft would take the flag away from + * the invocation itself. A value that really begins with `-` is written + * `--props-name=-value`. + */ + reserved?: (token: string) => boolean; +} + /** * Remove `--props` and `--props-*` tokens from argv, keeping their * original text. Configliere's own option matching coerces every value @@ -530,7 +547,12 @@ export interface Extraction { * Only the generated bindings are recognized, so this stays a source * adapter rather than a second argument parser. */ -export function extractPropsArgs(args: string[], bindings: Binding[]): Extraction { +export function extractPropsArgs( + args: string[], + bindings: Binding[], + options?: ExtractOptions, +): Extraction { + const reserved = options?.reserved ?? (() => false); const byOption = new Map(bindings.map((binding) => [binding.option, binding])); const collected = new Map(); const individual: { binding: Binding; value: string | string[] }[] = []; @@ -608,6 +630,12 @@ export function extractPropsArgs(args: string[], bindings: Binding[]): Extractio if (next === undefined) { throw new PropsError(`${binding.option} requires a value`); } + if (reserved(next)) { + throw new PropsError( + `${binding.option} requires a value — \`${next}\` is an xmd option, so write ` + + `\`${binding.option}=${next}\` for a value that begins with "-"`, + ); + } record(binding, next); index += 2; } @@ -819,6 +847,39 @@ export function resolveProps(options: ResolveOptions): Record { return props; } +/** + * Resolve one document's properties from every source this host offers. + * + * The command line arrives already extracted; the environment is read here, one + * lookup per declared binding plus the aggregate. Both commands that supply root + * props go through this, so precedence and decoding exist once — a second copy + * would be a second set of rules the specification does not describe. + */ +export function* resolvePropsFromSources(options: { + propsSchema: unknown; + bindings: Binding[]; + extraction: Extraction; +}): Operation> { + const { propsSchema, bindings, extraction } = options; + const individualEnv: { binding: Binding; value: string }[] = []; + for (const binding of bindings) { + const value = yield* readEnv(binding.env); + if (value !== undefined) { + individualEnv.push({ binding, value }); + } + } + const aggregateEnv = yield* readEnv(AGGREGATE_ENV); + + return resolveProps({ + propsSchema, + bindings, + individual: extraction.individual, + ...(extraction.aggregate === undefined ? {} : { aggregateCli: extraction.aggregate }), + ...(aggregateEnv === undefined ? {} : { aggregateEnv }), + individualEnv, + }); +} + /** * Render the document-property section of `xmd run --help`. * Everything shown is declarative — accepted bindings, descriptions, diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index bd91e5f83..873c3a4bd 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -52,15 +52,28 @@ import { WEB_REGISTRATIONS } from "@executablemd/web"; */ export function* syntaxCatalog(includes: readonly string[]): Operation { return yield* scoped(function* () { - yield* registerComponents([ - ...AGENT_REGISTRATIONS, - ...TESTING_REGISTRATIONS, - ...WEB_REGISTRATIONS, - ]); + yield* useRunProfileRegistry(); return yield* inspectSyntax({ includes, components: agentIdentityComponents() }); }); } +/** + * The registrations the `run` profile installs, as registry state and nothing + * else. + * + * Shared with `xmd prompt`, which both describes this vocabulary to a generator + * and validates what comes back. Registering only here would make the catalog + * advertise `` while validation reported it unresolved — a document told + * to use a component nobody would accept. + */ +export function* useRunProfileRegistry(): Operation { + yield* registerComponents([ + ...AGENT_REGISTRATIONS, + ...TESTING_REGISTRATIONS, + ...WEB_REGISTRATIONS, + ]); +} + /** * The catalog as JSON: two-space indent, one trailing newline. * diff --git a/packages/cli/tests/prompt-args.test.ts b/packages/cli/tests/prompt-args.test.ts new file mode 100644 index 000000000..8927e74b8 --- /dev/null +++ b/packages/cli/tests/prompt-args.test.ts @@ -0,0 +1,234 @@ +/** + * Tier PR — `xmd prompt` fixed grammar (specs/prompt-command-spec.md). + * + * Rows P1–P6, in the half that is decidable without a document. `scanPromptArgs` + * is a pure function over argv, so what the command line means — and every + * refusal it earns — is asserted directly rather than inferred from a process + * that printed nothing. + * + * The rest of P5 and P6 live in `prompt.test.ts`, where a candidate schema + * exists to bind against. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; + +import { + isReservedOption, + namesPrompt, + scanPromptArgs, + signatureFailure, + signatureOf, + strayPropertyValue, +} from "../src/prompt-args.ts"; +import type { OptionSignature } from "../src/prompt-args.ts"; +import { buildBindings, extractPropsArgs, PropsError } from "../src/props.ts"; +import type { Binding } from "../src/props.ts"; + +const REQUEST = "ask me for my age and write the result to a file"; + +function bindingsFor(properties: Record): Binding[] { + return buildBindings({ type: "object", properties, additionalProperties: false }); +} + +function frozen(entries: Record): Map { + return new Map(Object.entries(entries)); +} + +describe("Tier PR — xmd prompt fixed grammar", () => { + it("P1: exactly one request, kept byte for byte", function* () { + expect(namesPrompt(["prompt", REQUEST])).toBe(true); + expect(namesPrompt(["run", "doc.md"])).toBe(false); + + const one = scanPromptArgs(["prompt", REQUEST]); + expect(one.error).toBe(undefined); + expect(one.request).toBe(REQUEST); + + // Preserved, not trimmed: trimming is only how emptiness is tested. + const padded = scanPromptArgs(["prompt", ` ${REQUEST}\n`]); + expect(padded.error).toBe(undefined); + expect(padded.request).toBe(` ${REQUEST}\n`); + + expect(scanPromptArgs(["prompt"]).error).toContain("requires one request"); + expect(scanPromptArgs(["prompt", "", "--raw"]).error).toContain("non-whitespace"); + expect(scanPromptArgs(["prompt", " \t\n "]).error).toContain("non-whitespace"); + expect(scanPromptArgs(["prompt", REQUEST, "second"]).error).toContain( + "unrecognized argument for xmd prompt: second", + ); + + // A request that begins with a dash is written after the separator, and is + // still exactly one request. + const separated = scanPromptArgs(["prompt", "--", "--not-an-option"]); + expect(separated.error).toBe(undefined); + expect(separated.request).toBe("--not-an-option"); + // And it is kept out of the parser's argv, so nothing reads it as an option. + expect(separated.fixed).toEqual(["prompt"]); + }); + + it("P2: individual options follow the request, aggregate props may precede it", function* () { + const early = scanPromptArgs(["prompt", "--props-name", "Ada", REQUEST]); + expect(early.error).toContain("unrecognized option: --props-name"); + expect(early.error).toContain("follow the request"); + // Refused before anything is classified: no request was adopted from the + // tokens that followed, and no occurrence was recorded. + expect(early.request).toBe(undefined); + expect(early.occurrences).toEqual([]); + + const aggregate = scanPromptArgs(["prompt", "--props", '{"name":"Ada"}', REQUEST, "--raw"]); + expect(aggregate.error).toBe(undefined); + expect(aggregate.request).toBe(REQUEST); + // The aggregate never reaches the parser: it coerces a separated value + // through Number() before any schema could judge it. + expect(aggregate.fixed).toEqual(["prompt", REQUEST, "--raw"]); + + const inline = scanPromptArgs(["prompt", '--props={"name":"Ada"}', REQUEST]); + expect(inline.error).toBe(undefined); + expect(inline.fixed).toEqual(["prompt", REQUEST]); + }); + + it("P3: built-in options after generated props stay with the invocation", function* () { + const scan = scanPromptArgs([ + "prompt", + REQUEST, + "--props-name", + "Ada", + "--raw", + "--include", + "lib", + "--props-loud", + "--journal", + "trace.jsonl", + "--save", + "out.md", + ]); + expect(scan.error).toBe(undefined); + expect(scan.request).toBe(REQUEST); + expect(scan.fixed).toEqual([ + "prompt", + REQUEST, + "--raw", + "--include", + "lib", + "--journal", + "trace.jsonl", + "--save", + "out.md", + ]); + // `--props-loud` did not swallow `--journal`: a known option is never read + // as a generated property's value. + expect(scan.occurrences).toEqual([ + { option: "--props-name", provisional: "Ada" }, + { option: "--props-loud" }, + ]); + + // Nor at extraction, once a candidate declares `loud` a value option. + const bindings = bindingsFor({ loud: { type: "string" } }); + let failure: unknown; + try { + extractPropsArgs(["prompt", REQUEST, "--props-loud", "--raw"], bindings, { + reserved: isReservedOption, + }); + } catch (error) { + failure = error; + } + expect(failure).toBeInstanceOf(PropsError); + expect(String(failure)).toContain("--props-loud requires a value"); + expect(String(failure)).toContain("--props-loud=--raw"); + + // `xmd run` supplies no reserved list, so its behavior is unchanged. + const asRun = extractPropsArgs(["--props-loud", "--raw"], bindings); + expect(asRun.individual).toEqual([{ binding: bindings[0], value: "--raw" }]); + + expect(isReservedOption("--raw")).toBe(true); + expect(isReservedOption("--include=lib")).toBe(true); + expect(isReservedOption("--props-other")).toBe(true); + expect(isReservedOption("Ada")).toBe(false); + expect(isReservedOption("-5")).toBe(false); + }); + + it("P4: scalar, boolean and aggregate sources are all recorded", function* () { + const scan = scanPromptArgs([ + "prompt", + REQUEST, + "--props-name", + "Ada", + "--props-loud", + "--props-tag=alpha", + "--props-tag=beta", + "--props", + '{"count":2}', + ]); + expect(scan.error).toBe(undefined); + expect(scan.occurrences).toEqual([ + { option: "--props-name", provisional: "Ada" }, + { option: "--props-loud" }, + { option: "--props-tag", inline: "alpha" }, + { option: "--props-tag", inline: "beta" }, + ]); + + const bindings = bindingsFor({ + name: { type: "string" }, + loud: { type: "boolean" }, + tag: { type: "array", items: { type: "string" } }, + count: { type: "number" }, + }); + const extraction = extractPropsArgs( + [ + "prompt", + REQUEST, + "--props-name", + "Ada", + "--props-loud", + "--props-tag=alpha", + "--props-tag=beta", + "--props", + '{"count":2}', + ], + bindings, + { reserved: isReservedOption }, + ); + expect(extraction.aggregate).toBe('{"count":2}'); + expect(extraction.individual.map((entry) => [entry.binding.option, entry.value])).toEqual([ + ["--props-name", "Ada"], + ["--props-loud", "true"], + ["--props-tag", ["alpha", "beta"]], + ]); + }); + + it("P5: a boolean binding turns its provisional value into a second request", function* () { + const scan = scanPromptArgs(["prompt", REQUEST, "--props-loud", "true"]); + expect(scan.error).toBe(undefined); + expect(scan.occurrences).toEqual([{ option: "--props-loud", provisional: "true" }]); + + // A candidate that declares `loud` a value option accepts it. + expect(strayPropertyValue(scan.occurrences, bindingsFor({ loud: { type: "string" } }))).toBe( + undefined, + ); + + // One that declares it a switch does not: `true` is then a positional. + const stray = strayPropertyValue(scan.occurrences, bindingsFor({ loud: { type: "boolean" } })); + expect(stray).toContain("unrecognized argument for xmd prompt: true"); + expect(stray).toContain("--props-loud=true"); + }); + + it("P6: a frozen option's shape is what a later candidate may not change", function* () { + const scalar = bindingsFor({ name: { type: "string" } }); + const boolean = bindingsFor({ name: { type: "boolean" } }); + const array = bindingsFor({ name: { type: "array", items: { type: "string" } } }); + const absent = bindingsFor({ other: { type: "string" } }); + + expect(signatureOf(scalar[0])).toEqual({ boolean: false, array: false }); + const stable = frozen({ "--props-name": signatureOf(scalar[0]) }); + + // Unchanged: nothing is refused, so the same sources resolve again. + expect(signatureFailure(stable, scalar)).toBe(undefined); + + expect(signatureFailure(stable, absent)).toContain("declares no such property"); + expect(signatureFailure(stable, boolean)).toContain("single-value option"); + expect(signatureFailure(stable, boolean)).toContain("bare switch"); + expect(signatureFailure(stable, array)).toContain("repeated value option"); + + // An option nobody supplied is never frozen, so a candidate may add, + // remove or reshape it freely. + expect(signatureFailure(new Map(), absent)).toBe(undefined); + }); +}); diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts new file mode 100644 index 000000000..14bbda5f1 --- /dev/null +++ b/packages/cli/tests/prompt-cli.test.ts @@ -0,0 +1,458 @@ +/** + * Tier PR — the `xmd prompt` command lifecycle + * (specs/prompt-command-spec.md). + * + * Rows P2, P4 and P13–P16: what the command does to the filesystem, the + * journal, the process status and the document it ends in. The approved + * document runs through the production executor, so what these prove about + * execution is what `xmd run` does with a supplied root. + * + * The grammar rows shell out, because exit status and help text are what an + * operator sees. Everything with a phase to observe runs in process, where a + * refusal is proven by the tripwires that stayed at zero. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { runCli } from "@executablemd/test-support/launch"; +import { readTextFile, writeTextFile } from "@effectionx/fs"; +import { stat } from "@executablemd/runtime"; +import { Ok, scoped, spawn } from "effection"; +import type { Operation, Result } from "effection"; +import { join } from "node:path"; +import { readdir } from "node:fs/promises"; +import { until } from "effection"; + +import { promptExecutor } from "../src/cli.ts"; +import { runPrompt } from "../src/prompt.ts"; +import type { PromptCommand, PromptExecution } from "../src/prompt.ts"; +import { scanPromptArgs } from "../src/prompt-args.ts"; +import { + AGENT, + createPromptHarness, + useEnvironment, + useWorkingDirectory, +} from "./support/prompt-harness.ts"; +import type { PromptHarness } from "./support/prompt-harness.ts"; + +const REQUEST = "write a greeting"; + +/** A document that declares props and writes what it resolved. */ +const GREETER = [ + "---", + "props:", + " type: object", + " properties:", + " name: { type: string }", + " loud: { type: boolean, default: false }", + " count: { type: number }", + " required: [name]", + " additionalProperties: false", + "---", + "", + 'name={props.name} loud={props.loud} count={props.count}', + "", +].join("\n"); + +/** + * A document that validates and then fails at run time. + * + * Validation runs no command, so nothing before execution can know this one + * exits nonzero — which is the point: a runtime failure is not a candidate + * defect and never returns the command to generation or review. + */ +const FAILS_AT_RUN = ["```bash exec", "exit 3", "```", ""].join("\n"); + +const PLAIN = "Nothing but prose.\n"; + +function command(dir: string, args: string[], save?: string): PromptCommand { + const argv = ["prompt", ...args]; + return { + argv, + scan: scanPromptArgs(argv), + include: [dir], + ...(save === undefined ? {} : { save }), + agent: { + agentProvider: "acpx", + defaultAgent: AGENT, + approveAll: false, + approveReads: false, + denyAll: true, + }, + }; +} + +/** The production executor, configured the way the dispatch configures it. */ +function executor( + dir: string, + journal?: string, +): (approved: PromptExecution) => Operation> { + return promptExecutor( + { + include: [dir], + verbose: false, + journal, + raw: true, + secretDetection: true, + agentProvider: "acpx", + defaultAgent: AGENT, + approveAll: false, + approveReads: false, + denyAll: true, + }, + undefined, + function* () {}, + ); +} + +function* exists(path: string): Operation { + return (yield* stat(path)).exists; +} + +/** Every phase after the refusal, at zero. */ +function untouched(harness: PromptHarness): Record { + return { + catalogs: harness.catalogCalls.length, + runtimes: harness.fake.created.length, + started: harness.fake.started, + turns: harness.fake.prompts.length, + reviews: harness.reviews.length, + executions: harness.executions.length, + }; +} + +const NOTHING = { + catalogs: 0, + runtimes: 0, + started: false, + turns: 0, + reviews: 0, + executions: 0, +}; + +describe( + "Tier PR — the xmd prompt command", + { sanitizeOps: false, sanitizeResources: false }, + () => { + it("P2: a misplaced individual option refuses before any phase begins", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + const code = yield* runPrompt( + command(dir, ["--props-name", "Ada", REQUEST], "out.md"), + harness.deps, + ); + + expect(code).toBe(1); + expect(untouched(harness)).toEqual(NOTHING); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + }); + + // The agent configuration is settled before the catalog is built, so an + // incompatible pair of permission flags costs no inspection at all. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + const request = command(dir, [REQUEST], "out.md"); + const code = yield* runPrompt( + { ...request, agent: { ...request.agent, approveAll: true, denyAll: true } }, + harness.deps, + ); + + expect(code).toBe(1); + expect(untouched(harness)).toEqual(NOTHING); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + }); + }); + + it("P2: help needs no request and touches nothing", function* () { + yield* useWorkingDirectory(function* (dir) { + const { code, stdout, stderr } = yield* runCli( + [ + "prompt", + "--help", + "--save", + "out.md", + "--journal", + "trace.jsonl", + "--default-agent", + "xmd-nonexistent-agent", + ], + { cwd: dir }, + ).join(); + + expect(code).toBe(0); + expect(stdout).toContain("Usage: xmd prompt [OPTIONS] [request]"); + expect(stdout).toContain("Exactly one request is required"); + expect(stdout).toContain("--props "); + expect(stdout).toContain("XMD_PROPS"); + expect(stdout).toContain("--save "); + expect(stderr).not.toContain("unavailable"); + // No catalog was rendered, and neither file the options named was made. + expect(stdout).not.toContain("## Built-in components"); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* exists(join(dir, "trace.jsonl"))).toBe(false); + }); + + const program = yield* runCli(["--help"]).expect(); + expect(program.stdout).toMatch(/^\s+prompt\s/m); + }); + + it("P4: individual, aggregate and environment sources resolve and reach the run", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "approve" }); + + yield* useEnvironment({ XMD_PROPS: '{"name":"FromEnv","count":7}' }); + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-name", "Ada", "--props-loud"]), + harness.deps, + ); + + expect(code).toBe(0); + // Individual CLI beats the aggregate environment; the switch is true; the + // property only the aggregate supplied is still there. + expect(yield* readTextFile(join(dir, "greeting.txt"))).toBe("name=Ada loud=true count=7"); + }); + }); + + it("P13: every refusal is nonzero and leaves no save, journal or run", function* () { + const journalName = "trace.jsonl"; + + // Abort at review. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, join(dir, journalName)); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "abort" }); + + const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + + expect(code).toBe(1); + expect(harness.executions).toHaveLength(0); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* exists(join(dir, journalName))).toBe(false); + }); + + // A generation failure. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, join(dir, journalName)); + harness.fake.script({ reply: PLAIN, stopReason: "refusal" }); + + const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + + expect(code).toBe(1); + expect(harness.reviews).toHaveLength(0); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* exists(join(dir, journalName))).toBe(false); + }); + + // A terminal property-source failure. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, join(dir, journalName)); + harness.fake.script({ reply: PLAIN }); + + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-absent", "x"], "out.md"), + harness.deps, + ); + + expect(code).toBe(1); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* exists(join(dir, journalName))).toBe(false); + }); + + // An approved run's journal holds the document's events and no authorship. + yield* useWorkingDirectory(function* (dir) { + const journal = join(dir, journalName); + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, journal); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + const trace = yield* readTextFile(journal); + expect(trace).toContain("__root__"); + expect(trace).toContain(PLAIN.trim()); + // Nothing about generation is in it: not the request, not a turn, not a + // review, not a repair. + expect(trace).not.toContain(REQUEST); + expect(trace).not.toContain("agent_prompt"); + expect(trace).not.toContain("elicit"); + }); + }); + + it("P14: the approved bytes are created exclusively, before the run", function* () { + // Created before execution, and byte for byte. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + const run = executor(dir); + const seen: boolean[] = []; + harness.deps.execute = function* (approved) { + seen.push(yield* exists(join(dir, "out.md"))); + return yield* run(approved); + }; + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-name", "Ada"], "out.md"), + harness.deps, + ); + + expect(code).toBe(0); + // The saved file already existed when the document started. + expect(seen).toEqual([true]); + // Source only: no diagnostics, no decision, no wrapper. + expect(yield* readTextFile(join(dir, "out.md"))).toBe(GREETER); + }); + + // An existing path is left exactly as it is, and stops the run. + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "out.md"), "keep me\n"); + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + + expect(code).toBe(1); + expect(yield* readTextFile(join(dir, "out.md"))).toBe("keep me\n"); + expect(harness.executions).toHaveLength(0); + }); + + // Without the option, no generated source file is created at all. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); + + expect(code).toBe(0); + // Only what the document itself wrote. + expect((yield* until(readdir(dir))).sort()).toEqual(["greeting.txt"]); + }); + }); + + it("P15: the approved source runs as an ordinary document under ", function* () { + yield* useWorkingDirectory(function* (dir) { + const journal = join(dir, "trace.jsonl"); + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, journal); + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-name", "Ada"], "out.md"), + harness.deps, + ); + expect(code).toBe(0); + + // The identity the run reports is the deliberate one. + expect(yield* readTextFile(journal)).toContain(""); + // Relative filesystem operations resolved the contextual cwd, not the + // identity, so the document's own write landed beside the save. + expect(yield* exists(join(dir, "greeting.txt"))).toBe(true); + expect(yield* exists(join(dir, "out.md"))).toBe(true); + }); + + // A runtime failure is an ordinary run failure: nonzero, with the save the + // caller asked for still on disk to hand-edit. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: FAILS_AT_RUN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + + expect(code).toBe(1); + expect(yield* readTextFile(join(dir, "out.md"))).toBe(FAILS_AT_RUN); + // The failure did not send the run back to generation or review. + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.reviews).toHaveLength(1); + }); + }); + + it("P16: the deadline encloses every phase, and teardown gates what follows", function* () { + // Expiry is cancellation, so the proof is what cancelling the command does: + // the turn in flight is cancelled, the provider is dismantled, and no later + // phase begins. The barrier is what makes this a gate rather than a race — + // the turn it interrupts is known to be running. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: PLAIN, manual: true }); + + yield* scoped(function* () { + const running = yield* spawn(() => + runPrompt(command(dir, [REQUEST], "out.md"), harness.deps), + ); + yield* harness.fake.startedTurns(1); + yield* running.halt(); + }); + + expect(harness.fake.cancels).toBeGreaterThanOrEqual(1); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + }); + + // A generator teardown failure prevents the save and the run. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.closeFailure = new Error("the generator would not close"); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + + expect(code).toBe(1); + expect(harness.executions).toHaveLength(0); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + }); + + // Nothing bounds a generation turn: the exec and fetch defaults belong to + // the document, and the run deadline is the enclosing timebox above. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + // deno-lint-ignore require-yield + harness.deps.execute = function* () { + return Ok(undefined); + }; + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + expect(harness.fake.turns[0]?.timeoutMs).toBe(undefined); + }); + }); + + it("P16: the deadline is the whole prompt command's, as it is a run's", function* () { + // Read for prompt exactly as for run: a value that is not a duration fails + // the invocation before it prepares anything. + for (const flag of ["--timeout", "--timeout-exec", "--timeout-fetch"]) { + const { code, stderr } = yield* runCli(["prompt", REQUEST, `${flag}=abc`]).join(); + expect(code).toBe(1); + expect(stderr).toContain(flag); + } + + // And it bounds the command rather than only its final document: the + // deadline expires while the catalog is still being built, long before any + // document exists to bound. + const expired = yield* runCli(["prompt", REQUEST, "--timeout=1ms"]).join(); + expect(expired.code).toBe(1); + expect(expired.stderr).toContain("exceeded its --timeout of 1ms and was cancelled"); + }); + }, +); diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts new file mode 100644 index 000000000..587d700c7 --- /dev/null +++ b/packages/cli/tests/prompt.test.ts @@ -0,0 +1,486 @@ +/** + * Tier PR — `xmd prompt` generation, validation and review + * (specs/prompt-command-spec.md). + * + * Rows P5–P12. The ACPX runtime is the scriptable fake, the review provider is a + * scripted `Elicitation` handler, the executor records what it was handed, and + * the contextual working directory is a temporary one. Nothing here starts an + * agent, opens a browser or reaches a network. + * + * Every refusal is proven by the phases that stayed at zero — turns not sent, + * reviews not asked, executions not handed anything — rather than by output + * nobody produced. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation } from "effection"; +import { writeTextFile } from "@effectionx/fs"; +import { join } from "node:path"; + +import { runPrompt } from "../src/prompt.ts"; +import type { PromptCommand } from "../src/prompt.ts"; +import { scanPromptArgs } from "../src/prompt-args.ts"; +import { useGeneratorAgent } from "../src/agent-stack.ts"; +import { AGENT, createPromptHarness, useWorkingDirectory } from "./support/prompt-harness.ts"; +import type { PromptHarness } from "./support/prompt-harness.ts"; +import { createFakeAcp, makeRegistry, makeStore } from "./support/fake-acp.ts"; + +const REQUEST = "write a greeting"; + +/** A document that validates and runs. */ +const VALID = "Hello from the agent.\n"; + +/** A document that resolves no such component. */ +const UNRESOLVED = "\n"; + +/** A root whose own source cannot be read: the frontmatter never closes. */ +const BROKEN_SOURCE = ["---", "props: [", "---", "", "hi", ""].join("\n"); + +/** A root whose two declared properties generate one option. */ +const COLLIDING = [ + "---", + "props:", + " type: object", + " properties:", + " firstName: { type: string }", + " first_name: { type: string }", + "---", + "", + "hi", + "", +].join("\n"); + +/** A root that declares one required scalar property. */ +const REQUIRES_NAME = [ + "---", + "props:", + " type: object", + " properties:", + " name: { type: string }", + " required: [name]", + " additionalProperties: false", + "---", + "", + "Hello, {props.name}!", + "", +].join("\n"); + +/** The same document with `name` declared a switch instead. */ +const NAME_IS_BOOLEAN = [ + "---", + "props:", + " type: object", + " properties:", + " name: { type: boolean }", + " additionalProperties: false", + "---", + "", + "Hello, {props.name}!", + "", +].join("\n"); + +function command(dir: string, args: string[], save?: string): PromptCommand { + const argv = ["prompt", ...args]; + return { + argv, + scan: scanPromptArgs(argv), + include: [dir], + ...(save === undefined ? {} : { save }), + agent: { + agentProvider: "acpx", + defaultAgent: AGENT, + approveAll: false, + approveReads: false, + denyAll: true, + }, + }; +} + +/** Every session key the fake was asked to establish, deduplicated in order. */ +function sessions(harness: PromptHarness): string[] { + return [...new Set(harness.fake.ensured.map((input) => input.sessionKey))]; +} + +/** The decisions one review request offered. */ +function decisions(request: { schema: Record }): unknown { + const properties = request.schema.properties; + if (typeof properties !== "object" || properties === null) { + return undefined; + } + const decision = (properties as Record).decision; + if (typeof decision !== "object" || decision === null) { + return undefined; + } + return (decision as Record).enum; +} + +describe( + "Tier PR — xmd prompt authorship", + { sanitizeOps: false, sanitizeResources: false }, + () => { + it("P5: candidate defects earn a repair turn; caller defects terminate", function* () { + // A defect the agent authored: the root's own frontmatter. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: BROKEN_SOURCE }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(0); + expect(harness.fake.prompts).toHaveLength(2); + expect(harness.fake.prompts[0]).toBe(REQUEST); + expect(harness.fake.prompts[1]).toContain("complete replacement document"); + expect(harness.fake.prompts[1]).toContain("source-invalid"); + // The whole versioned value, as data. + expect(harness.fake.prompts[1]).toContain('"version": 1'); + expect(harness.executions).toHaveLength(1); + }); + + // A defect the agent authored: two properties generating one option. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: COLLIDING }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(0); + expect(harness.fake.prompts).toHaveLength(2); + expect(harness.fake.prompts[1]).toContain("generated-binding-collision"); + expect(harness.fake.prompts[1]).toContain("--props-first-name"); + // Carried as this command's own finding, not as a core diagnostic code. + expect(harness.fake.prompts[1]).not.toContain("DocumentValidationCode"); + }); + + // A defect the caller wrote: an option the candidate never declares. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-nothing", "here"]), + harness.deps, + ); + + expect(code).toBe(1); + // One turn, no repair: the agent cannot fix a command line. + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + + // A defect the caller wrote: aggregate JSON that is not JSON. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + + const code = yield* runPrompt(command(dir, [REQUEST, "--props", "{oops"]), harness.deps); + + expect(code).toBe(1); + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + + // A defect the caller wrote: a value this candidate's schema rejects. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: NAME_IS_BOOLEAN }); + + const code = yield* runPrompt( + command(dir, [REQUEST, "--props-name=not-a-boolean"]), + harness.deps, + ); + + expect(code).toBe(1); + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("P6: a revision may not change what the command line means", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + // The first draft binds `--props-name` as a single-value option. + harness.fake.script({ reply: REQUIRES_NAME }); + harness.script({ decision: "revise", feedback: "make it shout" }); + // The revision declares the same name as a switch. + harness.fake.script({ reply: NAME_IS_BOOLEAN }); + + const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); + + expect(code).toBe(1); + // The change was caught before the second candidate was presented. + expect(harness.reviews).toHaveLength(1); + expect(harness.executions).toHaveLength(0); + expect(harness.fake.prompts).toHaveLength(2); + }); + + // An unchanged signature re-resolves from the same unchanged sources. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: REQUIRES_NAME }); + harness.script({ decision: "revise", feedback: "greet louder" }); + harness.fake.script({ reply: `${REQUIRES_NAME}\nLOUDER\n` }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); + + expect(code).toBe(0); + expect(harness.executions[0]?.props).toEqual({ name: "Ada" }); + }); + }); + + it("P7: one run-profile catalog becomes the fresh session's system prompt", function* () { + yield* useWorkingDirectory(function* (dir) { + // A repository TypeScript component, so the catalog has to state the one + // thing it honestly cannot know without importing the module. + yield* writeTextFile(join(dir, "Widget.ts"), "export default function Widget() {}\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + // Exactly one catalog, built with the invocation's own includes. + expect(harness.catalogCalls).toEqual([[dir]]); + + const instructions = harness.fake.ensured[0]?.sessionOptions?.systemPrompt; + expect(typeof instructions).toBe("string"); + const stated = typeof instructions === "string" ? instructions : ""; + expect(stated).toContain("Every answer is replacement document source and nothing else"); + expect(stated).toContain("## Built-in structural syntax"); + expect(stated).toContain("## Built-in components"); + expect(stated).toContain("### ``"); + expect(stated).toContain("### ``"); + // The renderer's origin-only wording, unchanged. + expect(stated).toContain("This component is a repository TypeScript module."); + // The instructions are the session's, not the user turn's. + expect(harness.fake.prompts[0]).toBe(REQUEST); + expect(harness.fake.prompts[0]).not.toContain("Built-in components"); + }); + }); + + it("P8: one fresh session per invocation, contacted only by a subscribed turn", function* () { + // Placement contacts no backend: the runtime is not even created until the + // first turn is subscribed. + yield* useWorkingDirectory(function* () { + const fake = createFakeAcp(); + yield* scoped(function* (): Operation { + const provider = yield* useGeneratorAgent( + { + provider: "acpx", + defaultAgent: AGENT, + permissionMode: "deny-all", + }, + "instructions", + { + createRuntime: fake.create, + sessionStore: makeStore(), + agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), + }, + ); + const session = yield* provider.session("xmd-prompt:placement"); + expect(session.sessionKey.startsWith("xmd:v1:")).toBe(true); + expect(fake.ensured).toHaveLength(0); + expect(fake.started).toBe(false); + + fake.script({ reply: VALID }); + const subscription = yield* provider.promptStream("hello", { session }); + let next = yield* subscription.next(); + while (!next.done) { + next = yield* subscription.next(); + } + expect(fake.ensured).toHaveLength(1); + expect(fake.ensured[0]?.sessionKey).toBe(session.sessionKey); + }); + }); + + // Initial, repair and revision turns all land in one conversation. + const keys: string[] = []; + for (const _invocation of [0, 1]) { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.script({ decision: "revise", feedback: "try again" }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + // The base candidate, one repair, an exhausting pair, a revision. + harness.fake.script({ reply: VALID }); + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + expect(sessions(harness)).toHaveLength(1); + keys.push(sessions(harness)[0]); + }); + } + // A second invocation places a different session: nothing is inherited. + expect(keys[0]).not.toBe(keys[1]); + }); + + it("P9: only a completed terminal produces a candidate", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + // A turn that streams a perfectly good document and then fails. + harness.fake.script({ reply: VALID, stopReason: "max_tokens" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + expect(harness.fake.prompts).toHaveLength(1); + // The partial text was discarded: nobody was asked and nothing ran. + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("P10: the base candidate plus exactly three repairs", function* () { + // Three repairs are available. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(0); + expect(harness.fake.prompts).toHaveLength(4); + expect(harness.reviews).toHaveLength(1); + expect(decisions(harness.reviews[0])).toEqual(["approve", "revise", "abort"]); + }); + + // A fourth invalid candidate is exhausted, and cannot be approved. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script({ decision: "abort" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + expect(harness.fake.prompts).toHaveLength(4); + expect(harness.reviews).toHaveLength(1); + expect(decisions(harness.reviews[0])).toEqual(["revise", "abort"]); + expect(harness.reviews[0].message).toContain("could not produce a document that validates"); + expect(harness.reviews[0].message).toContain("component-unresolved"); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("P11: review offers what the candidate allows, and a revision starts over", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + // Four invalid drafts: the exhausted review. + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script({ decision: "revise", feedback: "use plain text" }); + // The revision's own budget: four more before the next review. + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script({ decision: "abort" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + // Four, then the revision's own draft and three more repairs: the budget + // reset rather than carrying the exhausted one forward. + expect(harness.fake.prompts).toHaveLength(8); + expect(harness.fake.prompts[4]).toContain("use plain text"); + expect(harness.fake.prompts[4]).toContain("complete replacement document"); + expect(harness.reviews).toHaveLength(2); + expect(decisions(harness.reviews[0])).toEqual(["revise", "abort"]); + expect(decisions(harness.reviews[1])).toEqual(["revise", "abort"]); + // The revision stayed in the same conversation. + expect(sessions(harness)).toHaveLength(1); + + // Non-empty feedback is the contract, and it is the schema's. + const rule = harness.reviews[0].schema.then; + expect(rule).toEqual({ + properties: { feedback: { type: "string", minLength: 1 } }, + required: ["feedback"], + }); + expect(harness.reviews[0].schema.if).toEqual({ + properties: { decision: { const: "revise" } }, + required: ["decision"], + }); + expect(harness.reviews[0].schema.additionalProperties).toBe(false); + }); + + // A review answer the schema rejects fails the command the way an abort + // does: nothing is saved and nothing runs. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "revise", raw: { decision: "revise", feedback: "" } }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("P12: the exact agent bytes are shown, approved and executed", function* () { + yield* useWorkingDirectory(function* (dir) { + // A document that holds a fence of its own, and a run of five backticks. + const fenced = [ + "Here is a block:", + "", + "```bash", + "echo hi", + "```", + "", + "and `````five````` backticks.", + "", + ].join("\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: fenced }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + const message = harness.reviews[0].message; + // The fence is longer than every run inside the candidate, so nothing in + // the source can close it. + expect(message).toContain("``````md\n"); + expect(message).toContain(`\n${fenced}\n\`\`\`\`\`\``); + // Nothing was stripped: what runs is the close value, byte for byte. + expect(harness.executions[0]?.root.source).toBe(fenced); + expect(harness.executions[0]?.root.path).toBe(""); + }); + + // A reply wrapped in a fence is shown as it is, not unwrapped. + yield* useWorkingDirectory(function* (dir) { + const wrapped = ["```md", "Hello.", "```", ""].join("\n"); + const harness = createPromptHarness(); + harness.fake.script({ reply: wrapped }); + // Wrapped source is not a document, so it earns repairs and then a review. + for (const _draft of [0, 1, 2]) { + harness.fake.script({ reply: wrapped }); + } + harness.script({ decision: "abort" }); + + yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(harness.reviews[0].message).toContain(wrapped); + }); + }); + }, +); diff --git a/packages/cli/tests/support/fake-acp.ts b/packages/cli/tests/support/fake-acp.ts index ee3e5dd24..7dcd48db8 100644 --- a/packages/cli/tests/support/fake-acp.ts +++ b/packages/cli/tests/support/fake-acp.ts @@ -82,6 +82,14 @@ export interface ScriptedTurn { * the turns that had already committed. */ readonly manual?: boolean; + /** + * The stop reason this turn settles with. + * + * `end_turn` unless a case says otherwise. Anything else is a turn the + * provider reports as failed while keeping whatever text it streamed, which is + * how a partial reply is driven without an adapter that misbehaves on purpose. + */ + readonly stopReason?: string; /** * The App Server turn this reply is, as an adapter that names its turns says * it: on this exact response's own `_meta`. @@ -100,10 +108,24 @@ export interface FakeAcp { readonly ensured: AcpRuntimeEnsureInput[]; /** Every prompt text this provider sent, in order. */ readonly prompts: string[]; + /** Every turn this provider started, whole, in order. */ + readonly turns: AcpRuntimeTurnInput[]; /** Every permission decision this fake was answered with. */ readonly decisions: string[]; /** Whether an agent process would have been started at all. */ readonly started: boolean; + /** Every handle this runtime was told to close, by the reason it was given. */ + readonly closes: string[]; + /** How many turns were cancelled rather than allowed to settle. */ + readonly cancels: number; + /** + * Fail every close. + * + * A provider whose teardown fails is the case a host has to survive without + * acting on what the failed scope produced, and only a close that rejects + * makes the provider report one. + */ + closeFailure?: Error; script(turn: ScriptedTurn): void; /** * Settles once `count` turns have been started. @@ -170,11 +192,14 @@ export function createFakeAcp(): FakeAcp { const created: AcpRuntimeOptions[] = []; const ensured: AcpRuntimeEnsureInput[] = []; const prompts: string[] = []; + const turns: AcpRuntimeTurnInput[] = []; const decisions: string[] = []; + const closes: string[] = []; const waiting: Array<{ count: number; settle: () => void }> = []; const reading: Array<{ count: number; settle: () => void }> = []; let consumed = 0; let started = false; + let cancels = 0; function announceTurn(): void { for (const barrier of [...waiting]) { @@ -199,10 +224,15 @@ export function createFakeAcp(): FakeAcp { created, ensured, prompts, + turns, decisions, + closes, get started(): boolean { return started; }, + get cancels(): number { + return cancels; + }, script(turn) { scripted.push(turn); }, @@ -285,6 +315,7 @@ export function createFakeAcp(): FakeAcp { }, startTurn(input: AcpRuntimeTurnInput): AcpRuntimeTurn { prompts.push(input.text); + turns.push(input); announceTurn(); const turn = scripted.shift() ?? { reply: "" }; const settled = withResolvers(); @@ -348,7 +379,7 @@ export function createFakeAcp(): FakeAcp { void asked.then(() => { settled.resolve({ status: "completed", - stopReason: "end_turn", + stopReason: turn.stopReason ?? "end_turn", ...(turn.turnId === undefined ? {} : { _meta: { codex: { turnId: turn.turnId } } }), }); }); @@ -387,6 +418,7 @@ export function createFakeAcp(): FakeAcp { }, result: run(() => settled.operation), cancel(): Promise { + cancels += 1; settled.resolve({ status: "cancelled" }); released.resolve(); return Promise.resolve(); @@ -402,8 +434,9 @@ export function createFakeAcp(): FakeAcp { cancel(): Promise { return Promise.resolve(); }, - close(): Promise { - return Promise.resolve(); + close(input?: { reason?: string }): Promise { + closes.push(input?.reason ?? ""); + return fake.closeFailure ? Promise.reject(fake.closeFailure) : Promise.resolve(); }, }; }, diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts new file mode 100644 index 000000000..94533e4be --- /dev/null +++ b/packages/cli/tests/support/prompt-harness.ts @@ -0,0 +1,160 @@ +/** + * The deterministic seams `xmd prompt` is proven against. + * + * Every phase the command owns is driven in process: the ACPX runtime is the + * scriptable fake, the review provider is a scripted `Elicitation` handler, the + * catalog and the execution are recorded, and the contextual working directory + * is a temporary one. No live agent, browser, or network belongs in this + * evidence. + * + * Each recorder is a tripwire as well as a fake. A refusal is proven by the + * phases that stayed at zero, never by empty output — a command that printed + * nothing and still opened a session would pass such a check. + */ + +import { Elicitation } from "@executablemd/core"; +import type { ElicitationRequest, SyntaxCatalog } from "@executablemd/core"; +import { Ok } from "effection"; +import type { Operation, Result } from "effection"; +import { ensure, scoped } from "effection"; +import { ensureDir, rm } from "@effectionx/fs"; +import { randomUUID } from "node:crypto"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { API, useHostFiles } from "@executablemd/runtime"; +import { syntaxCatalog } from "../../src/syntax.ts"; +import type { PromptDependencies, PromptExecution } from "../../src/prompt.ts"; +import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; +import type { FakeAcp } from "./fake-acp.ts"; + +/** The agent every prompt case drives, and the command it resolves to. */ +export const AGENT = "scripted-agent"; + +/** One review answer, scripted. */ +export interface ScriptedReview { + decision: "approve" | "revise" | "abort"; + feedback?: string; + /** Answer with this instead, to drive a response the schema rejects. */ + raw?: unknown; +} + +export interface PromptHarness { + fake: FakeAcp; + /** Every catalog request, by the includes it was made with. */ + catalogCalls: string[][]; + /** Every review request a provider was asked, in order. */ + reviews: ElicitationRequest[]; + /** Every approved execution that reached the executor. */ + executions: PromptExecution[]; + /** What the executor answers with, once each, then `Ok`. */ + executionResults: Result[]; + /** Review answers, taken in order. Running out is a test defect, not a case. */ + script(review: ScriptedReview): void; + /** The dependencies `runPrompt` is driven with. */ + deps: PromptDependencies; +} + +export function createPromptHarness(options?: { + /** Replace the catalog entirely, for a case about catalog failure. */ + catalog?: (includes: readonly string[]) => Operation; +}): PromptHarness { + const fake = createFakeAcp(); + const catalogCalls: string[][] = []; + const reviews: ElicitationRequest[] = []; + const executions: PromptExecution[] = []; + const executionResults: Result[] = []; + const answers: ScriptedReview[] = []; + + const harness: PromptHarness = { + fake, + catalogCalls, + reviews, + executions, + executionResults, + script(review) { + answers.push(review); + }, + deps: { + acp: { + createRuntime: fake.create, + sessionStore: makeStore(), + agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), + }, + *catalog(includes) { + catalogCalls.push([...includes]); + return yield* (options?.catalog ?? syntaxCatalog)(includes); + }, + *installElicitation() { + yield* Elicitation.around( + { + // deno-lint-ignore require-yield + *elicit([request], _next) { + reviews.push(request); + const answer = answers.shift(); + if (answer === undefined) { + throw new Error("the case scripted no answer for this review"); + } + if (answer.raw !== undefined) { + return answer.raw; + } + return { + decision: answer.decision, + ...(answer.feedback === undefined ? {} : { feedback: answer.feedback }), + }; + }, + }, + { at: "min" }, + ); + }, + // deno-lint-ignore require-yield + *execute(approved) { + executions.push(approved); + return executionResults.shift() ?? Ok(undefined); + }, + }, + }; + return harness; +} + +/** + * A temporary directory that is also the contextual working directory. + * + * Both, because the two answer different questions: session placement and + * `--save` resolve the contextual one, while a real file has to live somewhere. + */ +export function* useWorkingDirectory(body: (dir: string) => Operation): Operation { + const dir = join(tmpdir(), `xmd-prompt-${randomUUID()}`); + yield* ensureDir(dir); + return yield* scoped(function* () { + yield* ensure(() => rm(dir, { recursive: true, force: true })); + yield* API.Env.around({ + // deno-lint-ignore require-yield + *cwd() { + return dir; + }, + }); + // The provider `API.Files` has no host default for, installed exactly where + // the runtime entrypoint installs it: a document that reaches the + // filesystem must reach the caller's, or fail. + yield* useHostFiles(); + return yield* body(dir); + }); +} + +/** + * Environment values this scope answers with, delegating every other name. + * + * Delegation matters: the agent stack reads `DEFAULT_AGENT_NAME` through the + * same Api, and answering `undefined` for everything would make a case about + * `XMD_PROPS` quietly also be a case about the default agent. + */ +export function* useEnvironment(values: Record): Operation { + yield* API.Env.around({ + *env([name], next) { + if (Object.hasOwn(values, name)) { + return values[name]; + } + return yield* next(name); + }, + }); +} diff --git a/packages/cli/tests/syntax-cli.test.ts b/packages/cli/tests/syntax-cli.test.ts index f57f5c079..10da2db14 100644 --- a/packages/cli/tests/syntax-cli.test.ts +++ b/packages/cli/tests/syntax-cli.test.ts @@ -384,10 +384,13 @@ describe("Tier SX — the command line", { sanitizeOps: false, sanitizeResources ); }); - it("SX11: adds no prompt command", function* () { + it("SX11: the catalog is inspection, and prompt is the command that writes with it", function* () { const { stdout } = yield* runCli(["--help"]).expect(); expect(stdout).toContain("syntax"); - expect(stdout).not.toMatch(/^\s+prompt\s/m); + // `xmd prompt` is the other reader of this catalog: it feeds the same + // structured value to a generator rather than spawning this command and + // parsing what it printed. + expect(stdout).toMatch(/^\s+prompt\s/m); }); }); diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 2d3fa7179..d6332fa6f 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -477,9 +477,9 @@ own session store and one empty working directory per session, both disposable. ## Command-line configuration -`xmd run` configures the agent stack; the options are exclusive to it, and -`xmd test` rejects them, driving agents through the deterministic test-agent -stack instead. +`xmd run` and `xmd prompt` configure the agent stack; the options belong to +those two commands, and `xmd test` rejects them, driving agents through the +deterministic test-agent stack instead. That division also applies to a nested run profile. The outer `xmd test` invocation supplies no live Agent configuration to a child and no contextual @@ -498,7 +498,7 @@ Agent provider from the outer test. The permission options are mutually exclusive. -`xmd run` also takes the three timeout options, which are exclusive to it in the +Both commands also take the three timeout options, which belong to them in the same way: | Option | Establishes | @@ -531,6 +531,39 @@ The default agent resolves in order, each entry overriding the ones above it: The installed provider belongs to the run's `DocumentExecution` scope and closes during its teardown. +### The `xmd prompt` generator + +`xmd prompt` resolves that configuration once and uses it twice: for the session +that writes the document, and for the document that runs. The provider name, the +default agent with its `DEFAULT_AGENT_NAME` precedence, the permission mode and +the host's own machine-session assembly are settled before any catalog is built +or any document executes, so an unknown provider or an incompatible pair of +permission flags fails first. + +Generation takes the provider's operations directly — `useAcpxProvider`, without +the Agent install. It needs turns, not authority: no document is installed +around it, no root provider is registered, and no foreground launcher exists for +a `` to reach. The one Agent operation the provider performs for +itself, resolving the agent a session and a turn belong to, is answered by the +generator handle; every other Agent operation keeps its base behavior and +refuses. The selected permission policy is installed around those turns, so a +native permission request during generation is answered by the same +`--approve-all`, `--approve-reads` or `--deny-all` choice a run uses. + +Only the generator provider receives `newSessionOptions.systemPrompt`. It +carries the invocation's structured syntax catalog and the fixed statement of +what the session writes ([`xmd prompt`](./prompt-command-spec.md)). ACPX applies +those options when it creates a session and ignores them when it reuses a +record, and the command places a session name unique to the invocation, so each +invocation is a fresh conversation and the instructions are the ones it was +created under. + +The generator's scope closes after approval and before the save and the +execution. A teardown failure fails the command and neither later phase happens. +The executed program is an ordinary `xmd run` document with its own root +provider and its own lifetime: it inherits neither the generator session nor its +system prompt. + ### Availability Installing a provider starts nothing. The **first** Agent API use validates the diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index fe2cbb79b..361c22962 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -8237,6 +8237,18 @@ identity, and no temporary file is created: the text is captured inside the durable root import, so the journal holds it and a replay restores it without reading anything. +Text a host generated reports `` on the same terms. `xmd prompt` asks an +agent for a complete root, validates it, has a person approve it, and executes +`retainedSource("", source)` — so the identity says where the bytes came +from, and a source position reads `(:5:1)`. It is a deliberate identity +rather than a path nobody could read back, and it is the only thing that +differs: the approved document runs through the ordinary supplied-source path, +renders and returns exactly as any other root, resolves the contextual working +directory for every relative operation, and creates its journal only when +execution starts. Generation, repair and approval happen before that execution +exists, so nothing about them is journaled or replayed. The command's complete +contract is [`xmd prompt`](./prompt-command-spec.md). + `inspectDocument(root)` loads and validates the root definition and returns what it declares — without executing the document or creating a journal: @@ -10005,7 +10017,7 @@ itself, so an execution starting anywhere fails the row. Defined in §5.3. | SX4–SX6 | Renderers take a value | Both formats render from a supplied catalog with the filesystem refusing every call, twice with identical bytes, under the fixed category headings; every table cell is escaped, a prop name holding a pipe included | | SX7/SX8 | Includes | Repeated values select in caller order and replace the defaults; absent, the defaults apply | | SX9 | Failure | An unusable include exits 1, reports on stderr and prints no catalog | -| SX10/SX11 | Formats | Markdown by default, version-1 JSON with `--json`, and no `prompt` command | +| SX10/SX11 | Formats | Markdown by default, version-1 JSON with `--json`; the catalog is inspection, and `xmd prompt` is the command that writes with the same structured value | | SX12 | A package tree | Bare `xmd syntax` succeeds with the default includes in a repository whose `node_modules` holds directory links | ### Tier SM — `xmd syntax` end to end @@ -10019,6 +10031,24 @@ One checked-in Markdown suite runs the real command against a fixture directory. | SM6/SM7 | Nothing runs | A `.ts` fixture that throws at top level is origin-only and the command still succeeds; a fixture whose body writes a file leaves no file | | SM8–SM11 | Discovery | Direct, dotted and index paths; the two formats agree; repeated includes replace the defaults in caller order; the built-in category holds the run profile including `` | +### Tier PR — The `xmd prompt` command + +Authorship around one ordinary document: generation, validation, repair, live +review, saving and execution. Defined in [`xmd prompt`](./prompt-command-spec.md), +whose acceptance table this points at rather than restating. The ACPX runtime is +a scriptable fake, the review provider is a scripted `Elicitation` handler, and +the contextual working directory is a temporary one — no live agent, browser or +network appears in this evidence, and every refusal is proven by the phase +tripwires that stayed at zero. + +| # | Test | Verify | +|---|------|--------| +| P1–P3 | Fixed grammar | One request preserved byte for byte; missing, repeated, empty and whitespace-only requests refused; individual options after the request and aggregate props before it; a built-in option never read as a generated property's value | +| P4–P6 | Generated props | Scalar, switch, repeated and aggregate sources resolve with ordinary precedence and reach validation and execution; a draft's defect is repaired while a caller's terminates; a frozen option's shape is what no later draft may change | +| P7–P9 | Generation | One run-profile catalog, origin-only TypeScript truth included, becomes the fresh session's system prompt; one exact session carries every turn and a second invocation places another; only a completed terminal produces a candidate | +| P10–P12 | Repair and review | The base candidate plus exactly three repairs; an exhausted candidate is shown with its diagnostics and cannot be approved; revise needs non-empty feedback and resets the budget; the exact bytes are shown inside a fence they cannot close and executed unchanged | +| P13–P16 | Lifecycle | Every refusal is non-zero with no save, journal or run; an approval's journal holds only the document's events; the approved bytes are created exclusively before execution; the run reports `` and behaves as `xmd run` does; the deadline encloses every phase and teardown gates what follows | + ### Tier WB — The workflow component bundle in core Provider-neutral, and portable across every runtime. Defined in §5.3 and §8.1. diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md new file mode 100644 index 000000000..982a80623 --- /dev/null +++ b/specs/prompt-command-spec.md @@ -0,0 +1,343 @@ +# The `xmd prompt` command + +Ask the configured agent for an executable Markdown document, prove it is one, +decide whether to run it, and run it. + +```console +$ xmd prompt "ask me for my age and write the result to a file" +``` + +The command writes no code of its own. It adds *authorship* around one ordinary +document: the agent produces a complete root, `xmd` validates that root without +executing any of it, repairs the definite defects by asking again, shows a +person the exact bytes, and — once they approve — executes those bytes through +the same path `xmd run` uses for a supplied document. There is no second +execution model, no second props model and no second journal. + +## The flow + +```text +fixed CLI preflight + -> structured syntax catalog + -> fresh generator session + -> candidate props + validation + -> automatic repair, at most three turns + -> live approval or human revision + -> generator teardown + -> optional exclusive save + -> ordinary in-memory document execution +``` + +Each phase hands the next one a value. No phase after the first failure begins, +so a refused command line reaches no catalog, a failed generation reaches no +review, and an aborted review reaches no file and no run. + +Generation, repair and approval are a conversation, and a conversation is not a +run: they create no journal, retain no durable identity and replay nothing. Only +the final document execution owns `--journal`. + +## Command grammar + +```console +xmd prompt [options] [--props- ]… +``` + +The request is one positional argument and is text for the agent, never a path. +It must hold at least one non-whitespace character; that is the only test +applied to it. Its original text is preserved and sent byte for byte, so leading +and trailing whitespace survives. + +No request, more than one request, an empty string and a whitespace-only string +each fail the invocation. There is no stdin form and no editor form: a request +nobody wrote is never guessed, and no token is silently dropped. + +`--` ends option parsing, so a request that begins with `-` is written after it: + +```console +xmd prompt -- "--this is the request" +``` + +`xmd prompt` takes the complete `xmd run -e` execution flag set — `--include`, +`--verbose`, `--journal`/`-j`, `--raw`, `--agent-provider`, `--default-agent`, +`--approve-all`, `--approve-reads`, `--deny-all`, `--no-secret-detection`, and +the three timeout options — plus `--save`. Those options configure one +invocation: the same Agent and permission choices answer the generator's +requests and the executed document's, and the same includes describe the +vocabulary and resolve the run's components. + +`-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a +document. + +### `--save ` + +Write the approved source to `path` before executing it. The path is resolved +against the contextual working directory and **created exclusively**: an +existing path is left exactly as it is, the command fails, and nothing runs. +There is no check-then-write — the exclusive create *is* the check. + +The file holds the approved source and nothing else: no diagnostics, no +decision, no wrapper. Without `--save`, no generated-source file is created +anywhere; the approved document runs from memory. + +### Help + +```console +xmd prompt --help +``` + +Help needs no request. It describes the request, `--save`, the aggregate +`--props`/`XMD_PROPS` sources, that candidate-declared individual options follow +the request, and every shared execution flag. It describes no individual +property, because the document that would declare one does not exist yet — +answering otherwise would mean generating a document in order to describe one. + +Help reads no catalog, contacts no provider, places no session, asks nobody +anything, creates no file and runs nothing. + +## Generated document properties + +An `xmd prompt` invocation resolves root props exactly as `xmd run` does — same +sources, same precedence, same decoding ([Root Document +Props](./root-document-props-spec.md)) — with one difference: the schema comes +from the *candidate*, and there may be several candidates. + +```console +xmd prompt "greet someone" --props-name Ada --props-loud +``` + +The aggregate `--props` may be written before the request, because its meaning +never depends on a document. An individual `--props-*` option before the request +fails in preflight, before any catalog, Agent, elicitation, save, journal or +document operation. + +The original argv is the command line source for every candidate. `XMD_PROPS` +and the candidate's `XMD_PROPS_*` variables are read through the contextual +runtime environment. No resolved props object is carried from one candidate to +the next. + +For each candidate — the first draft, each repair and each answer to a human +revision — the command: + +1. builds the root as supplied text under the `` identity; +2. establishes that the root's own declaration is readable, from + `validateDocument()`'s structured answer rather than from an exception's + prose; +3. reads the usable props schema with `inspectDocument()`, once that declaration + is known good; +4. builds the individual bindings that schema generates; +5. holds every supplied option to its frozen signature; +6. extracts the unchanged raw property arguments, reads their environment + values, and resolves them; and +7. validates the exact candidate again with the resolved props, the ordered + includes and the host's identity-component declarations. + +### Frozen signatures + +An individual option's **signature** is its generated option name, its token +arity — a bare switch or one value — and its accumulation behavior — scalar +last-wins or repeated array. The first candidate that successfully binds a +supplied option freezes that option's signature. + +Before a later candidate extracts anything, every frozen supplied option is +compared with that candidate's binding. A removed option, a changed arity or a +changed accumulation is a terminal caller-source failure. The comparison happens +*before* extraction, so a switch that became a value option cannot reach forward +and read the `--raw`, `--include` or other built-in option written after it. + +A built-in option is never read as a generated property's value. A value that +really begins with `-` is written in the unambiguous inline form: + +```console +xmd prompt "" --props-name=-Ada +``` + +### Candidate failures and caller failures + +These are **repairable candidate failures**. The agent authored them, so the +agent is asked again: + +- source, frontmatter and root declaration diagnostics; +- a generated binding-name collision — two declared properties producing one + option or one environment variable; +- missing required root props and every other `props-invalid` result; and +- every other definite document diagnostic. + +These are **terminal property-source failures**. The caller wrote them, so no +repair turn is spent and no review, save, journal or execution happens: + +- a supplied individual option the usable candidate schema does not declare; +- malformed aggregate CLI or environment JSON; +- an individual CLI or environment value the candidate schema cannot decode; +- an extra positional the candidate's binding arity exposes; and +- a later candidate removing a frozen option or changing its signature. + +The split is about authorship, not severity. A collision is repairable because +the candidate authored both colliding names; an undeclared option is terminal +because only the caller supplied it, and teaching the agent about it would spend +a repair turn on a defect the agent cannot fix. + +An extra positional a candidate's arity exposes is decided as soon as the first +usable schema exists, and always before anything is presented. + +## Generation + +One catalog is built for the invocation, from the run profile's declarations in +the contextual working directory and the configured includes — the same +structured `SyntaxCatalog` value `xmd syntax` renders, rendered by the same +renderer. Nothing spawns `xmd syntax`, parses its Markdown, or keeps a second +catalog. Its statement that a repository TypeScript component's contract was not +read is the honest one, and it reaches the generator unchanged. + +The system instruction layer states that the session writes complete executable +Markdown roots for the requests that follow, that every answer is replacement +source only with no enclosing fence and no prose, that a repair request carries +validation facts and needs another complete replacement, and that the appended +catalog is the complete available vocabulary. It is the session's system prompt, +not part of any user message. + +The initial user turn is the request string itself, byte for byte. + +The command resolves one logical session name unique to the invocation and one +exact `Session` value from it, and passes that same value to the initial turn, +every repair turn and every human revision turn. Placing the session contacts no +backend: a fresh placement materializes when the first subscribed turn is +accepted. A second `xmd prompt` places a different session, so no invocation +inherits another's history. + +A candidate is the stream's complete close value from a terminal `completed` +turn. A failed, cancelled, unavailable or protocol-invalid turn is a generation +failure however much text it emitted: the partial candidate is discarded, and +nothing is presented, saved or executed. + +The generator's scope closes after approval and before `--save` or execution. A +teardown failure fails the command and neither later phase happens. + +## Repair + +Each first draft, and each answer to a human `revise`, starts its own repair +budget of three. A candidate with repairable failures earns one repair turn +carrying a fixed instruction to return a complete replacement root, core's +complete versioned `DocumentValidation` value serialized as JSON when core +produced one, and the prompt-owned generated-binding diagnostic when that is the +failure. Nothing renders a diagnostic into prose and nothing parses one back +out. + +The whole candidate is replaced by the new close value and validated again. The +base candidate is not repair turn one, so a candidate may be followed by at most +three replacements. A fourth invalid candidate is **exhausted** and goes to human +review carrying its diagnostics. + +An opaque `not-statically-checkable` invocation is not a diagnostic and does not +make a document invalid. It may be approved and executed under the ordinary +permission model. + +No fence is stripped and no Markdown-looking substring is extracted. The exact +close value is always the candidate: if a fenced reply is otherwise a valid +document, that exact document is shown and the person decides. + +## Review + +The command installs the CLI's WebForm elicitation provider around authorship — +outside any document and any journal — and asks through core's `elicit()` +operation. + +A valid candidate is offered a draft-07 object schema with a required `decision` +of `approve`, `revise` or `abort`, an optional string `feedback`, +`additionalProperties: false`, and an `if`/`then` rule requiring `feedback` with +`minLength: 1` when the decision is `revise`. An exhausted invalid candidate is +offered the same schema allowing only `revise` and `abort`: there is no approve +value to reject later. + +The message presents the exact candidate in a Markdown source fence whose +delimiter is at least three backticks and longer than every backtick run in the +candidate, so arbitrary source cannot close it. An invalid candidate's source is +followed by a `json` fence holding its complete structured diagnostics. Schemas, +session identifiers and repair budgets are machinery and stay out of what the +person reads. + +- `approve` returns the candidate and the props resolved for that candidate. +- `revise` sends the feedback to the same generator session, asks for a complete + replacement, resets the automatic repair budget to three, and reviews again. + Human revision rounds have no numeric bound. +- `abort` fails the command and performs no save and no execution. + +The contract is non-empty feedback, not non-whitespace feedback. A provider +failure, or an answer the schema rejects, fails the command the way an abort +does. + +## Execution + +After approval and successful generator teardown, `--save` writes, and then the +approved source executes as `retainedSource("", source)` through the +same document runner `xmd run -e` uses, with the props resolved under that exact +source and the same includes, output, journal, secret detection, Agent, +permission and timeout configuration. No temporary Markdown file is created. + +The `` identity affects positions and diagnostics only. The contextual +working directory still resolves relative filesystem operations, repository +components and includes. + +The executed program receives a fresh ordinary document Agent provider. It +inherits neither the generator session nor its system prompt. + +`--journal` is created when execution starts, and only then. Generation, +repairs, review and abort never enter it. Rendered output, a value root's JSON +result, runtime failure reporting and exit codes are byte-for-byte ordinary +`xmd run` behavior. A runtime failure does not return the command to generation +or review; `--save` has already completed, and the source is there to hand-edit. + +## Timeouts + +`--timeout` bounds the whole command: preparation, catalog inspection, +generation, every repair, human review, generator teardown, the save and the +execution. Expiry is Effection cancellation, so structured teardown completes +before the failure is reported. + +`--timeout-exec` and `--timeout-fetch` configure the final document's effects +only, exactly as under `xmd run`. Nothing bounds a generation turn but the +command deadline. + +## Failures + +Every failure below exits non-zero, and each one stops the phases after it: + +| Failure | Reaches | +| --- | --- | +| a malformed command line | nothing | +| incompatible permission flags or an unknown `--agent-provider` | nothing | +| a catalog an include makes unreadable | no provider | +| a turn that did not complete | no review, save or run | +| a terminal property-source failure | no repair, review, save or run | +| `abort`, or a review the provider could not answer | no save or run | +| generator teardown | no save or run | +| a `--save` path that exists, or a write that fails | no run | +| the document's own runtime failure | nothing after it; the save stands | + +## Acceptance + +Tier PR. Rows P1–P6 are proven in `packages/cli/tests/prompt-args.test.ts`, +P5–P12 in `packages/cli/tests/prompt.test.ts`, and P2, P4 and P13–P16 in +`packages/cli/tests/prompt-cli.test.ts`. The ACPX runtime is a scriptable fake, +the review provider is a scripted `Elicitation` handler, and the contextual +working directory is a temporary one: no live agent, browser or network appears +in this evidence. Every refusal is proven by the phase tripwires that stayed at +zero rather than by output nobody produced. + +| # | Criterion | Required observation | +| --- | --- | --- | +| P1 | Request grammar | Missing, repeated, empty and whitespace-only requests fail; a valid request reaches the initial Agent turn byte for byte | +| P2 | Ordering and help | An individual option before the request fails with zero downstream effects; aggregate props before it work; help needs no request and has zero effects | +| P3 | Fixed option safety | Built-in flags after generated props are parsed for generation and execution; a later signature change cannot consume one as a value | +| P4 | Scalar, boolean and aggregate props | Candidate-declared scalar and bare boolean CLI options plus aggregate CLI/environment JSON resolve with ordinary precedence and reach validation and execution | +| P5 | Candidate/source classification | An invalid declaration and a generated binding collision receive repair turns; an undeclared supplied option, malformed caller JSON and an invalid caller value terminate with no repair or review | +| P6 | Revision signature | Removing an option, changing boolean/scalar arity, changing scalar/array accumulation or rejecting the original value fails before extraction and presentation; unchanged signatures re-resolve from unchanged sources | +| P7 | Catalog and system layer | One structured run-profile catalog, including its origin-only TypeScript truth, becomes the fresh session's system prompt; no subprocess and no duplicate catalog exists | +| P8 | Fresh conversation | Initial, repair and human revision turns use one exact session object; a second invocation has a different placement; no backend is contacted before the first subscribed turn | +| P9 | Generation terminal | Only a completed terminal yields a candidate; a failed turn discards its partial text and causes no review, save or execution | +| P10 | Repair budget | The base candidate plus exactly three repairs are possible; the last invalid candidate is shown with complete diagnostics and cannot be approved | +| P11 | Human review | Valid source offers approve/revise/abort; invalid source offers revise/abort; revise requires non-empty feedback, continues the session and resets the repair budget | +| P12 | Exact source | Fences inside a candidate cannot close the review fence; approval, save and execution receive the Agent close value byte for byte; no fence stripping occurs | +| P13 | Abort and journal boundary | Abort, generation failure, a terminal props failure and an exhausted-invalid abort are non-zero with no save, journal or execution; an approval's journal contains only final document events | +| P14 | Save | Approved bytes are exclusively created before execution; an existing path is unchanged and prevents execution; no save flag creates no generated file | +| P15 | Ordinary execution | Approved source reports ``, resolves the contextual cwd and includes normally, receives approved-candidate props and the shared Agent flags, and preserves run output, result and runtime-failure behavior | +| P16 | Lifetime | The whole-command deadline cancels every phase and awaits provider teardown; generator teardown failure prevents save and execution; exec and fetch timeouts remain document-only | diff --git a/specs/root-document-props-spec.md b/specs/root-document-props-spec.md index 62f8d963b..cc08db533 100644 --- a/specs/root-document-props-spec.md +++ b/specs/root-document-props-spec.md @@ -354,9 +354,56 @@ than another spelling of the same one. ## Command Scope -Root document props belong to `xmd run`, and so does the inline root document. -`xmd test` accepts neither `--props`, `--props-*`, `XMD_PROPS`, `XMD_PROPS_*`, -nor `--eval`/`-e`. +Root document props belong to `xmd run` and `xmd prompt`. The inline root +document belongs to `xmd run` alone. `xmd test` accepts neither `--props`, +`--props-*`, `XMD_PROPS`, `XMD_PROPS_*`, nor `--eval`/`-e`. + +## Generated documents + +`xmd prompt` resolves root props from the same sources, in the same order, with +the same decoding ([`xmd prompt`](./prompt-command-spec.md)). What differs is +where the schema comes from: the document is written by an agent while the +command runs, and one invocation may see several drafts of it. + +Individual options follow the **request** rather than a document path, for the +same reason they follow the path under `xmd run` — the schema that gives them +meaning arrives after them: + +```console +xmd prompt "greet someone" --props-name Ada --props-loud +``` + +An individual `--props-*` written before the request fails in preflight, before +any catalog, agent, elicitation, save, journal or document operation. The +aggregate `--props` may be written before it, because its meaning never depends +on a document. + +`xmd prompt --help` describes the request, `--save`, the aggregate sources and +where individual options go. It names no individual option and no default: the +document that would declare them does not exist yet, and generating one in order +to describe it is not what help does. + +Every draft is bound afresh. The original argv is the command-line source for +all of them, the environment is read for each candidate's own bindings, and no +resolved props object is carried from one draft to the next. + +An individual option's **signature** is its generated option name, its token +arity — a bare switch or one value — and its accumulation behavior — scalar +last-wins or repeated array. The first draft that successfully binds a supplied +option freezes that option's signature, and every later draft is compared with +it before a single token is extracted. A removed option, a changed arity or a +changed accumulation fails the command. Checking before extraction is what stops +a switch that became a value option from reading the `--raw` or `--include` +written after it; a built-in option is never a generated property's value, and a +value that really begins with `-` is written `--props-name=-value`. + +Failures divide by who authored them. A draft's own defect — an unreadable +declaration, a collision between two properties generating one option or one +variable, a missing required property, any other document diagnostic — is +repairable: the agent is asked for a replacement. A defect the caller wrote — an +option no draft declares, malformed aggregate JSON, a value the schema cannot +decode, an extra positional a draft's arity exposes, or a signature change — +terminates the command with no repair, no review, no save and no run. ## Targeted roots From d6ca47d25de2195afb8d1dce9030e42d038ec309 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 06:58:18 -0400 Subject: [PATCH 02/15] =?UTF-8?q?=F0=9F=90=9B=20Settle=20one=20agent=20sta?= =?UTF-8?q?ck,=20refuse=20-e,=20and=20report=20a=20run's=20failure=20(#260?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three review findings on the xmd prompt command. An inline document reached the generator. `preparePropsPhase` skips the parse that carries the other commands' `-e` refusal, so a supplied document was dropped in silence and the command generated a different one. Refused in prompt's own branch, before the catalog, the provider, the review, the save, the journal or any execution. The agent configuration was resolved twice, once for generation and again for the document, so one command line had two chances to read DEFAULT_AGENT_NAME differently. A dispatch now settles it once and hands the resolved value to both consumers; DocumentMode carries the settled stack rather than the flags. The executed program still gets a fresh ordinary provider and inherits neither the generator session nor its system prompt. A document failing its own boundary printed a bare message where a run prints a `tests failed:` heading. reportFailure moves to a module both commands share. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/src/cli.ts | 123 ++++++++------ packages/cli/src/prompt.ts | 27 +-- packages/cli/src/report.ts | 22 +++ packages/cli/tests/prompt-cli.test.ts | 168 ++++++++++++++++--- packages/cli/tests/prompt.test.ts | 16 +- packages/cli/tests/support/prompt-harness.ts | 20 +++ specs/acp-client-spec.md | 5 +- specs/prompt-command-spec.md | 16 +- 8 files changed, 306 insertions(+), 91 deletions(-) create mode 100644 packages/cli/src/report.ts diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 4811716a8..151731396 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -80,7 +80,6 @@ import { command as hostCommand } from "@executablemd/runtime"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { installTestingComponents, - TestFailureError, testHarnessInstallation, useTesting, } from "@executablemd/testing"; @@ -93,6 +92,8 @@ import { installWebComponents, installWebElicitation } from "@executablemd/web"; import { timebox } from "@effectionx/timebox"; import { timeout as runTimeout } from "@executablemd/runtime"; import { installRunAgentStack, resolveAgentStack } from "./agent-stack.ts"; +import type { AgentStack } from "./agent-stack.ts"; +import { reportFailure } from "./report.ts"; import { TIMEOUT_FLAGS, resolveRunTimeouts } from "./timeouts.ts"; import type { RunTimeouts } from "./timeouts.ts"; import type { AgentFlags } from "./agent-config.ts"; @@ -517,22 +518,25 @@ function* underRunDeadline(timeouts: RunTimeouts, body: () => Operation): } /** - * Install the agent stack for `xmd run`: permission mode, the ACPX - * registration, and the components with the resolved root provider. Invalid flags and an unknown --agent-provider fail here - * — before any document executes. Nothing starts an agent: the provider - * validates availability on first use. + * Settle the Agent configuration for one invocation, or report why it cannot + * be settled. + * + * Called once per command, before anything a document or an agent could + * observe: incompatible permission flags and an unknown `--agent-provider` are + * command-line failures, and a command line is wrong before any of it runs. + * `undefined` means the caller has already reported and should stop. */ -function* installAgentStack( +function* settleAgentStack( flags: AgentFlags, sessions: MachineSessionAssembly | undefined, -): Operation { +): Operation { const stack = yield* resolveAgentStack(flags, sessions); if (!stack.ok) { console.error(stack.error.message); yield* exit(1); - return; + return undefined; } - yield* installRunAgentStack(stack.value); + return stack.value; } /** @@ -601,7 +605,16 @@ interface DocumentConfig { export interface DocumentMode { testing: boolean; - agent?: AgentFlags; + /** + * The Agent configuration this invocation already settled. + * + * Resolved, not the flags that produced it: an invocation reads + * `DEFAULT_AGENT_NAME` and decides its permission mode once, and a command + * that generates a document before running one has two consumers for that one + * answer. Passing the flags instead would let the second consumer reach a + * different conclusion than the first from the same command line. + */ + agent?: AgentStack; /** * What this host states about machine-wide agent sessions: who owns one, * how it was constructed, which build it belongs to, and which adapters this @@ -673,7 +686,7 @@ export function* installDocumentComponents(mode: DocumentMode, verbose: boolean) // Agent flags belong to the two commands that end in a document execution — // `xmd test` drives agents through the deterministic TestAgent stack instead. if (mode.agent) { - yield* installAgentStack(mode.agent, mode.machineSessions); + yield* installRunAgentStack(mode.agent); } } @@ -888,11 +901,6 @@ export interface PromptExecutionConfig { journal: string | undefined; raw: boolean; secretDetection: boolean; - agentProvider: string; - defaultAgent: string | undefined; - approveAll: boolean; - approveReads: boolean; - denyAll: boolean; } /** @@ -907,6 +915,7 @@ export interface PromptExecutionConfig { */ export function promptExecutor( config: PromptExecutionConfig, + stack: AgentStack, sessions: MachineSessionAssembly | undefined, installService: HostServiceInstaller, ): (approved: PromptExecution) => Operation> { @@ -928,13 +937,11 @@ export function promptExecutor( testing: false, props: approved.props, ...(sessions === undefined ? {} : { machineSessions: sessions }), - agent: { - agentProvider: config.agentProvider, - defaultAgent: config.defaultAgent, - approveAll: config.approveAll, - approveReads: config.approveReads, - denyAll: config.denyAll, - }, + // The same object generation was configured from. The provider it + // installs is a fresh ordinary one — the generator's scope is already + // gone — but which agent it defaults to and what it may do were + // decided once, for the whole invocation. + agent: stack, }, installService, ); @@ -970,16 +977,6 @@ function targetFailureReport(root: RootDocumentSource, error: unknown): string | return [outcome, heading, ...references].join("\n"); } -/** Print a completed document's failure the way `xmd` has always printed it. */ -function reportFailure(error: Error, prefix?: string): void { - const label = prefix === undefined ? "" : `${prefix}: `; - if (error instanceof TestFailureError) { - console.error(`\n${label}tests failed: ${error.message}`); - return; - } - console.error(`${label}${error.message}`); -} - interface TestConfig extends Omit { /** * Optional because `field` types a schema by what it accepts, and @@ -1261,6 +1258,19 @@ function* preparePropsPhase(args: string[], evalFlags: EvalFlags): Operation> { // `` reaches a person through the browser form, and `xmd run` @@ -1849,13 +1872,7 @@ function* dispatch( // it, which is what keeps a machine session from being acted on by // a command that never said it could own one. ...(sessions === undefined ? {} : { machineSessions: sessions }), - agent: { - agentProvider: config.agentProvider, - defaultAgent: config.defaultAgent, - approveAll: config.approveAll, - approveReads: config.approveReads, - denyAll: config.denyAll, - }, + agent: runStack, }, installService, ); @@ -1883,19 +1900,29 @@ function* dispatch( yield* exit(1); break; } + // Once, here, and handed to both consumers below. Generation and the + // execution that follows it are one invocation, so they answer to one + // `--default-agent`, one `DEFAULT_AGENT_NAME` and one permission mode. + const promptStack = yield* settleAgentStack( + { + agentProvider: config.agentProvider, + defaultAgent: config.defaultAgent, + approveAll: config.approveAll, + approveReads: config.approveReads, + denyAll: config.denyAll, + }, + sessions, + ); + if (promptStack === undefined) { + break; + } const exitCode = yield* runPrompt( { argv: helpRequest.args, scan, include: config.include, ...(config.save === undefined ? {} : { save: config.save }), - agent: { - agentProvider: config.agentProvider, - defaultAgent: config.defaultAgent, - approveAll: config.approveAll, - approveReads: config.approveReads, - denyAll: config.denyAll, - }, + stack: promptStack, }, { ...(sessions === undefined ? {} : { sessions }), @@ -1905,7 +1932,7 @@ function* dispatch( // A host that answers installs a provider; one that does not installs // none, and nothing downstream reads a profile to find out which. installElicitation: installWebElicitation, - execute: promptExecutor(config, sessions, installService), + execute: promptExecutor(config, promptStack, sessions, installService), }, ); if (exitCode !== 0) { diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index cda8bd3a0..d86b4b0b1 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -55,9 +55,8 @@ import type { import type { AcpxProvider, AcpxProviderDependencies } from "@executablemd/acp"; import { cwd } from "@executablemd/runtime"; -import { resolveAgentStack, useGeneratorAgent } from "./agent-stack.ts"; +import { useGeneratorAgent } from "./agent-stack.ts"; import type { AgentStack } from "./agent-stack.ts"; -import type { AgentFlags } from "./agent-config.ts"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { buildBindings, @@ -66,6 +65,7 @@ import { resolvePropsFromSources, } from "./props.ts"; import type { Binding, Extraction } from "./props.ts"; +import { reportFailure } from "./report.ts"; import { renderSyntaxMarkdown, useRunProfileRegistry } from "./syntax.ts"; import { isReservedOption, @@ -110,7 +110,14 @@ export interface PromptCommand { include: string[]; /** Where the approved source is written before it runs, when asked for. */ save?: string; - agent: AgentFlags; + /** + * The Agent configuration this invocation settled, before the command ran. + * + * Settled by the caller rather than here, because the execution that follows + * approval is configured from the same answer: two resolutions of one command + * line is two chances to read `DEFAULT_AGENT_NAME` differently. + */ + stack: AgentStack; } /** What the host supplies. Every entry is a decision only a host can make. */ @@ -182,12 +189,6 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op } const request = scan.request; - const stack = yield* resolveAgentStack(command.agent, deps.sessions); - if (!stack.ok) { - console.error(stack.error.message); - return 1; - } - let instructions: string; try { instructions = generatorInstructions(yield* deps.catalog(command.include)); @@ -207,7 +208,7 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op // catalog read one registry, so a component the generator was told about // is one validation resolves. yield* useRunProfileRegistry(); - const provider = yield* useGeneratorAgent(stack.value, instructions, deps.acp); + const provider = yield* useGeneratorAgent(command.stack, instructions, deps.acp); const session = yield* provider.session(generatorSessionName()); return yield* author(command, provider, session, request); }); @@ -235,7 +236,11 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op props, }); if (!executed.ok) { - console.error(executed.error.message); + // Reported exactly as `xmd run` reports the same failure: what ran is one + // ordinary document, and how it failed is not this command's news to + // rephrase. A runtime failure ends the command here — there is nothing for + // generation or review to reconsider about source a person already approved. + reportFailure(executed.error); return 1; } return 0; diff --git a/packages/cli/src/report.ts b/packages/cli/src/report.ts new file mode 100644 index 000000000..73be4b0de --- /dev/null +++ b/packages/cli/src/report.ts @@ -0,0 +1,22 @@ +/** + * How a completed document's failure reaches the terminal. + * + * Shared because `xmd run` and `xmd prompt` end in the same thing — one + * ordinary document execution — and a person reading the output of either has + * no reason to see the same failure worded two ways. A test failure is the case + * that makes the difference visible: it earns a heading and a blank line above + * it, and printing only its message would report a failed suite as if it were + * an ordinary error. + */ + +import { TestFailureError } from "@executablemd/testing"; + +/** Print a completed document's failure the way `xmd` has always printed it. */ +export function reportFailure(error: Error, prefix?: string): void { + const label = prefix === undefined ? "" : `${prefix}: `; + if (error instanceof TestFailureError) { + console.error(`\n${label}tests failed: ${error.message}`); + return; + } + console.error(`${label}${error.message}`); +} diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 14bbda5f1..64a834267 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -16,20 +16,24 @@ import { expect } from "@executablemd/test-support/expect"; import { runCli } from "@executablemd/test-support/launch"; import { readTextFile, writeTextFile } from "@effectionx/fs"; import { stat } from "@executablemd/runtime"; -import { Ok, scoped, spawn } from "effection"; +import { ensure, Ok, scoped, spawn } from "effection"; import type { Operation, Result } from "effection"; import { join } from "node:path"; import { readdir } from "node:fs/promises"; import { until } from "effection"; import { promptExecutor } from "../src/cli.ts"; +import { resolveAgentStack } from "../src/agent-stack.ts"; +import type { AgentStack } from "../src/agent-stack.ts"; import { runPrompt } from "../src/prompt.ts"; import type { PromptCommand, PromptExecution } from "../src/prompt.ts"; import { scanPromptArgs } from "../src/prompt-args.ts"; import { AGENT, createPromptHarness, + timesRead, useEnvironment, + useRecordedEnvironment, useWorkingDirectory, } from "./support/prompt-harness.ts"; import type { PromptHarness } from "./support/prompt-harness.ts"; @@ -64,6 +68,29 @@ const FAILS_AT_RUN = ["```bash exec", "exit 3", "```", ""].join("\n"); const PLAIN = "Nothing but prose.\n"; +/** + * A document that validates, runs, and fails its own tests. + * + * A `` boundary is what makes an assertion decide the outcome of an + * ordinary run, so this is the approved document that ends in the one failure + * `xmd run` reports differently from every other: with a heading of its own. + */ +const FAILING_TEST = [ + "", + '', + "", + "", + "", + "", +].join("\n"); + +/** The Agent configuration a dispatch settles once and hands to both consumers. */ +const STACK: AgentStack = { + provider: "acpx", + defaultAgent: AGENT, + permissionMode: "deny-all", +}; + function command(dir: string, args: string[], save?: string): PromptCommand { const argv = ["prompt", ...args]; return { @@ -71,13 +98,7 @@ function command(dir: string, args: string[], save?: string): PromptCommand { scan: scanPromptArgs(argv), include: [dir], ...(save === undefined ? {} : { save }), - agent: { - agentProvider: "acpx", - defaultAgent: AGENT, - approveAll: false, - approveReads: false, - denyAll: true, - }, + stack: STACK, }; } @@ -85,6 +106,7 @@ function command(dir: string, args: string[], save?: string): PromptCommand { function executor( dir: string, journal?: string, + stack?: AgentStack, ): (approved: PromptExecution) => Operation> { return promptExecutor( { @@ -93,12 +115,8 @@ function executor( journal, raw: true, secretDetection: true, - agentProvider: "acpx", - defaultAgent: AGENT, - approveAll: false, - approveReads: false, - denyAll: true, }, + stack ?? STACK, undefined, function* () {}, ); @@ -146,22 +164,54 @@ describe( expect(yield* exists(join(dir, "out.md"))).toBe(false); }); - // The agent configuration is settled before the catalog is built, so an - // incompatible pair of permission flags costs no inspection at all. + // The agent configuration is settled before the command begins, so an + // incompatible pair of permission flags costs no inspection at all. Read + // at the boundary an operator uses, because that is where the resolution + // the whole invocation shares now happens. yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); - const request = command(dir, [REQUEST], "out.md"); - const code = yield* runPrompt( - { ...request, agent: { ...request.agent, approveAll: true, denyAll: true } }, - harness.deps, - ); + const { code, stderr } = yield* runCli( + ["prompt", REQUEST, "--approve-all", "--deny-all", "--save", "out.md"], + { cwd: dir }, + ).join(); expect(code).toBe(1); - expect(untouched(harness)).toEqual(NOTHING); + expect(stderr).toContain( + "--approve-all, --approve-reads, and --deny-all are mutually exclusive", + ); + // No provider was built: reaching one is what reports an agent as + // unavailable, and this command line never got that far. + expect(stderr).not.toContain("unavailable"); expect(yield* exists(join(dir, "out.md"))).toBe(false); }); }); + it("P2: inline source is refused before any phase begins", function* () { + // `-e` belongs to `xmd run`. `xmd prompt` is the command that *writes* a + // document, so a second one supplied on the command line is a + // contradiction — and one the parser used to drop in silence, leaving the + // caller watching a different document get generated. + for (const flag of ["-e", "--eval"]) { + yield* useWorkingDirectory(function* (dir) { + const { code, stdout, stderr } = yield* runCli( + ["prompt", REQUEST, flag, "# supplied", "--save", "out.md", "--journal", "trace.jsonl"], + { cwd: dir }, + ).join(); + + expect(code).toBe(1); + expect(stderr).toContain( + "unrecognized option for xmd prompt: --eval — inline documents are exclusive to xmd run", + ); + // Every later phase, unreached: no catalog was rendered, no provider + // was built, no review was asked, and neither named file was made. + expect(stdout).not.toContain("## Built-in components"); + expect(stderr).not.toContain("unavailable"); + expect(stdout).toBe(""); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* exists(join(dir, "trace.jsonl"))).toBe(false); + }); + } + }); + it("P2: help needs no request and touches nothing", function* () { yield* useWorkingDirectory(function* (dir) { const { code, stdout, stderr } = yield* runCli( @@ -382,6 +432,80 @@ describe( }); }); + it("P15: a failing test in the approved document reports as a run reports it", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: FAILING_TEST }); + harness.script({ decision: "approve" }); + + const written = console.error; + const lines: string[] = []; + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + console.error = written; + }); + console.error = (...parts: unknown[]) => { + lines.push(parts.map((part) => String(part)).join(" ")); + }; + return yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); + }); + + expect(code).toBe(1); + // Byte for byte what `xmd run` prints for this failure: the heading and + // the blank line above it are how a failed suite is told apart from an + // ordinary error, and printing only the message would lose both. + expect(lines.at(-1)).toBe("\ntests failed: 1 test(s) failed in "); + + // A runtime failure is not a candidate defect: the agent was asked once + // and the person was asked once, and neither was asked again. + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.reviews).toHaveLength(1); + // The save already happened, and a failing run leaves it to hand-edit. + expect(yield* readTextFile(join(dir, "out.md"))).toBe(FAILING_TEST); + }); + }); + + it("P15: one Agent resolution serves generation and the execution after it", function* () { + yield* useWorkingDirectory(function* (dir) { + const reads: string[] = []; + yield* useRecordedEnvironment(reads, { DEFAULT_AGENT_NAME: "settled-agent" }); + + // What a dispatch settles, once, for the whole invocation. + const settled = yield* resolveAgentStack( + { + agentProvider: "acpx", + defaultAgent: undefined, + approveAll: false, + approveReads: false, + denyAll: true, + }, + undefined, + ); + if (!settled.ok) { + throw settled.error; + } + const stack = settled.value; + expect(stack.defaultAgent).toBe("settled-agent"); + expect(timesRead(reads, "DEFAULT_AGENT_NAME")).toBe(1); + + const harness = createPromptHarness(); + harness.deps.execute = executor(dir, undefined, stack); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt({ ...command(dir, [REQUEST]), stack }, harness.deps); + + expect(code).toBe(0); + // Generation resolved the settled agent rather than a name of its own. + expect(harness.fake.ensured.map((input) => input.agent)).toEqual(["settled-agent"]); + // And nothing after it read the name again: authorship and the document + // installation that followed were both configured from the one answer, + // so they cannot disagree about which agent this invocation meant. + expect(timesRead(reads, "DEFAULT_AGENT_NAME")).toBe(1); + }); + }); + it("P16: the deadline encloses every phase, and teardown gates what follows", function* () { // Expiry is cancellation, so the proof is what cancelling the command does: // the turn in flight is cancelled, the provider is dismantled, and no later diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 587d700c7..8a0ca94b2 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -22,6 +22,7 @@ import { runPrompt } from "../src/prompt.ts"; import type { PromptCommand } from "../src/prompt.ts"; import { scanPromptArgs } from "../src/prompt-args.ts"; import { useGeneratorAgent } from "../src/agent-stack.ts"; +import type { AgentStack } from "../src/agent-stack.ts"; import { AGENT, createPromptHarness, useWorkingDirectory } from "./support/prompt-harness.ts"; import type { PromptHarness } from "./support/prompt-harness.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./support/fake-acp.ts"; @@ -80,6 +81,13 @@ const NAME_IS_BOOLEAN = [ "", ].join("\n"); +/** The Agent configuration a dispatch settles once and hands to both consumers. */ +const STACK: AgentStack = { + provider: "acpx", + defaultAgent: AGENT, + permissionMode: "deny-all", +}; + function command(dir: string, args: string[], save?: string): PromptCommand { const argv = ["prompt", ...args]; return { @@ -87,13 +95,7 @@ function command(dir: string, args: string[], save?: string): PromptCommand { scan: scanPromptArgs(argv), include: [dir], ...(save === undefined ? {} : { save }), - agent: { - agentProvider: "acpx", - defaultAgent: AGENT, - approveAll: false, - approveReads: false, - denyAll: true, - }, + stack: STACK, }; } diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts index 94533e4be..e55732849 100644 --- a/packages/cli/tests/support/prompt-harness.ts +++ b/packages/cli/tests/support/prompt-harness.ts @@ -149,8 +149,23 @@ export function* useWorkingDirectory(body: (dir: string) => Operation): Op * `XMD_PROPS` quietly also be a case about the default agent. */ export function* useEnvironment(values: Record): Operation { + yield* useRecordedEnvironment([], values); +} + +/** + * The same environment, with every name it was asked recorded in order. + * + * What a resolution costs is visible only as the reads it performs: a value + * settled once and handed to both consumers reads its name once, while two + * consumers each settling their own read it twice and agree only by accident. + */ +export function* useRecordedEnvironment( + reads: string[], + values: Record, +): Operation { yield* API.Env.around({ *env([name], next) { + reads.push(name); if (Object.hasOwn(values, name)) { return values[name]; } @@ -158,3 +173,8 @@ export function* useEnvironment(values: Record): Operation }, }); } + +/** How many times one name was read. */ +export function timesRead(reads: readonly string[], name: string): number { + return reads.filter((read) => read === name).length; +} diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index d6332fa6f..3cabb11c2 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -538,7 +538,10 @@ that writes the document, and for the document that runs. The provider name, the default agent with its `DEFAULT_AGENT_NAME` precedence, the permission mode and the host's own machine-session assembly are settled before any catalog is built or any document executes, so an unknown provider or an incompatible pair of -permission flags fails first. +permission flags fails first. It is the settled value that reaches both +consumers, not the flags that produced it: `DEFAULT_AGENT_NAME` is read once per +invocation, and generation and the executed document cannot reach different +conclusions from one command line. Generation takes the provider's operations directly — `useAcpxProvider`, without the Agent install. It needs turns, not authority: no document is installed diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 982a80623..50f0c31a2 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -66,7 +66,18 @@ requests and the executed document's, and the same includes describe the vocabulary and resolve the run's components. `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a -document. +document. Supplying one anyway is refused in the command's own preflight, with +`unrecognized option for xmd prompt: --eval — inline documents are exclusive to +xmd run`, before the catalog, the generator provider, the review, the save, the +journal or any execution exists. A generated document is what this command +writes, so a second one on the command line is a contradiction rather than an +option the command is missing. + +Those options are resolved into one Agent configuration once per invocation — +one `--agent-provider`, one `--default-agent` or `DEFAULT_AGENT_NAME`, one +permission mode — and that settled answer is what both the generator and the +approved document's installation are configured from. Incompatible permission +flags and an unknown provider fail there, before the catalog is built. ### `--save ` @@ -283,7 +294,8 @@ inherits neither the generator session nor its system prompt. `--journal` is created when execution starts, and only then. Generation, repairs, review and abort never enter it. Rendered output, a value root's JSON result, runtime failure reporting and exit codes are byte-for-byte ordinary -`xmd run` behavior. A runtime failure does not return the command to generation +`xmd run` behavior — including a failing `` boundary, which is reported +under its own `tests failed:` heading rather than as a bare message. A runtime failure does not return the command to generation or review; `--save` has already completed, and the source is there to hand-edit. ## Timeouts From ddd2999f6012f1bb2c095d18aa7fb381693365d6 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 12:44:58 -0400 Subject: [PATCH 03/15] =?UTF-8?q?=E2=9C=A8=20Ship=20the=20plan=20program?= =?UTF-8?q?=20as=20a=20packaged=20Markdown=20document=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first document-backed command needs its Markdown where the command is: a source checkout, a published npm package, and a compiled binary with no checkout. The document is the deployed program, so it ships as a file rather than as a generated string. src/documents/prompt-plan.md is that program. A package declares what it ships by putting it in that directory; test documents and scenario fixtures live under src/ too, and a build that swept those up would publish fixtures and grow every binary carrying them. The command locates it from its module's own URL, never from the working directory and never through the component search path, so it finds the same policy whatever directory a person stands in and no repository file can answer for it. Each build keeps the directory beside its module: both deno compile sites embed it, and the npm build copies it into the emitted tree, because dnt emits the module graph only and an asset nothing imports would otherwise reach Node and Bun missing while Deno stayed green. A regression holds the two compile sites to the documents that exist, since that list is the one thing no build discovers. The host still runs the TypeScript authorship loop; wiring the plan program into the plan profile follows. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- .github/workflows/release.yml | 1 + deno.json | 2 +- packages/cli/src/documents/prompt-plan.md | 132 +++++++++++++++++++ packages/cli/src/packaged-document.ts | 55 ++++++++ packages/cli/tests/packaged-document.test.ts | 57 ++++++++ scripts/build-npm.ts | 33 ++++- scripts/tests/cli-npm-bin.test.ts | 9 ++ scripts/tests/packaged-document.test.ts | 55 ++++++++ specs/release-process-spec.md | 51 ++++++- 9 files changed, 391 insertions(+), 4 deletions(-) create mode 100644 packages/cli/src/documents/prompt-plan.md create mode 100644 packages/cli/src/packaged-document.ts create mode 100644 packages/cli/tests/packaged-document.test.ts create mode 100644 scripts/tests/packaged-document.test.ts diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c339ac14a..bc5db4ef3 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -98,6 +98,7 @@ jobs: --exclude-unused-npm \ --allow-all \ --include packages/code-review-agent \ + --include packages/cli/src/documents/prompt-plan.md \ --target ${{ matrix.target }} \ --output dist/${{ matrix.artifact }} \ packages/cli/src/compiled.ts diff --git a/deno.json b/deno.json index 5b4d4107e..e89db7542 100644 --- a/deno.json +++ b/deno.json @@ -57,7 +57,7 @@ "verify:clean": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/verify-clean.ts", "deps": "deno run --allow-all scripts/deps.ts", "deps:target": "deno run --allow-all scripts/deps-target.ts", - "build": "deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --output dist/xmd packages/cli/src/compiled.ts", + "build": "deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --include packages/cli/src/documents/prompt-plan.md --output dist/xmd packages/cli/src/compiled.ts", "build:web": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/preflight.ts scripts/build-web-client.ts", "gen:publish-workflow": "deno run --allow-all packages/cli/src/deno.ts run scripts/gen-publish-workflow.md", "bump": "deno run -A scripts/bump-version.ts", diff --git a/packages/cli/src/documents/prompt-plan.md b/packages/cli/src/documents/prompt-plan.md new file mode 100644 index 000000000..ac740fc04 --- /dev/null +++ b/packages/cli/src/documents/prompt-plan.md @@ -0,0 +1,132 @@ +--- +props: + type: object + properties: + request: { type: string } + syntax: { type: string } + session: { type: string } + required: [request, syntax, session] + additionalProperties: false +returns: + type: string +--- + +# The plan `xmd prompt` follows + +This is the policy behind `xmd prompt`, and it is a program rather than a +setting: what the agent is asked, how many times a draft may be repaired, what a +person is shown, and what happens when nobody approves anything are all written +here, in the open, where they can be read and argued with. + +The command supplies three things — the request as the person typed it, the +syntax catalog for this working directory, and the name of the session every +turn belongs to. Nothing here runs the document being written. A candidate is +text until a person approves it and the command runs it separately. + + + + + + +Write one complete Executable Markdown root document that accomplishes this: + +{props.request} + +Reply with the document source and nothing else. No enclosing code fence, no +explanation before or after it, no commentary about what you did. + +Write it as a document a person would want to read, not as a script with +comments bolted on. State what it does and why in ordinary prose, and let the +components appear where that prose leads to them. The person asked for the +outcome above; the document should still read as though it is about that +outcome. + +Everything you may use is described below. Use nothing that is not here. + +{props.syntax} + + + + + +Each draft gets one validation and up to three chances to fix what validation +found. A revision the person asked for is a new draft, so it gets its own three. + + + + + + + + + +That document does not validate. These are the exact findings: + + + +Return one complete replacement root document that resolves every finding above. +Source only, no enclosing fence, no explanation. Keep the narrative that explains +what the document is for. + + + + + +The person decides. A valid draft can be approved; one that is still invalid +after three repairs is shown with its findings and can only be sent back or +abandoned. On the tenth and last presentation there is nothing left to revise +into, so revision is not offered. + + +This document was written for: {props.request} + + + + +It still does not validate after three attempts to repair it. These are the +remaining findings: + + + + + + + + + + + + + + + +The person read that document and asked for this to change: + +{review.feedback} + +Return one complete replacement root document. Source only, no enclosing fence, +no explanation. Keep the narrative that explains what the document is for. + + + + + + diff --git a/packages/cli/src/packaged-document.ts b/packages/cli/src/packaged-document.ts new file mode 100644 index 000000000..daa1b19aa --- /dev/null +++ b/packages/cli/src/packaged-document.ts @@ -0,0 +1,55 @@ +/** + * The Markdown documents this CLI ships and executes itself. + * + * `xmd prompt` runs a first-party document rather than a TypeScript policy, so + * that document has to be present wherever the command is: a source checkout, a + * published npm package, and a compiled binary with no checkout at all. It is + * located from this module's own URL, never from the working directory and never + * through the component search path — the command must find the same program + * whatever directory a person happens to be standing in, and a repository file + * must not be able to answer for it. + * + * Each build keeps `src/documents/` beside its module: `deno compile --include` + * embeds each file at the same relative path, and the npm build copies the + * directory into the emitted tree. So the one lookup below is correct in all + * four places, and a build that forgets the asset fails loudly on first use + * rather than silently choosing different behavior. + */ + +import { readFile } from "node:fs/promises"; +import { until } from "effection"; +import type { Operation } from "effection"; + +/** The plan program `xmd prompt` executes. */ +export const PROMPT_PLAN = "prompt-plan.md"; + +/** + * Where a packaged document lives, as a URL beside this module. + * + * One directory rather than "any Markdown under `src/`": packages keep test + * documents and scenario fixtures beside their modules too, and a build that + * swept those up would publish them and grow the binary for no reason. Being in + * here is what declares a document shipped. + */ +export function packagedDocumentUrl(name: string): URL { + return new URL(`./documents/${name}`, import.meta.url); +} + +/** + * Read one packaged document. + * + * `node:fs` rather than a host File Api: this is the CLI reading its own + * program, not a document reaching the caller's filesystem, and it must not be + * answerable by whatever the running document installed. + */ +export function* readPackagedDocument(name: string): Operation { + const url = packagedDocumentUrl(name); + try { + return yield* until(readFile(url, "utf8")); + } catch (error) { + throw new Error( + `the packaged document ${name} is missing from this build (looked in ${url.href})`, + { cause: error }, + ); + } +} diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts new file mode 100644 index 000000000..351cf55bc --- /dev/null +++ b/packages/cli/tests/packaged-document.test.ts @@ -0,0 +1,57 @@ +/** + * The documents the CLI ships and executes itself. + * + * `xmd prompt` runs a first-party Markdown program, so that program has to be + * present and identical wherever the command is. This suite runs under Deno, + * Node and Bun, which is what makes it evidence: the lookup is package-relative, + * and a resolution that only works under the runtime the author happened to use + * is the failure this catches. + * + * The compiled binary and the published npm package are proven where those + * artifacts exist — `scripts/tests/cli-npm-bin.test.ts` for the package, and the + * `dist/xmd` suites for the binary. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { readTextFile } from "@effectionx/fs"; +import { fileURLToPath } from "node:url"; + +import { + packagedDocumentUrl, + PROMPT_PLAN, + readPackagedDocument, +} from "../src/packaged-document.ts"; +import { useWorkingDirectory } from "./support/prompt-harness.ts"; + +describe("packaged documents", () => { + it("reads the plan program from beside its module, whatever the cwd is", function* () { + // A temporary contextual cwd with no document in it. A lookup that reached + // for the working directory would find nothing here, and one that reached + // through the component search path would be answerable by a repository + // file — neither may decide which program this command runs. + const source = yield* useWorkingDirectory(function* () { + return yield* readPackagedDocument(PROMPT_PLAN); + }); + + expect(source).toBe(yield* readTextFile(fileURLToPath(packagedDocumentUrl(PROMPT_PLAN)))); + // It is the plan program, not merely some file that exists. + expect(source).toContain("returns:"); + expect(source).toContain(" { + const documents = new URL("src/documents/", pkgDir); + if (!(yield* exists(documents))) { + return []; + } + const names = yield* until(readdir(fromFileUrl(documents), { recursive: true })); + return names.map((name) => `src/documents/${name.split(sep).join("/")}`); +} + const ExportsSchema = z.union([z.string(), z.record(z.string(), z.string())]); const DenoJsonSchema = z.object({ @@ -312,6 +330,17 @@ function* buildPackage(pkgArg: string, version: string, ctx: BuildContext): Oper yield* copyFile(license, new URL("LICENSE", outDir)); } + // A package that executes its own Markdown ships that Markdown beside its + // emitted module. dnt emits the module graph and nothing else, so an asset no + // TypeScript imports would be absent from the published package while the + // source checkout kept working — the command would find nothing at runtime, + // on Node and Bun only. + for (const asset of yield* packagedDocuments(pkgDir)) { + const target = new URL(`esm/${asset}`, outDir); + yield* ensureDir(fromFileUrl(new URL(".", target))); + yield* copyFile(new URL(asset, pkgDir), target); + } + ctx.built.add(denoJson.name); const provenance = ctx.localSiblings && workspaceDeps.length > 0 ? " (local siblings — not publishable)" : ""; diff --git a/scripts/tests/cli-npm-bin.test.ts b/scripts/tests/cli-npm-bin.test.ts index 1be7be83d..29bc820cc 100644 --- a/scripts/tests/cli-npm-bin.test.ts +++ b/scripts/tests/cli-npm-bin.test.ts @@ -101,5 +101,14 @@ describe("npm CLI package", { sanitizeOps: false, sanitizeResources: false }, () // the whole of what a built npm bin has to get right for one. expect(run.stdout).toContain("You chose to approve the review."); expect(run.stdout).not.toContain("ERROR"); + + // The Markdown this package executes itself ships beside the module that + // reads it. dnt emits the module graph only, so an asset nothing imports is + // absent from the package unless the build copies it — and the command + // would then find no program to run, on Node and Bun while Deno stayed + // green. + expect(yield* readTextFile(path.join(OUT_DIR, "esm/src/documents/prompt-plan.md"))).toBe( + yield* readTextFile(path.join(ROOT, PKG_DIR, "src/documents/prompt-plan.md")), + ); }); }); diff --git a/scripts/tests/packaged-document.test.ts b/scripts/tests/packaged-document.test.ts new file mode 100644 index 000000000..a70f3c5ec --- /dev/null +++ b/scripts/tests/packaged-document.test.ts @@ -0,0 +1,55 @@ +/** + * Every packaged Markdown document reaches every build that ships it. + * + * The npm build discovers these documents, so it needs no list. `deno compile` + * names files one at a time, and it is named twice — in `deno task build` and in + * the release matrix. A document added beside its module and forgotten in either + * place produces a binary that runs until the moment it looks for its own + * program, which is the failure this refuses to let reach a release. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { readTextFile } from "@effectionx/fs"; +import type { Operation } from "effection"; +import { readdir } from "node:fs/promises"; +import { until } from "effection"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const ROOT = fileURLToPath(new URL("../../", import.meta.url)); + +/** Every document a package ships, repository-relative. */ +function* packagedDocuments(): Operation { + const found: string[] = []; + for (const member of yield* until(readdir(path.join(ROOT, "packages")))) { + const documents = path.join(ROOT, "packages", member, "src/documents"); + let entries: string[]; + try { + entries = yield* until(readdir(documents, { recursive: true })); + } catch { + continue; + } + for (const entry of entries) { + found.push(`packages/${member}/src/documents/${entry.split(path.sep).join("/")}`); + } + } + return found.sort(); +} + +describe("packaged documents reach every build", () => { + it("is embedded by both compile sites", function* () { + const documents = yield* packagedDocuments(); + // The suite is only meaningful while at least one exists; an empty sweep + // would pass while proving nothing. + expect(documents.length).toBeGreaterThan(0); + + const denoJson = yield* readTextFile(path.join(ROOT, "deno.json")); + const release = yield* readTextFile(path.join(ROOT, ".github/workflows/release.yml")); + const [buildTask = ""] = denoJson.split("\n").filter((line) => line.includes('"build":')); + + for (const document of documents) { + expect(buildTask).toContain(`--include ${document}`); + expect(release).toContain(`--include ${document}`); + } + }); +}); diff --git a/specs/release-process-spec.md b/specs/release-process-spec.md index a5a83ccc9..b02e032a6 100644 --- a/specs/release-process-spec.md +++ b/specs/release-process-spec.md @@ -92,7 +92,8 @@ documents at the revision it checks. prerelease marker — so a forgotten bump is visible where the release was made, then refuses to build. On a valid tag it compiles `packages/cli/src/compiled.ts` per target with - `--include packages/code-review-agent` and attaches the binaries and + `--include packages/code-review-agent --include packages/cli/src/prompt-plan.md` + and attaches the binaries and sha256 checksums to the tag's GitHub Release. That module is the compiled-binary entrypoint: it installs the `API.Env.command` adapter that relaunches the binary as itself, which a source entrypoint cannot do. Between @@ -543,3 +544,51 @@ A publishable `@executablemd/web` is an atomic configuration state: public inclusion of `generated/client-bundle.ts` in both published artifacts. The configuration elements change atomically; the package is never published without its browser asset and never published while private. + +## 9. Packaged documents (`@executablemd/cli`) + +A document-backed command executes first-party Markdown through the ordinary XMD +engine rather than a TypeScript policy. A package declares which Markdown it +ships by putting it in `src/documents/`; every other Markdown under `src/` — +test documents, scenario fixtures — stays out of the product. `xmd prompt` is +the first such command, and +`packages/cli/src/documents/prompt-plan.md` is its program: the checked-in Markdown is the +deployed artifact and the single source of truth, not a generated string mirror +of one. + +The command locates it from its own module URL — never from the contextual +working directory, and never through the component search path. Both are +answerable by whatever directory a person is standing in, and which policy the +command runs is not a thing a repository file may decide. + +Every build therefore keeps the asset beside its module, at the same relative +path: + +- **source checkout** — the file as committed; +- **`deno compile`** — embedded by `--include packages/cli/src/documents/prompt-plan.md`, + in `deno task build` and in `release.yml`'s matrix compile; +- **npm (dnt)** — copied by `scripts/build-npm.ts`, which copies each package's + `src/documents/` into `esm/src/documents/`, preserving relative location. dnt + emits the module graph and nothing else, so an asset no TypeScript imports is + absent from the published package unless the build copies it. That failure is + invisible under Deno and reaches only Node and Bun. + +A missing asset fails loudly, naming the path it looked at, rather than +selecting different behavior. + +The checks that hold this together, each proving a different build: + +- `packages/cli/tests/packaged-document.test.ts` reads the document from a + temporary working directory and compares it to the committed bytes. It runs + under Deno, Node and Bun, which is what makes it evidence rather than one + runtime's opinion. +- `scripts/tests/cli-npm-bin.test.ts` builds the real package and asserts + `esm/src/documents/prompt-plan.md` is byte-identical to the source. +- `scripts/tests/packaged-document.test.ts` holds the two `deno compile` sites + to the documents that exist, because that list is the one thing no build + discovers for itself. + +Adding another packaged document needs no npm-build change — `build-npm.ts` +copies the directory. Only `deno compile` names files individually, so a new +document must be added to `deno task build` and to `release.yml`, which +`scripts/tests/packaged-document.test.ts` enforces. From a22c7b14164e5a730418d6d8399bf4d61e74e9f3 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 13:19:32 -0400 Subject: [PATCH 04/15] =?UTF-8?q?=F0=9F=90=9B=20Make=20plan=20approval=20a?= =?UTF-8?q?nd=20exhaustion=20mutually=20exclusive=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A selects the value a value body publishes; it does not end that body, and a later failure wins over a value already selected. The approve branch returned the candidate and broke out of the review loop, then execution left the Session and reached the unconditional exhaustion . Every approval settled as the ten-draft failure and no approved candidate could reach the host. Approval now binds the exact candidate to a null-sentinel result, and one exhaustive branch after the Session either returns it or fails. Only the arm taken runs, so an approved candidate has no later sibling failure to overtake it. Bounds, prompt wording, diagnostics, presentation, Session structure and the authored abort remain as they were. The review schema also needed an explicit string type: Ajv strict mode refuses minLength on an untyped property, so every review was refused before a person saw anything. Diagnostics reach their code block through rather than a JSON.stringify call in an expression, so serializing them is a document construct like everything else this policy does. The regression runs the packaged document itself, resolved against first-party declarations with an empty include list, against a scripted agent turn, a scripted approval and a test validator. It fails with the exhaustion message before this change. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/src/documents/prompt-plan.md | 21 ++- packages/cli/tests/prompt-plan.test.ts | 184 ++++++++++++++++++++++ 2 files changed, 201 insertions(+), 4 deletions(-) create mode 100644 packages/cli/tests/prompt-plan.test.ts diff --git a/packages/cli/src/documents/prompt-plan.md b/packages/cli/src/documents/prompt-plan.md index ac740fc04..3af290e82 100644 --- a/packages/cli/src/documents/prompt-plan.md +++ b/packages/cli/src/documents/prompt-plan.md @@ -24,6 +24,7 @@ turn belongs to. Nothing here runs the document being written. A candidate is text until a person approves it and the command runs it separately. + @@ -62,7 +63,8 @@ found. A revision the person asked for is a new draft, so it gets its own three. That document does not validate. These are the exact findings: - + + Return one complete replacement root document that resolves every finding above. Source only, no enclosing fence, no explanation. Keep the narrative that explains @@ -93,7 +95,7 @@ into, so revision is not offered. required: ["decision"], additionalProperties: false, if: { properties: { decision: { const: "revise" } }, required: ["decision"] }, - then: { required: ["feedback"], properties: { feedback: { minLength: 1 } } }, + then: { required: ["feedback"], properties: { feedback: { type: "string", minLength: 1 } } }, }} > This document was written for: {props.request} @@ -104,12 +106,13 @@ This document was written for: {props.request} It still does not validate after three attempts to repair it. These are the remaining findings: - + + - + @@ -129,4 +132,14 @@ no explanation. Keep the narrative that explains what the document is for. +A `` selects the value this document publishes; it does not end the +document. So approval and exhaustion are written as the two arms of one branch, +and only the arm taken runs — an approved candidate is returned with no later +failure to overtake it, and a review that approved nothing fails instead. + + + + + + diff --git a/packages/cli/tests/prompt-plan.test.ts b/packages/cli/tests/prompt-plan.test.ts new file mode 100644 index 000000000..691ec1a54 --- /dev/null +++ b/packages/cli/tests/prompt-plan.test.ts @@ -0,0 +1,184 @@ +/** + * The packaged plan program, executed as itself. + * + * This runs the exact Markdown the CLI ships — read through the packaged + * loader, not copied into a fixture — so what it proves is what a release does. + * The seams around it are deterministic: a scriptable ACP runtime for the one + * Agent turn, a scripted Elicitation answer for the review, and a test-only + * validator standing in for the host capability that does not exist yet. + * + * The include list is empty on purpose. Repository component search must not be + * able to supply `Loop`, `If`, `Return`, `Fail`, `CodeBlock` or the validator: + * the policy under test is the one in the packaged document, resolved against + * first-party declarations only. + */ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped } from "effection"; +import type { Operation } from "effection"; +import { + agentIdentityComponents, + collect, + Elicitation, + installAgentComponents, + registerAgentProvider, + retainedSource, +} from "@executablemd/core"; +import type { ElicitationRequest, Json } from "@executablemd/core"; +import { executeInstalled } from "@executablemd/core/host"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { createAcpxProvider } from "@executablemd/acp"; + +import { PROMPT_PLAN, readPackagedDocument } from "../src/packaged-document.ts"; +import { AGENT, useWorkingDirectory } from "./support/prompt-harness.ts"; +import { createFakeAcp, makeRegistry, makeStore } from "./support/fake-acp.ts"; + +/** + * A candidate whose exact bytes are worth preserving. + * + * Leading and trailing blank lines, interior indentation, and a five-backtick + * run: enough that trimming, re-fencing, or reconstructing it through the + * presentation would all be visible in the assertion. + */ +const CANDIDATE = [ + "", + "---", + "returns: { type: string }", + "---", + "", + "# A greeting", + "", + "This document explains itself before it does anything.", + "", + "`````markdown", + " ", + "`````", + "", + '', + "", +].join("\n"); + +/** What the plan asked the validator about, in order. */ +interface PlanRun { + validated: string[]; + reviews: ElicitationRequest[]; + prompts: string[]; + value: Json | undefined; + failure: string | undefined; +} + +function* runPlan(): Operation { + const source = yield* readPackagedDocument(PROMPT_PLAN); + const fake = createFakeAcp(); + fake.script({ reply: CANDIDATE }); + + const validated: string[] = []; + const reviews: ElicitationRequest[] = []; + let value: Json | undefined; + let failure: string | undefined; + + yield* scoped(function* () { + const acpx = createAcpxProvider({ + createRuntime: fake.create, + sessionStore: makeStore(), + agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), + }); + yield* registerAgentProvider("acpx", acpx); + yield* installAgentComponents({ + defaultAgent: AGENT, + permissionMode: "deny-all", + rootProvider: { factory: acpx, options: { defaultAgent: AGENT, permissionMode: "deny-all" } }, + }); + + yield* Elicitation.around( + { + // deno-lint-ignore require-yield + *elicit([request], _next) { + reviews.push(request); + return { decision: "approve" }; + }, + }, + { at: "min" }, + ); + + try { + value = yield* collect( + yield* executeInstalled( + { + ...retainedSource("", source), + stream: new InMemoryStream(), + includes: [], + props: { + request: "ask me for my age and write the result to a file", + syntax: "## Built-in components\n\n### ``\n", + session: "plan-regression", + }, + }, + [ + { + components: [ + ...agentIdentityComponents(), + { + name: "ValidateCandidate", + origin: "test", + forms: ["self-closing"] as const, + props: { + type: "object", + properties: { source: { type: "string" } }, + required: ["source"], + additionalProperties: false, + }, + returns: { + type: "object", + properties: { + valid: { type: "boolean" }, + diagnostics: { type: "object" }, + }, + required: ["valid", "diagnostics"], + additionalProperties: false, + }, + // The deterministic seam only. It records what it was asked + // about and says yes; the production validator is later work. + factory: () => + // deno-lint-ignore require-yield + function* validateCandidate(props: Record) { + validated.push(String(props.source)); + return { valid: true, diagnostics: {} }; + }, + }, + ], + }, + ], + ), + ); + } catch (error) { + failure = error instanceof Error ? error.message : String(error); + } + }); + + return { validated, reviews, prompts: fake.prompts, value, failure }; +} + +describe("the packaged plan program", () => { + it("returns the approved candidate's exact bytes and never reaches exhaustion", function* () { + const run = yield* useWorkingDirectory(function* () { + return yield* runPlan(); + }); + + // Approving the first valid candidate is one turn and one question. A + // repair or revision turn here would mean the loops ran when they had + // nothing to fix. + expect(run.prompts).toHaveLength(1); + expect(run.reviews).toHaveLength(1); + + // The validator saw the Agent's complete close value, once, unaltered. + expect(run.validated).toEqual([CANDIDATE]); + + // The plan settled with a value rather than an authored failure. Before the + // control-flow correction this was the ten-draft exhaustion message: a + // `` selects a value but does not end the body, so the unconditional + // `` after the Session ran and won over every approval. + expect(run.failure).toBe(undefined); + expect(run.value).toBe(CANDIDATE); + }); +}); From e06e783c7eb2aea04f476d5e7f5e4da054fcab55 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 14:11:55 -0400 Subject: [PATCH 05/15] =?UTF-8?q?=E2=9C=A8=20Turn=20a=20request=20into=20a?= =?UTF-8?q?=20Plan=20with=20a=20packaged=20command=20document=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `xmd prompt` now executes two roots with a scope boundary between them. First `src/documents/prompt-command.md`, the checked-in first-party Markdown that implements the conversion and its review workflow; then the Plan it returned, through the ordinary supplied-source path. The TypeScript authorship loop — `author()`, `evaluate()` and `review()` — is gone: what the assistant is asked, how many drafts may be fixed, how many you are shown and what happens when you approve nothing are readable Markdown now, and arguable. A Plan is not a script with comments. The shipped generation, repair and revision instructions ask for every requested outcome as reader-facing prose with each component placed immediately after the sentences describing what it does, so the source can be audited and the execution followed by the same narrative. That authorship rule is the document's and human review's; it is never a hidden validation rule. The prompt profile is what that document runs inside. The assistant gets an empty directory of this host's own, no additional directories, no MCP servers, no native tools and a strict denial answered inside the provider — so nothing composed around it can widen a ceiling with nothing in it, and `--approve-all` configures the approved Plan rather than the conversation that wrote it. The document itself reaches no Files, command, service or network capability, runs against no repository component search, and cannot opt out of a failing turn ending it: a turn that streamed half a program and failed presents nothing. `` is declared to that execution rather than resolved from a repository, and it keeps the two kinds of failure apart. A defect a draft authored comes back as structured facts the document may repair; a defect the command line authored raises out of it, so no policy can catch a caller's mistake and call it feedback. The host takes the returned Plan back as untrusted text. Only after every Prompt task, provider and Elicitation resource has torn down does it validate those exact bytes again and resolve their props — a revision that changed a property's declared type changes what the run receives — and only then does `--save` create the file and the Plan run. `--session ` names the assistant session; without it each invocation places its own. The profile's working directory is one fixed host-owned path because a session's key includes the directory it lives in, and a location that changed every time would leave that option unable to name anything. Evidence runs the packaged document itself against a scripted agent and a scripted review: the shipped words reach the turn, ten presentations bound the review with no revision on the tenth, an interleaved Plan returns byte for byte, and what a person reads says each thing once however many rounds it took. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- .github/workflows/release.yml | 2 +- architecture.md | 6 +- deno.json | 2 +- packages/cli/src/agent-stack.ts | 60 +- packages/cli/src/cli.ts | 31 +- packages/cli/src/documents/prompt-command.md | 168 +++++ packages/cli/src/documents/prompt-plan.md | 145 ---- packages/cli/src/packaged-document.ts | 4 +- packages/cli/src/prompt-args.ts | 27 +- packages/cli/src/prompt-profile.ts | 339 +++++++++ packages/cli/src/prompt.ts | 544 +++++---------- packages/cli/tests/packaged-document.test.ts | 20 +- packages/cli/tests/prompt-args.test.ts | 16 +- packages/cli/tests/prompt-cli.test.ts | 59 +- ...est.ts => prompt-command-document.test.ts} | 34 +- packages/cli/tests/prompt.test.ts | 645 ++++++++++++------ scripts/tests/cli-npm-bin.test.ts | 4 +- specs/acp-client-spec.md | 83 ++- specs/executable-mdx-spec.md | 55 +- specs/prompt-command-spec.md | 528 ++++++++------ specs/release-process-spec.md | 13 +- specs/root-document-props-spec.md | 30 +- 22 files changed, 1685 insertions(+), 1130 deletions(-) create mode 100644 packages/cli/src/documents/prompt-command.md delete mode 100644 packages/cli/src/documents/prompt-plan.md create mode 100644 packages/cli/src/prompt-profile.ts rename packages/cli/tests/{prompt-plan.test.ts => prompt-command-document.test.ts} (81%) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bc5db4ef3..c9cb72f06 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -98,7 +98,7 @@ jobs: --exclude-unused-npm \ --allow-all \ --include packages/code-review-agent \ - --include packages/cli/src/documents/prompt-plan.md \ + --include packages/cli/src/documents/prompt-command.md \ --target ${{ matrix.target }} \ --output dist/${{ matrix.artifact }} \ packages/cli/src/compiled.ts diff --git a/architecture.md b/architecture.md index 1529c6895..1b067a74a 100644 --- a/architecture.md +++ b/architecture.md @@ -33,6 +33,10 @@ Existing documents and code get aligned to this section retroactively. | Repository base | the optional Git revision from which one named Workspace Repository initializes its primary checkout | | pinned commit | the commit obtained by resolving a base once; it remains the workflow run's starting repository state even as the run creates descendant commits | | document target | an addressable static heading in a root document's own Markdown flow, named by the canonical path of heading labels that reaches it; selecting one executes the preamble, each ancestor's own content, and that heading's complete subtree | +| Prompt | a person's original request, in ordinary natural language. `xmd prompt` takes exactly one | +| Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. A Plan is what `xmd prompt` approves, optionally saves and runs; it is not a synonym for a workflow, a policy document or any executable Markdown file | +| prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the request wording, the draft and repair loops, the check branches, human review, revision, approval, abort and exhaustion — and returns the exact approved Plan source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | +| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line | | expansion | one logical evaluation of an authored executable element within a document execution | | expansion ID | a deterministic identifier for one logical expansion; restoring or retrying that expansion preserves the ID, while a distinct evaluation requested by the document receives another | | Git capability | the contextual interface through which workflow infrastructure queries the Git repository associated with the current working directory | @@ -3541,7 +3545,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd prompt` | asks the configured agent for a complete executable Markdown root, validates it without executing any of it, repairs its definite defects by asking again at most three times per draft, shows a person the exact bytes, and executes the approved ones through the ordinary supplied-source path under the `` identity. Authorship sits outside durability: the catalog, the fresh generator session, every repair and the review create no journal, retain no durable identity and replay nothing, and the generator's scope closes before the optional exclusive save and before execution. What runs is one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A defect the draft authored is repairable; one the command line authored terminates without spending a turn on it | built on the #656 stack | +| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of an empty host-owned working directory with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/deno.json b/deno.json index e89db7542..322b3d94b 100644 --- a/deno.json +++ b/deno.json @@ -57,7 +57,7 @@ "verify:clean": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/verify-clean.ts", "deps": "deno run --allow-all scripts/deps.ts", "deps:target": "deno run --allow-all scripts/deps-target.ts", - "build": "deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --include packages/cli/src/documents/prompt-plan.md --output dist/xmd packages/cli/src/compiled.ts", + "build": "deno task build:web && deno compile --node-modules-dir=none --cached-only --frozen --exclude-unused-npm --allow-all --include packages/code-review-agent --include packages/cli/src/documents/prompt-command.md --output dist/xmd packages/cli/src/compiled.ts", "build:web": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/preflight.ts scripts/build-web-client.ts", "gen:publish-workflow": "deno run --allow-all packages/cli/src/deno.ts run scripts/gen-publish-workflow.md", "bump": "deno run -A scripts/bump-version.ts", diff --git a/packages/cli/src/agent-stack.ts b/packages/cli/src/agent-stack.ts index df1d99c12..3aca46d7e 100644 --- a/packages/cli/src/agent-stack.ts +++ b/packages/cli/src/agent-stack.ts @@ -6,9 +6,9 @@ * options, so they resolve them once, here, rather than each reading the flags * again. What they do with the result differs, and deliberately: a run installs * the registered provider into the Agent Api so a document may reach it, while - * prompt generation holds a provider handle of its own and installs no document - * components, no root provider and no foreground launcher — it needs turns, not - * authority. + * `xmd prompt` hands the same answer to two consumers — the prompt profile, which + * takes the provider name and the default agent and nothing else, and the + * approved document, which runs the ordinary run stack. * * `agent-config.ts` stays the pure flag-to-permission mapping. This module is * where the environment, the provider registry and the host's own machine @@ -16,15 +16,14 @@ */ import { - Agent, installAgentComponents, installPermissionMode, registerAgentProvider, } from "@executablemd/core"; import type { AgentProviderFactory, PermissionMode } from "@executablemd/core"; import { installForegroundLauncher, env as readEnv } from "@executablemd/runtime"; -import { createAcpxProvider, DEFAULT_AGENT_NAME, useAcpxProvider } from "@executablemd/acp"; -import type { AcpxProvider, AcpxProviderDependencies } from "@executablemd/acp"; +import { createAcpxProvider, DEFAULT_AGENT_NAME } from "@executablemd/acp"; +import type { AcpxProviderDependencies } from "@executablemd/acp"; import { Err, Ok } from "effection"; import type { Operation, Result } from "effection"; @@ -123,52 +122,3 @@ export function* installRunAgentStack(stack: AgentStack): Operation { // reaches under any of them refuses instead of spawning. yield* installForegroundLauncher(); } - -/** - * A provider that can run turns, and nothing else. - * - * `xmd prompt` writes a document before one exists, so there is nothing to - * install components into and no launch for anyone to authorize. It takes the - * provider's operations directly and answers only the one Agent operation the - * provider itself performs — resolving the agent a session and a turn belong - * to. Everything else on the Agent Api stays at its base behavior, so a launch - * or a document prompt reaching this scope is refused rather than served. - * - * The permission policy is installed first, so it encloses the provider state's - * own scope and answers the native permission requests generation raises with - * the same `--approve-all`, `--approve-reads` or `--deny-all` choice a run uses. - * - * `instructions` become the fresh session's system prompt. They reach ACPX - * through `newSessionOptions`, which it applies when it creates a session and - * ignores when it reuses a record. - */ -export function* useGeneratorAgent( - stack: AgentStack, - instructions: string, - dependencies?: AcpxProviderDependencies, -): Operation { - yield* installPermissionMode(stack.permissionMode); - - let provider: AcpxProvider | undefined; - yield* Agent.around( - { - *agent([name], _next) { - if (provider === undefined) { - throw new Error("the generator agent provider is not established yet"); - } - return yield* provider.agent(name); - }, - }, - { at: "min" }, - ); - - provider = yield* useAcpxProvider( - { defaultAgent: stack.defaultAgent, permissionMode: stack.permissionMode }, - { - ...hostAcpDependencies(stack.sessions), - ...dependencies, - newSessionOptions: { systemPrompt: instructions }, - }, - ); - return provider; -} diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 151731396..7528c68c4 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -109,7 +109,7 @@ import { resolvePropsFromSources, } from "./props.ts"; import type { Binding, Extraction } from "./props.ts"; -import { namesPrompt, scanPromptArgs, SAVE_OPTION } from "./prompt-args.ts"; +import { namesPrompt, scanPromptArgs, SAVE_OPTION, SESSION_OPTION } from "./prompt-args.ts"; import type { PromptScan } from "./prompt-args.ts"; import { runPrompt } from "./prompt.ts"; import type { PromptExecution } from "./prompt.ts"; @@ -256,6 +256,10 @@ const promptConfig = object({ description: "write the approved document here before running it (path must not exist)", ...field(z.string().optional()), }, + session: { + description: "logical name for the assistant session (default: unique to this invocation)", + ...field(z.string().optional()), + }, ...executionFields, }); @@ -907,9 +911,9 @@ export interface PromptExecutionConfig { * How `xmd prompt` runs the document a person approved: exactly as `xmd run` * runs a supplied one. * - * The generator's scope is already gone by the time this is called, so the + * The prompt profile's scope is already gone by the time this is called, so the * executed program gets a fresh ordinary Agent provider and inherits neither - * the generator session nor its system prompt. The browser form is composed + * the assistant session nor its instruction layer. The browser form is composed * around the document for the same reason a run composes one: `xmd prompt` is a * command a person is sitting in front of. */ @@ -938,7 +942,7 @@ export function promptExecutor( props: approved.props, ...(sessions === undefined ? {} : { machineSessions: sessions }), // The same object generation was configured from. The provider it - // installs is a fresh ordinary one — the generator's scope is already + // installs is a fresh ordinary one — the profile's scope is already // gone — but which agent it defaults to and what it may do were // decided once, for the whole invocation. agent: stack, @@ -1659,8 +1663,11 @@ const PROMPT_REQUEST_HELP = [ "path. Quote it so the shell passes it as a single argument:", ' xmd prompt "ask me for my age and write the result to a file"', "", - "The agent writes a complete document, xmd validates it and repairs definite", - "defects, and you approve, revise or abort before anything runs.", + "A first-party command document turns your request into a Plan: an executable", + "Markdown document that states what you asked for in ordinary language and", + "places the components that do the work alongside those words. xmd checks each", + "draft and repairs definite defects; you approve, revise or abort before", + "anything runs.", "", "Document properties follow the request. The individual options a document", "declares are the generated document's, so they are not listed here:", @@ -1673,6 +1680,13 @@ const PROMPT_REQUEST_HELP = [ ` ${SAVE_OPTION} `, " Write the approved document there before running it. The path must not", " exist; an existing one is left alone and the run stops.", + "", + ` ${SESSION_OPTION} `, + " Use this logical assistant session instead of one unique to this run.", + "", + "Permission flags configure the approved document. Writing the Plan is a", + "conversation about text and gives the assistant nothing: an empty directory of", + "its own, no tools, no MCP servers, and every native permission request denied.", ].join("\n"); /** @@ -1900,7 +1914,7 @@ function* dispatch( yield* exit(1); break; } - // Once, here, and handed to both consumers below. Generation and the + // Once, here, and handed to both consumers below. Authorship and the // execution that follows it are one invocation, so they answer to one // `--default-agent`, one `DEFAULT_AGENT_NAME` and one permission mode. const promptStack = yield* settleAgentStack( @@ -1922,6 +1936,7 @@ function* dispatch( scan, include: config.include, ...(config.save === undefined ? {} : { save: config.save }), + ...(config.session === undefined ? {} : { session: config.session }), stack: promptStack, }, { @@ -2137,7 +2152,7 @@ export function* runXmd( const provisional = xmd.parse({ args: helpRequest.args }); const selected = provisional.ok ? provisional.value.config : undefined; // The two commands that end in a document execution. `xmd prompt`'s deadline - // encloses more than a run's — the catalog, the generator session, every + // encloses more than a run's — the catalog, the assistant session, every // repair, the human review, provider teardown, the save and the execution — // because all of it is what the caller asked to be bounded. const executes = diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md new file mode 100644 index 000000000..923bf9592 --- /dev/null +++ b/packages/cli/src/documents/prompt-command.md @@ -0,0 +1,168 @@ +--- +props: + type: object + properties: + request: { type: string } + syntax: { type: string } + session: { type: string } + required: [request, syntax, session] + additionalProperties: false +returns: + type: string +--- + +# What `xmd prompt` does + +`xmd prompt` turns your request into a Plan: an Executable Markdown document +that states what you asked for in ordinary language and places the components +that perform the work alongside those words. + +This document is how that happens. It is a program rather than a setting: what +the assistant is asked for, how many times a draft may be fixed, what you are +shown, and what happens when you approve nothing are all written here, in the +open, where you can read them and argue with them. + +The command gives it three things — your request as you typed it, the components +available in this directory, and the name of the assistant session every turn +belongs to. Nothing here runs the document being written. A draft is text until +you approve it, and the command runs it separately afterwards. + + + + + + + +Write one complete Executable Markdown document that does this: + +{props.request} + +The document you write is a Plan. A Plan is not a script with comments: it says +what is being done in ordinary reader-facing prose, and places each component +immediately after the sentences describing the action that component performs. +The prose is part of the program and part of what the document prints when it +runs, so somebody watching it run follows the same words somebody reading the +source did. + +For the request "ask me for my age and write it to a file", the shape is: + +```markdown +Ask me for my age. + + + +Write it to a file. + +{answer.age} +``` + +You do not have to repeat the request word for word. Divide it, clarify it and +rewrite it into natural prose — but keep every outcome it asked for, keep them in +an order that makes sense, and keep each one next to the component that carries +it out. Somebody should be able to audit the source and follow the execution by +the same narrative. + +Everything you may use is described below. Use nothing that is not here. + +{props.syntax} + +Reply with the document source and nothing else. No enclosing code fence, no +explanation before or after it, no commentary about what you did. + + + + + +Each draft is checked once, and gets up to three chances to fix what the check +found. A change you ask for produces a new draft, so it gets its own three. + + + + + + + + + +That document has problems. These are the exact ones: + + + + +Send one complete replacement document that resolves every problem above. Keep +the reader-facing prose that says what the document is for, and keep each +component immediately after the sentences describing what it does. Source only, +no enclosing fence, no explanation. + + + + + +You decide what happens next. A draft that passed its check can be approved; one +that still has problems after three attempts is shown with them, and can only be +sent back or abandoned. On the tenth and last time you are asked there is +nothing left to change it into, so asking for a change is not offered. + + +This document was written for: {props.request} + + + + +Three attempts to fix it did not clear everything. These problems remain: + + + + + + + + + + + + + + + + +The person who asked for this document read it and wants this changed: + +{review.feedback} + +Send one complete replacement document. Keep the reader-facing prose that says +what the document is for, and keep each component immediately after the +sentences describing what it does. Source only, no enclosing fence, no +explanation. + + + + + +If you approved a document, return its source unchanged. Otherwise, stop without +saving or running anything. + + + + + + + diff --git a/packages/cli/src/documents/prompt-plan.md b/packages/cli/src/documents/prompt-plan.md deleted file mode 100644 index 3af290e82..000000000 --- a/packages/cli/src/documents/prompt-plan.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -props: - type: object - properties: - request: { type: string } - syntax: { type: string } - session: { type: string } - required: [request, syntax, session] - additionalProperties: false -returns: - type: string ---- - -# The plan `xmd prompt` follows - -This is the policy behind `xmd prompt`, and it is a program rather than a -setting: what the agent is asked, how many times a draft may be repaired, what a -person is shown, and what happens when nobody approves anything are all written -here, in the open, where they can be read and argued with. - -The command supplies three things — the request as the person typed it, the -syntax catalog for this working directory, and the name of the session every -turn belongs to. Nothing here runs the document being written. A candidate is -text until a person approves it and the command runs it separately. - - - - - - - -Write one complete Executable Markdown root document that accomplishes this: - -{props.request} - -Reply with the document source and nothing else. No enclosing code fence, no -explanation before or after it, no commentary about what you did. - -Write it as a document a person would want to read, not as a script with -comments bolted on. State what it does and why in ordinary prose, and let the -components appear where that prose leads to them. The person asked for the -outcome above; the document should still read as though it is about that -outcome. - -Everything you may use is described below. Use nothing that is not here. - -{props.syntax} - - - - - -Each draft gets one validation and up to three chances to fix what validation -found. A revision the person asked for is a new draft, so it gets its own three. - - - - - - - - - -That document does not validate. These are the exact findings: - - - - -Return one complete replacement root document that resolves every finding above. -Source only, no enclosing fence, no explanation. Keep the narrative that explains -what the document is for. - - - - - -The person decides. A valid draft can be approved; one that is still invalid -after three repairs is shown with its findings and can only be sent back or -abandoned. On the tenth and last presentation there is nothing left to revise -into, so revision is not offered. - - -This document was written for: {props.request} - - - - -It still does not validate after three attempts to repair it. These are the -remaining findings: - - - - - - - - - - - - - - - - -The person read that document and asked for this to change: - -{review.feedback} - -Return one complete replacement root document. Source only, no enclosing fence, -no explanation. Keep the narrative that explains what the document is for. - - - - - -A `` selects the value this document publishes; it does not end the -document. So approval and exhaustion are written as the two arms of one branch, -and only the arm taken runs — an approved candidate is returned with no later -failure to overtake it, and a review that approved nothing fails instead. - - - - - - - diff --git a/packages/cli/src/packaged-document.ts b/packages/cli/src/packaged-document.ts index daa1b19aa..4a99111c8 100644 --- a/packages/cli/src/packaged-document.ts +++ b/packages/cli/src/packaged-document.ts @@ -20,8 +20,8 @@ import { readFile } from "node:fs/promises"; import { until } from "effection"; import type { Operation } from "effection"; -/** The plan program `xmd prompt` executes. */ -export const PROMPT_PLAN = "prompt-plan.md"; +/** The document `xmd prompt` executes to turn a request into a Plan. */ +export const PROMPT_COMMAND_DOCUMENT = "prompt-command.md"; /** * Where a packaged document lives, as a URL beside this module. diff --git a/packages/cli/src/prompt-args.ts b/packages/cli/src/prompt-args.ts index a9d49397e..83708898e 100644 --- a/packages/cli/src/prompt-args.ts +++ b/packages/cli/src/prompt-args.ts @@ -21,6 +21,7 @@ import type { Binding } from "./props.ts"; export const PROMPT_COMMAND = "prompt"; export const SAVE_OPTION = "--save"; +export const SESSION_OPTION = "--session"; /** The built-in options that take a separated value. */ const VALUE_OPTIONS: readonly string[] = [ @@ -33,6 +34,7 @@ const VALUE_OPTIONS: readonly string[] = [ "--timeout-exec", "--timeout-fetch", SAVE_OPTION, + SESSION_OPTION, ]; /** The built-in options that take none. */ @@ -171,9 +173,30 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { continue; } + const separated = + equals === -1 && VALUE.has(name) && args[index + 1] !== undefined + ? args[index + 1] + : undefined; + // Read here rather than after parsing, because an empty value is exactly + // what the parser cannot report: an option it reads as absent falls back + // to the default, so a caller who asked for a session and named none would + // silently get the generated one instead. + if (name === SESSION_OPTION) { + const value = equals === -1 ? separated : token.slice(equals + 1); + if (value === undefined || value.length === 0) { + return { + ...(request === undefined ? {} : { request }), + fixed, + occurrences, + error: + `${SESSION_OPTION} needs a name — write \`${SESSION_OPTION} \` or leave ` + + "it out for a session unique to this invocation", + }; + } + } fixed.push(token); - if (equals === -1 && VALUE.has(name) && args[index + 1] !== undefined) { - fixed.push(args[index + 1]); + if (separated !== undefined) { + fixed.push(separated); index += 2; continue; } diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts new file mode 100644 index 000000000..aa67369c3 --- /dev/null +++ b/packages/cli/src/prompt-profile.ts @@ -0,0 +1,339 @@ +/** + * The prompt profile — the trusted-host assembly the prompt command document + * runs under, and the only thing that ever runs under it + * (specs/prompt-command-spec.md). + * + * `xmd prompt` owns two root document executions with a complete scope boundary + * between them. This module is the first one. It supplies that document's + * inputs, a + * constrained Agent provider, Elicitation, the fixed first-party components and + * the host-declared candidate validator, and it exposes no custom root and no + * repository component search: the program it runs is the one the CLI ships. + * + * The Agent ceiling is assembled here rather than read from the command line, + * because it is not the caller's to choose. Writing a Plan is a conversation + * about text; it never lets an agent touch anything. So the provider gets a fresh empty + * directory of this host's own, no additional directories, no MCP servers, an + * empty native-tool allowlist and a private strict denial of every native + * permission request — one that answers inside the provider and consults no + * authored approval scope, so nothing composed around it can widen a ceiling + * with nothing in it. `--approve-all`, `--approve-reads` and `--deny-all` + * configure the approved document, later, and reach none of this. + * + * The document itself is given no Files, command, service or XMD-mediated + * network capability either. It decides what to write; it writes nothing. + */ + +import { Err, Ok, scoped, until } from "effection"; +import type { Operation, Result } from "effection"; +import { mkdir } from "node:fs/promises"; +import { homedir } from "node:os"; +import { join } from "node:path"; + +import { + agentIdentityComponents, + collect, + installAgentComponents, + installPermissionMode, + installPromptFailurePolicy, + registerAgentProvider, + retainedSource, +} from "@executablemd/core"; +import type { Json } from "@executablemd/core"; +import type { IdentityComponent } from "@executablemd/core/host"; +import { executeInstalled } from "@executablemd/core/host"; +import { createAcpxProvider } from "@executablemd/acp"; +import type { AcpxProviderDependencies } from "@executablemd/acp"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { API } from "@executablemd/runtime"; + +import { hostAcpDependencies } from "./agent-stack.ts"; +import type { AgentStack } from "./agent-stack.ts"; +import { PROMPT_COMMAND_DOCUMENT, readPackagedDocument } from "./packaged-document.ts"; + +/** + * The identity the prompt command document runs under. + * + * Stable and internal: no path selects it, no include resolves it, and a + * position reading `(:12:1)` says the source is the CLI's own. + */ +export const PROMPT_COMMAND_IDENTITY = ""; + +/** + * The permission mode the prompt command document runs under. + * + * Fixed rather than configured. The provider denies native requests privately, + * so this is what an authored approval scope inside that document would + * compose around if one existed — and the honest answer for a profile that + * grants no native authority is the one that grants none. + */ +const PROFILE_PERMISSION_MODE = "deny-all"; + +/** The closed answer the host gives about one candidate. */ +export interface CandidateAssessment { + valid: boolean; + /** Empty when valid; the complete structured findings when not. */ + diagnostics: Json; +} + +/** What the host supplies to one prompt command document execution. */ +export interface PromptProfile { + /** The request as the person typed it. */ + request: string; + /** The rendered syntax catalog for this run profile and these includes. */ + syntax: string; + /** The logical name every turn in this invocation belongs to. */ + session: string; + /** The one Agent configuration this invocation settled. */ + stack: AgentStack; + /** What the constrained provider is built on, beyond the host's assembly. */ + acp?: AcpxProviderDependencies; + /** Who answers the review question. */ + installElicitation(): Operation; + /** + * The host's assessment of one candidate. + * + * A candidate-authored failure comes back as `valid: false` and is repairable. + * A caller-source failure raises, which ends that execution: the document + * has no way to catch it and no way to recategorize it as feedback + * for an agent that could not have caused it. + */ + assess(source: string): Operation; +} + +/** + * Run the packaged prompt command document and answer with the Plan it approved. + * + * Every resource this builds lives inside one scope, so leaving it is what tears + * the Prompt tasks, the provider and the Elicitation provider down. A teardown + * failure raises out of here rather than being folded into the result, because + * a failure to release is not an outcome the source that was selected survives. + */ +export function* runPromptCommandDocument(profile: PromptProfile): Operation> { + // Before a directory exists, before a provider exists, and therefore before + // any session could be placed or any turn started. A host that cannot + // establish this ceiling refuses rather than writing a Plan under a weaker one. + if (profile.stack.provider !== "acpx") { + return Err( + new Error( + `the ${profile.stack.provider} provider cannot establish the prompt profile's ` + + "ceiling — nothing was written or run", + ), + ); + } + + return yield* scoped(function* (): Operation> { + const workdir = yield* useProfileDirectory(); + yield* refuseDocumentCapabilities(); + yield* profile.installElicitation(); + + const acpx = createAcpxProvider(profileCeiling(profile, workdir)); + yield* registerAgentProvider("acpx", acpx); + const options = { + defaultAgent: profile.stack.defaultAgent, + permissionMode: PROFILE_PERMISSION_MODE, + } as const; + yield* installAgentComponents({ ...options, rootProvider: { factory: acpx, options } }); + yield* installPermissionMode(PROFILE_PERMISSION_MODE); + // A candidate comes from a turn's complete successful close value or from + // nowhere. `` ordinarily renders whatever a failed turn managed to + // emit and carries on, which for a policy that reviews source would mean + // showing a person half a program; the host decides otherwise here, so a + // failed, cancelled or protocol-invalid turn ends the command before anything + // is presented. The document cannot opt out of it. + yield* installPromptFailurePolicy(function* () { + return true; + }); + + const source = yield* readPackagedDocument(PROMPT_COMMAND_DOCUMENT); + try { + const approved = yield* collect( + yield* executeInstalled( + { + ...retainedSource(PROMPT_COMMAND_IDENTITY, source), + // Invocation-owned and thrown away with the scope. Ordinary document + // and Prompt semantics need a durable stream; nothing about writing a + // Plan needs a durable one, and `--journal` belongs to the Plan you + // approved rather than to the conversation that wrote it. + stream: new InMemoryStream(), + // No repository component search. What the document may name is + // what this profile declares, so a file in the caller's tree cannot + // answer for ``, `` or anything else. + includes: [], + props: { + request: profile.request, + syntax: profile.syntax, + session: profile.session, + }, + }, + [{ components: [...agentIdentityComponents(), validator(profile)] }], + ), + ); + if (typeof approved !== "string") { + return Err(new Error("the prompt command document returned something that is not a Plan")); + } + return Ok(approved); + } catch (error) { + return Err(error instanceof Error ? error : new Error(String(error))); + } + }); +} + +/** + * The host-declared candidate validator, as an internal value component. + * + * Declared to the execution, so canonical execution supplies its invocation + * identity and repository resolution cannot replace it. It executes nothing it + * is given: a candidate is a string here, and stays one until a person has + * approved it and the host has validated it again. + */ +function validator(profile: PromptProfile): IdentityComponent { + return { + name: "ValidateCandidate", + origin: "xmd prompt", + forms: ["self-closing"] as const, + props: { + type: "object", + properties: { source: { type: "string" } }, + required: ["source"], + additionalProperties: false, + }, + returns: { + type: "object", + properties: { valid: { type: "boolean" }, diagnostics: { type: "object" } }, + required: ["valid", "diagnostics"], + additionalProperties: false, + }, + factory: () => + function* validateCandidate(props: Record) { + const assessment = yield* profile.assess(String(props.source)); + return { valid: assessment.valid, diagnostics: assessment.diagnostics }; + }, + }; +} + +/** + * The Agent ceiling, stated as the dependencies the provider is built from. + * + * Each entry is the whole of one clause: the working directory the agent runs + * in, the MCP servers it configures, the native tools a fresh session may use, + * and who answers a native permission request. `mcpServers: []` and + * `allowedTools: []` are statements rather than omissions — leaving either off + * is the backend's default, which is not this host's. + * + * The host's own assembly is passed through, then overridden: a coordinator or + * a route store says who owns a session, which this profile still has to respect, + * while nothing a caller wrote may reach the four fields below. + */ +function profileCeiling(profile: PromptProfile, workdir: string): AcpxProviderDependencies { + return { + ...hostAcpDependencies(profile.stack.sessions), + ...profile.acp, + // deno-lint-ignore require-yield + *agentCwd() { + return workdir; + }, + mcpServers: [], + permissions: "strict", + newSessionOptions: { systemPrompt: PROMPT_INSTRUCTIONS, allowedTools: [] }, + }; +} + +/** + * What the assistant session is told once, before it is asked anything. + * + * The host owns this layer, and it owns only this: the shape of every answer, + * which is the one thing a policy cannot state about itself convincingly. What + * to write, which vocabulary is available, how to repair a draft and what a + * person asked to change are the prompt command document's text, and they arrive + * in the turns it sends. + */ +export const PROMPT_INSTRUCTIONS = [ + "You write complete executable Markdown root documents. Each user message is a", + "request about one document.", + "", + "Every answer is replacement document source and nothing else: no enclosing code", + "fence, no explanation, and no prose around it. What you return is validated and", + "executed exactly as you wrote it.", +].join("\n"); + +/** + * Where the assistant runs while it writes a Plan: this host's own directory, + * and an empty one. + * + * The caller's working directory is deliberately not offered — an agent writing + * a document has no reason to read the tree it will run in, and a ceiling that + * starts at the caller's checkout is not a ceiling. Nothing this profile grants + * can write here either, so what is created empty stays empty. + * + * One fixed path rather than a new directory each time, because a session's + * identity includes the directory it lives in: a location that changed every + * invocation would mean `--session` could never name a conversation that + * already exists, which is the whole of what that option is for. + */ +export const PROMPT_PROFILE_DIRECTORY: string = join(homedir(), ".xmd", "prompt"); + +function* useProfileDirectory(): Operation { + yield* until(mkdir(PROMPT_PROFILE_DIRECTORY, { recursive: true })); + return PROMPT_PROFILE_DIRECTORY; +} + +/** + * The capabilities the prompt command document does not get. + * + * Installed above whatever the entrypoint provided, so the document is + * refused rather than served. It decides what to write; writing a file, + * running a command, starting a service and reaching the network are all the + * approved document's business, under the caller's own configuration. + */ +function* refuseDocumentCapabilities(): Operation { + const refuse = (capability: string) => () => { + throw new Error( + `xmd prompt asked for ${capability}, which the prompt profile grants to nothing`, + ); + }; + yield* API.Files.around({ + // deno-lint-ignore require-yield + *checkFilePath() { + return refuse("a file")(); + }, + // deno-lint-ignore require-yield + *readTextFile() { + return refuse("a file")(); + }, + // deno-lint-ignore require-yield + *writeTextFile() { + return refuse("a file")(); + }, + // deno-lint-ignore require-yield + *deleteFile() { + return refuse("a file")(); + }, + // deno-lint-ignore require-yield + *globFiles() { + return refuse("a file")(); + }, + // deno-lint-ignore require-yield + *temporaryDirectory() { + return refuse("a directory")(); + }, + }); + yield* API.Process.around({ + // deno-lint-ignore require-yield + *exec() { + return refuse("a command")(); + }, + }); + yield* API.Fetch.around({ + // deno-lint-ignore require-yield + *fetch() { + return refuse("the network")(); + }, + }); + yield* API.Service.around({ + // deno-lint-ignore require-yield + *start() { + return refuse("a service")(); + }, + }); +} diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index d86b4b0b1..8e561931d 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -1,32 +1,34 @@ /** - * `xmd prompt` — ask the configured agent for a document, prove it is one, let - * a person decide, and then run it (specs/prompt-command-spec.md). + * `xmd prompt` — the trusted host around the prompt command document + * (specs/prompt-command-spec.md). * - * The command adds authorship around one ordinary document; it adds no second - * execution model. Everything before approval happens with no journal, no - * durable identity and no execution: generation is a conversation, and a - * conversation is not a run. - * - * The lifecycle is one ordered ownership chain, and no phase after the first - * failure begins: + * The command owns two root document executions with a complete scope boundary + * between them: * * ```text - * fixed CLI preflight - * -> structured syntax catalog - * -> fresh generator session - * -> candidate props + validation - * -> automatic repair, at most three turns - * -> live approval or human revision - * -> generator teardown - * -> optional exclusive save - * -> ordinary in-memory document execution + * fixed command preflight + * -> build the run-profile syntax catalog + * -> execute the exact packaged prompt command document + * -> await that execution and provider teardown + * -> validate the returned source again + * -> optionally save the exact bytes + * -> execute retainedSource("", source) through the ordinary run path * ``` * + * What a person is asked, how many drafts may be repaired, how many may be + * reviewed and what happens when nobody approves anything are not here. They are + * in `src/documents/prompt-command.md`, written in the open, where they can be + * read and argued with. This module is what a policy cannot be trusted to do for + * itself: settle the command line, build the ceiling the assistant runs under, + * answer honestly about a draft, and hold the boundary between text an agent + * wrote and a program this host runs. + * * Two kinds of failure are told apart throughout, because they have different - * remedies. A *candidate* failure is something the agent wrote, so the agent is - * asked to write it again with the validation facts. A *caller* failure is - * something the command line or the environment said, so it terminates: teaching - * the agent about it would spend a repair turn on a defect the agent cannot fix. + * remedies. A *draft* failure is something the agent wrote, so the prompt + * command document is told the facts and may ask for another draft. A *caller* + * failure is something the command line or the environment said, so it raises + * out of the validator and ends that execution: no draft the agent could write + * would fix it, and a policy that could catch it could call it feedback. */ import { Err, Ok, scoped, until } from "effection"; @@ -38,7 +40,6 @@ import { resolve } from "node:path"; import { agentIdentityComponents, - elicit, inspectDocument, retainedSource, validateDocument, @@ -49,14 +50,14 @@ import type { Json, PropsSchema, RootDocumentSource, - Session, SyntaxCatalog, } from "@executablemd/core"; -import type { AcpxProvider, AcpxProviderDependencies } from "@executablemd/acp"; +import type { AcpxProviderDependencies } from "@executablemd/acp"; import { cwd } from "@executablemd/runtime"; -import { useGeneratorAgent } from "./agent-stack.ts"; import type { AgentStack } from "./agent-stack.ts"; +import { runPromptCommandDocument } from "./prompt-profile.ts"; +import type { CandidateAssessment } from "./prompt-profile.ts"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { buildBindings, @@ -76,7 +77,7 @@ import { import type { OptionSignature, PromptScan } from "./prompt-args.ts"; /** - * The identity generated text runs under. + * The identity approved text runs under. * * Deliberate and fixed, the way `` is for an inline document: the bytes * came from an agent, not from a file, and a source position reading @@ -86,21 +87,12 @@ import type { OptionSignature, PromptScan } from "./prompt-args.ts"; */ export const PROMPT_IDENTITY = ""; -/** How many replacement drafts one candidate may be asked for automatically. */ -export const REPAIR_TURNS = 3; - /** The approved bytes, and the props resolved under exactly those bytes. */ export interface PromptExecution { root: RootDocumentSource; props: Record; } -/** What review approved: the exact candidate, and its own resolved props. */ -interface ApprovedPrompt { - source: string; - props: Record; -} - /** What one `xmd prompt` invocation was asked to do. */ export interface PromptCommand { /** The argv this invocation holds, and the props source for every candidate. */ @@ -110,6 +102,8 @@ export interface PromptCommand { include: string[]; /** Where the approved source is written before it runs, when asked for. */ save?: string; + /** The logical assistant-session name, when the caller chose one. */ + session?: string; /** * The Agent configuration this invocation settled, before the command ran. * @@ -124,7 +118,7 @@ export interface PromptCommand { export interface PromptDependencies { /** What this host states about machine-wide agent sessions, if anything. */ sessions?: MachineSessionAssembly; - /** What the generator provider is built on, beyond the host's own assembly. */ + /** What the profile's provider is built on, beyond the host's own assembly. */ acp?: AcpxProviderDependencies; /** The run profile's complete structured vocabulary. */ catalog(includes: readonly string[]): Operation; @@ -153,16 +147,6 @@ type CandidateOutcome = | { kind: "repairable"; defects: CandidateDefects } | { kind: "terminal"; error: Error }; -/** A candidate offered for review, and what may be decided about it. */ -type ReviewSubject = - | { candidate: string; valid: true; props: Record } - | { candidate: string; valid: false; defects: CandidateDefects }; - -interface ReviewDecision { - decision: "approve" | "revise" | "abort"; - feedback?: string; -} - /** The codes that say the root's own declaration could not be read. */ const DECLARATION_CODES: ReadonlySet = new Set([ "source-unreadable", @@ -173,8 +157,6 @@ const DECLARATION_CODES: ReadonlySet = new Set; + // Every supplied individual option's shape, as the first draft that bound it + // declared it. Frozen while the Plan is being written and carried into the + // final gate, so the bytes that run are checked against the command line that + // was written rather than against whichever draft happened to be last. + const frozen = new Map(); + + // The command document lives and dies inside that call's scope. Leaving it + // closes the Prompt tasks, the provider and the Elicitation provider, so a + // teardown failure raises out here — before the validation, the save and the + // execution that would otherwise already have happened. + let authored: Result; try { - approved = yield* scoped(function* (): Operation> { - yield* deps.installElicitation(); - // The same vocabulary the catalog just described. Validation and the - // catalog read one registry, so a component the generator was told about - // is one validation resolves. - yield* useRunProfileRegistry(); - const provider = yield* useGeneratorAgent(command.stack, instructions, deps.acp); - const session = yield* provider.session(generatorSessionName()); - return yield* author(command, provider, session, request); + authored = yield* runPromptCommandDocument({ + request, + syntax, + session: command.session ?? invocationSessionName(), + stack: command.stack, + ...(deps.acp === undefined ? {} : { acp: deps.acp }), + installElicitation: deps.installElicitation, + assess: (source) => assess(command, frozen, source), }); } catch (error) { console.error(describeError(error)); return 1; } - if (!approved.ok) { - console.error(approved.error.message); + if (!authored.ok) { + console.error(authored.error.message); return 1; } - const { source, props } = approved.value; + // The returned Plan is untrusted again. Whatever the command document concluded + // about a candidate, these are the bytes that would run, and they are checked + // as though nothing had ever validated them. + const admitted = yield* assessCandidate(command, frozen, authored.value); + if (admitted.kind === "terminal") { + console.error(admitted.error.message); + return 1; + } + if (admitted.kind === "repairable") { + console.error( + `the approved document does not validate:\n${JSON.stringify(admitted.defects, null, 2)}`, + ); + return 1; + } + + const source = authored.value; if (command.save !== undefined) { const saved = yield* saveSource(command.save, source); if (!saved.ok) { @@ -233,13 +236,13 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op const executed = yield* deps.execute({ root: retainedSource(PROMPT_IDENTITY, source), - props, + props: admitted.props, }); if (!executed.ok) { // Reported exactly as `xmd run` reports the same failure: what ran is one // ordinary document, and how it failed is not this command's news to // rephrase. A runtime failure ends the command here — there is nothing for - // generation or review to reconsider about source a person already approved. + // the command document to reconsider about a Plan you already approved. reportFailure(executed.error); return 1; } @@ -251,354 +254,117 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op * * The conversation belongs to this invocation: a second `xmd prompt` places a * different session rather than continuing this one, and a request nobody meant - * to repeat never arrives in a history it did not create. + * to repeat never arrives in a history it did not create. `--session` replaces + * it when a caller wants the provider's ordinary continuation instead. */ -function generatorSessionName(): string { +function invocationSessionName(): string { return `xmd-prompt:${randomUUID()}`; } /** - * What the generator session is told once, before it is asked anything. - * - * The catalog is the run profile's own, rendered by the same renderer - * `xmd syntax` uses — including its statement that a repository TypeScript - * component's contract was not read, which is the honest thing to tell a writer - * about a component nobody imported. - */ -export function generatorInstructions(catalog: SyntaxCatalog): string { - return [ - "You write complete executable Markdown root documents. Each user message is a", - "request for one document.", - "", - "Every answer is replacement document source and nothing else: no enclosing code", - "fence, no explanation, and no prose around it. What you return is written to a", - "file and executed exactly as you wrote it.", - "", - "A repair request carries the validation facts about the document you last sent.", - "Answer it with another complete replacement document, not a patch and not a", - "description of the change.", - "", - "The catalog below is the complete vocabulary available in the working directory", - "this command runs in and in its configured include paths. Nothing outside it is", - "available.", - "", - renderSyntaxMarkdown(catalog), - ].join("\n"); -} - -/** - * The authorship loop: draft, repair, review, revise. + * The host's answer about one draft, in the shape the command document reads. * - * Each draft — the first one, and each answer to a human revision — starts its - * own repair budget. A human revision has no numeric bound: the person asking - * for changes is the one deciding when to stop. + * A caller-source failure raises rather than answering. That is the whole of + * the classification the command document can observe: it sees facts about drafts, + * and it never sees an argument the command line got wrong. */ -function* author( +function* assess( command: PromptCommand, - provider: AcpxProvider, - session: Session, - request: string, -): Operation> { - const frozen = new Map(); - let draft = yield* runTurn(provider, session, request); - - while (true) { - if (!draft.ok) { - return draft; - } - let candidate = draft.value; - let budget = REPAIR_TURNS; - let subject: ReviewSubject | undefined; - - while (subject === undefined) { - const outcome = yield* evaluate(command, frozen, candidate); - if (outcome.kind === "terminal") { - return Err(outcome.error); - } - if (outcome.kind === "valid") { - subject = { candidate, valid: true, props: outcome.props }; - break; - } - if (budget === 0) { - subject = { candidate, valid: false, defects: outcome.defects }; - break; - } - budget -= 1; - const repaired = yield* runTurn(provider, session, repairRequest(outcome.defects)); - if (!repaired.ok) { - return repaired; - } - candidate = repaired.value; - } - - const decided = yield* review(subject); - if (!decided.ok) { - return decided; - } - const decision = decided.value; - if (decision.decision === "abort") { - return Err(new Error(ABORTED)); - } - if (decision.decision === "approve") { - if (!subject.valid) { - // The schema an exhausted candidate is offered has no approve value, so - // an answer carrying one came from somewhere the question did not. - return Err( - new Error( - "the review provider approved a document that does not validate, which the " + - "question did not offer — nothing was saved and nothing ran", - ), - ); - } - return Ok({ source: subject.candidate, props: subject.props }); - } - draft = yield* runTurn(provider, session, revisionRequest(decision.feedback ?? "")); + frozen: Map, + source: string, +): Operation { + const outcome = yield* assessCandidate(command, frozen, source); + if (outcome.kind === "terminal") { + throw outcome.error; } -} - -/** - * One turn, consumed whole inside the provider's scope. - * - * Only a terminal `completed` produces a candidate. A failed, cancelled or - * unavailable turn is a generation failure however much text it emitted, and the - * partial text is discarded rather than presented: half a program is not one. - */ -function* runTurn( - provider: AcpxProvider, - session: Session, - content: string, -): Operation> { - return yield* scoped(function* (): Operation> { - try { - const subscription = yield* provider.promptStream(content, { session }); - let status: string | undefined; - let stopReason: string | undefined; - let failure: Error | undefined; - let next = yield* subscription.next(); - while (!next.done) { - const event = next.value; - if (event.type === "terminal") { - status = event.status; - stopReason = event.stopReason; - failure = event.error; - } - next = yield* subscription.next(); - } - if (status !== "completed") { - return Err(turnFailure(status, stopReason, failure)); - } - return Ok(next.value); - } catch (error) { - return Err(toError(error)); - } - }); -} - -function turnFailure( - status: string | undefined, - stopReason: string | undefined, - failure: Error | undefined, -): Error { - const detail = - failure?.message ?? (stopReason === undefined ? undefined : `stop reason ${stopReason}`); - const outcome = status ?? "produced no terminal event"; - return new Error( - `the agent did not complete the turn (${outcome})` + - (detail === undefined ? "" : `: ${detail}`) + - " — nothing was reviewed, saved or run", - ); + if (outcome.kind === "valid") { + return { valid: true, diagnostics: {} }; + } + return { valid: false, diagnostics: outcome.defects as unknown as Json }; } /** * Everything decidable about one candidate, in the order that keeps a caller's * mistake from being taught to the agent. + * + * The registry is installed here, around the two questions that need it, rather + * than around the command document: validation and the catalog have to agree + * about what `` is, and that document has no business reaching a + * vocabulary it only describes. */ -function* evaluate( +function* assessCandidate( command: PromptCommand, frozen: Map, candidate: string, ): Operation { - const root = retainedSource(PROMPT_IDENTITY, candidate); - const includes = command.include; - const components = agentIdentityComponents(); - - // Whether the root declares itself readably. Inspection would raise on a - // malformed declaration, and recovering a code from an exception's prose is - // exactly what the structured answer exists to replace. - const declaration = yield* validateDocument({ ...root, includes, components }); - if (declaration.diagnostics.some((entry) => DECLARATION_CODES.has(entry.code))) { - return { kind: "repairable", defects: { validation: declaration } }; - } - - let propsSchema: PropsSchema; - try { - propsSchema = (yield* inspectDocument(root)).props; - } catch (error) { - return { - kind: "repairable", - defects: { prompt: { code: "root-props-unreadable", message: describeError(error) } }, - }; - } - - let bindings: Binding[]; - try { - bindings = buildBindings(propsSchema); - } catch (error) { - return { - kind: "repairable", - defects: { - prompt: { code: "generated-binding-collision", message: describeError(error) }, - }, - }; - } - - // Before a single token is extracted: an option that changed shape would - // otherwise reach forward and read the `--raw` written after it as its value. - const drift = signatureFailure(frozen, bindings); - if (drift !== undefined) { - return { kind: "terminal", error: new Error(drift) }; - } - const stray = strayPropertyValue(command.scan.occurrences, bindings); - if (stray !== undefined) { - return { kind: "terminal", error: new Error(stray) }; - } - - let extraction: Extraction; - let props: Record; - try { - extraction = extractPropsArgs(command.argv, bindings, { reserved: isReservedOption }); - props = yield* resolvePropsFromSources({ propsSchema, bindings, extraction }); - } catch (error) { - return { kind: "terminal", error: toError(error) }; - } - - for (const supplied of extraction.individual) { - frozen.set(supplied.binding.option, signatureOf(supplied.binding)); - } - - const validation = yield* validateDocument({ ...root, props, includes, components }); - if (validation.outcome === "invalid") { - return { kind: "repairable", defects: { validation } }; - } - return { kind: "valid", props }; -} + return yield* scoped(function* (): Operation { + yield* useRunProfileRegistry(); + const root = retainedSource(PROMPT_IDENTITY, candidate); + const includes = command.include; + const components = agentIdentityComponents(); + + // Whether the root declares itself readably. Inspection would raise on a + // malformed declaration, and recovering a code from an exception's prose is + // exactly what the structured answer exists to replace. + const declaration = yield* validateDocument({ ...root, includes, components }); + if (declaration.diagnostics.some((entry) => DECLARATION_CODES.has(entry.code))) { + return { kind: "repairable", defects: { validation: declaration } }; + } -/** - * The repair turn: the fixed instruction, and the facts, as data. - * - * Core's own versioned value travels whole. Nothing here renders a diagnostic - * into prose and nothing parses one back out, so what the agent reads is what - * validation concluded rather than a summary of it. - */ -function repairRequest(defects: CandidateDefects): string { - return [ - "The document you sent does not validate. Send a complete replacement document", - "that fixes every defect below. Reply with document source only.", - "", - fenced("json", JSON.stringify(defects, null, 2)), - ].join("\n"); -} + let propsSchema: PropsSchema; + try { + propsSchema = (yield* inspectDocument(root)).props; + } catch (error) { + return { + kind: "repairable", + defects: { prompt: { code: "root-props-unreadable", message: describeError(error) } }, + }; + } -function revisionRequest(feedback: string): string { - return [ - "A person reviewed the document and asked for this change:", - "", - feedback, - "", - "Send a complete replacement document. Reply with document source only.", - ].join("\n"); -} + let bindings: Binding[]; + try { + bindings = buildBindings(propsSchema); + } catch (error) { + return { + kind: "repairable", + defects: { + prompt: { code: "generated-binding-collision", message: describeError(error) }, + }, + }; + } -/** - * Ask a person, and read the one answer. - * - * The whole candidate is shown exactly as it will run, inside a fence chosen to - * be longer than any backtick run the candidate holds — arbitrary source cannot - * close a fence it cannot spell. Machinery stays out of what the person reads: - * the schema, the session and the repair budget are none of their business. - */ -function* review(subject: ReviewSubject): Operation> { - const decisions = subject.valid ? ["approve", "revise", "abort"] : ["revise", "abort"]; - const message = subject.valid - ? [ - "The agent wrote this document. Approve it to run it, ask for a change, or abort.", - "", - fenced("md", subject.candidate), - ].join("\n") - : [ - "The agent could not produce a document that validates. Ask for a change, or abort.", - "", - fenced("md", subject.candidate), - "", - "Validation reported these defects:", - "", - fenced("json", JSON.stringify(subject.defects, null, 2)), - ].join("\n"); + // Before a single token is extracted: an option that changed shape would + // otherwise reach forward and read the `--raw` written after it as its value. + const drift = signatureFailure(frozen, bindings); + if (drift !== undefined) { + return { kind: "terminal", error: new Error(drift) }; + } + const stray = strayPropertyValue(command.scan.occurrences, bindings); + if (stray !== undefined) { + return { kind: "terminal", error: new Error(stray) }; + } - try { - const answer = yield* elicit({ - message, - schema: reviewSchema(decisions), - label: "PromptReview", - }); - return readDecision(answer); - } catch (error) { - return Err(toError(error)); - } -} + let extraction: Extraction; + let props: Record; + try { + extraction = extractPropsArgs(command.argv, bindings, { reserved: isReservedOption }); + props = yield* resolvePropsFromSources({ propsSchema, bindings, extraction }); + } catch (error) { + return { kind: "terminal", error: toError(error) }; + } -function reviewSchema(decisions: readonly string[]): Json { - return { - type: "object", - title: "Review the generated document", - properties: { - decision: { - type: "string", - title: "Decision", - enum: [...decisions], - }, - feedback: { - type: "string", - title: "Feedback", - description: "What to change. Required when asking for a revision.", - }, - }, - required: ["decision"], - additionalProperties: false, - if: { properties: { decision: { const: "revise" } }, required: ["decision"] }, - then: { properties: { feedback: { type: "string", minLength: 1 } }, required: ["feedback"] }, - }; -} + for (const supplied of extraction.individual) { + frozen.set(supplied.binding.option, signatureOf(supplied.binding)); + } -/** Read the provider's answer rather than assert it. */ -function readDecision(answer: Json): Result { - if (typeof answer !== "object" || answer === null || Array.isArray(answer)) { - return Err(new Error("the review provider answered with something that is not a decision")); - } - const decision = answer.decision; - if (decision !== "approve" && decision !== "revise" && decision !== "abort") { - return Err(new Error("the review provider answered with no decision")); - } - const feedback = answer.feedback; - return Ok({ - decision, - ...(typeof feedback === "string" ? { feedback } : {}), + const validation = yield* validateDocument({ ...root, props, includes, components }); + if (validation.outcome === "invalid") { + return { kind: "repairable", defects: { validation } }; + } + return { kind: "valid", props }; }); } -/** - * A fence long enough that the body cannot close it. Three backticks at least, - * and one more than the longest run the body holds. - */ -function fenced(language: string, body: string): string { - let longest = 2; - for (const run of body.match(/`+/g) ?? []) { - longest = Math.max(longest, run.length); - } - const delimiter = "`".repeat(longest + 1); - return [`${delimiter}${language}`, body, delimiter].join("\n"); -} - /** * Create the destination and write the approved bytes, or refuse. * diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts index 351cf55bc..e7fed733b 100644 --- a/packages/cli/tests/packaged-document.test.ts +++ b/packages/cli/tests/packaged-document.test.ts @@ -18,26 +18,36 @@ import { fileURLToPath } from "node:url"; import { packagedDocumentUrl, - PROMPT_PLAN, + PROMPT_COMMAND_DOCUMENT, readPackagedDocument, } from "../src/packaged-document.ts"; import { useWorkingDirectory } from "./support/prompt-harness.ts"; describe("packaged documents", () => { - it("reads the plan program from beside its module, whatever the cwd is", function* () { + it("reads the prompt command document from beside its module, whatever the cwd is", function* () { // A temporary contextual cwd with no document in it. A lookup that reached // for the working directory would find nothing here, and one that reached // through the component search path would be answerable by a repository // file — neither may decide which program this command runs. const source = yield* useWorkingDirectory(function* () { - return yield* readPackagedDocument(PROMPT_PLAN); + return yield* readPackagedDocument(PROMPT_COMMAND_DOCUMENT); }); - expect(source).toBe(yield* readTextFile(fileURLToPath(packagedDocumentUrl(PROMPT_PLAN)))); - // It is the plan program, not merely some file that exists. + const committed = yield* readTextFile( + fileURLToPath(packagedDocumentUrl(PROMPT_COMMAND_DOCUMENT)), + ); + expect(source).toBe(committed); + // It is the prompt command document, not merely some file that exists. expect(source).toContain("returns:"); expect(source).toContain("): Map { - it("P1: exactly one request, kept byte for byte", function* () { + it("C1: exactly one request, kept byte for byte", function* () { expect(namesPrompt(["prompt", REQUEST])).toBe(true); expect(namesPrompt(["run", "doc.md"])).toBe(false); @@ -64,7 +64,7 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(separated.fixed).toEqual(["prompt"]); }); - it("P2: individual options follow the request, aggregate props may precede it", function* () { + it("C1: individual options follow the request, aggregate props may precede it", function* () { const early = scanPromptArgs(["prompt", "--props-name", "Ada", REQUEST]); expect(early.error).toContain("unrecognized option: --props-name"); expect(early.error).toContain("follow the request"); @@ -85,7 +85,7 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(inline.fixed).toEqual(["prompt", REQUEST]); }); - it("P3: built-in options after generated props stay with the invocation", function* () { + it("C1: built-in options after generated props stay with the invocation", function* () { const scan = scanPromptArgs([ "prompt", REQUEST, @@ -99,6 +99,8 @@ describe("Tier PR — xmd prompt fixed grammar", () => { "trace.jsonl", "--save", "out.md", + "--session", + "ada", ]); expect(scan.error).toBe(undefined); expect(scan.request).toBe(REQUEST); @@ -112,6 +114,8 @@ describe("Tier PR — xmd prompt fixed grammar", () => { "trace.jsonl", "--save", "out.md", + "--session", + "ada", ]); // `--props-loud` did not swallow `--journal`: a known option is never read // as a generated property's value. @@ -145,7 +149,7 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(isReservedOption("-5")).toBe(false); }); - it("P4: scalar, boolean and aggregate sources are all recorded", function* () { + it("C1: scalar, boolean and aggregate sources are all recorded", function* () { const scan = scanPromptArgs([ "prompt", REQUEST, @@ -194,7 +198,7 @@ describe("Tier PR — xmd prompt fixed grammar", () => { ]); }); - it("P5: a boolean binding turns its provisional value into a second request", function* () { + it("C1: a boolean binding turns its provisional value into a second request", function* () { const scan = scanPromptArgs(["prompt", REQUEST, "--props-loud", "true"]); expect(scan.error).toBe(undefined); expect(scan.occurrences).toEqual([{ option: "--props-loud", provisional: "true" }]); @@ -210,7 +214,7 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(stray).toContain("--props-loud=true"); }); - it("P6: a frozen option's shape is what a later candidate may not change", function* () { + it("C7: a frozen option's shape is what a later candidate may not change", function* () { const scalar = bindingsFor({ name: { type: "string" } }); const boolean = bindingsFor({ name: { type: "boolean" } }); const array = bindingsFor({ name: { type: "array", items: { type: "string" } } }); diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 64a834267..7ade28b48 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -2,10 +2,10 @@ * Tier PR — the `xmd prompt` command lifecycle * (specs/prompt-command-spec.md). * - * Rows P2, P4 and P13–P16: what the command does to the filesystem, the - * journal, the process status and the document it ends in. The approved - * document runs through the production executor, so what these prove about - * execution is what `xmd run` does with a supplied root. + * The command's grammar, filesystem, journal, lifetime and execution rows: what + * `xmd prompt` does to the disk, to the process status and to the document it + * ends in. The approved document runs through the production executor, so what + * these prove about execution is what `xmd run` does with a supplied root. * * The grammar rows shell out, because exit status and help text are what an * operator sees. Everything with a phase to observe runs in process, where a @@ -151,7 +151,7 @@ describe( "Tier PR — the xmd prompt command", { sanitizeOps: false, sanitizeResources: false }, () => { - it("P2: a misplaced individual option refuses before any phase begins", function* () { + it("C1: a misplaced individual option refuses before any phase begins", function* () { yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); const code = yield* runPrompt( @@ -185,7 +185,7 @@ describe( }); }); - it("P2: inline source is refused before any phase begins", function* () { + it("C1: inline source is refused before any phase begins", function* () { // `-e` belongs to `xmd run`. `xmd prompt` is the command that *writes* a // document, so a second one supplied on the command line is a // contradiction — and one the parser used to drop in silence, leaving the @@ -212,7 +212,7 @@ describe( } }); - it("P2: help needs no request and touches nothing", function* () { + it("C1: help needs no request and touches nothing", function* () { yield* useWorkingDirectory(function* (dir) { const { code, stdout, stderr } = yield* runCli( [ @@ -234,6 +234,11 @@ describe( expect(stdout).toContain("--props "); expect(stdout).toContain("XMD_PROPS"); expect(stdout).toContain("--save "); + expect(stdout).toContain("--session "); + // The permission flags are the approved document's, and help says so + // rather than letting a caller believe they configure how the Plan is + // written. + expect(stdout).toContain("Permission flags configure the approved document"); expect(stderr).not.toContain("unavailable"); // No catalog was rendered, and neither file the options named was made. expect(stdout).not.toContain("## Built-in components"); @@ -243,9 +248,19 @@ describe( const program = yield* runCli(["--help"]).expect(); expect(program.stdout).toMatch(/^\s+prompt\s/m); + + // A session is named or it is not selected, and the refusal happens in + // preflight, beside the request's own — a parser that read the empty value + // as absent would have used the generated name instead. + yield* useWorkingDirectory(function* (dir) { + const empty = yield* runCli(["prompt", REQUEST, "--session", ""], { cwd: dir }).join(); + expect(empty.code).toBe(1); + expect(empty.stderr).toContain("--session needs a name"); + expect(empty.stdout).not.toContain("## Built-in components"); + }); }); - it("P4: individual, aggregate and environment sources resolve and reach the run", function* () { + it("C15: individual, aggregate and environment sources resolve and reach the run", function* () { yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.deps.execute = executor(dir); @@ -265,10 +280,10 @@ describe( }); }); - it("P13: every refusal is nonzero and leaves no save, journal or run", function* () { + it("C12: every refusal is nonzero and leaves no save, journal or run", function* () { const journalName = "trace.jsonl"; - // Abort at review. + // Abort at review, through the command document's authored failure. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.deps.execute = executor(dir, join(dir, journalName)); @@ -283,7 +298,7 @@ describe( expect(yield* exists(join(dir, journalName))).toBe(false); }); - // A generation failure. + // A turn that produced text and then failed. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.deps.execute = executor(dir, join(dir, journalName)); @@ -337,7 +352,7 @@ describe( }); }); - it("P14: the approved bytes are created exclusively, before the run", function* () { + it("C11: the approved bytes are created exclusively, before the run", function* () { // Created before execution, and byte for byte. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); @@ -392,7 +407,7 @@ describe( }); }); - it("P15: the approved source runs as an ordinary document under ", function* () { + it("C15: the approved source runs as an ordinary document under ", function* () { yield* useWorkingDirectory(function* (dir) { const journal = join(dir, "trace.jsonl"); const harness = createPromptHarness(); @@ -432,7 +447,7 @@ describe( }); }); - it("P15: a failing test in the approved document reports as a run reports it", function* () { + it("C15: a failing test in the approved document reports as a run reports it", function* () { yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.deps.execute = executor(dir); @@ -466,7 +481,7 @@ describe( }); }); - it("P15: one Agent resolution serves generation and the execution after it", function* () { + it("C15: one Agent resolution serves generation and the execution after it", function* () { yield* useWorkingDirectory(function* (dir) { const reads: string[] = []; yield* useRecordedEnvironment(reads, { DEFAULT_AGENT_NAME: "settled-agent" }); @@ -497,7 +512,8 @@ describe( const code = yield* runPrompt({ ...command(dir, [REQUEST]), stack }, harness.deps); expect(code).toBe(0); - // Generation resolved the settled agent rather than a name of its own. + // The command document resolved the settled agent rather than a name of + // its own. expect(harness.fake.ensured.map((input) => input.agent)).toEqual(["settled-agent"]); // And nothing after it read the name again: authorship and the document // installation that followed were both configured from the one answer, @@ -506,7 +522,7 @@ describe( }); }); - it("P16: the deadline encloses every phase, and teardown gates what follows", function* () { + it("C13: the deadline encloses every phase, and teardown gates what follows", function* () { // Expiry is cancellation, so the proof is what cancelling the command does: // the turn in flight is cancelled, the provider is dismantled, and no later // phase begins. The barrier is what makes this a gate rather than a race — @@ -530,11 +546,12 @@ describe( expect(yield* exists(join(dir, "out.md"))).toBe(false); }); - // A generator teardown failure prevents the save and the run. + // A teardown failure prevents the final validation, the save and the run, + // whatever the command document selected. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.deps.execute = executor(dir); - harness.fake.closeFailure = new Error("the generator would not close"); + harness.fake.closeFailure = new Error("the profile provider would not close"); harness.fake.script({ reply: PLAIN }); harness.script({ decision: "approve" }); @@ -545,7 +562,7 @@ describe( expect(yield* exists(join(dir, "out.md"))).toBe(false); }); - // Nothing bounds a generation turn: the exec and fetch defaults belong to + // Nothing bounds an authoring turn: the exec and fetch defaults belong to // the document, and the run deadline is the enclosing timebox above. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); @@ -562,7 +579,7 @@ describe( }); }); - it("P16: the deadline is the whole prompt command's, as it is a run's", function* () { + it("C13: the deadline is the whole prompt command's, as it is a run's", function* () { // Read for prompt exactly as for run: a value that is not a duration fails // the invocation before it prepares anything. for (const flag of ["--timeout", "--timeout-exec", "--timeout-fetch"]) { diff --git a/packages/cli/tests/prompt-plan.test.ts b/packages/cli/tests/prompt-command-document.test.ts similarity index 81% rename from packages/cli/tests/prompt-plan.test.ts rename to packages/cli/tests/prompt-command-document.test.ts index 691ec1a54..f08bc5e96 100644 --- a/packages/cli/tests/prompt-plan.test.ts +++ b/packages/cli/tests/prompt-command-document.test.ts @@ -1,11 +1,11 @@ /** - * The packaged plan program, executed as itself. + * The packaged prompt command document, executed as itself. * * This runs the exact Markdown the CLI ships — read through the packaged * loader, not copied into a fixture — so what it proves is what a release does. * The seams around it are deterministic: a scriptable ACP runtime for the one * Agent turn, a scripted Elicitation answer for the review, and a test-only - * validator standing in for the host capability that does not exist yet. + * validator in the place the prompt profile declares the production one. * * The include list is empty on purpose. Repository component search must not be * able to supply `Loop`, `If`, `Return`, `Fail`, `CodeBlock` or the validator: @@ -29,7 +29,7 @@ import { executeInstalled } from "@executablemd/core/host"; import { InMemoryStream } from "@executablemd/durable-streams"; import { createAcpxProvider } from "@executablemd/acp"; -import { PROMPT_PLAN, readPackagedDocument } from "../src/packaged-document.ts"; +import { PROMPT_COMMAND_DOCUMENT, readPackagedDocument } from "../src/packaged-document.ts"; import { AGENT, useWorkingDirectory } from "./support/prompt-harness.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./support/fake-acp.ts"; @@ -58,8 +58,8 @@ const CANDIDATE = [ "", ].join("\n"); -/** What the plan asked the validator about, in order. */ -interface PlanRun { +/** What the command document asked the validator about, in order. */ +interface CommandRun { validated: string[]; reviews: ElicitationRequest[]; prompts: string[]; @@ -67,8 +67,8 @@ interface PlanRun { failure: string | undefined; } -function* runPlan(): Operation { - const source = yield* readPackagedDocument(PROMPT_PLAN); +function* runDocument(): Operation { + const source = yield* readPackagedDocument(PROMPT_COMMAND_DOCUMENT); const fake = createFakeAcp(); fake.script({ reply: CANDIDATE }); @@ -105,13 +105,13 @@ function* runPlan(): Operation { value = yield* collect( yield* executeInstalled( { - ...retainedSource("", source), + ...retainedSource("", source), stream: new InMemoryStream(), includes: [], props: { request: "ask me for my age and write the result to a file", syntax: "## Built-in components\n\n### ``\n", - session: "plan-regression", + session: "prompt-command-regression", }, }, [ @@ -137,8 +137,10 @@ function* runPlan(): Operation { required: ["valid", "diagnostics"], additionalProperties: false, }, - // The deterministic seam only. It records what it was asked - // about and says yes; the production validator is later work. + // The deterministic seam only, standing where the command + // declares its own validator. It records what it was asked + // about and says yes, so what this case observes is the + // program's control flow rather than validation's answers. factory: () => // deno-lint-ignore require-yield function* validateCandidate(props: Record) { @@ -159,10 +161,10 @@ function* runPlan(): Operation { return { validated, reviews, prompts: fake.prompts, value, failure }; } -describe("the packaged plan program", () => { - it("returns the approved candidate's exact bytes and never reaches exhaustion", function* () { +describe("the packaged prompt command document", () => { + it("C2: returns the approved candidate's exact bytes and never reaches exhaustion", function* () { const run = yield* useWorkingDirectory(function* () { - return yield* runPlan(); + return yield* runDocument(); }); // Approving the first valid candidate is one turn and one question. A @@ -174,8 +176,8 @@ describe("the packaged plan program", () => { // The validator saw the Agent's complete close value, once, unaltered. expect(run.validated).toEqual([CANDIDATE]); - // The plan settled with a value rather than an authored failure. Before the - // control-flow correction this was the ten-draft exhaustion message: a + // The document settled with a value rather than an authored failure. Before + // the control-flow correction this was the ten-draft exhaustion message: a // `` selects a value but does not end the body, so the unconditional // `` after the Session ran and won over every approval. expect(run.failure).toBe(undefined); diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 8a0ca94b2..6465ef791 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -1,11 +1,16 @@ /** - * Tier PR — `xmd prompt` generation, validation and review - * (specs/prompt-command-spec.md). + * Tier PR — `xmd prompt` writing a Plan, as the host and the packaged prompt + * command document perform it together (specs/prompt-command-spec.md). * - * Rows P5–P12. The ACPX runtime is the scriptable fake, the review provider is a - * scripted `Elicitation` handler, the executor records what it was handed, and - * the contextual working directory is a temporary one. Nothing here starts an - * agent, opens a browser or reaches a network. + * The policy under test is `src/documents/prompt-command.md`, reached the way + * the command reaches it: `runPrompt` builds the prompt profile, the packaged + * document runs inside it, and every observation here is of that document's + * behaviour rather than of a TypeScript loop standing in for it. + * + * The seams are deterministic. The ACPX runtime is the scriptable fake, the + * review provider is a scripted `Elicitation` handler, the executor records what + * it was handed, and the contextual working directory is a temporary one. + * Nothing here starts an agent, opens a browser or reaches a network. * * Every refusal is proven by the phases that stayed at zero — turns not sent, * reviews not asked, executions not handed anything — rather than by output @@ -13,19 +18,22 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { scoped } from "effection"; +import { ensure, scoped, until } from "effection"; import type { Operation } from "effection"; import { writeTextFile } from "@effectionx/fs"; +import { readdir } from "node:fs/promises"; import { join } from "node:path"; +import { API } from "@executablemd/runtime"; +import { Elicitation } from "@executablemd/core"; +import type { ElicitationRequest } from "@executablemd/core"; import { runPrompt } from "../src/prompt.ts"; import type { PromptCommand } from "../src/prompt.ts"; +import { PROMPT_INSTRUCTIONS, PROMPT_PROFILE_DIRECTORY } from "../src/prompt-profile.ts"; import { scanPromptArgs } from "../src/prompt-args.ts"; -import { useGeneratorAgent } from "../src/agent-stack.ts"; import type { AgentStack } from "../src/agent-stack.ts"; import { AGENT, createPromptHarness, useWorkingDirectory } from "./support/prompt-harness.ts"; import type { PromptHarness } from "./support/prompt-harness.ts"; -import { createFakeAcp, makeRegistry, makeStore } from "./support/fake-acp.ts"; const REQUEST = "write a greeting"; @@ -81,6 +89,25 @@ const NAME_IS_BOOLEAN = [ "", ].join("\n"); +/** One document under two schemas that bind `--props-count` the same way. */ +function counting(type: "number" | "string"): string { + return [ + "---", + "props:", + " type: object", + " properties:", + ` count: { type: ${type} }`, + " additionalProperties: false", + "---", + "", + "Counting to {props.count}.", + "", + ].join("\n"); +} + +/** A Plan whose effect is visible on the filesystem if anything runs it. */ +const WRITES_A_FILE = ['the draft ran', ""].join("\n"); + /** The Agent configuration a dispatch settles once and hands to both consumers. */ const STACK: AgentStack = { provider: "acpx", @@ -88,15 +115,9 @@ const STACK: AgentStack = { permissionMode: "deny-all", }; -function command(dir: string, args: string[], save?: string): PromptCommand { +function command(dir: string, args: string[], stack: AgentStack = STACK): PromptCommand { const argv = ["prompt", ...args]; - return { - argv, - scan: scanPromptArgs(argv), - include: [dir], - ...(save === undefined ? {} : { save }), - stack: STACK, - }; + return { argv, scan: scanPromptArgs(argv), include: [dir], stack }; } /** Every session key the fake was asked to establish, deduplicated in order. */ @@ -105,7 +126,7 @@ function sessions(harness: PromptHarness): string[] { } /** The decisions one review request offered. */ -function decisions(request: { schema: Record }): unknown { +function decisions(request: ElicitationRequest): unknown { const properties = request.schema.properties; if (typeof properties !== "object" || properties === null) { return undefined; @@ -117,11 +138,226 @@ function decisions(request: { schema: Record }): unknown { return (decision as Record).enum; } +/** How many times one exact string appears. */ +function occurrences(haystack: string, needle: string): number { + return haystack.split(needle).length - 1; +} + +/** What the command wrote to stderr while it ran. */ +function* reported(body: () => Operation): Operation<{ value: T; lines: string[] }> { + const written = console.error; + const lines: string[] = []; + const value = yield* scoped(function* (): Operation { + yield* ensure(() => { + console.error = written; + }); + console.error = (...parts: unknown[]) => { + lines.push(parts.map((part) => String(part)).join(" ")); + }; + return yield* body(); + }); + return { value, lines }; +} + describe( - "Tier PR — xmd prompt authorship", + "Tier PR — xmd prompt writing a Plan", { sanitizeOps: false, sanitizeResources: false }, () => { - it("P5: candidate defects earn a repair turn; caller defects terminate", function* () { + it("C2, C3, C14: the packaged program's own words ask for the document", function* () { + yield* useWorkingDirectory(function* (dir) { + // A repository TypeScript component, so the catalog has to state the one + // thing it honestly cannot know without importing the module. + yield* writeTextFile(join(dir, "Widget.ts"), "export default function Widget() {}\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + // Exactly one catalog, built with the invocation's own includes. + expect(harness.catalogCalls).toEqual([[dir]]); + + // The turn is the shipped Markdown's, word for word: the sentences below + // exist nowhere in TypeScript, so a host that chose its own policy could + // not produce them. + const turn = harness.fake.prompts[0]; + expect(turn).toContain("Write one complete Executable Markdown document that does this"); + // C14: the shipped instruction is the narrative-plus-components rule. + expect(turn).toContain("The document you write is a Plan"); + expect(turn).toContain("ordinary reader-facing prose"); + expect(turn).toContain( + "places each component\nimmediately after the sentences describing the action", + ); + // The worked example travels with it, unparsed. + expect(turn).toContain('{answer.age}'); + expect(turn).toContain("keep every outcome it asked for"); + // The request travels inside it, byte for byte. + expect(turn).toContain(REQUEST); + // So does the host's catalog, including the renderer's honest statement + // about a component nobody imported. + expect(turn).toContain("## Built-in components"); + expect(turn).toContain("### ``"); + expect(turn).toContain("### ``"); + expect(turn).toContain("This component is a repository TypeScript module."); + + // The host owns the instruction layer, and owns only the shape of an + // answer: the catalog and the request are the program's to send. + expect(harness.fake.ensured[0]?.sessionOptions?.systemPrompt).toBe(PROMPT_INSTRUCTIONS); + expect(PROMPT_INSTRUCTIONS).not.toContain("Built-in components"); + + // C3: one is one turn. Nothing repaired, nothing retried. + expect(harness.fake.prompts).toHaveLength(1); + expect(harness.fake.turns).toHaveLength(1); + expect(harness.reviews).toHaveLength(1); + }); + }); + + it("C4: one Session carries every turn, and --session names it", function* () { + // Initial, repair and revision turns all land in one conversation, and a + // second invocation places a different one. + const keys: string[] = []; + for (const _invocation of [0, 1]) { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: UNRESOLVED }); + harness.script({ decision: "revise", feedback: "try plain text" }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + // A draft, three repairs, a revision: five turns, one session. + expect(harness.fake.prompts).toHaveLength(5); + expect(sessions(harness)).toHaveLength(1); + keys.push(sessions(harness)[0]); + }); + } + expect(keys[0]).not.toBe(keys[1]); + + // `--session` replaces the generated name, so two invocations name one + // conversation and the provider's ordinary continuation applies. + const named: string[] = []; + for (const _invocation of [0, 1]) { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt( + { ...command(dir, [REQUEST]), session: "ada" }, + harness.deps, + ); + expect(code).toBe(0); + named.push(sessions(harness)[0]); + }); + } + expect(named[0]).toBe(named[1]); + expect(named[0]).not.toBe(keys[0]); + }); + + it("C5: the prompt profile's ceiling is the host's, and no flag widens it", function* () { + // An empty directory of the host's own, no MCP servers, no native tools — + // observed while the command document is still running, because the + // directory is gone + // by the time the command returns. + yield* useWorkingDirectory(function* (dir) { + yield* writeTextFile(join(dir, "secret.txt"), "the caller's tree\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + const seen: { cwd?: string; entries?: string[]; refusals: string[] } = { refusals: [] }; + harness.deps.installElicitation = function* () { + yield* Elicitation.around( + { + *elicit([request], _next) { + harness.reviews.push(request); + const cwd = harness.fake.created[0]?.cwd; + seen.cwd = cwd; + seen.entries = cwd === undefined ? undefined : yield* until(readdir(cwd)); + // The command document's own capabilities, asked for from inside + // its scope. It decides what to write; it writes nothing. + for (const [name, ask] of [ + ["a directory", () => API.Files.operations.temporaryDirectory()], + ["a command", () => API.Process.operations.exec({ command: ["true"] })], + ["the network", () => API.Fetch.operations.fetch("http://localhost")], + ] as const) { + try { + yield* ask(); + seen.refusals.push(`${name} was granted`); + } catch (error) { + seen.refusals.push(error instanceof Error ? error.message : String(error)); + } + } + return { decision: "approve" }; + }, + }, + { at: "min" }, + ); + }; + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + // Not the caller's working directory, and empty. + expect(seen.cwd).not.toBe(dir); + expect(seen.cwd).toBe(PROMPT_PROFILE_DIRECTORY); + expect(seen.entries).toEqual([]); + // Stated rather than omitted: this host configures no MCP server and + // allows no native tool on a fresh session. + expect(harness.fake.created[0]?.mcpServers).toEqual([]); + expect(harness.fake.ensured[0]?.sessionOptions?.allowedTools).toEqual([]); + // And the command document is given nothing to act with. + expect(seen.refusals).toEqual([ + "xmd prompt asked for a directory, which the prompt profile grants to nothing", + "xmd prompt asked for a command, which the prompt profile grants to nothing", + "xmd prompt asked for the network, which the prompt profile grants to nothing", + ]); + }); + + // `--approve-all` configures the approved document. A native permission + // request while the Plan is being written is still denied, privately, and + // the turn it + // belongs to fails. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID, requestsTool: "Bash" }); + + const code = yield* runPrompt( + command(dir, [REQUEST], { ...STACK, permissionMode: "approve-all" }), + harness.deps, + ); + + expect(code).toBe(1); + expect(harness.fake.decisions).toEqual(["reject_once"]); + // The denial ended the command: nobody was asked and nothing ran. + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("C6: a candidate is inert until the approved document runs", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: WRITES_A_FILE }); + harness.script({ decision: "abort" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + // Validated, presented, and never run: the document's own write is the + // effect that would have happened if anything had executed it. + expect(harness.reviews).toHaveLength(1); + expect((yield* until(readdir(dir))).includes("drafted.txt")).toBe(false); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("C7: candidate defects earn a repair turn; caller defects escape", function* () { // A defect the agent authored: the root's own frontmatter. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); @@ -133,10 +369,9 @@ describe( expect(code).toBe(0); expect(harness.fake.prompts).toHaveLength(2); - expect(harness.fake.prompts[0]).toBe(REQUEST); - expect(harness.fake.prompts[1]).toContain("complete replacement document"); + expect(harness.fake.prompts[1]).toContain("That document has problems"); expect(harness.fake.prompts[1]).toContain("source-invalid"); - // The whole versioned value, as data. + // The whole versioned value, as data the program serialized. expect(harness.fake.prompts[1]).toContain('"version": 1'); expect(harness.executions).toHaveLength(1); }); @@ -158,7 +393,8 @@ describe( expect(harness.fake.prompts[1]).not.toContain("DocumentValidationCode"); }); - // A defect the caller wrote: an option the candidate never declares. + // A defect the caller wrote: an option the candidate never declares. It + // raises out of the validator, so the program never sees it as feedback. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.fake.script({ reply: VALID }); @@ -203,150 +439,26 @@ describe( expect(harness.reviews).toHaveLength(0); expect(harness.executions).toHaveLength(0); }); - }); - it("P6: a revision may not change what the command line means", function* () { + // A revision that changes what the command line means is the caller's + // failure too, and it is caught before the candidate is presented. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); - // The first draft binds `--props-name` as a single-value option. harness.fake.script({ reply: REQUIRES_NAME }); harness.script({ decision: "revise", feedback: "make it shout" }); - // The revision declares the same name as a switch. harness.fake.script({ reply: NAME_IS_BOOLEAN }); const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); expect(code).toBe(1); - // The change was caught before the second candidate was presented. expect(harness.reviews).toHaveLength(1); expect(harness.executions).toHaveLength(0); expect(harness.fake.prompts).toHaveLength(2); }); - - // An unchanged signature re-resolves from the same unchanged sources. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); - harness.fake.script({ reply: REQUIRES_NAME }); - harness.script({ decision: "revise", feedback: "greet louder" }); - harness.fake.script({ reply: `${REQUIRES_NAME}\nLOUDER\n` }); - harness.script({ decision: "approve" }); - - const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); - - expect(code).toBe(0); - expect(harness.executions[0]?.props).toEqual({ name: "Ada" }); - }); }); - it("P7: one run-profile catalog becomes the fresh session's system prompt", function* () { - yield* useWorkingDirectory(function* (dir) { - // A repository TypeScript component, so the catalog has to state the one - // thing it honestly cannot know without importing the module. - yield* writeTextFile(join(dir, "Widget.ts"), "export default function Widget() {}\n"); - - const harness = createPromptHarness(); - harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); - - const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); - expect(code).toBe(0); - - // Exactly one catalog, built with the invocation's own includes. - expect(harness.catalogCalls).toEqual([[dir]]); - - const instructions = harness.fake.ensured[0]?.sessionOptions?.systemPrompt; - expect(typeof instructions).toBe("string"); - const stated = typeof instructions === "string" ? instructions : ""; - expect(stated).toContain("Every answer is replacement document source and nothing else"); - expect(stated).toContain("## Built-in structural syntax"); - expect(stated).toContain("## Built-in components"); - expect(stated).toContain("### ``"); - expect(stated).toContain("### ``"); - // The renderer's origin-only wording, unchanged. - expect(stated).toContain("This component is a repository TypeScript module."); - // The instructions are the session's, not the user turn's. - expect(harness.fake.prompts[0]).toBe(REQUEST); - expect(harness.fake.prompts[0]).not.toContain("Built-in components"); - }); - }); - - it("P8: one fresh session per invocation, contacted only by a subscribed turn", function* () { - // Placement contacts no backend: the runtime is not even created until the - // first turn is subscribed. - yield* useWorkingDirectory(function* () { - const fake = createFakeAcp(); - yield* scoped(function* (): Operation { - const provider = yield* useGeneratorAgent( - { - provider: "acpx", - defaultAgent: AGENT, - permissionMode: "deny-all", - }, - "instructions", - { - createRuntime: fake.create, - sessionStore: makeStore(), - agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), - }, - ); - const session = yield* provider.session("xmd-prompt:placement"); - expect(session.sessionKey.startsWith("xmd:v1:")).toBe(true); - expect(fake.ensured).toHaveLength(0); - expect(fake.started).toBe(false); - - fake.script({ reply: VALID }); - const subscription = yield* provider.promptStream("hello", { session }); - let next = yield* subscription.next(); - while (!next.done) { - next = yield* subscription.next(); - } - expect(fake.ensured).toHaveLength(1); - expect(fake.ensured[0]?.sessionKey).toBe(session.sessionKey); - }); - }); - - // Initial, repair and revision turns all land in one conversation. - const keys: string[] = []; - for (const _invocation of [0, 1]) { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); - harness.fake.script({ reply: UNRESOLVED }); - harness.fake.script({ reply: UNRESOLVED }); - harness.script({ decision: "revise", feedback: "try again" }); - harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); - - // The base candidate, one repair, an exhausting pair, a revision. - harness.fake.script({ reply: VALID }); - const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); - expect(code).toBe(0); - - expect(sessions(harness)).toHaveLength(1); - keys.push(sessions(harness)[0]); - }); - } - // A second invocation places a different session: nothing is inherited. - expect(keys[0]).not.toBe(keys[1]); - }); - - it("P9: only a completed terminal produces a candidate", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); - // A turn that streams a perfectly good document and then fails. - harness.fake.script({ reply: VALID, stopReason: "max_tokens" }); - - const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); - - expect(code).toBe(1); - expect(harness.fake.prompts).toHaveLength(1); - // The partial text was discarded: nobody was asked and nothing ran. - expect(harness.reviews).toHaveLength(0); - expect(harness.executions).toHaveLength(0); - }); - }); - - it("P10: the base candidate plus exactly three repairs", function* () { - // Three repairs are available. + it("C8: one base draft, three repairs, and ten presentations", function* () { + // Three repairs are available, and the fourth draft is what a person sees. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.fake.script({ reply: UNRESOLVED }); @@ -363,7 +475,8 @@ describe( expect(decisions(harness.reviews[0])).toEqual(["approve", "revise", "abort"]); }); - // A fourth invalid candidate is exhausted, and cannot be approved. + // A fourth invalid candidate is repair-exhausted: it reaches review with + // its diagnostics, and there is no value that would approve it. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); for (const _draft of [0, 1, 2, 3]) { @@ -377,68 +490,44 @@ describe( expect(harness.fake.prompts).toHaveLength(4); expect(harness.reviews).toHaveLength(1); expect(decisions(harness.reviews[0])).toEqual(["revise", "abort"]); - expect(harness.reviews[0].message).toContain("could not produce a document that validates"); + expect(harness.reviews[0].message).toContain( + "Three attempts to fix it did not clear everything", + ); expect(harness.reviews[0].message).toContain("component-unresolved"); expect(harness.executions).toHaveLength(0); }); - }); - it("P11: review offers what the candidate allows, and a revision starts over", function* () { + // Ten presentations: nine revisions, and a tenth round with nothing left + // to revise into. Each revision starts its own repair budget. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); - // Four invalid drafts: the exhausted review. - for (const _draft of [0, 1, 2, 3]) { - harness.fake.script({ reply: UNRESOLVED }); - } - harness.script({ decision: "revise", feedback: "use plain text" }); - // The revision's own budget: four more before the next review. - for (const _draft of [0, 1, 2, 3]) { - harness.fake.script({ reply: UNRESOLVED }); + for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { + harness.fake.script({ reply: VALID }); + if (round < 10) { + harness.script({ decision: "revise", feedback: `round ${round}` }); + } } harness.script({ decision: "abort" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(1); - // Four, then the revision's own draft and three more repairs: the budget - // reset rather than carrying the exhausted one forward. - expect(harness.fake.prompts).toHaveLength(8); - expect(harness.fake.prompts[4]).toContain("use plain text"); - expect(harness.fake.prompts[4]).toContain("complete replacement document"); - expect(harness.reviews).toHaveLength(2); - expect(decisions(harness.reviews[0])).toEqual(["revise", "abort"]); - expect(decisions(harness.reviews[1])).toEqual(["revise", "abort"]); - // The revision stayed in the same conversation. + expect(harness.reviews).toHaveLength(10); + // Every draft validated first time, so each round cost exactly one turn. + expect(harness.fake.prompts).toHaveLength(10); + expect(harness.fake.prompts[1]).toContain("round 1"); + for (const round of [0, 1, 2, 3, 4, 5, 6, 7, 8]) { + expect(decisions(harness.reviews[round])).toEqual(["approve", "revise", "abort"]); + } + // The last presentation offers no revision, because there is no eleventh + // round to revise into. + expect(decisions(harness.reviews[9])).toEqual(["approve", "abort"]); + // One conversation held all ten. expect(sessions(harness)).toHaveLength(1); - - // Non-empty feedback is the contract, and it is the schema's. - const rule = harness.reviews[0].schema.then; - expect(rule).toEqual({ - properties: { feedback: { type: "string", minLength: 1 } }, - required: ["feedback"], - }); - expect(harness.reviews[0].schema.if).toEqual({ - properties: { decision: { const: "revise" } }, - required: ["decision"], - }); - expect(harness.reviews[0].schema.additionalProperties).toBe(false); - }); - - // A review answer the schema rejects fails the command the way an abort - // does: nothing is saved and nothing runs. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); - harness.fake.script({ reply: VALID }); - harness.script({ decision: "revise", raw: { decision: "revise", feedback: "" } }); - - const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); - - expect(code).toBe(1); - expect(harness.executions).toHaveLength(0); }); }); - it("P12: the exact agent bytes are shown, approved and executed", function* () { + it("C9: arbitrary source cannot close the presentation, and abort is authored", function* () { yield* useWorkingDirectory(function* (dir) { // A document that holds a fence of its own, and a run of five backticks. const fenced = [ @@ -462,20 +551,55 @@ describe( const message = harness.reviews[0].message; // The fence is longer than every run inside the candidate, so nothing in // the source can close it. - expect(message).toContain("``````md\n"); + expect(message).toContain("``````markdown\n"); expect(message).toContain(`\n${fenced}\n\`\`\`\`\`\``); - // Nothing was stripped: what runs is the close value, byte for byte. - expect(harness.executions[0]?.root.source).toBe(fenced); + }); + + // Abort reaches the command document's own ``, with the message the + // shipped Markdown wrote. Nothing about it is host policy. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "abort" }); + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST]), harness.deps), + ); + + expect(value).toBe(1); + expect(lines.join("\n")).toContain( + "xmd prompt: you ended the review, so nothing was saved and nothing ran.", + ); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("C10, C11: the approved bytes are what runs, and props are theirs", function* () { + // The command line is unchanged across a revision, and the schema that + // resolves it is the approved document's rather than the first draft's. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + harness.fake.script({ reply: counting("number") }); + harness.script({ decision: "revise", feedback: "count in words" }); + harness.fake.script({ reply: counting("string") }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST, "--props-count", "7"]), harness.deps); + + expect(code).toBe(0); + // Resolved under the approved bytes: a props object kept from the first + // candidate would carry the number 7. + expect(harness.executions[0]?.props).toEqual({ count: "7" }); + expect(harness.executions[0]?.root.source).toBe(counting("string")); expect(harness.executions[0]?.root.path).toBe(""); }); - // A reply wrapped in a fence is shown as it is, not unwrapped. + // Nothing is stripped. A reply wrapped in a fence is not a document, so it + // earns repairs and a review — and what is shown is exactly what arrived. yield* useWorkingDirectory(function* (dir) { const wrapped = ["```md", "Hello.", "```", ""].join("\n"); const harness = createPromptHarness(); - harness.fake.script({ reply: wrapped }); - // Wrapped source is not a document, so it earns repairs and then a review. - for (const _draft of [0, 1, 2]) { + for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: wrapped }); } harness.script({ decision: "abort" }); @@ -484,5 +608,114 @@ describe( expect(harness.reviews[0].message).toContain(wrapped); }); }); + + it("C14: an interleaved Plan survives approval and execution byte for byte", function* () { + yield* useWorkingDirectory(function* (dir) { + // What the shipped instruction asks for: the request restated in prose a + // reader was written for, with each component beside the sentences that + // describe what it does. + const plan = [ + "# Your age", + "", + "You asked to be asked for your age and to have it written down, so this", + "document does exactly that, in that order.", + "", + "First, the question.", + "", + '', + "", + "Then the answer goes to a file beside this document.", + "", + '{answer.age}', + "", + ].join("\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: plan }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt( + command(dir, ["ask me for my age and write it to a file"]), + harness.deps, + ); + + expect(code).toBe(0); + // The prose reached the person who approved it, inside the presentation + // fence and unaltered. + expect(harness.reviews[0].message).toContain(plan); + // And the exact bytes are what runs: nothing trimmed, re-fenced or + // reflowed between approval and execution. + expect(harness.executions[0]?.root.source).toBe(plan); + }); + }); + + it("C3, C9: what you read says each thing once, however many rounds it took", function* () { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + // Round one: a draft that cannot be repaired, presented with its problems. + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script({ decision: "revise", feedback: "use plain text" }); + // Round two: a draft that passes, and is approved. + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + const [exhausted, approvable] = harness.reviews.map((review) => review.message); + // The heading each presentation carries appears once in it, and the + // second presentation carries neither the first draft nor its problems: + // a loop that accumulated its explanation would show both. + for (const message of [exhausted, approvable]) { + expect(occurrences(message, "This document was written for:")).toBe(1); + } + expect(occurrences(exhausted, "Three attempts to fix it did not clear everything")).toBe(1); + expect(approvable).not.toContain("Three attempts to fix it"); + expect(approvable).not.toContain("NoSuchComponent"); + expect(exhausted).not.toContain(VALID.trim()); + + // The repair turns say what they need once each, and carry only the + // problems of the draft they are repairing. + for (const repair of harness.fake.prompts.slice(1, 4)) { + expect(occurrences(repair, "That document has problems")).toBe(1); + expect(occurrences(repair, "Send one complete replacement document")).toBe(1); + } + // The revision turn says what changed, once, and asks for a whole + // document rather than repeating the original brief. + const revision = harness.fake.prompts[4]; + expect(occurrences(revision, "use plain text")).toBe(1); + expect(revision).not.toContain("The document you write is a Plan"); + }); + + // Abort and the tenth-round exhaustion each print one sentence, once. + for (const scripted of [ + { rounds: 1, ending: "you ended the review" }, + { rounds: 10, ending: "ten drafts were reviewed and none was approved" }, + ]) { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + for (const round of Array.from({ length: scripted.rounds }, (_, i) => i + 1)) { + harness.fake.script({ reply: VALID }); + if (round < scripted.rounds) { + harness.script({ decision: "revise", feedback: `round ${round}` }); + } + } + harness.script({ decision: "abort" }); + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST]), harness.deps), + ); + + expect(value).toBe(1); + // One line, from the document's own , and no accumulated + // repetition of it however many rounds preceded it. + expect(lines).toHaveLength(1); + expect(occurrences(lines[0], "xmd prompt:")).toBe(1); + expect(lines[0]).toContain("nothing was saved and nothing ran"); + }); + } + }); }, ); diff --git a/scripts/tests/cli-npm-bin.test.ts b/scripts/tests/cli-npm-bin.test.ts index 29bc820cc..74955cff3 100644 --- a/scripts/tests/cli-npm-bin.test.ts +++ b/scripts/tests/cli-npm-bin.test.ts @@ -107,8 +107,8 @@ describe("npm CLI package", { sanitizeOps: false, sanitizeResources: false }, () // absent from the package unless the build copies it — and the command // would then find no program to run, on Node and Bun while Deno stayed // green. - expect(yield* readTextFile(path.join(OUT_DIR, "esm/src/documents/prompt-plan.md"))).toBe( - yield* readTextFile(path.join(ROOT, PKG_DIR, "src/documents/prompt-plan.md")), + expect(yield* readTextFile(path.join(OUT_DIR, "esm/src/documents/prompt-command.md"))).toBe( + yield* readTextFile(path.join(ROOT, PKG_DIR, "src/documents/prompt-command.md")), ); }); }); diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 3cabb11c2..f26a16f42 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -531,41 +531,60 @@ The default agent resolves in order, each entry overriding the ones above it: The installed provider belongs to the run's `DocumentExecution` scope and closes during its teardown. -### The `xmd prompt` generator - -`xmd prompt` resolves that configuration once and uses it twice: for the session -that writes the document, and for the document that runs. The provider name, the -default agent with its `DEFAULT_AGENT_NAME` precedence, the permission mode and -the host's own machine-session assembly are settled before any catalog is built -or any document executes, so an unknown provider or an incompatible pair of -permission flags fails first. It is the settled value that reaches both +### The `xmd prompt` prompt profile + +`xmd prompt` resolves that configuration once and uses it twice: for the prompt +command document that writes the Plan, and for the Plan that runs. The provider +name, the default agent with its `DEFAULT_AGENT_NAME` precedence, the permission +mode and the host's own machine-session assembly are settled before any catalog +is built or any document executes, so an unknown provider or an incompatible +pair of permission flags fails first. It is the settled value that reaches both consumers, not the flags that produced it: `DEFAULT_AGENT_NAME` is read once per -invocation, and generation and the executed document cannot reach different +invocation, and authorship and the executed Plan cannot reach different conclusions from one command line. -Generation takes the provider's operations directly — `useAcpxProvider`, without -the Agent install. It needs turns, not authority: no document is installed -around it, no root provider is registered, and no foreground launcher exists for -a `` to reach. The one Agent operation the provider performs for -itself, resolving the agent a session and a turn belong to, is answered by the -generator handle; every other Agent operation keeps its base behavior and -refuses. The selected permission policy is installed around those turns, so a -native permission request during generation is answered by the same -`--approve-all`, `--approve-reads` or `--deny-all` choice a run uses. - -Only the generator provider receives `newSessionOptions.systemPrompt`. It -carries the invocation's structured syntax catalog and the fixed statement of -what the session writes ([`xmd prompt`](./prompt-command-spec.md)). ACPX applies -those options when it creates a session and ignores them when it reuses a -record, and the command places a session name unique to the invocation, so each -invocation is a fresh conversation and the instructions are the ones it was -created under. - -The generator's scope closes after approval and before the save and the -execution. A teardown failure fails the command and neither later phase happens. -The executed program is an ordinary `xmd run` document with its own root -provider and its own lifetime: it inherits neither the generator session nor its -system prompt. +The prompt profile takes the provider name and the default agent from that +answer, and nothing else. Its ceiling is the host's, assembled for that one +document and not readable from the command line: + +| The profile's provider gets | Stated as | +| --- | --- | +| a host-owned working directory that is not the caller's | `agentCwd`, one fixed empty path | +| no MCP servers | `mcpServers: []`, an empty set rather than an omission | +| no native tools on a fresh session | `newSessionOptions.allowedTools: []` | +| a private refusal of every native permission request | `permissions: "strict"` | + +`"strict"` answers the request inside the provider: the request is denied, the +turn it belongs to fails, and no public Agent handler is consulted or can +intervene. So `--approve-all`, `--approve-reads` and `--deny-all` cannot widen a +ceiling that has nothing in it; they configure the approved document later. A +provider that cannot establish this ceiling refuses before session +materialization or a turn. + +The command document itself is given no Files, command, service or XMD-mediated +network capability, and the host decides for that whole execution that a failing +`` ends it — so a turn that streamed text and then failed presents +nothing. + +The profile's working directory is one fixed host-owned path rather than a new one each +invocation, because a session's key includes the directory it lives in: a +location that changed every time would leave `--session` unable to name a +conversation that already exists. Nothing this profile grants can write there. + +Only the profile's provider receives `newSessionOptions.systemPrompt`. It carries the +fixed statement of what the session writes, and nothing else — the catalog and +the request are the command document's to send in its own turns +([`xmd prompt`](./prompt-command-spec.md)). ACPX applies those options when it +creates a session and ignores them when it reuses a record. Without `--session` +the command places a name unique to the invocation, so each invocation is a fresh +conversation created under those instructions; `--session ` selects an +existing one under ordinary continuation semantics. + +That document's scope closes before the final validation, the save and the +execution. A teardown failure fails the command and no later phase happens. The +executed program is an ordinary `xmd run` document with its own root provider and +its own lifetime: it inherits neither the assistant session nor its instruction +layer. ### Availability diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 361c22962..b6aa9e6a4 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -8237,17 +8237,24 @@ identity, and no temporary file is created: the text is captured inside the durable root import, so the journal holds it and a replay restores it without reading anything. -Text a host generated reports `` on the same terms. `xmd prompt` asks an -agent for a complete root, validates it, has a person approve it, and executes -`retainedSource("", source)` — so the identity says where the bytes came -from, and a source position reads `(:5:1)`. It is a deliberate identity -rather than a path nobody could read back, and it is the only thing that -differs: the approved document runs through the ordinary supplied-source path, -renders and returns exactly as any other root, resolves the contextual working -directory for every relative operation, and creates its journal only when -execution starts. Generation, repair and approval happen before that execution -exists, so nothing about them is journaled or replayed. The command's complete -contract is [`xmd prompt`](./prompt-command-spec.md). +Text a host generated reports `` on the same terms. `xmd prompt` runs a +packaged first-party command document that turns a request into a Plan — asking +an agent for a complete root, having the host check it, and having a person +approve it; the host then validates the returned bytes again and executes +`retainedSource("", source)` — so the +identity says where the bytes came from, and a source position reads +`(:5:1)`. It is a deliberate identity rather than a path nobody could +read back, and it is the only thing that differs: the approved document runs +through the ordinary supplied-source path, renders and returns exactly as any +other root, resolves the contextual working directory for every relative +operation, and creates its journal only when execution starts. + +That command document is the other root the command executes, under its own +stable internal identity ``, on an invocation-owned in-memory +durable stream that is discarded when authorship is over. So nothing about +generation, repair or approval is journaled or replayed, and the two executions +share no scope. The command's complete contract is +[`xmd prompt`](./prompt-command-spec.md). `inspectDocument(root)` loads and validates the root definition and returns what it declares — without executing the document or creating a journal: @@ -10033,21 +10040,23 @@ One checked-in Markdown suite runs the real command against a fixture directory. ### Tier PR — The `xmd prompt` command -Authorship around one ordinary document: generation, validation, repair, live -review, saving and execution. Defined in [`xmd prompt`](./prompt-command-spec.md), -whose acceptance table this points at rather than restating. The ACPX runtime is -a scriptable fake, the review provider is a scripted `Elicitation` handler, and -the contextual working directory is a temporary one — no live agent, browser or -network appears in this evidence, and every refusal is proven by the phase -tripwires that stayed at zero. +Two root executions with a scope boundary between them: the packaged prompt +command document, which owns the visible generation, repair, review and failure +policy, and the Plan it returned, which runs as any supplied root does. Defined in +[`xmd prompt`](./prompt-command-spec.md), whose acceptance table this points at +rather than restating. The ACPX runtime is a scriptable fake, the review provider +is a scripted `Elicitation` handler, and the contextual working directory is a +temporary one — no live agent, browser or network appears in this evidence, and +every refusal is proven by the phase tripwires that stayed at zero. | # | Test | Verify | |---|------|--------| -| P1–P3 | Fixed grammar | One request preserved byte for byte; missing, repeated, empty and whitespace-only requests refused; individual options after the request and aggregate props before it; a built-in option never read as a generated property's value | -| P4–P6 | Generated props | Scalar, switch, repeated and aggregate sources resolve with ordinary precedence and reach validation and execution; a draft's defect is repaired while a caller's terminates; a frozen option's shape is what no later draft may change | -| P7–P9 | Generation | One run-profile catalog, origin-only TypeScript truth included, becomes the fresh session's system prompt; one exact session carries every turn and a second invocation places another; only a completed terminal produces a candidate | -| P10–P12 | Repair and review | The base candidate plus exactly three repairs; an exhausted candidate is shown with its diagnostics and cannot be approved; revise needs non-empty feedback and resets the budget; the exact bytes are shown inside a fence they cannot close and executed unchanged | -| P13–P16 | Lifecycle | Every refusal is non-zero with no save, journal or run; an approval's journal holds only the document's events; the approved bytes are created exclusively before execution; the run reports `` and behaves as `xmd run` does; the deadline encloses every phase and teardown gates what follows | +| C1 | Fixed grammar and help | One request preserved byte for byte; missing, repeated, empty and whitespace-only requests refused; individual options after the request and aggregate props before it; a built-in option never read as a generated property's value; `--session` named or refused; generic help with no effects | +| C2–C3 | The packaged document is the policy | The command executes the checked-in Markdown value root under ``, and the turn text is that document's own words; generation, repair, review, revision, approval, abort and exhaustion are Markdown, `` stays one turn, and what a person reads says each thing once however many rounds it took | +| C4–C6 | Session and ceiling | One enclosing Session carries every turn, defaults differ per invocation and `--session` supplies the exact override; the prompt profile gives the assistant an empty host-owned directory, no MCP servers, no native tools and a private strict denial no permission flag widens; a draft is data until the approved Plan runs | +| C7–C9 | Classification, bounds and presentation | Draft defects return structured facts and caller defects escape the validator; one base draft plus three repairs, and ten presentations with no revision on the last; arbitrary source cannot close ``, and abort reaches the document's own `` | +| C10–C13 | Admission and lifetime | The host revalidates after the command document has completely torn down and resolves props for the exact returned bytes; those bytes are created exclusively before execution; the final journal holds only the document's events; cancellation and teardown failure settle before anything later begins | +| C14–C15 | Result | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, and a Plan of prose interleaved with components survives approval and execution byte for byte; the run reports `` and behaves as `xmd run` does | ### Tier WB — The workflow component bundle in core diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 50f0c31a2..fa065e32e 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -1,40 +1,66 @@ # The `xmd prompt` command -Ask the configured agent for an executable Markdown document, prove it is one, -decide whether to run it, and run it. +`xmd prompt` turns your **Prompt** — the request you typed, in ordinary language +— into a **Plan**: an Executable Markdown document that states what you asked +for in ordinary language and places the components that perform the work +alongside those words. ```console $ xmd prompt "ask me for my age and write the result to a file" ``` -The command writes no code of its own. It adds *authorship* around one ordinary -document: the agent produces a complete root, `xmd` validates that root without -executing any of it, repairs the definite defects by asking again, shows a -person the exact bytes, and — once they approve — executes those bytes through -the same path `xmd run` uses for a supplied document. There is no second -execution model, no second props model and no second journal. +A Plan is not merely executable code. It preserves the Prompt's intent as +reader-facing prose interleaved with the components that fulfil it, so the +request above should produce a document shaped like this: + +```markdown +Ask me for my age. + + + +Write it to a file. + +{answer.age} +``` + +and an execution organized by the same narrative: the first sentence, the +interaction, the second sentence, the file. The Prompt need not be copied +literally — it may be divided, clarified and rewritten into natural prose — but +every requested outcome, their meaningful ordering, and their relationship to the +components performing them are preserved. + +The command writes no policy of its own. It executes two root documents with a +complete scope boundary between them: first the **prompt command document**, a +checked-in first-party Markdown value root that implements this conversion and +its review workflow — what the assistant is asked, how many drafts may be +repaired, what you are shown, and what happens when you approve nothing — and +then the Plan that document returned, through the same path `xmd run` uses for a +supplied one. The prompt command document is not itself a Plan. There is no +second execution model, no second props model and no second journal. ## The flow ```text -fixed CLI preflight - -> structured syntax catalog - -> fresh generator session - -> candidate props + validation - -> automatic repair, at most three turns - -> live approval or human revision - -> generator teardown - -> optional exclusive save - -> ordinary in-memory document execution +fixed command preflight + -> build the run-profile syntax catalog + -> execute the exact packaged prompt command document + -> one Session + -> generate, check, repair, review, revise, approve or fail + -> Return the exact approved Plan source + -> await that execution and provider teardown + -> validate the returned source again + -> optionally save the exact bytes + -> execute retainedSource("", source) through the ordinary run path ``` Each phase hands the next one a value. No phase after the first failure begins, -so a refused command line reaches no catalog, a failed generation reaches no -review, and an aborted review reaches no file and no run. +so a refused command line reaches no catalog, a failed turn reaches no review, +and an aborted review reaches no file and no run. -Generation, repair and approval are a conversation, and a conversation is not a -run: they create no journal, retain no durable identity and replay nothing. Only -the final document execution owns `--journal`. +Writing a Plan is a conversation, and a conversation is not a run: the prompt +command document runs on an invocation-owned in-memory durable stream that is +never written to `--journal`, persisted, reused or replayed. Only the approved +Plan's execution owns `--journal`. ## Command grammar @@ -60,23 +86,24 @@ xmd prompt -- "--this is the request" `xmd prompt` takes the complete `xmd run -e` execution flag set — `--include`, `--verbose`, `--journal`/`-j`, `--raw`, `--agent-provider`, `--default-agent`, `--approve-all`, `--approve-reads`, `--deny-all`, `--no-secret-detection`, and -the three timeout options — plus `--save`. Those options configure one -invocation: the same Agent and permission choices answer the generator's -requests and the executed document's, and the same includes describe the -vocabulary and resolve the run's components. +the three timeout options — plus `--save` and `--session`. + +Agent provider selection, default agent selection and `--timeout` configure how +the Plan is written. The permission flags and the exec and fetch timeouts +configure only the approved Plan; authorship does not inherit the final run's +permission mode, and no permission flag widens the prompt profile's ceiling +below. `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with `unrecognized option for xmd prompt: --eval — inline documents are exclusive to -xmd run`, before the catalog, the generator provider, the review, the save, the -journal or any execution exists. A generated document is what this command -writes, so a second one on the command line is a contradiction rather than an -option the command is missing. +xmd run`, before the catalog, the command document, the review, the save, the +journal or any execution exists. Those options are resolved into one Agent configuration once per invocation — one `--agent-provider`, one `--default-agent` or `DEFAULT_AGENT_NAME`, one -permission mode — and that settled answer is what both the generator and the -approved document's installation are configured from. Incompatible permission +permission mode — and that settled answer is what both the prompt profile and +the approved Plan's installation are configured from. Incompatible permission flags and an unknown provider fail there, before the catalog is built. ### `--save ` @@ -86,9 +113,20 @@ against the contextual working directory and **created exclusively**: an existing path is left exactly as it is, the command fails, and nothing runs. There is no check-then-write — the exclusive create *is* the check. -The file holds the approved source and nothing else: no diagnostics, no -decision, no wrapper. Without `--save`, no generated-source file is created -anywhere; the approved document runs from memory. +The file holds the approved Plan and nothing else: no problems, no decision, no +wrapper. Without `--save`, no file is created anywhere; the approved Plan runs +from memory. + +### `--session ` + +Use this logical assistant session instead of one unique to the invocation. The +name must be non-empty; `--session` with an empty value is refused by fixed +grammar rather than read as absent, so a caller who asked for a named session +never falls back to the generated one by accident. + +Ordinary provider session continuation applies when the configured provider +already holds that name. The prompt command document still supplies the current +request and the current catalog in this invocation's initial turn. ### Help @@ -96,15 +134,145 @@ anywhere; the approved document runs from memory. xmd prompt --help ``` -Help needs no request. It describes the request, `--save`, the aggregate -`--props`/`XMD_PROPS` sources, that candidate-declared individual options follow -the request, and every shared execution flag. It describes no individual -property, because the document that would declare one does not exist yet — -answering otherwise would mean generating a document in order to describe one. +Help needs no request. It describes the request, `--save`, `--session`, the +aggregate `--props`/`XMD_PROPS` sources, that candidate-declared individual +options follow the request, that the permission flags configure the approved +document, and every shared execution flag. It describes no individual property, +because the document that would declare one does not exist yet — answering +otherwise would mean generating a document in order to describe one. Help reads no catalog, contacts no provider, places no session, asks nobody anything, creates no file and runs nothing. +## The packaged prompt command document + +The host executes one immutable packaged Markdown value root, +`packages/cli/src/documents/prompt-command.md`, under the stable internal source +identity ``. Its declared return schema is exactly +`{ type: "string" }`. It is located from the CLI module's own URL — never from +the working directory and never through the component search path — so the same +policy is found whatever directory a person stands in, and no repository file can +answer for it ([release process](./release-process-spec.md)). + +The host supplies three fixed internal inputs as that root's props: + +- `request` — the original request text; +- `syntax` — the rendered syntax catalog for the current run profile and ordered + `--include` values; and +- `session` — the resolved logical assistant-session name. + +They are not Plan root props and consume none of the approved Plan's property +sources. + +The document contains one enclosing `` expansion whose body holds every +initial, repair and human-revision ``. A loop inside that one Session +creates no second placement; sibling Sessions are not used, because sibling +placements stay distinct even when their authored names match. + +The host owns the provider instruction layer and the Agent ceiling. The Markdown +document owns the text of each generation, repair and revision request: the +initial prompt preserves the Prompt, includes the host's catalog, and asks for +one complete replacement root as source only — written as a Plan, with every +requested outcome kept as reader-facing prose and each component placed +immediately after the sentences describing the action it performs. That +authorship rule is repeated in every repair and revision request, so the +narrative survives a draft being replaced. + +`` remains one Agent turn. It gains no hidden repair, retry, review or +approval behaviour. + +## The prompt profile + +The prompt profile is the trusted-host assembly used only for that exact root. It +supplies the document's inputs, a constrained Agent provider, Elicitation, the +fixed first-party components and the host-declared draft validator. It uses no +repository component search — that execution's include list is empty — and +exposes no custom root. + +### Agent authority under the prompt profile + +The assistant that writes a Plan is assembled separately from the final run +provider: + +- a host-owned working directory that is not the caller's and is created empty; +- no additional directories; +- no MCP servers — stated as an empty set, not omitted; +- an empty requested native-tool allowlist on a fresh session; +- strict denial of every native permission request, answered inside the provider + without consulting an authored approval scope; and +- no Files, command, service or XMD-mediated network capability for the command + document itself. + +The provider may use its own transport to perform the model turn; that does not +grant the Agent a native network tool. + +`--approve-all`, `--approve-reads` and `--deny-all` do not change this ceiling. +They apply to the approved Plan later. A provider that cannot establish this +ceiling refuses before session materialization or a turn; there is no silent +downgrade. + +The profile's working directory is one fixed host-owned path rather than a new +directory each invocation, because a session's identity includes the directory it +lives in: a location that changed every time would mean `--session` could never +name a conversation that already exists. Nothing this profile grants can write +there, so what is created empty stays empty. + +The approved Plan later receives the caller-selected ordinary run Agent and +permission configuration. It inherits neither the assistant Session nor its +instruction layer. + +### The assistant Session + +Without `--session`, the host generates a logical name unique to the invocation. +The exact name is supplied to the command document, and its one enclosing +`` materializes only at the first consuming ``. Every turn +within this invocation uses that one Session. + +A turn produces a draft only from its complete successful close value. The host +decides, for the whole command document execution, that a failing `` ends +it: a failed, cancelled, unavailable or protocol-invalid turn discards its +partial text and reaches no human review, save or final execution, and the +document cannot opt out of that. + +That execution closes after approval or failure. The host observes its result +only after every Prompt task, Agent provider resource, Elicitation resource and +other child has completed teardown. A teardown failure wins over a selected Plan +and prevents final validation, save and execution. + +### A draft is data + +An Agent reply is an inert string while the Plan is being written. The command +document may bind it, pass it to the validator, serialize its problems, present +it with `` and return it. It never evaluates the draft and never +dynamically imports it. Only after approval, teardown and final host validation +may those exact bytes enter ordinary execution. + +## Host-declared draft validation + +The prompt profile declares one internal value component to the execution: + +```md + +``` + +Canonical execution supplies its invocation identity, so repository resolution +cannot replace it. Its result is a closed candidate assessment: either +`{ valid: true, diagnostics: {} }`, or `{ valid: false, diagnostics }` where +`diagnostics` carries the complete versioned `DocumentValidation` when core +produced one and the structured generated-binding diagnostic when property +binding failed. + +The component performs no candidate execution. For each source it: + +1. validates the supplied root declaration without executing it; +2. once a usable root props schema exists, derives that schema's bindings; +3. checks every frozen supplied individual-option signature before consuming + tokens; +4. resolves the original CLI and invocation-environment property sources under + that schema; and +5. calls `validateDocument()` with those exact props, the caller's includes and + the run profile's identity-component declarations. + ## Generated document properties An `xmd prompt` invocation resolves root props exactly as `xmd run` does — same @@ -124,28 +292,12 @@ document operation. The original argv is the command line source for every candidate. `XMD_PROPS` and the candidate's `XMD_PROPS_*` variables are read through the contextual runtime environment. No resolved props object is carried from one candidate to -the next. - -For each candidate — the first draft, each repair and each answer to a human -revision — the command: - -1. builds the root as supplied text under the `` identity; -2. establishes that the root's own declaration is readable, from - `validateDocument()`'s structured answer rather than from an exception's - prose; -3. reads the usable props schema with `inspectDocument()`, once that declaration - is known good; -4. builds the individual bindings that schema generates; -5. holds every supplied option to its frozen signature; -6. extracts the unchanged raw property arguments, reads their environment - values, and resolves them; and -7. validates the exact candidate again with the resolved props, the ordered - includes and the host's identity-component declarations. +the next, or from any candidate to the approved bytes. ### Frozen signatures An individual option's **signature** is its generated option name, its token -arity — a bare switch or one value — and its accumulation behavior — scalar +arity — a bare switch or one value — and its accumulation behaviour — scalar last-wins or repeated array. The first candidate that successfully binds a supplied option freezes that option's signature. @@ -165,16 +317,18 @@ xmd prompt "" --props-name=-Ada ### Candidate failures and caller failures These are **repairable candidate failures**. The agent authored them, so the -agent is asked again: +assessment answers `valid: false` and the command document may ask again: -- source, frontmatter and root declaration diagnostics; +- source, frontmatter, target, root-props and return-declaration diagnostics; - a generated binding-name collision — two declared properties producing one option or one environment variable; - missing required root props and every other `props-invalid` result; and - every other definite document diagnostic. -These are **terminal property-source failures**. The caller wrote them, so no -repair turn is spent and no review, save, journal or execution happens: +These are **terminal caller-source failures**. The caller wrote them, so they +raise out of the validator and end the command document immediately, with no +repair turn, no presentation, no approval, no save, no final journal and no final +execution: - a supplied individual option the usable candidate schema does not declare; - malformed aggregate CLI or environment JSON; @@ -184,129 +338,97 @@ repair turn is spent and no review, save, journal or execution happens: The split is about authorship, not severity. A collision is repairable because the candidate authored both colliding names; an undeclared option is terminal -because only the caller supplied it, and teaching the agent about it would spend -a repair turn on a defect the agent cannot fix. - -An extra positional a candidate's arity exposes is decided as soon as the first -usable schema exists, and always before anything is presented. - -## Generation - -One catalog is built for the invocation, from the run profile's declarations in -the contextual working directory and the configured includes — the same -structured `SyntaxCatalog` value `xmd syntax` renders, rendered by the same -renderer. Nothing spawns `xmd syntax`, parses its Markdown, or keeps a second -catalog. Its statement that a repository TypeScript component's contract was not -read is the honest one, and it reaches the generator unchanged. - -The system instruction layer states that the session writes complete executable -Markdown roots for the requests that follow, that every answer is replacement -source only with no enclosing fence and no prose, that a repair request carries -validation facts and needs another complete replacement, and that the appended -catalog is the complete available vocabulary. It is the session's system prompt, -not part of any user message. - -The initial user turn is the request string itself, byte for byte. - -The command resolves one logical session name unique to the invocation and one -exact `Session` value from it, and passes that same value to the initial turn, -every repair turn and every human revision turn. Placing the session contacts no -backend: a fresh placement materializes when the first subscribed turn is -accepted. A second `xmd prompt` places a different session, so no invocation -inherits another's history. - -A candidate is the stream's complete close value from a terminal `completed` -turn. A failed, cancelled, unavailable or protocol-invalid turn is a generation -failure however much text it emitted: the partial candidate is discarded, and -nothing is presented, saved or executed. - -The generator's scope closes after approval and before `--save` or execution. A -teardown failure fails the command and neither later phase happens. - -## Repair - -Each first draft, and each answer to a human `revise`, starts its own repair -budget of three. A candidate with repairable failures earns one repair turn -carrying a fixed instruction to return a complete replacement root, core's -complete versioned `DocumentValidation` value serialized as JSON when core -produced one, and the prompt-owned generated-binding diagnostic when that is the -failure. Nothing renders a diagnostic into prose and nothing parses one back -out. - -The whole candidate is replaced by the new close value and validated again. The -base candidate is not repair turn one, so a candidate may be followed by at most -three replacements. A fourth invalid candidate is **exhausted** and goes to human -review carrying its diagnostics. +because only the caller supplied it. The command document has no way to catch a +terminal failure and no way to recategorize it as draft feedback. An opaque `not-statically-checkable` invocation is not a diagnostic and does not -make a document invalid. It may be approved and executed under the ordinary -permission model. - -No fence is stripped and no Markdown-looking substring is extracted. The exact -close value is always the candidate: if a fenced reply is otherwise a valid -document, that exact document is shown and the person decides. - -## Review - -The command installs the CLI's WebForm elicitation provider around authorship — -outside any document and any journal — and asks through core's `elicit()` -operation. - -A valid candidate is offered a draft-07 object schema with a required `decision` -of `approve`, `revise` or `abort`, an optional string `feedback`, -`additionalProperties: false`, and an `if`/`then` rule requiring `feedback` with -`minLength: 1` when the decision is `revise`. An exhausted invalid candidate is -offered the same schema allowing only `revise` and `abort`: there is no approve -value to reject later. - -The message presents the exact candidate in a Markdown source fence whose -delimiter is at least three backticks and longer than every backtick run in the -candidate, so arbitrary source cannot close it. An invalid candidate's source is -followed by a `json` fence holding its complete structured diagnostics. Schemas, -session identifiers and repair budgets are machinery and stay out of what the -person reads. - -- `approve` returns the candidate and the props resolved for that candidate. -- `revise` sends the feedback to the same generator session, asks for a complete - replacement, resets the automatic repair budget to three, and reviews again. - Human revision rounds have no numeric bound. -- `abort` fails the command and performs no save and no execution. - -The contract is non-empty feedback, not non-whitespace feedback. A provider -failure, or an answer the schema rejects, fails the command the way an abort -does. - -## Execution - -After approval and successful generator teardown, `--save` writes, and then the -approved source executes as `retainedSource("", source)` through the -same document runner `xmd run -e` uses, with the props resolved under that exact -source and the same includes, output, journal, secret detection, Agent, -permission and timeout configuration. No temporary Markdown file is created. +by itself make a candidate invalid. + +## The policy the prompt command document owns + +The following is the shipped program's behaviour, not the host's. It is stated +here because it is what a caller sees; it is changed by editing +`prompt-command.md`, and nothing in TypeScript decides it. Prose quality is that +document's instructions and your review, never a hidden TypeScript validation +rule: `` reports structural facts and executes nothing. + +**Automatic repair.** The initial draft and every human-requested revision each +start a fresh repair budget: the base candidate is attempt one, at most three +repair turns may replace it, every repair prompt carries the complete structured +diagnostics, and every answer must be another complete replacement root. The +fourth invalid candidate is repair-exhausted and goes to human review with its +diagnostics. No fence is stripped, no Markdown substring is extracted and no +patch is applied. + +**Human review.** At most ten candidate presentations: the initial review plus at +most nine human revisions. Rounds one through nine offer `approve`, `revise` or +`abort` for a valid candidate, and `revise` or `abort` for an invalid +repair-exhausted one. `revise` requires non-empty feedback, sends one +complete-replacement request through the same enclosing Session, and resets the +three-turn repair budget. Round ten offers no further revision: a valid +candidate offers `approve` or `abort`, and an invalid one offers only `abort`. + +**Presentation.** The review message presents the exact draft with ``, +whose fence is longer than every backtick run the draft holds, so draft text +cannot close it and is never interpreted. An invalid draft is followed by its +complete JSON problems, serialized by ``. Prose outside `` +addresses you; text inside `` instructs the assistant. + +**Failure.** `approve` selects the draft, and one exhaustive branch after the +Session either returns its source unchanged or stops without saving or running +anything. `abort` and a review that approved nothing +reach `` with an actionable message. Failure is authored in Markdown rather +than hidden in the host or represented by a missing-`` accident. + +The command document's rendered output is not command output. Everything you see +while a Plan is written reaches you through Elicitation, and that document's +successful public result is the approved Plan source and nothing else. + +## Final admission, save and execution + +After the command document has completely torn down, the host treats the +returned string as untrusted again. It repeats candidate validation and property resolution using the exact +returned source, the original raw CLI and invocation-environment property +sources, the individual-option signatures frozen while it was written, the +caller's +ordered includes and the ordinary run profile declarations. + +A final caller-source failure or document validation failure exits non-zero +before save or execution. It does not re-enter the command document or ask for a +repair. The final resolved props belong to those exact source bytes; no props +object from an earlier candidate is reused. + +`--save` then exclusively creates the target with those exact bytes. The host +then executes `retainedSource("", source)` through the ordinary +supplied-source run path, with the same includes, output, value result, secret +detection, Agent configuration, permission mode and timeouts `xmd run -e` uses. +No temporary Markdown file is created. The `` identity affects positions and diagnostics only. The contextual working directory still resolves relative filesystem operations, repository -components and includes. +components and includes. The executed program receives a fresh ordinary document +Agent provider and inherits neither the assistant Session nor its instruction +layer. -The executed program receives a fresh ordinary document Agent provider. It -inherits neither the generator session nor its system prompt. - -`--journal` is created when execution starts, and only then. Generation, -repairs, review and abort never enter it. Rendered output, a value root's JSON +`--journal` is created when the final execution starts, and only then. That +journal contains no command document, assistant Session, Agent turn, draft +validation, repair or human-review event. Rendered output, a value root's JSON result, runtime failure reporting and exit codes are byte-for-byte ordinary -`xmd run` behavior — including a failing `` boundary, which is reported -under its own `tests failed:` heading rather than as a bare message. A runtime failure does not return the command to generation -or review; `--save` has already completed, and the source is there to hand-edit. +`xmd run` behaviour — including a failing `` boundary, which is reported +under its own `tests failed:` heading rather than as a bare message. A runtime +failure does not send the command back to authorship; `--save` has already +completed, +and the source is there to hand-edit. ## Timeouts -`--timeout` bounds the whole command: preparation, catalog inspection, -generation, every repair, human review, generator teardown, the save and the -execution. Expiry is Effection cancellation, so structured teardown completes -before the failure is reported. +`--timeout` bounds the whole command: preflight, catalog construction, the +command document's execution, Elicitation, its teardown, final validation, the +save and the execution. Expiry is Effection cancellation, so structured teardown completes +before the failure is reported. A teardown failure prevents every later phase. `--timeout-exec` and `--timeout-fetch` configure the final document's effects -only, exactly as under `xmd run`. Nothing bounds a generation turn but the +only, exactly as under `xmd run`. Nothing bounds an authoring turn but the command deadline. ## Failures @@ -317,39 +439,45 @@ Every failure below exits non-zero, and each one stops the phases after it: | --- | --- | | a malformed command line | nothing | | incompatible permission flags or an unknown `--agent-provider` | nothing | -| a catalog an include makes unreadable | no provider | +| a catalog an include makes unreadable | no command document | +| a provider that cannot establish the prompt profile's ceiling | no session, no turn | | a turn that did not complete | no review, save or run | -| a terminal property-source failure | no repair, review, save or run | -| `abort`, or a review the provider could not answer | no save or run | -| generator teardown | no save or run | +| a terminal caller-source failure | no repair, review, save or run | +| the command document's authored `` | no save or run | +| command document teardown | no final validation, save or run | +| final validation of the approved bytes | no save or run | | a `--save` path that exists, or a write that fails | no run | | the document's own runtime failure | nothing after it; the save stands | ## Acceptance -Tier PR. Rows P1–P6 are proven in `packages/cli/tests/prompt-args.test.ts`, -P5–P12 in `packages/cli/tests/prompt.test.ts`, and P2, P4 and P13–P16 in -`packages/cli/tests/prompt-cli.test.ts`. The ACPX runtime is a scriptable fake, -the review provider is a scripted `Elicitation` handler, and the contextual -working directory is a temporary one: no live agent, browser or network appears -in this evidence. Every refusal is proven by the phase tripwires that stayed at -zero rather than by output nobody produced. +Tier PR. The evidence lives in `packages/cli/tests/prompt-args.test.ts` (fixed +grammar and signatures), `packages/cli/tests/prompt-command-document.test.ts` +(the packaged document executed as itself), `packages/cli/tests/prompt.test.ts` +(the host and the packaged document writing a Plan together) and +`packages/cli/tests/prompt-cli.test.ts` (the command lifecycle, filesystem, +journal and execution). + +The ACPX runtime is a scriptable fake, the review provider is a scripted +`Elicitation` handler, and the contextual working directory is a temporary one: +no live agent, browser or network appears in this evidence. Every refusal is +proven by the phase tripwires that stayed at zero rather than by output nobody +produced. | # | Criterion | Required observation | | --- | --- | --- | -| P1 | Request grammar | Missing, repeated, empty and whitespace-only requests fail; a valid request reaches the initial Agent turn byte for byte | -| P2 | Ordering and help | An individual option before the request fails with zero downstream effects; aggregate props before it work; help needs no request and has zero effects | -| P3 | Fixed option safety | Built-in flags after generated props are parsed for generation and execution; a later signature change cannot consume one as a value | -| P4 | Scalar, boolean and aggregate props | Candidate-declared scalar and bare boolean CLI options plus aggregate CLI/environment JSON resolve with ordinary precedence and reach validation and execution | -| P5 | Candidate/source classification | An invalid declaration and a generated binding collision receive repair turns; an undeclared supplied option, malformed caller JSON and an invalid caller value terminate with no repair or review | -| P6 | Revision signature | Removing an option, changing boolean/scalar arity, changing scalar/array accumulation or rejecting the original value fails before extraction and presentation; unchanged signatures re-resolve from unchanged sources | -| P7 | Catalog and system layer | One structured run-profile catalog, including its origin-only TypeScript truth, becomes the fresh session's system prompt; no subprocess and no duplicate catalog exists | -| P8 | Fresh conversation | Initial, repair and human revision turns use one exact session object; a second invocation has a different placement; no backend is contacted before the first subscribed turn | -| P9 | Generation terminal | Only a completed terminal yields a candidate; a failed turn discards its partial text and causes no review, save or execution | -| P10 | Repair budget | The base candidate plus exactly three repairs are possible; the last invalid candidate is shown with complete diagnostics and cannot be approved | -| P11 | Human review | Valid source offers approve/revise/abort; invalid source offers revise/abort; revise requires non-empty feedback, continues the session and resets the repair budget | -| P12 | Exact source | Fences inside a candidate cannot close the review fence; approval, save and execution receive the Agent close value byte for byte; no fence stripping occurs | -| P13 | Abort and journal boundary | Abort, generation failure, a terminal props failure and an exhausted-invalid abort are non-zero with no save, journal or execution; an approval's journal contains only final document events | -| P14 | Save | Approved bytes are exclusively created before execution; an existing path is unchanged and prevents execution; no save flag creates no generated file | -| P15 | Ordinary execution | Approved source reports ``, resolves the contextual cwd and includes normally, receives approved-candidate props and the shared Agent flags, and preserves run output, result and runtime-failure behavior | -| P16 | Lifetime | The whole-command deadline cancels every phase and awaits provider teardown; generator teardown failure prevents save and execution; exec and fetch timeouts remain document-only | +| C1 | Fixed grammar and help | Request cardinality, individual-property ordering, aggregate props before the request, `--session` including its empty-value refusal, and effect-free generic help | +| C2 | Exact packaged root | The command executes the checked-in Markdown value root under ``; the turn text is that document's own words, and no TypeScript authorship loop or custom root chooses policy | +| C3 | Visible policy | Generation, three-turn repair, ten-round review, revision, approval, abort and exhaustion are present in Markdown; `` remains one turn | +| C4 | One Session | One enclosing Session expansion carries every turn; default names differ across invocations and `--session` supplies the exact override | +| C5 | Prompt profile ceiling | An empty host-owned working directory, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it | +| C6 | Draft inertness | A draft is only data while the Plan is written, and no draft effect occurs before the final execution | +| C7 | Validation classification | Candidate failures return structured facts; caller-source failures escape immediately; frozen signatures are checked before token extraction | +| C8 | Bounds | One base plus three automatic repairs per draft, and no more than ten human presentations, with no revision offered on the last | +| C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; abort and exhaustion reach ``, not host policy and not a missing `` | +| C10 | Final gate | The host revalidates after the command document has completely torn down, and resolves props for the exact returned bytes | +| C11 | Exact bytes | Approval, exclusive save and `` execution receive the Agent close value without rewriting or fence removal | +| C12 | Journal separation | Authorship uses only disposable in-memory history; the final journal begins with the approved Plan | +| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution | +| C14 | Narrative preservation | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, the assistant receives it with the request and the catalog, and a scripted Plan of prose interleaved with components returns byte for byte after approval | +| C15 | Ordinary run | The approved source keeps normal cwd, includes, props, output/value, permission, timeout, failure and save behaviour | diff --git a/specs/release-process-spec.md b/specs/release-process-spec.md index b02e032a6..005ede27e 100644 --- a/specs/release-process-spec.md +++ b/specs/release-process-spec.md @@ -92,7 +92,7 @@ documents at the revision it checks. prerelease marker — so a forgotten bump is visible where the release was made, then refuses to build. On a valid tag it compiles `packages/cli/src/compiled.ts` per target with - `--include packages/code-review-agent --include packages/cli/src/prompt-plan.md` + `--include packages/code-review-agent --include packages/cli/src/documents/prompt-command.md` and attaches the binaries and sha256 checksums to the tag's GitHub Release. That module is the compiled-binary entrypoint: it installs the `API.Env.command` adapter that @@ -551,10 +551,9 @@ A document-backed command executes first-party Markdown through the ordinary XMD engine rather than a TypeScript policy. A package declares which Markdown it ships by putting it in `src/documents/`; every other Markdown under `src/` — test documents, scenario fixtures — stays out of the product. `xmd prompt` is -the first such command, and -`packages/cli/src/documents/prompt-plan.md` is its program: the checked-in Markdown is the -deployed artifact and the single source of truth, not a generated string mirror -of one. +the first such command, and `packages/cli/src/documents/prompt-command.md` is +the document that implements it: the checked-in Markdown is the deployed +artifact and the single source of truth, not a generated string mirror of one. The command locates it from its own module URL — never from the contextual working directory, and never through the component search path. Both are @@ -565,7 +564,7 @@ Every build therefore keeps the asset beside its module, at the same relative path: - **source checkout** — the file as committed; -- **`deno compile`** — embedded by `--include packages/cli/src/documents/prompt-plan.md`, +- **`deno compile`** — embedded by `--include packages/cli/src/documents/prompt-command.md`, in `deno task build` and in `release.yml`'s matrix compile; - **npm (dnt)** — copied by `scripts/build-npm.ts`, which copies each package's `src/documents/` into `esm/src/documents/`, preserving relative location. dnt @@ -583,7 +582,7 @@ The checks that hold this together, each proving a different build: under Deno, Node and Bun, which is what makes it evidence rather than one runtime's opinion. - `scripts/tests/cli-npm-bin.test.ts` builds the real package and asserts - `esm/src/documents/prompt-plan.md` is byte-identical to the source. + `esm/src/documents/prompt-command.md` is byte-identical to the source. - `scripts/tests/packaged-document.test.ts` holds the two `deno compile` sites to the documents that exist, because that list is the one thing no build discovers for itself. diff --git a/specs/root-document-props-spec.md b/specs/root-document-props-spec.md index cc08db533..382e4929d 100644 --- a/specs/root-document-props-spec.md +++ b/specs/root-document-props-spec.md @@ -365,6 +365,12 @@ the same decoding ([`xmd prompt`](./prompt-command-spec.md)). What differs is where the schema comes from: the document is written by an agent while the command runs, and one invocation may see several drafts of it. +That command executes two roots. The first is the packaged prompt command +document, whose own props — the request, the rendered syntax catalog and the +assistant-session name — are host-supplied internal inputs. They are not draft +props, they are declared by no command-line option, and they consume none of the +sources below. + Individual options follow the **request** rather than a document path, for the same reason they follow the path under `xmd run` — the schema that gives them meaning arrives after them: @@ -378,10 +384,10 @@ any catalog, agent, elicitation, save, journal or document operation. The aggregate `--props` may be written before it, because its meaning never depends on a document. -`xmd prompt --help` describes the request, `--save`, the aggregate sources and -where individual options go. It names no individual option and no default: the -document that would declare them does not exist yet, and generating one in order -to describe it is not what help does. +`xmd prompt --help` describes the request, `--save`, `--session`, the aggregate +sources and where individual options go. It names no individual option and no +default: the document that would declare them does not exist yet, and generating +one in order to describe it is not what help does. Every draft is bound afresh. The original argv is the command-line source for all of them, the environment is read for each candidate's own bindings, and no @@ -400,10 +406,18 @@ value that really begins with `-` is written `--props-name=-value`. Failures divide by who authored them. A draft's own defect — an unreadable declaration, a collision between two properties generating one option or one variable, a missing required property, any other document diagnostic — is -repairable: the agent is asked for a replacement. A defect the caller wrote — an -option no draft declares, malformed aggregate JSON, a value the schema cannot -decode, an extra positional a draft's arity exposes, or a signature change — -terminates the command with no repair, no review, no save and no run. +repairable: the host's assessment answers `valid: false` with the structured +findings, and the command document may ask for a replacement. A defect the caller +wrote — an option no draft declares, malformed aggregate JSON, a value the schema +cannot decode, an extra positional a draft's arity exposes, or a signature change +— raises out of that assessment instead, ending the command document with no +repair, no review, no save and no run. That document cannot catch it and cannot +recategorize it as feedback for an agent that could not have caused it. + +The props the approved Plan runs with are resolved once more, after the command +document has completely torn down, from those exact returned bytes and the +original unchanged sources. No props object any draft produced is reused, so a +revision that changed a property's declared type changes what the run receives. ## Targeted roots From 9ee0c40cdefe14044a47ad59c763cf3ebb6d7e95 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 14:33:38 -0400 Subject: [PATCH 06/15] =?UTF-8?q?=F0=9F=90=9B=20Give=20each=20prompt=20ses?= =?UTF-8?q?sion=20its=20own=20directory=20and=20a=20reachable=20exhaustion?= =?UTF-8?q?=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four corrections to the prompt profile and the command document it runs. One directory per logical session, at `~/.xmd/prompt/sessions/`, replaces the single `~/.xmd/prompt` every invocation shared. A shared directory was one ambient location two conversations could both stand in; a fresh one each time would have left `--session` unable to name anything, since a session's key includes the directory it lives in. The digest gives both: a generated name reaches a location nothing else does, and the same explicit name reaches the same one, so the provider continues the session it established rather than placing a second. The leaf is the digest and never the name — a caller's string that becomes a path is a caller's string that can escape one. Emptiness is now proven rather than assumed. The directory is created empty and required to be empty before the provider is constructed or a session is materialized, and anything already in there is a terminal refusal naming the path. Nothing is deleted: what is in a directory this host did not authorize anything to write to is not this command's to clean up. Stopping on the tenth draft says two different things, and now the document says both. When that draft still has problems, `abort` is the only choice offered and there was never a Plan to approve, so it routes to the authored exhaustion message. Every other abort — including one on a tenth draft that could have been approved — stays the ordinary ending. The branch after the Session remains an exhaustive fallback. Final admission is now shown to be independently effective. A scripted draft uses a repository component that exists while the command document runs and is removed as its scope tears down; the unchanged approved bytes then fail the host's second validation, in that order, with no save, journal or execution. Both validations run the production path — leaving the component in place makes the case pass and execute, which is what makes it evidence. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 4 +- packages/cli/src/documents/prompt-command.md | 16 +- packages/cli/src/prompt-profile.ts | 102 ++++++--- packages/cli/tests/prompt.test.ts | 209 +++++++++++++++++-- packages/cli/tests/support/fake-acp.ts | 5 +- packages/cli/tests/support/prompt-harness.ts | 45 +++- specs/acp-client-spec.md | 15 +- specs/prompt-command-spec.md | 52 +++-- 8 files changed, 373 insertions(+), 75 deletions(-) diff --git a/architecture.md b/architecture.md index 1b067a74a..711f38087 100644 --- a/architecture.md +++ b/architecture.md @@ -36,7 +36,7 @@ Existing documents and code get aligned to this section retroactively. | Prompt | a person's original request, in ordinary natural language. `xmd prompt` takes exactly one | | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. A Plan is what `xmd prompt` approves, optionally saves and runs; it is not a synonym for a workflow, a policy document or any executable Markdown file | | prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the request wording, the draft and repair loops, the check branches, human review, revision, approval, abort and exhaustion — and returns the exact approved Plan source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | -| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line | +| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to remain empty | | expansion | one logical evaluation of an authored executable element within a document execution | | expansion ID | a deterministic identifier for one logical expansion; restoring or retrying that expansion preserves the ID, while a distinct evaluation requested by the document receives another | | Git capability | the contextual interface through which workflow infrastructure queries the Git repository associated with the current working directory | @@ -3545,7 +3545,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of an empty host-owned working directory with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | +| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, and refused rather than cleaned when it is not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md index 923bf9592..f421e2d75 100644 --- a/packages/cli/src/documents/prompt-command.md +++ b/packages/cli/src/documents/prompt-command.md @@ -139,8 +139,19 @@ Three attempts to fix it did not clear everything. These problems remain: +Stopping says two different things depending on where you are. On the tenth +draft, when it still has problems, `abort` is the only choice left and there was +never a Plan to approve — so that ending says so. Everywhere else, including a +tenth draft you could have approved, stopping is your decision and is reported +as one. + + + + + + @@ -157,8 +168,9 @@ explanation. -If you approved a document, return its source unchanged. Otherwise, stop without -saving or running anything. +If you approved a document, return its source unchanged. Nothing should reach +this point without approving or stopping first, so the other branch says what it +would mean if anything did. diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index aa67369c3..008f12222 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -5,16 +5,18 @@ * * `xmd prompt` owns two root document executions with a complete scope boundary * between them. This module is the first one. It supplies that document's - * inputs, a - * constrained Agent provider, Elicitation, the fixed first-party components and - * the host-declared candidate validator, and it exposes no custom root and no - * repository component search: the program it runs is the one the CLI ships. + * inputs, a constrained Agent provider, Elicitation, the fixed first-party + * components and the host-declared draft validator, and it exposes no custom + * root and no repository component search: the document it runs is the one the + * CLI ships. * * The Agent ceiling is assembled here rather than read from the command line, * because it is not the caller's to choose. Writing a Plan is a conversation - * about text; it never lets an agent touch anything. So the provider gets a fresh empty - * directory of this host's own, no additional directories, no MCP servers, an - * empty native-tool allowlist and a private strict denial of every native + * about text; it never lets an agent touch anything. So the provider gets a + * host-owned directory dedicated to this logical session — created empty, and + * required to be empty before anything is built — no additional directories, no + * MCP servers, an empty native-tool allowlist and a private strict denial of + * every native * permission request — one that answers inside the provider and consults no * authored approval scope, so nothing composed around it can widen a ceiling * with nothing in it. `--approve-all`, `--approve-reads` and `--deny-all` @@ -26,7 +28,8 @@ import { Err, Ok, scoped, until } from "effection"; import type { Operation, Result } from "effection"; -import { mkdir } from "node:fs/promises"; +import { createHash } from "node:crypto"; +import { mkdir, readdir } from "node:fs/promises"; import { homedir } from "node:os"; import { join } from "node:path"; @@ -122,8 +125,16 @@ export function* runPromptCommandDocument(profile: PromptProfile): Operation> { - const workdir = yield* useProfileDirectory(); yield* refuseDocumentCapabilities(); yield* profile.installElicitation(); @@ -215,8 +226,9 @@ function validator(profile: PromptProfile): IdentityComponent { /** * The Agent ceiling, stated as the dependencies the provider is built from. * - * Each entry is the whole of one clause: the working directory the agent runs - * in, the MCP servers it configures, the native tools a fresh session may use, + * Each entry is the whole of one clause: the directory the agent runs in — this + * session's own, proven empty above — the MCP servers it configures, the native + * tools a fresh session may use, * and who answers a native permission request. `mcpServers: []` and * `allowedTools: []` are statements rather than omissions — leaving either off * is the backend's default, which is not this host's. @@ -258,24 +270,64 @@ export const PROMPT_INSTRUCTIONS = [ ].join("\n"); /** - * Where the assistant runs while it writes a Plan: this host's own directory, - * and an empty one. + * Where every profile directory lives. * - * The caller's working directory is deliberately not offered — an agent writing - * a document has no reason to read the tree it will run in, and a ceiling that - * starts at the caller's checkout is not a ceiling. Nothing this profile grants - * can write here either, so what is created empty stays empty. + * Under this host's own state directory rather than the caller's tree: an agent + * writing a document has no reason to read the checkout it will run in, and a + * ceiling that starts there is not a ceiling. + */ +export const PROMPT_PROFILE_SESSIONS: string = join(homedir(), ".xmd", "prompt", "sessions"); + +/** + * The directory one logical session's conversation runs in. * - * One fixed path rather than a new directory each time, because a session's - * identity includes the directory it lives in: a location that changed every - * invocation would mean `--session` could never name a conversation that - * already exists, which is the whole of what that option is for. + * Dedicated to that session rather than shared by every invocation, so two + * conversations never see one ambient directory. The leaf is the digest of the + * name and never the name itself: a logical session name is a caller's string, + * and a caller's string that becomes a path is a caller's string that can escape + * one. + * + * A digest also gives the identity `--session` needs. The generated + * invocation-unique name digests to a location nothing else reaches, while the + * same explicit name digests to the same one — which is what lets ACPX find the + * session record it established last time, since a session's key includes the + * directory it lives in. */ -export const PROMPT_PROFILE_DIRECTORY: string = join(homedir(), ".xmd", "prompt"); +export function profileDirectoryFor(session: string): string { + return join(PROMPT_PROFILE_SESSIONS, createHash("sha256").update(session).digest("hex")); +} -function* useProfileDirectory(): Operation { - yield* until(mkdir(PROMPT_PROFILE_DIRECTORY, { recursive: true })); - return PROMPT_PROFILE_DIRECTORY; +/** + * Establish that directory, or refuse. + * + * Created empty, and required to be empty every time — not cleaned. Whatever is + * in there was put there by something this host did not authorize, and deleting + * a stranger's files to get on with the work is the opposite of what a ceiling + * is for. So the command says what it found and where, and stops. + */ +function* useProfileDirectory(session: string): Operation> { + const directory = profileDirectoryFor(session); + try { + yield* until(mkdir(directory, { recursive: true })); + const entries = yield* until(readdir(directory)); + if (entries.length > 0) { + return Err( + new Error( + `${directory} is not empty, and xmd prompt writes a Plan in a directory of its own ` + + "with nothing in it. Move or remove what is in there, or name a different " + + "--session; nothing was written or run", + ), + ); + } + return Ok(directory); + } catch (error) { + return Err( + new Error( + `could not establish ${directory}: ` + + (error instanceof Error ? error.message : String(error)), + ), + ); + } } /** diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 6465ef791..d814bc846 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -20,19 +20,24 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; import { ensure, scoped, until } from "effection"; import type { Operation } from "effection"; -import { writeTextFile } from "@effectionx/fs"; +import { ensureDir, readTextFile, rm, writeTextFile } from "@effectionx/fs"; import { readdir } from "node:fs/promises"; -import { join } from "node:path"; +import { join, sep } from "node:path"; import { API } from "@executablemd/runtime"; import { Elicitation } from "@executablemd/core"; import type { ElicitationRequest } from "@executablemd/core"; import { runPrompt } from "../src/prompt.ts"; import type { PromptCommand } from "../src/prompt.ts"; -import { PROMPT_INSTRUCTIONS, PROMPT_PROFILE_DIRECTORY } from "../src/prompt-profile.ts"; +import { + profileDirectoryFor, + PROMPT_INSTRUCTIONS, + PROMPT_PROFILE_SESSIONS, +} from "../src/prompt-profile.ts"; import { scanPromptArgs } from "../src/prompt-args.ts"; import type { AgentStack } from "../src/agent-stack.ts"; import { AGENT, createPromptHarness, useWorkingDirectory } from "./support/prompt-harness.ts"; +import { makeStore } from "./support/fake-acp.ts"; import type { PromptHarness } from "./support/prompt-harness.ts"; const REQUEST = "write a greeting"; @@ -216,8 +221,9 @@ describe( it("C4: one Session carries every turn, and --session names it", function* () { // Initial, repair and revision turns all land in one conversation, and a - // second invocation places a different one. + // second invocation places a different one in a directory of its own. const keys: string[] = []; + const directories: string[] = []; for (const _invocation of [0, 1]) { yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); @@ -235,16 +241,24 @@ describe( expect(harness.fake.prompts).toHaveLength(5); expect(sessions(harness)).toHaveLength(1); keys.push(sessions(harness)[0]); + directories.push(String(harness.fake.created[0]?.cwd)); }); } expect(keys[0]).not.toBe(keys[1]); + // Different conversations, so different directories: a generated name + // reaches a location nothing else does. + expect(directories[0]).not.toBe(directories[1]); // `--session` replaces the generated name, so two invocations name one - // conversation and the provider's ordinary continuation applies. + // conversation in one directory — and one ACPX store is what turns that + // from equal keys into an actually continued session. + const store = makeStore(); const named: string[] = []; + const namedDirectories: string[] = []; + const materializations: (string | undefined)[] = []; for (const _invocation of [0, 1]) { yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + const harness = createPromptHarness({ store }); harness.fake.script({ reply: VALID }); harness.script({ decision: "approve" }); @@ -254,17 +268,30 @@ describe( ); expect(code).toBe(0); named.push(sessions(harness)[0]); + namedDirectories.push(String(harness.fake.created[0]?.cwd)); + materializations.push(harness.fake.ensured[0]?.materialization); }); } expect(named[0]).toBe(named[1]); expect(named[0]).not.toBe(keys[0]); + expect(namedDirectories[0]).toBe(namedDirectories[1]); + // The name never reaches the path, and the digest is what the host derived + // from it. + expect(namedDirectories[0]).toBe(profileDirectoryFor("ada")); + expect(namedDirectories[0]).not.toContain("ada"); + expect(namedDirectories[0].startsWith(`${PROMPT_PROFILE_SESSIONS}${sep}`)).toBe(true); + + // The second invocation continued the record the first established rather + // than placing a second one: the store holds one, and only the first + // ensure asked for a session to be materialized by its first turn. + expect([...store.records.keys()]).toEqual([named[0]]); + expect(materializations).toEqual(["first-turn-acceptance", undefined]); }); it("C5: the prompt profile's ceiling is the host's, and no flag widens it", function* () { - // An empty directory of the host's own, no MCP servers, no native tools — - // observed while the command document is still running, because the - // directory is gone - // by the time the command returns. + // This session's own directory, empty, no MCP servers and no native tools + // — observed while the command document is still running, because that is + // the only moment the claim is about. yield* useWorkingDirectory(function* (dir) { yield* writeTextFile(join(dir, "secret.txt"), "the caller's tree\n"); @@ -300,12 +327,16 @@ describe( ); }; - const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + const code = yield* runPrompt( + { ...command(dir, [REQUEST]), session: "ceiling" }, + harness.deps, + ); expect(code).toBe(0); - // Not the caller's working directory, and empty. + // Not the caller's working directory: this session's, and empty while + // the conversation ran. expect(seen.cwd).not.toBe(dir); - expect(seen.cwd).toBe(PROMPT_PROFILE_DIRECTORY); + expect(seen.cwd).toBe(profileDirectoryFor("ceiling")); expect(seen.entries).toEqual([]); // Stated rather than omitted: this host configures no MCP server and // allows no native tool on a fresh session. @@ -319,10 +350,41 @@ describe( ]); }); + // Something already in that directory is a refusal, not a cleanup. It + // happens before the provider exists, so no session is placed and no turn + // is started — and what was there is still there afterwards. + yield* useWorkingDirectory(function* (dir) { + const occupied = profileDirectoryFor("occupied"); + yield* ensureDir(occupied); + yield* writeTextFile(join(occupied, "someone-elses.txt"), "not mine to delete\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: VALID }); + + const { value, lines } = yield* reported(() => + runPrompt({ ...command(dir, [REQUEST]), session: "occupied" }, harness.deps), + ); + + expect(value).toBe(1); + expect(lines.join("\n")).toContain("is not empty"); + expect(lines.join("\n")).toContain("name a different --session"); + expect(lines.join("\n")).toContain(occupied); + // Nothing downstream of the refusal happened. + expect(harness.fake.created).toHaveLength(0); + expect(harness.fake.ensured).toHaveLength(0); + expect(harness.fake.started).toBe(false); + expect(harness.fake.prompts).toHaveLength(0); + expect(harness.reviews).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + // The contents were left exactly as they were found. + expect(yield* readTextFile(join(occupied, "someone-elses.txt"))).toBe( + "not mine to delete\n", + ); + }); + // `--approve-all` configures the approved document. A native permission // request while the Plan is being written is still denied, privately, and - // the turn it - // belongs to fails. + // the turn it belongs to fails. yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); harness.fake.script({ reply: VALID, requestsTool: "Bash" }); @@ -572,6 +634,40 @@ describe( ); expect(harness.executions).toHaveLength(0); }); + + // Exhaustion is reachable, and it is a different ending. Ten presentations + // that never validated, the last offering nothing but `abort`, and the + // sentence says there was never a Plan to approve rather than that + // somebody decided to stop. + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness(); + for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { + // A base draft and its three repairs, none of which validates. + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script( + round < 10 ? { decision: "revise", feedback: `round ${round}` } : { decision: "abort" }, + ); + } + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST], STACK), harness.deps), + ); + + expect(value).toBe(1); + expect(harness.reviews).toHaveLength(10); + // Rounds one to nine could be sent back; the tenth had one choice, and + // taking it is what reaches the exhaustion ending. + expect(decisions(harness.reviews[8])).toEqual(["revise", "abort"]); + expect(decisions(harness.reviews[9])).toEqual(["abort"]); + expect(lines).toHaveLength(1); + expect(lines[0]).toBe( + "xmd prompt: ten drafts were reviewed and none was approved, so nothing was saved " + + "and nothing ran. Try again with a more specific request.", + ); + expect(harness.executions).toHaveLength(0); + }); }); it("C10, C11: the approved bytes are what runs, and props are theirs", function* () { @@ -609,6 +705,74 @@ describe( }); }); + it("C10: final admission vetoes after the profile has torn down", function* () { + yield* useWorkingDirectory(function* (dir) { + // A repository component the draft uses. It exists while the command + // document runs, so the same production validator that answers + // finds the draft sound. + const widget = join(dir, "Widget.md"); + yield* writeTextFile(widget, "A widget.\n"); + const draft = ["# Uses a widget", "", "", ""].join("\n"); + + const harness = createPromptHarness(); + harness.fake.script({ reply: draft }); + const events: string[] = []; + harness.deps.installElicitation = function* () { + // Registered inside the profile's scope, so it runs while that scope + // is being torn down — after the provider is gone and before the host + // looks at what was approved. + yield* ensure(function* () { + events.push("teardown"); + yield* rm(widget, { force: true }); + }); + yield* Elicitation.around( + { + // deno-lint-ignore require-yield + *elicit([request], _next) { + harness.reviews.push(request); + return { decision: "approve" }; + }, + }, + { at: "min" }, + ); + }; + + const written = console.error; + const lines: string[] = []; + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + console.error = written; + }); + console.error = (...parts: unknown[]) => { + events.push("reported"); + lines.push(parts.map((part) => String(part)).join(" ")); + }; + return yield* runPrompt(command(dir, [REQUEST]), harness.deps); + }); + + // The draft was sound enough to approve, and the approved bytes are + // unchanged — only the tree they resolve against moved. + expect(harness.reviews).toHaveLength(1); + expect(harness.reviews[0].message).toContain(""); + + // The gate is effective on its own: the same bytes now fail, because the + // host validates them again rather than trusting what the document + // concluded while its own scope was still standing. + expect(code).toBe(1); + expect(lines.join("\n")).toContain("the approved document does not validate"); + expect(lines.join("\n")).toContain("component-unresolved"); + expect(lines.join("\n")).toContain("Widget"); + + // And it is ordered: teardown finished first, which is the only reason + // the component was missing when the second validation ran. + expect(events).toEqual(["teardown", "reported"]); + + // Nothing after the veto happened. + expect(harness.executions).toHaveLength(0); + expect((yield* until(readdir(dir))).sort()).toEqual([]); + }); + }); + it("C14: an interleaved Plan survives approval and execution byte for byte", function* () { yield* useWorkingDirectory(function* (dir) { // What the shipped instruction asks for: the request restated in prose a @@ -689,16 +853,15 @@ describe( expect(revision).not.toContain("The document you write is a Plan"); }); - // Abort and the tenth-round exhaustion each print one sentence, once. - for (const scripted of [ - { rounds: 1, ending: "you ended the review" }, - { rounds: 10, ending: "ten drafts were reviewed and none was approved" }, - ]) { + // Stopping prints one sentence, once, whether it happened on the first + // presentation or the tenth: an approvable Plan existed and you chose to + // stop, and ten rounds of it do not accumulate into a different sentence. + for (const rounds of [1, 10]) { yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness(); - for (const round of Array.from({ length: scripted.rounds }, (_, i) => i + 1)) { + for (const round of Array.from({ length: rounds }, (_, i) => i + 1)) { harness.fake.script({ reply: VALID }); - if (round < scripted.rounds) { + if (round < rounds) { harness.script({ decision: "revise", feedback: `round ${round}` }); } } @@ -713,7 +876,7 @@ describe( // repetition of it however many rounds preceded it. expect(lines).toHaveLength(1); expect(occurrences(lines[0], "xmd prompt:")).toBe(1); - expect(lines[0]).toContain("nothing was saved and nothing ran"); + expect(lines[0]).toContain("you ended the review"); }); } }); diff --git a/packages/cli/tests/support/fake-acp.ts b/packages/cli/tests/support/fake-acp.ts index 7dcd48db8..4a751e095 100644 --- a/packages/cli/tests/support/fake-acp.ts +++ b/packages/cli/tests/support/fake-acp.ts @@ -29,7 +29,10 @@ import type { ProbeCapableRuntime, } from "@executablemd/acp"; -export function makeStore(): AcpxSessionStore & { records: Map } { +/** A session store whose records a case can read back. */ +export type FakeStore = AcpxSessionStore & { records: Map }; + +export function makeStore(): FakeStore { const records = new Map(); return { records, diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts index e55732849..406e4bbe1 100644 --- a/packages/cli/tests/support/prompt-harness.ts +++ b/packages/cli/tests/support/prompt-harness.ts @@ -16,16 +16,18 @@ import { Elicitation } from "@executablemd/core"; import type { ElicitationRequest, SyntaxCatalog } from "@executablemd/core"; import { Ok } from "effection"; import type { Operation, Result } from "effection"; -import { ensure, scoped } from "effection"; +import { ensure, scoped, until } from "effection"; import { ensureDir, rm } from "@effectionx/fs"; import { randomUUID } from "node:crypto"; +import { readdir } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { API, useHostFiles } from "@executablemd/runtime"; +import { PROMPT_PROFILE_SESSIONS } from "../../src/prompt-profile.ts"; import { syntaxCatalog } from "../../src/syntax.ts"; import type { PromptDependencies, PromptExecution } from "../../src/prompt.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; -import type { FakeAcp } from "./fake-acp.ts"; +import type { FakeAcp, FakeStore } from "./fake-acp.ts"; /** The agent every prompt case drives, and the command it resolves to. */ export const AGENT = "scripted-agent"; @@ -57,6 +59,14 @@ export interface PromptHarness { export function createPromptHarness(options?: { /** Replace the catalog entirely, for a case about catalog failure. */ catalog?: (includes: readonly string[]) => Operation; + /** + * The ACPX session store this invocation reads and writes. + * + * One store shared by two harnesses is two invocations of the command against + * one provider's memory, which is the only way to observe whether a named + * session is continued or created a second time. + */ + store?: FakeStore; }): PromptHarness { const fake = createFakeAcp(); const catalogCalls: string[][] = []; @@ -77,7 +87,7 @@ export function createPromptHarness(options?: { deps: { acp: { createRuntime: fake.create, - sessionStore: makeStore(), + sessionStore: options?.store ?? makeStore(), agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), }, *catalog(includes) { @@ -127,6 +137,7 @@ export function* useWorkingDirectory(body: (dir: string) => Operation): Op yield* ensureDir(dir); return yield* scoped(function* () { yield* ensure(() => rm(dir, { recursive: true, force: true })); + yield* useProfileDirectories(); yield* API.Env.around({ // deno-lint-ignore require-yield *cwd() { @@ -178,3 +189,31 @@ export function* useRecordedEnvironment( export function timesRead(reads: readonly string[], name: string): number { return reads.filter((read) => read === name).length; } + +/** + * Leave the host's profile session directories as this case found them. + * + * A profile directory is deliberately durable — its stable identity is what + * `--session` continues — so a suite that made a few dozen of them under the + * developer's own home directory would be leaving litter behind. Only entries + * that appeared while the case ran are removed, so a directory somebody else + * owns is never touched. + */ +function* useProfileDirectories(): Operation { + const before = new Set(yield* listSessionDirectories()); + yield* ensure(function* () { + for (const name of yield* listSessionDirectories()) { + if (!before.has(name)) { + yield* rm(join(PROMPT_PROFILE_SESSIONS, name), { recursive: true, force: true }); + } + } + }); +} + +function* listSessionDirectories(): Operation { + try { + return yield* until(readdir(PROMPT_PROFILE_SESSIONS)); + } catch { + return []; + } +} diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index f26a16f42..3749836bf 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -549,7 +549,7 @@ document and not readable from the command line: | The profile's provider gets | Stated as | | --- | --- | -| a host-owned working directory that is not the caller's | `agentCwd`, one fixed empty path | +| one host-owned directory per logical session | `agentCwd`, `~/.xmd/prompt/sessions/`, required empty | | no MCP servers | `mcpServers: []`, an empty set rather than an omission | | no native tools on a fresh session | `newSessionOptions.allowedTools: []` | | a private refusal of every native permission request | `permissions: "strict"` | @@ -566,10 +566,15 @@ network capability, and the host decides for that whole execution that a failing `` ends it — so a turn that streamed text and then failed presents nothing. -The profile's working directory is one fixed host-owned path rather than a new one each -invocation, because a session's key includes the directory it lives in: a -location that changed every time would leave `--session` unable to name a -conversation that already exists. Nothing this profile grants can write there. +The profile's working directory is derived from the logical session name rather +than shared or freshly made: `~/.xmd/prompt/sessions/`, with the +digest in the path and never the name. A session's key includes the directory it +lives in, so a shared location would put two conversations in one ambient +directory while a fresh one would leave `--session` unable to name a conversation +that already exists. It is created empty and required to be empty before the +provider is constructed or a session is materialized; a non-empty one is a +terminal refusal rather than something to clean. Nothing this profile grants can +write there. Only the profile's provider receives `newSessionOptions.systemPrompt`. It carries the fixed statement of what the session writes, and nothing else — the catalog and diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index fa065e32e..29fdbce52 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -194,7 +194,8 @@ exposes no custom root. The assistant that writes a Plan is assembled separately from the final run provider: -- a host-owned working directory that is not the caller's and is created empty; +- one host-owned directory dedicated to the logical prompt session, created + empty and required to remain empty; - no additional directories; - no MCP servers — stated as an empty set, not omitted; - an empty requested native-tool allowlist on a fresh session; @@ -211,11 +212,25 @@ They apply to the approved Plan later. A provider that cannot establish this ceiling refuses before session materialization or a turn; there is no silent downgrade. -The profile's working directory is one fixed host-owned path rather than a new -directory each invocation, because a session's identity includes the directory it -lives in: a location that changed every time would mean `--session` could never -name a conversation that already exists. Nothing this profile grants can write -there, so what is created empty stays empty. +The profile's working directory is +`~/.xmd/prompt/sessions/`. It is dedicated to that +one logical session rather than shared by every invocation, so two conversations +never see one ambient directory. The leaf is the digest and never the name +itself: a logical session name is a caller's string, and a caller's string that +becomes a path is one that can escape a path. + +The digest also carries the identity `--session` needs. A generated +invocation-unique name reaches a location nothing else does; the same explicit +name reaches the same one, and because a session's key includes the directory it +lives in, that is what lets the provider continue the session it established. + +The directory is created empty on first use and **required to be empty** before +the provider is constructed or a session is materialized. A non-empty directory +is a terminal, actionable refusal naming the path: nothing is deleted or cleaned, +because whatever is in there was put there by something this host did not +authorize. It may remain after execution — its stable identity is part of +named-session continuation — and nothing this profile grants can write there, so +what is created empty stays empty. The approved Plan later receives the caller-selected ordinary run Agent and permission configuration. It inherits neither the assistant Session nor its @@ -375,10 +390,19 @@ complete JSON problems, serialized by ``. Prose outside `` addresses you; text inside `` instructs the assistant. **Failure.** `approve` selects the draft, and one exhaustive branch after the -Session either returns its source unchanged or stops without saving or running -anything. `abort` and a review that approved nothing -reach `` with an actionable message. Failure is authored in Markdown rather -than hidden in the host or represented by a missing-`` accident. +Session returns its source unchanged. Stopping reaches `` with one of two +authored messages, and which one depends on whether an approvable Plan ever +existed: + +- the tenth presentation of a draft that still has problems offers only `abort`, + and taking that sole choice is **exhaustion** — ten drafts were reviewed and + none was approved; +- every other `abort`, including one on a tenth draft that could have been + approved, is the ordinary ending: you decided to stop. + +The branch after the Session stays as an exhaustive fallback for a body that +somehow reached it having approved nothing. Failure is authored in Markdown +rather than hidden in the host or represented by a missing-`` accident. The command document's rendered output is not command output. Everything you see while a Plan is written reaches you through Elicitation, and that document's @@ -469,13 +493,13 @@ produced. | C1 | Fixed grammar and help | Request cardinality, individual-property ordering, aggregate props before the request, `--session` including its empty-value refusal, and effect-free generic help | | C2 | Exact packaged root | The command executes the checked-in Markdown value root under ``; the turn text is that document's own words, and no TypeScript authorship loop or custom root chooses policy | | C3 | Visible policy | Generation, three-turn repair, ten-round review, revision, approval, abort and exhaustion are present in Markdown; `` remains one turn | -| C4 | One Session | One enclosing Session expansion carries every turn; default names differ across invocations and `--session` supplies the exact override | -| C5 | Prompt profile ceiling | An empty host-owned working directory, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it | +| C4 | One Session | One enclosing Session expansion carries every turn; two default invocations get different profile directories and session keys, two `--session` invocations get the same directory and key with the raw name absent from the path, and two named invocations sharing one ACPX store continue the established record rather than placing a second | +| C5 | Prompt profile ceiling | This session's own host-owned directory, empty while the command document runs, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it; pre-existing content in that directory refuses before any provider, session, turn, review, save or execution and is left untouched | | C6 | Draft inertness | A draft is only data while the Plan is written, and no draft effect occurs before the final execution | | C7 | Validation classification | Candidate failures return structured facts; caller-source failures escape immediately; frozen signatures are checked before token extraction | | C8 | Bounds | One base plus three automatic repairs per draft, and no more than ten human presentations, with no revision offered on the last | -| C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; abort and exhaustion reach ``, not host policy and not a missing `` | -| C10 | Final gate | The host revalidates after the command document has completely torn down, and resolves props for the exact returned bytes | +| C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; an ordinary abort and a tenth invalid presentation's sole `abort` reach their two distinct authored `` messages, not host policy and not a missing `` | +| C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no save, journal or execution — and resolves props for the exact returned bytes | | C11 | Exact bytes | Approval, exclusive save and `` execution receive the Agent close value without rewriting or fence removal | | C12 | Journal separation | Authorship uses only disposable in-memory history; the final journal begins with the approved Plan | | C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution | From 6348b73b99bf2aaa0d1557cd549887a92b01ffa9 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 14:54:32 -0400 Subject: [PATCH 07/15] =?UTF-8?q?=F0=9F=90=9B=20Own=20the=20profile=20root?= =?UTF-8?q?=20in=20tests,=20and=20give=20a=20default=20session's=20directo?= =?UTF-8?q?ry=20back=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two ownership corrections: one about which directories a test may touch, and one about which directories an invocation may keep. The suite no longer reads, creates or removes anything under a real `~/.xmd/prompt`. Snapshotting that root and deleting whatever appeared after the snapshot was unsafe on its face — absence from a snapshot says nothing about who created a directory, so an overlapping case or a real invocation could have its state removed by a test that never owned it. Where profile session directories live is now a host dependency: production keeps its default, and a harness names a tree it made itself, passed as a value with no flag, environment variable, document prop or replaceable context anywhere near it. `createPromptHarness` requires that root, so no case can fall back to the host's, and each root is a per-scope temporary directory removed whole — which is safe precisely because the scope created everything in it. The one case that needs two invocations to share a root says so explicitly. Only an explicitly named session needs its directory afterwards. Its identity is what a later `--session` derives the same ACPX session from, so it stays, is required empty on the way in every time, and is never cleaned. An invocation-unique default names nothing anybody can ask for again, so its directory belongs to the command: after the command document and every provider, Prompt task and Elicitation resource inside it has torn down, exactly one cleanup is attempted, and it settles before final admission, the save or the execution begins. Still empty, and the leaf is removed non-recursively. No longer empty, and the directory and its contents are preserved and the command fails terminally — this host authorized nothing to write there, and deleting a stranger's files to get on with the work is the opposite of what a ceiling is for. Being an `ensure` is what makes an abort, a failed turn and a cancellation settle the same way a success does. Which of the two applies is a trusted host value — whether the caller wrote `--session` — never inferred from the shape of the generated name, and never visible to the command document. Both outcomes are held by evidence that fails without them: disabling the cleanup leaves the leaf behind on success and before admission, and making the removal recursive turns the preserved-and-refused case into a silent success. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 4 +- packages/cli/src/prompt-profile.ts | 99 ++++- packages/cli/src/prompt.ts | 16 +- packages/cli/tests/prompt-cli.test.ts | 64 +-- packages/cli/tests/prompt.test.ts | 407 +++++++++++++++---- packages/cli/tests/support/prompt-harness.ts | 68 ++-- specs/acp-client-spec.md | 9 + specs/prompt-command-spec.md | 45 +- 8 files changed, 526 insertions(+), 186 deletions(-) diff --git a/architecture.md b/architecture.md index 711f38087..aae86a3d0 100644 --- a/architecture.md +++ b/architecture.md @@ -36,7 +36,7 @@ Existing documents and code get aligned to this section retroactively. | Prompt | a person's original request, in ordinary natural language. `xmd prompt` takes exactly one | | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. A Plan is what `xmd prompt` approves, optionally saves and runs; it is not a synonym for a workflow, a policy document or any executable Markdown file | | prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the request wording, the draft and repair loops, the check branches, human review, revision, approval, abort and exhaustion — and returns the exact approved Plan source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | -| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to remain empty | +| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when still empty, and preserved whole with the command failing terminally when it is not. Where those directories live is a host dependency no caller or document selects | | expansion | one logical evaluation of an authored executable element within a document execution | | expansion ID | a deterministic identifier for one logical expansion; restoring or retrying that expansion preserves the ID, while a distinct evaluation requested by the document receives another | | Git capability | the contextual interface through which workflow infrastructure queries the Git repository associated with the current working directory | @@ -3545,7 +3545,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, and refused rather than cleaned when it is not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | +| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index 008f12222..b7ca0ce53 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -26,10 +26,10 @@ * network capability either. It decides what to write; it writes nothing. */ -import { Err, Ok, scoped, until } from "effection"; +import { ensure, Err, Ok, scoped, until } from "effection"; import type { Operation, Result } from "effection"; import { createHash } from "node:crypto"; -import { mkdir, readdir } from "node:fs/promises"; +import { mkdir, readdir, rmdir } from "node:fs/promises"; import { homedir } from "node:os"; import { join } from "node:path"; @@ -87,6 +87,24 @@ export interface PromptProfile { syntax: string; /** The logical name every turn in this invocation belongs to. */ session: string; + /** + * Whether the caller named that session. + * + * A trusted host value rather than something read back out of the name: only + * the host knows whether `--session` was written, and the difference decides + * whether this conversation's directory outlives the invocation. It reaches + * the command document nowhere. + */ + explicitSession: boolean; + /** + * Where this host keeps its profile session directories. + * + * A host dependency, not a caller's: no flag, environment variable, document + * prop or replaceable context reaches it. Production leaves it at the default + * below; a harness that owns a temporary tree supplies that tree instead, so a + * test never reads, creates or removes anything under a real one. + */ + root: string; /** The one Agent configuration this invocation settled. */ stack: AgentStack; /** What the constrained provider is built on, beyond the host's assembly. */ @@ -125,16 +143,24 @@ export function* runPromptCommandDocument(profile: PromptProfile): Operation> { + // First, and before anything is built: this session's directory is + // established and proven empty, or the command stops here. Nothing has been + // installed yet, so a refusal reaches no provider, no session and no turn. + const established = yield* establishDirectory(profile.root, profile.session); + if (!established.ok) { + return established; + } + const workdir = established.value; + if (!profile.explicitSession) { + // Registered before every other resource, so it runs after all of them: + // the provider, the Prompt tasks and the Elicitation provider are already + // gone when this hands the directory back. Being an `ensure` is what makes + // an abort, a failed turn and a cancellation clean up the same way a + // success does. + yield* ensure(() => removeEmptyDirectory(workdir)); + } + yield* refuseDocumentCapabilities(); yield* profile.installElicitation(); @@ -270,13 +296,13 @@ export const PROMPT_INSTRUCTIONS = [ ].join("\n"); /** - * Where every profile directory lives. + * Where this host keeps its profile session directories by default. * - * Under this host's own state directory rather than the caller's tree: an agent - * writing a document has no reason to read the checkout it will run in, and a - * ceiling that starts there is not a ceiling. + * Under its own state directory rather than the caller's tree: an agent writing + * a document has no reason to read the checkout it will run in, and a ceiling + * that starts there is not a ceiling. */ -export const PROMPT_PROFILE_SESSIONS: string = join(homedir(), ".xmd", "prompt", "sessions"); +export const DEFAULT_PROFILE_ROOT: string = join(homedir(), ".xmd", "prompt", "sessions"); /** * The directory one logical session's conversation runs in. @@ -293,20 +319,20 @@ export const PROMPT_PROFILE_SESSIONS: string = join(homedir(), ".xmd", "prompt", * session record it established last time, since a session's key includes the * directory it lives in. */ -export function profileDirectoryFor(session: string): string { - return join(PROMPT_PROFILE_SESSIONS, createHash("sha256").update(session).digest("hex")); +export function profileDirectoryFor(root: string, session: string): string { + return join(root, createHash("sha256").update(session).digest("hex")); } /** - * Establish that directory, or refuse. + * Establish this session's directory, or refuse. * * Created empty, and required to be empty every time — not cleaned. Whatever is * in there was put there by something this host did not authorize, and deleting * a stranger's files to get on with the work is the opposite of what a ceiling * is for. So the command says what it found and where, and stops. */ -function* useProfileDirectory(session: string): Operation> { - const directory = profileDirectoryFor(session); +function* establishDirectory(root: string, session: string): Operation> { + const directory = profileDirectoryFor(root, session); try { yield* until(mkdir(directory, { recursive: true })); const entries = yield* until(readdir(directory)); @@ -330,6 +356,37 @@ function* useProfileDirectory(session: string): Operation> { } } +/** + * Give an invocation-unique conversation's directory back when it is over. + * + * Only an explicitly named session needs its directory afterwards: its identity + * is what a later `--session` derives the same ACPX session from. A generated + * name names nothing anybody can ask for again, so its directory is this scope's + * and goes away with it — before the approved Plan is validated, saved or run. + * + * Non-recursive, always. If something is in there, this host did not authorize + * whatever put it there, and the honest answer is to say so rather than to + * delete it: the same refusal an occupied directory earns on the way in. + */ +function* removeEmptyDirectory(directory: string): Operation { + try { + yield* until(rmdir(directory)); + } catch (error) { + const code = error instanceof Error && "code" in error ? error.code : undefined; + if (code === "ENOENT") { + return; + } + if (code === "ENOTEMPTY" || code === "EEXIST") { + throw new Error( + `${directory} was empty when this conversation started and is not now. It belongs to ` + + "one invocation, so nothing should have written there; its contents were left alone " + + "and nothing was saved or run", + ); + } + throw error instanceof Error ? error : new Error(String(error)); + } +} + /** * The capabilities the prompt command document does not get. * diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index 8e561931d..ce5289dfa 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -56,7 +56,7 @@ import type { AcpxProviderDependencies } from "@executablemd/acp"; import { cwd } from "@executablemd/runtime"; import type { AgentStack } from "./agent-stack.ts"; -import { runPromptCommandDocument } from "./prompt-profile.ts"; +import { DEFAULT_PROFILE_ROOT, runPromptCommandDocument } from "./prompt-profile.ts"; import type { CandidateAssessment } from "./prompt-profile.ts"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { @@ -124,6 +124,15 @@ export interface PromptDependencies { catalog(includes: readonly string[]): Operation; /** Who answers the review question. */ installElicitation(): Operation; + /** + * Where this host keeps its profile session directories. + * + * Absent is the ordinary host default. A harness that owns a temporary tree + * names that tree here, which is the only way anything but production selects + * one — there is no flag, no environment variable and no contextual Api to + * reach, so a document cannot move where the ceiling lives. + */ + profileRoot?: string; /** Run the approved document the way this host runs any supplied one. */ execute(approved: PromptExecution): Operation>; } @@ -195,6 +204,11 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op request, syntax, session: command.session ?? invocationSessionName(), + // Read from what the caller wrote, not from the shape of the name. Only a + // session somebody can ask for again needs its directory to outlive the + // invocation, and only the host knows whether somebody named one. + explicitSession: command.session !== undefined, + root: deps.profileRoot ?? DEFAULT_PROFILE_ROOT, stack: command.stack, ...(deps.acp === undefined ? {} : { acp: deps.acp }), installElicitation: deps.installElicitation, diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 7ade28b48..94dce6bfa 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -152,8 +152,8 @@ describe( { sanitizeOps: false, sanitizeResources: false }, () => { it("C1: a misplaced individual option refuses before any phase begins", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); const code = yield* runPrompt( command(dir, ["--props-name", "Ada", REQUEST], "out.md"), harness.deps, @@ -261,8 +261,8 @@ describe( }); it("C15: individual, aggregate and environment sources resolve and reach the run", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: GREETER }); harness.script({ decision: "approve" }); @@ -284,8 +284,8 @@ describe( const journalName = "trace.jsonl"; // Abort at review, through the command document's authored failure. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, join(dir, journalName)); harness.fake.script({ reply: PLAIN }); harness.script({ decision: "abort" }); @@ -299,8 +299,8 @@ describe( }); // A turn that produced text and then failed. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, join(dir, journalName)); harness.fake.script({ reply: PLAIN, stopReason: "refusal" }); @@ -313,8 +313,8 @@ describe( }); // A terminal property-source failure. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, join(dir, journalName)); harness.fake.script({ reply: PLAIN }); @@ -331,9 +331,9 @@ describe( }); // An approved run's journal holds the document's events and no authorship. - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { const journal = join(dir, journalName); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, journal); harness.fake.script({ reply: PLAIN }); harness.script({ decision: "approve" }); @@ -354,8 +354,8 @@ describe( it("C11: the approved bytes are created exclusively, before the run", function* () { // Created before execution, and byte for byte. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); const run = executor(dir); const seen: boolean[] = []; harness.deps.execute = function* (approved) { @@ -378,9 +378,9 @@ describe( }); // An existing path is left exactly as it is, and stops the run. - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { yield* writeTextFile(join(dir, "out.md"), "keep me\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: PLAIN }); harness.script({ decision: "approve" }); @@ -393,8 +393,8 @@ describe( }); // Without the option, no generated source file is created at all. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: GREETER }); harness.script({ decision: "approve" }); @@ -408,9 +408,9 @@ describe( }); it("C15: the approved source runs as an ordinary document under ", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { const journal = join(dir, "trace.jsonl"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, journal); harness.fake.script({ reply: GREETER }); harness.script({ decision: "approve" }); @@ -431,8 +431,8 @@ describe( // A runtime failure is an ordinary run failure: nonzero, with the save the // caller asked for still on disk to hand-edit. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: FAILS_AT_RUN }); harness.script({ decision: "approve" }); @@ -448,8 +448,8 @@ describe( }); it("C15: a failing test in the approved document reports as a run reports it", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: FAILING_TEST }); harness.script({ decision: "approve" }); @@ -482,7 +482,7 @@ describe( }); it("C15: one Agent resolution serves generation and the execution after it", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { const reads: string[] = []; yield* useRecordedEnvironment(reads, { DEFAULT_AGENT_NAME: "settled-agent" }); @@ -504,7 +504,7 @@ describe( expect(stack.defaultAgent).toBe("settled-agent"); expect(timesRead(reads, "DEFAULT_AGENT_NAME")).toBe(1); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, undefined, stack); harness.fake.script({ reply: PLAIN }); harness.script({ decision: "approve" }); @@ -527,8 +527,8 @@ describe( // the turn in flight is cancelled, the provider is dismantled, and no later // phase begins. The barrier is what makes this a gate rather than a race — // the turn it interrupts is known to be running. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: PLAIN, manual: true }); @@ -548,8 +548,8 @@ describe( // A teardown failure prevents the final validation, the save and the run, // whatever the command document selected. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.closeFailure = new Error("the profile provider would not close"); harness.fake.script({ reply: PLAIN }); @@ -564,8 +564,8 @@ describe( // Nothing bounds an authoring turn: the exec and fetch defaults belong to // the document, and the run deadline is the enclosing timebox above. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); // deno-lint-ignore require-yield harness.deps.execute = function* () { return Ok(undefined); diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index d814bc846..28dd5d742 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -18,25 +18,30 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { ensure, scoped, until } from "effection"; +import { ensure, scoped, spawn, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, readTextFile, rm, writeTextFile } from "@effectionx/fs"; import { readdir } from "node:fs/promises"; import { join, sep } from "node:path"; -import { API } from "@executablemd/runtime"; +import { API, stat } from "@executablemd/runtime"; import { Elicitation } from "@executablemd/core"; import type { ElicitationRequest } from "@executablemd/core"; import { runPrompt } from "../src/prompt.ts"; import type { PromptCommand } from "../src/prompt.ts"; import { + DEFAULT_PROFILE_ROOT, profileDirectoryFor, PROMPT_INSTRUCTIONS, - PROMPT_PROFILE_SESSIONS, } from "../src/prompt-profile.ts"; import { scanPromptArgs } from "../src/prompt-args.ts"; import type { AgentStack } from "../src/agent-stack.ts"; -import { AGENT, createPromptHarness, useWorkingDirectory } from "./support/prompt-harness.ts"; +import { + AGENT, + createPromptHarness, + useProfileRoot, + useWorkingDirectory, +} from "./support/prompt-harness.ts"; import { makeStore } from "./support/fake-acp.ts"; import type { PromptHarness } from "./support/prompt-harness.ts"; @@ -143,6 +148,35 @@ function decisions(request: ElicitationRequest): unknown { return (decision as Record).enum; } +/** + * A review provider that looks at the profile directory before it answers. + * + * Installed in place of the harness's own, so the observation happens inside the + * profile's scope — the only moment the directory exists. + */ +function watching( + harness: PromptHarness, + observe: (workdir: string) => Operation, +): () => Operation { + return function* () { + yield* Elicitation.around( + { + *elicit([request], _next) { + harness.reviews.push(request); + yield* observe(String(harness.fake.created[0]?.cwd)); + return { decision: "approve" }; + }, + }, + { at: "min" }, + ); + }; +} + +/** Whether a path is there at all. */ +function* exists(path: string): Operation { + return (yield* stat(path)).exists; +} + /** How many times one exact string appears. */ function occurrences(haystack: string, needle: string): number { return haystack.split(needle).length - 1; @@ -169,12 +203,12 @@ describe( { sanitizeOps: false, sanitizeResources: false }, () => { it("C2, C3, C14: the packaged program's own words ask for the document", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { // A repository TypeScript component, so the catalog has to state the one // thing it honestly cannot know without importing the module. yield* writeTextFile(join(dir, "Widget.ts"), "export default function Widget() {}\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); harness.script({ decision: "approve" }); @@ -225,8 +259,8 @@ describe( const keys: string[] = []; const directories: string[] = []; for (const _invocation of [0, 1]) { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); @@ -241,7 +275,11 @@ describe( expect(harness.fake.prompts).toHaveLength(5); expect(sessions(harness)).toHaveLength(1); keys.push(sessions(harness)[0]); - directories.push(String(harness.fake.created[0]?.cwd)); + const workdir = String(harness.fake.created[0]?.cwd); + directories.push(workdir); + // This suite reaches no directory this host would use for real. + expect(workdir.startsWith(`${profileRoot}${sep}`)).toBe(true); + expect(workdir.startsWith(DEFAULT_PROFILE_ROOT)).toBe(false); }); } expect(keys[0]).not.toBe(keys[1]); @@ -251,51 +289,213 @@ describe( // `--session` replaces the generated name, so two invocations name one // conversation in one directory — and one ACPX store is what turns that - // from equal keys into an actually continued session. - const store = makeStore(); - const named: string[] = []; - const namedDirectories: string[] = []; - const materializations: (string | undefined)[] = []; - for (const _invocation of [0, 1]) { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness({ store }); - harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + // from equal keys into an actually continued session. The root is shared + // deliberately, and by this case alone. + yield* useProfileRoot(function* (profileRoot) { + const store = makeStore(); + const named: string[] = []; + const namedDirectories: string[] = []; + const materializations: (string | undefined)[] = []; + const survived: boolean[] = []; + for (const _invocation of [0, 1]) { + yield* useWorkingDirectory(function* (dir) { + const harness = createPromptHarness({ profileRoot, store }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt( + { ...command(dir, [REQUEST]), session: "ada" }, + harness.deps, + ); + expect(code).toBe(0); + named.push(sessions(harness)[0]); + namedDirectories.push(String(harness.fake.created[0]?.cwd)); + materializations.push(harness.fake.ensured[0]?.materialization); + // A named conversation's directory outlives the invocation, because + // its identity is what the next `--session ada` derives. + survived.push(yield* exists(profileDirectoryFor(profileRoot, "ada"))); + }); + } + expect(named[0]).toBe(named[1]); + expect(namedDirectories[0]).toBe(namedDirectories[1]); + expect(survived).toEqual([true, true]); + // The name never reaches the path, and the digest is what the host + // derived from it. + expect(namedDirectories[0]).toBe(profileDirectoryFor(profileRoot, "ada")); + expect(namedDirectories[0]).not.toContain("ada"); + + // The second invocation continued the record the first established + // rather than placing a second one: the store holds one, and only the + // first ensure asked for a session to be materialized by its first turn. + expect([...store.records.keys()]).toEqual([named[0]]); + expect(materializations).toEqual(["first-turn-acceptance", undefined]); + }); + }); - const code = yield* runPrompt( - { ...command(dir, [REQUEST]), session: "ada" }, - harness.deps, - ); - expect(code).toBe(0); - named.push(sessions(harness)[0]); - namedDirectories.push(String(harness.fake.created[0]?.cwd)); - materializations.push(harness.fake.ensured[0]?.materialization); + it("C4, C13: a default session's directory belongs to its invocation", function* () { + // It exists and is empty while the turn runs, and it is gone once the + // profile has torn down — before anything the host does with what was + // approved. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID }); + const seen: { workdir?: string; entries?: string[] } = {}; + harness.deps.installElicitation = watching(harness, function* (workdir) { + seen.workdir = workdir; + seen.entries = yield* until(readdir(workdir)); + }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(0); + expect(seen.entries).toEqual([]); + expect(seen.workdir?.startsWith(`${profileRoot}${sep}`)).toBe(true); + // Handed back non-recursively when the conversation ended, and the root + // it lived under is still there for the next one. + expect(yield* exists(String(seen.workdir))).toBe(false); + expect(yield* exists(profileRoot)).toBe(true); + // The approved Plan still ran: cleanup is not a failure. + expect(harness.executions).toHaveLength(1); + }); + + // Abort, a turn that failed, and a cancelled command each hand the + // directory back the same way a success does. + for (const ending of [ + { name: "abort", drive: (harness: PromptHarness) => harness.script({ decision: "abort" }) }, + { + name: "a failed turn", + drive: (harness: PromptHarness) => { + harness.fake.script({ reply: VALID, stopReason: "refusal" }); + }, + }, + ]) { + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + if (ending.name === "abort") { + harness.fake.script({ reply: VALID }); + } + ending.drive(harness); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + + expect(code).toBe(1); + expect(harness.executions).toHaveLength(0); + // That exact leaf, and the root it lived under, both answer the + // question: the directory this ending made is gone, and nothing else + // was made in its place. + expect(yield* exists(String(harness.fake.created[0]?.cwd))).toBe(false); + expect(yield* until(readdir(profileRoot))).toEqual([]); }); } - expect(named[0]).toBe(named[1]); - expect(named[0]).not.toBe(keys[0]); - expect(namedDirectories[0]).toBe(namedDirectories[1]); - // The name never reaches the path, and the digest is what the host derived - // from it. - expect(namedDirectories[0]).toBe(profileDirectoryFor("ada")); - expect(namedDirectories[0]).not.toContain("ada"); - expect(namedDirectories[0].startsWith(`${PROMPT_PROFILE_SESSIONS}${sep}`)).toBe(true); - - // The second invocation continued the record the first established rather - // than placing a second one: the store holds one, and only the first - // ensure asked for a session to be materialized by its first turn. - expect([...store.records.keys()]).toEqual([named[0]]); - expect(materializations).toEqual(["first-turn-acceptance", undefined]); + + // Cancellation: the turn in flight is interrupted, and the ensure that + // hands the directory back runs on the way out like every other one. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID, manual: true }); + + yield* scoped(function* () { + const running = yield* spawn(() => runPrompt(command(dir, [REQUEST]), harness.deps)); + yield* harness.fake.startedTurns(1); + yield* running.halt(); + }); + + expect(yield* exists(String(harness.fake.created[0]?.cwd))).toBe(false); + expect(yield* until(readdir(profileRoot))).toEqual([]); + }); + + // The other outcome of the one attempt. A directory this invocation was + // given empty and did not leave empty is preserved and the command fails: + // something wrote there while the conversation ran, and this host + // authorized nothing to. The draft was approved first, so what is being + // observed is a Plan that would otherwise have been saved and run. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID }); + let workdir: string | undefined; + let planted: string | undefined; + harness.deps.installElicitation = watching(harness, function* (directory) { + workdir = directory; + planted = join(directory, "stowaway.txt"); + yield* writeTextFile(planted, "not this command's doing\n"); + }); + + const { value, lines } = yield* reported(() => + runPrompt({ ...command(dir, [REQUEST]), save: "out.md" }, harness.deps), + ); + + // Terminal, and said once: the attempt happens once and either settles + // the directory or ends the command. + expect(value).toBe(1); + expect(lines).toHaveLength(1); + expect(lines[0]).toContain("was empty when this conversation started"); + expect(lines[0]).toContain("its contents were left alone"); + // The one line is the directory's, not admission's: the attempt settles + // before the host looks at what was approved, so a run that had reached + // admission would have reported that instead. + expect(lines[0]).not.toContain("does not validate"); + + // Preserved whole — the directory and what appeared in it. A recursive + // removal would have taken both. + expect(harness.reviews).toHaveLength(1); + expect(yield* exists(String(workdir))).toBe(true); + expect(yield* readTextFile(String(planted))).toBe("not this command's doing\n"); + + // And nothing after the failure began: no save, and no execution — so no + // journal, which only an execution creates. + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* until(readdir(dir))).toEqual([]); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("C13: a default directory is handed back before final admission", function* () { + // The same veto C10 proves, watched from the other side: by the time the + // host reports what it decided about the approved bytes, the conversation + // and its directory are both already gone. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const widget = join(dir, "Widget.md"); + yield* writeTextFile(widget, "A widget.\n"); + const draft = ["# Uses a widget", "", "", ""].join("\n"); + + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: draft }); + let workdir: string | undefined; + harness.deps.installElicitation = watching(harness, function* (directory) { + workdir = directory; + yield* rm(widget, { force: true }); + }); + + const events: string[] = []; + const written = console.error; + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + console.error = written; + }); + console.error = () => { + events.push("reported"); + }; + return yield* runPrompt(command(dir, [REQUEST]), harness.deps); + }); + + expect(code).toBe(1); + // Admission ran, and it ran after the directory was handed back: the + // observation below is taken at the moment the host reported its + // decision. + expect(events).toEqual(["reported"]); + expect(yield* exists(String(workdir))).toBe(false); + expect(harness.executions).toHaveLength(0); + }); }); it("C5: the prompt profile's ceiling is the host's, and no flag widens it", function* () { // This session's own directory, empty, no MCP servers and no native tools // — observed while the command document is still running, because that is // the only moment the claim is about. - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { yield* writeTextFile(join(dir, "secret.txt"), "the caller's tree\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); const seen: { cwd?: string; entries?: string[]; refusals: string[] } = { refusals: [] }; harness.deps.installElicitation = function* () { @@ -336,7 +536,7 @@ describe( // Not the caller's working directory: this session's, and empty while // the conversation ran. expect(seen.cwd).not.toBe(dir); - expect(seen.cwd).toBe(profileDirectoryFor("ceiling")); + expect(seen.cwd).toBe(profileDirectoryFor(profileRoot, "ceiling")); expect(seen.entries).toEqual([]); // Stated rather than omitted: this host configures no MCP server and // allows no native tool on a fresh session. @@ -350,15 +550,15 @@ describe( ]); }); - // Something already in that directory is a refusal, not a cleanup. It - // happens before the provider exists, so no session is placed and no turn - // is started — and what was there is still there afterwards. - yield* useWorkingDirectory(function* (dir) { - const occupied = profileDirectoryFor("occupied"); + // Something already in a named session's directory is a refusal, not a + // cleanup. It happens before the provider exists, so no session is placed + // and no turn is started — and what was there is still there afterwards. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const occupied = profileDirectoryFor(profileRoot, "occupied"); yield* ensureDir(occupied); yield* writeTextFile(join(occupied, "someone-elses.txt"), "not mine to delete\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); const { value, lines } = yield* reported(() => @@ -385,8 +585,8 @@ describe( // `--approve-all` configures the approved document. A native permission // request while the Plan is being written is still denied, privately, and // the turn it belongs to fails. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID, requestsTool: "Bash" }); const code = yield* runPrompt( @@ -402,9 +602,36 @@ describe( }); }); + it("C5: two cases' profile roots cannot see or remove one another", function* () { + // Roots are made per scope and named by a UUID, so one case's cleanup + // cannot reach another's directory even while both are live. + yield* useProfileRoot(function* (mine) { + const marker = join(mine, "mine.txt"); + yield* writeTextFile(marker, "still here\n"); + + yield* useWorkingDirectory(function* (dir, profileRoot) { + expect(profileRoot).not.toBe(mine); + expect(profileRoot.startsWith(`${mine}${sep}`)).toBe(false); + expect(mine.startsWith(`${profileRoot}${sep}`)).toBe(false); + + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + // The other root is untouched, and this one holds nothing afterwards. + expect(yield* readTextFile(marker)).toBe("still here\n"); + expect(yield* until(readdir(profileRoot))).toEqual([]); + }); + + expect(yield* readTextFile(marker)).toBe("still here\n"); + }); + }); + it("C6: a candidate is inert until the approved document runs", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: WRITES_A_FILE }); harness.script({ decision: "abort" }); @@ -421,8 +648,8 @@ describe( it("C7: candidate defects earn a repair turn; caller defects escape", function* () { // A defect the agent authored: the root's own frontmatter. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: BROKEN_SOURCE }); harness.fake.script({ reply: VALID }); harness.script({ decision: "approve" }); @@ -439,8 +666,8 @@ describe( }); // A defect the agent authored: two properties generating one option. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: COLLIDING }); harness.fake.script({ reply: VALID }); harness.script({ decision: "approve" }); @@ -457,8 +684,8 @@ describe( // A defect the caller wrote: an option the candidate never declares. It // raises out of the validator, so the program never sees it as feedback. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); const code = yield* runPrompt( @@ -474,8 +701,8 @@ describe( }); // A defect the caller wrote: aggregate JSON that is not JSON. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); const code = yield* runPrompt(command(dir, [REQUEST, "--props", "{oops"]), harness.deps); @@ -487,8 +714,8 @@ describe( }); // A defect the caller wrote: a value this candidate's schema rejects. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: NAME_IS_BOOLEAN }); const code = yield* runPrompt( @@ -504,8 +731,8 @@ describe( // A revision that changes what the command line means is the caller's // failure too, and it is caught before the candidate is presented. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: REQUIRES_NAME }); harness.script({ decision: "revise", feedback: "make it shout" }); harness.fake.script({ reply: NAME_IS_BOOLEAN }); @@ -521,8 +748,8 @@ describe( it("C8: one base draft, three repairs, and ten presentations", function* () { // Three repairs are available, and the fourth draft is what a person sees. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); @@ -539,8 +766,8 @@ describe( // A fourth invalid candidate is repair-exhausted: it reaches review with // its diagnostics, and there is no value that would approve it. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: UNRESOLVED }); } @@ -561,8 +788,8 @@ describe( // Ten presentations: nine revisions, and a tenth round with nothing left // to revise into. Each revision starts its own repair budget. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { harness.fake.script({ reply: VALID }); if (round < 10) { @@ -590,7 +817,7 @@ describe( }); it("C9: arbitrary source cannot close the presentation, and abort is authored", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { // A document that holds a fence of its own, and a run of five backticks. const fenced = [ "Here is a block:", @@ -603,7 +830,7 @@ describe( "", ].join("\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: fenced }); harness.script({ decision: "approve" }); @@ -619,8 +846,8 @@ describe( // Abort reaches the command document's own ``, with the message the // shipped Markdown wrote. Nothing about it is host policy. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); harness.script({ decision: "abort" }); @@ -639,8 +866,8 @@ describe( // that never validated, the last offering nothing but `abort`, and the // sentence says there was never a Plan to approve rather than that // somebody decided to stop. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { // A base draft and its three repairs, none of which validates. for (const _draft of [0, 1, 2, 3]) { @@ -673,8 +900,8 @@ describe( it("C10, C11: the approved bytes are what runs, and props are theirs", function* () { // The command line is unchanged across a revision, and the schema that // resolves it is the approved document's rather than the first draft's. - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: counting("number") }); harness.script({ decision: "revise", feedback: "count in words" }); harness.fake.script({ reply: counting("string") }); @@ -692,9 +919,9 @@ describe( // Nothing is stripped. A reply wrapped in a fence is not a document, so it // earns repairs and a review — and what is shown is exactly what arrived. - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { const wrapped = ["```md", "Hello.", "```", ""].join("\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: wrapped }); } @@ -706,7 +933,7 @@ describe( }); it("C10: final admission vetoes after the profile has torn down", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { // A repository component the draft uses. It exists while the command // document runs, so the same production validator that answers // finds the draft sound. @@ -714,7 +941,7 @@ describe( yield* writeTextFile(widget, "A widget.\n"); const draft = ["# Uses a widget", "", "", ""].join("\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: draft }); const events: string[] = []; harness.deps.installElicitation = function* () { @@ -774,7 +1001,7 @@ describe( }); it("C14: an interleaved Plan survives approval and execution byte for byte", function* () { - yield* useWorkingDirectory(function* (dir) { + yield* useWorkingDirectory(function* (dir, profileRoot) { // What the shipped instruction asks for: the request restated in prose a // reader was written for, with each component beside the sentences that // describe what it does. @@ -794,7 +1021,7 @@ describe( "", ].join("\n"); - const harness = createPromptHarness(); + const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: plan }); harness.script({ decision: "approve" }); @@ -814,8 +1041,8 @@ describe( }); it("C3, C9: what you read says each thing once, however many rounds it took", function* () { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); // Round one: a draft that cannot be repaired, presented with its problems. for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: UNRESOLVED }); @@ -857,8 +1084,8 @@ describe( // presentation or the tenth: an approvable Plan existed and you chose to // stop, and ten rounds of it do not accumulate into a different sentence. for (const rounds of [1, 10]) { - yield* useWorkingDirectory(function* (dir) { - const harness = createPromptHarness(); + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); for (const round of Array.from({ length: rounds }, (_, i) => i + 1)) { harness.fake.script({ reply: VALID }); if (round < rounds) { diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts index 406e4bbe1..a71de417a 100644 --- a/packages/cli/tests/support/prompt-harness.ts +++ b/packages/cli/tests/support/prompt-harness.ts @@ -16,14 +16,12 @@ import { Elicitation } from "@executablemd/core"; import type { ElicitationRequest, SyntaxCatalog } from "@executablemd/core"; import { Ok } from "effection"; import type { Operation, Result } from "effection"; -import { ensure, scoped, until } from "effection"; +import { ensure, scoped } from "effection"; import { ensureDir, rm } from "@effectionx/fs"; import { randomUUID } from "node:crypto"; -import { readdir } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { API, useHostFiles } from "@executablemd/runtime"; -import { PROMPT_PROFILE_SESSIONS } from "../../src/prompt-profile.ts"; import { syntaxCatalog } from "../../src/syntax.ts"; import type { PromptDependencies, PromptExecution } from "../../src/prompt.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; @@ -56,7 +54,16 @@ export interface PromptHarness { deps: PromptDependencies; } -export function createPromptHarness(options?: { +export function createPromptHarness(options: { + /** + * Where this harness keeps its profile session directories. + * + * Required, and always a tree the case itself created: a suite that fell back + * to the host default would be reading and removing directories under the + * developer's own home, and two cases running close together could not tell + * whose was whose. + */ + profileRoot: string; /** Replace the catalog entirely, for a case about catalog failure. */ catalog?: (includes: readonly string[]) => Operation; /** @@ -87,13 +94,14 @@ export function createPromptHarness(options?: { deps: { acp: { createRuntime: fake.create, - sessionStore: options?.store ?? makeStore(), + sessionStore: options.store ?? makeStore(), agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), }, *catalog(includes) { catalogCalls.push([...includes]); - return yield* (options?.catalog ?? syntaxCatalog)(includes); + return yield* (options.catalog ?? syntaxCatalog)(includes); }, + profileRoot: options.profileRoot, *installElicitation() { yield* Elicitation.around( { @@ -132,12 +140,22 @@ export function createPromptHarness(options?: { * Both, because the two answer different questions: session placement and * `--save` resolve the contextual one, while a real file has to live somewhere. */ -export function* useWorkingDirectory(body: (dir: string) => Operation): Operation { +export function* useWorkingDirectory( + body: (dir: string, profileRoot: string) => Operation, +): Operation { const dir = join(tmpdir(), `xmd-prompt-${randomUUID()}`); + // A sibling rather than a child: the working directory is what the approved + // document writes into and what several cases read back, and a profile root + // inside it would show up in those listings. + const profileRoot = `${dir}-profile`; yield* ensureDir(dir); + yield* ensureDir(profileRoot); return yield* scoped(function* () { yield* ensure(() => rm(dir, { recursive: true, force: true })); - yield* useProfileDirectories(); + // Recursive, and safe because it is: everything under this root was created + // by this scope, so nothing here can reach a directory another case or a + // real invocation owns. + yield* ensure(() => rm(profileRoot, { recursive: true, force: true })); yield* API.Env.around({ // deno-lint-ignore require-yield *cwd() { @@ -148,7 +166,7 @@ export function* useWorkingDirectory(body: (dir: string) => Operation): Op // the runtime entrypoint installs it: a document that reaches the // filesystem must reach the caller's, or fail. yield* useHostFiles(); - return yield* body(dir); + return yield* body(dir, profileRoot); }); } @@ -191,29 +209,17 @@ export function timesRead(reads: readonly string[], name: string): number { } /** - * Leave the host's profile session directories as this case found them. + * A profile-session root this scope creates, owns and removes whole. * - * A profile directory is deliberately durable — its stable identity is what - * `--session` continues — so a suite that made a few dozen of them under the - * developer's own home directory would be leaving litter behind. Only entries - * that appeared while the case ran are removed, so a directory somebody else - * owns is never touched. + * Owning it is what makes recursive removal safe: everything under it was made + * by this scope, so nothing here can reach a directory another case — or a real + * invocation — is using. */ -function* useProfileDirectories(): Operation { - const before = new Set(yield* listSessionDirectories()); - yield* ensure(function* () { - for (const name of yield* listSessionDirectories()) { - if (!before.has(name)) { - yield* rm(join(PROMPT_PROFILE_SESSIONS, name), { recursive: true, force: true }); - } - } +export function* useProfileRoot(body: (root: string) => Operation): Operation { + const root = join(tmpdir(), `xmd-prompt-profile-${randomUUID()}`); + yield* ensureDir(root); + return yield* scoped(function* () { + yield* ensure(() => rm(root, { recursive: true, force: true })); + return yield* body(root); }); } - -function* listSessionDirectories(): Operation { - try { - return yield* until(readdir(PROMPT_PROFILE_SESSIONS)); - } catch { - return []; - } -} diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 3749836bf..158ff9a1f 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -576,6 +576,15 @@ provider is constructed or a session is materialized; a non-empty one is a terminal refusal rather than something to clean. Nothing this profile grants can write there. +An explicitly named session keeps its directory afterwards, because that is what +the next invocation derives the same session identity from, and no cleanup +applies to it. An invocation-unique default keeps nothing: once the profile has +torn down, on every ending, exactly one cleanup is attempted — the leaf is +removed non-recursively if it is still empty, and if it is not, it is preserved +whole and the command fails terminally. Which applies is a trusted host value — +whether `--session` was written — and where the directories live is a host +dependency no caller or document can select. + Only the profile's provider receives `newSessionOptions.systemPrompt`. It carries the fixed statement of what the session writes, and nothing else — the catalog and the request are the command document's to send in its own turns diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 29fdbce52..fbb33faa3 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -195,7 +195,8 @@ The assistant that writes a Plan is assembled separately from the final run provider: - one host-owned directory dedicated to the logical prompt session, created - empty and required to remain empty; + empty, required to be empty on the way in, and — for an invocation-unique + default session — handed back non-recursively on the way out; - no additional directories; - no MCP servers — stated as an empty set, not omitted; - an empty requested native-tool allowlist on a fresh session; @@ -224,13 +225,39 @@ invocation-unique name reaches a location nothing else does; the same explicit name reaches the same one, and because a session's key includes the directory it lives in, that is what lets the provider continue the session it established. -The directory is created empty on first use and **required to be empty** before -the provider is constructed or a session is materialized. A non-empty directory -is a terminal, actionable refusal naming the path: nothing is deleted or cleaned, -because whatever is in there was put there by something this host did not -authorize. It may remain after execution — its stable identity is part of -named-session continuation — and nothing this profile grants can write there, so -what is created empty stays empty. +The directory is created empty and **required to be empty** before the provider +is constructed or a session is materialized. A non-empty one is a terminal, +actionable refusal naming the path: nothing is deleted or cleaned, because +whatever is in there was put there by something this host did not authorize. +Nothing this profile grants can write there, so what is created empty stays +empty. + +The two kinds of session then have different directory lifetimes, because only +one of them has an identity worth keeping: + +- an **invocation-unique default** directory is the command's own. Exactly one + cleanup is attempted, after the command document and every provider, Prompt + task and Elicitation resource inside it has torn down, and it settles before + final validation, the save or the execution begins. Success, abort, a failed + turn and cancellation all reach it. It has exactly two outcomes: + - **still empty** — that exact leaf is removed, non-recursively; + - **no longer empty** — the directory and everything in it are preserved and + the command fails terminally. It is not a warning and not a silent skip, and + no final admission, save or execution follows. +- an **explicitly named** directory survives the invocation and is not subject to + that cleanup at all, because a later `--session ` derives the same + location and therefore the same ACPX session identity from it. It is required + empty again on the way in every time, and its contents are never cleaned or + overwritten. + +Which of the two applies is a trusted host value — whether the caller wrote +`--session` — not something read back out of the name, and the command document +is told neither the directory nor which kind it is. + +Where those directories live is a host dependency as well. No flag, environment +variable, document prop or replaceable context selects it; production uses its +own default, and a harness that owns a temporary tree is given that tree +directly. The approved Plan later receives the caller-selected ordinary run Agent and permission configuration. It inherits neither the assistant Session nor its @@ -502,6 +529,6 @@ produced. | C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no save, journal or execution — and resolves props for the exact returned bytes | | C11 | Exact bytes | Approval, exclusive save and `` execution receive the Agent close value without rewriting or fence removal | | C12 | Journal separation | Authorship uses only disposable in-memory history; the final journal begins with the approved Plan | -| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution | +| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content is preserved whole while the command fails terminally, with no admission, save, journal or execution after it | | C14 | Narrative preservation | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, the assistant receives it with the request and the catalog, and a scripted Plan of prose interleaved with components returns byte for byte after approval | | C15 | Ordinary run | The approved source keeps normal cwd, includes, props, output/value, permission, timeout, failure and save behaviour | From 1e449879c04e7c458ff6738ae13c2da8abb290dd Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 16:22:44 -0400 Subject: [PATCH 08/15] =?UTF-8?q?=E2=9C=A8=20Capture=20the=20problems=20wi?= =?UTF-8?q?th=20,=20and=20claim=20the=20directory=20before=20ma?= =?UTF-8?q?king=20it=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebased onto `` (#667), which supplies what the single-child `` wrapper was standing in for. Both wrappers in the packaged prompt command document are gone; `` binds the serialized text directly, and the bytes reaching `` are the same bytes — the repair turn and the review presentation still carry the complete structured problems, now proven by parsing the fence back rather than by matching a substring of it. The default session's directory is now a scope-owned resource rather than a directory with an ensure attached afterwards. The release is registered before the `mkdir` that could create it, so there is no window in which a leaf exists that nothing is responsible for: a failure between making it and using it hands it back like every other ending. What the release knows is whether the directory was ever handed over, and that decides which question it is answering. Once established, the exit has exactly two outcomes and no third. Still the empty directory it was given, and the leaf is removed, non-recursively. Anything else — content that appeared, or a directory that vanished — is preserved as found and fails the command terminally. A leaf that disappears under a live conversation was silently accepted before, which was wrong twice over: nothing here is allowed to remove it, so its absence is interference, and treating interference as a clean exit would let admission, the save and the execution proceed on a conversation something else had already reached into. A directory establishment never handed over is a different question, already answered by what establishment reported. The release leaves that report and the directory's contents alone, while still handing back an empty leaf it did create. Explicitly named directories are untouched by any of this: they stay, because the next `--session` derives the same ACPX session identity from where they are. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 2 +- packages/cli/src/documents/prompt-command.md | 4 +- packages/cli/src/prompt-profile.ts | 79 +++++++++---- packages/cli/tests/packaged-document.test.ts | 5 + packages/cli/tests/prompt.test.ts | 115 ++++++++++++++++++- specs/acp-client-spec.md | 10 +- specs/prompt-command-spec.md | 32 ++++-- 7 files changed, 206 insertions(+), 41 deletions(-) diff --git a/architecture.md b/architecture.md index aae86a3d0..d2379af72 100644 --- a/architecture.md +++ b/architecture.md @@ -36,7 +36,7 @@ Existing documents and code get aligned to this section retroactively. | Prompt | a person's original request, in ordinary natural language. `xmd prompt` takes exactly one | | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. A Plan is what `xmd prompt` approves, optionally saves and runs; it is not a synonym for a workflow, a policy document or any executable Markdown file | | prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the request wording, the draft and repair loops, the check branches, human review, revision, approval, abort and exhaustion — and returns the exact approved Plan source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | -| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when still empty, and preserved whole with the command failing terminally when it is not. Where those directories live is a host dependency no caller or document selects | +| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | | expansion | one logical evaluation of an authored executable element within a document execution | | expansion ID | a deterministic identifier for one logical expansion; restoring or retrying that expansion preserves the ID, while a distinct evaluation requested by the document receives another | | Git capability | the contextual interface through which workflow infrastructure queries the Git repository associated with the current working directory | diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md index f421e2d75..aba268aa0 100644 --- a/packages/cli/src/documents/prompt-command.md +++ b/packages/cli/src/documents/prompt-command.md @@ -86,7 +86,7 @@ found. A change you ask for produces a new draft, so it gets its own three. That document has problems. These are the exact ones: - + Send one complete replacement document that resolves every problem above. Keep @@ -129,7 +129,7 @@ This document was written for: {props.request} Three attempts to fix it did not clear everything. These problems remain: - + diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index b7ca0ce53..d50ce341f 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -144,22 +144,16 @@ export function* runPromptCommandDocument(profile: PromptProfile): Operation> { - // First, and before anything is built: this session's directory is + // First, and before anything is built: this session's directory is claimed, // established and proven empty, or the command stops here. Nothing has been - // installed yet, so a refusal reaches no provider, no session and no turn. - const established = yield* establishDirectory(profile.root, profile.session); + // installed yet, so a refusal reaches no provider, no session and no turn — + // and because the claim is taken before the directory is made, an ending of + // any kind, including a cancellation, still hands it back. + const established = yield* useSessionDirectory(profile); if (!established.ok) { return established; } const workdir = established.value; - if (!profile.explicitSession) { - // Registered before every other resource, so it runs after all of them: - // the provider, the Prompt tasks and the Elicitation provider are already - // gone when this hands the directory back. Being an `ensure` is what makes - // an abort, a failed turn and a cancellation clean up the same way a - // success does. - yield* ensure(() => removeEmptyDirectory(workdir)); - } yield* refuseDocumentCapabilities(); yield* profile.installElicitation(); @@ -323,6 +317,37 @@ export function profileDirectoryFor(root: string, session: string): string { return join(root, createHash("sha256").update(session).digest("hex")); } +/** + * This conversation's directory, for as long as the conversation lasts. + * + * An explicitly named session's directory is durable — a later `--session` finds + * the same location and therefore the same ACPX session — so nothing is + * registered against it and nothing removes it. + * + * An invocation-unique one belongs to this scope, and the release is registered + * *before* the first filesystem call that could create it. A leaf this call + * made and then failed on is still a leaf this call made; registering after the + * `mkdir` would leave one behind exactly in the case nobody is watching. Being + * the first thing registered in the scope is also what puts it last in teardown, + * after every provider, Prompt task and Elicitation resource has gone. + */ +function* useSessionDirectory(profile: PromptProfile): Operation> { + const directory = profileDirectoryFor(profile.root, profile.session); + if (profile.explicitSession) { + return yield* establishDirectory(directory); + } + const claim: DirectoryClaim = { established: false }; + yield* ensure(() => releaseSessionDirectory(directory, claim)); + const established = yield* establishDirectory(directory); + claim.established = established.ok; + return established; +} + +/** Whether the directory was ever handed to this conversation to use. */ +interface DirectoryClaim { + established: boolean; +} + /** * Establish this session's directory, or refuse. * @@ -331,8 +356,7 @@ export function profileDirectoryFor(root: string, session: string): string { * a stranger's files to get on with the work is the opposite of what a ceiling * is for. So the command says what it found and where, and stops. */ -function* establishDirectory(root: string, session: string): Operation> { - const directory = profileDirectoryFor(root, session); +function* establishDirectory(directory: string): Operation> { try { yield* until(mkdir(directory, { recursive: true })); const entries = yield* until(readdir(directory)); @@ -359,23 +383,34 @@ function* establishDirectory(root: string, session: string): Operation { +function* releaseSessionDirectory(directory: string, claim: DirectoryClaim): Operation { try { yield* until(rmdir(directory)); } catch (error) { const code = error instanceof Error && "code" in error ? error.code : undefined; - if (code === "ENOENT") { + if (!claim.established) { return; } + if (code === "ENOENT") { + throw new Error( + `${directory} was made for this conversation and is already gone. Something removed ` + + "it while the conversation was still running, which nothing here is allowed to do; " + + "nothing was saved or run", + ); + } if (code === "ENOTEMPTY" || code === "EEXIST") { throw new Error( `${directory} was empty when this conversation started and is not now. It belongs to ` + diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts index e7fed733b..fa575fef1 100644 --- a/packages/cli/tests/packaged-document.test.ts +++ b/packages/cli/tests/packaged-document.test.ts @@ -48,6 +48,11 @@ describe("packaged documents", () => { "places each component\nimmediately after the sentences describing the action", ); expect(source).toContain("ordinary reader-facing prose"); + // It captures the serialized problems with `` directly. The + // single-child `` wrapper existed only because the component refused a + // literal `as`, and main supplies that now. + expect(source).toContain(''); + expect(source).not.toContain(''); }); it("says which build is missing a document rather than behaving differently", function* () { diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 28dd5d742..d7b4e857e 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -177,6 +177,30 @@ function* exists(path: string): Operation { return (yield* stat(path)).exists; } +/** What one message's `json` fence holds, parsed back. */ +interface FencedDiagnostics { + validation?: { + version?: number; + outcome?: string; + diagnostics?: { code?: string }[]; + }; +} + +/** + * The complete JSON a message carried, read out of its fence. + * + * Parsed rather than string-matched: what matters is that ``'s serialized + * text reached `` whole, and a structure that parses back to the same + * shape is what says so. + */ +function fencedJson(message: string): FencedDiagnostics { + const fence = /```json\n([\s\S]*?)\n```/.exec(message); + if (fence === null) { + throw new Error(`no json fence in: ${message}`); + } + return JSON.parse(fence[1]); +} + /** How many times one exact string appears. */ function occurrences(haystack: string, needle: string): number { return haystack.split(needle).length - 1; @@ -404,6 +428,79 @@ describe( expect(yield* until(readdir(profileRoot))).toEqual([]); }); + // A failure between making the leaf and using it still hands it back. The + // claim is taken before the `mkdir`, so there is no window in which a + // directory exists that nothing is responsible for. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID }); + // deno-lint-ignore require-yield + harness.deps.installElicitation = function* () { + throw new Error("this host could not install a review provider"); + }; + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST]), harness.deps), + ); + + expect(value).toBe(1); + expect(lines.join("\n")).toContain("could not install a review provider"); + // Nothing was built after it, and no empty leaf was left behind. + expect(harness.fake.created).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + expect(yield* until(readdir(profileRoot))).toEqual([]); + }); + + // Establishment failing for a reason of its own leaves nothing behind and + // says what it found. The claim is already taken here, so the release runs + // and finds no directory it was ever given — which is the one case where + // an absent directory is the ordinary answer rather than interference. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const blocked = join(profileRoot, "not-a-directory"); + yield* writeTextFile(blocked, "in the way\n"); + + const harness = createPromptHarness({ profileRoot: blocked }); + harness.fake.script({ reply: VALID }); + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST]), harness.deps), + ); + + expect(value).toBe(1); + expect(lines).toHaveLength(1); + expect(lines[0]).toContain("could not establish"); + // What was in the way is untouched, and no phase after it began. + expect(yield* readTextFile(blocked)).toBe("in the way\n"); + expect(harness.fake.created).toHaveLength(0); + expect(harness.executions).toHaveLength(0); + }); + + // A leaf that disappears under a live conversation is interference, not a + // tidy exit. The command fails terminally rather than shrugging at an + // absent directory, and nothing it would have done next happens. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.fake.script({ reply: VALID }); + harness.deps.installElicitation = watching(harness, function* (workdir) { + yield* rm(workdir, { recursive: true, force: true }); + }); + + const { value, lines } = yield* reported(() => + runPrompt({ ...command(dir, [REQUEST]), save: "out.md" }, harness.deps), + ); + + expect(value).toBe(1); + expect(lines).toHaveLength(1); + expect(lines[0]).toContain("was made for this conversation and is already gone"); + expect(lines[0]).not.toContain("does not validate"); + // The Plan was approved and still reached nothing: no admission that + // could have saved it, no save, no execution and so no journal. + expect(harness.reviews).toHaveLength(1); + expect(yield* exists(join(dir, "out.md"))).toBe(false); + expect(yield* until(readdir(dir))).toEqual([]); + expect(harness.executions).toHaveLength(0); + }); + // The other outcome of the one attempt. A directory this invocation was // given empty and did not leave empty is preserved and the command fails: // something wrote there while the conversation ran, and this host @@ -659,6 +756,16 @@ describe( expect(code).toBe(0); expect(harness.fake.prompts).toHaveLength(2); expect(harness.fake.prompts[1]).toContain("That document has problems"); + // `` captured the whole serialized value, and the fence it went + // into holds every byte of it: the wrapper's removal changed where the + // text is bound, not what it says. Parsed back rather than matched, so + // what is proven is that the complete structure survived the trip. + const repaired = fencedJson(harness.fake.prompts[1]); + expect(repaired.validation?.version).toBe(1); + expect(repaired.validation?.outcome).toBe("invalid"); + expect( + repaired.validation?.diagnostics?.some((entry) => entry.code === "source-invalid"), + ).toBe(true); expect(harness.fake.prompts[1]).toContain("source-invalid"); // The whole versioned value, as data the program serialized. expect(harness.fake.prompts[1]).toContain('"version": 1'); @@ -782,7 +889,13 @@ describe( expect(harness.reviews[0].message).toContain( "Three attempts to fix it did not clear everything", ); - expect(harness.reviews[0].message).toContain("component-unresolved"); + // The presentation carries the same complete captured JSON the repair + // turns did, through the same `` binding. + const shown = fencedJson(harness.reviews[0].message); + expect(shown.validation?.version).toBe(1); + expect( + shown.validation?.diagnostics?.some((entry) => entry.code === "component-unresolved"), + ).toBe(true); expect(harness.executions).toHaveLength(0); }); diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 158ff9a1f..2a87f6936 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -578,10 +578,12 @@ write there. An explicitly named session keeps its directory afterwards, because that is what the next invocation derives the same session identity from, and no cleanup -applies to it. An invocation-unique default keeps nothing: once the profile has -torn down, on every ending, exactly one cleanup is attempted — the leaf is -removed non-recursively if it is still empty, and if it is not, it is preserved -whole and the command fails terminally. Which applies is a trusted host value — +applies to it. An invocation-unique default keeps nothing: its release is +registered before the directory is created, and once the profile has torn down, +on every ending, exactly one cleanup is attempted — the leaf is removed +non-recursively if it is still the empty directory that was handed over, and if +it has gained content or disappeared it is left as found and the command fails +terminally. Which applies is a trusted host value — whether `--session` was written — and where the directories live is a host dependency no caller or document can select. diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index fbb33faa3..22636246f 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -235,15 +235,25 @@ empty. The two kinds of session then have different directory lifetimes, because only one of them has an identity worth keeping: -- an **invocation-unique default** directory is the command's own. Exactly one - cleanup is attempted, after the command document and every provider, Prompt - task and Elicitation resource inside it has torn down, and it settles before - final validation, the save or the execution begins. Success, abort, a failed - turn and cancellation all reach it. It has exactly two outcomes: +- an **invocation-unique default** directory is the command's own, held as a + scope-owned resource: the release is registered *before* the first filesystem + call that could create it, so a leaf this invocation made and then failed on is + still a leaf this invocation hands back. Exactly one cleanup is attempted, after + the command document and every provider, Prompt task and Elicitation resource + inside it has torn down, and it settles before final validation, the save or + the execution begins. Success, abort, a failed turn and cancellation all reach + it. Once the directory has been established, it has exactly two outcomes: - **still empty** — that exact leaf is removed, non-recursively; - - **no longer empty** — the directory and everything in it are preserved and - the command fails terminally. It is not a warning and not a silent skip, and - no final admission, save or execution follows. + - **anything else** — the directory and whatever is in it are preserved and the + command fails terminally. A leaf that has gained content, and one that has + disappeared, are both interference: something acted on a directory this host + authorized nothing to touch. Neither is a warning and neither is a silent + skip, and no final admission, save or execution follows. + + A directory establishment never handed over — refused as non-empty, or never + created at all — is a different question, already answered by what + establishment reported. The release then removes an empty leaf it did create + and otherwise leaves both the report and the directory's contents alone. - an **explicitly named** directory survives the invocation and is not subject to that cleanup at all, because a later `--session ` derives the same location and therefore the same ACPX session identity from it. It is required @@ -413,8 +423,8 @@ candidate offers `approve` or `abort`, and an invalid one offers only `abort`. **Presentation.** The review message presents the exact draft with ``, whose fence is longer than every backtick run the draft holds, so draft text cannot close it and is never interpreted. An invalid draft is followed by its -complete JSON problems, serialized by ``. Prose outside `` -addresses you; text inside `` instructs the assistant. +complete JSON problems, serialized and captured by ``. Prose outside +`` addresses you; text inside `` instructs the assistant. **Failure.** `approve` selects the draft, and one exhaustive branch after the Session returns its source unchanged. Stopping reaches `` with one of two @@ -529,6 +539,6 @@ produced. | C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no save, journal or execution — and resolves props for the exact returned bytes | | C11 | Exact bytes | Approval, exclusive save and `` execution receive the Agent close value without rewriting or fence removal | | C12 | Journal separation | Authorship uses only disposable in-memory history; the final journal begins with the approved Plan | -| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content is preserved whole while the command fails terminally, with no admission, save, journal or execution after it | +| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content or vanished under the conversation is preserved as found while the command fails terminally, with no admission, save, journal or execution after it | | C14 | Narrative preservation | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, the assistant receives it with the request and the catalog, and a scripted Plan of prose interleaved with components returns byte for byte after approval | | C15 | Ordinary run | The approved source keeps normal cwd, includes, props, output/value, permission, timeout, failure and save behaviour | From 7f2a10fce012776445029ade5e9bd213127f5e90 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 16:46:53 -0400 Subject: [PATCH 09/15] =?UTF-8?q?=E2=9C=A8=20Make=20the=20approved=20Plan?= =?UTF-8?q?=20the=20result,=20and=20the=20workflow=20readable=20end=20to?= =?UTF-8?q?=20end=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The command document is now a workflow somebody can read straight through. It opens by saying what `xmd prompt` is for — a Prompt describes steps, the components carry them out, and the coding agent turns both into one document that explains and executes the sequence — and every stage after that is a heading: creating the first draft, checking and repairing it, reviewing it, continuing from the decision, returning the approved Plan. Every Plan now begins with one descriptive level-one title, and the generation, repair and revision turns each ask for it alongside the steps-beside-components structure, so a replacement cannot quietly drop either. The worked example is a titled Plan itself. This is an authorship and human-review requirement, not a rule hidden in the checker: `` reports structural facts and would accept a titleless Plan that a person would send back. `` is `` throughout. It checks a draft rather than validating a candidate, and nothing about what it does changed. The review choices are the words a person reads — Approve, Request changes, Stop — with no internal spelling behind them. A tenth draft that still has problems now offers something better than only stopping: Explain what went wrong makes one more ordinary turn in the same Session, carrying only the final diagnostics, because the conversation already holds the Prompt, the catalog and every draft. It asks for an explanation and explicitly not another Plan; its answer is inert text, reopens no draft limit, is reported as the coding agent's own words, and ends the command. So the host's instruction layer no longer demands that every answer be document source — it says only that an answer belongs to the message that asked for it, and each authored turn owns its own shape. The approved Plan is the command's result. By default it goes to stdout, byte for byte, with nothing this command added, so it can be piped, diffed or read before anybody commits to it. `--output` puts those bytes in a file instead, exclusively created. `--run` runs it. With both, the file is written first and only a successful write is followed by the run. `--save` is gone rather than aliased; nothing has been released to keep compatible with. That makes most execution flags conditional, so they are refused rather than ignored: a caller who asks for a journal, a permission mode or an exec deadline from a command that runs nothing has not been answered. `--include`, `--agent-provider`, `--default-agent`, `--session` and `--timeout` keep working always — they build the catalog, settle the agent, name the conversation, admit properties and bound the command. A journal exists only when `--run` begins. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 2 +- packages/cli/src/cli.ts | 69 ++++--- packages/cli/src/documents/prompt-command.md | 175 ++++++++++------ packages/cli/src/prompt-args.ts | 56 ++++- packages/cli/src/prompt-profile.ts | 34 +-- packages/cli/src/prompt.ts | 42 ++-- packages/cli/tests/packaged-document.test.ts | 41 +++- packages/cli/tests/prompt-args.test.ts | 58 +++++- packages/cli/tests/prompt-cli.test.ts | 194 +++++++++++++++--- .../cli/tests/prompt-command-document.test.ts | 4 +- packages/cli/tests/prompt.test.ts | 188 ++++++++++++----- packages/cli/tests/support/prompt-harness.ts | 9 +- specs/acp-client-spec.md | 8 +- specs/prompt-command-spec.md | 193 +++++++++++------ specs/root-document-props-spec.md | 9 +- 15 files changed, 792 insertions(+), 290 deletions(-) diff --git a/architecture.md b/architecture.md index d2379af72..d9b42257a 100644 --- a/architecture.md +++ b/architecture.md @@ -3545,7 +3545,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require the Plan to keep every requested outcome as reader-facing prose with each component beside the sentences describing it, through repairs and revisions alike, and prose quality stays that document's policy rather than a TypeScript rule. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, optionally creates the save exclusively, and executes them through the ordinary supplied-source path under the `` identity — one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior | built on the #659 stack | +| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, and then delivers the approved Plan where the caller asked: to stdout byte for byte by default, to an exclusively created `--output` path, and — only under `--run` — through the ordinary supplied-source path under the `` identity, one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A journal exists only when `--run` begins, and the flags that configure only a run are refused in preflight without it | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 7528c68c4..5501c4f65 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -109,7 +109,13 @@ import { resolvePropsFromSources, } from "./props.ts"; import type { Binding, Extraction } from "./props.ts"; -import { namesPrompt, scanPromptArgs, SAVE_OPTION, SESSION_OPTION } from "./prompt-args.ts"; +import { + namesPrompt, + OUTPUT_OPTION, + RUN_OPTION, + scanPromptArgs, + SESSION_OPTION, +} from "./prompt-args.ts"; import type { PromptScan } from "./prompt-args.ts"; import { runPrompt } from "./prompt.ts"; import type { PromptExecution } from "./prompt.ts"; @@ -239,8 +245,7 @@ const runConfig = object({ }); /** - * `xmd prompt` — the request, where to keep the source, and the execution the - * approved document runs under. + * `xmd prompt` — the Prompt, where the approved Plan goes, and whether it runs. * * The individual `--props-*` options a candidate declares are deliberately * absent: they exist only once a document does, and the props phase binds them @@ -249,13 +254,17 @@ const runConfig = object({ */ const promptConfig = object({ request: { - description: "what the agent should write a document to do", + description: "the steps the coding agent should turn into a Plan", ...field(z.string().optional(), cli.argument()), }, - save: { - description: "write the approved document here before running it (path must not exist)", + output: { + description: "write the approved Plan here instead of to stdout (path must not exist)", ...field(z.string().optional()), }, + run: { + description: "run the approved Plan instead of writing it", + ...field(z.boolean(), field.default(false)), + }, session: { description: "logical name for the assistant session (default: unique to this invocation)", ...field(z.string().optional()), @@ -1659,34 +1668,47 @@ const RUN_SOURCE_HELP = [ * are. Answering otherwise would mean generating a document to describe one. */ const PROMPT_REQUEST_HELP = [ - "Exactly one request is required, and it is text for the agent rather than a", - "path. Quote it so the shell passes it as a single argument:", + "Exactly one Prompt is required, and it is the steps you want carried out", + "rather than a path. Quote it so the shell passes it as a single argument:", ' xmd prompt "ask me for my age and write the result to a file"', "", - "A first-party command document turns your request into a Plan: an executable", - "Markdown document that states what you asked for in ordinary language and", - "places the components that do the work alongside those words. xmd checks each", - "draft and repairs definite defects; you approve, revise or abort before", - "anything runs.", + "A first-party command document turns your Prompt into a Plan: an executable", + "Markdown document that states each step in ordinary language and places the", + "component that carries it out beside those words. xmd checks each draft and", + "repairs definite defects; you approve, request changes or stop before anything", + "leaves the command.", "", - "Document properties follow the request. The individual options a document", - "declares are the generated document's, so they are not listed here:", - ' xmd prompt "" --props-name Ada', + "The approved Plan is the result. By default it is written to stdout and nothing", + "runs, so it can be piped, diffed or read before you commit to it.", + "", + "Document properties follow the Prompt. The individual options a Plan declares", + "are the generated document's, so they are not listed here:", + ' xmd prompt "" --props-name Ada', "", ` ${AGGREGATE_OPTION} `, " Set document properties as a JSON object", ` Environment: ${AGGREGATE_ENV}`, "", - ` ${SAVE_OPTION} `, - " Write the approved document there before running it. The path must not", - " exist; an existing one is left alone and the run stops.", + ` ${OUTPUT_OPTION} `, + " Write the approved Plan there instead of to stdout. The path must not", + " exist; an existing one is left alone and the command stops.", + "", + ` ${RUN_OPTION}`, + " Run the approved Plan instead of writing it. With --output the file is", + " written first, and only a successful write is followed by the run.", "", ` ${SESSION_OPTION} `, " Use this logical assistant session instead of one unique to this run.", "", - "Permission flags configure the approved document. Writing the Plan is a", - "conversation about text and gives the assistant nothing: an empty directory of", - "its own, no tools, no MCP servers, and every native permission request denied.", + `Options that only configure running a Plan — --journal, --raw, --verbose, the`, + "exec and fetch timeouts, the permission flags and secret detection — are", + `refused without ${RUN_OPTION}, because without it nothing runs for them to`, + "configure.", + "", + "Permission flags configure the approved Plan. Writing the Plan is a", + "conversation about text and gives the coding agent nothing: an empty directory", + "of its own, no tools, no MCP servers, and every native permission request", + "denied.", ].join("\n"); /** @@ -1935,7 +1957,8 @@ function* dispatch( argv: helpRequest.args, scan, include: config.include, - ...(config.save === undefined ? {} : { save: config.save }), + ...(config.output === undefined ? {} : { output: config.output }), + run: config.run, ...(config.session === undefined ? {} : { session: config.session }), stack: promptStack, }, diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md index aba268aa0..efc1c088a 100644 --- a/packages/cli/src/documents/prompt-command.md +++ b/packages/cli/src/documents/prompt-command.md @@ -11,42 +11,46 @@ returns: type: string --- -# What `xmd prompt` does +# `xmd prompt` turns steps into a program -`xmd prompt` turns your request into a Plan: an Executable Markdown document -that states what you asked for in ordinary language and places the components -that perform the work alongside those words. +This document is a workflow that generates an executable Plan from a sequence of +steps. It combines the original Prompt, which describes those steps, with the XMD +components available to carry them out. A coding agent turns both into one +document that explains and executes the sequence. -This document is how that happens. It is a program rather than a setting: what -the assistant is asked for, how many times a draft may be fixed, what you are -shown, and what happens when you approve nothing are all written here, in the -open, where you can read them and argue with them. +The result is the XMD version of a coding agent's plan. A conventional Markdown +plan must be interpreted again before its steps can happen. An XMD Plan already +contains those executable steps, so running it simply executes them. -The command gives it three things — your request as you typed it, the components -available in this directory, and the name of the assistant session every turn -belongs to. Nothing here runs the document being written. A draft is text until -you approve it, and the command runs it separately afterwards. +A draft remains text while this workflow reviews it. Nothing in it runs before +you approve it. After approval, `xmd prompt` validates the exact source again. By +default it prints the approved XMD source. `--output` writes that source to a +file instead, and `--run` executes the Plan. With both options, the command +writes the source before running it. +## Create the first draft + -Write one complete Executable Markdown document that does this: +Create one complete XMD Plan from this Prompt: {props.request} -The document you write is a Plan. A Plan is not a script with comments: it says -what is being done in ordinary reader-facing prose, and places each component -immediately after the sentences describing the action that component performs. -The prose is part of the program and part of what the document prints when it -runs, so somebody watching it run follows the same words somebody reading the -source did. +Begin the Plan with one level-one Markdown title that describes it. + +Write the Plan as a sequence of readable steps. Follow each step with the XMD +component that carries it out. The prose is part of the program and appears when +the Plan runs, so its source and its execution tell the same story. -For the request "ask me for my age and write it to a file", the shape is: +For the Prompt "ask me for my age and write it to a file", the shape is: ```markdown +# Ask for and save your age + Ask me for my age. @@ -56,27 +60,30 @@ Write it to a file. {answer.age} ``` -You do not have to repeat the request word for word. Divide it, clarify it and +You do not have to repeat the Prompt word for word. Divide it, clarify it and rewrite it into natural prose — but keep every outcome it asked for, keep them in -an order that makes sense, and keep each one next to the component that carries -it out. Somebody should be able to audit the source and follow the execution by -the same narrative. +an order that makes sense, and keep each one beside the component that carries it +out. Everything you may use is described below. Use nothing that is not here. {props.syntax} -Reply with the document source and nothing else. No enclosing code fence, no -explanation before or after it, no commentary about what you did. +Reply with the Plan source and nothing else. No enclosing code fence, no +explanation before or after it. -Each draft is checked once, and gets up to three chances to fix what the check -found. A change you ask for produces a new draft, so it gets its own three. +## Check and repair the draft + +Each new draft is checked before it is shown to you. If the check finds problems, +the coding agent gets up to three repair attempts to replace it with a corrected +Plan. A revision you request later is a new draft and receives three repair +attempts of its own. - + @@ -84,24 +91,34 @@ found. A change you ask for produces a new draft, so it gets its own three. -That document has problems. These are the exact ones: +That Plan has problems. These are the exact ones: -Send one complete replacement document that resolves every problem above. Keep -the reader-facing prose that says what the document is for, and keep each -component immediately after the sentences describing what it does. Source only, -no enclosing fence, no explanation. +Send one complete replacement Plan that resolves every problem above. Keep its +level-one title, keep the Prompt's sequence of readable steps in a meaningful +order, and keep each XMD component beside the step it performs. + +Reply with the Plan source and nothing else. No enclosing code fence, no +explanation before or after it. - + -You decide what happens next. A draft that passed its check can be approved; one -that still has problems after three attempts is shown with them, and can only be -sent back or abandoned. On the tenth and last time you are asked there is -nothing left to change it into, so asking for a change is not offered. +## Review the draft + +The workflow shows you the complete draft after its repair attempts. + +- Choose **Approve** to accept a draft that passed its check. +- Choose **Request changes** to send feedback to the coding agent and create a + new draft. +- Choose **Stop** to end without outputting or running anything. + +A draft with remaining problems cannot be approved. You may review at most ten +drafts, and the tenth cannot be revised. If the tenth draft still has problems, +you may ask the coding agent to explain what went wrong or stop. -This document was written for: {props.request} +### Original Prompt + +{props.request} + +### Draft Plan -Three attempts to fix it did not clear everything. These problems remain: +### Problems that remain + +The coding agent used all three repair attempts, but the draft still has these +problems: - +## Continue from your decision + +Approve keeps this exact draft and leaves the review. Request changes sends your +feedback to the coding agent and starts a new draft. Stop ends here. On a tenth +draft that still has problems there is nothing left to revise into, so the +remaining choice is to ask the coding agent what went wrong. + + -Stopping says two different things depending on where you are. On the tenth -draft, when it still has problems, `abort` is the only choice left and there was -never a Plan to approve — so that ending says so. Everywhere else, including a -tenth draft you could have approved, stopping is your decision and is reported -as one. - - + - + - + + + +The final Plan still has these problems: + + + + +Explain briefly why the attempts did not resolve them and what the person should +clarify in their next Prompt. Do not create another Plan. + + + + + -The person who asked for this document read it and wants this changed: +You read that Plan and asked for this to change: {review.feedback} -Send one complete replacement document. Keep the reader-facing prose that says -what the document is for, and keep each component immediately after the -sentences describing what it does. Source only, no enclosing fence, no -explanation. +Send one complete replacement Plan. Keep its level-one title, keep the Prompt's +sequence of readable steps in a meaningful order, and keep each XMD component +beside the step it performs. + +Reply with the Plan source and nothing else. No enclosing code fence, no +explanation before or after it. -If you approved a document, return its source unchanged. Nothing should reach -this point without approving or stopping first, so the other branch says what it -would mean if anything did. +## Return the approved Plan + +Only an approved Plan can leave this workflow. Its source is returned exactly as +the coding agent wrote it. - + diff --git a/packages/cli/src/prompt-args.ts b/packages/cli/src/prompt-args.ts index 83708898e..d30b00778 100644 --- a/packages/cli/src/prompt-args.ts +++ b/packages/cli/src/prompt-args.ts @@ -20,8 +20,9 @@ import { AGGREGATE_OPTION } from "./props.ts"; import type { Binding } from "./props.ts"; export const PROMPT_COMMAND = "prompt"; -export const SAVE_OPTION = "--save"; +export const OUTPUT_OPTION = "--output"; export const SESSION_OPTION = "--session"; +export const RUN_OPTION = "--run"; /** The built-in options that take a separated value. */ const VALUE_OPTIONS: readonly string[] = [ @@ -33,7 +34,7 @@ const VALUE_OPTIONS: readonly string[] = [ "--timeout", "--timeout-exec", "--timeout-fetch", - SAVE_OPTION, + OUTPUT_OPTION, SESSION_OPTION, ]; @@ -47,11 +48,43 @@ const SWITCH_OPTIONS: readonly string[] = [ "--deny-all", "--secret-detection", "--no-secret-detection", + RUN_OPTION, "--help", "-h", "--version", ]; +/** + * The options that configure the Plan's execution and nothing else. + * + * `xmd prompt` prints an approved Plan unless `--run` asks for it to be run, so + * each of these describes work that would not happen. Accepting one silently + * would mean answering a caller who asked for a journal, a permission mode or an + * exec deadline with a command that creates none of them. + * + * Everything absent from this list is here for a reason the command always has: + * `--include` builds the catalog and admits properties, `--agent-provider` and + * `--default-agent` settle who writes the Plan, `--session` names the + * conversation, `--timeout` bounds the whole command, and `--output` is where an + * approved Plan goes. + */ +const RUN_ONLY_OPTIONS: readonly string[] = [ + "--journal", + "-j", + "--raw", + "--verbose", + "-V", + "--timeout-exec", + "--timeout-fetch", + "--approve-all", + "--approve-reads", + "--deny-all", + "--secret-detection", + "--no-secret-detection", +]; + +const RUN_ONLY = new Set(RUN_ONLY_OPTIONS); + const VALUE = new Set(VALUE_OPTIONS); const KNOWN = new Set([...VALUE_OPTIONS, ...SWITCH_OPTIONS, AGGREGATE_OPTION]); @@ -128,6 +161,8 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { const occurrences: PropertyOccurrence[] = []; let request: string | undefined; let extra: string | undefined; + let runs = false; + let runOnly: string | undefined; let parsingOptions = true; let index = 1; @@ -173,6 +208,12 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { continue; } + if (name === RUN_OPTION) { + runs = true; + } + if (runOnly === undefined && RUN_ONLY.has(name)) { + runOnly = name; + } const separated = equals === -1 && VALUE.has(name) && args[index + 1] !== undefined ? args[index + 1] @@ -247,6 +288,17 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { }; } + if (runOnly !== undefined && !runs) { + return { + request, + fixed, + occurrences, + error: + `${runOnly} configures running the Plan, and without ${RUN_OPTION} this command ` + + `writes the Plan instead of running it — add ${RUN_OPTION}, or drop ${runOnly}`, + }; + } + return { request, fixed, occurrences }; } diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index d50ce341f..bde862b8b 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -189,7 +189,7 @@ export function* runPromptCommandDocument(profile: PromptProfile): Operation`, `` or anything else. + // answer for ``, `` or anything else. includes: [], props: { request: profile.request, @@ -211,16 +211,16 @@ export function* runPromptCommandDocument(profile: PromptProfile): Operation - function* validateCandidate(props: Record) { + function* checkDraft(props: Record) { const assessment = yield* profile.assess(String(props.source)); return { valid: assessment.valid, diagnostics: assessment.diagnostics }; }, @@ -274,19 +274,21 @@ function profileCeiling(profile: PromptProfile, workdir: string): AcpxProviderDe /** * What the assistant session is told once, before it is asked anything. * - * The host owns this layer, and it owns only this: the shape of every answer, - * which is the one thing a policy cannot state about itself convincingly. What - * to write, which vocabulary is available, how to repair a draft and what a - * person asked to change are the prompt command document's text, and they arrive - * in the turns it sends. + * The host owns this layer, and it owns only this: that an answer belongs to the + * message that asked for it. Which shape any particular message wants — a Plan, + * or an explanation of why there is not one — is that message's own business, and + * every message is the prompt command document's text. Hiding a shape here would + * be hiding a policy decision in a place nobody reviewing the workflow can read. */ export const PROMPT_INSTRUCTIONS = [ - "You write complete executable Markdown root documents. Each user message is a", - "request about one document.", + "You are the coding agent behind `xmd prompt`. A workflow asks you for one thing", + "at a time, on behalf of one person, and every message states what its answer has", + "to be.", "", - "Every answer is replacement document source and nothing else: no enclosing code", - "fence, no explanation, and no prose around it. What you return is validated and", - "executed exactly as you wrote it.", + "Answer the message you were sent, in the shape it asked for, and nothing else. A", + "message asking for a Plan is answered with Plan source; a message asking for an", + "explanation is answered with an explanation. Never answer one in the shape the", + "other asked for.", ].join("\n"); /** diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index ce5289dfa..6fcb20472 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -37,6 +37,7 @@ import { open } from "node:fs/promises"; import type { FileHandle } from "node:fs/promises"; import { randomUUID } from "node:crypto"; import { resolve } from "node:path"; +import process from "node:process"; import { agentIdentityComponents, @@ -100,8 +101,10 @@ export interface PromptCommand { /** What fixed grammar established about that argv. */ scan: PromptScan; include: string[]; - /** Where the approved source is written before it runs, when asked for. */ - save?: string; + /** Where the approved Plan is written, when the caller asked for a file. */ + output?: string; + /** Whether the caller asked for the approved Plan to be run. */ + run: boolean; /** The logical assistant-session name, when the caller chose one. */ session?: string; /** @@ -240,14 +243,27 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op } const source = authored.value; - if (command.save !== undefined) { - const saved = yield* saveSource(command.save, source); - if (!saved.ok) { - console.error(saved.error.message); + if (command.output !== undefined) { + // Before the run, so a Plan that fails at run time is still on disk to read + // and hand-edit. An existing path is refused and nothing after it happens. + const written = yield* writeOutput(command.output, source); + if (!written.ok) { + console.error(written.error.message); return 1; } } + if (!command.run) { + // The approved Plan is the result. It goes to stdout exactly as the agent + // wrote it — no fence, no heading, no trailing newline of this command's — + // so a caller can pipe it into a file, a diff or another program. A caller + // who named `--output` already has it, and gets a quiet command instead. + if (command.output === undefined) { + process.stdout.write(source); + } + return 0; + } + const executed = yield* deps.execute({ root: retainedSource(PROMPT_IDENTITY, source), props: admitted.props, @@ -380,13 +396,13 @@ function* assessCandidate( } /** - * Create the destination and write the approved bytes, or refuse. + * Create the destination and write the approved Plan, or refuse. * - * Exclusive creation, so an existing path is left exactly as it is and the run - * stops rather than replacing work somebody kept. There is no check-then-write: - * the open is the check. + * Exclusive creation, so an existing path is left exactly as it is and the + * command stops rather than replacing work somebody kept. There is no + * check-then-write: the open is the check. */ -function* saveSource(path: string, source: string): Operation> { +function* writeOutput(path: string, source: string): Operation> { const target = resolve(yield* cwd(), path); let handle: FileHandle; try { @@ -398,7 +414,7 @@ function* saveSource(path: string, source: string): Operation> { if (existing) { return Err( new Error( - `${target} already exists — choose another --save path; the approved document was ` + + `${target} already exists — choose another --output path; the approved Plan was ` + "not written and nothing ran", ), ); @@ -419,7 +435,7 @@ function* writeAll(handle: FileHandle, source: string): Operation> yield* until(handle.writeFile(source, "utf8")); return Ok(undefined); } catch (error) { - return Err(new Error(`could not write the approved document: ${describeError(error)}`)); + return Err(new Error(`could not write the approved Plan: ${describeError(error)}`)); } } diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts index fa575fef1..da4d2622c 100644 --- a/packages/cli/tests/packaged-document.test.ts +++ b/packages/cli/tests/packaged-document.test.ts @@ -39,19 +39,44 @@ describe("packaged documents", () => { expect(source).toBe(committed); // It is the prompt command document, not merely some file that exists. expect(source).toContain("returns:"); - expect(source).toContain("` directly. The // single-child `` wrapper existed only because the component refused a // literal `as`, and main supplies that now. expect(source).toContain(''); + // The closing branch is an unexpected-no-decision fallback, not a second + // copy of exhaustion: exhaustion is decided inside review, and saying it + // twice would make the two endings indistinguishable to a reader. + expect(source).toContain( + '', + ); + expect(source.split("reviewed ten drafts without an approved Plan").length - 1).toBe(2); + // The choices are the words a person reads, with no internal spelling + // behind them. + expect(source).toContain('["Approve", "Request changes", "Stop"]'); + expect(source).toContain('["Explain what went wrong", "Stop"]'); + expect(source).not.toContain('"revise"'); + expect(source).not.toContain('"abort"'); expect(source).not.toContain(''); }); diff --git a/packages/cli/tests/prompt-args.test.ts b/packages/cli/tests/prompt-args.test.ts index 36dffa92b..b2ae5231a 100644 --- a/packages/cli/tests/prompt-args.test.ts +++ b/packages/cli/tests/prompt-args.test.ts @@ -73,12 +73,19 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(early.request).toBe(undefined); expect(early.occurrences).toEqual([]); - const aggregate = scanPromptArgs(["prompt", "--props", '{"name":"Ada"}', REQUEST, "--raw"]); + const aggregate = scanPromptArgs([ + "prompt", + "--props", + '{"name":"Ada"}', + REQUEST, + "--raw", + "--run", + ]); expect(aggregate.error).toBe(undefined); expect(aggregate.request).toBe(REQUEST); // The aggregate never reaches the parser: it coerces a separated value // through Number() before any schema could judge it. - expect(aggregate.fixed).toEqual(["prompt", REQUEST, "--raw"]); + expect(aggregate.fixed).toEqual(["prompt", REQUEST, "--raw", "--run"]); const inline = scanPromptArgs(["prompt", '--props={"name":"Ada"}', REQUEST]); expect(inline.error).toBe(undefined); @@ -97,10 +104,11 @@ describe("Tier PR — xmd prompt fixed grammar", () => { "--props-loud", "--journal", "trace.jsonl", - "--save", + "--output", "out.md", "--session", "ada", + "--run", ]); expect(scan.error).toBe(undefined); expect(scan.request).toBe(REQUEST); @@ -112,10 +120,11 @@ describe("Tier PR — xmd prompt fixed grammar", () => { "lib", "--journal", "trace.jsonl", - "--save", + "--output", "out.md", "--session", "ada", + "--run", ]); // `--props-loud` did not swallow `--journal`: a known option is never read // as a generated property's value. @@ -214,6 +223,47 @@ describe("Tier PR — xmd prompt fixed grammar", () => { expect(stray).toContain("--props-loud=true"); }); + it("C1: options that only configure a run need --run to mean anything", function* () { + // Each of them describes work that a command writing a Plan never does. A + // caller who asked for a journal, a permission mode or an exec deadline and + // got a command that creates none of them was not answered. + for (const flag of [ + ["--journal", "trace.jsonl"], + ["-j", "trace.jsonl"], + ["--raw"], + ["--verbose"], + ["--timeout-exec", "5s"], + ["--timeout-fetch", "5s"], + ["--approve-all"], + ["--approve-reads"], + ["--deny-all"], + ["--no-secret-detection"], + ]) { + const refused = scanPromptArgs(["prompt", REQUEST, ...flag]); + expect(refused.error).toContain(`${flag[0]} configures running the Plan`); + expect(refused.error).toContain("add --run"); + + // With `--run` the same command line is ordinary, wherever the two are + // written relative to each other. + expect(scanPromptArgs(["prompt", REQUEST, ...flag, "--run"]).error).toBe(undefined); + expect(scanPromptArgs(["prompt", REQUEST, "--run", ...flag]).error).toBe(undefined); + } + + // The options the command always uses are never refused: they build the + // catalog, settle the agent, name the session, bound the command and say + // where the Plan goes. + for (const flag of [ + ["--include", "lib"], + ["--agent-provider", "acpx"], + ["--default-agent", "codex"], + ["--session", "ada"], + ["--timeout", "5s"], + ["--output", "plan.md"], + ]) { + expect(scanPromptArgs(["prompt", REQUEST, ...flag]).error).toBe(undefined); + } + }); + it("C7: a frozen option's shape is what a later candidate may not change", function* () { const scalar = bindingsFor({ name: { type: "string" } }); const boolean = bindingsFor({ name: { type: "boolean" } }); diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 94dce6bfa..5efb064cb 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -16,11 +16,11 @@ import { expect } from "@executablemd/test-support/expect"; import { runCli } from "@executablemd/test-support/launch"; import { readTextFile, writeTextFile } from "@effectionx/fs"; import { stat } from "@executablemd/runtime"; -import { ensure, Ok, scoped, spawn } from "effection"; +import { ensure, Ok, scoped, spawn, until } from "effection"; import type { Operation, Result } from "effection"; import { join } from "node:path"; import { readdir } from "node:fs/promises"; -import { until } from "effection"; +import process from "node:process"; import { promptExecutor } from "../src/cli.ts"; import { resolveAgentStack } from "../src/agent-stack.ts"; @@ -91,13 +91,35 @@ const STACK: AgentStack = { permissionMode: "deny-all", }; -function command(dir: string, args: string[], save?: string): PromptCommand { +/** + * One invocation, asking for the approved Plan to be run. + * + * `--run` is the default here because these cases are about the run: what the + * journal holds, what the document writes, how a runtime failure reports. The + * modes that write the Plan instead are exercised by their own cases below, + * which pass `run: false` and say which destination they mean. + */ +function command(dir: string, args: string[], output?: string): PromptCommand { + const argv = ["prompt", ...args, "--run"]; + return { + argv, + scan: scanPromptArgs(argv), + include: [dir], + ...(output === undefined ? {} : { output }), + run: true, + stack: STACK, + }; +} + +/** The same invocation, writing the Plan rather than running it. */ +function writing(dir: string, args: string[], output?: string): PromptCommand { const argv = ["prompt", ...args]; return { argv, scan: scanPromptArgs(argv), include: [dir], - ...(save === undefined ? {} : { save }), + ...(output === undefined ? {} : { output }), + run: false, stack: STACK, }; } @@ -193,7 +215,17 @@ describe( for (const flag of ["-e", "--eval"]) { yield* useWorkingDirectory(function* (dir) { const { code, stdout, stderr } = yield* runCli( - ["prompt", REQUEST, flag, "# supplied", "--save", "out.md", "--journal", "trace.jsonl"], + [ + "prompt", + REQUEST, + flag, + "# supplied", + "--output", + "out.md", + "--run", + "--journal", + "trace.jsonl", + ], { cwd: dir }, ).join(); @@ -218,7 +250,7 @@ describe( [ "prompt", "--help", - "--save", + "--output", "out.md", "--journal", "trace.jsonl", @@ -230,15 +262,22 @@ describe( expect(code).toBe(0); expect(stdout).toContain("Usage: xmd prompt [OPTIONS] [request]"); - expect(stdout).toContain("Exactly one request is required"); + expect(stdout).toContain("Exactly one Prompt is required"); expect(stdout).toContain("--props "); expect(stdout).toContain("XMD_PROPS"); - expect(stdout).toContain("--save "); + // Where an approved Plan goes, and what changes that. + expect(stdout).toContain("The approved Plan is the result. By default it is written to"); + expect(stdout).toContain("--output "); + expect(stdout).toContain("Write the approved Plan there instead of to stdout"); + expect(stdout).toContain("--run"); + expect(stdout).toContain("Run the approved Plan instead of writing it"); expect(stdout).toContain("--session "); - // The permission flags are the approved document's, and help says so - // rather than letting a caller believe they configure how the Plan is - // written. - expect(stdout).toContain("Permission flags configure the approved document"); + // The run-only flags are named as such rather than left to be + // discovered by a refusal. + expect(stdout).toContain("are\nrefused without --run"); + // The permission flags are the approved Plan's, and help says so rather + // than letting a caller believe they configure how the Plan is written. + expect(stdout).toContain("Permission flags configure the approved Plan"); expect(stderr).not.toContain("unavailable"); // No catalog was rendered, and neither file the options named was made. expect(stdout).not.toContain("## Built-in components"); @@ -257,6 +296,20 @@ describe( expect(empty.code).toBe(1); expect(empty.stderr).toContain("--session needs a name"); expect(empty.stdout).not.toContain("## Built-in components"); + + // A flag that only configures running a Plan, without --run: refused + // before authorship and before anything reaches the filesystem. + for (const flag of [["--journal", "trace.jsonl"], ["--raw"], ["--deny-all"]]) { + const stray = yield* runCli(["prompt", REQUEST, ...flag], { cwd: dir }).join(); + expect(stray.code).toBe(1); + expect(stray.stderr).toContain(`${flag[0]} configures running the Plan`); + expect(stray.stderr).toContain("add --run"); + expect(stray.stdout).toBe(""); + expect(yield* exists(join(dir, "trace.jsonl"))).toBe(false); + } + // That the same flag is ordinary again once `--run` is present is fixed + // grammar, and is proven there rather than by an invocation that would + // have to reach a real agent to say so. }); }); @@ -265,7 +318,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: GREETER }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); yield* useEnvironment({ XMD_PROPS: '{"name":"FromEnv","count":7}' }); const code = yield* runPrompt( @@ -288,7 +341,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, join(dir, journalName)); harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); @@ -336,7 +389,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, journal); harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -352,6 +405,99 @@ describe( }); }); + it("C11: the approved Plan is the result, and where it goes is the caller's", function* () { + // Default: the exact source on stdout, and nothing runs. Written with + // `process.stdout.write`, so what a pipe receives is the bytes and not a + // line the command added. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "Approve" }); + + const written: string[] = []; + const original = process.stdout.write.bind(process.stdout); + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + process.stdout.write = original; + }); + process.stdout.write = ((chunk: string | Uint8Array) => { + written.push(typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk)); + return true; + }) as typeof process.stdout.write; + return yield* runPrompt(writing(dir, [REQUEST]), harness.deps); + }); + + expect(code).toBe(0); + // Byte for byte, once, with nothing around it. + expect(written.join("")).toBe(PLAIN); + // And nothing ran: no execution, so no journal and no document effects. + expect(harness.executions).toHaveLength(0); + expect(yield* until(readdir(dir))).toEqual([]); + }); + + // `--output`: the same bytes in the file, a quiet stdout, and still no run. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "Approve" }); + + const written: string[] = []; + const original = process.stdout.write.bind(process.stdout); + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + process.stdout.write = original; + }); + process.stdout.write = ((chunk: string | Uint8Array) => { + written.push(typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk)); + return true; + }) as typeof process.stdout.write; + return yield* runPrompt( + { + ...writing(dir, [REQUEST, "--props-name", "Ada"], "plan.md"), + argv: ["prompt", REQUEST, "--props-name", "Ada", "--output", "plan.md"], + }, + harness.deps, + ); + }); + + expect(code).toBe(0); + expect(written.join("")).toBe(""); + expect(yield* readTextFile(join(dir, "plan.md"))).toBe(GREETER); + // The Plan was not run, so what it would have written is not there. + expect(harness.executions).toHaveLength(0); + expect((yield* until(readdir(dir))).sort()).toEqual(["plan.md"]); + }); + + // `--run`: the Plan runs, and stdout is the Plan's own to use. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + harness.deps.execute = executor(dir); + harness.fake.script({ reply: GREETER }); + harness.script({ decision: "Approve" }); + + const written: string[] = []; + const original = process.stdout.write.bind(process.stdout); + const code = yield* scoped(function* (): Operation { + yield* ensure(() => { + process.stdout.write = original; + }); + process.stdout.write = ((chunk: string | Uint8Array) => { + written.push(typeof chunk === "string" ? chunk : new TextDecoder().decode(chunk)); + return true; + }) as typeof process.stdout.write; + return yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); + }); + + expect(code).toBe(0); + // The source itself was never printed. + expect(written.join("")).not.toContain("props:"); + // The document ran, and did what it says. + expect(yield* readTextFile(join(dir, "greeting.txt"))).toBe("name=Ada loud=false count="); + }); + }); + it("C11: the approved bytes are created exclusively, before the run", function* () { // Created before execution, and byte for byte. yield* useWorkingDirectory(function* (dir, profileRoot) { @@ -363,7 +509,7 @@ describe( return yield* run(approved); }; harness.fake.script({ reply: GREETER }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt( command(dir, [REQUEST, "--props-name", "Ada"], "out.md"), @@ -383,7 +529,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); @@ -397,7 +543,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: GREETER }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); @@ -413,7 +559,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, journal); harness.fake.script({ reply: GREETER }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt( command(dir, [REQUEST, "--props-name", "Ada"], "out.md"), @@ -435,7 +581,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: FAILS_AT_RUN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); @@ -452,7 +598,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir); harness.fake.script({ reply: FAILING_TEST }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const written = console.error; const lines: string[] = []; @@ -507,7 +653,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, undefined, stack); harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt({ ...command(dir, [REQUEST]), stack }, harness.deps); @@ -553,7 +699,7 @@ describe( harness.deps.execute = executor(dir); harness.fake.closeFailure = new Error("the profile provider would not close"); harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST], "out.md"), harness.deps); @@ -571,7 +717,7 @@ describe( return Ok(undefined); }; harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); diff --git a/packages/cli/tests/prompt-command-document.test.ts b/packages/cli/tests/prompt-command-document.test.ts index f08bc5e96..0d62aa955 100644 --- a/packages/cli/tests/prompt-command-document.test.ts +++ b/packages/cli/tests/prompt-command-document.test.ts @@ -95,7 +95,7 @@ function* runDocument(): Operation { // deno-lint-ignore require-yield *elicit([request], _next) { reviews.push(request); - return { decision: "approve" }; + return { decision: "Approve" }; }, }, { at: "min" }, @@ -119,7 +119,7 @@ function* runDocument(): Operation { components: [ ...agentIdentityComponents(), { - name: "ValidateCandidate", + name: "CheckDraft", origin: "test", forms: ["self-closing"] as const, props: { diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index d7b4e857e..0eb2a1edb 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -125,9 +125,16 @@ const STACK: AgentStack = { permissionMode: "deny-all", }; +/** + * One invocation, asking for the approved Plan to be run. + * + * `--run` is the default here because these cases are about what reaches the + * execution: the modes that write the Plan instead have their own cases, and + * name the mode they mean. + */ function command(dir: string, args: string[], stack: AgentStack = STACK): PromptCommand { - const argv = ["prompt", ...args]; - return { argv, scan: scanPromptArgs(argv), include: [dir], stack }; + const argv = ["prompt", ...args, "--run"]; + return { argv, scan: scanPromptArgs(argv), include: [dir], run: true, stack }; } /** Every session key the fake was asked to establish, deduplicated in order. */ @@ -164,7 +171,7 @@ function watching( *elicit([request], _next) { harness.reviews.push(request); yield* observe(String(harness.fake.created[0]?.cwd)); - return { decision: "approve" }; + return { decision: "Approve" }; }, }, { at: "min" }, @@ -234,7 +241,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -246,16 +253,19 @@ describe( // exist nowhere in TypeScript, so a host that chose its own policy could // not produce them. const turn = harness.fake.prompts[0]; - expect(turn).toContain("Write one complete Executable Markdown document that does this"); + expect(turn).toContain("Create one complete XMD Plan from this Prompt"); // C14: the shipped instruction is the narrative-plus-components rule. - expect(turn).toContain("The document you write is a Plan"); - expect(turn).toContain("ordinary reader-facing prose"); expect(turn).toContain( - "places each component\nimmediately after the sentences describing the action", + "Begin the Plan with one level-one Markdown title that describes it", ); + expect(turn).toContain("Write the Plan as a sequence of readable steps"); + expect(turn).toContain("Follow each step with the XMD\ncomponent that carries it out"); // The worked example travels with it, unparsed. expect(turn).toContain('{answer.age}'); expect(turn).toContain("keep every outcome it asked for"); + // The worked example is a titled Plan, so the shape being asked for is + // the shape being shown. + expect(turn).toContain("# Ask for and save your age"); // The request travels inside it, byte for byte. expect(turn).toContain(REQUEST); // So does the host's catalog, including the renderer's honest statement @@ -289,9 +299,9 @@ describe( harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); - harness.script({ decision: "revise", feedback: "try plain text" }); + harness.script({ decision: "Request changes", feedback: "try plain text" }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -325,7 +335,7 @@ describe( yield* useWorkingDirectory(function* (dir) { const harness = createPromptHarness({ profileRoot, store }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt( { ...command(dir, [REQUEST]), session: "ada" }, @@ -385,7 +395,7 @@ describe( // Abort, a turn that failed, and a cancelled command each hand the // directory back the same way a success does. for (const ending of [ - { name: "abort", drive: (harness: PromptHarness) => harness.script({ decision: "abort" }) }, + { name: "stop", drive: (harness: PromptHarness) => harness.script({ decision: "Stop" }) }, { name: "a failed turn", drive: (harness: PromptHarness) => { @@ -395,7 +405,7 @@ describe( ]) { yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); - if (ending.name === "abort") { + if (ending.name === "stop") { harness.fake.script({ reply: VALID }); } ending.drive(harness); @@ -486,7 +496,7 @@ describe( }); const { value, lines } = yield* reported(() => - runPrompt({ ...command(dir, [REQUEST]), save: "out.md" }, harness.deps), + runPrompt({ ...command(dir, [REQUEST]), output: "out.md" }, harness.deps), ); expect(value).toBe(1); @@ -518,7 +528,7 @@ describe( }); const { value, lines } = yield* reported(() => - runPrompt({ ...command(dir, [REQUEST]), save: "out.md" }, harness.deps), + runPrompt({ ...command(dir, [REQUEST]), output: "out.md" }, harness.deps), ); // Terminal, and said once: the attempt happens once and either settles @@ -617,7 +627,7 @@ describe( seen.refusals.push(error instanceof Error ? error.message : String(error)); } } - return { decision: "approve" }; + return { decision: "Approve" }; }, }, { at: "min" }, @@ -713,7 +723,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -730,7 +740,7 @@ describe( yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: WRITES_A_FILE }); - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); @@ -749,13 +759,13 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: BROKEN_SOURCE }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); expect(harness.fake.prompts).toHaveLength(2); - expect(harness.fake.prompts[1]).toContain("That document has problems"); + expect(harness.fake.prompts[1]).toContain("That Plan has problems"); // `` captured the whole serialized value, and the fence it went // into holds every byte of it: the wrapper's removal changed where the // text is bound, not what it says. Parsed back rather than matched, so @@ -777,7 +787,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: COLLIDING }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); @@ -841,7 +851,7 @@ describe( yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: REQUIRES_NAME }); - harness.script({ decision: "revise", feedback: "make it shout" }); + harness.script({ decision: "Request changes", feedback: "make it shout" }); harness.fake.script({ reply: NAME_IS_BOOLEAN }); const code = yield* runPrompt(command(dir, [REQUEST, "--props-name", "Ada"]), harness.deps); @@ -861,14 +871,14 @@ describe( harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: UNRESOLVED }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); expect(harness.fake.prompts).toHaveLength(4); expect(harness.reviews).toHaveLength(1); - expect(decisions(harness.reviews[0])).toEqual(["approve", "revise", "abort"]); + expect(decisions(harness.reviews[0])).toEqual(["Approve", "Request changes", "Stop"]); }); // A fourth invalid candidate is repair-exhausted: it reaches review with @@ -878,16 +888,16 @@ describe( for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: UNRESOLVED }); } - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(1); expect(harness.fake.prompts).toHaveLength(4); expect(harness.reviews).toHaveLength(1); - expect(decisions(harness.reviews[0])).toEqual(["revise", "abort"]); + expect(decisions(harness.reviews[0])).toEqual(["Request changes", "Stop"]); expect(harness.reviews[0].message).toContain( - "Three attempts to fix it did not clear everything", + "The coding agent used all three repair attempts", ); // The presentation carries the same complete captured JSON the repair // turns did, through the same `` binding. @@ -906,10 +916,10 @@ describe( for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { harness.fake.script({ reply: VALID }); if (round < 10) { - harness.script({ decision: "revise", feedback: `round ${round}` }); + harness.script({ decision: "Request changes", feedback: `round ${round}` }); } } - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); @@ -919,11 +929,11 @@ describe( expect(harness.fake.prompts).toHaveLength(10); expect(harness.fake.prompts[1]).toContain("round 1"); for (const round of [0, 1, 2, 3, 4, 5, 6, 7, 8]) { - expect(decisions(harness.reviews[round])).toEqual(["approve", "revise", "abort"]); + expect(decisions(harness.reviews[round])).toEqual(["Approve", "Request changes", "Stop"]); } // The last presentation offers no revision, because there is no eleventh // round to revise into. - expect(decisions(harness.reviews[9])).toEqual(["approve", "abort"]); + expect(decisions(harness.reviews[9])).toEqual(["Approve", "Stop"]); // One conversation held all ten. expect(sessions(harness)).toHaveLength(1); }); @@ -945,7 +955,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: fenced }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -962,7 +972,7 @@ describe( yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const { value, lines } = yield* reported(() => runPrompt(command(dir, [REQUEST]), harness.deps), @@ -970,7 +980,7 @@ describe( expect(value).toBe(1); expect(lines.join("\n")).toContain( - "xmd prompt: you ended the review, so nothing was saved and nothing ran.", + "xmd prompt stopped at your request. Nothing was output or run.", ); expect(harness.executions).toHaveLength(0); }); @@ -987,7 +997,9 @@ describe( harness.fake.script({ reply: UNRESOLVED }); } harness.script( - round < 10 ? { decision: "revise", feedback: `round ${round}` } : { decision: "abort" }, + round < 10 + ? { decision: "Request changes", feedback: `round ${round}` } + : { decision: "Stop" }, ); } @@ -999,13 +1011,74 @@ describe( expect(harness.reviews).toHaveLength(10); // Rounds one to nine could be sent back; the tenth had one choice, and // taking it is what reaches the exhaustion ending. - expect(decisions(harness.reviews[8])).toEqual(["revise", "abort"]); - expect(decisions(harness.reviews[9])).toEqual(["abort"]); + expect(decisions(harness.reviews[8])).toEqual(["Request changes", "Stop"]); + expect(decisions(harness.reviews[9])).toEqual(["Explain what went wrong", "Stop"]); expect(lines).toHaveLength(1); expect(lines[0]).toBe( - "xmd prompt: ten drafts were reviewed and none was approved, so nothing was saved " + - "and nothing ran. Try again with a more specific request.", + "xmd prompt reviewed ten drafts without an approved Plan. Nothing was output or run.", ); + // Stopping asks the coding agent nothing: forty drafting turns and no + // forty-first. + expect(harness.fake.prompts).toHaveLength(40); + expect(harness.executions).toHaveLength(0); + }); + }); + + it("C3, C9: the last invalid draft can be explained rather than only stopped", function* () { + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + for (const round of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) { + for (const _draft of [0, 1, 2, 3]) { + harness.fake.script({ reply: UNRESOLVED }); + } + harness.script( + round < 10 + ? { decision: "Request changes", feedback: `round ${round}` } + : { decision: "Explain what went wrong" }, + ); + } + // The explanation the coding agent gives is prose, and stays prose. + const explanation = [ + "Every draft used , which this profile does not offer.", + "", + "Say which of the available components should do the work, or describe the", + "outcome without naming a component: ", + ].join("\n"); + harness.fake.script({ reply: explanation }); + + const { value, lines } = yield* reported(() => + runPrompt(command(dir, [REQUEST]), harness.deps), + ); + + expect(value).toBe(1); + // Exactly one turn more than the forty drafting turns, in the same + // conversation. It is not another draft: no eleventh review, and the + // ten-draft limit is not reopened. + expect(harness.fake.prompts).toHaveLength(41); + expect(harness.reviews).toHaveLength(10); + expect(sessions(harness)).toHaveLength(1); + + // It carries the final problems, which were produced after the agent's + // last draft and have not appeared in the conversation — and nothing + // else. The Session already holds the Prompt, the catalog and every + // draft, so none of them is resent. + const asked = harness.fake.prompts[40]; + expect( + fencedJson(asked).validation?.diagnostics?.some( + (entry) => entry.code === "component-unresolved", + ), + ).toBe(true); + expect(asked).toContain("Do not create another Plan"); + expect(asked).not.toContain(REQUEST); + expect(asked).not.toContain("## Built-in components"); + expect(asked).not.toContain(UNRESOLVED.trim()); + + // What it said is reported as the coding agent's words, and the command + // ends. Nothing about the explanation is treated as a Plan. + expect(lines).toHaveLength(1); + expect(lines[0]).toContain("reviewed ten drafts without an approved Plan"); + expect(lines[0]).toContain(explanation); + expect(lines[0]).toContain("Nothing was output or run."); expect(harness.executions).toHaveLength(0); }); }); @@ -1016,9 +1089,9 @@ describe( yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: counting("number") }); - harness.script({ decision: "revise", feedback: "count in words" }); + harness.script({ decision: "Request changes", feedback: "count in words" }); harness.fake.script({ reply: counting("string") }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST, "--props-count", "7"]), harness.deps); @@ -1038,7 +1111,7 @@ describe( for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: wrapped }); } - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(harness.reviews[0].message).toContain(wrapped); @@ -1049,7 +1122,7 @@ describe( yield* useWorkingDirectory(function* (dir, profileRoot) { // A repository component the draft uses. It exists while the command // document runs, so the same production validator that answers - // finds the draft sound. + // finds the draft sound. const widget = join(dir, "Widget.md"); yield* writeTextFile(widget, "A widget.\n"); const draft = ["# Uses a widget", "", "", ""].join("\n"); @@ -1070,7 +1143,7 @@ describe( // deno-lint-ignore require-yield *elicit([request], _next) { harness.reviews.push(request); - return { decision: "approve" }; + return { decision: "Approve" }; }, }, { at: "min" }, @@ -1136,7 +1209,7 @@ describe( const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: plan }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt( command(dir, ["ask me for my age and write it to a file"]), @@ -1160,10 +1233,10 @@ describe( for (const _draft of [0, 1, 2, 3]) { harness.fake.script({ reply: UNRESOLVED }); } - harness.script({ decision: "revise", feedback: "use plain text" }); + harness.script({ decision: "Request changes", feedback: "use plain text" }); // Round two: a draft that passes, and is approved. harness.fake.script({ reply: VALID }); - harness.script({ decision: "approve" }); + harness.script({ decision: "Approve" }); const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); expect(code).toBe(0); @@ -1173,24 +1246,25 @@ describe( // second presentation carries neither the first draft nor its problems: // a loop that accumulated its explanation would show both. for (const message of [exhausted, approvable]) { - expect(occurrences(message, "This document was written for:")).toBe(1); + expect(occurrences(message, "### Original Prompt")).toBe(1); + expect(occurrences(message, "### Draft Plan")).toBe(1); } - expect(occurrences(exhausted, "Three attempts to fix it did not clear everything")).toBe(1); - expect(approvable).not.toContain("Three attempts to fix it"); + expect(occurrences(exhausted, "### Problems that remain")).toBe(1); + expect(approvable).not.toContain("Problems that remain"); expect(approvable).not.toContain("NoSuchComponent"); expect(exhausted).not.toContain(VALID.trim()); // The repair turns say what they need once each, and carry only the // problems of the draft they are repairing. for (const repair of harness.fake.prompts.slice(1, 4)) { - expect(occurrences(repair, "That document has problems")).toBe(1); - expect(occurrences(repair, "Send one complete replacement document")).toBe(1); + expect(occurrences(repair, "That Plan has problems")).toBe(1); + expect(occurrences(repair, "Send one complete replacement Plan")).toBe(1); } // The revision turn says what changed, once, and asks for a whole // document rather than repeating the original brief. const revision = harness.fake.prompts[4]; expect(occurrences(revision, "use plain text")).toBe(1); - expect(revision).not.toContain("The document you write is a Plan"); + expect(revision).not.toContain("Create one complete XMD Plan from this Prompt"); }); // Stopping prints one sentence, once, whether it happened on the first @@ -1202,10 +1276,10 @@ describe( for (const round of Array.from({ length: rounds }, (_, i) => i + 1)) { harness.fake.script({ reply: VALID }); if (round < rounds) { - harness.script({ decision: "revise", feedback: `round ${round}` }); + harness.script({ decision: "Request changes", feedback: `round ${round}` }); } } - harness.script({ decision: "abort" }); + harness.script({ decision: "Stop" }); const { value, lines } = yield* reported(() => runPrompt(command(dir, [REQUEST]), harness.deps), @@ -1215,8 +1289,8 @@ describe( // One line, from the document's own , and no accumulated // repetition of it however many rounds preceded it. expect(lines).toHaveLength(1); - expect(occurrences(lines[0], "xmd prompt:")).toBe(1); - expect(lines[0]).toContain("you ended the review"); + expect(occurrences(lines[0], "xmd prompt ")).toBe(1); + expect(lines[0]).toContain("stopped at your request"); }); } }); diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts index a71de417a..becd62beb 100644 --- a/packages/cli/tests/support/prompt-harness.ts +++ b/packages/cli/tests/support/prompt-harness.ts @@ -30,9 +30,14 @@ import type { FakeAcp, FakeStore } from "./fake-acp.ts"; /** The agent every prompt case drives, and the command it resolves to. */ export const AGENT = "scripted-agent"; -/** One review answer, scripted. */ +/** + * One review answer, scripted. + * + * The values are the words a person reads and the words the provider answers + * with: the workflow keeps no internal spelling behind them. + */ export interface ScriptedReview { - decision: "approve" | "revise" | "abort"; + decision: "Approve" | "Request changes" | "Stop" | "Explain what went wrong"; feedback?: string; /** Answer with this instead, to drive a response the schema rejects. */ raw?: unknown; diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 2a87f6936..2d978276f 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -587,9 +587,11 @@ terminally. Which applies is a trusted host value — whether `--session` was written — and where the directories live is a host dependency no caller or document can select. -Only the profile's provider receives `newSessionOptions.systemPrompt`. It carries the -fixed statement of what the session writes, and nothing else — the catalog and -the request are the command document's to send in its own turns +Only the profile's provider receives `newSessionOptions.systemPrompt`. It carries +one fixed statement — that an answer belongs to the message that asked for it — +and nothing else. Which shape any particular message wants, a Plan or an +explanation of why there is not one, is that message's own business, and every +message is the command document's to send in its own turns ([`xmd prompt`](./prompt-command-spec.md)). ACPX applies those options when it creates a session and ignores them when it reuses a record. Without `--session` the command places a name unique to the invocation, so each invocation is a fresh diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 22636246f..3c0781d65 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -86,13 +86,27 @@ xmd prompt -- "--this is the request" `xmd prompt` takes the complete `xmd run -e` execution flag set — `--include`, `--verbose`, `--journal`/`-j`, `--raw`, `--agent-provider`, `--default-agent`, `--approve-all`, `--approve-reads`, `--deny-all`, `--no-secret-detection`, and -the three timeout options — plus `--save` and `--session`. +the three timeout options — plus `--output`, `--run` and `--session`. -Agent provider selection, default agent selection and `--timeout` configure how -the Plan is written. The permission flags and the exec and fetch timeouts -configure only the approved Plan; authorship does not inherit the final run's -permission mode, and no permission flag widens the prompt profile's ceiling -below. +`--include`, `--agent-provider`, `--default-agent`, `--session` and `--timeout` +are always in use: they build the catalog, settle who writes the Plan, name the +conversation, admit properties and bound the whole command. + +Every other execution flag configures **running** a Plan and nothing else — +`--journal`/`-j`, `--raw`, `--verbose`/`-V`, `--timeout-exec`, `--timeout-fetch`, +the three permission flags, and `--secret-detection`/`--no-secret-detection`. +Without `--run` nothing runs for them to configure, so writing one is refused in +fixed preflight rather than accepted and ignored: + +```console +$ xmd prompt "…" --journal trace.jsonl +--journal configures running the Plan, and without --run this command writes the +Plan instead of running it — add --run, or drop --journal +``` + +The permission flags configure only the approved Plan; authorship does not +inherit the final run's permission mode, and no permission flag widens the +prompt profile's ceiling below. `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with @@ -106,16 +120,26 @@ permission mode — and that settled answer is what both the prompt profile and the approved Plan's installation are configured from. Incompatible permission flags and an unknown provider fail there, before the catalog is built. -### `--save ` +### The approved Plan is the result + +The Plan is what the command produces, and where it goes is the caller's choice: -Write the approved source to `path` before executing it. The path is resolved -against the contextual working directory and **created exclusively**: an -existing path is left exactly as it is, the command fails, and nothing runs. -There is no check-then-write — the exclusive create *is* the check. +| Invocation | Result | +| --- | --- | +| `xmd prompt ""` | the exact approved source on **stdout**; nothing runs | +| `xmd prompt "" --output ` | the exact bytes in ``; no source on stdout; nothing runs | +| `xmd prompt "" --run` | the Plan runs; no source is printed, and stdout belongs to the Plan's own output | +| `xmd prompt "" --output --run` | the file is created first, and only a successful write is followed by the run | + +Stdout carries the approved source and nothing else: no fence, no label, no +delimiter and no newline this command added. What a caller reads is what the +coding agent wrote, so it can be piped into a file, a diff or another program. -The file holds the approved Plan and nothing else: no problems, no decision, no -wrapper. Without `--save`, no file is created anywhere; the approved Plan runs -from memory. +`--output ` is resolved against the contextual working directory and +**created exclusively**: an existing path is left exactly as it is, the command +fails, and nothing runs. There is no check-then-write — the exclusive create *is* +the check. The file holds the approved Plan and nothing else: no problems, no +decision, no wrapper. Without `--output`, no file is created anywhere. ### `--session ` @@ -134,10 +158,11 @@ request and the current catalog in this invocation's initial turn. xmd prompt --help ``` -Help needs no request. It describes the request, `--save`, `--session`, the -aggregate `--props`/`XMD_PROPS` sources, that candidate-declared individual -options follow the request, that the permission flags configure the approved -document, and every shared execution flag. It describes no individual property, +Help needs no request. It describes the Prompt, `--output`, `--run`, +`--session`, the aggregate `--props`/`XMD_PROPS` sources, that Plan-declared +individual options follow the Prompt, that stdout is where an approved Plan goes +by default, that the run-only flags are refused without `--run`, and that the +permission flags configure the approved Plan. It describes no individual property, because the document that would declare one does not exist yet — answering otherwise would mean generating a document in order to describe one. @@ -304,7 +329,7 @@ may those exact bytes enter ordinary execution. The prompt profile declares one internal value component to the execution: ```md - + ``` Canonical execution supplies its invocation identity, so repository resolution @@ -402,23 +427,53 @@ The following is the shipped program's behaviour, not the host's. It is stated here because it is what a caller sees; it is changed by editing `prompt-command.md`, and nothing in TypeScript decides it. Prose quality is that document's instructions and your review, never a hidden TypeScript validation -rule: `` reports structural facts and executes nothing. +rule: `` reports structural facts and executes nothing. + +**A complete titled Plan.** Every Plan begins with one descriptive level-one +Markdown heading, as the first body content after optional frontmatter, naming +what the Plan produces. The generation, repair and revision instructions each +require it, together with the Prompt's sequence of readable steps in a +meaningful order and each component beside the step it performs, so the +structure survives every replacement. It is an authorship and human-review +requirement: `` does not enforce it, and a titleless Plan is +something you send back rather than something the checker refuses. **Automatic repair.** The initial draft and every human-requested revision each -start a fresh repair budget: the base candidate is attempt one, at most three -repair turns may replace it, every repair prompt carries the complete structured -diagnostics, and every answer must be another complete replacement root. The -fourth invalid candidate is repair-exhausted and goes to human review with its +start a fresh repair budget: the base draft is attempt one, at most three repair +turns may replace it, every repair prompt carries the complete structured +diagnostics, and every answer must be another complete replacement Plan. The +fourth draft with problems is repair-exhausted and goes to human review with its diagnostics. No fence is stripped, no Markdown substring is extracted and no patch is applied. -**Human review.** At most ten candidate presentations: the initial review plus at -most nine human revisions. Rounds one through nine offer `approve`, `revise` or -`abort` for a valid candidate, and `revise` or `abort` for an invalid -repair-exhausted one. `revise` requires non-empty feedback, sends one -complete-replacement request through the same enclosing Session, and resets the -three-turn repair budget. Round ten offers no further revision: a valid -candidate offers `approve` or `abort`, and an invalid one offers only `abort`. +**Human review.** At most ten draft presentations: the initial review plus at +most nine revisions. The choices are the words shown, and they are the values the +provider answers with — there is no internal spelling behind them: + +| Round | Draft | Choices | +| --- | --- | --- | +| 1–9 | passed its check | **Approve**, **Request changes**, **Stop** | +| 1–9 | problems remain | **Request changes**, **Stop** | +| 10 | passed its check | **Approve**, **Stop** | +| 10 | problems remain | **Explain what went wrong**, **Stop** | + +**Request changes** requires non-empty feedback, sends one complete-replacement +request through the same enclosing Session, and resets the three-turn repair +budget. + +**The explanation turn.** **Explain what went wrong** is offered only on a tenth +draft that still has problems, and makes exactly one more `` in the same +enclosing Session. The Session already holds the original Prompt, the catalog, +every draft, every earlier diagnostic and every revision request, so nothing is +resent: the turn carries only the final diagnostics, which were produced after +the agent's last draft and have not appeared in the conversation. It asks for a +brief explanation and explicitly not another Plan. + +That turn is not a draft, a repair, a revision or a review round; it cannot +reopen the ten-draft limit; its answer is inert text that is never interpreted as +XMD; and it is reported to you and then ends the command with no approved +source, file, journal or execution. It is subject to the host's failed-turn +policy like every other turn, so a turn that fails ends the command immediately. **Presentation.** The review message presents the exact draft with ``, whose fence is longer than every backtick run the draft holds, so draft text @@ -426,44 +481,46 @@ cannot close it and is never interpreted. An invalid draft is followed by its complete JSON problems, serialized and captured by ``. Prose outside `` addresses you; text inside `` instructs the assistant. -**Failure.** `approve` selects the draft, and one exhaustive branch after the -Session returns its source unchanged. Stopping reaches `` with one of two +**Failure.** **Approve** selects the draft, and the branch after the Session +returns its source unchanged. Stopping reaches `` with one of two authored messages, and which one depends on whether an approvable Plan ever existed: -- the tenth presentation of a draft that still has problems offers only `abort`, - and taking that sole choice is **exhaustion** — ten drafts were reviewed and - none was approved; -- every other `abort`, including one on a tenth draft that could have been +- **Stop** on a tenth draft that still has problems is **exhaustion** — ten + drafts were reviewed and none was approved; +- **Explain what went wrong**, offered only there, makes one more turn (below) + and then ends the same way; +- every other **Stop**, including one on a tenth draft that could have been approved, is the ordinary ending: you decided to stop. -The branch after the Session stays as an exhaustive fallback for a body that -somehow reached it having approved nothing. Failure is authored in Markdown +The branch after the Session is only an unexpected-no-decision fallback and says +so; exhaustion is decided inside review, not duplicated there. Failure is authored in Markdown rather than hidden in the host or represented by a missing-`` accident. The command document's rendered output is not command output. Everything you see while a Plan is written reaches you through Elicitation, and that document's successful public result is the approved Plan source and nothing else. -## Final admission, save and execution +## Final admission and the result After the command document has completely torn down, the host treats the -returned string as untrusted again. It repeats candidate validation and property resolution using the exact -returned source, the original raw CLI and invocation-environment property -sources, the individual-option signatures frozen while it was written, the -caller's -ordered includes and the ordinary run profile declarations. +returned string as untrusted again. It repeats the draft check and property resolution using the exact returned +source, the original raw CLI and invocation-environment property sources, the +individual-option signatures frozen while it was written, the caller's ordered +includes and the ordinary run profile declarations. A final caller-source failure or document validation failure exits non-zero -before save or execution. It does not re-enter the command document or ask for a -repair. The final resolved props belong to those exact source bytes; no props -object from an earlier candidate is reused. - -`--save` then exclusively creates the target with those exact bytes. The host -then executes `retainedSource("", source)` through the ordinary -supplied-source run path, with the same includes, output, value result, secret -detection, Agent configuration, permission mode and timeouts `xmd run -e` uses. -No temporary Markdown file is created. +before any result at all: no stdout, no file and no execution. It does not +re-enter the command document or ask for a repair. The final resolved props +belong to those exact source bytes; no props object from an earlier draft is +reused. + +`--output` then exclusively creates the target with those exact bytes. With +`--run`, and only then, the host executes `retainedSource("", source)` +through the ordinary supplied-source run path, with the same includes, output, +value result, secret detection, Agent configuration, permission mode and +timeouts `xmd run -e` uses. No temporary Markdown file is created. Without +`--run` and without `--output`, the exact bytes go to stdout. The `` identity affects positions and diagnostics only. The contextual working directory still resolves relative filesystem operations, repository @@ -471,21 +528,21 @@ components and includes. The executed program receives a fresh ordinary document Agent provider and inherits neither the assistant Session nor its instruction layer. -`--journal` is created when the final execution starts, and only then. That -journal contains no command document, assistant Session, Agent turn, draft -validation, repair or human-review event. Rendered output, a value root's JSON +`--journal` is created when the final execution starts, and only then — so +source-only and `--output` invocations create no journal at all, and neither does +any ending before admission. That journal contains no command document, assistant +Session, Agent turn, draft check, repair or human-review event. Rendered output, a value root's JSON result, runtime failure reporting and exit codes are byte-for-byte ordinary `xmd run` behaviour — including a failing `` boundary, which is reported under its own `tests failed:` heading rather than as a bare message. A runtime -failure does not send the command back to authorship; `--save` has already -completed, -and the source is there to hand-edit. +failure does not send the command back to authorship; `--output` has already +completed, and the Plan is there to hand-edit. ## Timeouts `--timeout` bounds the whole command: preflight, catalog construction, the command document's execution, Elicitation, its teardown, final validation, the -save and the execution. Expiry is Effection cancellation, so structured teardown completes +file and the execution. Expiry is Effection cancellation, so structured teardown completes before the failure is reported. A teardown failure prevents every later phase. `--timeout-exec` and `--timeout-fetch` configure the final document's effects @@ -507,7 +564,7 @@ Every failure below exits non-zero, and each one stops the phases after it: | the command document's authored `` | no save or run | | command document teardown | no final validation, save or run | | final validation of the approved bytes | no save or run | -| a `--save` path that exists, or a write that fails | no run | +| an `--output` path that exists, or a write that fails | no stdout, no run | | the document's own runtime failure | nothing after it; the save stands | ## Acceptance @@ -527,18 +584,18 @@ produced. | # | Criterion | Required observation | | --- | --- | --- | -| C1 | Fixed grammar and help | Request cardinality, individual-property ordering, aggregate props before the request, `--session` including its empty-value refusal, and effect-free generic help | +| C1 | Fixed grammar and help | Prompt cardinality, individual-property ordering, aggregate props before the Prompt, `--session` including its empty-value refusal, run-only flags refused without `--run` before any authorship or filesystem effect, and effect-free generic help that explains `--output` and `--run` | | C2 | Exact packaged root | The command executes the checked-in Markdown value root under ``; the turn text is that document's own words, and no TypeScript authorship loop or custom root chooses policy | -| C3 | Visible policy | Generation, three-turn repair, ten-round review, revision, approval, abort and exhaustion are present in Markdown; `` remains one turn | +| C3 | Visible policy | Generation, three-turn repair, ten-round review, revision, approval, stopping, exhaustion and the explanation turn are present in Markdown under visible headings; the generation, repair and revision instructions each require the descriptive title and the steps-beside-components structure; `` remains one turn | | C4 | One Session | One enclosing Session expansion carries every turn; two default invocations get different profile directories and session keys, two `--session` invocations get the same directory and key with the raw name absent from the path, and two named invocations sharing one ACPX store continue the established record rather than placing a second | | C5 | Prompt profile ceiling | This session's own host-owned directory, empty while the command document runs, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it; pre-existing content in that directory refuses before any provider, session, turn, review, save or execution and is left untouched | | C6 | Draft inertness | A draft is only data while the Plan is written, and no draft effect occurs before the final execution | | C7 | Validation classification | Candidate failures return structured facts; caller-source failures escape immediately; frozen signatures are checked before token extraction | | C8 | Bounds | One base plus three automatic repairs per draft, and no more than ten human presentations, with no revision offered on the last | -| C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; an ordinary abort and a tenth invalid presentation's sole `abort` reach their two distinct authored `` messages, not host policy and not a missing `` | +| C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; the review schemas expose exactly the friendly choices for each round-and-state; an ordinary **Stop**, a tenth-round exhaustion and the explanation ending reach their distinct authored `` messages, and the closing fallback says no approval was reached rather than repeating exhaustion | | C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no save, journal or execution — and resolves props for the exact returned bytes | -| C11 | Exact bytes | Approval, exclusive save and `` execution receive the Agent close value without rewriting or fence removal | -| C12 | Journal separation | Authorship uses only disposable in-memory history; the final journal begins with the approved Plan | -| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, save or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content or vanished under the conversation is preserved as found while the command fails terminally, with no admission, save, journal or execution after it | +| C11 | Exact bytes | Approval, stdout, the exclusive `--output` create and `` execution each receive the Agent close value without rewriting or fence removal; `--output --run` writes before it runs, and an existing destination prevents the run | +| C12 | Journal separation | Authorship uses only disposable in-memory history; a journal exists only when `--run` begins the final execution, and it begins with the approved Plan | +| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, stdout, the file or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content or vanished under the conversation is preserved as found while the command fails terminally, with no admission, result, journal or execution after it | | C14 | Narrative preservation | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, the assistant receives it with the request and the catalog, and a scripted Plan of prose interleaved with components returns byte for byte after approval | | C15 | Ordinary run | The approved source keeps normal cwd, includes, props, output/value, permission, timeout, failure and save behaviour | diff --git a/specs/root-document-props-spec.md b/specs/root-document-props-spec.md index 382e4929d..331032d91 100644 --- a/specs/root-document-props-spec.md +++ b/specs/root-document-props-spec.md @@ -384,8 +384,8 @@ any catalog, agent, elicitation, save, journal or document operation. The aggregate `--props` may be written before it, because its meaning never depends on a document. -`xmd prompt --help` describes the request, `--save`, `--session`, the aggregate -sources and where individual options go. It names no individual option and no +`xmd prompt --help` describes the Prompt, `--output`, `--run`, `--session`, the +aggregate sources and where individual options go. It names no individual option and no default: the document that would declare them does not exist yet, and generating one in order to describe it is not what help does. @@ -414,8 +414,9 @@ cannot decode, an extra positional a draft's arity exposes, or a signature chang repair, no review, no save and no run. That document cannot catch it and cannot recategorize it as feedback for an agent that could not have caused it. -The props the approved Plan runs with are resolved once more, after the command -document has completely torn down, from those exact returned bytes and the +The props the approved Plan is admitted with are resolved once more, after the +command document has completely torn down, from those exact returned bytes and +the original unchanged sources. No props object any draft produced is reused, so a revision that changed a property's declared type changes what the run receives. From 869a001fcdb6ec06dbcd06c5e5f0254a7a89adb5 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 17:00:44 -0400 Subject: [PATCH 10/15] =?UTF-8?q?=F0=9F=90=9B=20State=20the=20whole=20Plan?= =?UTF-8?q?=20requirement=20in=20every=20turn=20that=20asks=20for=20one=20?= =?UTF-8?q?(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A repair or a revision was told to "keep" the draft's title. That is the wrong instruction for a replacement: a draft whose title is missing, in the wrong place, or does not describe the Plan has nothing worth keeping, and an instruction to keep it asks the coding agent to carry the defect forward. Each Plan-producing turn — the first draft, each repair, each revision — now states the complete requirement on its own: optional frontmatter and then one descriptive level-one heading as the first body content, the Prompt's complete sequence as readable steps, every requested outcome, a meaningful order, and each component beside the prose describing what it does. Replacements are told to write the title the Plan needs rather than the one the last draft had. None of that reaches ``, `validateDocument()` or TypeScript. It is what the workflow asks for and what a person reviews, which is where a judgement about whether a title describes a Plan belongs. The prose under "Continue from your decision" claimed a final invalid draft left only one choice. It leaves two: ask the coding agent what went wrong, or stop. The governing documents still described the earlier command. They were swept deliberately rather than by replacing words: the opening example is a titled Plan, the flow shows the four destinations, and the failure table, timeout paragraph, acceptance rows, terminology rows, construct inventory, executable-MDX identity section, ACP lifecycle paragraph and root-props admission paragraph no longer say that an approved Plan always runs, that `--save` is the output option, that stopping is called abort, that admission is followed unconditionally by execution, or that every success writes a journal. The evidence follows the same rule. The approved introduction is pinned whole, punctuation included, rather than sampled; the requirement block is asserted against the actual initial, repair and revision turns and counted three times in the shipped source; and the run-only preflight table covers every spelling, including `-V` and `--secret-detection`. The test-local `validateCandidate` is `checkDraft`, so a terminology sweep finds no survivor of the old name. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- architecture.md | 8 +- packages/cli/src/documents/prompt-command.md | 52 +++++++++--- packages/cli/tests/packaged-document.test.ts | 59 +++++++++++-- packages/cli/tests/prompt-args.test.ts | 5 ++ .../cli/tests/prompt-command-document.test.ts | 2 +- packages/cli/tests/prompt.test.ts | 74 ++++++++++++++--- specs/acp-client-spec.md | 11 ++- specs/executable-mdx-spec.md | 39 +++++---- specs/prompt-command-spec.md | 83 +++++++++++-------- specs/root-document-props-spec.md | 12 +-- 10 files changed, 252 insertions(+), 93 deletions(-) diff --git a/architecture.md b/architecture.md index d9b42257a..e060a9dd0 100644 --- a/architecture.md +++ b/architecture.md @@ -34,9 +34,9 @@ Existing documents and code get aligned to this section retroactively. | pinned commit | the commit obtained by resolving a base once; it remains the workflow run's starting repository state even as the run creates descendant commits | | document target | an addressable static heading in a root document's own Markdown flow, named by the canonical path of heading labels that reaches it; selecting one executes the preamble, each ancestor's own content, and that heading's complete subtree | | Prompt | a person's original request, in ordinary natural language. `xmd prompt` takes exactly one | -| Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. A Plan is what `xmd prompt` approves, optionally saves and runs; it is not a synonym for a workflow, a policy document or any executable Markdown file | -| prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the request wording, the draft and repair loops, the check branches, human review, revision, approval, abort and exhaustion — and returns the exact approved Plan source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | -| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared draft validator. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | +| Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd prompt` approves and then delivers: printed to stdout by default, written to an `--output` file, and run only under `--run`. It is not a synonym for a workflow, a policy document or any executable Markdown file | +| prompt command document | the one exact checked-in first-party Markdown value root `xmd prompt` executes to convert a Prompt into a Plan. It implements that conversion and its review workflow — the Prompt wording, the draft and repair loops, the `` branches, human review, revision, approval, stopping, exhaustion and the final explanation turn — and returns the exact approved Plan source. Every Plan-producing turn in it states the complete Plan requirements for itself, so a replacement may add or correct a title rather than only carry one forward. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | +| prompt profile | the trusted-host assembly used only for the prompt command document: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared ``. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical prompt session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | | expansion | one logical evaluation of an authored executable element within a document execution | | expansion ID | a deterministic identifier for one logical expansion; restoring or retrying that expansion preserves the ID, while a distinct evaluation requested by the document receives another | | Git capability | the contextual interface through which workflow infrastructure queries the Git repository associated with the current working directory | @@ -3545,7 +3545,7 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd prompt` | turns one Prompt into a Plan and runs it, by executing two root documents with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, and then delivers the approved Plan where the caller asked: to stdout byte for byte by default, to an exclusively created `--output` path, and — only under `--run` — through the ordinary supplied-source path under the `` identity, one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A journal exists only when `--run` begins, and the flags that configure only a run are refused in preflight without it | built on the #659 stack | +| `xmd prompt` | turns one Prompt into a Plan and delivers it, by executing one root document — and, only under `--run`, a second — with a complete scope boundary between them. First the packaged prompt command document, under the internal `` identity, inside the prompt profile: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and one host-declared `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after that execution and every provider, Prompt task and Elicitation resource inside it has torn down, the host validates the returned Plan again, resolves props for exactly those bytes, and then delivers the approved Plan where the caller asked: to stdout byte for byte by default, to an exclusively created `--output` path, and — only under `--run` — through the ordinary supplied-source path under the `` identity, one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A journal exists only when `--run` begins, and the flags that configure only a run are refused in preflight without it | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | | `` | stops authored work with the sentence its author wrote, raised where it is written. An ordinary overridable core default — never structural, never reserved, so a repository `Fail.md` is chosen ahead of it — with a closed schema of one required non-empty `message` and **self-closing only**: a paired spelling never enters its body, and `as` is refused by the body itself because there is nothing to bind. Every refusal reports the invocation and happens before the authored message, so a document that never reached its decision is never reported as having made one. A valid invocation is the ordinary failure of a function component: an `Error` carrying the exact authored message, positioned at the opening tag, rendering nothing and binding nothing. It carries no `printErrors()` declaration, which is what leaves recovery to an authored `` region under ordinary text-root modes; a value body's `throw` is not replaced there, so the authored failure settles the body ahead of missing-`` settlement. No authority, context, provider, resource, module state or durable operation of its own: replay of a completed root restores the recorded outcome without re-expanding the body | built on the #659 stack | | `` | binds one name in the current environment from exactly one source: the content it renders, or the exact value `value` names, bound by reference and never through the JSON boundary component props cross — the scanner resolves no JSON for that one prop, and expansion projects none. Which source it has is read from what the author wrote, before either one runs, so a construct naming both expands no child and evaluates no expression. It opens no scope, owns no resource, adds no middleware boundary and writes no journal record — replay reconstructs both sources through ordinary expansion | built on the #527 stack | diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md index efc1c088a..e12a4671b 100644 --- a/packages/cli/src/documents/prompt-command.md +++ b/packages/cli/src/documents/prompt-command.md @@ -40,11 +40,17 @@ Create one complete XMD Plan from this Prompt: {props.request} -Begin the Plan with one level-one Markdown title that describes it. +Every Plan is complete on its own: -Write the Plan as a sequence of readable steps. Follow each step with the XMD -component that carries it out. The prose is part of the program and appears when -the Plan runs, so its source and its execution tell the same story. +- optional frontmatter, and then one descriptive level-one Markdown heading as + the first body content; +- the Prompt's complete sequence, written as readable steps; +- every outcome the Prompt asked for; +- those steps in an order that makes sense; and +- each XMD component beside the prose describing the action it performs. + +The prose is part of the program and appears when the Plan runs, so its source +and its execution tell the same story. For the Prompt "ask me for my age and write it to a file", the shape is: @@ -96,9 +102,20 @@ That Plan has problems. These are the exact ones: -Send one complete replacement Plan that resolves every problem above. Keep its -level-one title, keep the Prompt's sequence of readable steps in a meaningful -order, and keep each XMD component beside the step it performs. +Send one complete replacement Plan that resolves every problem above. + +Every Plan is complete on its own: + +- optional frontmatter, and then one descriptive level-one Markdown heading as + the first body content; +- the Prompt's complete sequence, written as readable steps; +- every outcome the Prompt asked for; +- those steps in an order that makes sense; and +- each XMD component beside the prose describing the action it performs. + +Write the title the Plan needs rather than the one the last draft had: add it if +it was missing, move it if it was not the first body content, and replace it if +it did not describe the Plan. Reply with the Plan source and nothing else. No enclosing code fence, no explanation before or after it. @@ -167,8 +184,8 @@ problems: Approve keeps this exact draft and leaves the review. Request changes sends your feedback to the coding agent and starts a new draft. Stop ends here. On a tenth -draft that still has problems there is nothing left to revise into, so the -remaining choice is to ask the coding agent what went wrong. +draft that still has problems there is nothing left to revise into, so the two +remaining choices are to ask the coding agent what went wrong, or to stop. @@ -205,9 +222,20 @@ You read that Plan and asked for this to change: {review.feedback} -Send one complete replacement Plan. Keep its level-one title, keep the Prompt's -sequence of readable steps in a meaningful order, and keep each XMD component -beside the step it performs. +Send one complete replacement Plan. + +Every Plan is complete on its own: + +- optional frontmatter, and then one descriptive level-one Markdown heading as + the first body content; +- the Prompt's complete sequence, written as readable steps; +- every outcome the Prompt asked for; +- those steps in an order that makes sense; and +- each XMD component beside the prose describing the action it performs. + +Write the title the Plan needs rather than the one the last draft had: add it if +it was missing, move it if it was not the first body content, and replace it if +it did not describe the Plan. Reply with the Plan source and nothing else. No enclosing code fence, no explanation before or after it. diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts index da4d2622c..2d037bbca 100644 --- a/packages/cli/tests/packaged-document.test.ts +++ b/packages/cli/tests/packaged-document.test.ts @@ -23,6 +23,44 @@ import { } from "../src/packaged-document.ts"; import { useWorkingDirectory } from "./support/prompt-harness.ts"; +/** + * The approved introduction, exactly. + * + * Pinned rather than sampled: this is the passage the product contract settled, + * down to "coding agent's plan", and a paraphrase that still contained the + * chosen sentences would pass a looser check while saying something else. + */ +const INTRODUCTION = [ + "# `xmd prompt` turns steps into a program", + "", + "This document is a workflow that generates an executable Plan from a sequence of", + "steps. It combines the original Prompt, which describes those steps, with the XMD", + "components available to carry them out. A coding agent turns both into one", + "document that explains and executes the sequence.", + "", + "The result is the XMD version of a coding agent's plan. A conventional Markdown", + "plan must be interpreted again before its steps can happen. An XMD Plan already", + "contains those executable steps, so running it simply executes them.", + "", + "A draft remains text while this workflow reviews it. Nothing in it runs before", + "you approve it. After approval, `xmd prompt` validates the exact source again. By", + "default it prints the approved XMD source. `--output` writes that source to a", + "file instead, and `--run` executes the Plan. With both options, the command", + "writes the source before running it.", +].join("\n"); + +/** What every turn that asks for a Plan has to say, on its own. */ +const PLAN_REQUIREMENTS = [ + "Every Plan is complete on its own:", + "", + "- optional frontmatter, and then one descriptive level-one Markdown heading as", + " the first body content;", + "- the Prompt's complete sequence, written as readable steps;", + "- every outcome the Prompt asked for;", + "- those steps in an order that makes sense; and", + "- each XMD component beside the prose describing the action it performs.", +].join("\n"); + describe("packaged documents", () => { it("reads the prompt command document from beside its module, whatever the cwd is", function* () { // A temporary contextual cwd with no document in it. A lookup that reached @@ -41,14 +79,25 @@ describe("packaged documents", () => { expect(source).toContain("returns:"); expect(source).toContain(" { // Each of them describes work that a command writing a Plan never does. A // caller who asked for a journal, a permission mode or an exec deadline and // got a command that creates none of them was not answered. + // Every spelling, including the short forms and both secret-detection + // switches: a table that covered only the long names would leave `-V` and + // `--secret-detection` accepted and ignored. for (const flag of [ ["--journal", "trace.jsonl"], ["-j", "trace.jsonl"], ["--raw"], ["--verbose"], + ["-V"], ["--timeout-exec", "5s"], ["--timeout-fetch", "5s"], ["--approve-all"], ["--approve-reads"], ["--deny-all"], + ["--secret-detection"], ["--no-secret-detection"], ]) { const refused = scanPromptArgs(["prompt", REQUEST, ...flag]); diff --git a/packages/cli/tests/prompt-command-document.test.ts b/packages/cli/tests/prompt-command-document.test.ts index 0d62aa955..452991e34 100644 --- a/packages/cli/tests/prompt-command-document.test.ts +++ b/packages/cli/tests/prompt-command-document.test.ts @@ -143,7 +143,7 @@ function* runDocument(): Operation { // program's control flow rather than validation's answers. factory: () => // deno-lint-ignore require-yield - function* validateCandidate(props: Record) { + function* checkDraft(props: Record) { validated.push(String(props.source)); return { valid: true, diagnostics: {} }; }, diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 0eb2a1edb..0d900f9e2 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -137,6 +137,25 @@ function command(dir: string, args: string[], stack: AgentStack = STACK): Prompt return { argv, scan: scanPromptArgs(argv), include: [dir], run: true, stack }; } +/** + * What every turn that asks for a Plan has to say, on its own. + * + * Pinned as one block rather than as sentences found anywhere in the document: a + * repair or a revision that quietly dropped a clause would still satisfy a + * search for the words somewhere, and a replacement Plan is only complete if the + * message asking for it said so. + */ +const PLAN_REQUIREMENTS = [ + "Every Plan is complete on its own:", + "", + "- optional frontmatter, and then one descriptive level-one Markdown heading as", + " the first body content;", + "- the Prompt's complete sequence, written as readable steps;", + "- every outcome the Prompt asked for;", + "- those steps in an order that makes sense; and", + "- each XMD component beside the prose describing the action it performs.", +].join("\n"); + /** Every session key the fake was asked to establish, deduplicated in order. */ function sessions(harness: PromptHarness): string[] { return [...new Set(harness.fake.ensured.map((input) => input.sessionKey))]; @@ -254,18 +273,14 @@ describe( // not produce them. const turn = harness.fake.prompts[0]; expect(turn).toContain("Create one complete XMD Plan from this Prompt"); - // C14: the shipped instruction is the narrative-plus-components rule. - expect(turn).toContain( - "Begin the Plan with one level-one Markdown title that describes it", - ); - expect(turn).toContain("Write the Plan as a sequence of readable steps"); - expect(turn).toContain("Follow each step with the XMD\ncomponent that carries it out"); - // The worked example travels with it, unparsed. + // C14: the complete Plan requirement, stated in this turn rather than + // referred to from somewhere else in the document. + expect(turn).toContain(PLAN_REQUIREMENTS); + // The worked example travels with it, unparsed, and is a titled Plan — + // the shape being asked for is the shape being shown. expect(turn).toContain('{answer.age}'); - expect(turn).toContain("keep every outcome it asked for"); - // The worked example is a titled Plan, so the shape being asked for is - // the shape being shown. expect(turn).toContain("# Ask for and save your age"); + expect(turn).toContain("keep every outcome it asked for"); // The request travels inside it, byte for byte. expect(turn).toContain(REQUEST); // So does the host's catalog, including the renderer's honest statement @@ -287,6 +302,45 @@ describe( }); }); + it("C3, C14: every turn that asks for a Plan states the whole requirement", function* () { + yield* useWorkingDirectory(function* (dir, profileRoot) { + const harness = createPromptHarness({ profileRoot }); + // A draft that fails its check, then one that passes, then a revision. + // Three turns, one of each kind that produces a Plan. + harness.fake.script({ reply: UNRESOLVED }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "Request changes", feedback: "say it differently" }); + harness.fake.script({ reply: VALID }); + harness.script({ decision: "Approve" }); + + const code = yield* runPrompt(command(dir, [REQUEST]), harness.deps); + expect(code).toBe(0); + + const [initial, repair, revision] = harness.fake.prompts; + expect(harness.fake.prompts).toHaveLength(3); + expect(initial).toContain("Create one complete XMD Plan from this Prompt"); + expect(repair).toContain("Send one complete replacement Plan that resolves every problem"); + expect(revision).toContain("say it differently"); + + // Each of them carries the requirement itself. A turn that only said + // "keep what the last draft had" would leave a missing or wrong title + // missing or wrong, and none of these do. + for (const turn of [initial, repair, revision]) { + expect(turn).toContain(PLAN_REQUIREMENTS); + } + + // And a replacement is told to write the title the Plan needs rather + // than to carry the previous one forward. + for (const turn of [repair, revision]) { + expect(turn).toContain( + "Write the title the Plan needs rather than the one the last draft had", + ); + expect(turn).toContain("add it if\nit was missing"); + expect(turn).not.toContain("Keep its\nlevel-one title"); + } + }); + }); + it("C4: one Session carries every turn, and --session names it", function* () { // Initial, repair and revision turns all land in one conversation, and a // second invocation places a different one in a directory of its own. diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 2d978276f..9dc7779b3 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -533,14 +533,17 @@ during its teardown. ### The `xmd prompt` prompt profile -`xmd prompt` resolves that configuration once and uses it twice: for the prompt -command document that writes the Plan, and for the Plan that runs. The provider +`xmd prompt` resolves that configuration once and uses it for the prompt command +document that writes the Plan, and — when `--run` asks for one — for the run that +follows. Without `--run` the approved Plan is written rather than executed, so +the second consumer never appears and the resolved configuration is used once. +The provider name, the default agent with its `DEFAULT_AGENT_NAME` precedence, the permission mode and the host's own machine-session assembly are settled before any catalog is built or any document executes, so an unknown provider or an incompatible pair of permission flags fails first. It is the settled value that reaches both consumers, not the flags that produced it: `DEFAULT_AGENT_NAME` is read once per -invocation, and authorship and the executed Plan cannot reach different +invocation, and authorship and a run of the Plan cannot reach different conclusions from one command line. The prompt profile takes the provider name and the default agent from that @@ -602,7 +605,7 @@ That document's scope closes before the final validation, the save and the execution. A teardown failure fails the command and no later phase happens. The executed program is an ordinary `xmd run` document with its own root provider and its own lifetime: it inherits neither the assistant session nor its instruction -layer. +layer. It exists only under `--run`. ### Availability diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index b6aa9e6a4..dad20f66e 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -8238,16 +8238,19 @@ durable root import, so the journal holds it and a replay restores it without reading anything. Text a host generated reports `` on the same terms. `xmd prompt` runs a -packaged first-party command document that turns a request into a Plan — asking -an agent for a complete root, having the host check it, and having a person -approve it; the host then validates the returned bytes again and executes -`retainedSource("", source)` — so the -identity says where the bytes came from, and a source position reads -`(:5:1)`. It is a deliberate identity rather than a path nobody could -read back, and it is the only thing that differs: the approved document runs -through the ordinary supplied-source path, renders and returns exactly as any -other root, resolves the contextual working directory for every relative -operation, and creates its journal only when execution starts. +packaged first-party command document that turns a Prompt into a Plan — asking a +coding agent for a complete root, having the host check it, and having a person +approve it; the host then validates the returned bytes again. What happens to +them is the caller's choice: by default the exact source goes to stdout, and +nothing runs; `--output` writes those bytes to a file instead; and only `--run` +executes `retainedSource("", source)`, with `--output --run` writing the +file first. So the identity says where the bytes came from, and a source position +reads `(:5:1)`. It is a deliberate identity rather than a path nobody +could read back, and it is the only thing that differs: an approved Plan that is +run goes through the ordinary supplied-source path, renders and returns exactly +as any other root, resolves the contextual working directory for every relative +operation, and creates its journal only when that execution starts — so the modes +that write a Plan create none. That command document is the other root the command executes, under its own stable internal identity ``, on an invocation-owned in-memory @@ -10042,7 +10045,9 @@ One checked-in Markdown suite runs the real command against a fixture directory. Two root executions with a scope boundary between them: the packaged prompt command document, which owns the visible generation, repair, review and failure -policy, and the Plan it returned, which runs as any supplied root does. Defined in +policy, and — only under `--run` — the Plan it returned, which runs as any +supplied root does. Without `--run` the approved Plan is the result: stdout by +default, or an exclusively created `--output` file. Defined in [`xmd prompt`](./prompt-command-spec.md), whose acceptance table this points at rather than restating. The ACPX runtime is a scriptable fake, the review provider is a scripted `Elicitation` handler, and the contextual working directory is a @@ -10051,12 +10056,12 @@ every refusal is proven by the phase tripwires that stayed at zero. | # | Test | Verify | |---|------|--------| -| C1 | Fixed grammar and help | One request preserved byte for byte; missing, repeated, empty and whitespace-only requests refused; individual options after the request and aggregate props before it; a built-in option never read as a generated property's value; `--session` named or refused; generic help with no effects | -| C2–C3 | The packaged document is the policy | The command executes the checked-in Markdown value root under ``, and the turn text is that document's own words; generation, repair, review, revision, approval, abort and exhaustion are Markdown, `` stays one turn, and what a person reads says each thing once however many rounds it took | -| C4–C6 | Session and ceiling | One enclosing Session carries every turn, defaults differ per invocation and `--session` supplies the exact override; the prompt profile gives the assistant an empty host-owned directory, no MCP servers, no native tools and a private strict denial no permission flag widens; a draft is data until the approved Plan runs | -| C7–C9 | Classification, bounds and presentation | Draft defects return structured facts and caller defects escape the validator; one base draft plus three repairs, and ten presentations with no revision on the last; arbitrary source cannot close ``, and abort reaches the document's own `` | -| C10–C13 | Admission and lifetime | The host revalidates after the command document has completely torn down and resolves props for the exact returned bytes; those bytes are created exclusively before execution; the final journal holds only the document's events; cancellation and teardown failure settle before anything later begins | -| C14–C15 | Result | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, and a Plan of prose interleaved with components survives approval and execution byte for byte; the run reports `` and behaves as `xmd run` does | +| C1 | Fixed grammar and help | One Prompt preserved byte for byte; missing, repeated, empty and whitespace-only Prompts refused; individual options after the Prompt and aggregate props before it; a built-in option never read as a generated property's value; `--session` named or refused; every run-only flag refused without `--run` before any effect; generic help with no effects, explaining `--output` and `--run` | +| C2–C3 | The packaged document is the policy | The command executes the checked-in Markdown value root under ``, and the turn text is that document's own words; generation, repair, review, revision, approval, stopping, exhaustion and the final explanation are Markdown under visible headings, every Plan-producing turn states the complete Plan requirements for itself, `` stays one turn, and what a person reads says each thing once however many rounds it took | +| C4–C6 | Session and ceiling | One enclosing Session carries every turn, defaults differ per invocation and `--session` supplies the exact override; the prompt profile gives the assistant an empty host-owned directory, no MCP servers, no native tools and a private strict denial no permission flag widens; a draft is data until the approved Plan is delivered, and no draft effect happens before that | +| C7–C9 | Classification, bounds and presentation | Draft defects return structured facts and caller defects escape the checker; one base draft plus three repairs, and ten presentations with no revision on the last; arbitrary source cannot close ``, the review schemas expose exactly the friendly choices for each round and state, and stopping, exhaustion and the explanation ending each reach their own authored `` | +| C10–C13 | Admission and lifetime | The host revalidates after the command document has completely torn down and resolves props for the exact returned bytes; those bytes reach exactly one of stdout, an exclusively created `--output` file, a run, or a file and then a run; a journal exists only when `--run` begins and holds only the Plan's events; cancellation and teardown failure settle before anything later begins | +| C14–C15 | Result | The shipped generation, repair and revision instructions each carry the complete titled-Plan rule, and a titled Plan of prose interleaved with components survives approval byte for byte into stdout, a file and a run alike; a run reports `` and behaves as `xmd run` does | ### Tier WB — The workflow component bundle in core diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 3c0781d65..528ce8112 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -10,10 +10,13 @@ $ xmd prompt "ask me for my age and write the result to a file" ``` A Plan is not merely executable code. It preserves the Prompt's intent as -reader-facing prose interleaved with the components that fulfil it, so the -request above should produce a document shaped like this: +reader-facing prose interleaved with the components that fulfil it, beginning +with one descriptive level-one heading, so the request above should produce a +document shaped like this: ```markdown +# Ask for and save your age + Ask me for my age. @@ -23,20 +26,28 @@ Write it to a file. {answer.age} ``` -and an execution organized by the same narrative: the first sentence, the -interaction, the second sentence, the file. The Prompt need not be copied +and an execution organized by the same narrative: the title, the first sentence, +the interaction, the second sentence, the file. The Prompt need not be copied literally — it may be divided, clarified and rewritten into natural prose — but -every requested outcome, their meaningful ordering, and their relationship to the -components performing them are preserved. +the descriptive title, every requested outcome, their meaningful ordering, and +their relationship to the components performing them are preserved. Every +Plan-producing turn — the first draft, each repair and each revision — states +that whole requirement for itself, so a replacement can add or correct a title +rather than only carry one forward. + +That command prints the approved Plan and runs nothing. `--output` writes it to a +file instead, `--run` executes it, and the two together write it and then run +it. The command writes no policy of its own. It executes two root documents with a complete scope boundary between them: first the **prompt command document**, a checked-in first-party Markdown value root that implements this conversion and its review workflow — what the assistant is asked, how many drafts may be repaired, what you are shown, and what happens when you approve nothing — and -then the Plan that document returned, through the same path `xmd run` uses for a -supplied one. The prompt command document is not itself a Plan. There is no -second execution model, no second props model and no second journal. +then, when `--run` asks for it, the Plan that document returned, through the same +path `xmd run` uses for a supplied one. The prompt command document is not itself +a Plan. There is no second execution model, no second props model and no second +journal. ## The flow @@ -45,17 +56,18 @@ fixed command preflight -> build the run-profile syntax catalog -> execute the exact packaged prompt command document -> one Session - -> generate, check, repair, review, revise, approve or fail + -> generate, check, repair, review, revise, approve, explain or fail -> Return the exact approved Plan source -> await that execution and provider teardown -> validate the returned source again - -> optionally save the exact bytes - -> execute retainedSource("", source) through the ordinary run path + -> --output: exclusively create the file with the exact bytes + -> --run: execute retainedSource("", source) through the ordinary run + path; otherwise, with no --output, write the exact bytes to stdout ``` Each phase hands the next one a value. No phase after the first failure begins, so a refused command line reaches no catalog, a failed turn reaches no review, -and an aborted review reaches no file and no run. +and a review that stopped reaches no stdout, no file and no run. Writing a Plan is a conversation, and a conversation is not a run: the prompt command document runs on an invocation-owned in-memory durable stream that is @@ -111,7 +123,7 @@ prompt profile's ceiling below. `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with `unrecognized option for xmd prompt: --eval — inline documents are exclusive to -xmd run`, before the catalog, the command document, the review, the save, the +xmd run`, before the catalog, the command document, the review, the file, the journal or any execution exists. Those options are resolved into one Agent configuration once per invocation — @@ -265,15 +277,15 @@ one of them has an identity worth keeping: call that could create it, so a leaf this invocation made and then failed on is still a leaf this invocation hands back. Exactly one cleanup is attempted, after the command document and every provider, Prompt task and Elicitation resource - inside it has torn down, and it settles before final validation, the save or - the execution begins. Success, abort, a failed turn and cancellation all reach + inside it has torn down, and it settles before final validation and every way a + Plan could leave the command. Success, stopping, a failed turn and cancellation all reach it. Once the directory has been established, it has exactly two outcomes: - **still empty** — that exact leaf is removed, non-recursively; - **anything else** — the directory and whatever is in it are preserved and the command fails terminally. A leaf that has gained content, and one that has disappeared, are both interference: something acted on a directory this host authorized nothing to touch. Neither is a warning and neither is a silent - skip, and no final admission, save or execution follows. + skip, and no final admission, stdout, file or execution follows. A directory establishment never handed over — refused as non-empty, or never created at all — is a different question, already answered by what @@ -308,13 +320,13 @@ within this invocation uses that one Session. A turn produces a draft only from its complete successful close value. The host decides, for the whole command document execution, that a failing `` ends it: a failed, cancelled, unavailable or protocol-invalid turn discards its -partial text and reaches no human review, save or final execution, and the +partial text and reaches no human review, result or final execution, and the document cannot opt out of that. That execution closes after approval or failure. The host observes its result only after every Prompt task, Agent provider resource, Elicitation resource and other child has completed teardown. A teardown failure wins over a selected Plan -and prevents final validation, save and execution. +and prevents final validation and every way a Plan could leave the command. ### A draft is data @@ -363,7 +375,7 @@ xmd prompt "greet someone" --props-name Ada --props-loud The aggregate `--props` may be written before the request, because its meaning never depends on a document. An individual `--props-*` option before the request -fails in preflight, before any catalog, Agent, elicitation, save, journal or +fails in preflight, before any catalog, Agent, elicitation, file, journal or document operation. The original argv is the command line source for every candidate. `XMD_PROPS` @@ -404,8 +416,8 @@ assessment answers `valid: false` and the command document may ask again: These are **terminal caller-source failures**. The caller wrote them, so they raise out of the validator and end the command document immediately, with no -repair turn, no presentation, no approval, no save, no final journal and no final -execution: +repair turn, no presentation, no approval, no stdout, no file, no final journal +and no final execution: - a supplied individual option the usable candidate schema does not declare; - malformed aggregate CLI or environment JSON; @@ -541,9 +553,10 @@ completed, and the Plan is there to hand-edit. ## Timeouts `--timeout` bounds the whole command: preflight, catalog construction, the -command document's execution, Elicitation, its teardown, final validation, the -file and the execution. Expiry is Effection cancellation, so structured teardown completes -before the failure is reported. A teardown failure prevents every later phase. +command document's execution, Elicitation, its teardown, final validation, and +whichever result the caller asked for — stdout, the file, the run, or the file +and then the run. Expiry is Effection cancellation, so structured teardown +completes before the failure is reported. A teardown failure prevents every later phase. `--timeout-exec` and `--timeout-fetch` configure the final document's effects only, exactly as under `xmd run`. Nothing bounds an authoring turn but the @@ -559,13 +572,13 @@ Every failure below exits non-zero, and each one stops the phases after it: | incompatible permission flags or an unknown `--agent-provider` | nothing | | a catalog an include makes unreadable | no command document | | a provider that cannot establish the prompt profile's ceiling | no session, no turn | -| a turn that did not complete | no review, save or run | -| a terminal caller-source failure | no repair, review, save or run | -| the command document's authored `` | no save or run | -| command document teardown | no final validation, save or run | -| final validation of the approved bytes | no save or run | +| a turn that did not complete | no review, stdout, file or run | +| a terminal caller-source failure | no repair, review, stdout, file or run | +| the command document's authored `` — stopping, exhaustion, or the ending after an explanation | no stdout, file or run | +| command document teardown | no final validation, stdout, file or run | +| final validation of the approved bytes | no stdout, file or run | | an `--output` path that exists, or a write that fails | no stdout, no run | -| the document's own runtime failure | nothing after it; the save stands | +| the Plan's own runtime failure under `--run` | nothing after it; the `--output` file stands | ## Acceptance @@ -588,14 +601,14 @@ produced. | C2 | Exact packaged root | The command executes the checked-in Markdown value root under ``; the turn text is that document's own words, and no TypeScript authorship loop or custom root chooses policy | | C3 | Visible policy | Generation, three-turn repair, ten-round review, revision, approval, stopping, exhaustion and the explanation turn are present in Markdown under visible headings; the generation, repair and revision instructions each require the descriptive title and the steps-beside-components structure; `` remains one turn | | C4 | One Session | One enclosing Session expansion carries every turn; two default invocations get different profile directories and session keys, two `--session` invocations get the same directory and key with the raw name absent from the path, and two named invocations sharing one ACPX store continue the established record rather than placing a second | -| C5 | Prompt profile ceiling | This session's own host-owned directory, empty while the command document runs, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it; pre-existing content in that directory refuses before any provider, session, turn, review, save or execution and is left untouched | +| C5 | Prompt profile ceiling | This session's own host-owned directory, empty while the command document runs, no MCP servers, no native tools, strict private denial, no Files/command/network capability for the command document, and final-run permission flags that cannot widen any of it; pre-existing content in that directory refuses before any provider, session, turn, review, result or execution and is left untouched | | C6 | Draft inertness | A draft is only data while the Plan is written, and no draft effect occurs before the final execution | | C7 | Validation classification | Candidate failures return structured facts; caller-source failures escape immediately; frozen signatures are checked before token extraction | | C8 | Bounds | One base plus three automatic repairs per draft, and no more than ten human presentations, with no revision offered on the last | | C9 | Safe presentation and authored failure | Arbitrary source cannot close ``; the review schemas expose exactly the friendly choices for each round-and-state; an ordinary **Stop**, a tenth-round exhaustion and the explanation ending reach their distinct authored `` messages, and the closing fallback says no approval was reached rather than repeating exhaustion | -| C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no save, journal or execution — and resolves props for the exact returned bytes | +| C10 | Final gate | The host revalidates after the command document has completely torn down — a component removed during that teardown makes the unchanged approved bytes fail admission, with no stdout, file, journal or execution — and resolves props for the exact returned bytes | | C11 | Exact bytes | Approval, stdout, the exclusive `--output` create and `` execution each receive the Agent close value without rewriting or fence removal; `--output --run` writes before it runs, and an existing destination prevents the run | | C12 | Journal separation | Authorship uses only disposable in-memory history; a journal exists only when `--run` begins the final execution, and it begins with the approved Plan | -| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, stdout, the file or execution; a default session's directory is empty while its turn runs and gone after teardown on success, abort, a failed turn and cancellation alike; and one that gained content or vanished under the conversation is preserved as found while the command fails terminally, with no admission, result, journal or execution after it | +| C13 | Lifetime | Cancellation and every teardown failure settle before final validation, stdout, the file or execution; a default session's directory is empty while its turn runs and gone after teardown on success, stopping, a failed turn and cancellation alike; and one that gained content or vanished under the conversation is preserved as found while the command fails terminally, with no admission, result, journal or execution after it | | C14 | Narrative preservation | The shipped generation, repair and revision instructions carry the narrative-plus-components rule, the assistant receives it with the request and the catalog, and a scripted Plan of prose interleaved with components returns byte for byte after approval | -| C15 | Ordinary run | The approved source keeps normal cwd, includes, props, output/value, permission, timeout, failure and save behaviour | +| C15 | Ordinary run | Under `--run` the approved source keeps normal cwd, includes, props, output/value, permission, timeout and failure behaviour, and an `--output` file written before it stands | diff --git a/specs/root-document-props-spec.md b/specs/root-document-props-spec.md index 331032d91..dab78b1bb 100644 --- a/specs/root-document-props-spec.md +++ b/specs/root-document-props-spec.md @@ -411,14 +411,16 @@ findings, and the command document may ask for a replacement. A defect the calle wrote — an option no draft declares, malformed aggregate JSON, a value the schema cannot decode, an extra positional a draft's arity exposes, or a signature change — raises out of that assessment instead, ending the command document with no -repair, no review, no save and no run. That document cannot catch it and cannot -recategorize it as feedback for an agent that could not have caused it. +repair, no review, and no approved Plan on stdout, in a file or in a run. That +document cannot catch it and cannot recategorize it as feedback for an agent that +could not have caused it. The props the approved Plan is admitted with are resolved once more, after the command document has completely torn down, from those exact returned bytes and -the -original unchanged sources. No props object any draft produced is reused, so a -revision that changed a property's declared type changes what the run receives. +the original unchanged sources. Admission happens in every mode — before stdout, +before an `--output` file and before a run alike — and a failure there produces +none of them. No props object any draft produced is reused, so a revision that +changed a property's declared type changes what a run receives. ## Targeted roots From 0a927b4e3d48bc19755d1cf5c2970756f101e531 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 17:10:25 -0400 Subject: [PATCH 11/15] =?UTF-8?q?=F0=9F=90=9B=20Use=20the=20approved=20apo?= =?UTF-8?q?strophe,=20and=20refuse=20--save=20by=20name=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The introduction claimed to be the approved wording and was one byte away from it: `coding agent's plan` with an ASCII apostrophe where the contract has U+2019. The regression that was supposed to catch that had copied the same byte, so it pinned the file to itself rather than the file to the contract. Both now carry E2 80 99, and the fixture's comment says why it is the byte and not the sentence that is being held. Two contract passages still described the earlier command. The ACP lifecycle said the profile's scope closes before "the save and the execution"; it closes before final admission and before whichever result was asked for. The command spec said `xmd prompt` executes two root documents; it always executes the command document and executes a second only under `--run`. The terminology sweep reached the comments, messages and test names that the earlier passes left behind: "the save" in the deadline and teardown comments, the harness's `--save`, the profile's two directory failures now saying "nothing was output or run", and the review-decision comments and test names that still called stopping an abort. The unrelated abort in `runDocument` is left alone — it is about a document's own failure, not this command's. `--save` had no real proof it was gone. The case that passed it also passed two mutually exclusive permission flags, so the earlier failure decided the outcome and the assertion would have held whether or not the option still worked. That case no longer carries it, and a focused one does — on a command line that is otherwise entirely valid. Making that case honest exposed something worth fixing. `--save out.md` was refused, but as "unrecognized argument: out.md": the scanner passed the unknown option to a parser that stops at the first option it does not define and drops the rest, so what a caller heard about was the value rather than the flag. Fixed preflight now refuses an option this command does not define, by name, and names `--output` when the option was `--save`. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/src/cli.ts | 2 +- packages/cli/src/documents/prompt-command.md | 2 +- packages/cli/src/prompt-args.ts | 19 ++++++++++ packages/cli/src/prompt-profile.ts | 8 ++-- packages/cli/src/prompt.ts | 6 +-- packages/cli/tests/packaged-document.test.ts | 8 ++-- packages/cli/tests/prompt-args.test.ts | 17 +++++++++ packages/cli/tests/prompt-cli.test.ts | 39 ++++++++++++++++---- packages/cli/tests/prompt.test.ts | 14 +++---- packages/cli/tests/support/prompt-harness.ts | 3 +- specs/acp-client-spec.md | 5 ++- specs/prompt-command-spec.md | 26 ++++++++----- 12 files changed, 109 insertions(+), 40 deletions(-) diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 5501c4f65..05baf6359 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -2176,7 +2176,7 @@ export function* runXmd( const selected = provisional.ok ? provisional.value.config : undefined; // The two commands that end in a document execution. `xmd prompt`'s deadline // encloses more than a run's — the catalog, the assistant session, every - // repair, the human review, provider teardown, the save and the execution — + // repair, the human review, provider teardown, the output file and the run — // because all of it is what the caller asked to be bounded. const executes = !helpRequest.requested && diff --git a/packages/cli/src/documents/prompt-command.md b/packages/cli/src/documents/prompt-command.md index e12a4671b..a107a6d0c 100644 --- a/packages/cli/src/documents/prompt-command.md +++ b/packages/cli/src/documents/prompt-command.md @@ -18,7 +18,7 @@ steps. It combines the original Prompt, which describes those steps, with the XM components available to carry them out. A coding agent turns both into one document that explains and executes the sequence. -The result is the XMD version of a coding agent's plan. A conventional Markdown +The result is the XMD version of a coding agent’s plan. A conventional Markdown plan must be interpreted again before its steps can happen. An XMD Plan already contains those executable steps, so running it simply executes them. diff --git a/packages/cli/src/prompt-args.ts b/packages/cli/src/prompt-args.ts index d30b00778..73e59a671 100644 --- a/packages/cli/src/prompt-args.ts +++ b/packages/cli/src/prompt-args.ts @@ -208,6 +208,25 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { continue; } + if (!KNOWN.has(name)) { + // The parser stops at the first option it does not define and drops the + // rest, so an option nobody defines would otherwise be accepted in + // silence — and a caller who asked for something the command never did + // has not been answered. `--save` is named because it is the one + // spelling somebody may remember; it was replaced before release, so + // there is no alias to keep. + return { + ...(request === undefined ? {} : { request }), + fixed, + occurrences, + error: + name === "--save" + ? `unrecognized option for xmd prompt: --save — the approved Plan goes to stdout, ` + + `and ${OUTPUT_OPTION} writes it to a file` + : `unrecognized option for xmd prompt: ${name}`, + }; + } + if (name === RUN_OPTION) { runs = true; } diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index bde862b8b..61179d7d6 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -389,8 +389,8 @@ function* establishDirectory(directory: string): Operation> { * empty and is handing back empty is removed; anything else is reported and * nothing is deleted, because a leaf that changed underneath a conversation * nobody authorized to write there is interference, not a tidying job. The - * failure raises out of the profile's scope, so no final validation, save or - * execution follows it. + * failure raises out of the profile's scope, so no final admission follows it, + * and the approved Plan reaches no stdout, no file and no run. * * A directory the conversation never got — establishment refused it, or never * made it — is a different question, and one already answered: whatever @@ -410,14 +410,14 @@ function* releaseSessionDirectory(directory: string, claim: DirectoryClaim): Ope throw new Error( `${directory} was made for this conversation and is already gone. Something removed ` + "it while the conversation was still running, which nothing here is allowed to do; " + - "nothing was saved or run", + "nothing was output or run", ); } if (code === "ENOTEMPTY" || code === "EEXIST") { throw new Error( `${directory} was empty when this conversation started and is not now. It belongs to ` + "one invocation, so nothing should have written there; its contents were left alone " + - "and nothing was saved or run", + "and nothing was output or run", ); } throw error instanceof Error ? error : new Error(String(error)); diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index 6fcb20472..fca4cf2d8 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -11,7 +11,7 @@ * -> execute the exact packaged prompt command document * -> await that execution and provider teardown * -> validate the returned source again - * -> optionally save the exact bytes + * -> deliver the exact bytes: stdout, an --output file, a --run, or both * -> execute retainedSource("", source) through the ordinary run path * ``` * @@ -199,8 +199,8 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op // The command document lives and dies inside that call's scope. Leaving it // closes the Prompt tasks, the provider and the Elicitation provider, so a - // teardown failure raises out here — before the validation, the save and the - // execution that would otherwise already have happened. + // teardown failure raises out here — before the admission, the output file + // and the run that would otherwise already have happened. let authored: Result; try { authored = yield* runPromptCommandDocument({ diff --git a/packages/cli/tests/packaged-document.test.ts b/packages/cli/tests/packaged-document.test.ts index 2d037bbca..8c8bb46b8 100644 --- a/packages/cli/tests/packaged-document.test.ts +++ b/packages/cli/tests/packaged-document.test.ts @@ -27,8 +27,10 @@ import { useWorkingDirectory } from "./support/prompt-harness.ts"; * The approved introduction, exactly. * * Pinned rather than sampled: this is the passage the product contract settled, - * down to "coding agent's plan", and a paraphrase that still contained the - * chosen sentences would pass a looser check while saying something else. + * down to the typographic apostrophe in "coding agent’s plan". A paraphrase that + * still contained the chosen sentences would pass a looser check while saying + * something else, and an ASCII apostrophe here would quietly redefine the + * approved wording around whatever the file happens to hold. */ const INTRODUCTION = [ "# `xmd prompt` turns steps into a program", @@ -38,7 +40,7 @@ const INTRODUCTION = [ "components available to carry them out. A coding agent turns both into one", "document that explains and executes the sequence.", "", - "The result is the XMD version of a coding agent's plan. A conventional Markdown", + "The result is the XMD version of a coding agent’s plan. A conventional Markdown", "plan must be interpreted again before its steps can happen. An XMD Plan already", "contains those executable steps, so running it simply executes them.", "", diff --git a/packages/cli/tests/prompt-args.test.ts b/packages/cli/tests/prompt-args.test.ts index 53c7a76af..352068852 100644 --- a/packages/cli/tests/prompt-args.test.ts +++ b/packages/cli/tests/prompt-args.test.ts @@ -269,6 +269,23 @@ describe("Tier PR — xmd prompt fixed grammar", () => { } }); + it("C1: an option this command does not define is refused, not dropped", function* () { + // The parser stops at the first option it does not define and drops the + // rest, so silence here would mean accepting a command line nobody honoured. + const unknown = scanPromptArgs(["prompt", REQUEST, "--not-a-thing", "value"]); + expect(unknown.error).toBe("unrecognized option for xmd prompt: --not-a-thing"); + + // `--save` was replaced before release, so there is no alias — and the + // refusal says where the Plan goes now rather than only that the option is + // unknown. + const retired = scanPromptArgs(["prompt", REQUEST, "--save", "out.md"]); + expect(retired.error).toContain("unrecognized option for xmd prompt: --save"); + expect(retired.error).toContain("goes to stdout"); + expect(retired.error).toContain("--output writes it to a file"); + // It is not quietly read as the option that replaced it. + expect(retired.occurrences).toEqual([]); + }); + it("C7: a frozen option's shape is what a later candidate may not change", function* () { const scalar = bindingsFor({ name: { type: "string" } }); const boolean = bindingsFor({ name: { type: "boolean" } }); diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 5efb064cb..17a0d2d04 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -192,7 +192,7 @@ describe( // the whole invocation shares now happens. yield* useWorkingDirectory(function* (dir) { const { code, stderr } = yield* runCli( - ["prompt", REQUEST, "--approve-all", "--deny-all", "--save", "out.md"], + ["prompt", REQUEST, "--approve-all", "--deny-all", "--run"], { cwd: dir }, ).join(); @@ -203,7 +203,30 @@ describe( // No provider was built: reaching one is what reports an agent as // unavailable, and this command line never got that far. expect(stderr).not.toContain("unavailable"); + }); + }); + + it("C1: --save is gone, and is refused as the unknown option it is", function* () { + // Nothing was released under the old spelling, so there is no alias and + // nothing to keep compatible with. Proven on a command line that is + // otherwise entirely valid: a case that also carried a second, earlier + // failure would pass whether or not `--save` still worked. + yield* useWorkingDirectory(function* (dir) { + const { code, stdout, stderr } = yield* runCli(["prompt", REQUEST, "--save", "out.md"], { + cwd: dir, + }).join(); + + expect(code).toBe(1); + expect(stderr).toContain("--save"); + // Not accepted, not silently dropped, and not mistaken for `--output`. expect(yield* exists(join(dir, "out.md"))).toBe(false); + // And nothing downstream of the refusal ran: no catalog was rendered, no + // provider was reached, nobody was asked anything, no approved source + // was printed, and no journal exists. + expect(stdout).toBe(""); + expect(stderr).not.toContain("## Built-in components"); + expect(stderr).not.toContain("unavailable"); + expect(yield* until(readdir(dir))).toEqual([]); }); }); @@ -333,10 +356,10 @@ describe( }); }); - it("C12: every refusal is nonzero and leaves no save, journal or run", function* () { + it("C12: every refusal is nonzero and leaves no output file, journal or run", function* () { const journalName = "trace.jsonl"; - // Abort at review, through the command document's authored failure. + // Stop at review, through the command document's authored failure. yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.deps.execute = executor(dir, join(dir, journalName)); @@ -517,7 +540,7 @@ describe( ); expect(code).toBe(0); - // The saved file already existed when the document started. + // The output file already existed when the document started. expect(seen).toEqual([true]); // Source only: no diagnostics, no decision, no wrapper. expect(yield* readTextFile(join(dir, "out.md"))).toBe(GREETER); @@ -570,12 +593,12 @@ describe( // The identity the run reports is the deliberate one. expect(yield* readTextFile(journal)).toContain(""); // Relative filesystem operations resolved the contextual cwd, not the - // identity, so the document's own write landed beside the save. + // identity, so the document's own write landed beside the output file. expect(yield* exists(join(dir, "greeting.txt"))).toBe(true); expect(yield* exists(join(dir, "out.md"))).toBe(true); }); - // A runtime failure is an ordinary run failure: nonzero, with the save the + // A runtime failure is an ordinary run failure: nonzero, with the file the // caller asked for still on disk to hand-edit. yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); @@ -622,7 +645,7 @@ describe( // and the person was asked once, and neither was asked again. expect(harness.fake.prompts).toHaveLength(1); expect(harness.reviews).toHaveLength(1); - // The save already happened, and a failing run leaves it to hand-edit. + // The file was already written, and a failing run leaves it to hand-edit. expect(yield* readTextFile(join(dir, "out.md"))).toBe(FAILING_TEST); }); }); @@ -692,7 +715,7 @@ describe( expect(yield* exists(join(dir, "out.md"))).toBe(false); }); - // A teardown failure prevents the final validation, the save and the run, + // A teardown failure prevents the final admission, the output file and the run, // whatever the command document selected. yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); diff --git a/packages/cli/tests/prompt.test.ts b/packages/cli/tests/prompt.test.ts index 0d900f9e2..fb6c7d2d7 100644 --- a/packages/cli/tests/prompt.test.ts +++ b/packages/cli/tests/prompt.test.ts @@ -446,7 +446,7 @@ describe( expect(harness.executions).toHaveLength(1); }); - // Abort, a turn that failed, and a cancelled command each hand the + // Stopping, a turn that failed, and a cancelled command each hand the // directory back the same way a success does. for (const ending of [ { name: "stop", drive: (harness: PromptHarness) => harness.script({ decision: "Stop" }) }, @@ -558,7 +558,7 @@ describe( expect(lines[0]).toContain("was made for this conversation and is already gone"); expect(lines[0]).not.toContain("does not validate"); // The Plan was approved and still reached nothing: no admission that - // could have saved it, no save, no execution and so no journal. + // could have delivered it, no output file, no execution and so no journal. expect(harness.reviews).toHaveLength(1); expect(yield* exists(join(dir, "out.md"))).toBe(false); expect(yield* until(readdir(dir))).toEqual([]); @@ -569,7 +569,7 @@ describe( // given empty and did not leave empty is preserved and the command fails: // something wrote there while the conversation ran, and this host // authorized nothing to. The draft was approved first, so what is being - // observed is a Plan that would otherwise have been saved and run. + // observed is a Plan that would otherwise have been written and run. yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); harness.fake.script({ reply: VALID }); @@ -602,7 +602,7 @@ describe( expect(yield* exists(String(workdir))).toBe(true); expect(yield* readTextFile(String(planted))).toBe("not this command's doing\n"); - // And nothing after the failure began: no save, and no execution — so no + // And nothing after the failure began: no output file, and no execution — so no // journal, which only an execution creates. expect(yield* exists(join(dir, "out.md"))).toBe(false); expect(yield* until(readdir(dir))).toEqual([]); @@ -993,7 +993,7 @@ describe( }); }); - it("C9: arbitrary source cannot close the presentation, and abort is authored", function* () { + it("C9: arbitrary source cannot close the presentation, and stopping is authored", function* () { yield* useWorkingDirectory(function* (dir, profileRoot) { // A document that holds a fence of its own, and a run of five backticks. const fenced = [ @@ -1021,7 +1021,7 @@ describe( expect(message).toContain(`\n${fenced}\n\`\`\`\`\`\``); }); - // Abort reaches the command document's own ``, with the message the + // Stop reaches the command document's own ``, with the message the // shipped Markdown wrote. Nothing about it is host policy. yield* useWorkingDirectory(function* (dir, profileRoot) { const harness = createPromptHarness({ profileRoot }); @@ -1040,7 +1040,7 @@ describe( }); // Exhaustion is reachable, and it is a different ending. Ten presentations - // that never validated, the last offering nothing but `abort`, and the + // that never validated, the last offering nothing but a way out, and the // sentence says there was never a Plan to approve rather than that // somebody decided to stop. yield* useWorkingDirectory(function* (dir, profileRoot) { diff --git a/packages/cli/tests/support/prompt-harness.ts b/packages/cli/tests/support/prompt-harness.ts index becd62beb..59b7d6303 100644 --- a/packages/cli/tests/support/prompt-harness.ts +++ b/packages/cli/tests/support/prompt-harness.ts @@ -143,7 +143,8 @@ export function createPromptHarness(options: { * A temporary directory that is also the contextual working directory. * * Both, because the two answer different questions: session placement and - * `--save` resolve the contextual one, while a real file has to live somewhere. + * `--output` resolve the contextual one, while a real file has to live + * somewhere. */ export function* useWorkingDirectory( body: (dir: string, profileRoot: string) => Operation, diff --git a/specs/acp-client-spec.md b/specs/acp-client-spec.md index 9dc7779b3..dcec3b24d 100644 --- a/specs/acp-client-spec.md +++ b/specs/acp-client-spec.md @@ -601,8 +601,9 @@ the command places a name unique to the invocation, so each invocation is a fres conversation created under those instructions; `--session ` selects an existing one under ordinary continuation semantics. -That document's scope closes before the final validation, the save and the -execution. A teardown failure fails the command and no later phase happens. The +That document's scope closes before the final admission and before whichever +result the caller asked for: the source on stdout, the `--output` file, the run, +or the file followed by the run. A teardown failure fails the command and no later phase happens. The executed program is an ordinary `xmd run` document with its own root provider and its own lifetime: it inherits neither the assistant session nor its instruction layer. It exists only under `--run`. diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 528ce8112..330f13810 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -39,15 +39,15 @@ That command prints the approved Plan and runs nothing. `--output` writes it to file instead, `--run` executes it, and the two together write it and then run it. -The command writes no policy of its own. It executes two root documents with a -complete scope boundary between them: first the **prompt command document**, a -checked-in first-party Markdown value root that implements this conversion and -its review workflow — what the assistant is asked, how many drafts may be -repaired, what you are shown, and what happens when you approve nothing — and -then, when `--run` asks for it, the Plan that document returned, through the same -path `xmd run` uses for a supplied one. The prompt command document is not itself -a Plan. There is no second execution model, no second props model and no second -journal. +The command writes no policy of its own. It always executes one root document: +the **prompt command document**, a checked-in first-party Markdown value root +that implements this conversion and its review workflow — what the coding agent +is asked, how many drafts may be repaired, what you are shown, and what happens +when you approve nothing. It executes a second root only under `--run`: the Plan +that document returned, through the same path `xmd run` uses for a supplied one, +with a complete scope boundary between the two. The prompt command document is +not itself a Plan. There is no second execution model, no second props model and +no second journal. ## The flow @@ -120,6 +120,12 @@ The permission flags configure only the approved Plan; authorship does not inherit the final run's permission mode, and no permission flag widens the prompt profile's ceiling below. +An option this command does not define is refused by name in the same preflight, +because the ordinary parser stops at the first option it does not recognize and +drops the rest — accepting a command line nobody honoured. `--save` is named in +particular: it was replaced by `--output` before release, so there is no alias, +and the refusal says where an approved Plan goes now. + `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with `unrecognized option for xmd prompt: --eval — inline documents are exclusive to @@ -568,7 +574,7 @@ Every failure below exits non-zero, and each one stops the phases after it: | Failure | Reaches | | --- | --- | -| a malformed command line | nothing | +| a malformed command line, an unknown option, or `--save` | nothing | | incompatible permission flags or an unknown `--agent-provider` | nothing | | a catalog an include makes unreadable | no command document | | a provider that cannot establish the prompt profile's ceiling | no session, no turn | From 142272da3a9435a88c4aded1af39490d9e1a4146 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:26:49 -0400 Subject: [PATCH 12/15] =?UTF-8?q?=F0=9F=A4=96=20Say=20that=20the=20second?= =?UTF-8?q?=20root=20execution=20is=20optional=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The module contract opened by claiming `xmd prompt` owns two root document executions, and its flow diagram ended by running the second one unconditionally — one line after the line listing stdout, an `--output` file, a run, or both as the four ways the bytes are delivered. A reader following the diagram would conclude that a default invocation runs the Plan it just printed. Every invocation executes the packaged prompt command document. A second root is executed only under `--run`. The diagram now shows the four deliveries as the alternatives they are, and the paragraph after it puts the complete scope boundary where it belongs: before that optional second execution, so whatever result follows has already let go of the conversation that wrote it. The same claim opened the prompt profile's header and the Tier PR introduction, and three nearby comments still had execution as the thing that follows approval rather than one of the things that might. Corrected to the approved bytes, or to the run that may follow. Prose only: no runtime behaviour, no tests and no evidence changed. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/src/cli.ts | 4 ++-- packages/cli/src/prompt-profile.ts | 12 ++++++------ packages/cli/src/prompt.ts | 26 ++++++++++++++++++-------- specs/executable-mdx-spec.md | 11 ++++++----- 4 files changed, 32 insertions(+), 21 deletions(-) diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 05baf6359..ddb2e6132 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -1936,8 +1936,8 @@ function* dispatch( yield* exit(1); break; } - // Once, here, and handed to both consumers below. Authorship and the - // execution that follows it are one invocation, so they answer to one + // Once, here, and handed to both consumers below. Authorship and the run + // that may follow it are one invocation, so they answer to one // `--default-agent`, one `DEFAULT_AGENT_NAME` and one permission mode. const promptStack = yield* settleAgentStack( { diff --git a/packages/cli/src/prompt-profile.ts b/packages/cli/src/prompt-profile.ts index 61179d7d6..822a9eed5 100644 --- a/packages/cli/src/prompt-profile.ts +++ b/packages/cli/src/prompt-profile.ts @@ -3,12 +3,12 @@ * runs under, and the only thing that ever runs under it * (specs/prompt-command-spec.md). * - * `xmd prompt` owns two root document executions with a complete scope boundary - * between them. This module is the first one. It supplies that document's - * inputs, a constrained Agent provider, Elicitation, the fixed first-party - * components and the host-declared draft validator, and it exposes no custom - * root and no repository component search: the document it runs is the one the - * CLI ships. + * `xmd prompt` executes this root on every invocation, and a second one — the + * Plan it returns — only under `--run`, behind a complete scope boundary. This + * module is the one that always happens. It supplies that document's inputs, a + * constrained Agent provider, Elicitation, the fixed first-party components and + * the host-declared draft checker, and it exposes no custom root and no + * repository component search: the document it runs is the one the CLI ships. * * The Agent ceiling is assembled here rather than read from the command line, * because it is not the caller's to choose. Writing a Plan is a conversation diff --git a/packages/cli/src/prompt.ts b/packages/cli/src/prompt.ts index fca4cf2d8..5ec577321 100644 --- a/packages/cli/src/prompt.ts +++ b/packages/cli/src/prompt.ts @@ -2,8 +2,8 @@ * `xmd prompt` — the trusted host around the prompt command document * (specs/prompt-command-spec.md). * - * The command owns two root document executions with a complete scope boundary - * between them: + * Every invocation executes one root document — the packaged prompt command + * document — and a second one only when `--run` asks for it: * * ```text * fixed command preflight @@ -11,17 +11,26 @@ * -> execute the exact packaged prompt command document * -> await that execution and provider teardown * -> validate the returned source again - * -> deliver the exact bytes: stdout, an --output file, a --run, or both - * -> execute retainedSource("", source) through the ordinary run path + * -> deliver those exact bytes, in exactly one of four ways: + * (default) write the source to stdout + * --output exclusively create the file + * --run execute retainedSource("", source) + * through the ordinary run path + * --output --run create the file, then execute it * ``` * + * The complete scope boundary sits before that optional second execution: the + * command document and everything it built are gone before a Plan is admitted, + * so whichever result follows, it follows an invocation that has already let go + * of the conversation that wrote it. + * * What a person is asked, how many drafts may be repaired, how many may be * reviewed and what happens when nobody approves anything are not here. They are * in `src/documents/prompt-command.md`, written in the open, where they can be * read and argued with. This module is what a policy cannot be trusted to do for * itself: settle the command line, build the ceiling the assistant runs under, * answer honestly about a draft, and hold the boundary between text an agent - * wrote and a program this host runs. + * wrote and a Plan this host will hand over or run. * * Two kinds of failure are told apart throughout, because they have different * remedies. A *draft* failure is something the agent wrote, so the prompt @@ -110,7 +119,7 @@ export interface PromptCommand { /** * The Agent configuration this invocation settled, before the command ran. * - * Settled by the caller rather than here, because the execution that follows + * Settled by the caller rather than here, because the run that may follow * approval is configured from the same answer: two resolutions of one command * line is two chances to read `DEFAULT_AGENT_NAME` differently. */ @@ -193,8 +202,9 @@ export function* runPrompt(command: PromptCommand, deps: PromptDependencies): Op // Every supplied individual option's shape, as the first draft that bound it // declared it. Frozen while the Plan is being written and carried into the - // final gate, so the bytes that run are checked against the command line that - // was written rather than against whichever draft happened to be last. + // final gate, so the bytes that are delivered are checked against the command + // line that was written rather than against whichever draft happened to be + // last. const frozen = new Map(); // The command document lives and dies inside that call's scope. Leaving it diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index dad20f66e..990a9c669 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -10043,11 +10043,12 @@ One checked-in Markdown suite runs the real command against a fixture directory. ### Tier PR — The `xmd prompt` command -Two root executions with a scope boundary between them: the packaged prompt -command document, which owns the visible generation, repair, review and failure -policy, and — only under `--run` — the Plan it returned, which runs as any -supplied root does. Without `--run` the approved Plan is the result: stdout by -default, or an exclusively created `--output` file. Defined in +One root execution on every invocation — the packaged prompt command document, +which owns the visible generation, repair, review and failure policy — and a +second only under `--run`: the Plan it returned, which runs as any supplied root +does, behind a complete scope boundary. Without `--run` the approved Plan is the +result rather than a second execution: stdout by default, or an exclusively +created `--output` file. Defined in [`xmd prompt`](./prompt-command-spec.md), whose acceptance table this points at rather than restating. The ACPX runtime is a scriptable fake, the review provider is a scripted `Elicitation` handler, and the contextual working directory is a From 944c734861a0ba553c8247214c10ba6a0e051083 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:50:22 -0400 Subject: [PATCH 13/15] =?UTF-8?q?=F0=9F=90=9B=20Refuse=20a=20valued=20--ru?= =?UTF-8?q?n=20instead=20of=20reading=20it=20as=20the=20switch=20(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An option's name is read up to its first `=`, so `--run=false` arrived under the name of the switch and was taken for it. That set the flag saying execution had been requested, which is also what satisfies the gate refusing options that only configure a run — so `--run=false --journal trace.jsonl` was accepted. What followed is worth stating precisely, because it is not what the flag being set suggests. The ordinary parser reads `--run=false` as false and `--run=true` as the field's default, which is also false. So the command took the write path either way: `--run=false --journal ` completed successfully, created no journal, and never told the caller that the journal they asked for was not a thing this invocation would produce; and `--run=true` was silently answered by a command that wrote the Plan instead of running it. One half of the command believed a run had been requested and the other half did not, and neither told anybody. Every valued spelling — `--run=false`, `--run=true`, `--run=` — now fails fixed preflight, before it establishes that execution was requested and before it can answer for the gate. Nothing is loaded, built, contacted, opened, created or executed. The regressions fail without it: the scanner cases find no error, and the command-level case exits 0 where it now exits 1 — the old behaviour was a success, not an incidental failure, which is what makes the tripwires worth asserting. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/src/prompt-args.ts | 16 +++++++++ packages/cli/tests/prompt-args.test.ts | 30 ++++++++++++++++ packages/cli/tests/prompt-cli.test.ts | 47 ++++++++++++++++++++++++++ specs/prompt-command-spec.md | 17 +++++++++- 4 files changed, 109 insertions(+), 1 deletion(-) diff --git a/packages/cli/src/prompt-args.ts b/packages/cli/src/prompt-args.ts index 73e59a671..d370a2016 100644 --- a/packages/cli/src/prompt-args.ts +++ b/packages/cli/src/prompt-args.ts @@ -228,6 +228,22 @@ export function scanPromptArgs(args: readonly string[]): PromptScan { } if (name === RUN_OPTION) { + // `optionName` stops at the first `=`, so every `--run=…` spelling + // arrives here under the name of the switch. Reading one as the switch + // would let a token satisfy the gate below while meaning the opposite, + // and the ordinary parser reads `--run=true` as the default rather than + // as true — so a caller who spelled it that way is answered by neither + // half of this command. Refused instead, before it establishes anything. + if (equals !== -1) { + return { + ...(request === undefined ? {} : { request }), + fixed, + occurrences, + error: + `${RUN_OPTION} does not take a value — write ${RUN_OPTION} to execute the Plan ` + + "or leave it out to write the Plan", + }; + } runs = true; } if (runOnly === undefined && RUN_ONLY.has(name)) { diff --git a/packages/cli/tests/prompt-args.test.ts b/packages/cli/tests/prompt-args.test.ts index 352068852..97fa101cf 100644 --- a/packages/cli/tests/prompt-args.test.ts +++ b/packages/cli/tests/prompt-args.test.ts @@ -269,6 +269,36 @@ describe("Tier PR — xmd prompt fixed grammar", () => { } }); + it("C1: --run is a switch, and every valued spelling of it is refused", function* () { + // An option name is read up to its first `=`, so `--run=false` arrives under + // the name of the switch. Taken as the switch it would establish the + // opposite of what was written, and satisfy the run-only gate on the way. + const REFUSAL = + "--run does not take a value — write --run to execute the Plan " + + "or leave it out to write the Plan"; + + for (const spelling of ["--run=false", "--run=true", "--run="]) { + const scan = scanPromptArgs(["prompt", REQUEST, spelling]); + expect(scan.error).toBe(REFUSAL); + // It established nothing: the token reached neither the parser's argv nor + // the record of what this invocation asked for. + expect(scan.fixed).toEqual(["prompt", REQUEST]); + + // And it does not answer for `--run` where a run is what makes an option + // meaningful. Without the fix this command line is accepted, and then + // nothing runs — so the journal the caller asked for is never created. + const gated = scanPromptArgs(["prompt", REQUEST, spelling, "--journal", "trace.jsonl"]); + expect(gated.error).toBe(REFUSAL); + expect(gated.fixed).toEqual(["prompt", REQUEST]); + } + + // The switch itself is unaffected, wherever it is written. + expect(scanPromptArgs(["prompt", REQUEST, "--run"]).error).toBe(undefined); + expect(scanPromptArgs(["prompt", REQUEST, "--run", "--journal", "t.jsonl"]).error).toBe( + undefined, + ); + }); + it("C1: an option this command does not define is refused, not dropped", function* () { // The parser stops at the first option it does not define and drops the // rest, so silence here would mean accepting a command line nobody honoured. diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index 17a0d2d04..e66baa02a 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -206,6 +206,53 @@ describe( }); }); + it("C1: --run=false reaches no authorship and no durable effect", function* () { + // `--run=false` used to be read as the switch, which satisfied the gate + // that only exists because a run is what makes `--journal` mean anything. + // The command line was then accepted, nothing ran, and the journal the + // caller asked for was never created — so the gate answered a request it + // had not honoured. Nonzero alone would not catch that: the invocation + // below would have failed anyway, on the agent it cannot reach. + yield* useWorkingDirectory(function* (dir, profileRoot) { + const journal = join(dir, "trace.jsonl"); + const harness = createPromptHarness({ profileRoot }); + harness.deps.execute = executor(dir, journal); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "Approve" }); + + const argv = ["prompt", REQUEST, "--run=false", "--journal", journal]; + const written = console.error; + const lines: string[] = []; + const value = yield* scoped(function* (): Operation { + yield* ensure(() => { + console.error = written; + }); + console.error = (...parts: unknown[]) => { + lines.push(parts.map((part) => String(part)).join(" ")); + }; + return yield* runPrompt( + { argv, scan: scanPromptArgs(argv), include: [dir], run: false, stack: STACK }, + harness.deps, + ); + }); + + expect(value).toBe(1); + // The refusal is the one the fixed grammar owes this command line, not + // an incidental failure further along. + expect(lines).toEqual([ + "--run does not take a value — write --run to execute the Plan " + + "or leave it out to write the Plan", + ]); + // Every phase after preflight stayed at zero: no catalog, no provider, + // no session, no turn, no review, no execution. + expect(untouched(harness)).toEqual(NOTHING); + // And nothing durable exists — neither the journal it named nor + // anything else. + expect(yield* exists(journal)).toBe(false); + expect(yield* until(readdir(dir))).toEqual([]); + }); + }); + it("C1: --save is gone, and is refused as the unknown option it is", function* () { // Nothing was released under the old spelling, so there is no alias and // nothing to keep compatible with. Proven on a command line that is diff --git a/specs/prompt-command-spec.md b/specs/prompt-command-spec.md index 330f13810..e35aa57cc 100644 --- a/specs/prompt-command-spec.md +++ b/specs/prompt-command-spec.md @@ -126,6 +126,21 @@ drops the rest — accepting a command line nobody honoured. `--save` is named i particular: it was replaced by `--output` before release, so there is no alias, and the refusal says where an approved Plan goes now. +`--run` is a switch, and every valued spelling of it — `--run=false`, +`--run=true`, `--run=` — is refused there too: + +```console +$ xmd prompt "…" --run=false +--run does not take a value — write --run to execute the Plan or leave it out to +write the Plan +``` + +An option's name is read up to its first `=`, so such a token arrives under the +name of the switch. Taken as the switch it would establish the opposite of what +was written and satisfy the run-only gate on the way, which is how +`--run=false --journal ` came to be accepted by a command that then created +no journal. It establishes nothing and reaches no later phase. + `-e`/`--eval` stays exclusive to `xmd run`. A prompt supplies a request, not a document. Supplying one anyway is refused in the command's own preflight, with `unrecognized option for xmd prompt: --eval — inline documents are exclusive to @@ -574,7 +589,7 @@ Every failure below exits non-zero, and each one stops the phases after it: | Failure | Reaches | | --- | --- | -| a malformed command line, an unknown option, or `--save` | nothing | +| a malformed command line, an unknown option, `--save`, or a valued `--run=…` | nothing | | incompatible permission flags or an unknown `--agent-provider` | nothing | | a catalog an include makes unreadable | no command document | | a provider that cannot establish the prompt profile's ceiling | no session, no turn | From 1e782a83c8cb3b8022efa010e987e5e073a6664f Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:54:05 -0400 Subject: [PATCH 14/15] =?UTF-8?q?=F0=9F=90=9B=20Assert=20the=20session=20a?= =?UTF-8?q?nd=20output-file=20tripwires=20on=20the=20--run=3D=20refusal=20?= =?UTF-8?q?(#260)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The refusal has to prove authorship never began, and the two tripwires that say so most directly were implied rather than asserted. A provider count of zero already rules out an established session, but the claim worth making is the narrow one: no session was established, and no directory was made for one to run in. The absence of an output file was likewise folded into an empty working directory rather than stated. Evidence only: no behaviour, no message and no other case changed. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/tests/prompt-cli.test.ts | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index e66baa02a..f3e95101f 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -244,10 +244,14 @@ describe( "or leave it out to write the Plan", ]); // Every phase after preflight stayed at zero: no catalog, no provider, - // no session, no turn, no review, no execution. + // no turn, no review, no execution. expect(untouched(harness)).toEqual(NOTHING); - // And nothing durable exists — neither the journal it named nor - // anything else. + // No prompt-profile session was opened either — neither established + // with the provider nor given a directory to run in. + expect(harness.fake.ensured).toEqual([]); + expect(yield* until(readdir(profileRoot))).toEqual([]); + // And nothing durable exists — neither the journal it named, nor an + // output file, nor anything else. expect(yield* exists(journal)).toBe(false); expect(yield* until(readdir(dir))).toEqual([]); }); From 7d0889fe2b9229bc85e40ccbced7fbec64d28203 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:57:00 -0400 Subject: [PATCH 15/15] =?UTF-8?q?=F0=9F=90=9B=20Pin=20the=20valued=20--run?= =?UTF-8?q?=20refusal=20at=20the=20parser=20boundary=20it=20crossed=20(#26?= =?UTF-8?q?0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The defect was a disagreement between two layers: the scanner read `--run=false` as the switch while the parser read it as false. The command-level regression for it hand-built the scanned result and called `runPrompt` directly, so it never reached `preparePropsPhase`, the parser or dispatch — the boundary the bug lived on. It would have passed with the two layers still disagreeing. A real invocation now covers it, for `--run=false`, `--run=true` and `--run=`: the exact refusal, empty stdout so no approved Plan escaped, no unavailable-agent error because reaching a provider is what produces one, and an empty working directory holding neither the journal it named nor anything else. Both existing cases stay, because the three prove different things. The scanner table is the grammar. This one is the boundary. The direct one names the phases that stayed at zero, which a subprocess cannot see. Without the refusal this case fails at `Expected: 1 / Received: 0`, and its stderr carries the unavailable-agent error — the command line reached a provider, which is the authorship this refusal exists to prevent. Evidence only: no production change. Claude-Session: https://claude.ai/code/session_015HcqB9kJM9KFnMNToAuZF6 --- packages/cli/tests/prompt-cli.test.ts | 36 +++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/packages/cli/tests/prompt-cli.test.ts b/packages/cli/tests/prompt-cli.test.ts index f3e95101f..2386b82b0 100644 --- a/packages/cli/tests/prompt-cli.test.ts +++ b/packages/cli/tests/prompt-cli.test.ts @@ -213,6 +213,11 @@ describe( // caller asked for was never created — so the gate answered a request it // had not honoured. Nonzero alone would not catch that: the invocation // below would have failed anyway, on the agent it cannot reach. + // + // This case names the phases that stayed at zero, which a subprocess + // cannot see. The one after it drives the real parser and dispatch, which + // this one does not reach — the defect lived exactly between those two + // layers, so both are needed to pin it. yield* useWorkingDirectory(function* (dir, profileRoot) { const journal = join(dir, "trace.jsonl"); const harness = createPromptHarness({ profileRoot }); @@ -257,6 +262,37 @@ describe( }); }); + it("C1: a valued --run is refused by the real parser, not just the scanner", function* () { + // The defect was a disagreement between the scanner and the parser: the + // scanner read `--run=false` as the switch while the parser read it as + // false. A case that hands `runPrompt` an already-scanned command skips + // the boundary the bug lived on, so this one goes through the command line + // an operator actually types. + const REFUSAL = + "--run does not take a value — write --run to execute the Plan " + + "or leave it out to write the Plan"; + + for (const spelling of ["--run=false", "--run=true", "--run="]) { + yield* useWorkingDirectory(function* (dir) { + const { code, stdout, stderr } = yield* runCli( + ["prompt", REQUEST, spelling, "--journal", "trace.jsonl"], + { cwd: dir }, + ).join(); + + expect(code).toBe(1); + expect(stderr).toContain(REFUSAL); + // No approved Plan escaped: stdout is where one would have gone. + expect(stdout).toBe(""); + // No provider was reached — reaching one is what reports an agent as + // unavailable, and this command line never got that far. + expect(stderr).not.toContain("unavailable"); + // And nothing was written: not the journal it named, not an output + // file, not anything else. + expect(yield* until(readdir(dir))).toEqual([]); + }); + } + }); + it("C1: --save is gone, and is refused as the unknown option it is", function* () { // Nothing was released under the old spelling, so there is no alias and // nothing to keep compatible with. Proven on a command line that is