Skip to content

Validate supplied document structure without executing it #653

Description

@taras

Motivation

xmd prompt (#260) must reject a generated program with a missing component or
an invalid component invocation before asking a person to approve it. Core's
current document inspection reads the root source and validates root metadata,
but it does not resolve components in the document body or validate their
declared props. Those failures are currently discovered only when execution
begins.

Provide one reusable core boundary that validates a supplied document without
executing it. The CLI can then use the same result for generated source, and
other hosts do not need to reproduce expansion rules.

Contract

The validation operation accepts the same declarative document context needed
to interpret a supplied root source: source text and identity, contextual cwd,
ordered includes, root props, the contextual component registry, and the plain
identity-component declarations the host supplies. It does not run an
ExecutionInstallation, call an identity factory, or install an operational
provider. This is the same declaration-only environment inspectSyntax() uses,
not a second registry or a reconstruction of the execution host.

Validation starts with the selected root projection, using the existing root
target selection and definition-validation order. It then follows normal
component selection through the recursive closure of resolved Markdown
component definitions. Each definition is parsed once by source identity, so a
cycle does not recurse forever, and every authored invocation site in those
sources is reported in source order. Validation does not interpret control-flow
reachability: an invocation inside a branch is still authored program structure.
Children authored beneath an origin-only TypeScript invocation remain part of
the containing source and are inspected even though validation cannot know
whether that component will project them.

The operation returns one versioned result:

interface DocumentValidation {
  readonly version: 1;
  readonly outcome: "valid" | "invalid";
  readonly diagnostics: readonly DocumentValidationDiagnostic[];
  readonly invocations: readonly InvocationValidation[];
}

interface InvocationSite {
  readonly name: string;
  readonly position?: Readonly<SourcePosition>;
}

type InvocationValidation = InvocationSite &
  (
    | {
        readonly outcome: "valid";
        readonly origin: Readonly<ComponentOrigin>;
      }
    | {
        readonly outcome: "invalid";
        readonly origin?: Readonly<ComponentOrigin>;
        readonly diagnosticIndexes: readonly number[];
      }
    | {
        readonly outcome: "not-statically-checkable";
        readonly origin: Readonly<ComponentOrigin>;
        readonly reasons: readonly (
          | "dynamic-props"
          | "origin-only-contract"
        )[];
      }
  );

type DocumentValidationCode =
  | "source-unreadable"
  | "source-invalid"
  | "target-invalid"
  | "frontmatter-invalid"
  | "props-declaration-invalid"
  | "returns-declaration-invalid"
  | "component-unresolved"
  | "component-ambiguous"
  | "invocation-form-invalid"
  | "body-shape-invalid"
  | "props-invalid"
  | "binding-invalid"
  | "capture-invalid"
  | "return-usage-invalid"
  | "structural-usage-invalid";

interface DocumentValidationDiagnostic {
  readonly code: DocumentValidationCode;
  readonly message: string;
  readonly position?: Readonly<SourcePosition>;
  readonly component?: string;
  readonly issues?: readonly NormalizedIssue[];
}

code is a closed set for version 1 rather than rendered CLI prose. A schema
diagnostic carries the same normalized AJV issues normal validation already
produces; consumers do not parse message to recover a property or keyword.
Sources are ordered by putting the root first, then processing a FIFO queue of
Markdown definitions in the order their first invocation is encountered. Within
one source, diagnostics and invocation records are ordered by source position;
multiple diagnostics at one position use the DocumentValidationCode order
shown above. An invocation's indexes point into that one ordered diagnostic
array. Root parse, target, frontmatter, props-declaration and return-declaration
failures have no invocation record when no invocation exists.
The document outcome is invalid exactly when the result contains a diagnostic
that represents a definite validation failure. An opaque invocation by itself
does not make the document invalid.

The version-1 diagnostic codes cover every condition core can establish before
execution:

  • source failures emitted by the parser normal execution shares;
  • invalid root or Markdown-component frontmatter, props declarations, and
    return declarations;
  • target-selection failures for a targeted supplied root;
  • unresolved or ambiguous component selection;
  • invocation-form and engine-owned body-shape violations;
  • missing required props when their absence is statically established;
  • schema violations when every schema-visible prop value is static; and
  • invalid as, capture, return, or structural usage that the shared expansion
    rules can decide without evaluating document code.

Validation uses the existing parser exactly as normal execution does. It does
not add a strict MDX grammar beside that parser. Text the shared scanner does not
recognize as an invocation remains text, and spread attributes retain their
current execution semantics. A future change that makes either one an error
changes parsing for validation and execution together.

Invocation outcomes

Validation applies checks whose answers do not depend on runtime values even
when another part of the invocation is opaque. A definite resolution, authored
form, engine-owned body-shape, or independently established structural failure
makes the invocation invalid.

Full props-schema validation runs only when every schema-visible prop value is
static. A dynamic schema-visible expression or origin-only TypeScript contract
that prevents the complete check makes an otherwise non-invalid invocation
not-statically-checkable. A declared capture keeps the same deliberate schema
bypass it has during execution and does not by itself make the invocation
opaque. Validation does not implement a partial JSON Schema solver to infer
additional failures from a mixture of static and dynamic prop values.

Outcome precedence is therefore:

  1. invalid when any no-execution check proves a failure;
  2. not-statically-checkable when no failure is proven but at least one
    applicable check needs runtime information; and
  3. valid only when all applicable checks are statically proven.

Repository TypeScript components whose catalog entry identifies only their
origin have no invented props, forms, captures, returns, or documentation
contract. Their invocation is not statically checkable unless a separate
engine-owned rule already proves it invalid.

No-execution boundary

Validation does not invoke a component, evaluate a document expression, render
or project content, start an agent, prompt or elicit, create or read a journal,
execute a command, call an identity factory, install an operational provider, or
perform a filesystem effect expressed by the document. It may read the supplied
root and the component-definition files normal selection identifies, just as
inspection does.

Component resolution, declaration admission, schemas, authored forms, body
rules, target selection, and source positions come from the same definitions as
normal execution and the structured syntax catalog from #632. The operation
must not introduce a second component registry, schema dialect, parser, or
handwritten catalog.

Acceptance criteria

  • A supplied document containing <DefinitelyMissing /> returns an unresolved
    component diagnostic without starting execution.
  • A supplied document containing <File /> without its required path returns
    a declaration/schema diagnostic without starting execution.
  • A valid document using built-in, registered, and included Markdown components
    passes validation.
  • A defect inside a recursively selected Markdown component is reported at that
    component source's authored position, including when the invocation is beneath
    authored control flow that validation does not evaluate.
  • Existing root-document parse, target, and declaration failures remain
    represented in the returned diagnostics.
  • A dynamic prop makes a schema-dependent invocation not statically checkable
    when no independent failure is certain; a definite form or body-shape failure
    on the same invocation still makes it invalid.
  • An origin-only TypeScript component is reported as not statically checkable
    where its invocation contract is unknown, rather than accepted under an
    invented schema or rejected merely for being opaque.
  • Repeated validation returns diagnostics and invocation records in the same
    documented order.
  • Add xmd prompt: generate an executable markdown program from a request, approve it, run it #260 can send the versioned diagnostics to an ACP generator and show them to a
    person without parsing rendered CLI prose.
  • Focused evidence proves that validation creates no execution, agent, journal,
    elicitation, command, identity-factory, provider, or document-authored
    filesystem effects.

Specification

Update architecture.md and specs/executable-mdx-spec.md to define this
non-executing validation boundary, its traversal and ordering, its version-1
diagnostics, and the distinction between invalid and not-statically-checkable
invocations. Spec, tests, and mechanics move together in the implementation PR.

Sequencing

Complete this core capability before implementing #260's verification and
approval loop. #632's syntax-catalog dependency is complete; #260 consumes this
result rather than defining another validator.

Out of scope

  • restricting generated programs to a safe component subset;
  • interpreting control-flow reachability;
  • partially solving JSON Schema across dynamic and static values;
  • proving dynamic JavaScript expressions or runtime-dependent prop values;
  • defining a stricter MDX grammar than normal execution uses;
  • executing a program to discover whether it succeeds; and
  • generation, repair, approval, saving, or execution behavior owned by Add xmd prompt: generate an executable markdown program from a request, approve it, run it #260.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions