Skip to content

feature datacontract mock datacontract.yaml --output ./samples to generate fake datasets to ingest to sample test data #1581

Description

@dmaresma

Feature Request: Test Data Generation for Data Contracts

I would like to see a new feature added to the datacontract-cli utility that can automatically generate mock or sample datasets based on the entities, attributes, and schema definitions contained within a data contract.

For example:

datacontract mock datacontract.yaml --output ./samples

The command could generate realistic test data artifacts from the contract definition, including:

SQL DML scripts using INSERT INTO ... VALUES (...)
CSV files
JSON files
NDJSON files (optional)
Parquet files (optional)

This capability would help data product teams quickly bootstrap development, validate schema implementations, test ingestion pipelines, and provide sample datasets to consumers before production data becomes available.

Ideally, the generated data would:

Respect data types, constraints, and nullability rules defined in the contract.
Support configurable record counts.
Generate realistic sample values for common business attributes (dates, emails, identifiers, addresses, phone numbers, etc.).
Preserve entity relationships and foreign key references when defined.
Allow deterministic generation through a seed parameter for repeatable testing.

Example:

datacontract mock datacontract.yaml
--rows 1000
--output ./samples

This feature would significantly improve developer experience and accelerate the adoption of Data Contracts by providing instantly consumable test datasets directly from the contract specification.

Version plus concise (GitHub Issue style):

Request: Add a mock command to datacontract-cli that generates synthetic datasets from a Data Contract definition.

Example:

datacontract mock datacontract.yaml --output ./samples

Expected outputs:

SQL (INSERT INTO ... VALUES)
CSV
JSON
Optional: Parquet, NDJSON

Benefits:

Faster development and testing
Sample datasets for data consumers
Schema validation
Pipeline and integration testing
Better onboarding and adoption of Data Contracts

The generated data should respect the contract's schema, data types, constraints, relationships, and optionally support configurable row counts and deterministic generation through a seed value.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestwaiting-for-decisionWaiting for a decision of the maintainers.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions