simtools-tests

Versioned test resources and generated artifacts for simtools integration and science-validation workflows.

Resource bundles

Current bundles live below simtools-tests/<resource-version>/integration_tests/. Bundles below simtools-tests/legacy/ are historical and are not maintained by current CI.

Path	Lifecycle
config_files/	Versioned workflow inputs.
static/	Hand-maintained inputs; update static/static_manifest.yml with every change.
downloaded/	Inputs fetched from config_files/download_files.yml; regenerated from external GitLab URLs.
generated/	Workflow outputs collected as reference products.
run_time.yml	Container/runtime image, mounts, network, and environment-file configuration.

Science-test releases, when present, live in <resource-version>/science_tests/ and contain the release context and runner instructions for simtools-run-science-tests.

Do not commit tmp/, tmp_application_output/, or log_files/; the other directories are part of the versioned snapshot.

Generation requirements

Run generation from this repository root with the already configured simtools environment. The runtime in run_time.yml requires Podman or Docker and access to the configured image registry. Generation also needs network access to the GitLab URLs in download_files.yml. Simulation models are read from a filesystem repository or a local Git repository. Copy ../simtools/.env_template to .env and set the container paths SIMTOOLS_SIM_TELARRAY_PATH, SIMTOOLS_CORSIKA_PATH, and SIMTOOLS_CORSIKA_INTERACTION_TABLE_PATH, plus one of:

• SIMTOOLS_SIMULATION_MODELS_PATH for a checked-out simulation-model repository; or
• SIMTOOLS_SIMULATION_MODELS_GIT_PATH and SIMTOOLS_SIMULATION_MODELS_GIT_REVISION for a local Git repository.

Set SIMTOOLS_TESTS_PATH when the container needs to resolve versioned resources from this checkout. Keep .env out of commits.

Generate or regenerate

For a new bundle, replace the version values below. The target must not already exist; the template copies config_files/, static/, and run_time.yml, then generation creates downloaded/, generated/, and log_files/.

resource_version="vX.Y.Z"
template_version="vA.B.C"
runtime_file="simtools-tests/${template_version}/integration_tests/run_time.yml"
test ! -e "simtools-tests/${resource_version}"
simtools-resources-test-generate \
    --simtools_version "${resource_version}" \
    --template_version "${template_version}" \
    --test_directory . \
    --runtime_environment_file "${runtime_file}" \
    --overwrite_collection_files

For an existing bundle, review its run_time.yml first and rerun with:

resource_version="vX.Y.Z"
simtools-resources-test-generate \
    --simtools_version "${resource_version}" \
    --test_directory . \
    --runtime_environment_file "simtools-tests/${resource_version}/integration_tests/run_time.yml" \
    --overwrite_collection_files

A successful run exits 0, writes one log per workflow, and updates the collected outputs. Use --config_file path/to/workflow.config.yml to run one workflow or --download_only to fetch only external inputs.

Run simtools integration tests

From the simtools repository root, select a resource directory with --test_resources_path:

resource_version="vX.Y.Z"
pytest --no-cov -n auto --model_version=6.0.2 \
    --test_resources_path="../simtools-tests/simtools-tests/${resource_version}/integration_tests" \
    tests/integration_tests

Alternatively, select a versioned resource directory with --simtools_tests_resource_version (or SIMTOOLS_TESTS_RESOURCE_VERSION); the default is the resource-version in simtools/dependency_versions.yml. A full resource path can also be supplied through SIMTOOLS_TEST_RESOURCES.

When using a local model checkout, set SIMTOOLS_SIMULATION_MODELS_PATH. For a Git source, set SIMTOOLS_SIMULATION_MODELS_GIT_PATH and SIMTOOLS_SIMULATION_MODELS_GIT_REVISION instead; the two source types cannot be used together. This assumes sibling simtools and simtools-tests checkouts. See CONTRIBUTING.md and the RELEASING.md release guidance for project workflow.

Run simtools science tests

Science tests are longer-running release-validation workflows. A release directory contains a context template and workflow selection; copy the context example outside the repository, fill in the production paths, and run:

simtools-run-science-tests \
    --release_dir simtools-tests/vX.Y.Z/science_tests \
    --context_file /path/to/vX.Y.Z-context.yml \
    --dry_run

Remove --dry_run to execute the selected workflows. See the release directory's README.md for site and test-selection options.