Skip to content
43 changes: 25 additions & 18 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,7 @@ This repository contains reusable GitHub Actions workflows for the `datasciencec

## Repository purpose

- Use this repository only for workflows that require `datasciencecampus` organization-scoped credentials or policies.
- Prefer `ONSdigital/ons-github-actions` for broadly reusable workflows that do not depend on this org boundary.

- Use this repository when a workflow needs `datasciencecampus` credentials or policies, or when public repositories must be able to call it.
## Security posture

- Changes in this repository should follow secure-by-design principles.
Expand All @@ -16,61 +14,70 @@ This repository contains reusable GitHub Actions workflows for the `datasciencec

## Workflow structure

- Public caller-facing workflows use the clean `add-*` names:
- Project-routing public caller-facing workflows use the clean `add-*` names:
- `.github/workflows/add-issue-to-projects.yml`
- `.github/workflows/add-pr-to-projects.yml`
- Internal privileged implementations use `*-impl` names:
- Other public reusable workflow entry points are:
- `.github/workflows/security-analysis.yml` (orchestrates zizmor and Checkov)
- `.github/workflows/terraform-quality.yml`
- Security-analysis child workflows are `.github/workflows/zizmor.yml` and `.github/workflows/checkov.yml`; callers should use the orchestrator so both tools share its trigger and policy configuration.
- Internal project-routing implementations use `*-impl` names:
- `.github/workflows/add-issue-to-projects-impl.yml`
- `.github/workflows/add-pr-to-projects-impl.yml`
- Reusable workflow tests use `*-reusable` names:
- Reusable workflow tests use `test-*-reusable` names:
- `.github/workflows/test-add-issue-to-projects-reusable.yml`
- `.github/workflows/test-add-pr-to-projects-reusable.yml`
- `.github/workflows/test-terraform-quality-reusable.yml`

Do not collapse the public and internal workflows back into one file unless the user explicitly asks for that architectural change.
Do not collapse the project-routing public and internal workflows back into one file unless the user explicitly asks for that architectural change.

## Public contract conventions

- The public `add-*` workflows are the API that other repositories consume via `uses:`.
- The internal `*-impl` workflows are dispatch-only and should not be documented as the primary caller entrypoint.
- If caller-facing inputs change, update all of these together:
- The public `add-*` workflows are the caller API for project routing. The internal `*-impl` workflows are dispatch-only and should not be documented as the caller entrypoint.
- `security-analysis.yml` and `terraform-quality.yml` are public `workflow_call` entry points. Their inputs and permission requirements belong in the reusable-workflow reference and their relevant how-to guides.
- If a caller-facing contract changes, update the relevant items together:
- `README.md`
- `docs/reference/reusable-workflows.md`
- relevant `docs/how-to/*.md`
- If the change is architectural rather than cosmetic, add or update an ADR under `docs/explanation/`.
- `docs/how-to/README.md` or `docs/README.md` when pages or navigation change
- If the change is architectural rather than cosmetic, add or update an ADR under `docs/explanation/adr/` and link it from `docs/explanation/adr/README.md`.

## Pinning and dispatch gotchas

- These dispatch and `implementation_ref` rules apply to the project-routing workflows, not to security-analysis or terraform-quality.
- `workflow_dispatch` requires a branch or tag ref, not a raw commit SHA.
- Public reusable workflows support SHA pinning at the `uses:` boundary, but internal dispatch still needs a branch or tag ref.
- If `implementation_ref` is omitted, the public reusable workflow first uses `github.workflow_ref` when that is already a branch or tag.
- For SHA-pinned callers, the public reusable workflow reads `configs/implementation-ref.json` from the pinned workflow revision, takes `implementation_version`, and dispatches the matching `v...` tag.
- For pull request contexts, do not dispatch on `refs/pull/*/merge`; use the PR head branch.
- Project-routing public reusable workflows support SHA pinning at the `uses:` boundary, but their internal dispatch still needs a branch or tag ref.
- If `implementation_ref` is omitted, the project-routing workflow first uses `github.workflow_ref` when that is already a branch or tag.
- For SHA-pinned project-routing callers, the public workflow reads `configs/implementation-ref.json` from the pinned workflow revision, takes `implementation_version`, and dispatches the matching `v...` tag.
- For project-routing pull request contexts, do not dispatch on `refs/pull/*/merge`; use the PR head branch.

## Secrets and credentials

- Caller-side router credentials:
- Caller-side router credentials are used only by project-routing workflows:
- `PROJECT_ROUTER_BOT_APP_ID`
- `PROJECT_ROUTER_BOT_PRIVATE_KEY`
- Internal implementation credentials:
- Internal implementation credentials are used only by project-routing implementations:
- `PROJECT_HANDLER_BOT_APP_ID`
- `PROJECT_HANDLER_BOT_PRIVATE_KEY`
- Do not use `secrets: inherit` for this workflow family unless the user explicitly requests it.
- Prefer explicit secret mappings and least-privilege permissions.
- Keep credential use inside the narrowest possible workflow boundary.
- Do not broaden GitHub token or app permissions without a clear repository-specific reason.
- Validate repository ownership, organization scope, and object provenance before mutating projects or dispatching privileged workflows.
- Security-analysis callers should grant only `actions: read`, `contents: read`, and `security-events: write` to the reusable-workflow job. Terraform-quality callers need only `contents: read`. Prefer `permissions: {}` at workflow level and explicit job-level grants.

## Release Please conventions

- Release automation is managed by `.github/workflows/release-please.yml` and `release-please-config.json`.
- `configs/implementation-ref.json` stores the release-managed `implementation_version` used for SHA-pinned dispatch resolution.
- `configs/implementation-ref.json` stores the release-managed `implementation_version` used for project-routing SHA-pinned dispatch resolution.
- Release Please updates `configs/implementation-ref.json` via the JSON updater; workflows prepend `v` when dispatching.
- Internal test workflows use local reusable workflow paths and should not be version-pinned or wired into release-please version bumping.

## Documentation conventions

- Keep docs concise and task-oriented.
- Use `docs/how-to/` for usage steps, `docs/reference/` for exact contracts, and `docs/explanation/` for rationale.
- Use the category index pages in `docs/README.md` to navigate to lower-level indexes. Put ADRs in `docs/explanation/adr/` and maintain that folder's `README.md` index.
- When examples show consumer workflows, prefer `@<commit-sha>` without `implementation_ref` unless the example is explicitly showing an override.

## Change discipline
Expand Down
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@ This repository contains workflows that depend on `datasciencecampus` organizati

## Overview

This repository currently provides three public reusable workflows:
This repository currently provides four public reusable workflows:

- `add-issue-to-projects`: add new issues to one or more `datasciencecampus` ProjectsV2 boards.
- `add-pr-to-projects`: add new pull requests to one or more `datasciencecampus` ProjectsV2 boards and set configured field values.
- `security-analysis`: scan GitHub Actions workflows and infrastructure-as-code for security issues using zizmor and checkov.
- `terraform-quality`: check Terraform formatting, validate configuration, and run TFLint.

Each public reusable workflow has a matching internal implementation workflow. The public workflow is the caller-facing contract; the internal workflow owns the privileged `workflow_dispatch` path and project mutation logic.
The project workflows each have a matching internal implementation workflow. Their public workflows are the caller-facing contracts; the internal workflows own the privileged `workflow_dispatch` path and project mutation logic.

## Workflow Catalog

Expand All @@ -25,6 +26,13 @@ Orchestrates GitHub Actions security analysis with `zizmor` and infrastructure s
- Child workflow (checkov): [.github/workflows/checkov.yml](.github/workflows/checkov.yml)
- How-to guide: [docs/how-to/use-security-analysis-workflow.md](docs/how-to/use-security-analysis-workflow.md)

### `terraform-quality`

Checks Terraform configuration with `terraform fmt`, `terraform validate`, and TFLint. Callers can select validation directories and Terraform version, and enable or disable each check.

- Reusable workflow: [.github/workflows/terraform-quality.yml](.github/workflows/terraform-quality.yml)
- How-to guide: [docs/how-to/use-terraform-quality-workflow.md](docs/how-to/use-terraform-quality-workflow.md)

### `add-issue-to-projects`

Adds opened issues to one or more `datasciencecampus` Projects by project number, with optional field updates.
Expand All @@ -47,8 +55,8 @@ Adds opened pull requests to `datasciencecampus` Projects and sets configured fi

Consumers should call the public reusable workflows from other repositories using `uses:`.

- By default, the reusable workflows dispatch the release tag recorded in metadata stored alongside the invoked workflow revision.
- Set `implementation_ref` only when you need to override that release-managed dispatch target with a specific branch or tag.
- The `add-issue-to-projects` and `add-pr-to-projects` workflows dispatch the release tag recorded in metadata alongside the invoked workflow revision by default.
- Set `implementation_ref` on either project workflow only when you need to override that release-managed dispatch target with a specific branch or tag.

> [!IMPORTANT]
> Public repositories that trigger these workflows automatically from issue or pull request creation events must restrict those events to trusted actors, for example by allowing only collaborators to open issues or pull requests. Configure this in the caller repository at `https://github.com/<owner>/<repo>/settings` under `Settings > General > Features`, then use `Issues > Issue permissions` or `Pull requests > Pull request permissions` as appropriate.
Expand Down
29 changes: 8 additions & 21 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,13 @@
# Documentation (Diataxis)
# Documentation

Keep docs short and practical.
Browse by what you need to do or understand:

## Structure
The documentation is organized using the [Diataxis framework](https://diataxis.fr/).

- `docs/tutorials/`: learning by doing
- `docs/how-to/`: task steps
- `docs/reference/`: facts, inputs, outputs
- `docs/explanation/`: rationale and trade-offs
## Browse documentation

## Pages
- [How-to guides](how-to/README.md) — Follow steps to use the reusable workflows.
- [Reference](reference/README.md) — Look up workflow contracts, inputs, permissions, and GitHub Apps.
- [Explanations](explanation/README.md) — Understand the architecture, trust boundaries, and design decisions.
- [Tutorials](tutorials/README.md) — Learn through guided examples; no tutorials are published yet.

- How-to:
- `docs/how-to/use-add-issue-to-projects-workflow.md`
- `docs/how-to/use-add-pr-to-projects-workflow.md`
- Reference:
- `docs/reference/reusable-workflows.md`
- Explanation:
- `docs/explanation/README.md`

## Rules

- Put all docs under `docs/`.
- Pick one category per page.
- Prefer concise examples over long prose.
8 changes: 2 additions & 6 deletions docs/explanation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@ Background, rationale, and trade-offs.

- [Workflow trust boundaries](workflow-trust-boundaries.md)

## ADRs
## Architecture Decision Records

- [ADR-0001: Called workflow owns secret usage](ADR-0001-called-workflow-owns-secret-usage.md)
- [ADR-0002: Use workflow_dispatch instead of repository_dispatch](ADR-0002-workflow-dispatch-over-repository-dispatch.md)
- [ADR-0003: Unified project_field_values input with optional field updates](ADR-0003-unified-project-field-values-input.md)
- [ADR-0004: Separate reusable workflow pinning from dispatch ref](ADR-0004-separate-reusable-pinning-from-dispatch-ref.md)
- [ADR-0005: Security workflow orchestration pattern](ADR-0005-security-workflow-orchestration.md)
- [Browse architecture decision records](adr/README.md)
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
with:
repository: ${{ github.event.repository.name }}
issue_node_id: ${{ github.event.issue.node_id }}
project_field_values: '[{"project":1234}]'
project_field_values: '[{"project":1234}]'
```

## Rationale
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Create a single orchestrator workflow (`security-analysis.yml`) that runs on `pu
- Caller customization via workflow_call inputs (config paths, persona, advanced-security override)

5. **Clear caller contract**
Repositories should call `security-analysis.yml`, not individual tool workflows. This prevents confusion about which workflow to use and ensures both tools always run together.
Repositories should call `security-analysis.yml`, not individual tool workflows. This prevents confusion about which workflow to use and runs both tools together through the supported entry point.

6. **Parallel execution**
Both tools run in parallel, reducing overall workflow runtime compared to sequential execution.
Expand All @@ -46,15 +46,15 @@ Create a single orchestrator workflow (`security-analysis.yml`) that runs on `pu
Positive:

- Simpler mental model for repository maintainers (one workflow to add, not two)
- Guaranteed consistency: both tools always run together
- Consistent results when callers use the orchestrator: both tools run together
- Easier to update default behavior organization-wide (one place to change)
- Enforces "security by default" — repositories can't accidentally skip one tool
- Provides one recommended entry point that includes both tools
- Flexible customization for specific repositories via workflow_call

Negative:

- If one tool needs to run independently, we'd need to refactor the architecture
- Child workflows cannot be called directly (only via orchestrator), which limits flexibility
- Child workflows expose `workflow_call` and can be called directly, but direct calls bypass the orchestrator's shared triggers, concurrency, and combined-results contract
- Orchestrator adds one layer of indirection (minimal performance impact)

## Alternatives considered
Expand All @@ -68,7 +68,7 @@ Negative:
- Con: Composite actions don't support workflow triggers or SARIF uploads

3. **Orchestrator + direct child calls**
- Current decision; child workflows are reusable-only to prevent independent runs
- Current decision; child workflows expose only `workflow_call`. Callers should use the orchestrator to run both tools together, although GitHub does not prevent direct calls to an individual child workflow.

## Related decisions

Expand Down
9 changes: 9 additions & 0 deletions docs/explanation/adr/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Architecture Decision Records

Decisions that document the rationale and trade-offs behind the repository's workflow architecture.

- [ADR-0001: Called workflow owns secret usage](ADR-0001-called-workflow-owns-secret-usage.md)
- [ADR-0002: Use workflow_dispatch instead of repository_dispatch](ADR-0002-workflow-dispatch-over-repository-dispatch.md)
- [ADR-0003: Unified project_field_values input with optional field updates](ADR-0003-unified-project-field-values-input.md)
- [ADR-0004: Separate reusable workflow pinning from dispatch ref](ADR-0004-separate-reusable-pinning-from-dispatch-ref.md)
- [ADR-0005: Security workflow orchestration pattern](ADR-0005-security-workflow-orchestration.md)
6 changes: 3 additions & 3 deletions docs/explanation/workflow-trust-boundaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ flowchart LR

Related references:

- [ADR-0001: Called workflow owns secret usage](ADR-0001-called-workflow-owns-secret-usage.md)
- [ADR-0002: Use workflow_dispatch instead of repository_dispatch](ADR-0002-workflow-dispatch-over-repository-dispatch.md)
- [ADR-0004: Separate reusable workflow pinning from dispatch ref](ADR-0004-separate-reusable-pinning-from-dispatch-ref.md)
- [ADR-0001: Called workflow owns secret usage](adr/ADR-0001-called-workflow-owns-secret-usage.md)
- [ADR-0002: Use workflow_dispatch instead of repository_dispatch](adr/ADR-0002-workflow-dispatch-over-repository-dispatch.md)
- [ADR-0004: Separate reusable workflow pinning from dispatch ref](adr/ADR-0004-separate-reusable-pinning-from-dispatch-ref.md)
- [GitHub Apps reference](../reference/github-apps.md)
1 change: 1 addition & 0 deletions docs/how-to/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ Task-focused instructions.
- [Use the add-issue-to-projects reusable workflow](use-add-issue-to-projects-workflow.md)
- [Use the add-pr-to-projects reusable workflow](use-add-pr-to-projects-workflow.md)
- [Use the security-analysis reusable workflow](use-security-analysis-workflow.md)
- [Use the terraform-quality reusable workflow](use-terraform-quality-workflow.md)
Loading
Loading