Skip to content

Improve quickstart deployment validation - #501

Open
Steven Ma (stevenjma) wants to merge 5 commits into
masterfrom
stema-microsoft-improve-validation-workflow
Open

Steven Ma (stevenjma) wants to merge 5 commits into
masterfrom
stema-microsoft-improve-validation-workflow

Conversation

@stevenjma

@stevenjma Steven Ma (stevenjma) commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • replace automatic privileged quickstart PR E2E execution with an authorized maintainer /validate command
  • require fresh per-quickstart metadata.json evidence using ARM_CORRELATION_REQUEST_ID
  • validate correlated Terraform ARM writes against the approved regional ARMProd request stores
  • retain repository-owned E2E execution for changes to test/**
  • document the contributor and maintainer workflow

Security model

  • authorize /validate before entering the protected environment
  • run validators from a trusted default-branch checkout
  • never execute PR-controlled code with Azure OIDC credentials
  • block non-member changes to the workflow and transitive static-check implementation
  • use a subscriptionless OIDC application with no client secret, certificate, or Azure subscription role
  • publish terraform-deployment-validation against the exact PR head SHA

ARM telemetry behavior

  • query Requests.HttpIncomingRequests in armprodeus, armprodweu, and armprodsea
  • bind correlation ID and deployment start time as Kusto query parameters
  • ignore preliminary -1 telemetry records
  • collapse retried writes to their latest HTTP outcome
  • require two identical snapshots separated by the ingestion interval
  • fail closed unless all three regional queries succeed
  • report synchronous 2xx completion separately from 202 Accepted; the latter does not prove eventual provisioning state

Validation

  • metadata validator unit tests
  • ARM correlation validator unit tests
  • Python compilation
  • JSON schema parsing
  • workflow YAML parsing
  • git diff --check
  • live three-region ARMProd query: 14 completed writes, 0 failed writes
  • focused security/correctness review passes; findings addressed or explicitly documented

Provisioning status

  • adx-readonly GitHub environment: configured
  • Corporate Entra OIDC application: azure-terraform-adx-validation
  • Application/client ID: 2f9720c9-2bfa-4e78-8caf-a8f51d8c1c12
  • Tenant ID: 72f988bf-86f1-41af-91ab-2d7cd011db47
  • Federated subject: repo:Azure@6844498/terraform@117169328:environment:adx-readonly
  • Service Tree owner: Azure Deployments (67fa35d2-495a-4f32-aee6-9dac46f7ecce)
  • Azure subscription roles: none
  • ARMProd onboarding: completed under IcM 865765936
  • Approved stores: armprodgbl/ARMProd, armprodeus/Requests, armprodweu/Requests, armprodsea/Requests

The issue_comment workflow must exist on the default branch before /validate comments can execute. End-to-end command validation therefore occurs immediately after merge on a fresh single-quickstart PR containing valid deployment metadata.

Steven (stemaMSFT) and others added 5 commits August 18, 2026 11:21
Add correlation-backed metadata validation and a maintainer-triggered /validate workflow for Terraform quickstarts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Filter changed-files output to existing quickstart sample directories before invoking metadata and Terraform validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore PR coverage for E2E test changes and scope metadata freshness checks to deployable or metadata updates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Configure the validation workflow for a dedicated corporate OIDC application with no Azure subscription roles.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Query the three regional ARM request stores with stable fail-closed snapshots and distinguish synchronous completion from asynchronous acceptance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@stevenjma

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Microsoft"

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants