Migrate eng/pipelines from PublishBuildArtifacts to PublishPipelineArtifact by mmitche · Pull Request #128498 · dotnet/runtime

mmitche · 2026-05-22T18:15:42Z

Summary

Migrates all remaining PublishBuildArtifacts@1 / DownloadBuildArtifacts@0 usage in eng/pipelines/ to PublishPipelineArtifact@1 / DownloadPipelineArtifact@2.

Pipeline Artifacts use dedup-based upload and are 10-100x faster than legacy Build Artifacts. This migration reduces agent time spent on artifact publishing and returns agents to the pool sooner, reducing queue pressure for all pipelines sharing the same pools.

Artifact naming strategy

Pipeline Artifacts cannot be overwritten on retry (unlike Build Artifacts), so artifact names must account for System.JobAttempt:

Diagnostic artifacts (logs, binlogs, test results): suffixed with _Attempt$(System.JobAttempt) to avoid collisions on retry
Intermediate artifacts consumed by downstream jobs: use stable names (no Attempt suffix) and publish only on succeeded() so retries can publish cleanly

Changes

Publish wrapper

eng/pipelines/common/templates/publish-build-artifacts.yml: PublishBuildArtifacts@1 / 1ES.PublishBuildArtifacts@1 → PublishPipelineArtifact@1 / 1ES.PublishPipelineArtifact@1
upload-artifact-step.yml, upload-intermediate-artifacts-step.yml: updated input names (PathtoPublish → targetPath, etc.)

WASM

wasm-post-build-steps.yml: 2 direct calls migrated; stable artifact names preserved for downstream consumption

Performance templates

build-perf-sample-apps.yml: 10 binlog publish steps → _Attempt$(System.JobAttempt) suffix (diagnostic)
build-perf-bdn-app.yml: 1 binlog → _Attempt suffix
build-perf-maui-apps-net6.yml, build-perf-maui-apps-net7.yml: 6 binlogs each → _Attempt suffix

Downloads

download-artifact-step.yml, download-specific-artifact-step.yml, browser-wasm-build-tests.yml, browser-wasm-coreclr-build-tests.yml: migrated to DownloadPipelineArtifact@2

Not changed

eng/common/ files (managed by Arcade; the templates/job/job.yml wrapper already routes enablePublishBuildArtifacts through publish-pipeline-artifacts with _Attempt suffix)
enablePublishBuildArtifacts: true parameter references in SuperPMI/libraries templates (parameter name only; already handled by Arcade wrapper)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

dotnet-policy-service · 2026-05-22T18:17:00Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Copilot

Pull request overview

This PR updates the runtime repo’s Azure Pipelines YAML under eng/pipelines/ to use Pipeline Artifacts (PublishPipelineArtifact@1 / DownloadPipelineArtifact@2) instead of legacy Build Artifacts, including updating shared helper templates so callers pass the correct input names.

Changes:

Replace PublishBuildArtifacts@1 with PublishPipelineArtifact@1 (and 1ES equivalents) and update inputs (pathtoPublish/PathtoPublish/pathToPublish → targetPath).
Replace DownloadBuildArtifacts@0 with DownloadPipelineArtifact@2 and update download destinations (downloadPath → targetPath).
Update perf templates’ binlog artifact naming to include _Attempt$(System.JobAttempt) to avoid retry collisions.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
eng/pipelines/performance/templates/build-perf-sample-apps.yml	Switch binlog publishing to Pipeline Artifacts and suffix names with job attempt.
eng/pipelines/performance/templates/build-perf-maui-apps-net7.yml	Switch MAUI binlog publishing to Pipeline Artifacts with attempt-suffixed artifact names.
eng/pipelines/performance/templates/build-perf-maui-apps-net6.yml	Switch MAUI binlog publishing to Pipeline Artifacts with attempt-suffixed artifact names.
eng/pipelines/performance/templates/build-perf-bdn-app.yml	Switch Benchmarks.Droid binlog publishing to Pipeline Artifacts with attempt-suffixed artifact name.
eng/pipelines/common/wasm-post-build-steps.yml	Publish WASM intermediate artifacts/WBT via Pipeline Artifacts (stable names).
eng/pipelines/common/upload-intermediate-artifacts-step.yml	Update intermediate artifact publish helper to pass Pipeline Artifact inputs.
eng/pipelines/common/upload-artifact-step.yml	Update shared archive+publish helper to publish Pipeline Artifacts (uses `targetPath`).
eng/pipelines/common/templates/publish-build-artifacts.yml	Replace build-artifact publish wrapper with pipeline-artifact publish wrapper.
eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml	Migrate artifact downloads to `DownloadPipelineArtifact@2` with explicit `targetPath`.
eng/pipelines/common/templates/browser-wasm-build-tests.yml	Migrate artifact downloads to `DownloadPipelineArtifact@2` with explicit `targetPath`.
eng/pipelines/common/download-specific-artifact-step.yml	Migrate “download specific run” helper to `DownloadPipelineArtifact@2` and update input names.
eng/pipelines/common/download-artifact-step.yml	Migrate “download current run” helper to `DownloadPipelineArtifact@2` and update to `targetPath`.

mmitche · 2026-05-22T20:43:15Z

This will for sure need some scrutiny but should start to cut some time off PR builds.

mmitche · 2026-06-18T17:38:06Z

@copilot resolve the merge conflicts in this pull request

akoeplinger · 2026-07-03T16:45:47Z

@mmitche copilot can't resolve the merge conflicts here since the PR is opened from your fork

…e-artifacts # Conflicts: # eng/pipelines/performance/templates/build-perf-bdn-app.yml

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

- Rename publish-build-artifacts.yml -> publish-pipeline-artifacts.yml and update references - Update commented-out multi-threaded download block to DownloadPipelineArtifact@2 (targetPath) - Append _Attempt$(System.JobAttempt) to always()-published SuperPMI_Collection/SuperPMI_Logs diagnostic artifacts to avoid retry collisions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fd207c2-4d82-4608-aae9-e2fc3ea7bd2b

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

run-superpmi-collect-job.yml publishes SuperPMI_Collection_* with the default succeeded() condition but has fallible Azure-upload steps after the publish, and the artifact is not consumed downstream by name. Append _Attempt$(System.JobAttempt) so a job retry after a post-publish failure does not collide with the pipeline artifact from attempt 1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2fd207c2-4d82-4608-aae9-e2fc3ea7bd2b

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

eng/pipelines/common/templates/publish-pipeline-artifacts.yml:14

publish-pipeline-artifacts.yml always emits a condition: field, but the condition parameter defaults to an empty string. When a caller doesn't pass condition (e.g. upload-intermediate-artifacts-step.yml), this can expand to condition: with an empty value, which Azure Pipelines treats as an invalid/empty condition expression and can fail template expansion or job execution. Default this parameter to succeeded() (matching Azure Pipelines' task default), or gate emitting the condition: key on ne(parameters.condition, '').

The template's last consumer was removed by dotnet#111454 (perf YAML moved to dotnet/performance), leaving it orphaned. Delete it rather than migrate it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a76ca5e8-9da2-412f-8f6f-5a61d5ba62f5

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

mmitche · 2026-07-15T18:05:40Z

The BrowserWasm / BrowserWasmCoreCLR perf artifacts published here are consumed cross-repo by the runtime-wasm-perf pipeline, whose download side lives in dotnet/performance and still uses DownloadBuildArtifacts@0 (which cannot read Pipeline Artifacts).

Companion PR that migrates the download side to DownloadPipelineArtifact@2 (which reads both Build and Pipeline Artifacts): dotnet/performance#5262.

Merge order: dotnet/performance#5262 should merge before this PR so runtime-wasm-perf keeps working across the transition.

Note

This comment was generated with the assistance of GitHub Copilot.

Per review feedback, publish BrowserWasm/BrowserWasmCoreCLR via the shared upload-artifact-step.yml (Pipeline Artifact) rather than the interim legacy Build Artifact workaround. The dotnet/performance consumer is being updated to DownloadPipelineArtifact@2 to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a76ca5e8-9da2-412f-8f6f-5a61d5ba62f5

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

eng/pipelines/common/templates/publish-pipeline-artifacts.yml:17

publish-pipeline-artifacts.yml currently always emits a condition: field, even when parameters.condition is left as the default empty string. At least one caller (upload-intermediate-artifacts-step.yml) doesn’t pass condition, which would expand to an empty/null condition and can break the task evaluation. Consider defaulting to succeeded() when no explicit condition is provided (similar to the Arcade official template).

…ate-pipeline-artifacts

main added a "Download built nugets for CoreCLR runtime" step to the mono Wasm.Build.Tests template (browser-wasm-build-tests.yml) that still uses DownloadBuildArtifacts@0. This PR publishes the consumed artifact (BuildArtifacts_browser_wasm_<os>_Release_CoreCLR) as a Pipeline Artifact, which DownloadBuildArtifacts@0 cannot read ("Unsupported artifact type: PipelineArtifact") -- it downloads nothing, so the subsequent "Copy CoreCLR runtime pack" step fails with "Not found SourceFolder". Migrate the step to DownloadPipelineArtifact@2 with a targetPath that matches the existing CopyFiles SourceFolder, consistent with the other wasm build-test download steps. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a76ca5e8-9da2-412f-8f6f-5a61d5ba62f5

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

eng/pipelines/common/templates/publish-pipeline-artifacts.yml:17

This template always emits condition: ${{ parameters.condition }}. Since condition defaults to an empty string, callers that omit it will generate condition: '', which is not a valid Azure Pipelines condition and can break template expansion/runtime evaluation.

Consider only emitting condition when provided and otherwise defaulting to succeeded(), matching the pattern used in eng/common/templates-official/steps/publish-pipeline-artifacts.yml.

akoeplinger · 2026-07-19T08:24:19Z

/ba-g the windows failures on the runtime pipeline are due to #130961. failures on runtime-wasm-perf are preexisting on main, will file a separate PR to fix it.

…CoreCLR pack The previous approach invoked publish-pipeline-artifacts.yml directly as a postBuildStep. global-build-job.yml forwards osGroup/archType/buildConfig/etc. to every postBuildStep template, and publish-pipeline-artifacts.yml declares typed parameters, which enforce strict validation and reject those forwarded params -> 'Unexpected parameter' pipeline compile errors. Move the publish (CoreCLR job) and download+stage (Mono job) logic into the untyped perf-wasm-prepare-artifacts-steps.yml template behind publishCoreClrRuntimePack/downloadCoreClrRuntimePack params, mirroring the structure of dotnet#131026 but using the post-dotnet#128498 pipeline-artifact tasks (PublishPipelineArtifact/DownloadPipelineArtifact) instead of the removed publish-build-artifacts.yml template and legacy DownloadBuildArtifacts@0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef6da291-7777-4114-9c73-05398fb2b5f6

PR #128498 migrated eng/pipelines from PublishBuildArtifacts to PublishPipelineArtifact and removed eng/pipelines/common/templates/publish-build-artifacts.yml. This branch predates that merge, so it referenced the now-deleted publish-build-artifacts.yml template and the legacy DownloadBuildArtifacts@0 task, which fail to compile / target the wrong artifact store on current main. Switch the CoreCLR runtime pack publish/download to the post-migration tasks: - publish-build-artifacts.yml -> publish-pipeline-artifacts.yml (PathtoPublish -> targetPath) - DownloadBuildArtifacts@0 -> DownloadPipelineArtifact@2 (downloadPath -> targetPath); pipeline-artifact download doesn't nest under the artifact name, so copy the nupkgs directly from the target path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ef6da291-7777-4114-9c73-05398fb2b5f6

…1033) #128498 migrated this step to PublishPipelineArtifact, which would break runtimelab consumers: they still require multiple jobs publishing to the same 'IntermediateArtifacts' artifact. Revert just this step to 1ES.PublishBuildArtifacts@1 / PublishBuildArtifacts@1. This can be removed once all runtimelab branches moves to v3/v4 publishing.

## Summary - give Linux and Windows browser-wasm CoreCLR build and test Pipeline Artifacts host-specific names - update the browser CoreCLR runtime-test consumer to download the renamed Linux artifact - leave non-browser artifact names and consumers unchanged This preserves Pipeline Artifact publication while avoiding the duplicate name used by both browser hosts after #128498. ## Validation - parsed `eng/pipelines/runtime.yml` with Ruby YAML - verified four host-specific producers and the matching browser consumer - ran `git diff --check` - completed independent Claude and Gemini code reviews with no actionable findings > [!NOTE] > This pull request description was generated by GitHub Copilot.

Migrate eng pipelines to pipeline artifacts

513e76a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 22, 2026 18:15

mmitche requested a review from akoeplinger as a code owner May 22, 2026 18:15

Copilot started reviewing on behalf of mmitche May 22, 2026 18:15 View session

github-actions Bot added the area-Infrastructure label May 22, 2026

github-project-automation Bot added this to Runtime Infra May 22, 2026

dotnet-policy-service Bot assigned mmitche May 22, 2026

Copilot AI reviewed May 22, 2026

View reviewed changes

Comment thread eng/pipelines/common/templates/browser-wasm-coreclr-build-tests.yml

Comment thread eng/pipelines/common/upload-artifact-step.yml Outdated

build-analysis Bot mentioned this pull request May 22, 2026

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

3 tasks

Merge remote-tracking branch 'upstream/main' into dev/migrate-pipelin…

40ea69c

…e-artifacts # Conflicts: # eng/pipelines/performance/templates/build-perf-bdn-app.yml

mmitche commented Jul 14, 2026

View reviewed changes

Comment thread eng/pipelines/common/templates/publish-pipeline-artifacts.yml

mmitche commented Jul 14, 2026

View reviewed changes

Comment thread eng/pipelines/common/upload-intermediate-artifacts-step.yml

Copilot AI review requested due to automatic review settings July 14, 2026 16:05

Copilot started reviewing on behalf of akoeplinger July 14, 2026 16:06 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread eng/pipelines/common/download-specific-artifact-step.yml Outdated

Copilot AI review requested due to automatic review settings July 14, 2026 16:14

Copilot started reviewing on behalf of mmitche July 14, 2026 16:15 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

mmitche commented Jul 14, 2026

View reviewed changes

Comment thread eng/pipelines/common/upload-artifact-step.yml

Copilot AI review requested due to automatic review settings July 14, 2026 16:29

Copilot started reviewing on behalf of mmitche July 14, 2026 16:30 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

mmitche commented Jul 14, 2026

View reviewed changes

Comment thread eng/pipelines/common/download-specific-artifact-step.yml Outdated

Copilot AI review requested due to automatic review settings July 15, 2026 15:25

Copilot started reviewing on behalf of mmitche July 15, 2026 15:25 View session

mmitche commented Jul 15, 2026

View reviewed changes

Comment thread eng/pipelines/performance/templates/perf-wasm-prepare-artifacts-steps.yml Outdated

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread eng/pipelines/performance/templates/perf-wasm-prepare-artifacts-steps.yml Outdated

Comment thread eng/pipelines/performance/templates/perf-wasm-prepare-artifacts-steps.yml Outdated

mmitche mentioned this pull request Jul 15, 2026

Migrate perf artifact download steps to DownloadPipelineArtifact@2 dotnet/performance#5262

Merged

Copilot AI review requested due to automatic review settings July 17, 2026 15:43

Copilot started reviewing on behalf of mmitche July 17, 2026 15:43 View session

akoeplinger approved these changes Jul 17, 2026

View reviewed changes

Copilot AI reviewed Jul 17, 2026

View reviewed changes

mmitche and others added 2 commits July 17, 2026 11:06

Merge commit '034f7f5c843988fd82ddfb8369b27c67ab986c86' into dev/migr…

71b31a6

…ate-pipeline-artifacts

Copilot AI review requested due to automatic review settings July 17, 2026 18:09

Copilot started reviewing on behalf of mmitche July 17, 2026 18:10 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

akoeplinger approved these changes Jul 19, 2026

View reviewed changes

akoeplinger merged commit 2c21b93 into dotnet:main Jul 19, 2026
206 of 211 checks passed

github-project-automation Bot moved this to Done in Runtime Infra Jul 19, 2026

This was referenced Jul 19, 2026

[perf][wasm] Stage CoreCLR wasm runtime pack for Mono workload install #131026

Merged

Keep runtimelab IntermediateArtifacts on classic build artifacts #131033

Merged

dotnet-maestro Bot mentioned this pull request Jul 20, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#7846

Merged

dotnet-milestone-bot Bot added this to the 11.0-preview7 milestone Jul 20, 2026

radekdoulik mentioned this pull request Jul 20, 2026

Fix browser-wasm CoreCLR artifact collisions #131086

Merged

DrewScoggins mentioned this pull request Jul 20, 2026

Use stable names for perf sample-app binlog artifacts #131103

Open

Uh oh!

Conversation

mmitche commented May 22, 2026

Summary

Artifact naming strategy

Changes

Publish wrapper

WASM

Performance templates

Downloads

Not changed

Uh oh!

dotnet-policy-service Bot commented May 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

mmitche commented May 22, 2026

Uh oh!

mmitche commented Jun 18, 2026

Uh oh!

akoeplinger commented Jul 3, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

mmitche commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

akoeplinger commented Jul 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants