Collapse standalone Libraries Test Run jobs into build legs (Helix submission in-job) by Copilot · Pull Request #131948 · dotnet/runtime

Copilot · 2026-08-06T15:34:36Z

main PR N/A

Description

This refactor removes standalone libraries test-run jobs and submits the same Helix work directly from the build jobs that already produce the corresponding test assets. The goal is to keep test coverage unchanged while reducing pipeline job split overhead now that Helix Job Monitor is enabled.

Runtime pipeline (eng/pipelines/runtime.yml)
- Removed all jobTemplate: /eng/pipelines/libraries/run-test-job.yml usages.
- Added postBuildSteps Helix submission (/eng/pipelines/libraries/helix.yml) to the matching libraries/coreclr build legs.
- Preserved queue routing (libraries queue group where previously used) and trigger conditions per leg.
- Preserved checked-runtime + release-libraries behavior by explicitly setting:
  - /p:Configuration=Release /p:RuntimeConfiguration=Checked on the checked-runtime release-libraries leg.
SuperPMI pipeline (eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml)
- Removed the two standalone libraries test-run jobs (normal, no_tiered_compilation).
- Embedded two Helix submissions (same scenarios) into each checked build leg that produces the assets.
- Preserved superpmi queue group and checked-runtime intent via:
  - /p:Configuration=Release /p:RuntimeConfiguration=Checked.
Template cleanup
- Deleted now-unused eng/pipelines/libraries/run-test-job.yml.

Coverage parity (before → after)

Scope	Before	After	Parity
`runtime.yml` libraries test-run legs	5 standalone `run-test-job` matrix entries	Helix submit moved into corresponding `global-build-job` legs	Same platform/arch/config/trigger coverage, same queue template/group mapping
`superpmi-collect-pipeline.yml` libraries scenarios	2 standalone jobs (`normal`, `no_tiered_compilation`) across 7 platforms	Same 2 scenarios submitted from the checked build legs covering same platform set	Equivalent scenario + platform coverage

# Before
- template: /eng/pipelines/common/platform-matrix.yml
  parameters:
    jobTemplate: /eng/pipelines/libraries/run-test-job.yml
    ...

# After
- template: /eng/pipelines/common/platform-matrix.yml
  parameters:
    jobTemplate: /eng/pipelines/common/global-build-job.yml
    ...
    jobParameters:
      postBuildSteps:
        - template: /eng/pipelines/libraries/helix.yml
          parameters:
            ...

Customer Impact

Keeps the same runtime/libraries test surface while reducing pipeline fragmentation and simplifying failure triage for affected legs.

Regression

No known product regression; this is CI orchestration refactoring with explicit preservation of prior test-leg coverage and runtime/config targeting.

Testing

Pipeline-template/YAML refactor only; no product code path changes.

Risk

Low to moderate. Main risk is CI leg wiring (condition/queue/config mismatch). This change keeps prior leg mapping and explicitly pins runtime/config where the old split-job flow implied it.

Package authoring no longer needed in .NET 9

IMPORTANT: Starting with .NET 9, you no longer need to edit a NuGet package's csproj to enable building and bump the version.
Keep in mind that we still need package authoring in .NET 8 and older versions.

Copilot

Copilot wasn't able to review any files in this pull request.

azure-pipelines · 2026-08-06T15:34:53Z

Azure Pipelines: 16 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: akoeplinger <1376924+akoeplinger@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Suppressed comments (6)

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:64

SuperPMI collection submissions need a non-empty SuperPmiCollectionType. This helix.yml invocation doesn’t set it, so the default empty value will flow into the Helix post-commands and result filenames (double-dot artifacts), which can break collection processing.

                  testRunNamePrefixSuffix: coreclr_Checked
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked
                  SuperPmiCollect: true
                  SuperPmiCollectionName: libraries_tests_no_tiered_compilation

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:118

SuperPMI collection submissions need SuperPmiCollectionType set (used for result file naming and post-commands). Leaving it empty here will generate incorrectly named outputs and can break collection processing.

                  testScope: innerloop
                  testRunNamePrefixSuffix: coreclr_Checked
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked
                  SuperPmiCollect: true

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:128

SuperPMI collection submissions need SuperPmiCollectionType set. With the current helix.yml call it stays empty, which changes the expected .mch/.log names and can break collection processing.

                  creator: dotnet-bot
                  testScope: innerloop
                  testRunNamePrefixSuffix: coreclr_Checked
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:186

SuperPMI collection submissions need SuperPmiCollectionType set. Leaving it empty here will produce incorrectly named .mch/.log outputs (double-dot artifacts) and can break collection processing.

                parameters:
                  creator: dotnet-bot
                  testScope: innerloop
                  testRunNamePrefixSuffix: coreclr_Checked
                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:196

SuperPMI collection submissions need SuperPmiCollectionType set. If it remains empty here, the SuperPMI post-commands and downloaded result filenames won’t match expectations, which can break collection processing.

              - template: /eng/pipelines/libraries/helix.yml
                parameters:
                  creator: dotnet-bot
                  testScope: innerloop
                  testRunNamePrefixSuffix: coreclr_Checked

eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml:78

This platform-matrix leg now embeds the Helix SuperPMI collection submission into the build job, but the job still uses timeoutInMinutes: 120 later in this block. Previously the equivalent libraries SuperPMI test-run leg used a 300 minute timeout (notably for osx-arm64); collapsing the test leg into the build job without increasing the timeout risks the job timing out before Helix completes.

        parameters:
          jobTemplate: /eng/pipelines/common/global-build-job.yml
          helixQueueGroup: superpmi
          helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
          buildConfig: checked

+                  testRunNamePrefixSuffix: coreclr_Checked
+                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
+                  SuperPmiCollect: true
+                  SuperPmiCollectionName: libraries_tests
+                  scenarios:
+                  - normal


akoeplinger · 2026-08-06T16:14:15Z

@copilot address the CI failures and the code review comment

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

+                  testRunNamePrefixSuffix: coreclr_Checked
+                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
+                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked
+                  SuperPmiCollect: true
+                  SuperPmiCollectionName: libraries_tests
+                  scenarios:
+                  - normal
+              - template: /eng/pipelines/libraries/helix.yml
+                parameters:
+                  creator: dotnet-bot
+                  testScope: innerloop
+                  testRunNamePrefixSuffix: coreclr_Checked
+                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
+                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked
+                  SuperPmiCollect: true
+                  SuperPmiCollectionName: libraries_tests_no_tiered_compilation
+                  scenarios:
+                  - no_tiered_compilation


+                  testRunNamePrefixSuffix: coreclr_Checked
+                  useHelixMonitor: ${{ variables.enableHelixJobMonitor }}
+                  extraHelixArguments: /p:Configuration=Release /p:RuntimeConfiguration=Checked
+                  SuperPmiCollect: true


jkoritzinsky · 2026-08-06T17:37:01Z

Can we move the host/installer build+test steps into the main build job as well?

If we do that, we should also rename the jobs as "AllSubsets" is not an interesting moniker once all jobs are "all subsets"

dotnet-policy-service · 2026-08-06T23:18:12Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Initial plan

764a1d3

Copilot AI assigned Copilot and akoeplinger Aug 6, 2026

Copilot AI review requested due to automatic review settings August 6, 2026 15:34

Copilot AI reviewed Aug 6, 2026

View reviewed changes

Copilot started work on behalf of akoeplinger August 6, 2026 15:35 View session

Collapse standalone libraries test-run pipeline jobs

33db6e8

Co-authored-by: akoeplinger <1376924+akoeplinger@users.noreply.github.com>

Copilot AI review requested due to automatic review settings August 6, 2026 15:48

Copilot started reviewing on behalf of akoeplinger August 6, 2026 15:54 View session

Fix moved Helix send configuration parity

9b504b4

Co-authored-by: akoeplinger <1376924+akoeplinger@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Collapse test-only jobs into product jobs~~ Collapse standalone Libraries Test Run jobs into build legs (Helix submission in-job) Aug 6, 2026

Copilot finished work on behalf of akoeplinger August 6, 2026 16:04

Copilot AI requested a review from akoeplinger August 6, 2026 16:04

Copilot AI reviewed Aug 6, 2026

View reviewed changes

Comment thread eng/pipelines/coreclr/templates/superpmi-collect-pipeline.yml

Comment on lines +49 to +54

testRunNamePrefixSuffix: coreclr_Checked

useHelixMonitor: ${{ variables.enableHelixJobMonitor }}

SuperPmiCollect: true

SuperPmiCollectionName: libraries_tests

scenarios:

- normal

Copilot AI review requested due to automatic review settings August 6, 2026 16:04

Copilot started reviewing on behalf of akoeplinger August 6, 2026 16:11 View session

Copilot AI reviewed Aug 6, 2026

View reviewed changes

Copilot finished work on behalf of akoeplinger August 6, 2026 16:30

jkoritzinsky mentioned this pull request Aug 6, 2026

Consolidate libraries Helix submissions #131971

Merged

github-actions Bot added the area-Infrastructure-libraries label Aug 6, 2026

Navigation Menu

Collapse standalone Libraries Test Run jobs into build legs (Helix submission in-job) - #131948

Collapse standalone Libraries Test Run jobs into build legs (Helix submission in-job)#131948
akoeplinger with Copilot wants to merge 3 commits into
mainfrom
copilot/dotnetruntime-131615

Copilot AI commented Aug 6, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

azure-pipelines Bot commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

akoeplinger commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

jkoritzinsky commented Aug 6, 2026

Uh oh!

dotnet-policy-service Bot commented Aug 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Copilot AI commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Customer Impact

Regression

Testing

Risk

Package authoring no longer needed in .NET 9

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

azure-pipelines Bot commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

akoeplinger commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

jkoritzinsky commented Aug 6, 2026

Uh oh!

dotnet-policy-service Bot commented Aug 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Aug 6, 2026 •

edited

Loading