Skip to content

Restore with command: custom so nuget.config is not rewritten - #792

Merged
YunchuWang merged 1 commit into
mainfrom
yunchuwang/fix-cfs-package-source-mapping
Aug 3, 2026
Merged

Restore with command: custom so nuget.config is not rewritten#792
YunchuWang merged 1 commit into
mainfrom
yunchuwang/fix-cfs-package-source-mapping

Conversation

@YunchuWang

@YunchuWang YunchuWang commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

What changed?

  • eng/templates/build.yml: the Restore step now uses command: custom / custom: restore instead of command: restore with feedsToUse: config. NuGetAuthenticate@1 and nuget.config are unchanged.

Why is this change needed?

Every official build (durabletask-dotnet.official, definition 657) has failed at Restore since #789 routed restores through the Central Feed Service. main is red and no signed packages can be produced, which blocks publishing v1.25.0.

Root cause. DotNetCoreCLI@2's restore command does not pass nuget.config to the CLI. It copies the file into a generated temp config (D:\a\_work\1\Nuget\tempNuGet_<buildId>.config) and prefixes every <packageSources> key with feed-. The <packageSourceMapping> entries are not rewritten, and mapping matches on the source key — so after the rewrite the mapping covers no declared source. NuGet then excludes the only feed for every package, including the Microsoft.Build.Traversal MSBuild SDK that src/dirs.proj needs, so evaluation dies before a single package is fetched:

error : Could not resolve SDK "Microsoft.Build.Traversal".
error : Unable to resolve 'Microsoft.Build.Traversal (= 4.1.82)' for '.NETStandard,Version=v0.0'.
    PackageSourceMapping is enabled, the following source(s) were not considered: feed-upstream-public.
error MSB4236: The SDK 'Microsoft.Build.Traversal' specified could not be found.

Why this isn't fixed in nuget.config. It can't be. The rewrite prepends feed- unconditionally and non-idempotently. Let S be the source key and M the mapping key in the single nuget.config every environment shares. NuGet matches only when the two are equal, so:

  • local dev and GitHub Actions, no rewriting → requires S == M
  • Azure Pipelines through the task → requires feed-S == M

Both must hold for the same file, which requires S == feed-S. There is no such string, so no naming scheme satisfies both environments. Confirmed empirically across the entire permutation space for one feed:

<packageSources> key <packageSourceMapping> key Build Result
upstream-public upstream-public 295413 ❌ not considered: feed-upstream-public
upstream-public both keys 295418 ❌ multiple package sources associated with the same key(s)
both keys both keys 295505 ❌ not considered: feed-feed-upstream-public
feed-upstream-public feed-upstream-public 295506 ❌ not considered: feed-feed-upstream-public

The double feed-feed- prefix in the last two is direct evidence the rewrite is unconditional.

Why command: custom fixes it. feedsToUse and nugetConfigPath are inputs that only exist for the restore and push commands. With command: custom the task shells out to dotnet restore <arguments> and never enters the config-rewriting path, so nuget.config is used verbatim and the source keys match the mapping by construction. This is also how the Pack step in this same file already invokes the task.

Tracked upstream by azure-pipelines-tasks#15542.

Issues / work items


Project checklist

  • Release notes are not required for the next release
  • Backport is not required
  • All required tests have been added/updated (unit tests, E2E tests) — build infrastructure only, no product code
  • Breaking change? No

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • No
  • Yes, AI helped write parts of this PR (e.g., GitHub Copilot)
  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot CLI
  • AI-assisted areas/files: eng/templates/build.yml
  • What you changed after AI output: n/a

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Result: Passed

Manual validation (only if runtime/behavior changed)

Validated on the real pipeline (definition 657), one variable per run, all from base c3d5d87a.

Build Change under test Restore Outcome
295413 none (main as-is) SDK resolution failure
295418 nuget.config mapping workaround duplicate mapping key
295505 nuget.config both source keys feed-feed- double prefix
295506 nuget.config prefixed source key feed-feed- double prefix
295451 direct pwsh: dotnet restore green
295504 this PR (command: custom) green

Build 295504 completed every stage — Restore → Build → Sign assemblies → Pack → Sign NugetPackages → Component Governance → SBOM → artifact publish — and produced all 17 signed packages at the expected versions:

Microsoft.DurableTask.Abstractions.1.25.0.nupkg
Microsoft.DurableTask.Analyzers.1.25.0.nupkg
Microsoft.DurableTask.Client.1.25.0.nupkg
Microsoft.DurableTask.Client.AzureManaged.1.25.0.nupkg
Microsoft.DurableTask.Client.AzureManaged.Sandboxes.1.25.0.nupkg
Microsoft.DurableTask.Client.Grpc.1.25.0.nupkg
Microsoft.DurableTask.Client.OrchestrationServiceClientShim.1.25.0.nupkg
Microsoft.DurableTask.Extensions.AzureBlobPayloads.1.25.0.nupkg
Microsoft.DurableTask.Grpc.1.25.0.nupkg
Microsoft.DurableTask.Worker.1.25.0.nupkg
Microsoft.DurableTask.Worker.AzureManaged.1.25.0.nupkg
Microsoft.DurableTask.Worker.AzureManaged.Sandboxes.1.25.0.nupkg
Microsoft.DurableTask.Worker.Grpc.1.25.0.nupkg
Microsoft.DurableTask.ExportHistory.1.25.0-preview.2.nupkg
Microsoft.DurableTask.ScheduledTasks.1.25.0-preview.2.nupkg
Microsoft.DurableTask.Generators.2.1.0-preview.2.nupkg
Microsoft.DurableTask.InProcessTestHost.0.2.3-preview.1.nupkg

The YAML in this PR was verified to parse to a structure identical to the one build 295504 ran; the only textual difference is an added explanatory comment.


Notes for reviewers

  • nuget.config is deliberately untouched. The mapping there was already correct — the task was corrupting it.
  • Only the restore step used feedsToUse: config. Build and Pack pass --no-restore and never resolve packages, so they are unaffected.
  • NuGetAuthenticate@1 is retained and still succeeds; it exports the credential provider environment for the following step.
  • This makes the ADO path consistent with what Route NuGet restores through Central Feed Service #789 already does in GitHub Actions, which restores via dotnet restore --configfile nuget.config.
  • azure-pipelines-release.yml has the same pattern but is not referenced by any enabled definition (657 → eng/ci/official-build.yml, 1100 → eng/publish/publish.yml, 520 → eng/ci/code-mirror.yml), so it is left alone.
  • A direct pwsh: dotnet restore step also works (build 295451). command: custom was chosen because it keeps the task wrapper and matches the adjacent Pack step.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes NuGet restore failures in Azure Pipelines when using DotNetCoreCLI@2/NuGetCommand tasks with feedsToUse: config and PackageSourceMapping enabled. Those tasks rewrite the config and prefix feed keys with feed-, which breaks key-based package source mapping unless the prefixed key is also mapped.

Changes:

  • Add a feed-upstream-public entry under <packageSourceMapping> to match the Azure Pipelines task–rewritten source key.
  • Document the Azure Pipelines task behavior and link the upstream tracking issue for context.

Copilot AI review requested due to automatic review settings August 3, 2026 18:34
@YunchuWang
YunchuWang force-pushed the yunchuwang/fix-cfs-package-source-mapping branch from d571c49 to 5588164 Compare August 3, 2026 18:34
@YunchuWang YunchuWang changed the title Fix package source mapping under Azure Pipelines NuGet tasks Restore directly instead of through the DotNetCoreCLI task Aug 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

DotNetCoreCLI@2's restore command copies nuget.config into a generated temp
config and prefixes every package source key with "feed-". The
packageSourceMapping entries the Central Feed Service requires are not
rewritten and match on the source key, so after the rewrite the mapping covers
no declared source. NuGet excludes the only feed for every package, including
the Microsoft.Build.Traversal MSBuild SDK, so evaluating src/dirs.proj fails
before any package is fetched:

  error : Unable to resolve 'Microsoft.Build.Traversal (= 4.1.82)'.
  PackageSourceMapping is enabled, the following source(s) were not
  considered: feed-upstream-public.
  error MSB4236: The SDK 'Microsoft.Build.Traversal' could not be found.

Every official build has failed this way since the Central Feed Service change.

No nuget.config can work around this. The rewrite prepends "feed-"
unconditionally, so a source key never equals its own mapping key under the
task, while outside the task they must be equal. Renaming the source, adding
the prefixed key to the mapping, or declaring both were each tried and each
fails.

feedsToUse and nugetConfigPath only apply to the restore and push commands.
Using command: custom skips the rewriting entirely and passes nuget.config
straight to the CLI, matching how the Pack step already invokes the task.

See microsoft/azure-pipelines-tasks#15542

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3697a867-8972-4cde-81fd-b3dd94c55066
@YunchuWang
YunchuWang force-pushed the yunchuwang/fix-cfs-package-source-mapping branch from 5588164 to 840dde7 Compare August 3, 2026 18:54
Copilot AI review requested due to automatic review settings August 3, 2026 18:54
@YunchuWang YunchuWang changed the title Restore directly instead of through the DotNetCoreCLI task Restore with command: custom so nuget.config is not rewritten Aug 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

eng/templates/build.yml:54

  • The explanatory comment says the DotNetCoreCLI task rewrites package source keys but that the packageSourceMapping entries are “not rewritten”. That contradicts the PR description (and the linked failure mode) where the task also rewrites mapping keys, causing collisions/duplicate-key errors. Please adjust the comment to reflect the actual behavior so future maintainers don’t apply the wrong workaround.
        # Restore uses command: custom rather than command: restore. The restore command's
        # feedsToUse/nugetConfigPath inputs make the task copy nuget.config into a generated temp
        # config and prefix every package source key with "feed-". Package source mapping matches on
        # the source key and those entries are not rewritten, so the mapping the Central Feed Service
        # requires stops applying and NuGet excludes the only feed. See

@YunchuWang
YunchuWang merged commit 26bd006 into main Aug 3, 2026
9 of 10 checks passed
@YunchuWang
YunchuWang deleted the yunchuwang/fix-cfs-package-source-mapping branch August 3, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants