Enable Roslyn compiler output caching in non-official builds - #133748
Conversation
Configure the experimental Roslyn compiler output cache alongside sccache for non-official builds. Disable SourceLink for those builds, persist the cache through Azure Pipelines, and report per-build cache statistics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ab967da8-759b-499d-a20d-d6d6c2cc355d
|
Azure Pipelines: Successfully started running 5 pipeline(s). 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved concerns remain around cache growth, diagnostics preservation, and complete statistics coverage.
Get a fresh assessment by requesting another Copilot review.
Review tier: Lite
Findings: 1
Open findings (1)
What changed in this PR
Enables experimental Roslyn compiler-output caching for non-official Unix builds using sccache.
Changes:
- Adds Azure Pipelines cache setup and SourceLink disabling.
- Integrates cache arguments into global builds.
- Reports Roslyn cache statistics.
| File | Summary |
|---|---|
eng/pipelines/coreclr/templates/setup-sccache.yml |
Integrates Roslyn cache setup. |
eng/pipelines/coreclr/templates/setup-roslyn-cache.yml |
Restores, configures, and persists the compiler cache. |
eng/pipelines/coreclr/templates/sccache-stats.yml |
Reports compiler cache statistics. |
eng/pipelines/coreclr/templates/roslyn-cache-stats.yml |
Collects per-build cache statistics. |
eng/pipelines/common/global-build-job.yml |
Forwards cache-specific build arguments. |
|
Can we add a script to enable this cache (and sccache) locally? It would be great if we could get to a point where a new worktree from a known commit would pull cached artifacts and make the baseline build much shorter. |
|
Sccache just needs an environment variable set locally. Once this one is tested out I think it would make sense to have an opt-in as well. I want to try it out in CI first, though. |
|
Last I checked sccache didn't really work on Windows, can we decouple this from that so that Windows can benefit from the Roslyn feature as well? |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 501aac83-6191-40e5-8dc2-ef3acbcb1e41
|
I agree with decoupling from sccache but I want to enable incrementally, like I did with sccache. Let's start with Linux and go from there. |
|
Azure Pipelines: Successfully started running 5 pipeline(s). 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
@akoeplinger Fixed. Anything else? |
Enable Roslyn's experimental compiler output cache in the Unix x64/arm64 non-official build legs that already use sccache.
The setup:
Cache@2PathMapalready produced byContinuousIntegrationBuildOfficial builds are unaffected and continue to produce SourceLink-enabled outputs without using this cache.
In a two-commit
libs.testsexperiment using a Roslyn build containing dotnet/roslyn#84916, globally disabling SourceLink produced 358 hits and one store across 359 compiler invocations. The cached build took 4m53s versus 6m32s with the cache disabled, a 25.25% reduction (1.338x speedup). With SourceLink enabled, only 13 invocations hit.A local integration probe using
ROSLYN_CACHE_PATHproduced 41 stores on the first rebuild and 41 hits on the second rebuild.The Roslyn compiler output cache remains experimental. Cache read or write failures fall back to normal compilation, and the outer cache key is versioned so its provisional format can be invalidated independently.
Note
This pull request description was generated with GitHub Copilot.