Handle exceptions from parallel ReadyToRun compilation

[agocke]

Summary

• replace ReadyToRun's persistent raw-thread work list with Parallel.ForEach, relying on TPL to join active compilation work before returning or propagating an exception
• use a no-buffering dynamic partitioner in both ReadyToRun and NativeAOT because method compilation costs vary widely and buffered partitions can cause high-core load imbalance
• keep ReadyToRun's CorInfoImpl and recycling counter in partition-local state, bounding cache lifetime without tying large caches to ThreadPool threads
• retain one ReadyToRun WorkerState across single-threaded compilation waves because SuperPMI collection uses --parallelism:1 and requires stable ObjectToHandle handles
• dispose the ReadyToRun compilation when Compile throws and release single-threaded JIT state before object emission

Testing

• ./build.sh clr.jit+clr.hosts: succeeded
• ILCompiler.ReadyToRun and ILCompiler.RyuJit builds: succeeded with 0 warnings and 0 errors
• ILCompiler.ReadyToRun.Tests: 34 passed, 11 skipped, 0 failed
• ILCompiler.Compiler.Tests: 22 passed, 0 failed
• ReadyToRun compiler output was byte-for-byte identical between the dedicated-thread and TPL implementations
• ReadyToRun CoreLib compilation performance was neutral: parallelism 4 median 22.26s vs. 22.76s; parallelism 24 median 10.03s vs. 9.94s
• NativeAOT compiling crossgen2 with ILC (five alternating samples after warmup on a 32-logical-core Intel Xeon Platinum 8370C): parallelism 4 median 58.46s -> 57.09s (2.3% faster); parallelism 24 median 41.49s -> 39.91s (3.8% faster)
• NativeAOT crossgen2 object output was byte-for-byte identical across both implementations and both parallelism settings; median peak RSS was comparable (p4: 1282 MiB -> 1244 MiB, p24: 1296 MiB -> 1316 MiB)

Note

This pull request description was generated with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

This PR refactors parallel ReadyToRun compilation to centralize work distribution/worker lifetime management in a new CompilationWorklist<TWorkerState>, improving exception handling and cleanup behavior, and adjusts crossgen2 compilation ownership to ensure deterministic disposal.

Changes:

• Introduces CompilationWorklist<TWorkerState> to coordinate parallel work and propagate the first unexpected worker exception to the coordinating thread.
• Refactors ReadyToRunCodegenCompilation to use per-worker state and dispose worker/JIT state earlier to reduce peak memory during object emission.
• Refactors crossgen2 to build compilations via a helper and use scoped (using var) disposal for safer cleanup.

Show a summary per file

File	Description
src/coreclr/tools/aot/ILCompiler.ReadyToRun/ILCompiler.ReadyToRun.csproj	Adds the new CompilationWorklist.cs to the build.
src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs	Replaces bespoke thread/semaphore orchestration with CompilationWorklist, adds per-worker state tracking, and disposes worklist before emission.
src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/CompilationWorklist.cs	New helper to manage parallel work distribution, worker lifetime, and exception capture/rethrow.
src/coreclr/tools/aot/crossgen2/Program.cs	Refactors compilation creation into BuildCompilation and uses scoped disposal to simplify ownership and cleanup.

Review details

Suppressed comments (1)

src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/CompilationWorklist.cs:60

• Run(...) should validate that the provided items list is non-null before storing it and using items.Count in TryTake; otherwise a null argument would crash later with a NullReferenceException.

        public void Run(IReadOnlyList<DependencyNodeCore<NodeFactory>> items)
        {
            EnsureWorkers();
            Debug.Assert(_items is null);

            _items = items;

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes core parallel compilation orchestration and error propagation behavior in crossgen2, which merits careful human validation of concurrency/exception semantics.

Review tier: Lite
Findings: 1

New issues introduced by this change (1)

Severity	Finding
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​ReadyToRunCodegenCompilation.cs — The PR description mentions encapsulating work distribution/persistent worker lifetime in…

Issues resolved since last review (1)

Severity	Finding
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​CompilationWorklist.cs — Validate constructor arguments so invalid parallelism values (0/negative) and a null delegate fail… View resolved comment

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/coreclr/tools/aot/crossgen2/Program.cs:356

• determinismCheckFailed is only used to cache compilation.DeterminismCheckFailed and can be removed to simplify control flow without changing behavior.
  src/coreclr/tools/aot/crossgen2/Program.cs:378
• BuildCompilation always builds a ReadyToRunCodegenCompilation (via ReadyToRunCodegenCompilationBuilder.ToCompilation()), so returning ICompilation here hides the concrete type and forces casts at call sites.

[Copilot]

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Install Dependencies

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes core parallel compilation control flow and exception propagation behavior in a high-blast-radius area that warrants focused human review.

Review tier: Lite
Findings: 1 · 1

New issues introduced by this change (1)

Severity	Finding
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​ReadyToRunCodegenCompilation.cs — Parallel.ForEach will surface unexpected worker exceptions as an AggregateException, which can…

Pre-existing issues (1)

Severity	Finding
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​ReadyToRunCodegenCompilation.cs — The PR description mentions encapsulating work distribution/persistent worker lifetime in… View comment

[Copilot]

Copilot review overview

🔵 Needs a closer look

There is a confirmed cross-thread flag updated with Volatile APIs but reset with a non-volatile write, which should be made consistent to avoid mixed volatile/non-volatile access.

Review tier: Lite
Findings: 1 · 1

Pre-existing issues (2)

Severity	Finding
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​ReadyToRunCodegenCompilation.cs — Parallel.ForEach will surface unexpected worker exceptions as an AggregateException, which can… View comment
	src/​coreclr/​tools/​aot/​ILCompiler.ReadyToRun/​Compiler/​ReadyToRunCodegenCompilation.cs — The PR description mentions encapsulating work distribution/persistent worker lifetime in… View comment

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunCodegenCompilation.cs:869

• _compilationSessionGeneratedColdCode is written by worker threads via Volatile.Write and read via Volatile.Read; resetting it with a plain write introduces mixed volatile/non-volatile access on a cross-thread field. Reset it with Volatile.Write as well to keep the access pattern consistent and avoid reordering surprises.

[Copilot]

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review tier: Lite
Findings: None

Resolved findings (2)

• Parallel.ForEach will surface unexpected worker exceptions as an AggregateException, which can…
• The PR description mentions encapsulating work distribution/persistent worker lifetime in…

Previously missed findings (1)

In code that hasn't changed since last review

Preserve code-generation failures across the parallel loop

src/​coreclr/​tools/​aot/​ILCompiler.RyuJit/​Compiler/​RyuJitCompilation.cs:176

This overload reports worker failures as an AggregateException. In DEBUG NativeAOT, ILCompilerRootCommand catches only CodeGenerationFailedException directly to print the single-method repro arguments, so an unresilient worker failure now bypasses that handler; the release path also gets only the generic aggregate message as its headline. Please unwrap and rethrow the original failure here or update the top-level handler to inspect InnerExceptions.

[jkotas]

LGTM!

[agocke]

/ba-g failure is #133798