Support asynchronous runtime test entry points

[jakobbotsch]

Await Task and ValueTask tests in standalone, merged, and XHarness runners. Migrate blocking async test wrappers and remove redundant context restoration and WASM exclusions while preserving Task-returning dispatch coverage.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[azure-pipelines]

Azure Pipelines:
Successfully started running 4 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[jakobbotsch]

@jkoritzinsky Any thoughts on this? This is mostly an LLM PR with some iteration on the async tests, but I am not familiar enough with the test generator to have opinions on those changes...

[jkoritzinsky]

I think this looks fine. We could factor this differently but I don't think it's worth it if this works.

[azure-pipelines]

Azure Pipelines:
Successfully started running 4 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

This PR updates a broad set of CoreCLR async-related tests and infrastructure to run asynchronously end-to-end (avoiding synchronous blocking) and enables the generated runners to await Task/ValueTask-returning tests.

Changes:

• Convert many tests from blocking waits (.Wait(), .Result, .GetAwaiter().GetResult()) to async + await and/or Task-returning facts.
• Update the XUnit wrapper generator and XHarness runner plumbing to treat async tests as first-class and await them.
• Remove wasm-specific runtime-async opt-in properties and simplify async test project build gating; add documentation for async tests.

Reviewed changes

Copilot reviewed 83 out of 83 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/tests/readytorun/async-inline-thunks/async-inline-thunks.cs	Switch entrypoint and helpers to async/await to avoid sync blocking.
src/tests/async/without-yields/without-yields.cs	Convert test to [Fact] async entry method (no .Result).
src/tests/async/with-yields/with-yields.cs	Convert test to [Fact] async entry method (no .Result).
src/tests/async/widening-tailcall/widening-tailcall.cs	Convert conditional sync entrypoint to async [Fact].
src/tests/async/void/void.cs	Replace .Wait() with await in test entrypoint.
src/tests/async/varying-yields/varying-yields.cs	Return Task from test and avoid .Wait().
src/tests/async/valuetaskbased-asyncfibonacci-without-yields/valuetaskbased-asyncfibonacci-without-yields.cs	Make Main async and await async entry.
src/tests/async/valuetaskbased-asyncfibonacci-with-yields/valuetaskbased-asyncfibonacci-with-yields.cs	Make Main async and await async entry.
src/tests/async/valuetask/valuetask.cs	Make facts async/awaitable (Task/ValueTask) and remove sync blocking wrappers.
src/tests/async/valuetask-source/valuetask-source.cs	Convert tests to async and replace .GetResult() with await.
src/tests/async/valuetask-source/valuetask-source-stub.cs	Convert entrypoint to async and await test body.
src/tests/async/taskbased-asyncfibonacci-without-yields/taskbased-asyncfibonacci-without-yields.cs	Make Main async and await async entry.
src/tests/async/taskbased-asyncfibonacci-with-yields/taskbased-asyncfibonacci-with-yields.cs	Make Main async and await async entry.
src/tests/async/synchronization-context/synchronization-context.cs	Convert multiple sync-blocking sync-context tests to async.
src/tests/async/struct/struct.cs	Convert entrypoint to async and await calls.
src/tests/async/strength-reduction/strength-reduction.cs	Convert entrypoint to async and await async work.
src/tests/async/staticvirtual/staticvirtual.cs	Return Task and avoid .Wait().
src/tests/async/small/small.cs	Return Task and avoid .Wait().
src/tests/async/simple-eh/simple-eh.cs	Return Task/Task<int> from facts and remove sync wrappers.
src/tests/async/shared-generic/shared-generic.cs	Convert entrypoint and interface tests to async/await.
src/tests/async/runtime-wrapped-exception/runtime-wrapped-exception.cs	Convert facts/helpers to async/await and remove sync blocking.
src/tests/async/returns/returns.cs	Return Task and avoid .Wait().
src/tests/async/resumed-known-true/resumed-known-true.cs	Convert helper tasks into individual async facts.
src/tests/async/regression/synchronized-async-version.cs	Convert parameterized and fact tests to async and use ThrowsAsync.
src/tests/async/regression/managed-thread-id.cs	Remove sync wrappers and expose async facts directly.
src/tests/async/regression/132017.cs	Convert test entrypoint to async and await work.
src/tests/async/regression/132016.cs	Convert test entrypoint to async and await work.
src/tests/async/regression/132015.cs	Convert facts to async and await test methods.
src/tests/async/regression/128566.cs	Convert fact to async and await work.
src/tests/async/regression/126750.cs	Switch the helper to return Task and await it in the test.
src/tests/async/regression/125805.cs	Convert test flow to direct async fact.
src/tests/async/regression/125615.cs	Convert entrypoint to async and await work.
src/tests/async/regression/125042.cs	Remove sync wrapper; make async method the fact body.
src/tests/async/reflection/reflection.cs	Convert reflection-based async invocations to await.
src/tests/async/pinvoke/pinvoke.cs	Remove sync wrapper and make async method the fact body.
src/tests/async/pgo/pgo.cs	Convert entrypoint to async and await test invocation.
src/tests/async/parallel-continuations/parallel-continuations.cs	Convert to async and await WhenAll results.
src/tests/async/override/override.cs	Convert entrypoint to async and use awaits for virtual dispatch tests.
src/tests/async/osr-inlined-contexts/osr-inlined-contexts.cs	Convert OSR context tests to async/await.
src/tests/async/objects-captured/objects-captured.cs	Convert test to async and await entry.
src/tests/async/object/object.cs	Convert entrypoint to async and await entry.
src/tests/async/mincallcost-microbench/mincallcost-microbench.cs	Make Main async and await Task.Run.
src/tests/async/inst-unbox-thunks/inst-unbox-thunks.cs	Convert multiple facts to async and await results.
src/tests/async/inlined-frame-transition-suspends/inlined-frame-transition-suspends.cs	Convert to async and make RunOn return Task instead of blocking.
src/tests/async/inlined-frame-ldvirtftn-await/inlined-frame-ldvirtftn-await.cs	Convert to async and use ThrowsAsync.
src/tests/async/inlined-frame-exceptions/inlined-frame-exceptions.cs	Convert to async and use ThrowsAsync.
src/tests/async/inlined-frame-contexts/inlined-frame-contexts.cs	Convert tests to async/await.
src/tests/async/inlined-frame-configureawait/inlined-frame-configureawait.cs	Convert to async/await with ConfigureAwait(false) to avoid xUnit sync context interference.
src/tests/async/inlined-asyncversion-frames/inlined-asyncversion-frames.cs	Convert theory/fact to async and await calls.
src/tests/async/implement/implement.cs	Convert entrypoint to async; await interface calls.
src/tests/async/fibonacci-without-yields/fibonacci-without-yields.cs	Convert to async and await entry.
src/tests/async/fibonacci-without-yields-config-await/fibonacci-without-yields-config-await.cs	Convert to async and await entry.
src/tests/async/fibonacci-with-yields/fibonacci-with-yields.cs	Convert to async and await entry.
src/tests/async/fibonacci-with-yields-struct-return/fibonacci-with-yields-struct-return.cs	Convert to async and await entry.
src/tests/async/execution-context/execution-context.cs	Turn multiple sync wrappers into async facts and await bodies.
src/tests/async/eh-microbench/eh-microbench.cs	Make Main async and await Task.Run.
src/tests/async/diagnostic-ip-stacktrace/diagnostic-ip-stacktrace.csproj	Remove wasm runtime-async opt-in property.
src/tests/async/diagnostic-ip-stacktrace/diagnostic-ip-stacktrace.cs	Convert test to async and await the task.
src/tests/async/devirtualize/devirtualize.cs	Convert entrypoint to async and await all calls.
src/tests/async/devirtualize-inherited-nonasync/devirtualize-inherited-nonasync.cs	Convert entrypoint to async and await calls.
src/tests/async/delegates/delegates.cs	Convert entrypoint to async and await results.
src/tests/async/custom-struct-awaiters/custom-struct-awaiters.cs	Convert test to async and remove sync wrapper.
src/tests/async/cse-array-index-byref/cse-array-index-byref.cs	Convert entrypoint to async and await method.
src/tests/async/covariant-return/covariant-returns.cs	Convert multiple test entrypoints to async; remove blocking waits.
src/tests/async/collectible-alc/collectible-alc.cs	Remove sync wrapper and expose async fact method.
src/tests/async/byref-param/byref-param.cs	Convert tests to async facts and await bodies.
src/tests/async/awaitingnotasync/awaitingnotasync.cs	Replace sync wrapper with async [Fact] entrypoint.
src/tests/async/awaitingnoasyncgeneric/awaitingnoasyncgeneric.cs	Convert entrypoint to async and await calls.
src/tests/async/asyncversion-inline/asyncversion-inline.cs	Convert sync-context runner helper + tests to async/await.
src/tests/async/async.csproj	Remove wasm runtime-async opt-in property.
src/tests/async/async-versions-task-adapters/async-versions-task-adapters.cs	Convert tests to async and remove sync waits.
src/tests/async/Directory.Build.targets	Simplify build-disable condition (remove wasm opt-in gating).
src/tests/JIT/opt/Devirtualization/box2.cs	Return Task from fact instead of blocking wait.
src/tests/JIT/Regression_3/GitHub_27924/GitHub_27924.cs	Convert entrypoint to async and await completion.
src/tests/JIT/Regression_2/Runtime_133120/Runtime_133120.cs	Return Task from fact instead of sync wrapper.
src/tests/JIT/Regression_2/Runtime_131285/Runtime_131285.cs	Remove sync wrapper; make async method the fact body.
src/tests/Interop/COM/RuntimeAsync/RuntimeAsync.cs	Convert COM interop facts to async and await tasks.
src/tests/Directory.Build.props	Update xUnit warning suppression guidance regarding blocking.
src/tests/Common/XUnitWrapperGenerator/XUnitWrapperGenerator.cs	Generate async-capable executors and await async tests in runners.
src/tests/Common/XUnitWrapperGenerator/ITestInfo.cs	Add IsAsync to test model and emit await for awaitable returns.
src/tests/Common/XHarnessRunnerLibrary/RunnerEntryPoint.cs	Change callback to async (Task<TestSummary>) for async tests.
src/tests/Common/XHarnessRunnerLibrary/GeneratedTestRunner.cs	Await run callback during execution rather than returning completed task.
docs/workflow/testing/coreclr/testing.md	Document async test authoring expectations (avoid sync blocking).

Suppressed comments (5)

src/tests/async/inlined-frame-configureawait/inlined-frame-configureawait.cs:1

• This test sets a custom SynchronizationContext but does not restore the original context afterward. That can affect subsequent test execution (including xUnit infrastructure) on the same thread. Reintroduce saving/restoring the prior context via try/finally (apply similarly to ConfiguredAwaitInLoopDoesNotReturnToContext).
  src/tests/async/valuetask-source/valuetask-source.cs:1
• Two issues here: (1) the test sets SynchronizationContext to null but does not restore the prior context, which can leak into later tests; reintroduce save/restore via try/finally. (2) The assertions after AwaitConfigDefault/True on sIgnoreContext are checking sUseContext.trace (lines 39 and 42) instead of sIgnoreContext.trace, so they aren't validating the intended behavior.
  src/tests/async/valuetask-source/valuetask-source-stub.cs:1
• EntryPoint sets a custom SynchronizationContext but does not restore the original value afterward. This can pollute the environment for subsequent tests. Capture SynchronizationContext.Current before setting it and restore it in a finally block.
  src/tests/async/synchronization-context/synchronization-context.cs:1
• This test (and others in this file) sets a custom SynchronizationContext but no longer restores the previous context. That can cause cross-test contamination when tests share threads. Restore the prior SynchronizationContext in a finally block (consider a small helper to reduce repetition).
  src/tests/async/regression/126750.cs:1
• This test’s comment indicates it relies on leaving specific values on the stack ‘in the stack range that the resumed frame lands in’. Previously, Run synchronously drove the async continuation to completion (keeping Run’s stack frame alive). Returning the Task and awaiting it from the caller changes stack lifetime and may invalidate what this regression is trying to assert. Consider restoring the original structure where completion is forced before Run returns (e.g., complete the TCS, ensure the task has completed, then observe result) while still avoiding blocking on incomplete tasks.

[Copilot]

🔵 Needs a closer look

Several updated tests set a custom SynchronizationContext without restoring it, which can leak into subsequent in-process tests and cause nondeterministic failures.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

src/tests/async/synchronization-context/synchronization-context.cs:18

• TestSyncContextContinue sets a custom SynchronizationContext but does not restore the previous context. This can leak into later tests in the same assembly and change await/continuation behavior unexpectedly.
  src/tests/async/valuetask-source/valuetask-source-stub.cs:17
• EntryPoint sets a custom SynchronizationContext but never restores the previous context. Because this project runs many tests in-process, leaving a non-default context can affect unrelated tests executed afterwards.

src/tests/async/asyncversion-inline/asyncversion-inline.cs:164

• RunOn sets SynchronizationContext but never restores the previous context. Since the merged/standalone runners execute many tests sequentially in a single process, this can leak a custom SynchronizationContext into subsequent tests and cause cross-test interference.

    private static async Task RunOn(SynchronizationContext ctx, Func<Task> body)
    {
        SynchronizationContext.SetSynchronizationContext(ctx);
        await body();
    }

• Files reviewed: 83/83 changed files
• Comments generated: 0 new
• Review effort level: Lite

[jakobbotsch]

I pushed a commit to fix a test, can I get a reapproval?

[jakobbotsch]

Ping @jkoritzinsky