Skip to content

Add CoreCLR WASM composite R2R microbenchmark lane - #5324

Open
lewing wants to merge 1 commit into
mainfrom
lewing-wasm-coreclr-composite-r2r-lane
Open

lewing wants to merge 1 commit into
mainfrom
lewing-wasm-coreclr-composite-r2r-lane

Conversation

@lewing

@lewing lewing commented Sep 28, 2026

Copy link
Copy Markdown
Member

Adds a CoreCLR browser-WASM composite ReadyToRun microbenchmark lane next to the per-assembly R2R lane from #5297.

Dependencies

  • [wasm][coreclr] Enable composite ReadyToRun publishing runtime#134618 ([wasm][coreclr] Enable composite ReadyToRun publishing) must land, and the lane needs a perf-build BrowserWasmCoreCLR artifact built from a runtime commit that includes it. Until then, the lane fails with the runtime's existing PublishReadyToRunComposite is not supported for CoreCLR browser-wasm error.
  • ReadyToRun SDK tasks with wasm output naming (Port runtime PR dotnet/runtime#133111 R2R changes to SDK Crossgen tasks sdk#56395, still open). #134618 adds _WasmCoreClrValidateCompositeTasks, which fails before the composite compile when the SDK's ReadyToRun tasks plan <entry>.r2r.dll instead of <entry>.r2r.wasm. The SDK in the perf artifact (dotnet-none, pinned by runtime's global.json) needs that change. The alternative is for the artifact to ship runtime's Crossgen2Tasks and set Crossgen2SdkOverridePropsPath/Crossgen2SdkOverrideTargetsPath. Until one of those is in place, expect the first runs to fail fast with that clear error, not with a bad benchmark result.

Changes

  • CLI (micro_benchmarks.py): new --wasm-ready-to-run-composite option that implies R2R. --wasm-ready-to-run still means per-assembly. Both options require --wasm --wasm-runtime-flavor CoreCLR. configure_wasm_ready_to_run always writes both PERFLAB_WASM_READY_TO_RUN and PERFLAB_WASM_READY_TO_RUN_COMPOSITE, so a stale parent value can't select the wrong mode. benchmarks_ci.py accepts --wasm-workload-source with either mode.
  • Job script (run_performance_job.py): r2r_run_type == "r2r_composite" maps to R2RType=r2r_composite and passes --wasm-ready-to-run-composite to the Helix work item. The workload-source SDK cohort applies to both R2R modes. r2r_composite is rejected for any runtime type other than wasm_coreclr.
  • MSBuild (MicroBenchmarks.Wasm.targets):
    • PublishReadyToRunComposite now comes from the selected mode. PublishTrimmed, TrimMode, WebCIL and ContainerFormat=wasm are the same as before, so per-assembly mode is unchanged.
    • ValidateWasmReadyToRunConfiguration checks that the applied composite value matches the selected mode, and that composite mode is only used together with R2R.
    • ValidateWasmReadyToRunOutputs (after _CreateR2RImages), composite mode: requires exactly one _ReadyToRunCompileList item with CreateCompositeImage=true. Its OutputR2RImage must end in .r2r.wasm and exist on disk, and there must be at least one _ReadyToRunCompositeBuildInput or _ReadyToRunCompositeUnrootedBuildInput.
    • ValidateWasmReadyToRunOutputs, per-assembly mode: now also fails if a composite image was planned.
    • New ValidateWasmReadyToRunCompositePublishAssets (after ProcessPublishFilesForWasm): requires exactly one _WasmCompositePublishStaticWebAsset with AssetTraitValue=readyToRunComposite. GenerateWasmBootJson routes exactly that asset to resources.coreAssembly (with isCompositeImage). This catches a composite that was compiled but never shipped or loaded.
    • I ran every guard branch locally in a stub MSBuild harness: each failure path reported its error, and the valid composite and per-assembly paths passed.
  • Pipeline: new coreclr_r2r_composite_v8 job in runtime-wasm-perf-jobs.yml. It is identical to coreclr_r2r_v8 except r2rRunType: 'r2r_composite': same release-branch exclusion, payload, machine and engine. The run-performance-job.yml workload-source condition now covers both R2R types.
  • Tests and docs: test_wasm_coreclr_r2r.py covers mode parsing and validation, env configuration (including stale values), work item forwarding, run configuration, the MSBuild property and guard structure, and a check that the new pipeline lane matches the per-assembly lane apart from its identity. The docs describe the new option.

Results identity

Composite runs report R2RType=r2r_composite, so PerfLab keeps them in a separate history from per-assembly R2RType=r2r and from the interpreted coreclr_v8 lane. Existing histories don't change.

Risks to watch in the first validation run

  • Crossgen2 time/memory vs. --buildTimeout 1200: one composite covers the whole trimmed benchmark closure, including Roslyn (Microsoft.CodeAnalysis*). It is one large, serial compile rather than many small ones, and could exceed the BDN build timeout or the machine's memory.
  • BDN's WASM launcher and coreAssembly: the composite owner is loaded from the boot config's resources.coreAssembly before CoreCLR initializes, with component stubs staying in the TPA. BDN's generated WASM host must use the SDK's boot config and loader unchanged. If it doesn't, stubs will fail-fast because they can't find the owner.
  • ColdStart/instantiation cost: instantiating one large .wasm image may noticeably move first-iteration and startup-sensitive numbers compared with per-assembly R2R.

Notes (no changes made)

  • PublishReadyToRunExclude: none remain in MicroBenchmarks.csproj; Enable Jil for CoreCLR WASM R2R #5317, which removed the Jil exclusion, is already in main. Any future exclusion would keep that assembly out of the composite (it would stay IL/interpreted), which is the expected composite behavior.
  • _AOT_InternalForceInterpretAssemblies (Roslyn): this is a Mono AOT item and has no effect on CoreCLR Crossgen2. In composite mode, Roslyn is compiled into the composite, the main contributor to the compile-time risk above. If that turns out too slow, a follow-up could exclude Microsoft.CodeAnalysis* from the composite.

Note

This PR description was generated with GitHub Copilot assistance.

Add a coreclr_r2r_composite_v8 job (r2rRunType r2r_composite) alongside
the per-assembly coreclr_r2r_v8 lane. The mode flows through
run_performance_job.py (R2RType=r2r_composite, a separate PerfLab
history) to a new --wasm-ready-to-run-composite micro_benchmarks option,
which sets PERFLAB_WASM_READY_TO_RUN_COMPOSITE for MSBuild.

MicroBenchmarks.Wasm.targets now sets PublishReadyToRunComposite from
the selected mode, validates it, and in composite mode verifies that
exactly one <entry>.r2r.wasm composite image was produced with
component inputs and that it was defined as the readyToRunComposite
publish asset that GenerateWasmBootJson routes to coreAssembly.

Depends on dotnet/runtime#134618.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 28, 2026 15:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The WASM targets file is not imported, so the composite configuration and validation do not run.

Review effort: Lite
Findings: 1 High severity

Open (1)
What changed in this PR

Adds an opt-in CoreCLR browser-WASM composite ReadyToRun microbenchmark lane with distinct result tracking.

Changes:

  • Adds composite R2R CLI, validation, environment propagation, and job forwarding.
  • Adds MSBuild output and publish-asset checks.
  • Adds pipeline wiring, tests, and documentation.
File Summary
src/​benchmarks/​micro/​MicroBenchmarks.Wasm.targets Composite R2R configuration and validation; currently not imported by the project, so these changes are inactive.
scripts/​tests/​test_wasm_coreclr_r2r.py Tests mode parsing, forwarding, configuration, and pipeline behavior.
scripts/​run_performance_job.py Propagates composite mode and result identity.
scripts/​micro_benchmarks.py Adds composite R2R CLI support.
scripts/​benchmarks_ci.py Accepts composite workload-source runs.
eng/​pipelines/​templates/​run-performance-job.yml Enables workload-source forwarding.
eng/​pipelines/​runtime-wasm-perf-jobs.yml Defines the composite benchmark lane.
docs/​benchmarking-workflow-dotnet-runtime.md Documents composite R2R usage and dependencies.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/benchmarks/micro/MicroBenchmarks.Wasm.targets

@LoopedBard3 LoopedBard3 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems good to me, let's merge once we have the dependencies merged.

@DrewScoggins DrewScoggins left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants