Skip to content

Preserve vector results when folding AVX-512 mask complements - #134520

Merged
tannergooding merged 1 commit into
dotnet:mainfrom
tannergooding:tannergooding-fix-avx512-mask-folding
Sep 23, 2026
Merged

tannergooding merged 1 commit into
dotnet:mainfrom
tannergooding:tannergooding-fix-avx512-mask-folding

Conversation

@tannergooding

Copy link
Copy Markdown
Member

Preserve ConvertMaskToVector when folding a vector complement through an AVX-512 mask-producing operation. The comparison-mode reversal and double-NOT paths previously returned the inner TYP_MASK where the parent expected a vector, causing checked-build assertions and invalid release codegen.

Reuse the existing conversion around the simplified mask and assert that the replacement preserves the outer NOT's result type. This leaves comparison predicates, mask element sizes, and operand evaluation unchanged.

Add focused regressions for both folds, including both comparison-reversal directions, equal/unequal/NaN inputs, 128/256/512-bit vectors, and double-NOT masks with byte/short/int/long elements. Both regression paths independently reproduce the assertion with the unfixed JIT.

Validation on Windows x64:

  • Checked JIT build and JIT formatting passed.
  • Seven focused cases passed on AVX-512 hardware, with AVX-512 disabled, and with AVX2 disabled; three double-NOT cases passed with hardware intrinsics disabled.
  • Nine adjacent regression cases passed with AVX-512 enabled and disabled.

Release and other-architecture runs, and the full test suite, were not run. This fix is independent of the pending operand-ordering changes.

Resolves #134480

Note

This PR was prepared with GitHub Copilot.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 23, 2026 13:34
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Sep 23, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Release, other-architecture, and full-suite validation were not run.

Review effort: Lite
Findings: None

What changed in this PR

Fixes RyuJIT AVX-512 mask-complement folding so vector result conversions are preserved.

Changes:

  • Reuses ConvertMaskToVector in both affected fold paths.
  • Adds focused regression coverage across vector widths, comparisons, NaNs, and element sizes.
File Description
src/​tests/​JIT/​Regression_ro_2/​Runtime_134480.cs Adds focused regression tests.
src/​coreclr/​jit/​morph.cpp Preserves vector result types during mask folding.

@tannergooding
tannergooding merged commit 2af9a56 into dotnet:main Sep 23, 2026
143 of 146 checks passed
@tannergooding
tannergooding deleted the tannergooding-fix-avx512-mask-folding branch September 23, 2026 19:29
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects

None yet

3 participants