Skip to content

Delete ILCompiler.Compiler.Tests unit test - #133474

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:move-swift-lowering-tests
Sep 9, 2026
Merged

MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:move-swift-lowering-tests

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

I expect this to stir up controversy so I'll preface this that I do understand the value of unit testing and it was me who added this testing in the first place (dotnet/corert#4285).

Over the years, we have developed a sense for when adding a test to ILCompiler.Compiler.Tests is appropriate: never. In 9 years we accumulated testing for 3 things in here:

  • Dependency graph testing. Redundant with src/tests/nativeaot/SmokeTests/TrimmingBehaviors E2E tests (that don't just test absence of things in the graph, but also associated object writing, linking, and run-time behaviors)
  • Devirtualization testing. Redundant with all the devirtualization testing we have elsewhere (most of these tests were already added in the AI era by AI and are not as useful as they seem).
  • Swift lowing testing that is probably also redundant with Swift lowering testing elsewhere for CoreCLR sake

The tests never failed for me because they found a bug in my code. They only failed because they couldn't compile or had to be adapted to new behaviors.

The problem is the AI is defaulting to adding tests here. The tests that the AI adds here are two kinds:

  • Completely useless tests (tests that just mirrors code - like "in product code have a line to define a symbol, in test code we validate that the symbol was defined")
  • Tests that should have been an E2E test in the src/tests infrastructure, but AI was satisfied with a unit test and we now lack E2E coverage (AI was satisfied with the unit test, and let's not lie to ourselves, the developer in charge didn't even think about testing or read the test).

We could write AI guidance to never add tests here. But then why do we have these tests? So I'm deleting them. AI should follow the testing strategy that we have been successful with for 9 years and add E2E testing in src/tests.

Testing in src/tests is much higher in value because it runs for all our runtimes and AOT compilers, and executes with all our stress modes and optimization levels. The ILCompiler.Compiler.Tests does none of that and if we make it do that, then we're just replicating src/tests.

I'm moving the Swift lowering testing to type system tests because it's much more related to that than anything else. But if we think it's redundant with the Swift lowing testing that we have in src/tests, we could delete it too. I don't have strong opinion on that one.

MichalStrehovsky and others added 2 commits September 9, 2026 15:22
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The migrated tests introduce build-breaking issues (warnings-as-errors from unused usings, plus an ambiguous MetadataType reference) that must be fixed before merge.

Pull request overview

This PR removes the ILCompiler.Compiler.Tests unit test project and migrates the Swift lowering coverage into ILCompiler.TypeSystem.Tests, updating the associated test-data assembly and build wiring accordingly.

Changes:

  • Delete ILCompiler.Compiler.Tests (and its assets) and remove it from the AOT solution and toolstest subset.
  • Add SwiftLoweringTests to ILCompiler.TypeSystem.Tests, including the shared JIT-interface sources needed for Swift lowering verification.
  • Move/retarget Swift lowering test data into CoreTestAssembly under a TypeSystemTests.TestData.* namespace, and add missing test-data sources.
File summaries
File Description
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/SwiftLoweringTests.cs Moves Swift lowering test to TypeSystem tests and switches to TestTypeSystemContext/CoreTestAssembly-based discovery.
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/ILCompiler.TypeSystem.Tests.csproj Adds linked JIT-interface/Pgo sources and includes SwiftLoweringTests.cs.
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/SwiftTypesSupport.cs Renames Swift test-data namespace to TypeSystemTests.TestData.SwiftTypes.
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/SwiftTypes.cs Renames Swift test-data namespace to TypeSystemTests.TestData.SwiftTypes.
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/Platform.cs Adds InlineArrayAttribute stub needed for compiling inline-array test data in the custom corelib.
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/GvmVariantInterface.cs Moves the variant-GVM test data into CoreTestAssembly (used by existing TypeSystem tests).
src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/CoreTestAssembly.csproj Removes link to the old ILCompiler.Compiler.Tests asset path (now local file exists).
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/ILCompiler.Compiler.Tests.csproj Deleted test project.
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/ILCompiler.Compiler.Tests.Assets/ILCompiler.Compiler.Tests.Assets.csproj Deleted assets project.
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/ILCompiler.Compiler.Tests.Assets/Devirtualization.cs Deleted devirtualization asset source.
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/ILCompiler.Compiler.Tests.Assets/DependencyGraph.cs Deleted dependency-graph asset source.
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/DevirtualizationTests.cs Deleted unit tests.
src/coreclr/tools/aot/ILCompiler.Compiler.Tests/DependencyGraphTests.cs Deleted unit tests.
src/coreclr/tools/aot/ilc.slnx Removes the deleted test project from the solution.
eng/Subsets.props Removes the deleted test project from clr.toolstests.
Review details

Suppressed comments (4)

src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/SwiftLoweringTests.cs:35

  • MetadataType is ambiguous here because both System.Reflection.Metadata and Internal.TypeSystem define a MetadataType. With the current using System.Reflection.Metadata;, this should fail to compile with CS0104 unless the type is qualified or aliased.
    src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/SwiftTypesSupport.cs:7
  • using System.Runtime.InteropServices; is unused in this file, and the repo builds with TreatWarningsAsErrors=true, so this will fail the build (CS8019).
    src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/CoreTestAssembly/SwiftTypes.cs:8
  • using System; is unused in this file, and the repo builds with TreatWarningsAsErrors=true, so this will fail the build (CS8019).
    src/coreclr/tools/aot/ILCompiler.TypeSystem.Tests/SwiftLoweringTests.cs:39
  • Offsets parsing is currently checking the wrong level: NamedArguments.FirstOrDefault(...).Value is a CustomAttributeTypedArgument<TypeDesc>, so the pattern match to ImmutableArray<CustomAttributeTypedArgument<TypeDesc>> will never succeed. This makes offsets always null and can silently ignore explicit Offsets = [...] data in the test attribute.
  • Files reviewed: 14/15 changed files
  • Comments generated: 0
  • Review effort level: Lite

@jkotas jkotas left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the value of these tests ranges from negative to very low, and AI is not helping with that.

@agocke agocke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strong agree with: I've never seen these tests fail due to a bug that wasn't caught elswehere.

@MichalStrehovsky
MichalStrehovsky enabled auto-merge (squash) September 9, 2026 22:24
@MichalStrehovsky
MichalStrehovsky merged commit 78fcecd into dotnet:main Sep 9, 2026
157 of 160 checks passed
@MichalStrehovsky
MichalStrehovsky deleted the move-swift-lowering-tests branch September 9, 2026 22:26
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Sep 10, 2026
maximmenshikov added a commit to maximmenshikov/runtime that referenced this pull request Sep 22, 2026
Fourteen helpers - {FLT,DBL}{ADD,SUB,MUL,DIV}, {FLT,DBL}CMP_{LE,GE}, FLT2DBL and
DBL2FLT - bound by ilc to the toolchain's compiler-rt builtins (__adddf3,
__ledf2, ...), the same way CORINFO_HELP_DBLREM is already bound to fmod. No
floating-point arithmetic is implemented in the runtime or the libraries.

The target is selected explicitly as TargetAbi.NativeAotRiscV64SoftFloat,
spelled riscv64-lp64 on the ilc command line, following the armel precedent. It
drives the JIT flag, the RISC-V ELF float-ABI field in e_flags (which is what
lets the linker reject a mix of lp64 and lp64d objects), the instruction-set
defaults and a two-way validation: lp64d requires F and D, lp64 must not have
them. crossgen2 rejects the target - the helpers have no ReadyToRun encoding.

Value numbering models the helpers as the operations they implement, as
CORINFO_HELP_LMUL is modelled on 32-bit targets; only the two three-way
compares need VNFuncs of their own.

Note: the RiscV64ObjectWriterTests addition was dropped - ILCompiler.Compiler.Tests
was deleted upstream in dotnet#133474 and needs a new home.

Signed-off-by: Maxim Menshikov <maksim.menshikov@nethermind.io>
maximmenshikov added a commit to maximmenshikov/runtime that referenced this pull request Sep 23, 2026
Fourteen helpers - {FLT,DBL}{ADD,SUB,MUL,DIV}, {FLT,DBL}CMP_{LE,GE}, FLT2DBL and
DBL2FLT - bound by ilc to the toolchain's compiler-rt builtins (__adddf3,
__ledf2, ...), the same way CORINFO_HELP_DBLREM is already bound to fmod. No
floating-point arithmetic is implemented in the runtime or the libraries.

The target is selected explicitly as TargetAbi.NativeAotRiscV64SoftFloat,
spelled riscv64-lp64 on the ilc command line, following the armel precedent. It
drives the JIT flag, the RISC-V ELF float-ABI field in e_flags (which is what
lets the linker reject a mix of lp64 and lp64d objects), the instruction-set
defaults and a two-way validation: lp64d requires F and D, lp64 must not have
them. crossgen2 rejects the target - the helpers have no ReadyToRun encoding.

Value numbering models the helpers as the operations they implement, as
CORINFO_HELP_LMUL is modelled on 32-bit targets; only the two three-way
compares need VNFuncs of their own.

Note: the RiscV64ObjectWriterTests addition was dropped - ILCompiler.Compiler.Tests
was deleted upstream in dotnet#133474 and needs a new home.

Signed-off-by: Maxim Menshikov <maksim.menshikov@nethermind.io>
maximmenshikov added a commit to maximmenshikov/runtime that referenced this pull request Sep 25, 2026
Fourteen helpers - {FLT,DBL}{ADD,SUB,MUL,DIV}, {FLT,DBL}CMP_{LE,GE}, FLT2DBL and
DBL2FLT - bound by ilc to the toolchain's compiler-rt builtins (__adddf3,
__ledf2, ...), the same way CORINFO_HELP_DBLREM is already bound to fmod. No
floating-point arithmetic is implemented in the runtime or the libraries.

The target is selected explicitly as TargetAbi.NativeAotRiscV64SoftFloat,
spelled riscv64-lp64 on the ilc command line, following the armel precedent. It
drives the JIT flag, the RISC-V ELF float-ABI field in e_flags (which is what
lets the linker reject a mix of lp64 and lp64d objects), the instruction-set
defaults and a two-way validation: lp64d requires F and D, lp64 must not have
them. crossgen2 rejects the target - the helpers have no ReadyToRun encoding.

Value numbering models the helpers as the operations they implement, as
CORINFO_HELP_LMUL is modelled on 32-bit targets; only the two three-way
compares need VNFuncs of their own.

Note: the RiscV64ObjectWriterTests addition was dropped - ILCompiler.Compiler.Tests
was deleted upstream in dotnet#133474 and needs a new home.

Signed-off-by: Maxim Menshikov <maksim.menshikov@nethermind.io>
maximmenshikov added a commit to maximmenshikov/runtime that referenced this pull request Sep 25, 2026
Fourteen helpers - {FLT,DBL}{ADD,SUB,MUL,DIV}, {FLT,DBL}CMP_{LE,GE}, FLT2DBL and
DBL2FLT - bound by ilc to the toolchain's compiler-rt builtins (__adddf3,
__ledf2, ...), the same way CORINFO_HELP_DBLREM is already bound to fmod. No
floating-point arithmetic is implemented in the runtime or the libraries.

The target is selected explicitly as TargetAbi.NativeAotRiscV64SoftFloat,
spelled riscv64-lp64 on the ilc command line, following the armel precedent. It
drives the JIT flag, the RISC-V ELF float-ABI field in e_flags (which is what
lets the linker reject a mix of lp64 and lp64d objects), the instruction-set
defaults and a two-way validation: lp64d requires F and D, lp64 must not have
them. crossgen2 rejects the target - the helpers have no ReadyToRun encoding.

Value numbering models the helpers as the operations they implement, as
CORINFO_HELP_LMUL is modelled on 32-bit targets; only the two three-way
compares need VNFuncs of their own.

Note: the RiscV64ObjectWriterTests addition was dropped - ILCompiler.Compiler.Tests
was deleted upstream in dotnet#133474 and needs a new home.

Signed-off-by: Maxim Menshikov <maksim.menshikov@nethermind.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants