Sve: *FirstFaulting* are mostly failing in gcstress runs

[kunalspathak]

When working on #112362, I see there are other gcstress failures for SVE tests.

Discussion: #112389 (comment)

Sample run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=948285&view=ms.vss-test-web.build-test-results-tab&runId=25190942&resultId=169023&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

[kunalspathak]

@dotnet/arm64-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[a74nh]

I wonder if this a failure in the mechanism that tracks the first faulting state?

[kunalspathak]

It looks like we are in stuck in some kind of deadlock in gc code and not sure if it something that these tests exposed in GC or we are encoding wrong gc information that is making us end up in this state. @mrsharm - lets work offline to see what is going on.

call stack

 0:005> ~*kc

   0  Id: 2d00.3224 Suspend: 1 Teb: 00000091`d595c000 Unfrozen
 # Call Site
00 ntdll!NtWaitForSingleObject
01 KERNELBASE!WaitForSingleObjectEx
02 coreclr!GCEvent::Impl::Wait
03 coreclr!GCEvent::Wait
04 coreclr!WKS::gc_heap::wait_for_gc_done
05 coreclr!WKS::gc_heap::try_allocate_more_space
06 coreclr!WKS::gc_heap::allocate_more_space
07 coreclr!WKS::gc_heap::allocate
08 coreclr!WKS::GCHeap::Alloc
09 coreclr!Alloc
0a coreclr!Alloc
0b coreclr!AllocateObject
0c coreclr!AllocateObject
0d coreclr!EEException::CreateThrowable
0e coreclr!CreateCOMPlusExceptionObject
0f coreclr!ExceptionTracker::CreateThrowable
10 coreclr!ProcessCLRExceptionNew
11 coreclr!ProcessCLRException
12 ntdll!RtlpExecuteHandlerForUnwind
13 ntdll!RtlUnwindEx
14 ntdll!RtlUnwind
15 coreclr!ClrUnwindEx
16 coreclr!ProcessCLRExceptionNew
17 coreclr!ProcessCLRException
18 ntdll!RtlpExecuteHandlerForException
19 ntdll!RtlDispatchException
1a ntdll!KiUserExceptionDispatch
1b KERNELBASE!#DebugBreak
1c coreclr!WKS::FATAL_GC_ERROR
1d coreclr!WKS::gc_heap::verify_free_lists
1e coreclr!WKS::gc_heap::verify_heap
1f coreclr!WKS::gc_heap::garbage_collect
20 coreclr!WKS::GCHeap::GarbageCollectGeneration
21 coreclr!WKS::GCHeap::GarbageCollectTry
22 coreclr!WKS::GCHeap::StressHeap
23 coreclr!DoGcStress
24 coreclr!OnGcCoverageInterrupt
25 coreclr!IsGcMarker
26 coreclr!CLRVectoredExceptionHandlerShim
27 ntdll!RtlpCallVectoredHandlers
28 ntdll!RtlDispatchException
29 ntdll!KiUserExceptionDispatch
2a 0x0
2b 0x0
2c 0x0
2d 0x0
2e 0x0
2f coreclr!CallDescrWorkerInternal
30 coreclr!CallDescrWorkerWithHandler
31 coreclr!MethodDescCallSite::CallTargetWorker
32 coreclr!MethodDescCallSite::Call_RetArgSlot
33 coreclr!RunMainInternal
34 coreclr!RunMain::__l30::__Body::Run::__l5::__Body::Run
35 coreclr!`RunMain'::`30'::__Body::Run
36 coreclr!RunMain
37 coreclr!Assembly::ExecuteMainMethod
38 coreclr!CorHost2::ExecuteAssembly
39 coreclr!coreclr_execute_assembly
3a corerun
3b corerun!GetCurrentClrDetails
3c corerun!GetCurrentClrDetails
3d corerun!GetCurrentClrDetails
3e corerun!GetCurrentClrDetails
3f corerun!GetCurrentClrDetails
40 KERNEL32!BaseThreadInitThunk
41 ntdll!RtlUserThreadStart

   1  Id: 2d00.ed4 Suspend: 1 Teb: 00000091`d5962000 Unfrozen ".NET EventPipe"
 # Call Site
00 ntdll!NtWaitForMultipleObjects
01 KERNELBASE!WaitForMultipleObjectsEx
02 coreclr!ds_ipc_poll
03 coreclr!ds_ipc_stream_factory_get_next_available_stream
04 coreclr!server_thread
05 KERNEL32!BaseThreadInitThunk
06 ntdll!RtlUserThreadStart

   2  Id: 2d00.24e4 Suspend: 1 Teb: 00000091`d5964000 Unfrozen ".NET Debugger"
 # Call Site
00 ntdll!NtWaitForMultipleObjects
01 KERNELBASE!WaitForMultipleObjectsEx
02 coreclr!DebuggerRCThread::MainLoop
03 coreclr!DebuggerRCThread::ThreadProc
04 coreclr!DebuggerRCThread::ThreadProcStatic
05 KERNEL32!BaseThreadInitThunk
06 ntdll!RtlUserThreadStart

   3  Id: 2d00.eec Suspend: 1 Teb: 00000091`d5966000 Unfrozen ".NET Finalizer"
 # Call Site
00 ntdll!NtWaitForSingleObject
01 KERNELBASE!WaitForSingleObjectEx
02 coreclr!GCEvent::Impl::Wait
03 coreclr!GCEvent::Wait
04 coreclr!WKS::GCHeap::WaitUntilGCComplete
05 coreclr!Thread::RareDisablePreemptiveGC
06 coreclr!Thread::DisablePreemptiveGC
07 coreclr!Thread::PulseGCMode
08 coreclr!JIT_MonEnter_Helper
09 coreclr!JIT_MonReliableEnter_Portable
0a 0x0

   4  Id: 2d00.2954 Suspend: 1 Teb: 00000091`d5980000 Unfrozen
 # Call Site
00 ntdll!NtWaitForWorkViaWorkerFactory
01 ntdll!TppWorkerThread
02 KERNEL32!BaseThreadInitThunk
03 ntdll!RtlUserThreadStart

#  5  Id: 2d00.310c Suspend: 1 Teb: 00000091`d5982000 Unfrozen
 # Call Site
00 ntdll!DbgBreakPoint
01 ntdll!DbgUiRemoteBreakin
02 KERNEL32!BaseThreadInitThunk
03 ntdll!RtlUserThreadStart

[kunalspathak]

Seems like related to #112203, #110350 and #105780

[kunalspathak]

@mangod9