[Tests] Add deterministic InProc Crash Reporter tests - #134228
Merged
mdh1418 merged 12 commits intoSep 25, 2026
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Link the production PAL/reporter archive, initialize a private PAL instance, and validate deterministic JSON and compact output on desktop Unix and Android. Introduce isolated desktop runners, invocation-owned output, and bounded failure diagnostics in the shared harness. Retain failed outputs and collect Android archives through XHarness. Use standard C++ support and route native Mono/NativeAOT exclusions before PAL dependency checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Extend the shared fixtures and assertions with additional fatal-report shapes: native-only SIGABRT and a supplied compressed stack-overflow trace. Add matching desktop and Android projects and document their output expectations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Reuse the rich fixture to verify compact output remains available when lifecycle file output is disabled, without creating a report directory. Add matching desktop and Android projects and document the output expectation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Reuse the shared rich assertions for repeated caller-sink reports with changing signals. Cover null and nested requests, failed sinks and subsequent recovery, generation without services, and isolation from an enabled lifecycle sink. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Hold JSON and compact-report owners in an output callback while independent threads request both formats. Require immediate rejection without callbacks, preserve owner output, and verify recovery after owner success or sink failure. Use bounded handshakes and the production reporter without product changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Exercise signal-dispatch rejection while successful and failing on-demand owners hold the shared reporter guard. Finish the existing isolated OnDemand process with a signal-shaped owner, validating output and on-demand rejection both during and after signal reporting. Reuse bounded coordination and platform capture without product hooks or real signals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
Intentionally fail only RichSigsegv after its report assertions pass, before successful-run cleanup. Include an incomplete-file marker to exercise recursive retention alongside the real compact and JSON reports. Keep the expected success exit code unchanged so CI reports a genuine failure. Revert this entire commit after confirming downloadable Linux and Android Helix artifacts, then require normal passing CI before merging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: aa976a9e-b875-4524-82e1-1485d03d14fa
|
Azure Pipelines: Successfully started running 4 pipeline(s). 12 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The unconditional failure probe causes the RichSigsegv test and required suite to fail.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
What changed in this PR
Adds deterministic Unix and Android CoreCLR in-process crash reporter tests using managed/native fixtures, report validation, concurrency checks, and CI artifact handling.
Changes:
- Adds signal, stack-overflow, console-only, abort, and on-demand scenarios.
- Integrates native PAL-linked drivers and Android log capture.
- Wires tests into discovery, packaging, and Helix artifact collection.
- Includes an unconditional temporary failure probe in
Program.csthat must be removed or gated.
| File | Summary |
|---|---|
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/StackOverflow/Android.Device_Emulator.InProcCrashReport.StackOverflow.Test.csproj |
Android StackOverflow test adapter. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/Shared/InProcCrashReport.Common.props |
Shared Android build configuration. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/Shared/config.h |
Android platform definitions. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/Shared/android_log_interpose.h |
Android log capture declarations. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/Shared/android_log_interpose.c |
Android log interception implementation. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/RichSigsegv/Android.Device_Emulator.InProcCrashReport.RichSigsegv.Test.csproj |
Android rich-report adapter. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/OnDemand/Android.Device_Emulator.InProcCrashReport.OnDemand.Test.csproj |
Android on-demand adapter. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/ConsoleOnly/Android.Device_Emulator.InProcCrashReport.ConsoleOnly.Test.csproj |
Android console-only adapter. |
src/tests/FunctionalTests/Android/Device_Emulator/InProcCrashReport/Abort/Android.Device_Emulator.InProcCrashReport.Abort.Test.csproj |
Android abort adapter. |
src/tests/build.sh |
Passes runtime and build settings to CMake. |
src/tests/baseservices/exceptions/inproccrashreport/StackOverflow/InProcCrashReport.StackOverflow.csproj |
Unix StackOverflow test project. |
src/tests/baseservices/exceptions/inproccrashreport/Shared/Program.cs |
Managed harness and report validation; contains the unconditional failure probe. |
src/tests/baseservices/exceptions/inproccrashreport/Shared/InProcCrashReport.Unix.props |
Unix test configuration. |
src/tests/baseservices/exceptions/inproccrashreport/Shared/inproccrashreport_test_driver.cpp |
Synthetic native reporter driver. |
src/tests/baseservices/exceptions/inproccrashreport/Shared/CMakeLists.txt |
Native test build and linking. |
src/tests/baseservices/exceptions/inproccrashreport/RichSigsegv/InProcCrashReport.RichSigsegv.csproj |
Unix rich-report test project. |
src/tests/baseservices/exceptions/inproccrashreport/README.md |
Test scope and execution documentation. |
src/tests/baseservices/exceptions/inproccrashreport/OnDemand/InProcCrashReport.OnDemand.csproj |
Unix on-demand test project. |
src/tests/baseservices/exceptions/inproccrashreport/ConsoleOnly/InProcCrashReport.ConsoleOnly.csproj |
Unix console-only test project. |
src/tests/baseservices/exceptions/inproccrashreport/Abort/InProcCrashReport.Abort.csproj |
Unix abort test project. |
src/tasks/AndroidAppBuilder/Templates/CMakeLists-android.txt |
Enables C++ Android test sources. |
src/libraries/tests.proj |
Includes Android CoreCLR tests. |
src/libraries/sendtohelix-mobile.targets |
Configures Android failure artifact retrieval. |
This was referenced Sep 19, 2026
Contributor
|
Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag |
Contributor
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure |
Member
Author
|
/azp run runtime-android |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
lateralusX
reviewed
Sep 23, 2026
Remove the deliberate RichSigsegv failure and incomplete-file marker from normal test runs. Preserve the artifact collection work and probe on the local follow-up branch instead. This reverts commit f3b2d63. Generated by GitHub Copilot. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude Android crash reporter tests on every non-CoreCLR runtime. Use ELF linker wrapping for Android log capture and forward through __real___android_log_write instead of runtime symbol interposition and dlsym. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the shared validator and private production-PAL fixture in separate iOS, iOS simulator, and MacCatalyst arm64 app bundles. Package the matching native dylib using existing AppleAppBuilder support and keep desktop and Android behavior unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Register the five CoreCLR arm64 scenarios outside smoke coverage, preserve exit code 100 and Catalyst sandbox settings in archived builds, and document local execution and platform limits. Use the ordinary XHarness runner; defer app-owned failure-file collection to a follow-up. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The Unix native test target is missing minipal linkage, and the temporary-probe documentation is stale.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
Resolved since last review (1)
Member
Author
|
/azp run runtime-android,runtime-ioslike |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
This was referenced Sep 24, 2026
Member
Author
|
/azp run runtime-ioslikesimulator,runtime-maccatalyst |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add shared Unix, Android, and Apple mobile tests for CoreCLR's in-process crash reporter. Fixed thread, stack, exception, and register data provide stable expectations for checking report contents, output destinations, and the handling of repeated, failed, or overlapping requests.
Test structure
The tests pair a managed harness with a native library linked against CoreCLR's built PAL archive, which contains the reporter and its output components. The native driver supplies the fixed crash data through the reporter's callbacks. The managed harness invokes report generation and checks the resulting fields, frames, and files.
Signal-path tests call the reporter's dispatcher directly, allowing the process to continue so the harness can inspect its output. The test library keeps its PAL and reporter state private, so its callbacks are independent of the hosting runtime's callbacks. Each test project runs in a separate process to isolate the reporter's process-wide state.
Signal-path reporting writes JSON to a file and human-readable text to stderr on Unix or logcat on Android. The harness captures stderr or intercepts Android logging calls to validate that text alongside the JSON file. On-demand reporting delivers either format through a caller-provided callback; the test callbacks save the received bytes for the same report checks.
Test projects
The suite has matching Unix, Android, and Apple mobile projects for the following coverage:
RichSigsegvSIGSEGV: three thread records, interleaved managed/native frames, generic names, exact registers and frame metadata, managed exception details, and module associations.AbortSIGABRT, with a native-only crash stack and absent managed exception fields.StackOverflowConsoleOnlyOnDemandThese are surviving component tests: they do not raise fatal signals or cause an actual stack overflow.
On-demand checks
An on-demand caller chooses JSON or human-readable text and supplies a callback to receive the report bytes. The callback returns
trueto continue receiving output orfalseto stop. The reporter must stop calling a receiver that returnsfalse, report that generation failed, and remain available for a later request.OnDemandtests this contract using the rich thread and stack data fromRichSigsegv. It supplies callbacks that either save all the output or deliberately returnfalse. Sequential requests check both formats, recovery after stopped output, rejection of a missing callback, and rejection of a nested request made from inside an output callback.The initial JSON request runs before file-output services are initialized. Later requests run with those services enabled and verify that on-demand output still goes to the caller's callback while the reporter's JSON directory remains empty.
The concurrency checks test whether another request can interfere with a report already in progress. The test pauses that report inside a callback, then makes the following requests on other threads against the same reporter:
falsewithout invoking its output callback or writing bytes.falsewithout invoking its output callback or writing bytes.Each new request must return before the paused report resumes. The on-demand-first checks run with both JSON and text output, and with callbacks that either accept output or stop the report when it resumes. After either outcome, fresh requests in both formats must succeed. Completed reports are checked for the expected contents.
The signal-report-first check runs last. After rejecting the overlapping requests, that report must finish with valid JSON and console/logcat output. Further on-demand requests must remain rejected because signal-triggered reporting leaves the reporter unavailable for reuse; the process would normally terminate afterwards. All thread coordination uses explicit notifications and bounded waits.
Platform and CI integration
Linux and macOS use regular CoreCLR runtime test discovery. Android and Apple mobile use thin functional-test adapters that reuse the same managed validator and native driver. The native fixture links a private instance of the production PAL/reporter, separate from the hosting CoreCLR.
Android packages the tests into APKs and uses linker wrapping to capture reporter log output while forwarding it to Android's normal logging implementation. AppleAppBuilder packages and signs the fixture dylib for iOS, iOS simulator, and MacCatalyst arm64.
The mobile projects participate in full-suite CoreCLR archive builds, not the default smoke selection. Mono and NativeAOT are excluded. Apple run-only work items explicitly expect exit code 100, and archived builds preserve the requested MacCatalyst App Sandbox setting.
Desktop failures retain report files in the Helix upload directory. Android retrieves a failure archive before uninstalling the app. Apple currently retains standard XHarness logs, but app-owned report files are not copied to the host before cleanup. The Apple tests validate the original JSON and console output inside the app; host-side retrieval is not required for their assertions.
Validation
Local Apple validation
OnDemandruns in fresh processes.OnDemandruns in fresh processes.Simulator and Catalyst results do not establish physical-device execution or provisioning support. No Apple target was substituted with Mono or NativeAOT.
Temporary intentional validation failures on simulator and Catalyst confirmed that failed assertions remain failures through the normal XHarness runner. The failure injection has been removed. After the review fixes and probe removal, all five desktop macOS scenarios and
RichSigsegvon simulator and sandboxed Catalyst were rebuilt and passed.Linux and Android validation
Earlier validation passed the full suite on Linux x64 and an Android x64 emulator, including the on-demand concurrency checks. Ten additional Linux runs repeated those checks successfully.
After switching Android log capture to linker wrapping, the wrapper was compiled for arm64 and x64 using AndroidAppBuilder's C warning policy. A focused native regression ran on an Android arm64 emulator against the real Android logging library, checking capture, filtering, forwarding, buffer limits, and reset behavior.
The full Android CoreCLR APK suite was not rebuilt or rerun after that change because a matching Android CoreCLR product baseline was unavailable in the validation worktree. Linux was not rerun during the Apple/review-fix validation.
Archive and runner inspection
All 15 Apple input archives and generated Helix work-item metadata were inspected, including expected exit code 100, platform selection, fixture inclusion, relocation-sensitive inputs, and MacCatalyst sandbox forwarding.
Generated build/run scripts for
RichSigsegvandOnDemandwere replayed successfully from relocated archives on simulator and Catalyst. These are local results, not evidence of hosted Helix execution or hosted failure-artifact collection.Follow-up coverage
Apple failure-artifact retrieval: Copy app-owned JSON reports, captured console output, and incomplete files to the host before app cleanup, then use the existing host-to-Helix upload path. Validate retrieval with intentional failures while preserving the failing result. This PR retains standard XHarness logs but does not implement that file retrieval.
PAL dependency sharing and configuration consistency: Share or export PAL link-dependency information instead of maintaining a parallel platform-specific dependency list in the test fixture. The solution must also keep the test link configuration consistent with the matching prebuilt PAL's actual system-versus-bundled-libunwind choice. Extracting the list into a shared helper alone does not guarantee that agreement. This build-system work is deferred, including validation of the relevant Linux configurations and any installation or artifact-distribution changes it requires.
Additional component tests: Extend coverage of report-file cleanup, retention limits, and filesystem errors.
Fatal-path integration tests: Cover actual fatal signals, runtime stack walking, concurrent fatal signals, watchdog termination, and GC-state crashes using an external controller that can inspect output after the target exits. Deterministic GC-state crashes may require GC test hooks.