Add new SOS test harness

[max-charlamb]

Summary

• add a consolidated, cross-host SOS test harness covering CDB, dotnet-dump, and LLDB
• exercise dump and live targets across runtime versions, deployment flavors, and DAC modes
• isolate dbgeng and dump capture in helper processes with replay artifacts for failures
• disable W^X for live EngineHost debuggees so bpmd can patch JIT code safely
• restore Unix executable permissions after pipeline artifact transfer for the MTP runner and debuggee apphosts

This branch is based on Lee Culver's leculver/sos-testing work and is opened as a draft to inspect and review the complete diff.

Validation

• dotnet build src\tests\SOS.Tests\SOS.Tests.csproj --no-restore --configuration Debug
• SOS.Tests.exe --filter-method SOS.Tests.GcWhereTests.GcWhere_Moves --parallel none: 24 passed

Follow-up

• prebuild SingleFile debuggees for CI instead of publishing them during test execution

[noahfalk]

The shape of this seems reasonable to me and if we want to make some adjustments that seems easier to do after we've gotten a little experience and we can make targetted diffs rather than trying tweak this giant PR. I think the main thing we should do before committing this is:

1. Make a little audit confirming the new testing appears to cover the same (or more) coverage than before
2. Add a small design/usage document that describes how the harness works, how it is run, what a canonical test case looks like and what canonical test output/test artifacts get generated.
3. Test the tests by having a new copilot session select at random 5 places where a buggy change in SOS is deliberately inserted. Then confirm that the new tests are at least as good at catching those issues as the previous ones.

[steveisok]

The shape of this seems reasonable to me and if we want to make some adjustments that seems easier to do after we've gotten a little experience and we can make targetted diffs rather than trying tweak this giant PR. I think the main thing we should do before committing this is:

1. Make a little audit confirming the new testing appears to cover the same (or more) coverage than before
2. Add a small design/usage document that describes how the harness works, how it is run, what a canonical test case looks like and what canonical test output/test artifacts get generated.
3. Test the tests by having a new copilot session select at random 5 places where a buggy change in SOS is deliberately inserted. Then confirm that the new tests are at least as good at catching those issues as the previous ones.

Pushed a commit with what you asked for. Once we get this in I'll start integrating pieces of what I have in my helix PR.

[Copilot]

🟡 Changes recommended

There is at least one confirmed correctness risk in the new debuggee snapshot helper (possible deadlock when redirecting process output) that should be addressed before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces a new consolidated SOS test harness and a large set of SOS command tests that run across multiple debugger hosts (cdb/dbgeng, dotnet-dump, lldb) and target flavors (framework-dependent, single-file, and desktop .NET Framework), including both dump-backed and live-debugging scenarios.

Changes:

• Add a new SOS.TestHarness library plus helper executables (EngineHost/Capturer) to drive hosts, capture/record replay context, and manage host lifetime constraints.
• Add a new SOS.Tests xUnit v3 test suite with broad SOS command coverage and matrix helpers for host/flavor/runtime/dump-kind/DAC combinations.
• Update debuggee build/publish infrastructure to support single-file publish outputs and runtime version stamping for DAC resolution.

File summaries

File	Description
src/tests/SOS.UnitTests/Debuggees/SosHarnessScenarios/TestHarness.cs	Adds a marker debuggee helper that no-ops in live mode and self-snapshots via dotnet-dump in capture mode.
src/tests/SOS.UnitTests/Debuggees/SosHarnessScenarios/SosHarnessScenarios.csproj	New scenarios debuggee project used by the SOS harness test matrix.
src/tests/SOS.UnitTests/Debuggees/Directory.Build.props	Writes a runtime.version file into publish outputs to carry pinned runtime patch info.
src/tests/SOS.Tests/WindowsTheoryAttribute.cs	Adds a Windows-only theory attribute to avoid “empty data” failures on non-Windows for cdb-only matrices.
src/tests/SOS.Tests/TestObjectHelpers.cs	Adds shared helper(s) for locating exact heap objects used by multiple tests.
src/tests/SOS.Tests/TestMatrices.cs	Adds reusable matrix builders/filters for known dump/runtime/DAC limitations.
src/tests/SOS.Tests/StackInspectionTests.cs	Adds tests for stack-inspection commands across hosts (including Windows-only dbgeng stackwalk cases).
src/tests/SOS.Tests/SpecializedInspectionTests.cs	Adds tests for SOS commands that require specific staged runtime state in the debuggee.
src/tests/SOS.Tests/SosTheoryAttribute.cs	Adds a cross-platform theory attribute intended to keep empty-matrix cases as failures.
src/tests/SOS.Tests/SOS.Tests.csproj	Introduces the new SOS test project and its source-generator wiring and Unix exec-bit fixup for MTP runner.
src/tests/SOS.Tests/RuntimeInfoTests.cs	Adds tests for runtime/module/process listing commands (eeversion, modules, runtimes, etc.).
src/tests/SOS.Tests/PrintExceptionTests.cs	Adds cross-host/flavor tests for printexception on a nested-exception crash dump.
src/tests/SOS.Tests/ObjectGcHelperTests.cs	Adds tests for per-object GC helper commands (dumpobjgcrefs, pathto, dumpalc, etc.).
src/tests/SOS.Tests/ObjectFieldsTests.cs	Adds tests validating dumpobj/dumpvc field decoding against debuggee-oracle values.
src/tests/SOS.Tests/NativeAddressSpaceTests.cs	Adds coverage for notreachableinrange under dotnet-dump.
src/tests/SOS.Tests/ModuleResolveTests.cs	Adds round-trip tests for name/token resolution (name2ee, token2ee).
src/tests/SOS.Tests/MiscCommandTests.cs	Adds smoke coverage for session/diagnostic commands (dbgout, sosflush, enummem gating).
src/tests/SOS.Tests/MemoryAndDecodeTests.cs	Adds tests for REPL memory dumpers and state decoders (threadstate/taskstate/dumpexceptions).
src/tests/SOS.Tests/LiveBpmdTests.cs	Adds a live-debugging test that validates raw bpmd breakpoint mechanics.
src/tests/SOS.Tests/HeapAnalysisTests.cs	Adds tests for heap-wide analysis commands (sizestats, traverseheap, ephrefs).
src/tests/SOS.Tests/GcWhereTests.cs	Adds tests for gcwhere including a live generation-promotion scenario.
src/tests/SOS.Tests/EeHeapTests.cs	Adds structural tests for eeheap -gc parsing including server-GC multi-heap coverage.
src/tests/SOS.Tests/DumpSigTests.cs	Adds combined smoke tests for signature decoding commands (dumpsig, dumpsigelem).
src/tests/SOS.Tests/DumpModuleTests.cs	Adds tests for dumpmodule structure and -mt type table correctness.
src/tests/SOS.Tests/DumpIlTests.cs	Adds IL decoding tests for dumpil including pointer-based decode variant.
src/tests/SOS.Tests/DumpHeapThinLockTests.cs	Adds tests for dumpheap -thinlock using a staged thin-lock debuggee object.
src/tests/SOS.Tests/DumpHeapStringsTests.cs	Adds content-based tests for dumpheap -strings and -strings -stat.
src/tests/SOS.Tests/DumpHeapObjectsTests.cs	Adds tests for dumpheap listing/statistics and selection flags (-stat/-type/-mt/-min/-max).
src/tests/SOS.Tests/DumpHeapLiveDeadTests.cs	Adds tests for dumpheap -live/-dead partitioning correctness.
src/tests/SOS.Tests/DumpHeapGenerationsTests.cs	Adds tests for dumpheap generation/segment/heap selection validated against eeheap ranges.
src/tests/SOS.Tests/DumpDomainTests.cs	Adds tests for dumpdomain/dumpassembly including runtime-shape differences.
src/tests/SOS.Tests/DumpDelegateTests.cs	Adds tests for delegate decoding (dumpdelegate) on known staged delegates.
src/tests/SOS.Tests/DumpArrayTests.cs	Adds tests for dumparray parameters and documented edge cases.
src/tests/SOS.Tests/DiagnosticCommandTests.cs	Adds tests for diagnostic/status commands including clrma host constraints.
src/tests/SOS.Tests/CollectionQueueTests.cs	Adds tests for concurrent-collection dumpers (dcq, threadpoolqueue).
src/tests/SOS.Tests/CodeInfoTests.cs	Adds tests for ehinfo/gcinfo including dump-kind selection where required.
src/tests/SOS.Tests/ClrUTests.cs	Adds Windows-only cdb tests for !clru shape and interleaving options.
src/tests/SOS.Tests/ClrThreadsTests.cs	Adds cross-matrix tests for clrthreads including parser validation.
src/tests/SOS.Tests/ClrThreadsParsing.cs	Adds a dedicated parser for the clrthreads table’s atypical header/layout.
src/tests/SOS.Tests/ClrStackLinesTests.cs	Adds tests asserting source line behavior in clrstack and suppression with -n.
src/tests/SOS.Tests/ClrStackICorDebugTests.cs	Adds tests for experimental clrstack -i and variable decoding (-i -a).
src/tests/SOS.Tests/ClrStackFullTests.cs	Adds tests for clrstack -f with cross-variant self-consistency checks.
src/tests/SOS.Tests/ClrStackFrameCountTests.cs	Adds tests for clrstack -c frame limiting semantics.
src/tests/SOS.Tests/ClrStackArgsLocalsTests.cs	Adds tests for clrstack -p/-l/-a with value and oracle cross-checks.
src/tests/SOS.Tests/ClrStackAllThreadsTests.cs	Adds tests for clrstack -all with deterministic worker-thread staging.
src/tests/SOS.TestHarness/Targets.cs	Adds memoized dump-session resolution and global teardown for shared hosts/sessions.
src/tests/SOS.TestHarness/Target.cs	Adds the unified Target abstraction plus replay recording hooks for navigation/commands.
src/tests/SOS.TestHarness/SosToken.cs	Adds tokenized matching/parsing for nondeterministic SOS output values.
src/tests/SOS.TestHarness/SosTable.cs	Adds a fixed-width SOS table parser plus rich assertion helpers.
src/tests/SOS.TestHarness/SosRow.cs	Adds row model supporting extra scalar columns and structured attached data.
src/tests/SOS.TestHarness/SosDataRowExtensions.cs	Adds assertion helpers for structured “internal data” record sets.
src/tests/SOS.TestHarness/SosDataRow.cs	Adds structured sub-record model used by table data extractors.
src/tests/SOS.TestHarness/SosDataExtractor.cs	Adds delegate contract for consuming “internal data” lines during parsing.
src/tests/SOS.TestHarness/SosConvert.cs	Adds shared numeric/boolean converters tied to tokens and rich failure context.
src/tests/SOS.TestHarness/SosColumn.cs	Adds column-spec model including alignment and optional per-column validation token.
src/tests/SOS.TestHarness/SosCell.cs	Adds typed parsing helpers and consistent failure messages for cell values.
src/tests/SOS.TestHarness/SosAssertException.cs	Adds a rich assertion exception that includes command/host/output context.
src/tests/SOS.TestHarness/SOS.TestHarness.csproj	Introduces the harness library project and its dependencies (DbgEng utilities, xUnit v3 core).
src/tests/SOS.TestHarness/ReplayContext.cs	Adds per-test replay capture (commands/navigations/host diagnostics) for failure artifacts.
src/tests/SOS.TestHarness/lldbhelper.py	Adds an LLDB scripting helper to provide reliable per-command framing and success status.
src/tests/SOS.TestHarness/LldbCliHost.cs	Adds the dump-backed LLDB host implementation and DAC selection wiring.
src/tests/SOS.TestHarness/Liveness.cs	Adds the dump vs live axis used by the test matrix.
src/tests/SOS.TestHarness/ILiveDebuggerHost.cs	Adds the interface contract for live, advancing debugger hosts.
src/tests/SOS.TestHarness/IDiagnosticHost.cs	Adds optional diagnostics-capture capability surface for child-process hosts.
src/tests/SOS.TestHarness/IDebuggerHost.cs	Adds the base host abstraction for SOS + raw debugger command execution.
src/tests/SOS.TestHarness/HostSlot.cs	Adds single-slot pooling/eviction mechanics for resource-constrained host backends.
src/tests/SOS.TestHarness/Hosts.cs	Adds the host axis enum (cdb/lldb/dotnet-dump) for matrix expansion.
src/tests/SOS.TestHarness/HostFactory.cs	Adds backend selection for dump and live hosts, including DAC-dir overrides.
src/tests/SOS.TestHarness/HarnessSkip.cs	Adds a harness-level dynamic skip mechanism for intrinsic host/flavor limitations.
src/tests/SOS.TestHarness/HarnessCancellation.cs	Bridges runner cancellation into blocking host waits for responsive Ctrl+C behavior.
src/tests/SOS.TestHarness/GcType.cs	Adds the workstation vs server GC axis for capture/launch and layout coverage.
src/tests/SOS.TestHarness/Flavor.cs	Adds the Core/SingleFile/Framework flavor axis used across targets and hosts.
src/tests/SOS.TestHarness/EngineProtocol.cs	Adds the command framing protocol for the dbgeng EngineHost child process.
src/tests/SOS.TestHarness/DumpKind.cs	Adds dump-kind axis (Heap/Mini/Full) for reduced-dump behavior coverage.
src/tests/SOS.TestHarness/DumpGenerationRequirements.cs	Adds Windows reduced-dump prerequisite handling and fallback/skip behavior.
src/tests/SOS.TestHarness/DeadTarget.cs	Adds a dump-backed cursor target that resolves memoized DumpSessions on navigation.
src/tests/SOS.TestHarness/DbgEngDumpHost.cs	Adds an in-process dbgeng dump host implementation.
src/tests/SOS.TestHarness/DbgEngCapturer.cs	Adds dbgeng-based dump capture for flavors that can’t self-collect.
src/tests/SOS.TestHarness/DacPolicy.cs	Adds the DAC selection policy including an environment-variable clamp.
src/tests/SOS.TestHarness/Dac.cs	Adds the legacy vs cDAC axis and its validity constraints (net11+).
src/tests/SOS.TestHarness/CoreVersion.cs	Adds the core runtime version axis used to fan out builds/installs and disable versions.
src/tests/SOS.TestHarness/CaptureCli.cs	Adds the Capturer CLI entrypoint for out-of-process dump capture.
src/tests/SOS.TestHarness.SourceGen/SOS.TestHarness.SourceGen.csproj	Introduces the source generator project for mirroring debuggee constants into tests.
src/tests/SOS.TestHarness.SourceGen/MirrorGenerator.cs	Implements the incremental source generator for compile-time “oracle” constants.
src/tests/SOS.TestHarness.EngineHost/SOS.TestHarness.EngineHost.csproj	Introduces the dbgeng EngineHost child process project.
src/tests/SOS.TestHarness.EngineHost/Program.cs	Implements the EngineHost stdin/stdout REPL for dump/live dbgeng operations.
src/tests/SOS.TestHarness.Capturer/SOS.TestHarness.Capturer.csproj	Introduces the Capturer helper executable project.
src/tests/SOS.TestHarness.Capturer/Program.cs	Wires the Capturer exe entrypoint to the capture CLI.
src/tests/Debuggees.proj	Adds a post-build publish step to produce single-file SOS debuggees for the harness to consume.

Review details

• Files reviewed: 120/120 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[hoyosjs]

the stdin/out feedback is fair. The document is a little verbose, but a fair documentation of the env vars and the example summarizes most of the needed info anyway

[noahfalk]

👍

The copilot review comment about deadlocks redirecting output should probably be addressed.

[steveisok]

/ba-g known issues #5996 and #5997