[CreateDump] Use _exit after createdump launch failures

[mdh1418]

Fixes #133733

Use _exit(EXIT_FAILURE) instead of exit(-1) when the forked createdump launcher child fails before completing execve.

On Unix, CoreCLR forks a temporary child and expects it to replace itself with createdump. If the parent/child synchronization handshake fails or execve(createdump) fails, the child is still a forked copy of the managed runtime.

Calling exit() in that child runs inherited process destructors. In particular, the PAL process-shutdown destructor invokes the CoreCLR shutdown callback, which cleans up the debugger transport and diagnostic server. Because the child shares the parent's filesystem namespace, that cleanup unlinks paths belonging to the still-running parent:

/tmp/clr-debug-pipe-<pid>-<key>-in
/tmp/clr-debug-pipe-<pid>-<key>-out
/tmp/dotnet-diagnostic-<pid>-<key>-socket

The parent runtime remains alive and retains its diagnostic listener file descriptor, but new diagnostic clients cannot connect after the Unix-domain socket pathname has been removed.

Cleanup behavior before this change

On a child failure:

forked child
    |
    | handshake or execve failure
    v
exit(-1)
    |
    | runs inherited destructors
    v
PAL shutdown callback
    |
    +-- closes the child's inherited descriptor references
    +-- runs debugger transport cleanup
    +-- runs diagnostic server shutdown
    +-- unlinks debugger pipe and diagnostic socket paths

The kernel eventually closed the child's file descriptors when the child terminated, but exit() first ran userspace cleanup that belonged to the parent runtime. Unlinking the shared endpoint paths affected the parent even though closing the child's descriptor references alone would not have affected it.

Cleanup behavior after this change

On a child failure:

forked child
    |
    | handshake or execve failure
    v
_exit(EXIT_FAILURE)
    |
    +-- kernel closes the child's file descriptors
    +-- kernel releases the child's mappings and other process resources
    +-- parent observes EOF on the child stderr pipe
    +-- parent reaps the child with waitpid()
    |
    +-- does not run inherited destructors
    +-- does not invoke the PAL shutdown callback
    +-- does not unlink the parent's diagnostic endpoint paths

The child still closes all of its file descriptor references as part of kernel process termination. The difference is that it no longer performs inherited userspace teardown against copied runtime state.

On a successful execve, the existing SOCK_CLOEXEC and FD_CLOEXEC flags continue to close inherited diagnostic descriptors in the new createdump process. This change affects only failure paths before a successful exec.

Both forked-child failure paths are updated:

1. Failure to read the one-byte synchronization signal from the parent.
2. Failure to execute the external createdump binary.

Validation

The issue was reproduced by running a target with its colocated createdump set to mode 0644.

Before the change:

1. dotnet-dump collect reached the target runtime.
2. execve(createdump) failed with EACCES.
3. The target process remained alive.
4. The target's default diagnostic socket disappeared.

After the change:

1. Two consecutive dump requests failed with the expected execute-permission error.
2. The target process remained alive after both requests.
3. The default diagnostic socket remained present after both requests.
4. The second request successfully connected to the same runtime, confirming that the listener remained reachable.
5. After restoring execute permission, a subsequent minidump completed successfully.
6. The diagnostic socket remained present after successful collection.

CoreCLR Release was rebuilt successfully with the change.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

[Copilot]

🟡 Changes recommended

The analogous NativeAOT Unix failure paths still need the same fix or explicit scoping.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR prevents failed CoreCLR createdump launcher children from running inherited shutdown cleanup that can remove the parent’s diagnostic endpoints.

Changes:

• Replaces exit(-1) with _exit(EXIT_FAILURE) for handshake and execve failures.
• Preserves diagnostic sockets and pipes when launches fail.
• NativeAOT’s analogous Unix paths remain unchanged.

File summaries

File	Description
src/coreclr/pal/src/thread/process.cpp	Safely terminates failed CoreCLR createdump launcher children.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The reviewed failure paths now safely terminate without affecting the parent runtime.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Lite

[steveisok]

/backport to release/11.0

Started backporting to release/11.0 (link to workflow run)