ci: Backport earthflowAI solution to unbind ATS launches and run CTest in parallel - #4118
Merged
Merged
Conversation
rrsettgast
requested review from
dkachuma,
herve-gross and
jhuang2601
as code owners
August 14, 2026 19:49
rrsettgast
requested review from
MelReyCG,
joshua-white and
wrtobin
as code owners
August 14, 2026 19:49
rrsettgast
requested review from
cssherman,
frankfeifan,
jafranc,
kimtaeho07 and
tjb-ltk
as code owners
August 14, 2026 23:07
This was referenced Aug 15, 2026
Contributor
Author
|
@joshua-white I think this is ready to merge unless there is something else to add back in from your streak2 work. |
victorapm
self-requested a review
August 15, 2026 15:37
victorapm
approved these changes
Aug 15, 2026
victorapm
left a comment
Contributor
There was a problem hiding this comment.
Thank you for the improvements!
joshua-white
approved these changes
Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GEOS ATS starts each integrated test through an independent Open MPI
launcher. For serial tests, each launcher applied its default binding
policy independently and selected the first available core. As a result,
many concurrent one-rank tests could all be pinned to the same core on
the streak2-32core runner despite ATS accounting for 32 available slots.
Verify that the integrated-test launcher is Open MPI and disable its
hwloc binding policy before invoking geos_ats.sh. This leaves each rank
unbound within the container's available CPU set so the Linux scheduler
can use all available cores. Enable Open MPI binding reports so the
effective placement remains visible in retained ATS logs.
Also:
additional OpenMP thread pool;
Add GEOS_CTEST_PARALLEL_LEVEL to control parallel CTest execution:
Preserve an explicitly supplied CMAKE_CTEST_ARGUMENTS value and use the
resulting arguments for both the generated build_test target and CI test
execution.
Set the CTest PROCESSORS property from NUM_MPI_TASKS so parallel CTest
accounts for the full rank count of MPI tests instead of treating every
test as a single scheduling slot.
Parallel execution exposed tests that created and removed identically
named PVT table files from shared working directories. Give the affected
constitutive and wells test executables unique scratch filenames, and
give the BlackOil fixture a unique temporary directory. Include the
newer thermal-estimator well tests, which have the same shared-file
behavior but were not present in the original EarthFlowAI change.
This removes Open MPI's forced first-core binding for independent ATS
launchers. It intentionally does not provide exclusive core allocation;
the operating system remains responsible for scheduling unbound ranks
within the runner's available CPU set.
stats:
on rocky gcc13 (cpu), unit tests went from
397 sto71 sintegratedTests go from
43mmissing some tests, to18mwith all tests re-enabled.