Skip to content

Port ILU to GPU - #2858

Merged
pcarruscag merged 24 commits into
developfrom
pedro/ilu_gpu
Aug 10, 2026
Merged

Port ILU to GPU#2858
pcarruscag merged 24 commits into
developfrom
pedro/ilu_gpu

Conversation

@pcarruscag

@pcarruscag pcarruscag commented Aug 3, 2026

Copy link
Copy Markdown
Member

PR Checklist

  • I am submitting my contribution to the develop branch.
  • My contribution generates no new compiler warnings (try with --warnlevel=3 when using meson).
  • My contribution is commented and consistent with SU2 style (https://su2code.github.io/docs_v7/Style-Guide/).
  • I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
  • I have added a test case that demonstrates my contribution, if necessary.
  • I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp), if necessary.

pcarruscag and others added 3 commits August 3, 2026 22:21
@pcarruscag
update regressions
da08b49
@pcarruscag @claude
GPU ILU: level-scheduled triangular solves, colored build only
62a0c23 
Colored-iterative forward/backward substitution needed too many sweeps
to be useful on stiffer systems (e.g. large RANS cases with highly
anisotropic near-wall cells), so the triangular solves go back to
exact, level-scheduled kernels (one pass, no sweep count). The
factorization stays colored/iterative, which converges quickly
regardless of matrix stiffness.

LINEAR_SOLVER_ILU_GPU_SWEEPS is now a single scalar (build sweeps
only) instead of a 3-entry array, since the solves no longer sweep.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Group ILU options into CConfig::CIluOptions, single accessor
bcda669 
Consolidates LINEAR_SOLVER_ILU_FILL_IN, LINEAR_SOLVER_ILU_LEVEL_SCHEDULING,
and LINEAR_SOLVER_ILU_GPU_SWEEPS behind one struct/getter (GetIluOptions),
following the same pattern already used for other option families
(e.g. CMGOptions). Config keys and defaults are unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pcarruscag
pcarruscag commented Aug 8, 2026
View reviewed changes
Comment thread Common/include/linear_algebra/CSysMatrix.hpp Outdated
Comment thread Common/include/linear_algebra/CSysMatrix.hpp Outdated
Comment thread Common/include/linear_algebra/CSysMatrix.hpp Outdated
Comment thread Common/include/linear_algebra/CSysMatrix.hpp Outdated
Comment thread Common/include/linear_algebra/CSysMatrix.hpp Outdated
Comment thread Common/src/linear_algebra/CSysMatrix.cpp Outdated
Comment thread Common/src/linear_algebra/CSysMatrix.cpp Outdated
Comment thread Common/src/linear_algebra/CSysMatrix.cpp
Comment thread Common/src/linear_algebra/CSysMatrixGPU.cu Outdated
Comment thread Common/src/linear_algebra/CSysMatrixGPU.cu Outdated
pcarruscag and others added 7 commits August 8, 2026 12:35
@pcarruscag @claude
Address PR #2858 review comments
33b3c25 
- CSysMatrix.hpp: trim member-doc comments down to what the members
  are, drop design/alternatives narrative and CUDA-graph rationale
  duplicated at the call sites.
- CSysMatrix.cpp: factor repeated GPU LDU free/alloc sequences into
  local helper lambdas (freeLDU, reused GPUAllocAndInit/GPUAllocAndCopy);
  factor the repeated "GPU dispatch unavailable" error into one
  GPUNotAvailable(CURRENT_FUNCTION) helper.
- Move the static nVar==nEqn / nVar*nVar<=1024 validity checks for the
  GPU Jacobi/ILU/SpMV paths from every Build/Compute call into
  Initialize(), where CUDA and the block size are already known and
  the check only needs to run once.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Deduplicate host LDU free in CSysMatrix destructor
cde3315 
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Multi-seed RCM reordering to reduce ILU level-scheduling depth
2e24ccf 
Single-seed RCM grows from one peripheral, minimum-degree node, so the
number of BFS levels tracks the mesh graph diameter. Since the GPU ILU
triangular solves are level-scheduled (one kernel launch per level),
this produces hundreds of narrow, serialized launches on large meshes.

Adds RCM_NUM_SEEDS (default 1, unchanged behavior) to seed the RCM
growth from multiple fronts chosen by farthest-point sampling, which
bounds the number of levels by the seed set's covering radius instead
of the full component diameter while keeping the ordering locally
RCM-like around each front.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Make multi-seed RCM farthest-point sampling incremental
2a7b466 
Picking numSeeds seeds by restarting a full multi-source BFS for each
one costs O(numSeeds * (V+E)) and dominates SetRCM_Ordering for large
numSeeds. Maintaining the nearest-seed distance incrementally instead
- relaxing only the points a new seed brings strictly closer - reuses
prior work across rounds and keeps the total cost close to a single
BFS. Measured on oneram6 (38k points): 121ms -> 11ms for 128 seeds.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Report GPU ILU color/level scheduling stats, like edge coloring does
a05fe64 
Prints, once per rank-reduced worst case, the number of colors used
to schedule the factorization and the number of levels used to
schedule the triangular solves, plus their average group size. Narrow
levels are the direct symptom of a mesh needing more RCM_NUM_SEEDS.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag @claude
Print GPU ILU scheduling stats only once, shorten the message
4fb0272 
Initialize() runs once per multigrid level, so the report was printed
once per level; a function-local static now gates it to a single
print per run. Also trims the message to the two numbers that matter.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pcarruscag
Merge remote-tracking branch 'origin/develop' into pedro/ilu_gpu
e0f5c4a
@pcarruscag
pcarruscag merged commit 9405bb5 into develop Aug 10, 2026
38 checks passed
@pcarruscag
pcarruscag deleted the pedro/ilu_gpu branch August 10, 2026 04:10
@ddg93 ddg93 mentioned this pull request Aug 18, 2026
Merged
9 tasks
pcarruscag pushed a commit that referenced this pull request Sep 1, 2026
@ddg93 @bigfooted
[GSoC] Feature: LU-SGS preconditioner to GPU (#2872)
07aa46b 
This PR ports the LU-SGS preconditioner to GPU

## Proposed Changes
This PR introduces a build phase for the LU-SGS preconditioner where the
inverse of the diagonal D is calculated to facilitate the compute phase.
Then, forward and backward sweeps are implemented through custom CUDA
kernels matching the logic of the CPU section (except for the inverse of
D, which is directly multiplied in the GPU logic). These kernels are
managed by the compute method on GPU which maintains the MPI
communications.

## Related Work
This PR follows the discussion of #2843 . Kernel level dependencies are
managed as for the ILU preconditioner proposed in #2858 .

## Validation
CPU and GPU implementations of the LU-SGS preconditioner are compared
for validation of this PR on the rae2822 test-case. After one iteration,
the first, preconditioned Krylov vector is compared finding matching
results down to machine precision. Residuals are compared after 100
iterations finding matching results.

## TO-DO
- [X] complete with the quantized_mode option path;
- [ ] run benchmarks on modern GPUs. Current evaluations on a P620 are
inconclusive;
- [ ] profiling of the custom kernels;

## PR Checklist
*Put an X by all that apply. You can fill this out after submitting the
PR. If you have any questions, don't hesitate to ask! We want to help.
These are a guide for you to know what the reviewers will be looking for
in your contribution.*

- [X] I am submitting my contribution to the develop branch.
- [X] My contribution generates no new compiler warnings (try with
--warnlevel=3 when using meson).
- [X] My contribution is commented and consistent with SU2 style
(https://su2code.github.io/docs_v7/Style-Guide/).
- [X] I used the pre-commit hook to prevent dirty commits and used
`pre-commit run --all` to format old commits.
- [ ] I have added a test case that demonstrates my contribution, if
necessary.
- [ ] I have updated appropriate documentation (Tutorials, Docs Page,
config_template.cpp), if necessary.

---------

Co-authored-by: Nijso <bigfootedrockmidget@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog:feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@pcarruscag