Skip to content

Add CUDA NEP backend and GPU neighbor lists - #8005

Closed
19hello wants to merge 1 commit into
deepmodeling:developfrom
19hello:pr/nep-gpu
Closed

19hello wants to merge 1 commit into
deepmodeling:developfrom
19hello:pr/nep-gpu

Conversation

@19hello

@19hello 19hello commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds a CUDA NEP backend and GPU neighbor-list implementation for molecular dynamics.

Main changes:

  • Add the optional CUDA NEP backend under source/source_esolver/nep_gpu.
  • Add GPU neighbor-list construction under source/source_cell/module_neighlist/gpu_neighlist.
  • Split the existing CPU neighbor-list implementation into cpu_neighlist.
  • Move domain_decomposition directly under source/source_cell.
  • Add the ENABLE_NEP_GPU CMake option.
  • Add a silicon NEP GPU validation case.

Implementation

The GPU NEP backend provides two main interfaces:

  • initialize()

    • Selects the CUDA device.
    • Creates the GPU-side NEP model.
    • Uploads NEP parameters to the GPU.
  • compute()

    • Prepares coordinates and atom types.
    • Builds or reuses the GPU neighbor list.
    • Evaluates NEP descriptors on the GPU.
    • Computes radial forces, angular forces, energy, and virial.
    • Copies the results back to the MD layer.

The GPU neighbor-list implementation uses spatial binning and CUDA kernels for:

  • Bin construction.
  • Candidate-neighbor search.
  • Neighbor-count reduction.
  • Neighbor-index filling.
  • Candidate-list filtering.

Build

CUDA NEP can be enabled with:

cmake -S . -B build_nep_gpu \
  -DUSE_CUDA=ON \
  -DENABLE_NEP_GPU=ON \
  -DENABLE_LCAO=OFF \
  -DENABLE_LIBRI=OFF \
  -DNEP_DIR=<NEP installation path> \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.0/bin/nvcc \
  -DCMAKE_CUDA_ARCHITECTURES=75

## Verification

Environment:

- GPU: NVIDIA GeForce RTX 2080 Ti
- CUDA Toolkit: 12.0.76
- CUDA architecture: 75
- MPI processes: 1
- OMP_NUM_THREADS=1

Completed tests:

- GPU NEP initialization and MD execution.
- GPU neighbor-list path with 100,000 atoms.
- CPU neighbor-list plus GPU NEP with 100,000 atoms.
- CPU neighbor-list plus GPU NEP with 500,000 atoms.
- CPU neighbor-list plus GPU NEP with 1,000,000 atoms.
- CPU and GPU NEP energy results matched for the validation case.
- git diff --check passed.
- ABACUS governance check passed with header-dependency warnings only.

The large-scale MD tests used:

md_nstep          5
md_neighbor_skin  0
device            gpu
esolver_type      nep
OMP_NUM_THREADS   1
MPI processes      1

## Performance observation

For the CPU-neighbor-list plus GPU-NEP path:

500,000 atoms:
    Total time:              89.49 s
    CPU neighbor preparation: 86.42 s
    GPU NEP compute:           1.65 s

1,000,000 atoms:
    Total time:              176.50 s
    CPU neighbor preparation: 170.80 s
    GPU NEP compute:           2.95 s

With md_neighbor_skin=0, the neighbor list is rebuilt at every MD step.

## Known limitations

- The GPU neighbor-list implementation still performs some bin-offset preparation on the host.
- The GPU backend currently supports potential-only, non-ZBL NEP models.
- CPU and GPU neighbor lists are separate implementations. The GPU NEP path uses its own device-resident neighbor list.
- GPU-specific unit tests and broader numerical regression tests should be expanded in follow-up work.

## Build note

The previously configured ENABLE_LCAO=OFF incremental build reached the link stage but failed because existing module_current/td_current_io*.cpp
objects require LCAO-only symbols. This is unrelated to the GPU NEP changes. The GPU executable was built successfully in the earlier GPU validation
build and was used for the runtime tests described above.

@19hello
19hello force-pushed the pr/nep-gpu branch 2 times, most recently from 778985a to 8a2381b Compare September 23, 2026 08:30
@19hello 19hello closed this Sep 24, 2026
Critsium-xy added a commit to Critsium-xy/abacus-develop that referenced this pull request Sep 25, 2026
.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 253 of them.

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (deepmodeling#8000, deepmodeling#7924, deepmodeling#7906, deepmodeling#8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Critsium-xy added a commit to Critsium-xy/abacus-develop that referenced this pull request Sep 27, 2026
.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 253 of them.

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (deepmodeling#8000, deepmodeling#7924, deepmodeling#7906, deepmodeling#8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Critsium-xy added a commit to Critsium-xy/abacus-develop that referenced this pull request Sep 27, 2026
.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 252 of them
(253 when prepared; source_lcao/setup_dm.h has since been deleted
upstream by deepmodeling#8000).

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (deepmodeling#8000, deepmodeling#7924, deepmodeling#7906, deepmodeling#8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Critsium-xy added a commit to Critsium-xy/abacus-develop that referenced this pull request Sep 28, 2026
.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 248 of them.

253 files were selected when this was prepared. Since then, develop has
deleted source_lcao/setup_dm.h (deepmodeling#8000) and
source_pw/module_stodft/hamilt_sdft_pw.cpp (deepmodeling#8012), and already converted
source_base/module_out/binstream.{h,cpp} (deepmodeling#8025) and
source_pw/module_stodft/sto_hamilt_pw.h (renamed from hamilt_sdft_pw.h
in deepmodeling#8012), which leaves 248. The conversion follows files that develop
moved, e.g. onsite_proj_tools_stress.cpp is now under module_proj/
(deepmodeling#8007).

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (deepmodeling#8000, deepmodeling#7924, deepmodeling#7906, deepmodeling#8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Critsium-xy added a commit to Critsium-xy/abacus-develop that referenced this pull request Sep 30, 2026
.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 246 of them.

253 files were selected when this was prepared. Since then, develop has
deleted two of them and already converted five others, which leaves 246:

  - deleted: source_lcao/setup_dm.h (deepmodeling#8000),
    source_pw/module_stodft/hamilt_sdft_pw.cpp (deepmodeling#8012);
  - already converted: source_base/module_out/binstream.{h,cpp} (deepmodeling#8025),
    source_pw/module_stodft/sto_hamilt_pw.h (renamed from
    hamilt_sdft_pw.h in deepmodeling#8012), source_io/module_ctrl/ctrl_output_pw.h
    and source_pw/module_pwdft/op_pw_nl.cpp (deepmodeling#8043).

Files that develop moved are converted at their new path, e.g.
onsite_proj_tools_stress.cpp is now under source_pw/module_proj/
(deepmodeling#8007).

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (deepmodeling#8000, deepmodeling#7924, deepmodeling#7906, deepmodeling#8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mohanchen pushed a commit that referenced this pull request Sep 30, 2026
…o spaces (#8011)

* docs: fix two @file tags that name a different file

Doxygen's @file takes the name of the file it documents. These two
name a file that does not exist, so Doxygen attributes the block to the
wrong (or to no) file:

  source_base/ndarray.h                  said NDArray.h
  source_lcao/module_rt/band_energy.h    said bandenegy.h (also a typo)

Found by tools/03_code_analysis/code_quality_score.py (rule
doc_file_mismatch). The scan also flagged
source_pw/module_pwdft/radial_proj.h (said radial_projection.h), but
#8007 has since moved that header to source_pw/module_proj/ with the
tag already corrected, so it is no longer part of this commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* style: convert leading tab indentation to spaces in 246 source files

.editorconfig has required `indent_style = space` with `indent_size = 4`
for the whole repository, but 339 files under source/ still indent with
tabs. This converts the leading whitespace of 246 of them.

253 files were selected when this was prepared. Since then, develop has
deleted two of them and already converted five others, which leaves 246:

  - deleted: source_lcao/setup_dm.h (#8000),
    source_pw/module_stodft/hamilt_sdft_pw.cpp (#8012);
  - already converted: source_base/module_out/binstream.{h,cpp} (#8025),
    source_pw/module_stodft/sto_hamilt_pw.h (renamed from
    hamilt_sdft_pw.h in #8012), source_io/module_ctrl/ctrl_output_pw.h
    and source_pw/module_pwdft/op_pw_nl.cpp (#8043).

Files that develop moved are converted at their new path, e.g.
onsite_proj_tools_stress.cpp is now under source_pw/module_proj/
(#8007).

Scope. Three groups were deliberately left out:

  - 48 files touched by the pull requests open at the time of writing
    (#8000, #7924, #7906, #8005 and others), so this does not force a
    conflict on work in flight;
  - 5 vendored files: source_base/libm/ is ported from glibc-2.36 and
    carries its own LICENCE, and source_base/mcd.c is Softpixel
    MemCheckDeluxe under a BSD licence. Reformatting vendored sources
    makes future syncs with their upstream harder;
  - tabs that appear after the first non-blank character (alignment
    tabs, 1193 lines). Only leading indentation is converted.

Method and verification. `expand -i -t4`, which rewrites the initial
whitespace of a line and nothing else, followed by three checks:

  - `git diff -w --stat` is empty, so not one non-whitespace character
    changed anywhere in the diff;
  - of the 42 raw string literals in the changed files (222 lines, all
    in source_io/module_parameter/read_inp_out.cpp), none has a line
    that this commit touches. Leading whitespace inside `R"(...)"` is
    part of the string, so that was the one place a leading-whitespace
    rewrite could have changed behaviour;
  - no changed line follows a line ending in a backslash, so no
    backslash-continued string literal is affected either;
  - no file gained a CR.

Effect on tools/03_code_analysis/code_quality_score.py: average score
over source/ goes from 82.15 to 82.58 and the number of passing files
from 1560 to 1562.

Six files score 1 to 3 points lower, all through the `line_too_long`
rule, because the scorer counts a tab as a single character while it
renders as up to four columns. The lines were already over 120 columns
on screen; the tab was hiding it. One of them,
module_ri/exx_abfs_ctor_orbs.cpp, moves from 60 to 59 and so drops just
below the tool's pass line. Wrapping those lines would mean editing code
in a commit that is otherwise whitespace-only, so it is left for a
follow-up.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant