AMR scaling: comm/memory-scalable regrid + restart-IO, batched flux-capture, multi-level restart#7
Merged
Merged
Conversation
…atch (remove per-seam malloc in s_amr_fine_fine_halo; byte-identical)
…/ALLREDUCE/ALLGATHER over all blocks, not per-block; file layout byte-identical)
…+gctag) Replace the two O(global-grid) MPI_ALLREDUCE tag-frame reductions in s_amr_cluster (gtag, MPI_MAX) and s_amr_regrid child-nesting (gctag, MPI_LOR) with all-gathers of each rank's SPARSE tagged-cell linear indices (new helpers s_amr_union_gtag/gctag). Tags are 0/1 so the reconstructed field is byte-identical to the dense reduction and clustering stays rank-invariant; comm now scales with the number of tagged cells, not the whole grid. Validated byte-identical on 5 multi-rank AMR goldens (dynamic-regrid, multi-level-static, subcycle, restart, stretched-grid; np=2) + GPU np=1 on H200. NOTE: touching s_amr_cluster/s_amr_regrid made ffmt reindent their pre-existing +4 over-indentation (whitespace + comment re-wrap only, zero logic change) - this is the bulk of the diff; git diff -w isolates the ~90-line logical change.
…blocks) launches -> O(1) per category; byte-identical)
… (fm host array referenced in GPU_PARALLEL_LOOP bounds) s_amr_fine_slice (the c8b1006 device-packed seam halo) used the host local array fm(TB)/fm(TA) as GPU_PARALLEL_LOOP bounds, so nvfortran/Cray demanded fm PRESENT on device -> runtime 'FATAL ERROR: data in PRESENT clause was not found on device: name=fm(2:)'. AMD flang tolerated it; NVHPC (Phoenix) and CCE (Frontier) gpu-acc lanes aborted every np>=2 AMR test. Use the scalars na-1/nb-1 (na=fm(TA)+1 already computed on host) as the bounds - byte-identical, no host-array reference in the device region. Validated: 6 np=2 AMR goldens (244B1E42 F57C3A5B ADA042A2 EF58E377 4644A339 B7704247) pass on 2x H200 gpu-acc.
…d) dense tag field (per-rank memory now O(#tagged), byte-identical boxes)
…~1290^3 (2048^3+ target) The sparse-clustering encode gi+(mg+1)*(gj+(ng+1)*gk) in default integer overflows once (m_glb+1)*(n_glb+1)*(p_glb+1) >= 2^31 (~1290^3, and the 2048^3 this scaling work targets), silently corrupting decoded box coordinates. Widen locidx/allidx to integer(8) with int8 encode/decode and gather via MPI_INTEGER8; counts/displacements stay int32 (they count tagged cells, < 2^31). Byte-identical at tested grids (7 regrid/multilevel goldens pass).
…ustering compute O(nboxes*ntag) -> O(ntag*depth); byte-identical boxes)
Bumps the ffmt formatter pin in toolchain/pyproject.toml from 0.4.3 to 0.4.4 (fixes single-line-construct indent leak, sbryngelson/ffmt#5). Verified as a 0-change no-op on master formatting; Formatting/Lint Toolchain/Python 3.9-3.14 CI all green.
ffmt 0.4.4 (pinned by the prior commit, fixes sbryngelson/ffmt#5) corrects the pre-existing +4 over-indentation of s_amr_cluster/s_amr_regrid (a 0.3.x single-line 'do ...; end do' indent-leak that 0.4.3 tolerated as a buggy fixed point). Whitespace + doc-comment re-wrap only, zero logic change (git diff -w is comment-rewraps); m_amr.fpp and m_sfc_partition.fpp are the only two files affected. Unblocks clean formatting of subsequent edits.
…r level (was per parent-box; byte-identical)
…s (serial path) The serial AMR restart reader assumed every block's fine extent was ref_ratio*(region_width)-1 (single level), so it rejected every level>=2 block as 'block fine extents disagree with the region (corrupt file)' -- multi-level (amr_max_level>=2) runs could not restart. A level-l block covers ref_ratio**l fine cells per L0 cell of its region, not ref_ratio (see the work-weight at m_amr.fpp:1173 and s_set_amr_fine_geometry:1297-1321). The parallel_io read path also sizes each block from the region before extents are read, so the level cannot be back-derived -- it must be stored. Fix (serial path): the writer stores amr_block_level(k) in each per-block header; the reader reads it, sets amr_block_level(k) before the owner/geometry rebuild (s_amr_assign_block_owners and s_set_amr_fine_geometry key off it to place L>=2 blocks under their parent), and generalizes the extent check to ref_ratio**level. New test AMR -> 1D -> multi-level restart (4AF96C49) restart_checks the static L2 hierarchy; validated on gpu-acc np=1 (fix passes; single-level AMR restart 21C71558, 1D restart 1A379909, multi-level advance 75AD6885 unchanged). parallel_io path left unchanged (still fails-closed on size mismatch, no regression) -- a clean second increment.
# Conflicts: # src/simulation/m_amr.fpp
… MPI code (PR #7 review C1+I1) C1 (Critical): the parallel_io (MPI-IO) AMR restart path stored no per-block level and hard-coded a level-1 fine extent (ref_ratio*width), so multi-level (amr_max_level>1) restart read every L>=2 block at the wrong resolution and mislaid all downstream block offsets -- caught only by a fragile total-size tripwire. Mirror the serial fix onto the MPI-IO path: the per-block header is now 7 ints (region box + refinement level), the reader sets amr_block_level(k) before the owner/geometry rebuild and sizes each block with ref_ratio**level, with a level-range guard. New restart_check test 'AMR -> 1D -> multi-level restart parallel_io np=2' (78314D65). Single-level MPI-IO restart (5EFB3277) and serial multi-level restart (4AF96C49) unchanged; validated np=2 gpu-acc. I1 (Important): delete dead code. s_mpi_sendrecv_amr_fine_halo (~150 lines + GPU pack/unpack kernels) had zero call sites under the whole-block ownership model that superseded the continuation-face halo, yet s_initialize_amr_mpi_buffers allocated its device buffers (amr_buff_send/recv) on every AMR run. Removed both routines, the buffers, the finalize dealloc, and the call site (max_f1/2/3 stay -- used for slot coordinate sizing); fixed the stale comment referencing the nonexistent s_mpi_bcast_amr_reflux_faces. Net -159 lines.
review Perf-2) s_amr_fine_fine_halo rescanned all O(nblocks^2) block pairs with f_amr_seam every RK stage (6x per fine step) to find same-level adjacent seams. Block topology (regions/levels/count) changes only at regrid/restart, so cache the (xb, yb, seam-dim) list once via s_amr_build_seam_pairs and iterate O(#seams) per stage. Rebuilt when amr_seam_pairs_dirty is set (regrid, restart, init) with a block-count tripwire as backup. The list preserves the original (xb, yb) nested-loop order on all ranks (replicated region metadata), so the paired MPI_SENDRECVs stay matched -> byte-identical. New pure helper f_amr_seam_dim shares the adjacency logic between builder and halo. Validated: full AMR suite 57/57 byte-identical, np=2 gpu-acc.
…bug print (PR #7 review)
…e load_balance debug behind MFC_DEBUG (PR #7 review)
…ratio + np-flexible restart (PR #7 review)
…mirroring m_checker (PR #7 review)
…iew surface reduction)
sbryngelson
added a commit
that referenced
this pull request
Jul 16, 2026
…l) + parent-coarse invariants; disambiguate 2*r0 (PR #7 review)
sbryngelson
added a commit
that referenced
this pull request
Jul 16, 2026
…el case.md subsection; move fine-extent/parent-coarse invariants into common-pitfalls; refresh amr_golden_tests docstring (PR #7 review)
sbryngelson
added a commit
that referenced
this pull request
Jul 16, 2026
…morton/f_morton) (PR #7 review)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scaling + correctness work for the block-structured AMR path, branched off
up/megaand re-synced with it (0 behind). Every perf commit is byte-identical on the AMR golden suite; the three correctness commits are covered by tests. Validated on CPU (56/56 AMR goldens) and GPU (nvfortran gpu-acc, np=1 and np=2 on H200/V100).Scaling (byte-identical — boxes/fields unchanged)
Findings from a 5-reviewer HPC-perf review of MFlowCode#1628:
54810d6e— hoist the fine-fine seam pack buffers to persistent module scratch (removes a per-seam malloc ins_amr_fine_fine_halo).52f4bb2e— vectorize the AMR restart-I/O metadata collectives: one EXSCAN/ALLREDUCE/ALLGATHER over all blocks instead of per-block. File layout unchanged.76f55c72— sparse tagged-index ALLGATHERV replaces the two O(global-grid) MPI_ALLREDUCE tag frames in regrid clustering; comm now scales with #tagged, not the global grid (the critical scaling wall).15baa8a9— batch the per-block AMR flux-capture kernels over the slot dimension (O(blocks) launches → one collapse(4) kernel per category). Conservation-critical.6703e30d— sparse Berger-Rigoutsos clustering: drops the O(global-grid) dense tag field; per-rank regrid memory is now O(#tagged).205d4e6a— batch the multi-level child-nesting union to one ALLGATHERV per level (was per parent-box).07bddd81— partition the sparse tag list through Berger-Rigoutsos (clustering compute O(nboxes·ntag) → O(ntag·depth)).Together these make regrid scale in both comm (R6/#2) and memory (R6b), and remove the per-step malloc/launch overheads (R3/R4/R5).
Correctness
a9831a47— the seam-halo device kernel (s_amr_fine_slice) referenced a host array inGPU_PARALLEL_LOOPbounds, aborting every np≥2 AMR gpu-acc test on nvfortran/Cray withPRESENT clause not found: fm(2:). Loop bounds → scalars; byte-identical. (Also landed onup/megaas9ea61932.)baf8c4c1— the sparse tag linear indexgi + (mg+1)*(gj + (ng+1)*gk)overflows int32 above ~1290³; promoted tointeger(8)+MPI_INTEGER8. Unblocks the 2048³+ target.914a373e— the serial AMR restart reader assumed single-level fine extents (ref_ratio·width), so it rejected every level ≥ 2 block ascorrupt file— multi-level runs could not restart. Now the per-block level is stored in the restart header and the reader rebuilds multi-level geometry from it. Newrestart_checktestAMR -> 1D -> multi-level restart(4AF96C49); single-level restart unchanged. (Serial path only; parallel_io multi-level restart remains a follow-up — it still fails closed.)Validation