Fix: revert #7664 $m_y$ convention

[maki49]

Fix #7831 induced by #7664.
The fix in #7804 works for NiO (z-collinear AFM) but not work for Mn3Sn (in-plane 120° AFM). It is also reverted in this PR.

[Copilot]

Pull request overview

This PR reverts the $m_y$ (spin-density y component) convention change introduced in #7664 to fix the regression reported in #7831, restoring correct behavior for in-plane non-collinear magnetic moments (e.g., Mn3Sn 120° AFM) while removing the downstream symmetrization workaround that was added to match the prior convention.

Changes:

• Flip the extracted rho_y sign in DensityMatrix_Tools::func_xyz_to_updown specializations to account for the conjugated DM storage layout.
• Remove the y-channel handedness workaround (yflip) in nspin=4 spin-density symmetrization so spin_so3() is applied directly again.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
source/source_estate/module_dm/density_matrix.cpp	Reverts the rho_y extraction sign in Pauli decomposition to match conjugated DM storage and fix non-collinear magnetization regressions.
source/source_estate/module_charge/symm_rhog.cpp	Removes the yflip compensation so spin-density symmetrization uses the physical pseudovector rotation (spin_so3) directly.

Suppressed comments (1)

source/source_estate/module_dm/density_matrix.cpp:664

• The trailing formula comment is misleading: this line computes Im(tmp[ud]) - Im(tmp[du]) for the stored block (which your comment says is conj(P)). In terms of the physical 1-RDM P, that corresponds to -Im(P_ud) + Im(P_du), not Im(P_ud) - Im(P_du). Please adjust the comment to match the actual convention (and clarify whether tmp is DM=conj(P) vs P).

    target_DMR_mat[icol + step_trace[2]] = tmp[1].imag() - tmp[2].imag();  // rho_y = Im(P_updown) - Im(P_downup)
    target_DMR_mat[icol + step_trace[3]] = tmp[0].real() - tmp[3].real();  // rho_z = (rho_upup - rho_downdown).real()

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[dyzheng]

1. Summary

The PR reverts the rho_y (m_y) sign flip that PR #7664 introduced in the
spinor-to-Pauli density-matrix conversion (func_xyz_to_updown), removes the
compensating yflip conjugation patch that PR #7804 added to the SOC grid
symmetrization (symm_rhog.cpp), restores three nspin=4 reference results to
their pre-#7664 values, and adds a magnetization round-trip unit test.

The core diagnosis is correct and well supported. cal_dm_psi.cpp stores
the complex conjugate of the physical 1-RDM

// cal_dm_psi.cpp: dm(iw1,iw2) = sum_ib wfc(ib,iw1).T * wg * wfc(ib,iw2).conj()
//                i.e. DMK(μ,ν) = Σ wg C*_{μ,ib} C_{ν,ib} = (C f C†)^T = conj(P)

For a stored block D = conj(P) the magnetization components are

m_x = Re(D_ud) + Re(D_du)             (conjugation-invariant)
m_y = Im(D_ud) - Im(D_du)             (bare formula, correct)
m_z = Re(D_uu) - Re(D_dd)             (conjugation-invariant)

The "bare" formula is the only one that recovers the true m_y of the spinor;
the #7664 formula -Im(D_ud)+Im(D_du) returns -m_y for every spinor.
m_x and m_z read only real parts and are unaffected — which is why only the
transverse (y) channel is broken. I verified this numerically on concrete
spinors (pure +y/−y, in-plane 120°-like, and generic 3D directions) using the
exact cal_dm_psi storage convention; the result agrees with the new
round-trip test.

A decisive cross-check: the PW path builds rho[2] directly from the spinor
wavefunctions (elecstate_op.cpp):

rho[2][ir] += w1 * 2.0 * (up.real()*dn.imag() - dn.real()*up.imag());  // = +m_y

i.e. the PW path always uses the bare/physical sign. Pre-#7664 the LCAO path
agreed with the PW path; post-#7664 they disagree for identical spinors. This
LCAO-vs-PW inconsistency is unambiguous and is fixed by this PR.

The PR is nevertheless incomplete. It reverts only the DM/charge-density
leg of #7664. Other #7664 changes that encode the same wrong convention remain
in the tree (DeltaSpin pauli_to_moment / cal_coeff_lambda and the unit test
that pins them), a pre-existing Mulliken M_y bug is left in place, and the
#7804 symmetry unit test is not reconciled with the revert. These are detailed
in Section 4.

2. What the PR changes

File	Change
source/source_estate/module_dm/density_matrix.cpp	Both func_xyz_to_updown specializations: rho_y back to the bare formula (Im(ud)-Im(du) / -i*(ud-du)); comment documents the conj(P) storage invariant (issue #7831).
source/source_estate/module_charge/symm_rhog.cpp	Removes the #7804 yflip (S=diag(1,-1,1) conjugation) from psymmg_soc; wspin is again the plain spin_so3(gmatc).
source/source_estate/module_dm/test/test_soc_magnetization_roundtrip.cpp	New unit test: builds the 4-element spinor DM block exactly as cal_dm_psi does (conj(c_a)*c_b) and asserts that func_xyz_to_updown recovers the physical (m_x,m_y,m_z) for five directions with nonzero transverse components, including pure ±y.
source/source_estate/module_dm/test/CMakeLists.txt	Registers MODULE_ESTATE_dm_soc_magnetization_roundtrip_test.
tests/03_NAO_multik/scf_angle_spin4/result.ref	Restored to pre-#7664: etot -6267.4651896196382950, force 0.000000, stress 3912.920437.
tests/03_NAO_multik/scf_out_dos_spin4/result.ref	Restored to pre-#7664: etot -1964.0663947982770878, force 0.162158, stress 1877.059089.
tests/03_NAO_multik/scf_u_spin4/result.ref	Restored to pre-#7664: etot -6789.2816406266510967, force 11.331534, stress 4697.832232.

3. What the PR correctly keeps (do NOT revert)

• gint_common.cpp clx_j = {0,-1,1,0} (H_{ud} = B_x - iB_y): introduced
  before Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 (d5c2916 / Fix Pauli-to-Spinor Conversion in LCAO Non-Collinear Calculations #7513), correct, and retained by this PR. The bug
  was only in the inverse (spinor→Pauli) extraction, not in the Hamiltonian.
• dftu_pw.cpp transfer_vu (V_{ud} = 0.5(V_x - iV_y)): this Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 change
  is a genuine fix — it pairs with the PW occupation extraction
  (locale[2] = Im(occ[1]-occ[2]), which Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 never touched) to form the
  standard QE/VASP-style pairing. Consistently, 099_PW_DJ_SO/result.ref
  (updated by Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 for this fix) is not reverted here.
• The Pauli-basis DFT+U / DeltaSpin force-stress structure (no spinor
  conversion, no *2 for nspin=4 in dftu_fs.cpp/dspin_fs.cpp) predates
  Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 and is unaffected.

4. What still needs to be changed

4.1 DeltaSpin still uses the #7664 (wrong) y-convention — must be fixed

• source/source_lcao/module_deltaspin/spin_constrain.h (pauli_to_moment):
  My = -w*Im(occ[1]-occ[2]) (post-Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664). The occupations are built with the
  same conj-first structure as cal_dm_psi (occ[1] = conj(c_up)*c_dn,
  spin_constrain.cpp:184), so the bare formula +Im(occ[1]-occ[2]) was
  correct and Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 flipped it. Consequence: DeltaSpin measures -m_y, and
  since cal_coeff_lambda was flipped in tandem the constraint loop is
  self-consistent in code space but drives the y-mirror of the target while
  reporting convergence.
• source/source_lcao/module_operator_lcao/dspin_lcao.cpp
  (cal_coeff_lambda): λ_{ud} = λ_x - iλ_y (post-Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664). Must be reverted
  together with pauli_to_moment.
• source/source_lcao/module_deltaspin/test/deltaspin_core_test.cpp: updated by
  Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 to pin My = -0.4 for its general case; must be reverted to the bare
  formula.
• Risk after this PR: with the DM/charge-density rho_y reverted to +m_y
  but DeltaSpin still measuring -m_y, the PR replaces one inconsistency
  (LCAO vs PW) with another (DM vs DeltaSpin measurement).

4.2 Mulliken output M_y is a pre-existing bug (separate, but track it)

• source/source_io/module_mulliken/output_mulliken.cpp:448,456 computes
  M_y = -Im(mud_ud) + Im(mud_du) with mud = DMK·S (conj-first). The same
  analysis shows this yields -m_y. It predates Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 (since 3.7.0,
  commit a339356) and is likely why Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664's flip went unnoticed: the DMR
  change made func_xyz_to_updown agree with the (buggy) Mulliken sign.
• Recommend a separate fix (bare formula) plus a unit test, and at minimum a
  comment in this PR acknowledging that the Mulliken sign must not be used as
  an oracle for the DM convention.

4.3 The #7804 symmetry unit test is not reconciled

• source/source_cell/module_symmetry/test/symmetry_rho_soc_test.cpp
  (added by Fix: match spin density symmetrization with the new convention changed in #7664  #7804): SpinConventionCoupling hard-codes the Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 convention
  (ry = -D[1].imag() + D[2].imag()) in its local pauli_from_block helper
  and documents it as "the Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 convention". Fix: revert #7664 $m_y$ convention #7832 removes the yflip from
  symm_rhog.cpp but does not update this test. The helper, the surrounding
  comments, and the test design should be updated to the bare convention; the
  test should ideally exercise the real func_xyz_to_updown/psymmg_soc
  instead of a local re-implementation, otherwise it can pass vacuously while
  the code convention drifts.

4.4 References: regenerated vs. reverted

• The three result.ref files are restored to the exact pre-Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 values. This
  is expected to be consistent because Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664's final commit changed the LCAO
  path only through density_matrix.cpp plus the reference files (the DFT+U
  force code and H-construction fixes predate Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664), but the PR body reports
  no test run. The references must be regenerated/verified with the actual
  post-PR code and the exact ctest/integration-test commands and results
  reported (per the ABACUS contribution rules).
• Minor: scf_angle_spin4/threshold (relaxed to 1e-5 by Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664) is not
  reverted — acceptable, but should be noted.
• 099_PW_DJ_SO/result.ref is correctly left at its post-Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664 value (it
  tracks the retained dftu_pw fix).

4.5 Test coverage gaps

• The round-trip test covers only the <double> specialization; the
  <complex> (multi-k) specialization is changed identically but untested.
  Extend the test to the complex case.
• There is no integration regression test for the actual #7664 destroys the y-component of magnetic moment #7831 scenario
  (in-plane non-collinear AFM with symmetry enabled). The unit test pins the
  extraction but not the symmetry path; a case with nonzero in-plane moments
  (Mn3Sn-like) would directly pin the yflip removal.
• The test comment references ref/MSG/2026-08-#7664-..., which does not exist
  in the repository — either add the design note or drop the reference.

4.6 Restart / DM I/O compatibility (minor)

• DMR restart files written by Fix: correct Pauli-to-spinor Hamiltonian conversion for nspin=4  #7664-era code carry the flipped y-channel;
  after this revert, restarting from such files would be inconsistent for
  nspin=4. Check whether nspin=4 DMR restart is supported (write_dmr /
  read_dmr) and document the compatibility or add a version guard.

5. Suggested follow-up checklist

1. Revert DeltaSpin pauli_to_moment, cal_coeff_lambda, and
   deltaspin_core_test.cpp to the bare convention; add an nspin=4 DeltaSpin
   regression test with a nonzero m_y target.
2. Update symmetry_rho_soc_test.cpp to the bare convention and make it
   exercise the real functions.
3. Fix the Mulliken M_y sign (separate PR or included) and add a unit test.
4. Regenerate/verify the three nspin=4 references plus 099_PW_DJ_SO and
   report the exact test output in the PR body.
5. Extend the round-trip test to the <complex> specialization and add an
   in-plane non-collinear symmetry regression test.
6. Resolve the dangling doc reference and the DMR restart-compatibility
   question.

6. Verdict

Directionally correct and worth merging as a partial fix, but recommend
"changes requested" until at least §4.1 (DeltaSpin) and §4.3 (symmetry test)
are addressed in this PR or a directly linked follow-up. As-is, the PR fixes
the LCAO↔PW rho_y inconsistency that #7664 introduced but leaves the
DeltaSpin path and the Mulliken output on the opposite (wrong) sign, so the
codebase would still be internally inconsistent in the transverse channel.

[maki49]

@dyzheng Thanks for the suggestions. I've fixed the $m_y$ convention in DeltaSpin and updated the symmetry test to call the real function.