[aliemen]

closes #610

This RP was intended to implement a shifted Green's implementation such that we can run simulation with source planes and P3M in OPALX. The main idea is that the "image" solve is performed with a normal shifted Green's kernel. PP interaction in the images are ignored, since they are (this is an approximation) more than RCUT away from the source plane.

Apart from that, I also found a few optimizations and smaller fixes that help with using P3M on bigger problems.

Changes

• FFTOpenPoissonSolver::shiftedGreensFunction now accepts TRUNCATED. It still uses the Green's function without the erf factor, but the distinction is necessary to optimize Green's rebuilds. Then P3M can use FFTTruncatedGreenPeriodicPoissonSolver::shiftedGreensFunction(shift), which delegates to the open solver.
• You can now change $\alpha$ (a factor in the truncated Green's function depending on RCUT) between runs. That allows for "adaptive cutoff radii", which is necessary if we want to define RCUT with $h$. Everything in the layout is regenerated when $\alpha$ changes.
• ParticleSpatialOverlapLayout gets a few helper methods to change $\alpha$ and register its change.
• The number of local cells is now capped at max(16 N, 2^24) where N is the number of local plus ghost particles after exchange (this is necessary to not run into "exceeds INT_MAX" errors. If it runs into this cap, the cells are simply widened to reduce the number of cells necessary (which can increase PP count, but at least the program doesn't crash). This does not influence the set of interacting particle pairs.
• Bugs:
  • particleRanks is now constructed with an explicit extent of 0. Kokkos 5.2.0 treated the missing extent as a huge one, which caused illegal memory accesses on GPU.
  • In the non-neighbor rank scan, outsideCounts is now indexed by i (the position in outsideIds) instead of by the particle ID.
  • createPeriodicGhostParticles now re-reads the position view after the copy, because the copy may have reallocated it.
  • ParticleAttrib::internalCopy, used for periodic ghosts, had three problems: it copied the first particles instead of the boundary particles (view(i) instead of view(indices(i))), it grew the views destructively (dropping all attribute data), and it took the view before the views grew. All fixed now.
• Some smaller timer location/naming and assertion fixes/additions.

Testing

I ran OPALX against this branch on up to 8 GH200 nodes just fine. This branch also works fine with OPALX master (even though some adjustments are needed to use these new features). Also added some more unit tests that should check more edge cases.

There is still a bug with periodic ghost particles in the PP interaction, which has already existed in master before. I will make a separate issue and fix it at a different time. It's not critical, since we want to use open P3M anyways.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[aliemen]

Tests look good. But still, I cannot merge myself until the CI notification is fixed.