Repository navigation
Conversation
added files/directories: - interop/: header-only: KokkosDLPack.h (Kokkos::View -> DLPack capsule, zero-copy), IpplDLPack.h (ippl ParticleAttrib adapter) InferenceBridge.h (embedded python interpreter, sets the init/infer/finalize contract), exposed as INTERFACE target IPPL::python_bridge - cmake: IPPL_ENABLE_PYTHON_BRIDGE option (default OFF); pybind11 and DLPack resolved in Dependencies.cmake with overridable PYBIND11_VERSION / DLPACK_VERSION - demos/alpine: LandauDampingNeurOp, numeric field solve with par2grid/grid2par replaced by surrogate inference in pytorch
- InferenceBridge: replaced fixed init/infer methods with call(fn, args...), a variadic template that perfect-forwards arguments to the Python function and returns result as py::object. This means the bridge doesn't mandate the python side API except finalize() - guarded(): added catch for std::exception like from py::cas_error from unknown argument conversion (like a user defined class) - demos/alpine: LandauDampingNeurOp is built with hidden symbol visibility (CXX_VISIBILITY_PRESET hidden, VISIBILITY_INLINES_HIDDEN), recommended recommended by pybind11 for embedding code; removes -Wattributes warning during compilation. Visibility attribute in the header is dropped, so its compiler-neutral.
Collaborator
|
@srikrrish would this not somewhat best belong into the Stream directory? (so src/Stream/Interoperability). Also to not overfill our main demo, do we want to consider again a separate demo folder for neursl operators (sth like demos/alpineNO. |
Member
Yeah agree to both. I haven't still reviewed this. Will do it soon and put these in the comments. Thanks |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional interoperability layer (
src/Interoperability, namespaceippl::Interoperability) that lets IPPL share particle data with Python frameworks with no copying.KokkosDLPack.h: exports aKokkos::Viewas a DLPack capsule, zero-copy, in the View's memory space.IpplDLPack.h: exportsParticleAttribdata.InferenceBridge.h: loads a Python module through an embedded interpreter ( via pybind11), calls its functions with arbitrary arguments.demos/alpine/LandauDampingNeurOp: LandauDamping where the electric field is inferred by a Python model every step instead of computed by the PIC field solver.Build
It is opt-in with
-DIPPL_ENABLE_PYTHON_BRIDGE=ON(default OFF). pybind11 is used if installed and fetched otherwise (-DPYBIND11_VERSION). DLPack (just a header, C ABI) is fetched (-DDLPACK_VERSION).-DPython_EXECUTABLE=<python>selects the Python environment to embed. The interoperability headers are not installed when the option is OFF.Execution space, memory space and data movement
Physics / numerics impact
None on existing IPPL code: no existing kernels, floating-point behaviour or MPI/GPU
execution order are changed.
Testing
New unit test
unit_tests/Interoperability/DLPackExport(GoogleTest, typed over allexecution spaces * float/double * Dim 1–3). Through
InferenceBridge, it exportsParticleAttribs to PyTorch and checks:The test is only registered if the embedded Python can import torch.
Ran on JUWELS Booster after merging the current master: 36/36 tests passed on 2 MPI ranks.