peclet::core::halo::detail Namespace Reference

Classes
struct	NonDeduced

Functions
bool	mpiReportsCudaAware ()
	Does the MPI library itself claim CUDA awareness (OpenMPI's MPIX_Query_cuda_support)? Some stacks under-report (measured: user-space OpenMPI 5.0.7 + UCX 1.20.1 with working cuda_ipc returns 0 — see core/docs/cuda-aware-mpi.md), so this is only used to AUTO-enable, never to veto an explicit env request.
bool	probeGpuAwareMpi ()
	Probe whether MPI actually accepts DEVICE pointers (GPU-aware MPI): checksummed loopback Sendrecv on MPI_COMM_SELF with device buffers, errors returned (not fatal) and the payload verified on device.
bool	gpuAwareMpi ()
	Whether to hand DEVICE pointers straight to MPI (GPU-aware MPI) instead of host-staging.

Function Documentation

mpiReportsCudaAware()

bool peclet::core::halo::detail::mpiReportsCudaAware ( )

inline

Does the MPI library itself claim CUDA awareness (OpenMPI's MPIX_Query_cuda_support)? Some stacks under-report (measured: user-space OpenMPI 5.0.7 + UCX 1.20.1 with working cuda_ipc returns 0 — see core/docs/cuda-aware-mpi.md), so this is only used to AUTO-enable, never to veto an explicit env request.

Definition at line 38 of file grid_halo.hpp.

Referenced by gpuAwareMpi().

probeGpuAwareMpi()

bool peclet::core::halo::detail::probeGpuAwareMpi ( )

inline

Probe whether MPI actually accepts DEVICE pointers (GPU-aware MPI): checksummed loopback Sendrecv on MPI_COMM_SELF with device buffers, errors returned (not fatal) and the payload verified on device.

Necessary-but-not-sufficient (same-node cuda_ipc/cuda_copy can work while inter-node GPUDirect is broken). CAUTION: a stack with NO cuda transport SEGFAULTS inside UCX rather than returning an error (measured on the system OpenMPI here), so this probe is only run once the MPI has claimed CUDA awareness — it is the validation gate, not the discovery mechanism.

Definition at line 57 of file grid_halo.hpp.

References MPI_Comm_free(), MPI_Initialized(), MPI_INT, MPI_STATUS_IGNORE, and MPI_SUCCESS.

Referenced by gpuAwareMpi().

gpuAwareMpi()

bool peclet::core::halo::detail::gpuAwareMpi ( )

inline

Whether to hand DEVICE pointers straight to MPI (GPU-aware MPI) instead of host-staging.

Resolution (decided once per process): env PECLET_CORE_GPU_AWARE_MPI (legacy PECLET_CORE_CUDA_AWARE_MPI) set to 0 forces host staging, nonzero forces the device-pointer path (trusted, no probe — MPIX under-reports on some working stacks); UNSET auto-enables the device path when the MPI REPORTS CUDA awareness AND the loopback probe validates it, so a properly built GPU-aware MPI is used by default with no configuration. Stacks that work but under-report (see core/docs/cuda-aware-mpi.md) still need the env=1 opt-in — blind probing is not survivable there because an unsupported transport segfaults instead of erroring.

Definition at line 106 of file grid_halo.hpp.

References mpiReportsCudaAware(), and probeGpuAwareMpi().

Referenced by peclet::core::halo::GridHalo< T >::exchangeBegin(), peclet::core::halo::ParticleHalo< Dim >::forward(), peclet::core::amr::DistributedGatherHalo< Dim, Bits >::gather(), and peclet::core::halo::ParticleHalo< Dim >::reverse().