peclet.pnm — distributed (MPI) pore-network extraction on the core block decomposition. More...

#include <mpi.h>
#include <algorithm>
#include <array>
#include <cstdint>
#include <Kokkos_Core.hpp>
#include <Kokkos_UnorderedMap.hpp>
#include <map>
#include <stdexcept>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
#include "peclet/core/common/types.hpp"
#include "peclet/core/common/view.hpp"
#include "peclet/core/decomp/block_decomposer.hpp"
#include "peclet/core/halo/grid_halo.hpp"
#include "peclet/core/halo/grid_halo_topology.hpp"
#include "pore_extraction.hpp"

Classes
struct	pnm::detail_mpi::MergeRemap
	ONE global merge of a cross-block equivalence graph: the local (a,b) pairs are allgathered and union-found on the host (union-by-min-gid: the class root is the min gid, matching the single-rank fixpoint; deterministic on every rank — identical input). More...

struct	pnm::PoreRec
	One owned pore-peak record of the distributed network-flow extraction (namespace-scope: nvcc forbids function-local types in extended-lambda captures). More...

struct	pnm::MpiPoreNetwork
	Result of one distributed extraction. More...

Namespaces
namespace	pnm

namespace	pnm::detail_mpi

Functions
int	pnm::detail_mpi::allreduceMaxInt (int v, MPI_Comm comm)

template<class T >
std::vector< T >	pnm::detail_mpi::allgatherv (const std::vector< T > &local, MPI_Comm comm)
	Allgatherv a local vector of T (POD) — every rank returns the concatenation.

template<class T >
void	pnm::detail_mpi::sortUnique (std::vector< T > &v)
	Sort + unique in place.

template<class Topo >
BlockGeo	pnm::detail_mpi::blockGeoOf (const Topo &topo, std::array< int, 3 > gdims, int G)
	The extended-block geometry of a halo topology (ghost width G) on the global grid `gdims`.

template<class T >
peclet::core::View< T >	pnm::detail_mpi::stageInner (const Exec &space, const BlockGeo &geo, const std::vector< T > &h, const char *name)
	Stage a host inner block (x-fastest, no ghosts) onto a fresh extended device field (ghosts zero-initialised); the caller exchanges.

template<class T >
std::vector< T >	pnm::detail_mpi::packInner (const Exec &space, const BlockGeo &geo, const peclet::core::View< T > &e)
	Pack the owned box of an extended device field into a host vector.

template<class PairView >
std::vector< std::pair< Index, Index > >	pnm::detail_mpi::localPairs (const PairView &d, std::size_t count)
	Interleaved device (a,b) Index pairs -> host list, locally sorted/unique.

MergeRemap	pnm::detail_mpi::globalMerge (const std::vector< std::pair< Index, Index > > &loc, MPI_Comm comm)

template<class FieldView , class PredFn >
void	pnm::detail_mpi::applyRemap (const Exec &space, const MergeRemap &m, const FieldView &field, std::size_t n, const PredFn &pred)
	Apply a MergeRemap to the entries of `field` selected by `pred(i)`: label -> its class root.

template<class Halo , class TargetView >
void	pnm::detail_mpi::resolveToFixpoint (const Exec &space, const BlockGeo &geo, Halo &halo, const TargetView &target, MPI_Comm comm, const char *what)
	Cross-rank fixpoint of the hold-at-ghost pointer jumping (kernels::resolveHoldAtGhost): exchange the state field, resolve one round on every rank, Allreduce the pending count, until no cell is pending.

template<class KeyValFn >
std::vector< std::pair< Index, Index > >	pnm::detail_mpi::minByKey (std::size_t n, const KeyValFn &kv)
	Per-rank (key -> min value) reduction on device via UnorderedMap, with capacity-retry.

void	pnm::mpi_block_of (std::array< int, 3 > gdims, MPI_Comm comm, std::array< int, 3 > &offset, std::array< int, 3 > &size)
	This rank's ORB block of the global grid (deterministic; same partition as flow/dem): `offset` = the block's first voxel index per axis (an integer grid offset, not a physical origin), `size` = its voxel count per axis.

MpiPoreNetwork	pnm::extract_pore_network_mpi (const std::vector< float > &sdf_local, std::array< int, 3 > gdims, std::array< float, 3 > origin, std::array< float, 3 > spacing, MPI_Comm comm)
	Distributed fused extraction.

NetworkFlow	pnm::extract_network_flow_mpi (const std::vector< float > &sdf_local, std::array< int, 3 > gdims, std::array< float, 3 > origin, std::array< float, 3 > spacing, const std::vector< double > &u_h, const std::vector< double > &v_h, const std::vector< double > &w_h, const std::vector< double > &p_h, const std::vector< double > &ox_h, const std::vector< double > &oy_h, const std::vector< double > &oz_h, std::array< double, 3 > grad_p, MPI_Comm comm)

Detailed Description

peclet.pnm — distributed (MPI) pore-network extraction on the core block decomposition.

The SDF is decomposed over ranks by the shared ORB (peclet::core::decomp::BlockDecomposer, the same deterministic partition flow/dem use) and every stage of the single-rank pipeline (pore_extraction.hpp) runs per-rank on a g=1 extended block with core's GridHalo ghost exchange — the SAME stage kernels (pore_kernels.hpp), instantiated on the BlockGeo geometry instead of the single-rank GridGeo. Labels are GLOBAL voxel ids (Index/int64), which makes every fixpoint decomposition-independent, so the multi-rank result is BIT-EXACT to the single-rank pipeline. This file holds only what is distributed: the halo exchanges, the ownership/merge orchestration and the reductions:

pore detection — 3^3 stencil on the exchanged SDF; a rank emits the peaks it owns.
marker CCL — local union-find over owned cells (the single-rank kernels on the owned box), then per-component min-gid, then ONE global merge: the cross-block adjacency graph on boundary labels (surface data) is allgathered and union-found on the host — no iteration to convergence. Fixpoint = min gid of the global component, the same value the single-rank atomic_min CCL converges to.
flood fill — Jacobi (double-buffered) min-label sweeps with a halo exchange + changed-flag Allreduce per sweep; sweep-for-sweep identical to the single-rank Jacobi flood.
gradient-path roots — the 512-step walk does not distribute, but its RESULT is a forest (each pore voxel points at its steepest 26-neighbour under the (sdf, gid) lexicographic order; peaks are roots). Roots are resolved by pointer jumping: chase within the block, stop at a ghost, exchange the target field, repeat. The forest is fixed, so the fixpoint is the unique root regardless of order. (No 512-step cap — a >512-step monotone path would be the one divergence from the single-rank walk.)
renumbering — solid labels ARE their component min-gid, so solid ids are just the ascending sort of the global label set; pore ids need the first-appearance (min gid) per root, reduced per-rank with a device UnorderedMap then allgathered. Matches the single-rank first-encounter order exactly.
topology — local (+x/+y/+z) face pairs against exchanged seg ghosts, then a global sort/unique; every rank returns the identical list.

Include only from MPI-enabled TUs (PECLET_PNM_MPI bindings, tests/kokkos_mpi).

Definition in file pore_extraction_mpi.hpp.

Classes

Namespaces

Functions

Detailed Description