flow 1.3.0
Kokkos cut-cell IBM incompressible Navier-Stokes solver + pnm pore extraction
Loading...
Searching...
No Matches
peclet::flow::vof::VofBlockExchange Class Reference

The gather / scatter / table replication of the block container. More...

#include <block_exchange.hpp>

Inheritance diagram for peclet::flow::vof::VofBlockExchange:
peclet::flow::vof::VofBlockExchangeBase

Classes

struct  GatherTable
 
struct  Patch
 This rank's own patch: the extended block the face velocity and the union colour live on. More...
 

Public Member Functions

void init (I3 gs, std::array< bool, 3 > per, std::vector< VofBox > rankBox, int rank)
 
void setRankBoxes (std::vector< VofBox > rankBox)
 Re-bind the owned inner box of every rank WITHOUT disturbing the communicator or the local patch.
 
void setPatch (Patch p, SField uf, SField vf, SField wf)
 The local face velocity, in the ADVECTOR's high-face convention (i.e.
 
long gatherBytes () const override
 Bytes moved by the last gather / scatter on THIS rank (sent + received).
 
long scatterBytes () const override
 
long gatherMessages () const
 Messages posted by the last gather / scatter on this rank (sends + receives).
 
long scatterMessages () const
 
void gatherImpl (std::vector< VofBlock > &blocks, int ghost, bool useExtended, int nc, const SField *loc, SField(*blockView)(VofBlock &, int), int blockBase, int outside=0)
 patch -> block.
 
void scatterImpl (std::vector< VofBlock > &blocks, int nc, SField *loc, SField(*blockView)(VofBlock &, int), int blockBase, int op, bool gatherLocal=false)
 block -> patch, combined with op (0 = max, 1 = sum).
 
void gatherLocalSum (std::vector< VofBlock > &blocks, int nc, SField *loc, SField(*blockView)(VofBlock &, int), int blockBase)
 The local half of a gatherLocal UNPACK_SUM: every patch INNER cell written (so no zeroing pass is needed) with 0 + the values of the blocks this rank masters whose box holds it, in block order.
 
void gatherLocalLaunch (const GatherTable &T, SField *loc)
 
void zeroPatchInner (int nc, SField *loc)
 
void packFromPatch (const VofPiece &p, int nc, const SField *loc, SField buf, long &off)
 
void unpackToBlock (const VofPiece &p, int nc, SField buf, long &off, SField(*blockView)(VofBlock &, int), VofBlock &b, int blockBase)
 
void movePiece (const VofPiece &p, int nc, const SField *loc, SField(*blockView)(VofBlock &, int), VofBlock &b, int blockBase, bool toBlock)
 The master's OWN cells: patch <-> block with no buffer and no host traffic.
 
void appendMoveJobs (const VofPiece &p, int nc, const SField *loc, SField(*blockView)(VofBlock &, int), VofBlock &b, int blockBase, std::vector< VofMoveJob > &out)
 H-4(e): the (piece, component) copies of movePiece for one piece, appended to out.
 
template<class Exec = SExec>
void moveLocalPieces (const std::vector< VofMoveJob > &mv)
 H-4(e), host: every collected movePiece copy in one launch per chunk of kVofBlockBatch (piece, component) pairs, rows inside, through a by-value job table – instead of one MDRange3 per piece per component.
 
void packFromBlock (const VofPiece &p, int nc, SField(*blockView)(VofBlock &, int), VofBlock &b, int blockBase, SField buf, long &off)
 
void combineBlockIntoPatch (const VofPiece &p, int nc, SField(*blockView)(VofBlock &, int), VofBlock &b, int blockBase, SField *loc, int op)
 
void combineBufIntoPatch (const VofPiece &p, int nc, SField buf, long &off, SField *loc, int op)
 
void gatherFaceVel (std::vector< VofBlock > &blocks, int ghost) override
 Fill every MASTER block's faceU/V/W over its EXTENDED box, from the face velocity the ranks own.
 
void gatherColour (std::vector< VofBlock > &blocks, SField cLocal) override
 Fill every MASTER block's INNER colour from the caller's colour patch (the mirror of scatterColourMax).
 
void scatterColourMax (std::vector< VofBlock > &blocks, SField cLocal) override
 UNION the masters' inner colour into the caller's local colour patch: zero the inner region, then C = max(C, C_block) cell by cell (TBFsolver's UNPACK_MAX).
 
void scatterForceSum (std::vector< VofBlock > &blocks, SField fx, SField fy, SField fz) override
 SUM every master block's CSF face force into the caller's three face-force patches (TBFsolver's UNPACK_SUM, VOF.f90::computeSurfaceTension): overlapping markers ADD their forces, which is what makes two touching bubbles push on the fluid twice and not once.
 
void scatterColourSum (std::vector< VofBlock > &blocks, SField sLocal) override
 S = sum_k C_k (vof_overlap_design §5.6): the colour scatter with UNPACK_SUM.
 
void migrateColour (std::vector< VofBlock > &blocks, const std::vector< int > &oldMaster) override
 Move a block's colour from its old master to its new one after a re-assignment (rung W1 item a).
 
long migrateBytes () const
 
long migrateMessages () const
 
SField buffer (long n)
 Reusable device buffer of exactly n doubles, handed out in REQUEST ORDER — two live messages of the same size must not be the same View, so the pool is keyed by the request sequence (reset at the head of every transfer) and not by the size.
 
void gatherFaceVelHost (std::vector< VofBlock > &blocks, int ghost)
 
void scatterColourMaxHost (std::vector< VofBlock > &blocks, SField cLocal)
 
void syncTable (std::vector< VofBlock > &blocks) override
 Replicate the (possibly re-centred) boxes so every rank's table agrees again.
 
- Public Member Functions inherited from peclet::flow::vof::VofBlockExchangeBase
virtual ~VofBlockExchangeBase ()=default
 

Static Public Member Functions

static SField faceViewOf (VofBlock &b, int c)
 
static SField colourViewOf (VofBlock &b, int c)
 
static SField forceViewOf (VofBlock &b, int c)
 
static int ghostOf (const std::vector< VofBlock > &blocks)
 

Public Attributes

bool deviceStaging = true
 

Detailed Description

The gather / scatter / table replication of the block container.

Definition at line 146 of file block_exchange.hpp.

Member Function Documentation

◆ init()

void peclet::flow::vof::VofBlockExchange::init ( I3  gs,
std::array< bool, 3 >  per,
std::vector< VofBox >  rankBox,
int  rank 
)
inline
Parameters
rankBoxthe owned inner box of EVERY rank, in global cells (from the flow BlockDecomposer; a single element covering the grid when serial)

Definition at line 158 of file block_exchange.hpp.

◆ setRankBoxes()

void peclet::flow::vof::VofBlockExchange::setRankBoxes ( std::vector< VofBox >  rankBox)
inline

Re-bind the owned inner box of every rank WITHOUT disturbing the communicator or the local patch.

The gather/scatter pieces are computed from these boxes, so a decomposition that MOVES (dynamic load balancing: Solver::redistribute) must push the new ones through here — otherwise every piece still addresses the previous partition's rank blocks.

Definition at line 172 of file block_exchange.hpp.

◆ setPatch()

void peclet::flow::vof::VofBlockExchange::setPatch ( Patch  p,
SField  uf,
SField  vf,
SField  wf 
)
inline

The local face velocity, in the ADVECTOR's high-face convention (i.e.

already bridged with vof::copyFaceVelocity), on the local patch.

Definition at line 181 of file block_exchange.hpp.

◆ gatherBytes()

long peclet::flow::vof::VofBlockExchange::gatherBytes ( ) const
inlineoverridevirtual

Bytes moved by the last gather / scatter on THIS rank (sent + received).

Reimplemented from peclet::flow::vof::VofBlockExchangeBase.

Definition at line 195 of file block_exchange.hpp.

◆ scatterBytes()

long peclet::flow::vof::VofBlockExchange::scatterBytes ( ) const
inlineoverridevirtual

Reimplemented from peclet::flow::vof::VofBlockExchangeBase.

Definition at line 196 of file block_exchange.hpp.

◆ gatherMessages()

long peclet::flow::vof::VofBlockExchange::gatherMessages ( ) const
inline

Messages posted by the last gather / scatter on this rank (sends + receives).

Definition at line 198 of file block_exchange.hpp.

◆ scatterMessages()

long peclet::flow::vof::VofBlockExchange::scatterMessages ( ) const
inline

Definition at line 199 of file block_exchange.hpp.

◆ gatherImpl()

void peclet::flow::vof::VofBlockExchange::gatherImpl ( std::vector< VofBlock > &  blocks,
int  ghost,
bool  useExtended,
int  nc,
const SField *  loc,
SField(*)(VofBlock &, int)  blockView,
int  blockBase,
int  outside = 0 
)
inline

patch -> block.

useExtended picks the box; blockBase is 0 for an array indexed over the extended box and ghost for one indexed over the inner box.

Definition at line 226 of file block_exchange.hpp.

References appendMoveJobs(), buffer(), moveLocalPieces(), movePiece(), packFromPatch(), unpackToBlock(), and peclet::flow::vof::vofBuildPieces().

Referenced by gatherColour(), and gatherFaceVel().

◆ scatterImpl()

void peclet::flow::vof::VofBlockExchange::scatterImpl ( std::vector< VofBlock > &  blocks,
int  nc,
SField *  loc,
SField(*)(VofBlock &, int)  blockView,
int  blockBase,
int  op,
bool  gatherLocal = false 
)
inline

block -> patch, combined with op (0 = max, 1 = sum).

The patch's INNER region is zeroed first: both combines start from an empty field (UNPACK_MAX's empty union is pure gas; UNPACK_SUM's is no force), so a cell no marker covers reads exactly 0.

gatherLocal (UNPACK_SUM only; the CSF force, doc/vof_step_performance_design.md §5.9): the pieces of the blocks this rank masters are not combined block by block but GATHERED in one launch – per patch cell, the blocks in block order, each adding its value if its box holds the cell. That is the per-block combine's addition sequence (0, then the local masters in block order, then the received pieces), so the sum is the same double.

Definition at line 336 of file block_exchange.hpp.

References buffer(), combineBlockIntoPatch(), combineBufIntoPatch(), gatherLocalSum(), packFromBlock(), peclet::flow::vof::vofBuildPieces(), and zeroPatchInner().

Referenced by scatterColourMax(), scatterColourSum(), and scatterForceSum().

◆ gatherLocalSum()

void peclet::flow::vof::VofBlockExchange::gatherLocalSum ( std::vector< VofBlock > &  blocks,
int  nc,
SField *  loc,
SField(*)(VofBlock &, int)  blockView,
int  blockBase 
)
inline

The local half of a gatherLocal UNPACK_SUM: every patch INNER cell written (so no zeroing pass is needed) with 0 + the values of the blocks this rank masters whose box holds it, in block order.

A block box is at most one period long on a periodic axis, so a patch cell maps to at most one block-local cell – the one vofBuildPieces maps it to. More than kVofBlockBatch blocks: later chunks continue each cell's sum from the value the previous chunk left, which is the same addition sequence.

Definition at line 437 of file block_exchange.hpp.

References gatherLocalLaunch(), peclet::flow::vof::kVofBlockBatch, peclet::flow::I3::x, peclet::flow::I3::y, and peclet::flow::I3::z.

Referenced by scatterImpl().

◆ gatherLocalLaunch()

◆ zeroPatchInner()

void peclet::flow::vof::VofBlockExchange::zeroPatchInner ( int  nc,
SField *  loc 
)
inline

◆ packFromPatch()

void peclet::flow::vof::VofBlockExchange::packFromPatch ( const VofPiece &  p,
int  nc,
const SField *  loc,
SField  buf,
long &  off 
)
inline

◆ unpackToBlock()

void peclet::flow::vof::VofBlockExchange::unpackToBlock ( const VofPiece &  p,
int  nc,
SField  buf,
long &  off,
SField(*)(VofBlock &, int)  blockView,
VofBlock &  b,
int  blockBase 
)
inline

Definition at line 619 of file block_exchange.hpp.

References peclet::flow::L3().

Referenced by gatherImpl().

◆ movePiece()

void peclet::flow::vof::VofBlockExchange::movePiece ( const VofPiece &  p,
int  nc,
const SField *  loc,
SField(*)(VofBlock &, int)  blockView,
VofBlock &  b,
int  blockBase,
bool  toBlock 
)
inline

The master's OWN cells: patch <-> block with no buffer and no host traffic.

Definition at line 644 of file block_exchange.hpp.

References peclet::flow::vof::VofBlockExchange::Patch::e, peclet::flow::vof::VofBlockExchange::Patch::g, peclet::flow::L3(), and peclet::flow::vof::VofBlockExchange::Patch::o.

Referenced by gatherImpl().

◆ appendMoveJobs()

void peclet::flow::vof::VofBlockExchange::appendMoveJobs ( const VofPiece &  p,
int  nc,
const SField *  loc,
SField(*)(VofBlock &, int)  blockView,
VofBlock &  b,
int  blockBase,
std::vector< VofMoveJob > &  out 
)
inline

H-4(e): the (piece, component) copies of movePiece for one piece, appended to out.

Definition at line 666 of file block_exchange.hpp.

References peclet::flow::vof::VofBlockExchange::Patch::e, peclet::flow::vof::VofBlockExchange::Patch::g, and peclet::flow::vof::VofBlockExchange::Patch::o.

Referenced by gatherImpl().

◆ moveLocalPieces()

template<class Exec = SExec>
void peclet::flow::vof::VofBlockExchange::moveLocalPieces ( const std::vector< VofMoveJob > &  mv)
inline

H-4(e), host: every collected movePiece copy in one launch per chunk of kVofBlockBatch (piece, component) pairs, rows inside, through a by-value job table – instead of one MDRange3 per piece per component.

Pure copies; the pieces partition each block box, so no two jobs write the same cell (asserted in debug builds) and the result is the per-piece launches'.

Definition at line 689 of file block_exchange.hpp.

References peclet::flow::vof::kVofBlockBatch, and peclet::flow::vof::vofMoveOverlap().

Referenced by gatherImpl().

◆ packFromBlock()

void peclet::flow::vof::VofBlockExchange::packFromBlock ( const VofPiece &  p,
int  nc,
SField(*)(VofBlock &, int)  blockView,
VofBlock &  b,
int  blockBase,
SField  buf,
long &  off 
)
inline

Definition at line 717 of file block_exchange.hpp.

References peclet::flow::L3().

Referenced by scatterImpl().

◆ combineBlockIntoPatch()

void peclet::flow::vof::VofBlockExchange::combineBlockIntoPatch ( const VofPiece &  p,
int  nc,
SField(*)(VofBlock &, int)  blockView,
VofBlock &  b,
int  blockBase,
SField *  loc,
int  op 
)
inline

◆ combineBufIntoPatch()

void peclet::flow::vof::VofBlockExchange::combineBufIntoPatch ( const VofPiece &  p,
int  nc,
SField  buf,
long &  off,
SField *  loc,
int  op 
)
inline

◆ faceViewOf()

static SField peclet::flow::vof::VofBlockExchange::faceViewOf ( VofBlock &  b,
int  c 
)
inlinestatic

Definition at line 791 of file block_exchange.hpp.

Referenced by gatherFaceVel().

◆ colourViewOf()

static SField peclet::flow::vof::VofBlockExchange::colourViewOf ( VofBlock &  b,
int  c 
)
inlinestatic

Definition at line 792 of file block_exchange.hpp.

Referenced by gatherColour(), scatterColourMax(), and scatterColourSum().

◆ forceViewOf()

static SField peclet::flow::vof::VofBlockExchange::forceViewOf ( VofBlock &  b,
int  c 
)
inlinestatic

Definition at line 796 of file block_exchange.hpp.

Referenced by scatterForceSum().

◆ gatherFaceVel()

void peclet::flow::vof::VofBlockExchange::gatherFaceVel ( std::vector< VofBlock > &  blocks,
int  ghost 
)
inlineoverridevirtual

Fill every MASTER block's faceU/V/W over its EXTENDED box, from the face velocity the ranks own.

Cells outside a non-periodic domain come from the ghost layers of the rank whose box touches that domain face — the doubles the global-field advector consumes there. They are NOT a clamp fill: uf(i) is the HIGH face of cell i, so the LOW domain face (a wall's normal velocity, an inflow's datum) sits at global index -1. A zero-gradient copy of index 0 put the first interior face's velocity on a no-slip wall, and every marker with colour in the wall-adjacent layer then gained or lost volume through the wall (bubble column, 2026-09-25: up to 1e-5 of a marker per step).

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 798 of file block_exchange.hpp.

References deviceStaging, faceViewOf(), peclet::flow::vof::VofBlockExchange::Patch::g, gatherFaceVelHost(), and gatherImpl().

◆ gatherColour()

void peclet::flow::vof::VofBlockExchange::gatherColour ( std::vector< VofBlock > &  blocks,
SField  cLocal 
)
inlineoverridevirtual

Fill every MASTER block's INNER colour from the caller's colour patch (the mirror of scatterColourMax).

Used to seed a block from an arbitrary global colour field — the general seeding path, which a sphere seed is only a convenience over.

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 812 of file block_exchange.hpp.

References colourViewOf(), gatherImpl(), and ghostOf().

◆ scatterColourMax()

void peclet::flow::vof::VofBlockExchange::scatterColourMax ( std::vector< VofBlock > &  blocks,
SField  cLocal 
)
inlineoverridevirtual

UNION the masters' inner colour into the caller's local colour patch: zero the inner region, then C = max(C, C_block) cell by cell (TBFsolver's UNPACK_MAX).

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 817 of file block_exchange.hpp.

References colourViewOf(), deviceStaging, ghostOf(), scatterColourMaxHost(), and scatterImpl().

◆ scatterForceSum()

void peclet::flow::vof::VofBlockExchange::scatterForceSum ( std::vector< VofBlock > &  blocks,
SField  fx,
SField  fy,
SField  fz 
)
inlineoverridevirtual

SUM every master block's CSF face force into the caller's three face-force patches (TBFsolver's UNPACK_SUM, VOF.f90::computeSurfaceTension): overlapping markers ADD their forces, which is what makes two touching bubbles push on the fluid twice and not once.

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 826 of file block_exchange.hpp.

References forceViewOf(), ghostOf(), and scatterImpl().

◆ scatterColourSum()

void peclet::flow::vof::VofBlockExchange::scatterColourSum ( std::vector< VofBlock > &  blocks,
SField  sLocal 
)
inlineoverridevirtual

S = sum_k C_k (vof_overlap_design §5.6): the colour scatter with UNPACK_SUM.

With at most two markers on a cell the sum is exact whatever the order (IEEE addition is commutative).

Reimplemented from peclet::flow::vof::VofBlockExchangeBase.

Definition at line 833 of file block_exchange.hpp.

References colourViewOf(), ghostOf(), and scatterImpl().

◆ migrateColour()

void peclet::flow::vof::VofBlockExchange::migrateColour ( std::vector< VofBlock > &  blocks,
const std::vector< int > &  oldMaster 
)
inlineoverridevirtual

Move a block's colour from its old master to its new one after a re-assignment (rung W1 item a).

The two boxes are identical (the table was replicated before the re-assignment) so the whole extended array travels as one contiguous message and lands bit for bit.

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 841 of file block_exchange.hpp.

References buffer(), colour(), and peclet::flow::vof::VofBlock::kAuxLen.

◆ migrateBytes()

long peclet::flow::vof::VofBlockExchange::migrateBytes ( ) const
inline

Definition at line 897 of file block_exchange.hpp.

◆ migrateMessages()

long peclet::flow::vof::VofBlockExchange::migrateMessages ( ) const
inline

Definition at line 898 of file block_exchange.hpp.

◆ buffer()

SField peclet::flow::vof::VofBlockExchange::buffer ( long  n)
inline

Reusable device buffer of exactly n doubles, handed out in REQUEST ORDER — two live messages of the same size must not be the same View, so the pool is keyed by the request sequence (reset at the head of every transfer) and not by the size.

The request sequence of a step is deterministic, so after the first few steps nothing is allocated at all; an allocation per message would otherwise dominate the device-vs-host packing comparison.

Definition at line 905 of file block_exchange.hpp.

Referenced by gatherImpl(), migrateColour(), and scatterImpl().

◆ ghostOf()

static int peclet::flow::vof::VofBlockExchange::ghostOf ( const std::vector< VofBlock > &  blocks)
inlinestatic

◆ gatherFaceVelHost()

void peclet::flow::vof::VofBlockExchange::gatherFaceVelHost ( std::vector< VofBlock > &  blocks,
int  ghost 
)
inline

Definition at line 928 of file block_exchange.hpp.

References peclet::flow::vof::vofBuildPieces().

Referenced by gatherFaceVel().

◆ scatterColourMaxHost()

void peclet::flow::vof::VofBlockExchange::scatterColourMaxHost ( std::vector< VofBlock > &  blocks,
SField  cLocal 
)
inline

◆ syncTable()

void peclet::flow::vof::VofBlockExchange::syncTable ( std::vector< VofBlock > &  blocks)
inlineoverridevirtual

Replicate the (possibly re-centred) boxes so every rank's table agrees again.

Implements peclet::flow::vof::VofBlockExchangeBase.

Definition at line 1130 of file block_exchange.hpp.

Member Data Documentation

◆ deviceStaging

bool peclet::flow::vof::VofBlockExchange::deviceStaging = true

Definition at line 222 of file block_exchange.hpp.

Referenced by gatherFaceVel(), and scatterColourMax().


The documentation for this class was generated from the following file: