Characterization & benchmarks

Not what the suite can do — how well it does it: accuracy, performance, and parallel scaling, measured.

The examples gallery shows what the peclet suite can do. This companion set of pages measures how well it does it — the numbers you’d want before trusting a solver or choosing between its options:

Accuracy — how fast does a quantity of interest converge to ground truth as the mesh is refined, and to what limit? (Order of accuracy, and the discretization bias that survives refinement.)
Performance — wall-clock and solver-iteration cost per unknown, and how it scales with resolution.
Parallel scaling — strong and weak scaling across MPI ranks / GPUs on the shared core decomposition and halo exchange.

Each page is a self-contained, runnable characterization script: it builds its own geometry, sweeps a parameter (mesh, rank count, …), and reports a table plus a figure with a stated headline number. Like the examples, the solver-backed pages are executed on real hardware and frozen into the site, so the public build needs no GPU. Where there is an external ground truth (an analytic solution or a published benchmark), the characterization is measured against it.

Pages

peclet.dem vs the DEM8 bulk-process benchmark →

The three Dosta et al. (2024) community benchmark cases — silo emptying (100k), drum mixing (38k bimodal), and ball impact on 25–100k beds — run from the paper’s shared initial states, head-to-head against MUSEN (GPU) and LIGGGHTS (24 MPI ranks) re-run on the same machine. The first run found real gaps and drove a three-day solver campaign (per-pair materials, friction-cone tangential impulses, a staged stabilization pass); the final solver sits inside or at the reference envelope on silo rate and Beverloo ratio, drum amplitude, and impact penetration — at a 100× larger time step and 2.5–7× less GPU wall-clock than the study’s fastest code. Every intermediate configuration is kept as the diagnostic record.

dem · accuracy · performance · benchmark · validation

Staggered vs collocated: Stokes-drag convergence →

The two peclet.flow velocity placements, put head-to-head on the Zick–Homsy sphere array. The staggered MAC grid shows textbook second-order convergence onto the benchmark; the collocated grid’s error is bias-dominated — it sits ~1% high and mesh refinement does not remove it. Quantifies the intrinsic velocity-placement bias (and what it costs in wall-clock), so you know which grid to reach for and what the trade is.

flow · accuracy · staggered vs collocated · Stokes · verification

peclet.flow vs FoxBerry: strong scaling →

Head-to-head on FoxBerry’s own 64M-cell benchmark cases — single-phase flow and a 5000-sphere packed bed with FoxBerry’s inlet/outlet/wall BCs — 24 → 1536 cores on Snellius, plotted onto FoxBerry’s own graphs. peclet is 13–25× faster single-phase and 6.5–8× on the bed, with flat pressure-iteration counts across the whole ladder. The page tells the story of the three fixes the campaign forced: coarse-level telescoping of the geometric multigrid on the ORB tree (the old hierarchy stopped coarsening when a per-rank block turned odd, and that alone cost a third of the efficiency), a consensus-protocol race in the halo topology builder that hung the top rungs intermittently, and a momentum solve that was stopping on the wrong criterion — burning its sweep cap every step until a residual-based stop and a velocity multigrid under MPI halved the step again.

flow · parallel scaling · code comparison · multigrid

Channel DNS multi-GPU scaling on Snellius H100 →

The performance companion to the wall-bounded turbulence DNS: measured weak scaling of the distributed incompressible solver on NVIDIA H100 with GPU-aware MPI (46 M cells/GPU fixed, 1→8 GPUs / up to 364 M cells) — near-ideal on-node (80 % at the 182 M production point), with the inter-node pressure-solve reduction tax quantified. Includes the distributed-multigrid correctness fix that made the large runs possible.

flow · HPC · MPI · multi-GPU · weak scaling · Snellius

Parallel scaling vs CaNS, incflo & OpenFOAM →

The systematic scaling study on a boundary-free tiled Taylor–Green case: CPU (MPI × OpenMP hybrid) and multi-GPU weak scaling from a 24-core workstation to 32 H100s on Snellius, head-to-head against CaNS (the FFT state of the art), incflo (the AMReX exascale solver in peclet’s method family) and OpenFOAM on identical grids. Headlines: 90 % weak efficiency and 7.7 Gcell/s on 32 H100s (1.5 billion cells, five steps per second); the crossover against CaNS lands before two nodes on CPU and between 4 and 8 GPUs — its FFT all-to-all collapses below 10 % on both fabrics while peclet and incflo hold ~90 %; and peclet leads every general-geometry reference on every platform (4.2× OpenFOAM, 3.2× incflo per CPU node, 6.3× incflo on one GPU) at expert-tuned reference settings.

flow · HPC · MPI · CPU + GPU · weak scaling · CaNS · incflo · OpenFOAM · decomposition

Porous-bed scaling & permeability →

The parallel-scaling companion on a real geometry: creeping flow through DEM-grown random sphere packings (φ = 0.50), cut-cell and ghost-cell IBM, 1→32 H100s. Upscale ladder (256³ cells/GPU, per-rung beds): cut-cell pressure iterations flat at 21–33/step to 537 M cells with permeability domain-size independent. Refine ladder (one bed, sphere radius 16→64 cells): the two IBMs’ independent $k_\infty$ estimates agree to 0.1 %. Also documents two bugs the study flushed out: a deterministic dem packing corruption specific to the CUDA 12.6/sm_90 build, and a ghost-projection march instability at the largest elongated rungs.

flow · dem · HPC · MPI · multi-GPU · IBM · permeability · Snellius

Voronoi update throughput: repair vs rebuild, workstation → H100 →

The engine figure of merit for every moving-cell method in peclet.voro: how fast a Voronoi tessellation of moving points is brought up to date each step, against rebuilding it from scratch. A per-step repair (certificate → re-evaluate → re-clip the flagged cells) runs 8.7× faster than a cold build on an 8-thread host and 4.0–4.6× on H100s at 400k seeds per GPU, with 80 % weak-scaling efficiency to 4 GPUs and cell volumes exact to 1e-11 — but only after the benchmark itself exposed that the first certificate silently missed 0.1 % of the gained neighbours per step. Also records what is not yet fast: the SDF-walled repair costs a cold build while flagging no cell, and the RTX 5080 update path is launch-latency bound.

voro · performance · MPI · multi-GPU · weak scaling · Snellius

How to read these

A characterization is only as good as its ground truth and its honesty about what it didn’t measure. Every page states the reference it compares against, the regime it holds in, and — where a result plateaus or a run is resolution-limited — says so rather than tuning until the curve looks clean. Surprises get logged to ISSUES.md, the same backlog the examples feed.

--- title: "Characterization & benchmarks" subtitle: "Not what the suite can do — how *well* it does it: accuracy, performance, and parallel scaling, measured." toc: false --- The [examples](../index.qmd) gallery shows *what* the `peclet` suite can do. This companion set of pages measures *how well* it does it — the numbers you'd want before trusting a solver or choosing between its options: - **Accuracy** — how fast does a quantity of interest converge to ground truth as the mesh is refined, and to *what* limit? (Order of accuracy, and the discretization bias that survives refinement.) - **Performance** — wall-clock and solver-iteration cost per unknown, and how it scales with resolution. - **Parallel scaling** — strong and weak scaling across MPI ranks / GPUs on the shared `core` decomposition and halo exchange. Each page is a **self-contained, runnable characterization script**: it builds its own geometry, sweeps a parameter (mesh, rank count, …), and reports a table plus a figure with a stated headline number. Like the examples, the solver-backed pages are executed on real hardware and **frozen** into the site, so the public build needs no GPU. Where there is an external ground truth (an analytic solution or a published benchmark), the characterization is measured *against it*. ## Pages ::: {.grid} ::: {.g-col-12 .g-col-md-6} ### [peclet.dem vs the DEM8 bulk-process benchmark →](dem-bulk-dosta2024/index.qmd) The three Dosta et al. (2024) community benchmark cases — silo emptying (100k), drum mixing (38k bimodal), and ball impact on 25–100k beds — run from the paper's shared initial states, head-to-head against **MUSEN** (GPU) and **LIGGGHTS** (24 MPI ranks) re-run on the same machine. The first run found real gaps and drove a three-day solver campaign (per-pair materials, friction-cone tangential impulses, a staged stabilization pass); the final solver sits inside or at the reference envelope on silo rate and Beverloo ratio, drum amplitude, and impact penetration — at a 100× larger time step and 2.5–7× less GPU wall-clock than the study's fastest code. Every intermediate configuration is kept as the diagnostic record. *dem · accuracy · performance · benchmark · validation* ::: ::: {.g-col-12 .g-col-md-6} ### [Staggered vs collocated: Stokes-drag convergence →](staggered-vs-collocated/index.qmd) The two `peclet.flow` velocity placements, put head-to-head on the Zick–Homsy sphere array. The staggered MAC grid shows textbook second-order convergence **onto** the benchmark; the collocated grid's error is **bias-dominated** — it sits ~1% high and mesh refinement does not remove it. Quantifies the intrinsic velocity-placement bias (and what it costs in wall-clock), so you know which grid to reach for and what the trade is. *flow · accuracy · staggered vs collocated · Stokes · verification* ::: ::: {.g-col-12 .g-col-md-6} ### [peclet.flow vs FoxBerry: strong scaling →](foxberry-scaling/index.qmd) Head-to-head on **FoxBerry's own** 64M-cell benchmark cases — single-phase flow and a 5000-sphere packed bed with FoxBerry's inlet/outlet/wall BCs — 24 → 1536 cores on Snellius, plotted onto FoxBerry's own graphs. peclet is **13–25× faster** single-phase and **6.5–8×** on the bed, with **flat pressure-iteration counts across the whole ladder**. The page tells the story of the three fixes the campaign forced: coarse-level *telescoping* of the geometric multigrid on the ORB tree (the old hierarchy stopped coarsening when a per-rank block turned odd, and that alone cost a third of the efficiency), a consensus-protocol race in the halo topology builder that hung the top rungs intermittently, and a momentum solve that was stopping on the wrong criterion — burning its sweep cap every step until a residual-based stop and a velocity multigrid under MPI halved the step again. *flow · parallel scaling · code comparison · multigrid* ::: ::: {.g-col-12 .g-col-md-6} ### [Channel DNS multi-GPU scaling on Snellius H100 →](channel-scaling/index.qmd) The performance companion to the [wall-bounded turbulence](../examples/wall-bounded-turbulence/index.qmd) DNS: measured **weak scaling** of the distributed incompressible solver on NVIDIA H100 with GPU-aware MPI (46 M cells/GPU fixed, 1→8 GPUs / up to 364 M cells) — near-ideal on-node (80 % at the 182 M production point), with the inter-node pressure-solve reduction tax quantified. Includes the distributed-multigrid correctness fix that made the large runs possible. *flow · HPC · MPI · multi-GPU · weak scaling · Snellius* ::: ::: {.g-col-12 .g-col-md-6} ### [Parallel scaling vs CaNS, incflo & OpenFOAM →](parallel-scaling/index.qmd) The systematic scaling study on a boundary-free tiled Taylor–Green case: CPU (MPI × OpenMP hybrid) and multi-GPU weak scaling from a 24-core workstation to 32 H100s on Snellius, head-to-head against **CaNS** (the FFT state of the art), **incflo** (the AMReX exascale solver in peclet's method family) and **OpenFOAM** on identical grids. Headlines: **90 % weak efficiency and 7.7 Gcell/s on 32 H100s** (1.5 billion cells, five steps per second); the crossover against CaNS lands **before two nodes on CPU and between 4 and 8 GPUs** — its FFT all-to-all collapses below 10 % on both fabrics while peclet and incflo hold ~90 %; and peclet leads every general-geometry reference on every platform (4.2× OpenFOAM, 3.2× incflo per CPU node, 6.3× incflo on one GPU) at expert-tuned reference settings. *flow · HPC · MPI · CPU + GPU · weak scaling · CaNS · incflo · OpenFOAM · decomposition* ::: ::: {.g-col-12 .g-col-md-6} ### [Porous-bed scaling & permeability →](porous-scaling/index.qmd) The parallel-scaling companion on a *real geometry*: creeping flow through DEM-grown random sphere packings (φ = 0.50), cut-cell **and** ghost-cell IBM, 1→32 H100s. Upscale ladder (256³ cells/GPU, per-rung beds): cut-cell pressure iterations **flat at 21–33/step to 537 M cells** with permeability domain-size independent. Refine ladder (one bed, sphere radius 16→64 cells): the two IBMs' independent $k_\infty$ estimates **agree to 0.1 %**. Also documents two bugs the study flushed out: a deterministic dem packing corruption specific to the CUDA 12.6/sm_90 build, and a ghost-projection march instability at the largest elongated rungs. *flow · dem · HPC · MPI · multi-GPU · IBM · permeability · Snellius* ::: ::: {.g-col-12 .g-col-md-6} ### [Voronoi update throughput: repair vs rebuild, workstation → H100 →](voro-update-throughput/index.qmd) The engine figure of merit for every moving-cell method in `peclet.voro`: how fast a Voronoi tessellation of *moving* points is brought up to date each step, against rebuilding it from scratch. A per-step **repair** (certificate → re-evaluate → re-clip the flagged cells) runs **8.7× faster than a cold build on an 8-thread host** and 4.0–4.6× on H100s at 400k seeds per GPU, with **80 % weak-scaling efficiency to 4 GPUs** and cell volumes exact to 1e-11 — but only after the benchmark itself exposed that the first certificate silently missed 0.1 % of the gained neighbours per step. Also records what is *not* yet fast: the SDF-walled repair costs a cold build while flagging no cell, and the RTX 5080 update path is launch-latency bound. *voro · performance · MPI · multi-GPU · weak scaling · Snellius* ::: ::: ## How to read these A characterization is only as good as its ground truth and its honesty about what it *didn't* measure. Every page states the reference it compares against, the regime it holds in, and — where a result plateaus or a run is resolution-limited — says so rather than tuning until the curve looks clean. Surprises get logged to [`ISSUES.md`](../ISSUES.md), the same backlog the examples feed.