Design decisions — flow¶
Harvested verbatim from the project's working notes. Each entry records a decision, the
alternative it rejected, and the stated reason. This file is evidence, not a summary —
quotes are unedited. Read docs/DECISIONS.md first; come here for depth.
Bootstrapped by a one-time harvest of the working notes (2026-09-10); maintained by hand
from here on — add an entry when you make a decision a future session could undo. Regenerate
the index with docs/decisions/build_index.py after editing.
Do not reverse an entry here without recording a new decision that supersedes it.
Collocated momentum advection uses the PROJECTED divergence-free face field, not the cell→face average¶
- area: flow
- source: flow/doc/uf_advection.md; amr/docs/amr_flow_uniform_parity.md §3/§7 (P1)
- decided: 2026-09-21
- status: settled
-
quote: | On the collocated grid the advecting velocity is the projected, discretely divergence-free MAC face field
uf_/vf_/wf_produced by the previous step's approximate (ABC) projection — read verbatim at the control volume's faces — NOT the un-projected cell→face average ½(u_i+u_j). The average survives only as the developer-tier ablationdiagnostics.set_uf_advection(False), which reproduces the pre-change binary BIT FOR BIT (state_hashcolocated_advecte7e376f79469,colocated_advect_bc93bde5cd6f16).Three pre-existing statements all prescribed it: flow's own design note (
doc/flow_colocated_plan.md§1 step 3, "these u_f become the advecting velocities for the next step's advection"), the Almgren–Bell–Colella convention (the field the projection just made solenoidal IS the conservative advective flux), and the FOU operator's conservative row-sum identity, whichdocs/decisions/flow.mdalready recorded "holds only for uniform/div-free advecting field".colocated_advection.hpphad said "this header's adv_vel is where that swap happens" since phase 2 and the swap had never been made.Measured: max|div_h(advecting field)| 7.5e-06 → 4.6e-15 at N=32 (1.7e-07 → 2.1e-16 at N=64); Taylor–Green L2 vs the exact solution ~1 % BETTER at the same observed order at N=32/64/128; the amr↔flow uniform-grid NS parity gap 2.49e-04 → 1.90e-11 over 20 steps. The ten pre-existing state_hash cases (staggered bed, all four collocated Stokes schemes, channel, VoF droplet, scalar, porous, moving scene) are bit-identical: the staggered grid and every advection-off collocated baseline are untouched. - rejected: the un-projected cell→face average ½(u_i+u_j) as the collocated advecting velocity (the phase-2 form, flow's behaviour from phase 2 until 2026-09-21); also rejected: leaving the difference in place as amr_flow_uniform_parity.md P1's "not taken" default, now superseded by this decision. - why: "the field the projection just made solenoidal IS the conservative advective flux"; the FOU row-sum identity needs a divergence-free advecting field; it was the last uniform-grid difference between peclet.flow and peclet.amr. Accuracy is NOT the reason — on smooth flow the two are within 1 %; consistency is. - note: | Does NOT reverse
docs/decisions/amr.md:13(amr already advects with uf); it aligns flow to it. Does NOT touch the suite-wide ABC-not-Rhie–Chow prohibition: the residual CELL divergence is intrinsic to cell-centred placement and no advecting-velocity choice removes it. Consequence for amr: itstg_advect_matched.jsonparity case ablates amr to flow's OLD choice, so it now measures the difference rather than the agreement — that case wants updating on the amr side.
"Hand the stopping level to GraphAMG" telescoping idea is retired¶
- area: flow
- source: mg-decomposition-alignment.md:75
- decided: 2026-09-01
- status: settled
- quote: | The "hand the stopping level to GraphAMG" idea is retired — it gathers the operator to every rank and solves serially, scaling the wrong quantity.
- rejected: handing the MG-telescoping stopping level to GraphAMG
- why: "it gathers the operator to every rank and solves serially, scaling the wrong quantity"
"Intrinsic ~1% gap, don't chase it" corrected: collocated IBM is first-order at curved walls, staggered is second-order¶
- area: flow
- source: sdflow-collocated-solver.md:46-56
- decided: 2026-07-04
- status: superseded
- quote: | KEY FINDING — CORRECTED 2026-07-04 (supersedes the old "intrinsic ~1% gap, don't chase it"). The collocated solver is first-order accurate at curved immersed boundaries; the staggered solver is second-order. Both converge to the SAME Z&H limit. The "~1% gap" seen at N≈32 is first-order boundary error, NOT a resolution-independent floor — it roughly halves each time N doubles.
- rejected: "the old 'intrinsic ~1% gap, don't chase it'" conclusion
- why: "PROVEN by a staggered-vs-collocated Z&H sphere-drag convergence to N=128... collocated +0.60→+0.40→+0.30% at N=64/96/128 (clean local p≈1.0)"
"No portable GPU wheel" was a policy choice, not a hard limit — single-GPU CUDA wheels are feasible¶
- area: flow
- source: peclet-cuda-wheel-feasible.md:10-13
- decided: 2026-07-03
- status: superseded
- quote: |
DEPLOYMENT.md / the umbrella pyproject say GPU peclet is not pip-installable. That's a policy choice,
not a hard limit — the cited blocker (arch × CUDA × MPI-ABI) really only applies to multi-GPU MPI.
For single-GPU the Python module links no MPI, so a pip wheel is just a packaging exercise (like
cupy-cuda12x/ PyTorch). - rejected: the DEPLOYMENT.md / umbrella pyproject claim that GPU peclet cannot be pip-installable
- why: the arch x CUDA x MPI-ABI blocker only applies to multi-GPU MPI builds; a single-GPU module links no MPI
(1,2) mixed closure order is the recommended default going forward¶
- area: flow
- source: flow-ghost-projection.md:50
- decided: undated
- status: settled
- quote: | (1,2) MIXED is the recommended default going forward: quadratic RHS = 2nd-order steady constraint on the 7-point near-symmetric linear matrix; operator mismatch converges through time stepping (measured ρ(I−A_lin⁻¹A_quad)=0.40; vs binary-M lagging 1.087 = divergent). Z&H: all three modes identical drag to 3 decimals, order ~1.9; 7-point variants hold FLAT 7 BiCGStab iters N=32..128 (13-point grew 11→16).
- rejected: binary-M lagging (divergent, ρ=1.087) and linear-everywhere (1,1) (worse pointwise near the IB)
- why: "operator mismatch converges through time stepping (measured ρ(I−A_lin⁻¹A_quad)=0.40; vs binary-M lagging 1.087 = divergent)"
(superseded) fp32-floor-explains-everything hypothesis¶
- area: flow
- source: vof-campaign.md:366-381
- decided: 2026-08-31
- status: superseded
- quote: |
- (superseded) fp32 rival hypothesis, first raised (2026-08-31).
mac_cutcell_mg.hpp:51using MReal = float;— the MG operator storage is fp32 [...] Consequence: do NOT start S3 coefficient-coarsening work before ruling out the fp32 floor — a coarsening fix that ignores a precision floor is a wasted campaign.
- (superseded) fp32 rival hypothesis, first raised (2026-08-31).
- rejected: superseded by "WO-M FINAL... TWO mechanisms" (float floor is depth-independent; S3 indefiniteness deepens with depth — both real, separated)
- why: none stated at time of the original hypothesis; refuted later by the depth-independence discriminator
(superseded) precision-vs-indefiniteness verdict corrected — S3 indefiniteness is real and separate from the float-storage floor¶
- area: flow
- source: vof-campaign.md:346-347
- decided: 2026-08-31
- status: superseded
- quote: |
- (superseded) earlier verdict: "it is precision, not indefiniteness; S3's evidence is contaminated" — half right (the float defect is real) but the S3 conclusion did not survive.
- rejected: "it is precision, not indefiniteness; S3's evidence is contaminated"
- why: "WO-M FINAL 2026-08-31: S3 STANDS — the pivot SURVIVES in double" (from the surrounding ★★★ note)
1/2-1/2 constraint truncation exonerated as the plateau cause — a previously-published claim was wrong¶
- area: flow
- source: collocated-second-order-verdict.md:29-34
- decided: undated
- status: superseded
- quote: |
- The ½/½ CONSTRAINT truncation (and the embed port's "defect (a)" as the cause): a-priori vs
exact quadrature, signed net defect 1e-5→1e-6 of through-flux, 3 decades under the bias
(
flow/tests/study/collocated_constraint_consistency.py). This claim was in CLAUDE.md/the example page and is WRONG as an explanation of the plateau.
- The ½/½ CONSTRAINT truncation (and the embed port's "defect (a)" as the cause): a-priori vs
exact quadrature, signed net defect 1e-5→1e-6 of through-flux, 3 decades under the bias
(
- rejected: the ½/½ constraint truncation (embed port's "defect (a)") as the cause of the collocated accuracy plateau
- why: measured signed net defect is 3 decades under the observed bias; the claim had been published in CLAUDE.md/the example page and is wrong
2nd order finally resolved by ghost-cell projection FD family, not the full Basilisk embed rewrite¶
- area: flow
- source: sdflow-collocated-solver.md:10-27
- decided: 2026-07-17
- status: superseded
- quote: |
✅ 2ND-ORDER PROBLEM RESOLVED (2026-07-17): the collocated GHOST projection [...] Fix =
set_ghost_projection(True,1,2)on SolverColocated: staggered ghost closures/matrix verbatim on the face-averaged field + NEWgpCenterGraddirectional cell gradient (central / one-sided 2nd-order, never reads decoupled cells) for the predictor AND cell correction. [...] The Basilisk-embed conclusion below ("needs one shared FV geometry") is thereby SUPERSEDED — a consistent point-based FD family works; embed modes 4–7 remain ablations. - rejected: the earlier conclusion that a full Basilisk-embed FV rewrite was required
- why: "a consistent point-based FD family works" — defaults stay byte-identical, Z&H order restored
A masked solid cell is not a fluid sample¶
- area: flow
- source: sdf-scene-campaign.md:44
- decided: undated
- status: settled
- quote: | a masked solid cell is not a fluid sample (cost a sign-flipped moving-wall force, §7 item 2).
- rejected: none stated
- why: treating a masked solid cell as a fluid sample "cost a sign-flipped moving-wall force"
API/ordering: x-fastest is fixed; PyVista/VTK [x,y,z] + F-contiguous convention¶
- area: flow
- source: suite-distributed-status.md:177-184
- decided: undated (commits de16af5, c272046)
- status: settled
- quote: | API/ordering decision (commits de16af5, c272046): dcfd uses the PyVista/VTK convention (the leading package): get_u/v/w return 3-D arrays indexed u[x,y,z], shape (nx,ny,nz), F-contiguous → x fastest in memory (ParaView/VTI-compatible, bytes unchanged). set_ accept 3-D [x,y,z] arrays of any order (read via strides). To avoid the ravel('F') footgun for VTI export, added get_u/v/w_flat()* returning the x-fastest 1-D buffer directly. (The numpy C-vs-F tension is unavoidable: no convention is both [x,y,z]-indexed AND default-ravels-to-x-fastest; PyVista resolves it with [x,y,z]+order='F', which dcfd matches.) User: x-fastest is FIXED (non-negotiable, ParaView); dislikes order='F' and [z,y,x].
- rejected: order='F' as the user-facing default and [z,y,x] indexing
- why: x-fastest memory layout is required for ParaView/VTI compatibility (non-negotiable per user)
Agglomerated redundant coarse solve deferred pending multi-rank hardware¶
- area: flow
- source: suite-distributed-status.md:296-299
- decided: undated (MG Phase 3)
- status: uncertain
- quote: | The agglomerated redundant coarse solve (for large-np scaling) is DEFERRED — its benefit needs many-rank/multi-GPU hardware (dev box has 1 GPU; the threshold wouldn't even trigger at np≤4, so it'd ship untested).
- rejected: shipping the agglomerated coarse solve untested at np<=4
- why: dev box has 1 GPU; the feature's threshold wouldn't trigger at low np
Agglomerated/"auto" bottom solve default is gated to the singular (mean-removed) path only, not enabled universally¶
- area: flow
- source: agglomerated-bottom-ibm-fix.md:45
- decided: 2026-08-14
- status: settled
- quote: |
DEFAULT FLIPPED 2026-08-14 (flow 3493a89, umbrella 72c29c2):
set_pressure_bottomdefault = auto, gated to the SINGULAR path (removeMean_) — the sweep caught a real regression on the Dirichlet-anchored/outflow path (div floor 8e-8 → 2e-5 on a 128×32×32 inflow/outflow channel...). Anchored operators keep the smoothed bottom byte-identical... - rejected: enabling "auto" bottom solve on Dirichlet-anchored/outflow operators
- why: "the sweep caught a real regression on the Dirichlet-anchored/outflow path (div floor 8e-8 → 2e-5 ...); inner solve exact AND CSR consistent ... so NOT the fixed bugs; suspected float round-off"
Agreed IBM velocity-MG coarse-op scheme: volume-fraction Helmholtz, coarse-only solid masking, masked volume-weighted transfers¶
- area: flow
- source: velocity-mg-design.md:37-52
- decided: undated
- status: superseded
- quote: |
The agreed scheme (correction scheme, true residual, NO un-scale):
- Coarse op = clean volume-fraction Helmholtz
A_θ(smoothed momentum balance): θ = fluid volume fraction from the SDF (smoothed Heaviside, the volumetric analogue of the pressure path'smg_coarsen_open_avg_karea openness), coarsened 2:1; face β_f = νΔt·min(θ_i,θ_nbr)/h²; volume-weighted diagonal AC = θ_i + Σβ_f. Symmetric M-matrix. (Plain Galerkin/rediscretized of A_f is wrong — momentum is non-conservative across the IBM boundary.) - Small cells → solid ONLY on the coarse op (θ<ε ⇒ identity row). Safe because a misclassified coarse cell costs convergence RATE only, never accuracy (the fine A_f smoother + true residual fix it). Do NOT ε-mask the fine grid — that's exactly the small-cell job D_rescale already does accurately.
- Masked, volume-weighted transfers (the diagnosed real bias: unmasked transfers move corrections
ACROSS the wall, polluting the cut-cell skin). Restrict
Σθ_f r_f / Σθ_f(fluid-side weighted; an extensive boundary FORCE coarsens by fluid volume — this is the "apply the clean stencil to the IBM field ⇒ residual = wall force" view); prolong masked (no correction into θ<ε cells).
- Coarse op = clean volume-fraction Helmholtz
- rejected: plain Galerkin/rediscretized coarse operator of A_f (non-conservative across the IBM boundary); ε-masking the fine grid; unmasked transfers
- why: "momentum is non-conservative across the IBM boundary" for plain Galerkin; unmasked transfers "move corrections ACROSS the wall, polluting the cut-cell skin"
- conflict: this volume-fraction-weighted coarse op design was later replaced by the STAIRCASE (binary θ≥0.5 classification, no volume-fraction coefficients) as the consolidated default — see "STAIRCASE coarse operator is now the ONLY IBM coarse op" entry below
Agreement across configurations sharing a floor is not proof of convergence¶
- area: flow
- source: defect-correction-campaign.md:65-70
- decided: 2026-09-01
- status: settled
- quote: | Cold-start floor (collocated session, 2026-09-01) — CAN INVALIDATE P1: [...] (b) agreement across configs is NOT convergence when they share a floor — three R=48 runs agreed to 9 digits while all starved. P1's four columns agreeing would have read as GO. Gate 3 now needs cold AND warm legs.
- rejected: treating cross-config agreement alone as sufficient evidence of convergence
- why: "three R=48 runs agreed to 9 digits while all starved"
All level-0 partitioning must go through one shared factory; never hand-build a BlockDecomposer for the solver¶
- area: flow
- source: mg-decomposition-alignment.md:56
- decided: undated
- status: settled
- quote: |
- The three level-0 partition call sites (
mpi_block,IbmSolver::initMpi,CutcellMG::initMpi) now share ONE factoryCutcellMG::decomposition(). Never hand-build aBlockDecomposerfor the solver; they must agree or the block layout diverges silently.
- The three level-0 partition call sites (
- rejected: hand-building a separate BlockDecomposer at any of the three call sites
- why: "they must agree or the block layout diverges silently"
Allreduce diet: fuse mean-removal sum+count; 'fine' mean-removal scope as bench/pack default, 'all' stays solver default pending Snellius validation¶
- area: flow
- source: parallel-scaling-study.md:81-86
- decided: 2026-08-08
- status: settled
- quote: | (3) ALLREDUCE DIET: mean-removal {sum,count} fused to ONE allreduce (bit-identical, always on) + set_pressure_mean_removal('fine') drops interior-level/post-matvec projections → 17.6→5.4 allreduces/Krylov-iter, iters EXACTLY unchanged (7.5==7.5), KE digit-identical; 'all' stays solver default until Snellius validates; bench driver + snellius pack default MEANSCOPE=fine.
- rejected: none stated (interim state, not yet finalized)
- why: fusing sum+count is bit-identical; 'fine' scope needs Snellius validation before becoming the solver default
Anisotropic MG coarsening order: coarsen axis a iff H_a < 2·min H over coarsenable axes, engaged only under aniso¶
- area: flow
- source: physical-units-phase2-aniso.md:47-51
- decided: 2026-09-06/07
- status: settled
- quote: |
Decisions taken (do not reopen):
- ⚑A coarsening order (both MGs, one helper
mgChooseRatio): coarsen axis a iff coarsenable andH_a < 2 * min Hover the coarsenable axes,H_a = h_a' cfac_a; ENGAGED ONLY WHENaniso(the isotropic table, incl. after telescoping, stays today's — a lagging unblocked axis would otherwise change bits at extent=None).
- ⚑A coarsening order (both MGs, one helper
- rejected: applying the anisotropic coarsening rule to the isotropic path (would change isotropic bits)
- why: "a lagging unblocked axis would otherwise change bits at extent=None"
Anisotropic wall-gradient normal/foot-point convention (⚑B)¶
- area: flow
- source: physical-units-phase2-aniso.md:52-54
- decided: 2026-09-06/07
- status: settled
- quote: |
- ⚑B normal:
n_a = (g_a/h_a')/|g/h'|(physical),m_a = n_a/h_a'(index direction); foot pointxi - d' m; image distances physical; dominant axis by |m|; fragment areasqrt(sum W_a^2 w_a); slips*lambda'/(h_a'|n_a|);ibmVolfrac=0.5 + d'|m|under aniso only.
- ⚑B normal:
- rejected: none stated
- why: none stated
Aperture weighting (A1) is not indicated as a fix for the confined finite-Re deficit¶
- area: flow
- source: advective-cutwall-flux-plan.md:26
- decided: 2026-08-31
- status: superseded
- quote: |
ten-cate did NOT recover (E1
0.781/0.777/0.749 → 0.803/0.797/0.766vs 0.947; E3/E4 still above u∞) ⇒ the confined finite-Re deficit is NOT the advective cut-wall flux, and A1 (aperture weighting / D2) is NOT indicated — nothing in the remaining symptoms implicates it. - rejected: A1 (aperture weighting / D2) as the fix for the ten-cate deficit
- why: "nothing in the remaining symptoms implicates it"
- conflict: this conclusion is itself later reframed as moot once advective-cutwall-flux-plan.md:49 finds the true (geometric) cause — A1 remains correctly not-indicated, but for a different final reason.
Area fractions (proper face α, not min(θ)) extend the stable dt ceiling for the (now-removed) area-fraction coarse op¶
- area: flow
- source: velocity-mg-design.md:118-124
- decided: 2026-06-17
- status: superseded
- quote: |
Replaced the
min(θ)face coefficient with proper AREA fractions (ibm_areafrac_k: α=clamp(0.5+ψ_face,0,1) at each control-volume −face; coarsened by averaging via the pressure path'smg_coarsen_open_avg_k;mg_build_velocity_op_areafrac_k: diagonal 1+Σβ_f, β_f=νΔt·α_f/h², θ for the ε-solid test). Mirrors the proven rediscretized PRESSURE MG. Result: extends the stable dt ceiling — area-frac+exclude is exact & stable to dt=400 at φ=0.216 (was dt=200 with min(θ)); φ=0.5236 stays dt=200. Kept (genuine improvement). - rejected: min(θ) as the coarse-op face coefficient
- why: measured to extend stable dt ceiling to 400 (from 200)
- conflict: this area-fraction operator was itself removed in the later "staircase is the ONLY IBM coarse op" consolidation
Body-force ghost policy: Neumann copy, pinned to ρ's policy¶
- area: flow
- source: vof-campaign.md:145-154
- decided: 2026-08-30
- status: settled
- quote: |
Ghost policy chosen = Neumann copy, same
as ρ, because
buildRhsVarface-averages force and ρ with the SAME mean and the physical content is f_f/ρ_f — pin the force's policy to ρ's or the ratio breaks at the boundary. - rejected: none stated (implicitly, any policy not matching ρ's)
- why: "buildRhsVar face-averages force and ρ with the SAME mean and the physical content is f_f/ρ_f — pin the force's policy to ρ's or the ratio breaks at the boundary"
Boiling scope addition did not resurrect transported-φ CLSVOF¶
- area: flow
- source: vof-campaign.md:101-106
- decided: 2026-08-30
- status: settled
- quote: | Phase change = Part II of the plan (interfacial ṁ, velocity jump/divergence source in the projection, T_sat interface condition); strengthens the band-RDF investment (normal-probe gradients). Boiling did NOT resurrect transported-φ CLSVOF pending the Part II literature check.
- rejected: transported-φ CLSVOF
- why: none stated (pending Part II literature check)
Byte-identical gating is unmeasurable for atomic-add paths — gate against the path's own run-to-run spread instead¶
- area: flow
- source: defect-correction-campaign.md:55-57
- decided: 2026-09-02
- status: settled
- quote: |
TRAP — mode B is not bit-reproducible against itself:
starApplyDeltauses Kokkos::atomic_add (star_elimination.hpp:155,163), so 3 runs of one CUDA binary give 3 states (spread 1.1e-16). A "byte-identical" gate is UNMEASURABLE there; gate against the path's own run-to-run spread. - rejected: a byte-identical regression gate for atomic-add code paths
- why: "3 runs of one CUDA binary give 3 states"
CA (event-halving) smoothing ships default ON with an env kill switch¶
- area: flow
- source: comm-scaling-plan.md:23
- decided: undated
- status: settled
- quote: |
- CA smoothing (2.3) — DONE, flow b96dd74: one 2-deep exchange per RB pair, first colour
redundantly re-smooths the 1-deep ghost ring, second colour needs NO exchange. Env
PECLET_FLOW_CA(default ON, =0 kill switch); engages only where every rank's block extent ≥ 4 AND the operator is periodic/IBM.
- CA smoothing (2.3) — DONE, flow b96dd74: one 2-deep exchange per RB pair, first colour
redundantly re-smooths the 1-deep ghost ring, second colour needs NO exchange. Env
- rejected: none stated
- why: measured 35%→62% weak efficiency gain, "almost entirely 2.3's event halving"
CSF force is exact; curvature error is the real budget — improve κ, never the force¶
- area: flow
- source: vof-campaign.md:430-444
- decided: 2026-08-30
- status: settled
- quote: |
F_c = σ·κ_f·(C(i)−C(i−s_c))/hadded by a siblingaddCsfRhsusing the projection's own face difference. Loud gate: stationary droplet max|u| = 1.93e-17, spurious Ca 1.9e-18; the ablationset_csf_mode(1)(arithmetic cell-force interpolation) gives 5.76e-02 / Ca 5.8e-03 = 3.0e+15× worse. [...] ★ Ca ≲ 1e-7 IS NOT REACHABLE, structurally: with EXACT κ the force gives 1.9e-18, so the force is exact [...] Ca ≈ δκ·h ⇒ the budget is a CURVATURE requirement [...] Improve κ (V5 RDF, ELVIRA/LVIRA on 5³), never the force. - rejected: arithmetic cell-force interpolation (set_csf_mode(1)); improving the force term to chase lower Ca
- why: "the force is exact" once κ is exact; "the budget is a CURVATURE requirement"
CUDA wheel dependency name: nvidia-cuda-runtime, not the deprecated -cu13 suffix¶
- area: flow
- source: peclet-cuda-wheel-feasible.md:28-30
- decided: 2026-07-04
- status: settled
- quote: |
flow/packaging/pyproject-cuda.toml: namepeclet-flow-cu13, depnvidia-cuda-runtime(NOT-cu13— NVIDIA dropped the-cuXXsuffix at CUDA 13; the-cu13pkg is a deprecated placeholder; real one is 13.3.29, lays libcudart.so.13 at site-packages/nvidia/cu13/lib).
- rejected: depending on the
nvidia-cuda-runtime-cu13package name - why: NVIDIA dropped the -cuXX suffix at CUDA 13; that package name is now a deprecated placeholder
Cell-average IBM scheme, not point-value, for Poiseuille validation¶
- area: flow
- source: cuda-kokkos-migration.md:285-287
- decided: undated
- status: settled
- quote: | CRITICAL: use the CELL-AVERAGE scheme (ibmFillEntry<1>); point-value <0> is ~10% off at this resolution.
- rejected: point-value scheme (ibmFillEntry<0>)
- why: "point-value <0> is ~10% off at this resolution"
Channel DNS IC must seed streamwise rolls/streaks in wall units, not cold random noise¶
- area: flow
- source: channel-dns-isotropic-grid.md:46
- decided: 2026-08-05
- status: settled
- quote: | PRODUCTION LAMINARIZED → TRANSITION FIX (2026-08-05, peclet-examples 9b1c86b): ... the cold random-noise IC failed to trigger the self-sustaining cycle at low Re_tau=180 ... FIX (channel_dns_mpi.py): IC now seeds streamwise ROLLS+STREAKS at spanwise wavelength ~110 wall units, perturbation scales in WALL UNITS (resolution-independent; old lp() cutoff was cycles/CELL → shrank into viscous range at fine dx).
- rejected: cold random-noise initial condition; perturbation cutoff specified in cycles/cell
- why: "the cold random-noise IC failed to trigger the self-sustaining cycle at low Re_tau=180"; cycles/cell cutoff "shrank into viscous range at fine dx"
Chebyshev pressure driver's apparent win was a placement artifact, not a real gain¶
- area: flow
- source: momentum-solve-residual-stop.md:39
- decided: undated
- status: settled
- quote: | at 1536 ranks node placement moves the SAME config by 1.5x (single 0.391 vs 0.256 s) — every top-rung A/B needs a same-allocation control; the Chebyshev pressure driver looked like a 36 % win until the control showed parity (and 5x LOSS on the cut-cell bed, 238 vs 40 iterations).
- rejected: adopting the Chebyshev pressure driver based on the uncontrolled 36% win
- why: "the control showed parity (and 5x LOSS on the cut-cell bed, 238 vs 40 iterations)"
Clean-fluid-interior mask required on both restriction and prolongation for IBM velocity-MG correctness¶
- area: flow
- source: velocity-mg-design.md:54-64
- decided: 2026-06-16
- status: settled
- quote: |
## SOLVED (2026-06-16) — clean-fluid-interior coarse coupling makes the IBM velocity-MG exact + stiff-stable
Volume-fraction coarse op (
mg_build_velocity_volfrac_op_k, diagonal 1+Σβ_f, NOT θ-weighted — θ-weighting the I term shrinks the diagonal vs the fine As_[c] O(1) diagonal → overshoot; the backward-Euler op is a POINT eq, only the diffusion flux sees θ: β_f=νΔt·min(θ_i,θ_nbr)/h²). The KEY FIX (Frank's diagnosis of the logical error): the coarse grid must couple ONLY where its clean operator matches the fine one. So a clean-fluid-interior mask (ibm_clean_fluid_mask_k: 1 only at fluid cells with no solid neighbour; 0 at IBM cut cells AND solid cells) is applied to BOTH the restriction (zero the residual there:mg_mul_mask_konlv.res_maskbefore restrict) AND the prolongation (mg_prolong_masked_k, no correction INTO those cells). The fine IBM RB-GS smoother owns the cut-cell band + the stiff 1+6β solid interior; the coarse grid solves the clean interior. - rejected: θ-weighting the diagonal's identity (I) term; coupling the coarse grid at cut/solid cells (masking cut cells only, not solid cells, was tried and still failed — see below)
- why: "θ-weighting the I term shrinks the diagonal vs the fine As_[c] O(1) diagonal → overshoot"; "the coarse grid must couple ONLY where its clean operator matches the fine one"
Coarse-first decomposition ships opt-in, legacy remains the default¶
- area: flow
- source: mg-decomposition-alignment.md:22
- decided: 2026-08-11
- status: settled
- quote: |
Shipped 2026-08-11 (core c98964a, flow 7e0a3b1, umbrella 8ec5f39, all pushed): coarse-first
decomposition, OPT-IN (
flow.set_decomposition_levels(L)/PECLET_FLOW_DECOMP_LEVELS, 0 = legacy default). - rejected: making coarse-first decomposition the default
- why: none stated beyond shipping it opt-in
Coarse-level solve policy default stays "smoother", not "auto", because auto regresses the cut-cell IBM path¶
- area: flow
- source: mg-decomposition-alignment.md:28
- decided: 2026-08-11
- status: settled
- quote: |
coarse-level solve policy
set_pressure_bottom("smoother"|"auto"|"agglomerated")+BOTTOMenv in the channel driver. DEFAULT STAYS "smoother" —automakes the cut-cellrandom_spheresN=48 regression WORSE (442→622 iters, +41 %, accuracy bit-identical). An exact coarse solve cannot legitimately do that ⇒ the assembled coarse operator is inconsistent with the V-cycle's on the IBM path. That is now open problem #3 and it BLOCKS making auto the default. - rejected: making "auto" the default coarse-level solve policy
- why: "auto makes the cut-cell random_spheres N=48 regression WORSE (442→622 iters, +41%, accuracy bit-identical)"
Collocated pressure coupling is the ABC (MAC) approximate projection, NOT Rhie–Chow¶
- area: flow
- source: sdflow-collocated-solver.md:37-39
- decided: undated (2026-06-22 build)
- status: settled
- quote: |
The collocated pressure coupling is the Almgren–Bell–Colella
approximate (MAC) projection (NOT Rhie–Chow): average cell→face (
centerToFace), make the face field divergence-free with the SAMECutcellMG, then correct cell velocities with a (now openness-aware) cell-pressure gradient (projectCorrectCenter). - rejected: Rhie–Chow interpolation as the collocated coupling
- why: none stated (states what it is, not why it isn't R-C)
Container slab half-extent must equal L/2 + wall thickness, never more (periodic-image rule)¶
- area: flow
- source: advective-cutwall-flux-plan.md:57
- decided: 2026-09-02
- status: settled
- quote: | RULE: container slab half-extent = L/2 + wall thickness, never more.
- rejected: an oversized slab half-extent
- why: an oversized slab lets its periodic images refill the cavity (see the ten-cate root cause above)
Correction/supersession: plan §9.3 U5 was wrong about VoF spacing¶
- area: flow
- source: physical-units-plan.md:32
- decided: 2026-09-06
- status: superseded
- quote: |
- The plan's §9.3 U5 is wrong about the VoF spacing. Under representation (B) the advector runs on
the unit lattice and
v*dt'ISu*dt/h; passing hRef converts twice. The1.0s stay; sigma, kappa and every TIME convert at the boundary instead.
- The plan's §9.3 U5 is wrong about the VoF spacing. Under representation (B) the advector runs on
the unit lattice and
- rejected: superseded claim = "§9.3 U5" (passing hRef to convert v*dt' under representation B)
- why: "the advector runs on the unit lattice and vdt' IS udt/h; passing hRef converts twice"
Correction: the raw field registry hands out internal (unconverted) arrays¶
- area: flow
- source: physical-units-plan.md:28
- decided: 2026-09-06
- status: settled
- quote: |
- The RAW field registry (
field_view/get_field/set_field) hands out INTERNAL arrays, whileget_u/get_pconvert. Any driver writingforce_x/drag_betadirectly must scale — that is whats.unit_scales(a dict,identityTrue in cell units) exists for
- The RAW field registry (
- rejected: none stated
- why: none stated (documents an internal-vs-converted API split that must be respected)
Correction: §9.3 U3's setter list was incomplete, causing silent physical errors¶
- area: flow
- source: physical-units-plan.md:35
- decided: 2026-09-06
- status: superseded
- quote: |
- §9.3 U3's setter list was short by four —
set_property_modelon rho/mu (the documented two-phase spelling),enable_vof_momentum,set_vof_kappa_constant,set_phase_change_fit_curvature. Missing them made a two-phase physical run silently wrong bylam^3in density andlam^1.5in the capillary dt.
- §9.3 U3's setter list was short by four —
- rejected: superseded claim = the original (short) §9.3 U3 setter list
- why: "Missing them made a two-phase physical run silently wrong by lam^3 in density and lam^1.5 in the capillary dt"
Coupling partial/cut+solid cells to the coarse grid fails regardless of coarsening depth — capping depth cannot fix it¶
- area: flow
- source: velocity-mg-design.md:118-135
- decided: 2026-06-17
- status: settled
- quote: |
Couple-partial-cells (drop the exclude mask) — FAILS, and the cap does NOT rescue it (measured): coupling
the partial/cut+solid cells to the coarse grid diverges at dt=200 even with a SINGLE coarsening level (L=2).
So the overshoot is at the very first coarse correction into the row-scaled IBM cut/solid cells, independent of
coarsening depth ⇒ a pore-scale cap cannot enable coupling (capping depth is at best a minor compute saving
when the exclude mask is already on). The clean-fluid exclude mask stays REQUIRED + default (
res_mask=true). Corrected lever for coupling = Brinkman/Darcy, NOT capping: coupling overshoots because the clean coarse op (even with area fractions) lacks the WALL DRAG the fine IBM imposes on partial cells → no resistance → overshoot. A Brinkman drag−μ/k·uon partial coarse cells (k from the local geometry) is the physically-motivated way to make the coarse op resist like the fine one — the only remaining route to coupling, but speculative (needs a permeability model) and unproven. - rejected: capping coarsening depth as a fix for partial-cell coupling divergence
- why: "the overshoot is at the very first coarse correction into the row-scaled IBM cut/solid cells, independent of coarsening depth"
Crank-Nicolson ported then reverted; backward-Euler is the right default¶
- area: flow
- source: suite-distributed-status.md:320-324
- decided: undated
- status: settled
- quote: | Crank-Nicolson: ported then REVERTED — the simple explicit Laplacian is inconsistent with the Robust-Scaled cut-cell IBM (θ=0.5 gave 4% θ-dependent steady error, a bug); correct CN needs the cut-cell operator in the explicit half (research); no steady benefit anyway → backward-Euler is the right default.
- rejected: Crank-Nicolson (θ=0.5) time integration with the plain explicit Laplacian
- why: inconsistent with the Robust-Scaled cut-cell IBM, causing a 4% θ-dependent steady-state error
Cut-cell C convention: liquid fraction of fluid volume, openness-weighted¶
- area: flow
- source: vof-campaign.md:98-100
- decided: 2026-08-30
- status: settled
- quote: | Cut-cell C convention APPROVED: C = liquid fraction of the fluid volume, openness-weighted fluxes.
- rejected: none stated
- why: none stated
Defect-correction proposal: float hierarchy demoted to pure preconditioner, evaluate before double-diagonal¶
- area: flow
- source: vof-campaign.md:293-306
- decided: 2026-09-01
- status: superseded
- quote: |
★★ USER PROPOSAL 2026-09-01 — DEFECT CORRECTION, evaluate BEFORE the double-diagonal.
r = b − A_exact·xin double, matrix-free asr(x); float operator only solves the correctionA_float·dx = −(float)r,x ← x + (double)dx. This demotes the float hierarchy to a pure preconditioner — its errors then affect only the convergence RATE, never the fixed point — soA·1 = 0breaking inside it becomes IRRELEVANT and the double-diagonal becomes unnecessary rather than merely cheaper. - rejected: shipping the double-diagonal as the fix
- why: "its errors then affect only the convergence RATE, never the fixed point"
- conflict: defect-correction-campaign.md (P1 result formally retires the double-diagonal fallback, confirming this proposal's direction won)
Deferred correction is not viable as an undamped fallback¶
- area: flow
- source: flow-ghost-projection.md:29
- decided: undated
- status: settled
- quote: | Deferred correction is NOT a viable fallback undamped: measured iteration rate max|1−λ| = 1.087.
- rejected: undamped deferred correction as the C/F / ghost solve fallback
- why: "measured iteration rate max|1−λ| = 1.087" (divergent)
Design constraints for any collocated-plateau fix¶
- area: flow
- source: collocated-second-order-verdict.md:51-55
- decided: undated
- status: settled
- quote: | Design constraints for any fix (Frank): C1 cells=conserved quantities, faces=volume flux, flux = volume-flux × face-interpolated quantity; C2 dt-INDEPENDENT steady states (bars Basilisk's dt·a face event AND both embed-note salvages); C3 AMR 2:1 faces; C4 no fragmenting graphs, keep CG.
- rejected: Basilisk's dt·a face event and both embed-note salvage approaches (ruled out by C2); any fix that fragments the CG graph (ruled out by C4)
- why: C2 requires dt-independent steady states; C4 requires keeping a non-fragmenting graph compatible with CG
Distributed cut-cell MG coarse levels must be nested, not independently re-decomposed¶
- area: flow
- source: channel-dns-isotropic-grid.md:52 (line ~740 in original combined read; per-file line 52 of the "ROOT CAUSE FOUND" bullet)
- decided: 2026-07-31
- status: settled
- quote: |
ROOT CAUSE FOUND (2026-07-31): distributed cut-cell MG coarse levels are NOT nested.
CutcellMG::initMpi(flow/src/mac_cutcell_mg.hpp:228) decomposes EACH MG level with an INDEPENDENTBlockDecomposerORB of that level's coarsened global grid (:246) — vs single-rankinit(:170) which halves in place (nested). Independent ORBs don't align across levels ... sorestrictAvg/prolongAdd— which assume coarse-local i ↔ fine-local ratio·i — read across the misaligned block boundary → OOB. ... FIX DESIGN: nested coarse decomposition — (1) core BlockDecomposer: add split-alignment ... +coarsened(ratio)(copy tree w/ splitValue,origins,sizes scaled by 1/ratio; preserves leaf/rank order); (2) mac_cutcell_mg initMpi: level L+1 decomp = level L.coarsened(ratio)not independent ORB. - rejected: independent per-level BlockDecomposer ORB for MG coarse levels
- why: "Independent ORBs don't align across levels ... read across the misaligned block boundary → OOB"
DistributedNS declared canonical; clean API; periodic BCs only; MPI a true build option¶
- area: flow
- source: suite-distributed-status.md:247-251
- decided: 2026-06-09
- status: settled
- quote: |
cfd-gpu shipped TWO implementations of the same
cut-cell IBM NS physics: production
pnm_backend(cfd_solver*.cu, in-kernelget_idxwrap, richCFDSolverAPI) and the extended-blockDistributedNS/dcfd. User wants ONE best code + ONE API + MPI optional. Decided (user):DistributedNSis canonical; clean new API (no pnm_backend compat); periodic BCs only (future BCs via the cut-cell IBMu_bc, not a halo-BC system); MPI a true build option. - rejected: keeping pnm_backend API compatibility; a halo-BC system for future boundary conditions
- why: user wants one best code, one API, MPI optional
DistributedNS solves in physical units (ρ, dynamic μ, force/volume), not kinematic ν¶
- area: flow
- source: sdflow-dt-divided-convention.md:46-59
- decided: 2026-06
- status: settled
- quote: |
(2) DistributedNS now solves in PHYSICAL units (was kinematic, rho≡1).
initsignature changed toinit(res, rank, size, rho, mu, dt, comm)— density + DYNAMIC viscosity (was a singlenu). [...] The stored variablep_is the physical pressure p (not a potential, not kinematic) [...] It's an exact rescaling (×rho) of the kinematic system -> velocity invariant, physical pressure unchanged - rejected: the kinematic (ρ≡1, single-ν) formulation as the solved system
- why: none stated beyond the unit-system change itself; validated as an exact rescaling
DistributedStokes renamed to DistributedNS¶
- area: flow
- source: suite-distributed-status.md:14-18
- decided: 2026-06-08
- status: settled
- quote: |
RENAME (2026-06-08): cfd-gpu's
DistributedStokesis nowDistributedNS(it's a full NS solver). Everywhere belowDistributedStokes→DistributedNS,dstokes→dns,src/distributed_stokes.cuh→src/distributed_ns.cuh,tests/test_distributed_stokes.cu→test_distributed_ns.cu. - rejected: none stated
- why: it's a full NS solver, not just Stokes
Divergence in mixed-order ghost projection (GPORDER 1,2) is caused by the incremental-rotational pressure accumulation, not the linear solve¶
- area: flow
- source: ghost-hardening-plan.md:50-59
- decided: 2026-08-17
- status: settled
- quote: |
LOCAL FAILING CASE (the phase-B/C testbed):
GPORDER=1,2(matrix_order=1, rhs_order=2 — the mixed mode the plan wanted to build C on) DIVERGES on s116, k → 4.5e77, ×1.62/step, threshold between 1956 and 7823 spheres and RESOLUTION-INDEPENDENT... Mechanism nailed: trajectory identical to 5 digits at PRTOL 1e-4 vs 1e-12 (not the linear solve), growth monotone inset_pressure_underrelax(ω_p)(stabilises R=3 outright; at R=5 only slows it — a damper, not a cure), and 10× smaller dt makes it WORSE ⇒ amplifier = the incremental-rotational accumulationP += (ρ/dt)φ. - rejected: attributing the (1,2) divergence to the linear solve tolerance or resolution
- why: trajectory is identical across PRTOL settings (rules out the linear solve); divergence worsens with smaller dt, pointing to the incremental-rotational accumulation as the amplifier
Divergence uses flux openness (beta); MG uses operator openness (alpha)¶
- area: flow
- source: cuda-kokkos-migration.md:325-327
- decided: undated (channel WIP session)
- status: settled
- quote: | THREE open-BC bugs found: (1) FIXED flux-vs-operator openness -- divergence uses FLUX beta (inflow-with-nonzero-normal-vel + outflow OPEN; walls + tangential-Dirichlet/lid CLOSED), MG uses OPERATOR alpha (inflow Neumann via setBC); (2)
- rejected: using the same openness definition for both divergence and MG operator
- why: none stated beyond the bug description; the fix separated the two openness definitions
Do not make the preconditioner rho-aware — harmful, and no compatibility floor exists in practice¶
- area: flow
- source: ghost-hardening-plan.md:26-37
- decided: 2026-08-17
- status: settled
- quote: |
- H1 refuted 3×: GP_THETA_MIN (1e-4) NEVER fires on any bed (min rho 1e-3); max|w| ≤ 971 and rho*w is O(1); the worst rows sit at grid/surface INCIDENCE coincidences...
- Preconditioner is fine: spec(M⁻¹A) |λ|min 0.32, λmax 2.15, spread 6.7 on a real bed. Making the preconditioner rho-aware is HARMFUL (split radius 1.15 → 15.4). Don't.
- No compatibility floor in practice: rtol sweep 1e-4→1e-10 gives 56→137 iters (linear in log rtol) — the solve converges, nothing burns on the stagnation guard.
- rejected: making the preconditioner rho-aware; the H1 hypothesis that GP_THETA_MIN fires and causes the trouble
- why: rho-aware preconditioning measured to worsen the split radius (1.15→15.4); GP_THETA_MIN never fires on any tested bed
Domain-BC all-fluid velocity must use double diffSmoothColor, not float IBM stencil¶
- area: flow
- source: cuda-kokkos-migration.md:318-320
- decided: undated (Session-2, 2026-06-19)
- status: settled
- quote: | (b) domain-BC all-fluid velocity MUST use the DOUBLE diffSmoothColor(+dcorr) (= CUDA diff_k), NOT the float IBM stencil (float gave only 1.5e-7);
- rejected: float IBM stencil for domain-BC all-fluid velocity
- why: "float gave only 1.5e-7" (double needed for accuracy)
Domain-BC paths stay blocking, following the VelocityMG precedent¶
- area: flow
- source: comm-scaling-plan.md:22
- decided: undated
- status: settled
- quote: | Domain-BC paths stay blocking (VelocityMG precedent).
- rejected: applying halo-compute overlap to domain-BC paths
- why: "VelocityMG precedent" (no further reason stated)
Drag must be included in bcStencilPath() whenever advection is off¶
- area: flow
- source: porous-cfddem-cuda-two-bugs.md:14
- decided: undated
- status: settled
- quote: |
Drag never in the momentum operator on the all-fluid domain-BC path: with
advect_=false(default; the fluidized-bed example never enables advection) and no immersed solid,bcStencilPath()was false →smoothCompused the CONST-COEFFICIENT fold smoother (Ac=ρ/dt+6μinline) → the drag-loaded band fromrebuildStencils+addDragDiagonalwas built every step and never read. [...] Fixed:bcStencilPath()includeshasDrag_. - rejected: none stated
- why: "pressure-loop gain = β·dt/ρ (measured 3.84 vs predicted 3.85 — diverges whenever β > ρ/dt)"
E2 resolved: the aspect coarsening rule alone fixes MG-PCG stall on stretched grids — no auto-FCG/symmetric-V-cycle decision needed¶
- area: flow
- source: physical-units-phase2-aniso.md:28-31
- decided: 2026-09-07
- status: settled
- quote: | C3 LANDED (flow 6cf870b, umbrella cd50c56): E2 RESOLVED by the rule alone — stretched MG-PCG 500-cap → 10 (cubic 9-10), pr 0.356 → 0.068 (isotropic ~0.06), level table (1,2,1)(2,2,1)(2,2,2)…; new ctest cutcellmg_aniso_mpi (106 MPI tests).
- rejected: needing a separate symmetric-V-cycle-under-aniso or auto-FCG-on-aniso decision
- why: the aspect coarsening rule alone resolved the stall (measured pr 0.356→0.068)
E3 decided: v3 wall-torque uses the area factor V'/h_b', not h_a'V', correcting the design note's §4.4¶
- area: flow
- source: physical-units-phase2-aniso.md:34-37
- decided: 2026-09-07
- status: superseded
- quote: | E3 DECIDED: the v3 wall-torque traction takes the AREA factor V'/h_b' on the area component (F' = mu' A'×Omega'), NOT h_a'V' on the force component — the note's §4.4 sentence was wrong; C4b implements it, C5 = docs (flow/CLAUDE.md + umbrella CONVENTIONS §7, PLAN §9.4/§9.8).
- rejected: h_a'V' on the force component (the design note's §4.4 as originally written)
- why: "the note's §4.4 sentence was wrong"
Earlier "+13-83% above 2024" deviation was a normalization error, not a real discrepancy¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:79-81
- decided: undated
- status: superseded
- quote: |
The "+13–83% above 2024" I reported earlier this session was a NORMALIZATION ERROR
(equated raw K with superficial F̄_D, dropped the (1−φ)); raw-K/F̄_D
tracks 1/(1−φ) exactly (1.26/1.54/1.83 vs 1.25/1.67/2.0).
resconv_vs2024.pyreinterpreted: raw K=5.48 → (1−φ)K=4.38 vs 4.67 = −6% (consistent). - rejected: the earlier "+13-83% above 2024" claim
- why: raw K was equated with superficial F_D, dropping the (1-phi) factor
Earlier "slow convergence / non-converged" worry was a multiplied-dt float-precision artifact, fixed by the divided-dt engine convention¶
- area: flow
- source: ringbed-cfd-surrogate.md:63
- decided: 2026-06-18
- status: settled
- quote: |
The earlier "slow convergence/+19%/non-converged" worry was the OLD multiplied-
dtfloat-precision artifact at large dt — the divided-dtengine ([[sdflow-dt-divided-convention]], [[ibm-accuracy-sphere-validation]]) fixed it: grid-convergence is now monotone, order 1.25, k_inf=1.195e-3, only 1.4% residual at 14.8 cells/wall... - rejected: the multiplied-dt momentum-operator convention
- why: divided-dt fixes the float-precision artifact and restores monotone grid convergence
Embed momentum + mode-3 projection (mode 5) does not converge — over-drags¶
- area: flow
- source: embed-port-progress.md:23
- decided: undated
- status: settled
- quote: |
- mode 5 (embed + mode-3 projection): plateaus ~+0.7%, does NOT converge — mode-3 projection over-drags.
- rejected: mode 5 (embed momentum + mode-3 projection) as a viable configuration
- why: "does NOT converge — mode-3 projection over-drags"
Escalation rule: a twice-failed gate stops the work order, never gets its numerics tweaked to pass¶
- area: flow
- source: vof-campaign.md:495-496
- decided: 2026-08-30
- status: settled
- quote: | Escalation rule: twice-failed gate = stop + findings log, never tweak numerics to pass.
- rejected: tweaking numerics to force a failing gate to pass
- why: none stated
Exact-crossing openness overrides must be masked to 0 at solid velocity points¶
- area: flow
- source: flow-ghost-projection.md:61
- decided: undated
- status: settled
- quote: |
set_openness_override(exact apertures, MUST be masked to 0 where the face velocity point is solid — the flux DOF is pinned there; unmasked exact apertures broke the projected divergence at 1.9e-3). - rejected: unmasked exact apertures at solid-velocity-point faces
- why: "unmasked exact apertures broke the projected divergence at 1.9e-3"
FV viscous operator confirmed 2nd-order-consistent — "face-flux placement" hypothesis refuted; barrier is the pressure coupling¶
- area: flow
- source: sdflow-collocated-solver.md:152-166
- decided: 2026-07-05
- status: settled
- quote: | A-priori truncation test [...] proves the FV VISCOUS OPERATOR is clean O(h²) (order 1.98-2.00, N=32→128). So the face-centre two-point flux IS 2nd-order-consistent (Basilisk right); the "face-flux placement" hypothesis is REFUTED. [...] Tried the spec's wall pressure term −p_i·W_c on the momentum force [...] dissipative sign is stable but WORSE (N32 +0.81→+1.04%), opposite sign diverges — NOT the fix.
- rejected: the "face-flux placement" hypothesis; the wall-pressure-term-on-momentum-force fix (both signs)
- why: "the FV VISCOUS OPERATOR is clean O(h²)"
Face-primary uf reconstruction (mode 8) attempted and reverted — incompatible with flow's divided time convention¶
- area: flow
- source: embed-port-progress.md:33
- decided: undated
- status: settled
- quote: | Face-primary uf attempted (mode 8) and FAILED — reverted. Basilisk centered.h fix = acceleration event uf=fs(face_avg(u)+dta) so the cut-cell body-force flux odta enters div(ouf) as a source. But Basilisk's explicit u+=dt(a-grad p) correction needs its (I-dtmuLap) convention; flow uses the DIVIDED convention (rho/dtcs - muLap_embed), whose implicit viscous barely dissipates at large dt → explicit dta body force runs away → -75% drag at N=32. Closing p=2 needs switching the momentum step to the undivided time convention (solver-wide) OR an implicit face-flux body-force source.* NOT clean p=2; mode 0 / Solver remain production.
- rejected: mode 8 (face-primary uf via Basilisk's acceleration-event reconstruction) under flow's current divided momentum-operator convention
- why: flow's divided convention's implicit viscous "barely dissipates at large dt", so the explicit dt*a body force term "runs away → -75% drag at N=32"; production stays mode 0 / Solver
Falling-drop gate: WO-K's under-resolved-momentum-solve suspicion is refuted¶
- area: flow
- source: vof-campaign.md:459-464
- decided: 2026-08-31
- status: superseded
- quote: | Falling drop (WO-K's deferred gate): the GATE measured the wrong quantity. [...] WO-K's suspected under-resolved momentum solve is REFUTED (60→2649 sweeps swings lab-frame 5.7× but relative velocity stays 0.826/0.828/0.828).
- rejected: "WO-K's suspected under-resolved momentum solve"
- why: "A periodic zero-mean body force conserves MOMENTUM, not volume flux"
Final default: momentum residual tolerance follows the pressure solver's rtol, not a fixed constant¶
- area: flow
- source: momentum-solve-residual-stop.md:53
- decided: 2026-09-02
- status: settled
- quote: |
Final default (user decision 2026-09-02, later the same day): the momentum residual
tolerance FOLLOWS THE PRESSURE SOLVER'S rtol (
velResTol_ < 0; the projection consumes u and resolves the divergence the momentum residual leaves to its own tolerance — no free constant). Guards: >= 1 sweep/V-cycle always (else the hydrostatic acid test drifts 1e-8), round-off floor (r <= 1e-14scale or no decrease between checks). - rejected: a fixed residual-stop constant (1e-5), decided earlier the same day
- why: "the projection consumes u* and resolves the divergence the momentum residual leaves to its own tolerance — no free constant"
- conflict: supersedes the same file's earlier same-day decision (see next entry)
Fine-scope MG-PCG promoted to default after lever ablation¶
- area: flow
- source: parallel-scaling-study.md:162-164
- decided: 2026-08-09
- status: settled
- quote: | LEVER ABLATION @8/16 GPU: fine-scope MG-PCG WINS (default PROMOTED in flow: meanRemovalAll_=false, gates rerun); meanall +5.5%, hoststage +12% (GPU-aware halo worth it), Chebyshev 2.6x WORSE (38 vs 12.2 iters — reduction-free loses to PCG's convergence), GraphAMG bottom 3-8x worse on GPU.
- rejected: 'all'-scope mean removal as the default; Chebyshev smoother; GraphAMG bottom solve (on GPU)
- why: fine-scope MG-PCG measured fastest; Chebyshev's reduction-free approach loses to PCG's convergence rate; GraphAMG bottom is 3-8x worse on GPU
Fix: refresh the MG halo before computing the residual, not only before each colour sweep¶
- area: flow
- source: cpu-fat-rank-optimization.md:14-22
- decided: undated (flow 5d77deb)
- status: settled
- quote: |
ROOT CAUSE (lever 1), found by instrumenting, not guessing — flow 5d77deb. The pressure
V-cycle's smoother exchanges the halo BEFORE each colour sweep, so on return from the pre-smooth
the ghosts are ONE COLOUR-UPDATE STALE. The residual (→ restricted coarse rhs) was therefore wrong
on the whole block-boundary shell... The perturbation scales with
block SURFACE ⇒ the V-cycle's convergence rate depended on the decomposition. Fix = refresh the
halo before the residual (distributed: exchange overlapped with the interior residual via the new
residualCutcellBox, same interior/shell split the smoother uses; single-rank: periodic wrap copy).PECLET_FLOW_MG_RESFILL=0restores the old behaviour as an ablation. - rejected: computing the MG residual from ghosts left stale by the pre-smooth's own halo exchange
- why: "boundary-adjacent cells reported their Gauss–Seidel residual (≈0) instead of the true one, so the coarse grid never saw that error"
Float MReal operator storage silently breaks A*1=0 at high MG contrast; peer's FCG/S3-indefiniteness fix is refuted for this case¶
- area: flow
- source: collocated-attractor-campaign.md:56
- decided: 2026-08-31
- status: settled
- quote: | FLOAT-STORAGE ROOT CAUSE 2026-08-31 ...: the R>=24 order-2-aperture ladder blocker ("high- contrast MG degradation", row 55) is ROOT-CAUSED = float MReal operator storage breaks A*1=0 at eps_f32; at 3-decade MS contrast the defect ~1e-4 relative on mixed rows shifts the near-null vector off the deflated constant -> PCG rebound / FCG floor 2e-6 / Chebyshev cap-burn, ALL drivers ... PROOF: -DPECLET_FLOW_MREAL_DOUBLE A/B at 192^3 = clean 1e-8 ...; float path bit-reproduced post-refactor. Cross-session: VoF peer (suite-c8) supplied FCG (...) + the indefinite-pivot S3 model — REFUTED on this case, their dense-probe evidence self-declared contaminated (float-backed V-cycle); their VoF rho-ratio-1000 case may independently reproduce (contrast per se).
- rejected: the VoF peer's indefinite-pivot S3 model as the explanation for this case's MG degradation
- why: "their dense-probe evidence self-declared contaminated (float-backed V-cycle)"; float MReal storage A/B (double build clean) is the proven cause
Fragmentation guard: BFS-isolated pockets are treated as solid for the projection only¶
- area: flow
- source: flow-ghost-projection.md:81
- decided: 2026-07-16
- status: settled
- quote: | the binary COUPLED-face graph FRAGMENTS (25/50/83 components at Ng=32/44/56) → per-pocket null vectors → BiCGStab breakdown → fields to 1e152. FIXED: host-BFS fragmentation guard in setSolid (pockets outside the main component become solid for the projection only).
- rejected: none stated (the unguarded fragmentation caused breakdown)
- why: "per-pocket null vectors → BiCGStab breakdown → fields to 1e152"
Free-variable fix: pin phi=0 at decoupled cells after the solve¶
- area: flow
- source: flow-ghost-projection-mpi-plan.md:19
- decided: 2026-07-23
- status: settled
- quote: |
KEY BUG found by the np=2 gate (fixed; affects single-rank too): solid-centered cells + fully-BC_ONLY rows are FREE variables of the binary-openness operator (zero row + zero rhs). The Krylov path left arbitrary decomposition-dependent phi there [...] Fix: pin phi=0 at decoupled cells after the solve (
gp_pin_decoupledin flow_ibm project()), per the ghost_projection.hpp doc contract. Side effect: staggered gp residual floor 4.3e-8 → 1.4e-13 on separated spheres — the old "compatibility floor" was this pollution leaking through EXPLICIT sliver faces. - rejected: leaving free variables (solid-centered / BC_ONLY rows) unpinned after the Krylov solve
- why: "the Krylov path left arbitrary decomposition-dependent phi there — invisible to every gp diagnostic, but injected into near-wall fluid u"
Fresh (newly-uncovered) cells must be seeded with the local wall velocity, not inherit the solid value¶
- area: flow
- source: sdf-scene-campaign.md:97
- decided: 2026-08-31
- status: settled
- quote: |
§7 item 4 (FRESH CELLS) RESOLVED 2026-08-31 flow
1a01769... inheriting the solid's value in a cell a moving body uncovers costs a RESOLUTION-INDEPENDENT +2.6..2.9% drag bias ... Seeding with the local wall velocity (uBc_, already computed) is now the DEFAULT: bias → −0.04..+0.38%, spurious force oscillation 20–50× down ... DECISIVE DIAGNOSTIC: without seeding the oscillation gets WORSE as dt is refined — the literature's signature that the source is SPATIAL; with seeding it converges. - rejected: inheriting the solid's prior value in a freshly-uncovered cell
- why: "a RESOLUTION-INDEPENDENT +2.6..2.9% drag bias"; "without seeding the oscillation gets WORSE as dt is refined"
GPU binding must let SLURM cgroup-isolate; manual CUDA_VISIBLE_DEVICES remap fights SLURM¶
- area: flow
- source: channel-dns-isotropic-grid.md:45
- decided: 2026-07-29
- status: settled
- quote: |
GPU-BINDING GOTCHA: manual CUDA_VISIBLE_DEVICES remap (PECLET_BIND_GPU=1) FOUGHT SLURM → oversubscription
(N=4 put 3 ranks on GPU0, 4× SLOWER). FIX:
srun --gpus-per-task=1 --gpu-bind=per_task:1+ PECLET_BIND_GPU=0 (SLURM cgroup-isolates; driver must NOT remap). - rejected: PECLET_BIND_GPU=1 manual remap
- why: "manual CUDA_VISIBLE_DEVICES remap ... FOUGHT SLURM → oversubscription ... 4× SLOWER"
GPU-aware MPI is not the shipped default on Snellius 2024a due to a toolchain conflict¶
- area: flow
- source: channel-dns-isotropic-grid.md:44
- decided: undated
- status: settled
- quote: | GPU-aware MPI KNOT: Snellius 2023 stack UCX-CUDA is built only for CUDA 12.1.1, but Kokkos 5.1.1 requires NVCC≥12.2.0 — mutually exclusive. GPU-aware DID validate at 12.1.1 ... but you can't build the solver there. ... Default shipped = CUDA/12.4.0 + host-staging.
- rejected: building the solver against the GPU-aware-validated CUDA 12.1.1 stack
- why: "you can't build the solver there" (NVCC≥12.2.0 required by Kokkos 5.1.1)
Galilean identity (A−idiag·I)·1=0 is false in general — corrects the plan's inventory¶
- area: flow
- source: defect-correction-campaign.md:27-28
- decided: 2026-09-01
- status: superseded
- quote: | (3) Galilean identity (A-idiagI)1=0 is FALSE in general — fou_operator is conservative, row sum = rho_fdiv_h(u^k) (per axis dt(velp-velm), since max(v,0)+min(v,0)=v); holds only for uniform/div-free advecting field
- rejected: the plan's assumption that (A−idiag·I)·1=0 holds generally
- why: "fou_operator is conservative, row sum = rho_f*div_h(u^k)... holds only for uniform/div-free advecting field"
Geometric const-coeff + masking are the validated defaults; Galerkin/CG is opt-in¶
- area: flow
- source: suite-distributed-status.md:98-100
- decided: undated (Step 19)
- status: settled
- quote: | Geometric const-coeff path + masking remain validated DEFAULTS (galerkin_=false); cut-cell+Galerkin+CG+flux-div+no-mask is a self-consistent opt-in.
- rejected: making Galerkin/CG the default
- why: none stated beyond "validated"
Ghost's rho scalar is not a partial cut-cell transplant — premise correction¶
- area: flow
- source: ghost-hardening-plan.md:21-25
- decided: 2026-08-17
- status: superseded
- quote: |
Premise corrections (all measured, don't re-derive):
- Ghost's one-scalar
rho = min(1, min_f D_f)is NOT a partial transplant of cut-cell's D_rescale — for a single-sided closure it is algebraically IDENTICAL toR = D_rescale/D_axis(ghost divides each face's weights by its own D then scales the row by min D; cut-cell folds both intoK = poly_Nc*R). Frank's and the plan's premise here was wrong.
- Ghost's one-scalar
- rejected: the premise that ghost's rho is only a partial transplant of cut-cell's D_rescale
- why: algebraically it is identical to R = D_rescale/D_axis for a single-sided closure — "Frank's and the plan's premise here was wrong"
Ghost-plane MASK must be exchanged under motion, not just velocity fields¶
- area: flow
- source: advective-cutwall-flux-plan.md:75
- decided: 2026-09-02
- status: settled
- quote: | Gate 7 CLOSED 2026-09-02: np=2 1.45e-7 → 4.1e-16. Mechanism = ghost-plane MASK (ibmSolidMask uses the clamping ccSampleExt) deciding which ghost rows get the A0 fill; uBc_ exchange alone changed nothing. Fix: exchangeExtRaw(C[c].mask) under hasMotion_ (+ uBc_/uwCell_ exchange). Ablation: inner-only or ghost-only fill → 1.4e-3 (both rows classes needed).
- rejected: exchanging only uBc_/velocity without exchanging the mask ("uBc_ exchange alone changed nothing"); inner-only or ghost-only mask fill (both insufficient, 1.4e-3 residual)
- why: the ghost-plane mask, not the velocity field, decides which rows get the A0 wall-velocity fill
Ghost-projection MPI design: gp-row ownership = inner-block cell; BiCGStab stages the iterate on a g=2 block rather than doing a second exchange¶
- area: flow
- source: flow-ghost-projection-mpi-plan.md:13-16
- decided: 2026-07-23
- status: settled
- quote: |
gp-row ownership = inner-block cell (buildGpOverlay already iterates exactly the rank's inner cells); all ±2 closure couplings are reads into the exchanged g=2 halo. [...] BiCGStab distributed matvec: the MG block only has g=1 ghosts but the overlay reaches ±2 → stage the iterate on the solver's g=2 block (
gpX2_), one 2-deep exchange, read the g=1 halo back from the staged copy (no second exchange),gpApplyDeltain ghost mode. - rejected: a second separate exchange to satisfy the ±2 overlay reach
- why: none stated beyond the design itself (avoids a second exchange)
Gibou-style NS scheme identified as a trap (adopted MAC/staggered for stability, not a collocated precedent)¶
- area: flow
- source: sdflow-collocated-solver.md:169-174
- decided: 2026-07-05
- status: settled
- quote: | keeping ABC does NOT block 2nd order — the whole COLLELA SCHOOL (Basilisk embed.h [V-verified collocated approx projection], Trebotich-Graves/EBChombo, Johansen-Colella 1998 foundation) reaches 2nd-order curved-wall drag WITH an approximate projection. PRIMARY REC = copy Basilisk embed.h [...] Gibou path = TRAP (a Gibou NS scheme adopted MAC/staggered for stability).
- rejected: the Gibou-path literature precedent for a collocated 2nd-order scheme
- why: "a Gibou NS scheme adopted MAC/staggered for stability"
GraphAMG re-enabled as porous+drag default after two BC-blind defects fixed¶
- area: flow
- source: porous-cfddem-cuda-two-bugs.md:27
- decided: 2026-07-06
- status: settled
- quote: |
GraphAMG∧domain-BC FIXED (was gated off): two BC-blind defects in
flow/src/mac_cutcell_mg.hpp— (a) buildAmg wrapped off-diagonals periodically across ALL boundaries (open Dirichlet outflow face coupled top↔bottom; now boundary-crossing faces on non-periodic axes keep the coefficient diagonal-only); (b) pcgAmg unconditionally mean-projected rhs/z (now gated on removeMean_ = !hasOutflow_). GraphAMG re-enabled as porous+drag default; validated resid 3.9e-11; regression +0.00%. - rejected: leaving GraphAMG gated off for domain-BC problems
- why: none stated beyond the fix description
Grid-convergence studies must report dimensionless permeability k* = k/N², not dimensional k¶
- area: flow
- source: sdflow-regression-suite.md:20-22
- decided: undated
- status: settled
- quote: | Key gotcha: geometry is self-similar in N (R = r_frac·N), so dimensional permeability k_cells ∝ N² (NOT convergence) — must report the dimensionless k* (k/N²) for a real grid-convergence study. Z&H's K is already dimensionless. Order is fit as f(N)=f_inf + C·N^-p (grid-search p + linear LS, dependency-free).
- rejected: reporting dimensional k_cells directly as a convergence metric
- why: "geometry is self-similar in N ... so dimensional permeability k_cells ∝ N² (NOT convergence)"
Grid-dimension convention: MG per-axis coarsening depends on factors of two; always check halvings before proposing a grid¶
- area: flow
- source: channel-scaling-rebenchmark.md:182-200
- decided: 2026-08-10
- status: settled
- quote: |
★★★ THE REAL ROOT CAUSE 2026-08-10 (peclet-examples 9371067): GRID DIMENSIONS' FACTORS OF TWO.
MG coarsening rule, read from
flow/src/mac_cutcell_mg.hpp(NOT guessed):can(d) = (d%2==0) && (d/2>=2), applied PER AXIS INDEPENDENTLY → semi-coarsening is already automatic: the long axis keeps halving after the short ones stop (level dump showsratio(2,1,1)). Depth is NOT set by the smallest direction — it is set by factors of two. An ODD dimension never coarsens even once.- Hierarchy stops when NO axis can coarsen, or at
nLevels(MGLEVELS). - MPI ONLY:
evenBlocks(ax)— an axis coarsens only if EVERY rank's block origin AND size are even on it. So distributed depth is set by the PER-RANK BLOCK, not the global grid... - MEASURED, single GPU, 384×128×GNZ, all else identical: GNZ=256 (8 halvings) 5.0 iters/120 ms; GNZ=250 (1) 9.8/195 ms; GNZ=255 (0) 16.2/329 ms → ONE ODD NUMBER = 3.2×, no MPI involved. ... ALWAYS check halvings before proposing a grid.
- rejected: choosing benchmark grid dimensions without checking per-axis halving depth (led to a "WORSE" refine ladder caught before burning GPU hours)
- why: "An ODD dimension never coarsens even once" — measured 3.2x iteration/time penalty from one odd dimension
Guidance: ghost for resolved/smooth IBM geometry, cutcell aperture for tight-throat porous media¶
- area: flow
- source: flow-ghost-projection.md:86
- decided: 2026-07-16
- status: settled
- quote: | VERDICT: ghost equally good on resolved/smooth IBM geometries with fewer iterations; cutcell remains preferable for under-resolved tight-throat porous media; ghost iterations also rise there (33–42 vs 14).
- rejected: using ghost projection for under-resolved tight-throat porous media
- why: "point-based faces can't throttle sub-cell throats the way apertures do"
Host-serial-kernel threshold lever kept despite measuring as marginal at the fat-rank size¶
- area: flow
- source: cpu-fat-rank-optimization.md:43-49
- decided: undated (flow 56bc731)
- status: settled
- quote: |
LEVER 2 (serial-below-threshold host kernels) = MARGINAL, kept anyway (flow 56bc731). ccFor3 +
the cut-cell RB-GS line sweeps run sequentially below
hostSerialCellCutoff()(8192 cells,PECLET_FLOW_HOST_SERIAL_CELLSoverrides). Bit-identical; reductions deliberately NOT cut over (FP order vs the multi-rank contract). Per-level timer says the win is almost entirely the MG BOTTOM level (~24 trivial launches/V-cycle): 64³/rank 0.018→0.007 s per 50 V-cycles ≈ 8% of the V-cycle; 128³/rank ~1.5%; 256³/rank (the fat-rank size) ~0.2% = noise. The plan's 15–20% estimate did NOT survive measurement. A bigger cutoff back-fires (131072: 64³ projection 16.6→20.6 ms). - rejected: cutting reductions over to serial-below-threshold execution; a larger serial-cutoff (131072)
- why: reductions kept parallel to preserve "FP order vs the multi-rank contract"; a bigger cutoff measured worse (16.6→20.6 ms)
How to apply: staggered recommended for accuracy-critical drag/permeability; collocated for structural wins, at the cost of first order at curved walls¶
- area: flow
- source: sdflow-collocated-solver.md:202-207
- decided: undated
- status: settled
- quote: | How to apply: recommend staggered when permeability/drag accuracy is the goal (2nd order, lands on Z&H at modest N); recommend collocated for its structural wins (cell-centered storage, easy coupling to cell-centered scalars/physics) — but it is FIRST-ORDER at curved immersed walls, so it needs finer grids there (or the embed-style fix).
- rejected: none stated
- why: none stated beyond the accuracy/structure tradeoff
IBM overlay needs no separate scaling change — linear in the base stencil¶
- area: flow
- source: sdflow-dt-divided-convention.md:24-26
- decided: 2026-06
- status: settled
- quote: |
The IBM overlay
(
ibm_modify_stencil_k) is LINEAR in the base stencil, so scaling the base by 1/dt scales the whole modified row + inhom automatically — no IBM-side change. - rejected: none stated
- why: linearity of the overlay in the base stencil
IBM velocity-MG must NEVER un-scale the residual by 1/D_rescale¶
- area: flow
- source: velocity-mg-design.md:24-35
- decided: undated
- status: settled
-
quote: | ## IBM-path correctness fix — the DECISION (so the wrong fix never resurfaces) The IBM (packed-object) diffusion vmg with the geometry-blind
setDiffusionCoarseconst-coeff coarse has a +2–4% Z&H drag bias at a fixed cycle count. The fix is NOT, and must never become, "un-scale the residual by 1/D_rescale". D_rescale (mac_ibm.cuh:105) is amin|D|cut-face factor that →0 for thin slivers; it's a pure left row-scale of A and b → exactly RB-GS-invariant. Dividing the restricted residual by it AMPLIFIES the near-solid rows where D→0.mac_ibm.cuh:18itself says the IBM op is row-based, non-conservative across the wall, "NEVER multigridded; D_rescale is GS-invariant but not MG-invariant."Why un-scaling is unnecessary, not just dangerous: the correction-scheme V-cycle already restricts the TRUE residual
b' − A_f x(level-0 op = the IBMAs_[c]viasetDiffusionFine), so the fixed point is the exact sharp solution regardless of the coarse op —r=0 ⇒ correction=0. The coarse op only sets the RATE. So leave the residual scaled; fix the coarse op and the transfers instead. - rejected: un-scaling the restricted residual by 1/D_rescale to fix the +2-4% Z&H drag bias - why: "Dividing the restricted residual by it AMPLIFIES the near-solid rows where D→0"; D_rescale is "GS-invariant but not MG-invariant"; the correction-scheme V-cycle already uses the true residual so the coarse op only affects convergence rate, not correctness
If a pressure method clearly wins at scale, make it the default¶
- area: flow
- source: parallel-scaling-study.md:11-17
- decided: 2026-08-07
- status: settled
- quote: | The study (user-approved plan 2026-08-07): ... User decisions: BOTH CaNS and OpenFOAM as references ("close to SOTA, much better parallel than OpenFOAM"); 32 GPUs max; keep runs short (budget); if a pressure method clearly wins at scale, MAKE IT THE DEFAULT; workstation study FIRST and its results reported (workstation-vs-Snellius for small systems).
- rejected: none stated
- why: none stated (user directive)
Incremental (rotational) pressure now default ON in C++ DistributedNS, matching the Python binding¶
- area: flow
- source: sdflow-dt-divided-convention.md:40-44
- decided: 2026-06
- status: settled
- quote: |
(1)
incremental_rotational pressure is now default ON in C++DistributedNS(matches the sdflow Python binding, which already defaulted on); the cell-for-cell classical-Chorin ctests + then_pois=0(no-projection) ctests opt out withset_incremental_pressure(false) - rejected: classical (non-rotational) pressure as the C++ default
- why: to match the existing Python binding default
Incremental-rotational pressure default ON in sdflow, OFF in DistributedNS¶
- area: flow
- source: suite-distributed-status.md:325-332
- decided: undated
- status: settled
- quote: | Default OFF in DistributedNS (69 Chorin cell-for-cell tests byte-unchanged), default ON in the sdflow module. Validated: steady velocity unchanged; pressure vs pnm_backend 2.36%→0.42% (5.6× closer); and 28.6% FASTER (smaller correction → fewer pressure-PCG iters).
- rejected: none stated
- why: keeps DistributedNS's existing cell-for-cell tests byte-unchanged while giving sdflow closer parity to pnm_backend and faster convergence
Instability fix: a wall-banded rotational-term blend, not the full rotational update everywhere¶
- area: flow
- source: collocated-attractor-campaign.md:14
- decided: 2026-08-21
- status: settled
- quote: |
Fixed by Frank's wall-banded blend
set_rotational_wall_weight(0.5): P += (ρ/dt + w·μ)φ − (1−w)μ·div(u) at solid-adjacent cells only — stable + 11-digit convergence at dt=60/600/1e20*. - rejected: the unblended full rotational update on the cell-centered approximate projection (unstable, PM-II instability)
- why: "stable + 11-digit convergence at dt=60/600/1e20"
Interstitial vs superficial drag normalization: our K is interstitial (Zick-Homsy); literature (vdH/Tenneti/van Wachem) reports superficial F_D=(1-phi)K, not "friction"¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:45-50
- decided: undated
- status: superseded
- quote: |
Convention (DEFINITIVE — see RECENT PAPERS block below): our
K(=Zick&Homsy) is INTERSTITIAL- referenced; the literature (vdH/Tenneti/van Wachem) reports the SUPERFICIAL total drag F̄_D=(1−φ)·K (NOT friction — earlier "friction" label was wrong; the number (1−φ)K was right). Both →1 dilute so the dilute limit can't tell them apart — the trap. - rejected: the earlier "friction" label for (1-phi)K
- why: the dilute limit (both -> 1) hides the convention difference; the earlier label was simply wrong even though the number was right
Invariant: the ORB must never split the wall-normal axis (y) in wall-bounded flow¶
- area: flow
- source: channel-scaling-rebenchmark.md:85-88
- decided: undated
- status: settled
- quote: |
- The ORB must never split wall-normal y (an internal y-boundary decouples the halves at the centreline — validated bug; the driver hard-aborts). Weak scaling grows GNX only, so the box gets more elongated and ORB carves x/z first; y stayed whole to np≈32 on the production grid. 32 GPUs is therefore the natural ceiling for this case.
- rejected: allowing the ORB to split the wall-normal axis
- why: "an internal y-boundary decouples the halves at the centreline — validated bug"
Keep the staircase velocity-MG despite modest current efficiency win — it is the right operator for future AMR near contact points¶
- area: flow
- source: velocity-mg-design.md:113-116
- decided: undated
- status: settled
- quote: | Why conserved despite a modest current efficiency win (RB-GS converges in O(pore-cells) sweeps; pressure solve dominates the step at high res): it is the natural coarse operator for a future AMR with extreme refinement near contact points, where the velocity solve becomes genuinely stiff/multiscale and an O(1)-V-cycle exact, unconditionally-stable solver pays off. Keep intact.
- rejected: removing the staircase velocity-MG for lack of current efficiency win
- why: "the natural coarse operator for a future AMR with extreme refinement near contact points"
Lattice-plane trap: an SDF exactly 0 at a staggered point is invisible to both the fluid mask and the ghost detector¶
- area: flow
- source: sdf-scene-campaign.md:150
- decided: 2026-09-02
- status: settled
- quote: |
Lattice-plane trap ROOT-CAUSED (2026-09-02, §7 item 12): sdf EXACTLY 0 at a staggered point is fluid
to
ibmSolidMask(strict <0) and not a ghost inibmFillEntry(strict <0) → the wall has no row → inert. Detectormoving_instance_degenerate_points()... First detector version used face apertures — WRONG signal (plane cutting parallel faces leaves apertures 0/1); the gate caught it. - rejected: face-aperture-based detector for the degenerate lattice-plane condition
- why: "plane cutting parallel faces leaves apertures 0/1" — the wrong signal
Level-0 BC hook must use fold=0 (reflection) for the unfolded stencil¶
- area: flow
- source: momentum-solve-residual-stop.md:24
- decided: undated
- status: settled
- quote: | (2) Level-0 BC hook must use fold=0 (reflection) for the unfolded stencil; fold=1 diverges.
- rejected: fold=1
- why: "fold=1 diverges"
M2 chosen as plain Richardson, not BiCGStab¶
- area: flow
- source: defect-correction-campaign.md:31
- decided: 2026-09-01
- status: settled
- quote: | M2 = plain Richardson (not BiCGStab): 16 vs 48 B/cell, no breakdown mode.
- rejected: BiCGStab for M2
- why: "16 vs 48 B/cell, no breakdown mode"
MG telescoping ships off by default, byte-identical when off¶
- area: flow
- source: mg-decomposition-alignment.md:79
- decided: 2026-09-02
- status: settled
- quote: |
2026-09-02 — telescoping IMPLEMENTED and merged to main ...
set_pressure_telescope(True)/PECLET_FLOW_TELESCOPE=1, off by default, byte-identical off. - rejected: none stated
- why: none stated
MG-PCG stalls on ρ- and eps/drag-scaled coefficient operators; Chebyshev is the default driver for those paths¶
- area: flow
- source: multiphysics-framework-plan.md:391
- decided: undated (P5 section)
- status: settled
- quote: | CRITICAL FINDING: MG-PCG STALLS on ρ-scaled coefficient operators (5000 its stuck; openness-tuned transfer pair loses CG's SPD structure) — Chebyshev converges in ~20 and is the varRho DEFAULT driver (bounds re-estimated per rebuild); MG-transfer-symmetry fix = follow-up.
- rejected: MG-PCG as the default driver for variable-density (and, per line 379/382, porous+drag) coefficient operators
- why: "5000 its stuck; openness-tuned transfer pair loses CG's SPD structure"
MG-PCG's relative stopping test never fires on a near-quiescent field; a fixed-iteration cap is the workaround, not a proper fix¶
- area: flow
- source: flow-thermal-convection-validated.md:20
- decided: undated
- status: settled
- quote: |
Two flow pressure-driver pitfalls (logged in peclet-examples ISSUES.md, open in flow):
- MG-PCG's relative stopping test never fires on a near-quiescent field (velocities ~1e-5, e.g.
near RB onset): runs to max_iter at ANY rtol. Workaround: cap it —
set_pressure_pcg(True, 12, 1e-6)acts as a fixed-work MG solve (div ~1e-12). Proper fix would be an absolute/RHS-scaled floor in the criterion.
- MG-PCG's relative stopping test never fires on a near-quiescent field (velocities ~1e-5, e.g.
near RB onset): runs to max_iter at ANY rtol. Workaround: cap it —
- rejected: relying on the relative (rtol) stopping criterion alone near a quiescent field
- why: "runs to max_iter at ANY rtol" — the relative test cannot fire when velocities are ~1e-5
MPI-optional single-source build: default is single-rank with no MPI linked¶
- area: flow
- source: suite-distributed-status.md:332-345
- decided: undated (Phase 2)
- status: settled
- quote: |
cfd builds
sdflowin BOTH modes from one block — default = single-rank, NO MPI linked (ldd: no libmpi),-DCFD_BUILD_MPI=ON= MPI multi-rank +*_mpitests. - rejected: none stated
- why: none stated
Marching-squares (order-2) apertures are now the shipped default in both flow and AMR¶
- area: flow
- source: collocated-attractor-campaign.md:56
- decided: 2026-08-26
- status: settled
- quote: | APERTURE ORDER-2 DEFAULT SHIPPED 2026-08-26 (flow 2b3a6ce, core + umbrella pushed): marching-squares apertures are the DEFAULT in flow (trilinear samples) AND AMR (analytic samples); PECLET_FLOW_APERTURE_ORDER=1 reverts. TWO hardening lessons baked in: CENTER GATE (ungated MS opens masked-DOF staggered faces -> uncorrectable ~1.6e-4 div floor under plain RB-GS...) and floor 1e-3.
- rejected: order-1 (single-sample) apertures as default; an ungated marching-squares aperture (opens masked-DOF staggered faces)
- why: order-2 "recovers 90%/~100% of the exact-aperture bias at R=8/12"; ungated MS causes "uncorrectable ~1.6e-4 div floor under plain RB-GS"
Masking must exclude BOTH cut cells and solid cells, not cut cells alone¶
- area: flow
- source: velocity-mg-design.md:64-75
- decided: 2026-06-16
- status: settled
- quote: |
Why each exclusion matters (measured, SC sphere,
verify_velocity_mg_volfrac_zh_sdflow.py):- residual scaled-but-not-unscaled + plain transfers + const coarse: +bias, NaN at dt≥200.
- volfrac op alone: converges to RB-GS but SLOWER than const, still NaN at dt≥200.
- zero residual at CUT cells only: helps rate, still NaN at dt≥200.
- zero residual + mask prolong at CUT cells: better rate, still NaN at dt≥200 (the solid rows: fine 1+6β live during the cycle vs coarse identity θ<ε → factor-(1+6β) mismatch → overshoot at large β).
- zero residual + mask prolong at CUT and SOLID cells (clean-fluid mask): dt=60 0.000% (beats const 0.06%), dt=200 0.000% and STABLE where const NaNs (2× the stable steady-state dt), converged not under-resolved. dt≥400 (β≥40) still diverges — the practical ceiling.
- rejected: masking cut cells only (without also masking solid cells)
- why: "the solid rows: fine 1+6β live during the cycle vs coarse identity θ<ε → factor-(1+6β) mismatch → overshoot at large β"
Method verdict: WY-split PLIC (B) chosen; CICSAM (A) rejected; CLSVOF (D) dropped¶
- area: flow
- source: vof-campaign.md:89-92
- decided: 2026-08-30
- status: settled
- quote: |
- Method verdict: B builds (WY split PLIC + MYC + SZ/Lehmann-Gekle inversion + HF cascade
- PV paraboloid fallback + balanced-force CSF, momentum-consistent from day one); C = band RDF folded into B for SDF contact angle (ghost-fraction fill, n_wall=∇sdf, θ rotation); A (CICSAM) rejected (MTHINC optional fast mode later); D (CLSVOF) dropped.
- rejected: CICSAM (A); CLSVOF (D)
- why: none stated beyond the verdict itself
Mixed ghost widths require CutcellMG::parityOg to correct red-black colour parity¶
- area: flow
- source: comm-scaling-plan.md:33
- decided: undated
- status: settled
- quote: |
KEY TRAP (cost a bug-catch): mixed ghost widths swap red-black colours — parity uses og+local
index, so a g=2 level's colours flip vs the g=1 reference (3 axes → parity flips). Fix =
CutcellMG::parityOg(og − g + 1 per axis);Level::ogitself stays the true global origin (buildAmg needs it). - rejected: none stated
- why: "a g=2 level's colours flip vs the g=1 reference"
Mode 10 (open-centroid quadrature) is dead — worse on Z&H and diverges on RCP slivers¶
- area: flow
- source: flow-ghost-projection.md:119
- decided: 2026-07-18
- status: settled
- quote: | Mode 10 (open-centroid quadrature + good force) = DEAD: worse on Z&H, DIVERGES on RCP slivers (mode-3a non-telescoping row-sum runaway; telescoping force does NOT cure it) — constraint quadrature is definitively not the lever.
- rejected: mode 10 open-centroid quadrature constraint
- why: "worse on Z&H, DIVERGES on RCP slivers ... telescoping force does NOT cure it"
Mode 4 (fully-FV via defect correction) implemented and is a negative milestone — not 2nd order¶
- area: flow
- source: sdflow-collocated-solver.md:133-150
- decided: 2026-07-05
- status: settled
- quote: | RESULT: stable, converges to an ω-independent fixed point, but NOT 2nd-order — Z&H +0.81/+0.99/ +0.93% at N=32/48/64 (non-convergent) vs mode0 +1.00/+0.68/+0.60. ROOT CAUSE (documented in doc §"Implementation of the fully-FV route"): only the WALL flux got centroid placement; the o_f open-FACE fluxes in L_FV still use the face-CENTRE two-point gradient = the SAME O(h) sub-cell placement error
- rejected: mode 4 as shipped (face-centre flux placement retained on the 6 axis faces)
- why: "only the WALL flux got centroid placement; the o_f open-FACE fluxes... still use the SAME O(h) sub-cell placement error"
Mode-10 quadrature, Seo-Mittal pressure-only split, and ghost-as-production are all dead ends (ghost part superseded 2026-08-25: AUTO collocated default = ghost)¶
- area: flow
- source: collocated-second-order-verdict.md:40-41
- decided: undated
- status: settled
- quote: | Also dead: mode 10 quadrature (breaks D/G adjointness — conservation is the binding constraint), Seo–Mittal pressure-only split, ghost as production.
- rejected: mode 10 quadrature; Seo-Mittal pressure-only split; ghost scheme as the production path
- why: mode 10 quadrature breaks D/G adjointness (conservation is the binding constraint)
- conflict: scope annotation 2026-10-08 — the "ghost as production" verdict rests on the 2026-08-18 A/B that made gauge-exact the collocated default; five days later the collocated-attractor campaign made ghost the AUTO default in both flow and AMR (flow
c672014, core7472306). See "Collocated AUTO default = ghost projection (2026-08-25), superseding 'ghost as production is a dead end'". Mode 10 and the Seo–Mittal split stay dead.
Momentum solve uses the divided (1/dt-scaled) convention, replacing the dt-multiplied form¶
- area: flow
- source: sdflow-dt-divided-convention.md:10-16
- decided: 2026-06 (June 2026)
- status: settled
- quote: |
In cfd-gpu's sdflow (
dns::DistributedNS), the implicit momentum/diffusion equation is scaled by 1/dt ("divided convention"): the operator isidt*I - nu*Lap (+ FOU advection)with diagonalAc = idt + 6*beta,beta = nu,idt = 1/dt; the RHS time term carriesidt[...] This replaced the old dt-multiplied formI - nu*dt*Lap,b = u^n + dt*f. Same converged solution [...] but well-conditioned as dt→∞ / steady state — the change the user asked for (June 2026) - rejected: the dt-multiplied form
I - nu*dt*Lap - why: "at large dt the multiplied form's I and u^n terms become negligible vs the huge dt-scaled terms (float precision loss)"
Momentum tolerance-stop: adaptive tolerance instead of a fixed sweep cap¶
- area: flow
- source: parallel-scaling-study.md:74-77
- decided: 2026-08-08
- status: settled
- quote: |
(1) momentum TOLERANCE STOP
set_velocity_solver_params(iters, rtol, min_iters)— fused max-increment reduce on the black sweep, rank-uniform MPI stop; rtol=0 default = legacy byte-identical; TGV exits at 4 sweeps/component vs 20 (user floated "cap 5" — rejected: silently under-converges stiff regimes, tolerance adapts instead). - rejected: a fixed sweep cap of 5 (user's own suggestion)
- why: a fixed low cap silently under-converges stiff regimes; an adaptive tolerance is used instead
Momentum-advection kernels must use the actual wall velocity field, not maskVelocity's solid zeros¶
- area: flow
- source: advective-cutwall-flux-plan.md:15
- decided: 2026-09-02 (A0)
- status: settled
- quote: |
What A0 was: the momentum-advection kernels (
sadv::, geometry-blind) readmaskVelocity's solid ZEROS; zero is the wall velocity only for a STATIC wall. A0 feeds them scratch views holding the rigid-bodyu_wall(reusesuBc_), explicit + implicit paths both. - rejected: reading maskVelocity's solid-zero convention as the wall velocity for a moving wall
- why: "zero is the wall velocity only for a STATIC wall"
Multiphysics assembly is runtime-dispatched flags on existing kernels, not a Solver template¶
- area: flow
- source: multiphysics-framework-plan.md:403
- decided: undated
- status: settled
- quote: |
Core design verdicts (argued in the doc):
- Runtime-dispatched assembly, NO
Solver<Grid,Props>template: hot loops (RB-GS, CutcellMG) already run on stored coefficient fields; rho/mu enter only assembly kernels (ibmBuildDiffusioncut_cell_ibm.hpp:215, buildRhs, pressure update).varProps_flag selects sibling*Varkernels; existing validated kernels never edited (bit-exactness is structural). Pressure Poisson is already variable-coefficient in openness => variable density =openness/rho_faceon the same CutcellMG rails.
- Runtime-dispatched assembly, NO
- rejected: a templated Solver
design - why: "existing validated kernels never edited (bit-exactness is structural)"
Must re-mask solid velocity after grad(phi) correction¶
- area: flow
- source: cuda-kokkos-migration.md:354-355
- decided: undated
- status: settled
- quote: | (b) MUST re-mask solid velocity after the grad(phi) correction (CUDA apply_mask/mask_k) or decoupled solid velocity accumulates -> blow-up.
- rejected: skipping the re-mask
- why: "decoupled solid velocity accumulates -> blow-up"
Non-incremental Chorin projection gives wrong steady Z&H drag — incremental-rotational required¶
- area: flow
- source: embed-port-progress.md:21-22
- decided: undated
- status: settled
- quote: |
- Z&H drag steady state is dt-independent BUT slow to converge (incremental-rotational pressure accumulation, ~1000-3000 steps; NOT accelerated by larger dt). Non-incremental Chorin gives WRONG steady drag (−40%, splitting-error). Robust protocol: dt=400 + warmstart, run past min_steps floor until |dK| over 200 steps < 5e-5. The warm-detector protocol fires on false plateaus — don't use.
- rejected: non-incremental Chorin projection (−40% error, splitting-error); the warm-detector convergence protocol
- why: Chorin gives "WRONG steady drag (−40%, splitting-error)"; the warm-detector "fires on false plateaus"
Old "inter-node reduction tax" diagnosis for the channel 8-GPU scaling drop is superseded¶
- area: flow
- source: channel-scaling-rebenchmark.md:59-61, 137-139
- decided: 2026-08-10/11
- status: superseded
- quote: |
The published page (2026-08-02) reports 46 M cells/GPU, 1/2/4/8 GPUs =
1369/1300/1713/2942 ms/step = 100/105/80/47 % weak efficiency (33/70/106/124 Mcell/s) and
attributes the 8-GPU drop to "the inter-node pressure-solve reduction tax". That diagnosis is
now believed wrong/stale — see below.
...
COMMUNICATION IS NOT THE BOTTLENECK — the old page's "inter-node reduction tax" is dead.
Pressure allreduce = 0/1/3/9/14/16 ms of a 2.6–2.8 s step (0.3–0.6 %); 2.3–2.4 % with CFR. Step is
89–95 %
projection, and projection tracks the iteration count exactly. - rejected: "the inter-node pressure-solve reduction tax" as the explanation for weak-scaling loss
- why: measured pressure allreduce time is only 0.3-0.6% of the step; the loss tracks iteration count instead
Old grid-convergence table retired; Galerkin=True / set_pressure_pcg not for production¶
- area: flow
- source: suite-distributed-status.md:311-316
- decided: undated (MG Phase 4)
- status: superseded
- quote: |
This re-frames my earlier "Galerkin pressure buggy" saga (the old
grid_convergence_sdflow_vs_pnm.pyk/N² 5.700→5.603→5.586→5.33 table): sdflow side used the buggygalerkin=True; pnm side was MIS-CONFIGURED (time-marched w/ default outer-iters, not pnm's SIMPLE 800-outer steady solve). BOTH halves broken → retired; Z&H single-sphere supersedes it. (NOT Brinkman — that correction still stands, never active.) OPEN bug → backlog item 7: fix or remove sdflow's Galerkin-MG pressure path; until then don't usegalerkin=True/set_pressure_pcgfor production. - rejected: the old grid_convergence_sdflow_vs_pnm.py comparison table/methodology
- why: both sides of that comparison were misconfigured; Galerkin-MG pressure path has an open bug
Old single-GPU CFDSolver reference retired; pnm_backend is pore-network extraction only¶
- area: flow
- source: sdflow-dt-divided-convention.md:29-38
- decided: 2026-06
- status: settled
- quote: |
RETIREMENT (June 2026): the old single-GPU
CFDSolverreference (cfd_solver.cu/.cuh) is deleted (restore tagpnm_backend-reference); sdflow was validated bit-identical to it + Zick–Homsy first. [...] Thepnm_backendPython module is now pore-network extraction ONLY* (SDFReader / extract_pores / segment_volume / extract_topology_gpu, in bindings.cpp + pore_extraction.cu) — it no longer has a CFD solver. - rejected: keeping the CFDSolver single-GPU reference implementation; pnm_backend carrying a CFD solver
- why: "sdflow was validated bit-identical to it + Zick–Homsy first"
Only upwind/dissipative advection schemes exist — no central/energy-conserving option¶
- area: flow
- source: channel-dns-isotropic-grid.md:15
- decided: undated
- status: settled
- quote: |
Only advection schemes are SOU (scheme 0) and Koren-TVD (1) — both upwind/dissipative. NO
central/energy-conserving option (
staggered_advection.hpp). This is THE scientific risk: over-dissipation may relaminarize Re_tau=180 or flatten spectra. Must be measured empirically, not assumed. - rejected: central/energy-conserving advection scheme (does not exist)
- why: "none stated" (a limitation, not a chosen rejection)
Owner-boundary attribution fix: remove shared-cell pressure from both sides of cross-owner faces¶
- area: flow
- source: sdf-scene-campaign.md:82
- decided: 2026-08-31
- status: settled
- quote: |
§7 item 10 (OWNER-BOUNDARY ATTRIBUTION) RESOLVED 2026-08-31 flow
1d95260(v4): per-body reaction attribution carried the pressure flux through the owner partition's mid-surfaces — ZERO in the total (pairwise), zero for one instance, cancelling for symmetric arrays, and worth a FACTOR 2.2 on the drag of a sphere in a closed tank ... Fix: remove the shared-cell π from both sides of every cross-owner fluid-fluid staggered face (+s convention = visited once = MPI-clean); wall faces skipped (their remainder IS the wall pressure force). - rejected: leaving the shared-cell pressure flux attributed through owner mid-surfaces
- why: "worth a FACTOR 2.2 on the drag of a sphere in a closed tank ... and −44% on a Jeffery orbit period"
P1 passed: double-diagonal fallback retired as measurably worse, not merely unnecessary¶
- area: flow
- source: defect-correction-campaign.md:33-38
- decided: 2026-09-01
- status: settled
- quote: | P1 RESULT: RCP bed rtol 1e-8 cap 300: float 24/33/CAPPED vs exact 14/14/28 == full double, div 9.51e-12 vs float 4.51e-06. [...] KEY: exact vs DIAGRESUM agree on iters but separate 65x on divergence -> the double-diagonal converges to the FLOAT-FACE operator, exact to the true one => fallback RETIRED as measurably worse, not merely unnecessary.
- rejected: the double-diagonal fallback (converges to the float-face operator, not the true one)
- why: "the double-diagonal converges to the FLOAT-FACE operator, exact to the true one"
- conflict: vof-campaign.md's "USER PROPOSAL 2026-09-01 — DEFECT CORRECTION" (which this note explicitly supersedes)
PCG selector fixed; Chebyshev stays varRho/porous default (S0/S1 outcome)¶
- area: flow
- source: vof-campaign.md:172-201
- decided: 2026-08-30
- status: settled
- quote: |
Working spelling:
set_pressure_chebyshev(False,…). With PCG genuinely selected the stall reproduces at CONSTANT density on 3-D wall-bounded grids (200/200, div 1.2e-5 at nz≥8; Chebyshev 13-14) and NEVER on periodic+IBM at any ratio — present since the 2026-07-06 build [...] So S3/S4 are PARKED (no measurement shows a coefficient-coarsening failure: periodic handles 1e4 at parity with ρ≡1). [...] Two modes FCG does NOT cure (for WO-H): (a) gravity-driven hydrostatic column [...] (b) small ρ₀/ρ_f next to a prescribed-velocity face at ratio ≥1e2. So Chebyshev stays the varRho default (only driver healthy in all four regimes) and S2's 3× keeps its value. - rejected: PCG as the varRho/porous default driver
- why: "only driver healthy in all four regimes"
Pairing lesson: stability needs (G,D) structural match, accuracy needs closure-value consistency — only the ghost architecture has both¶
- area: flow
- source: collocated-attractor-campaign.md:35
- decided: 2026-08-23
- status: settled
- quote: | Adjoint family O(h) (3 rungs); Design A order 1.3; Design B (star) SPD-but-scheme-unstable => PAIRING LESSON (stability = (G,D) structural match; accuracy = closure-value consistency; ghost architecture = only scheme with both; doc/fluid_only_constraint_plan.md). B+ gates: sym(A_ghost) INDEFINITE (dead); star base λmax == binary surrogate (no quick precond win).
- rejected: Design B (star) — "SPD-but-scheme-unstable"; B+ gates ("dead")
- why: "stability = (G,D) structural match; accuracy = closure-value consistency; ghost architecture = only scheme with both"
Part II phase-change architecture: Robin IHTR, PLIC plane-shift regression, transported-φ CLSVOF stays dead¶
- area: flow
- source: vof-campaign.md:115-125
- decided: 2026-08-30
- status: settled
- quote: | ṁ from pure-cell weighted 5³ one-sided gradients on band-RDF normals; IHTR Robin (not hard T_sat — spurious-wave trap, Bureš-Sato); WY advects with band-extended div-free liquid velocity; interface regression by PLIC plane shift + clip-and-redistribute (NEVER volume-source-in-C — Hardt-Wondra wisp trap); div source shifted into pure GAS cells in the Poisson RHS; ρcpT advected with the same geometric fluxes; conjugate solid heat via cut-cell SDF diffusion. [...] Confirmed: transported-φ CLSVOF stays dead; band RDF sufficient (whole Basilisk/PARIS/NGA phase-change ecosystem is geometric VoF).
- rejected: hard T_sat interface condition; volume-source-in-C interface regression; transported-φ CLSVOF
- why: "spurious-wave trap, Bureš-Sato" (hard T_sat); "Hardt-Wondra wisp trap" (volume-source-in-C)
Part III bubbly-flow container reuses V0–V4 kernels; Dodd–Ferrante FFT not needed¶
- area: flow
- source: vof-campaign.md:107-114
- decided: 2026-08-30
- status: settled
- quote: |
In peclet: blocks = third
container over the SAME V0–V4 kernels; rungs W0–W5 in plan §10 (W2 gate: TBFsolver's own
channel_18case as cross-code validation; W5 = resolved↔unresolved switching with dem/coupling — flagship). TBFsolver's Dodd–Ferrante constant-coeff FFT NOT needed (varRho MG projection is stronger). - rejected: Dodd–Ferrante constant-coefficient FFT pressure solve
- why: "varRho MG projection is stronger"
Per-kernel space.fence() removed — default-exec-space kernels are stream-ordered¶
- area: flow
- source: cuda-kokkos-migration.md:534-539
- decided: 2026-06-20
- status: settled
- quote: |
- FENCE OPTIMISATION (commit 6f6ec32): dropped all 64 redundant per-kernel space.fence() from the cfd Kokkos operator headers (default-exec-space kernels are stream-ordered; only host reads -- deep_copy(host)/ parallel_reduce -- need sync, and those are inherently blocking). BIT-IDENTICAL (full verify suite + RingBed
- OpenMP + 15 MPI tests all unchanged). HUGE perf win: the velocity RB-GS alone fired ~1e4 host-blocking fences/step. => Kokkos sdflow is now FASTER than the hand-tuned CUDA at production res.
- rejected: per-kernel fence() calls
- why: "default-exec-space kernels are stream-ordered; only host reads ... need sync"
Phase B/C explicitly must not touch GP_THETA_MIN, the sliver branch, or the preconditioner scaling¶
- area: flow
- source: ghost-hardening-plan.md:60-64
- decided: 2026-08-17
- status: settled
- quote: | Phase B re-scoped: B0 = compatibility projector (project on the measured left null vector, not the constants) — the only genuine structural gap left; B1 conditioning-driven order reduction (τ≈1e-2 on D_f, counter-gated); B2 store ρ·w not w; B3 optional two-sided sandwich flux closure. DO NOT touch GP_THETA_MIN, the sliver branch, or the preconditioner scaling.
- rejected: touching GP_THETA_MIN, the sliver branch, or the preconditioner scaling in phases B/C
- why: these were already investigated and ruled out/confirmed fine (see H1-refuted and preconditioner findings above)
Plain incompressible continuity with ε only in drag is the WRONG constraint for a porous bed; volume-averaged (Model-A) continuity is required¶
- area: flow
- source: multiphysics-framework-plan.md:379
- decided: 2026-07-05
- status: superseded
- quote: |
VOLUME-AVERAGED (POROUS) CONTINUITY IMPLEMENTED (2026-07-05, ...) — was the plan's explicitly-
DEFERRED "Model-A continuity". User (CFD-DEM expert) correctly caught that the coupling solved
PLAIN incompressible NS (div(openu)=0, open = cut-cell WALL openness from the SDF, NOT the
particle porosity) with eps only in the drag — so reporting div(u)->0 as "converged" enforced the
WRONG constraint (for a bed div(eps u) = -d(eps)/dt != 0). FIX in flow:
set_porous_continuity(True)— divergOpenEps (eps-weighted divergence div(openeps_f*u)) ... - rejected: treating div(open*u)=0 (plain incompressible NS with eps only in drag) as the converged/correct constraint for a porous bed
- why: "for a bed div(eps u) = -d(eps)/dt != 0" — the plain constraint is physically wrong for a porous medium
- conflict: this is itself later refined/superseded by the semi-implicit-drag pressure correction (multiphysics-framework-plan.md:381, below).
Porous pressure correction must use drag-consistent relaxation (SIMPLE/PISO-with-implicit-drag), not a drag-free coefficient¶
- area: flow
- source: multiphysics-framework-plan.md:381
- decided: 2026-07-05
- status: settled
- quote: |
SEMI-IMPLICIT-DRAG PRESSURE CORRECTION (2026-07-05, flow
07ea855, umbrella1e219c8): USER (expert) identified the ROOT of the high-Re porous divergence: the pressure-correction coefficient was open*eps (drag-free), INCONSISTENT with the drag-loaded momentum diagonal A_P=idt+beta -> corrector over-shoots predictor -> diverges where drag is stiff. FIX = the standard SIMPLE/PISO- with-implicit-drag (OpenFOAM rAU=1/A_p, MFIX): drag-relaxation w_f=idt/(idt+beta_f)... - rejected: the drag-free pressure-correction coefficient (open*eps) used against a drag-loaded momentum diagonal
- why: "corrector over-shoots predictor -> diverges where drag is stiff"
- conflict: refines/partially supersedes multiphysics-framework-plan.md:379 (the earlier porous-continuity fix) for the high-Re stiff-drag regime.
Porous projection must run even with no immersed solid — flow now throws, CfdDem auto-installs set_pressure_geometry¶
- area: flow
- source: porous-cfddem-cuda-two-bugs.md:62
- decided: 2026-07-10
- status: settled
- quote: |
THE REAL BUG — FOUND & FIXED (flow
0e19de4, coupling78353b3): the porous projection lives ENTIRELY insideproject(), and step() hasif (cutcellPressure_) project();— a porous domain-BC-only box (no set_solid/set_pressure_geometry — exactly the bidisperse example) ran with NO projection at all [...] FIX: (a) flow step() throws when porous_&&!cutcellPressure_; (b) new bindinghas_cutcell_pressure(); (c) CfdDem auto-installsset_pressure_geometry(all-fluid)when missing. - rejected: silently running porous mode without any pressure-geometry / projection
- why: "no continuity constraint, gas never accelerated to interstitial U/ε in the bed"
Port production physics onto the distributed solver rather than retrofit MPI into production kernels¶
- area: flow
- source: suite-distributed-status.md:101-104
- decided: undated (Step 20)
- status: settled
- quote: |
closes the last capability gap vs production
cfd_solver.cu. Decided (with user): rather than retrofit MPI into the production kernels (every one wraps get_idx — unvalidatable on 1 GPU), port the production physics onto the already-distributed solver, then eventually replace. - rejected: retrofit MPI into the production kernels
- why: every production kernel wraps get_idx, which is unvalidatable on 1 GPU
Pre-fix porous coefficient pair kept for A/B only, never for publishable results¶
- area: flow
- source: porous-eps-conservative-momentum.md:38-39
- decided: undated
- status: settled
- quote: | Gotchas: (1) the pre-fix pair (buildPorousCoeffDrag etc.) kept for A/B only — never publish with it;
- rejected: publishing results computed with the pre-fix (buildPorousCoeffDrag) pair
- why: it is the known-buggy energy-pumping formulation, retained only for A/B comparison
Precision policy rule: identity-bearing quantities stored in the precision the identity is asserted; ship double-diagonal, not fp64 default¶
- area: flow
- source: vof-campaign.md:307-312
- decided: 2026-08-31
- status: settled
- quote: |
PRECISION POLICY (as a rule, not a patch): a quantity an algorithm requires to satisfy an
exact discrete identity must be stored in the precision in which that identity is asserted;
one carrying only an approximation may stay float. Identity-bearing = the operator DIAGONAL
in both operators.
PECLET_FLOW_MG_DIAGRESUM=1ablation recovers full-double behaviour 48³–160³. Recommend shipping double-diagonal (+17 B/cell) — NOT fp64 default (+120 B/cell, +12% time, zero gain on Z&H/permeability/regression). - rejected: full fp64 default operator storage
- why: "+120 B/cell, +12% time, zero gain on Z&H/permeability/regression"
Pressure solve must be PCG (Krylov), not RB-GS, for cut-cell IBM¶
- area: flow
- source: cuda-kokkos-migration.md:352-358
- decided: undated
- status: settled
- quote: | CRITICAL FINDINGS: (a) pressure solve MUST be PCG (Krylov), NOT RB-GS — RB-GS leaves smooth modes unresolved -> unstable at N>=32 (stable only small N); CUDA single-rank auto-runs MG-PCG (last_pressure_iterations=12 not 20). (b) MUST re-mask solid velocity after the grad(phi) correction (CUDA apply_mask/mask_k) or decoupled solid velocity accumulates -> blow-up. (c) use standard incremental (Goda) pressure P+=(rho/dt)phi; the rotational variant -mudiv(u) is UNSTABLE in this cut-cell port (descale amplification at thin cut cells) and unneeded (PCG -> div~1e-12).
- rejected: RB-GS for pressure; rotational pressure-update variant
- why: "RB-GS leaves smooth modes unresolved -> unstable at N>=32"; rotational variant "UNSTABLE in this cut-cell port (descale amplification at thin cut cells) and unneeded"
Public API renames: pressure_potential() → pressure() (returns physical pressure)¶
- area: flow
- source: sdflow-dt-divided-convention.md:36-38
- decided: 2026-06
- status: settled
- quote: |
sdflow renames:
pot_update_k→press_update_k,sub_gradpot_k→sub_gradp_k,pscale_k→press_from_phi_k, and the public accessorpressure_potential()→pressure()(returns the physical pressure p).phi()/phi_(projection potential) unchanged. - rejected: the name pressure_potential() for the physical-pressure accessor
- why: none stated beyond disambiguating potential vs physical pressure
Random-array drag deviation is a genuine method difference, not finite-size or arrangement effects¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:57-64
- decided: undated
- status: settled
- quote: |
DEVIATION INVESTIGATED (commit
8ee8bab): the ~+8–13% (vs vdH) / +16% (vs Tenneti) random-array drag deviation is a systematic METHOD difference, not finite-size or arrangement: (1) ... mean F_d FLAT across N_p=16→256 ... → NOT finite-size; (2) ... MD-equilibrated config (g(σ)=1.78 = Carnahan–Starling 1.758, a proper equilibrium ensemble) still F_d=4.54, +8% vs vdH → NOT arrangement. - rejected: finite-size effects and arrangement/crystallization as explanations for the deviation
- why: mean drag is flat across N_p and unchanged under a proper equilibrium (MD) ensemble
Reaction torque leak fix is a closed-form cut-cell moment; adding the pressure moment term makes it worse¶
- area: flow
- source: sdf-scene-campaign.md:70
- decided: 2026-08-31
- status: settled
- quote: |
Fix: mur×(n dA×Omega) over cut cells with the
EXACT apertures — closed form, no reconstruction, zero when nothing rotates, force untouched. Gates
(
rotation_gate.py): static −31% → +3.5/+2.4/+2.2% CONVERGING* ... TRAP: a failed repair is recorded — adding the pressure moment Σ r×grad(pi) makes it WORSE (−84%); do not re-try. - rejected: adding the pressure moment Σ r×grad(pi) to close the torque gap
- why: "makes it WORSE (−84%); do not re-try"
Repair attempts (modes 1–3b) inside the unidirectional IBM all fail to reach 2nd order¶
- area: flow
- source: sdflow-collocated-solver.md:79-108
- decided: 2026-07-04
- status: settled
- quote: | STRUCTURAL CONCLUSION (the real lesson): the unidirectional IBM momentum row is FINITE-DIFFERENCE (pressure force wants the point ∇p at the cell center) while the cut-cell constraint is FINITE-VOLUME (adjointness wants an o-weighted scatter); at cut cells these conflict and NO axis-only (T, force) pair satisfies both. Mode 0's wrong-geometry pair is self-consistent + error-compensating → best of the axis-only family. 2nd-order collocated requires momentum + constraint to share ONE finite-volume cut-cell geometry = the Basilisk embed route — now justified by an exhaustive ablation, not conjecture.
- rejected: mode 1 (wall-anchored weighted-LSQ), mode 2 (transpose pairing), mode 3a (open-centroid, unstable), mode 3b (o-weighted adjoint, stable but worse constant)
- why: "at cut cells these conflict and NO axis-only (T, force) pair satisfies both"
Report K normalized by superficial velocity (Zick-Homsy convention), not interstitial¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:110-112
- decided: undated
- status: settled
- quote: | Normalization is the main pitfall — report K normalized by SUPERFICIAL velocity (=Z&H, →1 dilute); vdH/Beetstra F is INTERSTITIAL so K_vdH=F/(1−φ).
- rejected: reporting an interstitial-normalized K without conversion
- why: none stated beyond avoiding the normalization pitfall
Resolved: the cell-average velocity scheme was a porting bug, not a real discrepancy — point-value is correct¶
- area: flow
- source: migration-faithful-port.md:31
- decided: 2026-06-19
- status: settled
- quote: |
RESOLVED (2026-06-19): the task-1 ~1% discrepancy was the cell-average velocity scheme (
buildIbmOverlay<1>) vs CUDA's POINT-VALUE (ibm_geometry_ext_k<0>, hardcoded in distributed_ns.cuh:999). Thepoly_*matched; the deviation was purely the SCHEME selector. Switching to<0>+ restoring the rotational pressure update -> Kokkos matches CUDA to ~1e-13 - rejected: the cell-average velocity scheme (buildIbmOverlay<1>) as used in the Kokkos port
- why: "the deviation was purely the SCHEME selector"; switching to point-value matches CUDA to ~1e-13
Richardson extrapolation is ill-conditioned for this dense-bed data; do not rely on a free-parameter fit¶
- area: flow
- source: ringbed-cfd-surrogate.md:31
- decided: 2026-06-15
- status: settled
- quote: | ... order UNCHANGED (per-grid drop ~0.68×, ~+19% at 12.7 cells/wall, k_inf~1.12e-3 geom; Richardson ILL-CONDITIONED — free-fit→nonsense 4e-4).
- rejected: relying on an unconstrained Richardson-extrapolation fit for k_inf on this data
- why: "free-fit→nonsense 4e-4"
Ring convergence failure was a solver-tolerance artifact, not a cut-cell/geometry limitation¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:14-30
- decided: undated
- status: superseded
- quote: |
The apparent "IBM breaks for rings" was a SOLVER-TOLERANCE ARTIFACT, not the cut-cell.
The convergence study stopped the steady-state TIME-MARCH on a loose
Δ⟨u⟩<3e-4·⟨u⟩criterion (inibm_shape_diagnostic.solve_kANDrun_ring_cfd_sdflow.stokes_permeability), halting before steady state by a shortfall that GROWS with resolution ... Fix = tight tol 1e-6: Z&H −0.03% at N=192 (clean 2nd order), single sphere flat to 0.5% (was spurious 23% drop); ALL single shapes then converge order 1.3–1.85 ... Ring report "first-order +14%" was substantially this artifact — tight-tol ring bed is order ≈1.5. - rejected: the earlier conclusion that ring geometry/thin walls caused first-order (+14%) convergence
- why: the loose 3e-4 steady-state stopping criterion halted the march before steady state, a shortfall that grows with resolution and mimics degraded convergence order
Ring-bed k convergence slowness is intrinsic dense-packing near-contact Stokes stiffness, not a thin-wall/cut-cell IBM defect¶
- area: flow
- source: ringbed-cfd-surrogate.md:33
- decided: 2026-06-15
- status: superseded
- quote: | (1) k convergence is SLOW (sub-2nd-order) — but it's NEAR-CONTACTS, not the wall or the cut-cell (CORRECTED 2026-06-15 via [[ibm-accuracy-sphere-validation]] IBM diagnostic). ... A SINGLE hollow cylinder converges FINE (order 1.70, 0.2% at 11.5 cells/wall, volume exact ~1e-4) ⇒ ring-bed slowness = dense packing near-contact fluid gaps (intrinsic Stokes stiffness, cf RCP φ0.62), NOT thin wall/IBM defect.
- rejected: the initial hypothesis that slow k convergence was caused by the thin wall or the cut-cell IBM discretization
- why: "A SINGLE hollow cylinder converges FINE ... ⇒ ring-bed slowness = dense packing near-contact fluid gaps"
- conflict: this note's own earlier framing (implicit prior suspicion of wall/IBM defect) is what's being corrected here.
Root cause of the RB-GS stall: update-criterion (max|du| <= rtol*first-sweep-update) chases noise on a warm-started step¶
- area: flow
- source: momentum-solve-residual-stop.md:8
- decided: undated
- status: settled
- quote: |
On the FoxBerry packed bed (domain BCs + IBM, nu dt/dx^2 ~ 4e4) the momentum RB-GS hit its
600-sweep cap every step = 63 % of the step at 384 ranks. Cause: the stop is max|du| <= rtol *
FIRST sweep's update, and a warm-started near-steady step's first update is already noise.
Fix (flow telescope branch d6b3eb5 + 3987e08):
set_velocity_residual_tolerance(rtol)(max|b-Au| <= rtol*max(|b|,|Au|); the inflow forcing enters via the ghost, held face excluded): 96^3: 468 -> 24 sweeps/step. - rejected: the update-criterion stop (max|du| <= rtol * first sweep's update)
- why: "a warm-started near-steady step's first update is already noise"
Rung 3 fix: openness-weighted centered gradient replaces plain ½(g⁻+g⁺) projection correction for embed mode¶
- area: flow
- source: embed-port-progress.md:24-25
- decided: undated
- status: settled
- quote: |
- mode 6 pre-Rung-3 (embed + plain projection): converges from below but asymptotically O(h) (−2.27→−1.09→−0.48→−0.37→−0.27 at N=32..128) — plain
projectCorrectCenteruses ½(g⁻+g⁺) with closed faces zeroed = O(h) under-correction at cut cells (analysis defect b). - Rung 3 fix: Basilisk
centered_gradient= openness-WEIGHTED cell gradient(o⁻g⁻+o⁺g⁺)/(o⁻+o⁺)— at a cut cell with one closed face it uses the OPEN face gradient at FULL weight (not ½). AddedcenterGradOpen/projectCorrectCenterOpen; wired into mode 6 for BOTH the incremental −grad(P^n) predictor AND the correction (adjoint pair).
- mode 6 pre-Rung-3 (embed + plain projection): converges from below but asymptotically O(h) (−2.27→−1.09→−0.48→−0.37→−0.27 at N=32..128) — plain
- rejected: plain projectCorrectCenter's ½(g⁻+g⁺) correction (with closed faces zeroed) for the embed-momentum mode
- why: plain ½/½ correction is "O(h) under-correction at cut cells" (defect b); the open-face-only weighting matches Basilisk's centered_gradient
S-ladder plan; Dodd–Ferrante splitting rejected for the pressure driver¶
- area: flow
- source: vof-campaign.md:473-479
- decided: 2026-08-30
- status: settled
- quote: | Key risk owned: PCG stalls on ρ-scaled coefficients → S-ladder in plan §5 (2026-08-30, user's main concern): S0 measure on static manufactured ρ fields (no VoF needed) → S1 flexible CG → S2 Chebyshev-bound amortization at capillary dt → S3 series-harmonic resistor-network coarsening → S4 Galerkin/BoxMG symmetric transfers. [...] Dodd–Ferrante splitting rejected (error ~ σκ = the dominant field at pore scale).
- rejected: Dodd–Ferrante splitting for the pressure driver
- why: "error ~ σκ = the dominant field at pore scale"
S3 coarsening indefiniteness is real, but coarsenOpenAvg must NOT switch to harmonic¶
- area: flow
- source: vof-campaign.md:253-260
- decided: 2026-08-31
- status: settled
- quote: |
Cause:
coarsenOpenAvg's arithmetic face-coefficient average (across a 1000:1 jump the coarse coefficient is ~0.5 where ~2e-3 is right). TRAP: do NOT just switchcoarsenOpenAvgto harmonic — that same field is the geometric openness on the periodic/IBM path. This is the honest reason Chebyshev stays the varRho/porous default. - rejected: switching coarsenOpenAvg's face-coefficient averaging to harmonic
- why: "that same field is the geometric openness on the periodic/IBM path"
SdflowIbm MPI requires constructing each rank with local ORB block dims (ctor refactor, not a gated add-on)¶
- area: flow
- source: cuda-kokkos-migration.md:558-570
- decided: undated
- status: settled
- quote: | SdflowIbm MPI = ARCHITECTURAL REFACTOR (not a clean gated add-on, unlike CutcellMG/VelocityMG): SdflowIbm is CONSTRUCTED for the grid it operates on -- e_=nx+2G block, the IBM overlay/openness/masks, and the g2<->g1 pressure bridge are all sized + built at construction/setSolid from (nx,ny,nz). MPI needs decompose-THEN- allocate-local (each rank's block dims come FROM the GridHalo decomposition, chicken-and-egg with the ctor). ... SdflowIbm MPI DONE (commit 47e52c3) = CFD MPI PORT COMPLETE. The refactor was clean after all: each rank CONSTRUCTS SdflowIbm with its LOCAL ORB block dims; initMpi(gnx,gny,gnz,comm) builds the g=2 velocity-block halo + records og_=block inner origin (global RB parity) + sets distributed_.
- rejected: none stated (this is the resolved architecture, not a rejected alternative)
- why: "CUDA DistributedNS was MPI-first (ctor takes global, decomposes internally) -- the Kokkos SdflowIbm was single-GPU-first, hence the mismatch."
Sequencing: "VoF vs multiphysics first" dissolves — VoF is the next multiphysics phase¶
- area: flow
- source: vof-campaign.md:480-484
- decided: 2026-08-30
- status: settled
- quote: | Sequencing answered 2026-08-30: multiphysics phases 1–8 are DONE (host-openmp); "VoF vs multiphysics first" dissolves — VoF IS the next multiphysics phase. [...] NOT on the path: VelocityMG varProps, porous ρε inertia, AMR unification.
- rejected: none stated (a false dichotomy dissolved)
- why: none stated
Sign convention: closed divergence of the corrected field equals Aφ − b = −residual¶
- area: flow
- source: flow-ghost-projection.md:43
- decided: undated
- status: settled
- quote: | Identity: closed div of the corrected field = Aφ − b = −residual (sign!).
- rejected: none stated
- why: none stated (sign convention)
Sliver mask must pin only cells with cs<1e-6, not every cell with sdf(center)<0¶
- area: flow
- source: embed-port-progress.md:27
- decided: undated
- status: settled
- quote: |
SLIVER-MASK breakthrough (Rung 4 core):
ibmSolidMask+maskVelocityzero every cell with sdf(center)<0 — INCLUDING solid-centred cut cells (cs>0). Correct for mode 0's IBM (no-slip on fluid rows) but WRONG for embed: a partial-fluid cut cell must hold its reconstructed near-wall velocity (e.g. the below-wall extrapolation U9=-0.152). Masking it to 0 leaves the adjacent fluid cell under-constrained → a UNIFORM +0.15 velocity shift on the flat channel (+7% at N=32). Fix (flow_ibm.hpp setSolid, gated faceInterp_>=5): re-mask from cs — pin ONLY cs<1e-6. Flat-wall N=32 went +7% -> ~0 (profile matches analytic to 0.04%). BUT it flips Z&H mode 6 from under-drag (converging) to over-drag increasing (-0.05,+0.29,+0.42 at N=32/48/64) — the live cut cells expose that the plain 1/2-1/2 constraint over-counts curved-wall flux. - rejected: the mode-0-style mask (zero every cell with sdf(center)<0, including solid-centred cut cells) for embed mode
- why: masking solid-centred cut cells to 0 leaves the adjacent fluid cell under-constrained, producing a uniform +7% velocity shift on a flat channel at N=32
SolverColocated is the ABC approximate projection, NOT Rhie–Chow — user correction¶
- area: flow
- source: vof-campaign.md:485-489
- decided: 2026-08-30
- status: settled
- quote: | USER CORRECTION 2026-08-30: SolverColocated = ABC approximate projection (they worked hard on it — [[collocated-attractor-campaign]]), NOT Rhie–Chow; never say R-C for it.
- rejected: describing SolverColocated's coupling as Rhie–Chow
- why: none stated beyond the correction itself (user built it as ABC)
Staggered converges onto Zick & Homsy; collocated is bias-dominated (irreducible ~0.9%)¶
- area: flow
- source: peclet-examples-gallery.md:165-171
- decided: 2026-07-04
- status: settled
- quote: | HEADLINE RESULT (frozen, N=16/24/32/40, ~5min CPU render): staggered converges monotonically ONTO Z&H (−1.74%→−0.18%, fit K∞+C·N^−p gives order p≈1.92, K∞=4.299, +0.16% bias); collocated is BIAS-DOMINATED — sits ~+0.9% high (+0.78/+1.06/+0.99/+0.73%), non-monotone, NO clean power law, K∞≈4.332 (+0.93%), irreducible by refinement — confirms [[sdflow-collocated-solver]]'s intrinsic velocity-placement gap.
- rejected: none stated (measurement result)
- why: confirms an intrinsic velocity-placement gap in the collocated scheme
- conflict: collocated-second-order-verdict.md (later campaign shows the plateau is NOT simply "bias-dominated collocated vs exact staggered" — it is a real non-vanishing wall-sourced plateau present with more nuance and multiple exonerated causes)
Staircase coarse operator (binary classification, no volume-fraction coefficients) is the DEFAULT for the IBM volfrac path, removing the dt ceiling¶
- area: flow
- source: velocity-mg-design.md:77-98
- decided: 2026-06-17
- status: settled
- quote: |
## STAIRCASE coarse operator (2026-06-17, Frank's idea) — DEFAULT; removes the dt restriction
Frank's variant, now the DEFAULT for the IBM volfrac path (
set_velocity_mg_volfrac(on, eps, res_mask=True, staircase=True)): keep the fine-level IBM residual exclude (clean-fluidres_mask) AND on the coarse levels use the volume fraction ONLY to CLASSIFY cells — θ≥0.5 fluid, θ<0.5 solid — then build a plain constant-coefficient Helmholtz (per-axis β,mg_build_velocity_op_staircase_k) at fluid cells with the solid cells pinned to 0 (a first-order staircase no-slip). The volume/area fractions are NOT used as coefficients (that was the area-fraction op; weak partial-cell coupling limited it). ... Result (SC sphere): exact (== RB-GS, 0.000%) and stable to dt=6400 (β=640) at φ=0.216 AND φ=0.5236 — vs area-fraction dt≤400, const-coarse dt≤200. Essentially no practical dt restriction; V-cycle ρ≈0.23. The const-coarse failure (coupling fluid pockets through thin walls) is fixed by the binary classification disconnecting pockets across resolved walls. - rejected: the volume-fraction-weighted (area-fraction) coarse operator as coefficients; the plain const-coeff coarse op without staircase classification
- why: "weak partial-cell coupling limited" the area-fraction op; binary classification "disconnect[s] pockets across resolved walls" whereas const-coarse coupled fluid pockets through thin walls
Staircase is consolidated as the ONLY IBM velocity-MG coarse op — const and area-fraction paths removed from the code¶
- area: flow
- source: velocity-mg-design.md:100-116
- decided: 2026-06-17
- status: settled
- quote: |
## CONSOLIDATED (2026-06-17) — staircase is the ONLY IBM coarse op; const + area-fraction REMOVED
Cleanup commit: the staircase is now the IBM velocity-MG coarse operator, reached directly via
set_velocity_multigrid(on, levels, vcycles)+set_ibm_solid(theset_velocity_mg_volfractoggle and thevmg_volfrac_/vmg_staircase_/vmg_eps_flags are GONE). Removed:setDiffusionCoarse+mg_const_diffusion_op_k(geometry-blind const-coarse for IBM),setVelocityVolfracCoarse+mg_build_velocity_op_areafrac_k+ibm_areafrac_k+ the fine area-fraction fields,mg_restrict_volwt_k+ thevel_mask_xfer_experiment. KEPT:setVelocityStaircaseCoarse,mg_build_velocity_op_staircase_k,mg_threshold_mask_k,MGLevel::pin+ the smoother/residualpinarg, the clean-fluid exclude (ibm_clean_fluid_mask_k/mg_mul_mask_k/mg_prolong_masked_k),vfine_/vresmask_/vtheta_lvl_/vpin_lvl_. Domain-BC const-coeff (cavity/BFS,setDiffusionConstAllLevels+mg_const_diffusion_op_aniso_k) and the upwind-convective (build_vmg_adv_stencil) paths are SEPARATE and untouched. - rejected: geometry-blind const-coarse (setDiffusionCoarse) and area-fraction (setVelocityVolfracCoarse) IBM coarse operators — both deleted from the code
- why: staircase superseded both in accuracy and dt-stability (see prior entries)
Stale-ghost pressure V-cycle bug invalidated the entire published parallel-scaling page's peclet numbers¶
- area: flow
- source: parallel-scaling-study.md:206-213
- decided: 2026-08-09
- status: superseded
- quote: | ⚠ ALL PECLET NUMBERS ON THE PUBLISHED PAGE ARE NOW STALE (2026-08-09, flow 5d77deb). The pressure V-cycle restricted a residual computed with one-colour-stale ghosts; fixing it made pressure iteration counts decomposition-independent AND ~2× lower.
- rejected: the pre-fix pressure iteration counts and all derived peclet-vs-reference ratios on the published page
- why: the V-cycle was restricting a residual computed with stale (one-colour-old) ghost values
Star half fix: phibar mean was not bitwise-annihilating even in double — replaced by flux form¶
- area: flow
- source: defect-correction-campaign.md:52-54
- decided: 2026-09-02
- status: settled
- quote: |
Star half LANDED (flow 7fb80e0):
phibar = Sum(a*x)/Sum(a)was NOT bitwise-annihilating even in double — the two sums round independently — so the star delta leaked into A*1 regardless of aperture precision, INDEPENDENT of P1; fixed by the flux form(a_k/D)*Sum_j a_j (x_k - x_j). - rejected: the phibar=Sum(a*x)/Sum(a) formulation
- why: "the two sums round independently — so the star delta leaked into A*1"
Strict staggered bit-identical guard must be held through every change; AMR must work for both velocity placements or be explicitly scoped¶
- area: flow
- source: sdflow-octree-amr-next.md:19-20
- decided: undated
- status: settled
- quote: |
Two velocity placements via the
GridLayoutpolicy (src/grid_layout.hpp): staggered MAC vs collocated — see [[sdflow-collocated-solver]]. Any AMR must work for both (or be scoped to one and keep the other bit-identical — the project has held a strict staggered bit-identical guard through every change). - rejected: none stated
- why: the project has held a strict staggered bit-identical guard through every change; a design that can't work for both must not break it
Success criterion for any A·1=0 repair: match the full-double floor, never demand rtol 1e-8¶
- area: flow
- source: vof-campaign.md:413-427
- decided: 2026-08-31
- status: superseded
- quote: | Consequences: (1) the honest success criterion for double-diagonal/any A·1=0 repair is "floor ≤ the full-double floor at MATCHED config", never "reaches rtol 1e-8" — evaluating against 1e-8 would REJECT A CORRECT FIX (I had written exactly that wrong criterion into WO-M and corrected it mid-flight).
- rejected: judging a precision fix by whether it reaches rtol 1e-8
- why: "a full-double hierarchy still floors at r/r0 ≈ 3–4e-8... the classical CG attainable-accuracy limit"
Superseded same-day decision: residual stop default was fixed at 1e-5¶
- area: flow
- source: momentum-solve-residual-stop.md:32
- decided: 2026-09-02
- status: superseded
- quote: |
Decided 2026-09-02 (user): residual stop is the DEFAULT (1e-5; regression suite passes,
metrics +0.00 %, steady-state runs need 5-20 % more steps); the const-coeff domain-BC smoother
got its residual (
diffResidual); AUTO velocity MG under MPI below PECLET_FLOW_VMG_AUTO_CELLS=65536 cells/rank and np>1 (flow telescope branch c600d79/500a9d8). - rejected: none stated (this decision was itself superseded later the same day by the rtol-following default)
- why: "regression suite passes, metrics +0.00 %, steady-state runs need 5-20 % more steps"
TRAP: -DPECLET_FLOW_MREAL_DOUBLE=ON on the cmake command line silently builds float¶
- area: flow
- source: defect-correction-campaign.md:45-46
- decided: 2026-09-01
- status: settled
- quote: |
TRAP:
-DPECLET_FLOW_MREAL_DOUBLE=ONon the cmake line SILENTLY BUILDS FLOAT (symbol is in no CMakeLists/preset -> uninitialized cache var). Correct form:-DCMAKE_CXX_FLAGS=-DPECLET_FLOW_MREAL_DOUBLE. - rejected: passing -DPECLET_FLOW_MREAL_DOUBLE=ON as a normal cmake cache variable
- why: "symbol is in no CMakeLists/preset -> uninitialized cache var"
Telescoping is default on for the pressure MG since 2026-09-02; the velocity solve does not need it¶
- area: flow
- source: momentum-solve-residual-stop.md:18
- decided: 2026-09-02
- status: settled
- quote: | Why: the user asked to "improve the velocity solve" and whether it needs telescoping. It does NOT: 2/3/5 levels identical (pore-confined problem). Telescoping is DEFAULT ON for the pressure MG since 2026-09-02.
- rejected: telescoping the velocity solve
- why: "2/3/5 levels identical (pore-confined problem)"
Ten-Cate periodic-image bug: periodic images are a union, not independent slabs¶
- area: flow
- source: sdf-scene-campaign.md:131
- decided: 2026-09-02
- status: settled
- quote: | 2026-09-02 — ten-cate CLOSED, §7 item 11 (periodic images are a UNION): the whole "creeping-valued confined drag / sharpest open defect" story was a CSG slab wider than the periodic box refilling its own cavity (tank 30% narrow). Corrected: E1 −2.6%, E4 +1.8% at d/h=8.
- rejected: the earlier CSG-slab-per-image geometry construction (implicitly non-union)
- why: "a CSG slab wider than the periodic box refilling its own cavity (tank 30% narrow)"
The 47%-at-8-GPU "reduction tax" diagnosis is stale, superseded by 2026-08 solver fixes¶
- area: flow
- source: snellius-parallel-benchmark-campaign.md:25-27
- decided: 2026-08-10
- status: superseded
- quote: | SUPERSEDED IN PART (2026-08-10): the measured baseline below predates the 2026-08 solver fixes (esp. the stale-ghost MG residual, which made V-cycle convergence decomposition-dependent) — the 47%@8-GPU "reduction tax" diagnosis is stale. Re-benchmark planned: see [[channel-scaling-rebenchmark]].
- rejected: the "inter-node pressure-solve reduction tax" explanation for the 47% weak efficiency at 8 GPU
- why: the baseline predates the stale-ghost MG residual fix, which made V-cycle convergence decomposition-dependent
The agglomerated-bottom MG anomaly required a per-fluid-component null-space projector, a double row-sum, and a looser inner tolerance¶
- area: flow
- source: agglomerated-bottom-ibm-fix.md:15
- decided: 2026-08-13
- status: settled
- quote: |
Root causes, in order of severity (all measured, not guessed):
pcgAmg::meanZeroprojected the ALL-cell mean; the true null space is one constant per connected FLUID component (solid cells are identity rows)... Fix: [per-component projector].- MG level coefficients are float (
MReal) → assembled double CSR had fluid row sums ~5e-8 relative instead of 0 → near-null vector off the assumed constant... Fix: on the singular path (removeMean_) resum each fluid diagonal in double = −Σ off-diags (exact by construction...). - Inner tol 1e-10 is at/below the double floor of the projected solve; parity already held at 1e-5. Now 1e-8, cap 100 (~7 median iters).
- rejected: projecting the all-cell mean (rather than per-connected-fluid-component); leaving MG coefficients in single-precision row sums uncorrected; an inner tolerance of 1e-10
- why: all three measured directly (per-iteration trace signature, row-sum defect measurement, floor comparison)
The defect-correction rule: Krylov matvec/residual must be the exact double operator in flux form; preconditioners below may stay float¶
- area: flow
- source: defect-correction-campaign.md:14-17
- decided: 2026-09-01
- status: settled
- quote: |
The rule: Krylov matvec + residual = exact double operator in FLUX form (A·1=0 bitwise); V-cycles/
smoothers/AMG below are preconditioners and may stay float. User's proposal; the architect's evaluation found
no outer loop is needed — PCG already separates
matvec/precondlambdas, onlymatvecOverlap(mac_cutcell_mg.hpp:1715) reads the float bands - rejected: none stated (this is the adopted rule)
- why: "PCG already separates matvec/precond lambdas"
The earlier "staircase not suited for packed materials" caveat is retracted¶
- area: flow
- source: velocity-mg-design.md:90-98
- decided: undated
- status: superseded
- quote: |
PACKED-BED VALIDATED (
scripts/verify_velocity_mg_staircase_packing_sdflow.py, random periodic sphere packings): on a moderate bed (21 sph, φ=0.245, N=64) staircase is exact (0.000%) and stable at dt=800 where const (0.19% biased→NaN) and area-fraction (NaN) both fail; on a DENSE thin-neck bed (53 sph, φ=0.29, ~1-cell necks, N=128) staircase is exact (0.000–0.010% vs RB-GS) and stable to dt=3200 (β=320) with 16 V-cycles (the ceiling rises with V-cycle count; thin necks lower it vs the SC sphere's dt=6400). Exact across coarsening levels 2–6 (the answer is correct regardless of depth — fine smoother + exclude mask own the boundary; deep coarsening only affects rate, so thelevelscap is a tuning knob, not a correctness requirement). ⇒ the earlier "not for packed materials" caveat is RETRACTED: the staircase velocity-MG handles packed beds, exact and stable to very large dt. - rejected: the earlier belief that staircase velocity-MG doesn't suit packed materials
- why: measured exact + stable results on moderate and dense thin-neck packed beds
The rotational (Timmermans) pressure update must be restored, not the non-rotational Goda form substituted¶
- area: flow
- source: migration-faithful-port.md:61
- decided: undated
- status: settled
- quote: |
Concrete violation to undo (task 1, cut-cell pressure, commit 290f8a4): I changed the incremental
pressure update from CUDA's ROTATIONAL form (
P += (rho/dt)*phi - mu*div(u*), Timmermans) to the non-rotational Goda form (P += (rho/dt)*phi), and substituted a diagonal-preconditioned CG for CUDA's geometric MULTIGRID / MG-PCG pressure solve, and stored the pressure operator in double instead of CUDA's floatmreal. Result: k_Kokkos=5.798 vs k_CUDA=5.836 (~0.65%, stable under heavy convergence -> a genuine method difference). MUST re-port faithfully: CUDA's mac_multigrid.cuh MGPoisson ... + the rotational pressure update. - rejected: non-rotational Goda pressure update form; diagonal-preconditioned CG in place of geometric MG/MG-PCG; double-precision pressure operator storage
- why: "Result: k_Kokkos=5.798 vs k_CUDA=5.836 (~0.65%, stable under heavy convergence -> a genuine method difference)"
The standalone V-cycle pressure driver does not honor set_pressure_solver_params(n) and is ~30x slower at small grids¶
- area: flow
- source: flow-thermal-convection-validated.md:25
- decided: undated
- status: settled
- quote: |
- The standalone V-cycle driver (no PCG/Chebyshev) is ~30× slower than capped PCG at small grids
and
set_pressure_solver_params(n)doesn't change its work — n_pois appears not honoured on that path.
- The standalone V-cycle driver (no PCG/Chebyshev) is ~30× slower than capped PCG at small grids
and
- rejected: none (this documents a bug/limitation, not a chosen alternative)
- why: none stated beyond the observed behavior (n_pois not honoured on that driver path)
The ten-cate confined-flow deficit was a geometry (oversized periodic slab) bug, not a solver/advection defect¶
- area: flow
- source: advective-cutwall-flux-plan.md:49
- decided: 2026-09-02
- status: superseded
- quote: |
CLOSED 2026-09-02 — the deficit was GEOMETRY: the ten-cate tank slab (
NX*0.7) was wider than the periodic box; the scene query takes the MIN OVER 27 IMAGES (union), so the slab's images refilled the cavity: 38-cell duct instead of 53 (d/W 0.21 vs 0.15 → Faxén K≈1.67 = exactly the measured effective K). Corrected tank at d/h=8: E1 0.922 vs 0.947, E4 0.972 vs 0.955, physical falls. - rejected: the hypothesis that the deficit was caused by the advective cut-wall flux / aperture weighting (A1), or by precision/momentum-convergence issues
- why: "the slab's images refilled the cavity ... d/W 0.21 vs 0.15 → Faxén K≈1.67 = exactly the measured effective K"
- conflict: supersedes the earlier framing in this same note ("ten-cate did NOT recover", advective-cutwall-flux-plan.md:26) that treated it as an open solver-physics deficit.
The ~0.3% collocated accuracy "plateau" is not a truncation ceiling — it is an instability + an invisible pressure-subspace attractor family¶
- area: flow
- source: collocated-attractor-campaign.md:11
- decided: 2026-08-21
- status: superseded
- quote: | State 2026-08-21 (supersedes [[collocated-second-order-verdict]] — the ~0.3% "plateau" is NOT a truncation ceiling): two-layer mechanism found and measured. (1) Guy–Fogelson PM-II instability of the rotational update on the cell-centered approximate projection ... (2) ROOT CAUSE of the residual bias: invisible pressure subspace — gpCenterGrad never reads solid-centered P but the aperture operator couples those DOFs (α>0 faces), so u-stationarity only needs the P-increment ∈ ker(G) ⇒ an affine FAMILY of steady states...
- rejected: the earlier "collocated-second-order-verdict" conclusion that ~0.3% was a truncation-order ceiling
- why: measured two-layer mechanism (PM-II instability + invisible pressure subspace / attractor family) fully explains the residual bias
- conflict: collocated-second-order-verdict.md (explicitly named as superseded; not in this slice)
UCX_RNDV_THRESH tuning is falsified as an explanation for the np8 anomaly — leave UCX defaults¶
- area: flow
- source: comm-scaling-plan.md:57
- decided: 2026-08-18
- status: settled
- quote: | UCX_RNDV_THRESH=256k = 2x WORSE (36.1) -> threshold hypothesis FALSIFIED, leave UCX defaults.
- rejected: tuning UCX_RNDV_THRESH=256k
- why: "2x WORSE (36.1)"
User decision: port Basilisk embed.h, not Trebotich–Graves, for 2nd-order collocated walls¶
- area: flow
- source: sdflow-collocated-solver.md:186-191
- decided: 2026-07-05
- status: settled
- quote: | USER DECISION 2026-07-05: copy Basilisk (not Trebotich–Graves — same core reconstruction, far more extractable, its flux-redistribution rigor targets an explicit small-cell-CFL problem we don't have in the Stokes regime; T–G kept only as a pore-scale-validation reference + a later add-on if we need strict discrete conservation through pores).
- rejected: Trebotich–Graves/EBChombo as the primary port target
- why: "far more extractable, its flux-redistribution rigor targets an explicit small-cell-CFL problem we don't have in the Stokes regime"
User directive: ghost-cell IBM must become production-grade (it generalizes to AMR better than cut-cell)¶
- area: flow
- source: ghost-hardening-plan.md:11-14
- decided: 2026-08-17
- status: settled
- quote: |
Frank's directive (2026-08-17): ghost-cell IBM must become production-grade — it generalizes
to AMR better than cut-cell (its closure cascade is ALREADY lifted into
peclet::core::scheme::ghost_closure, shared verbatim with core's octree AMR band). Plan:flow/doc/ghost_hardening_plan.md(flow 31a4675). - rejected: none stated (cut-cell is the implicit alternative, not rejected outright, but ghost is directed to be hardened)
- why: ghost-cell IBM "generalizes to AMR better than cut-cell"
V-cycle stale-ghost bug made MG convergence decomposition-dependent; fixed, now iteration count is decomposition-independent¶
- area: flow
- source: channel-scaling-rebenchmark.md:67-73
- decided: undated (commit 5d77deb)
- status: settled
- quote: |
5d77debthe decisive one: the V-cycle residual read STALE ghosts, so the coarse-grid rhs was wrong on the whole block-boundary shell → the V-cycle's convergence rate DEPENDED ON THE DECOMPOSITION. Post-fix, iteration counts are decomposition-independent and ~2× lower. Any weak-scaling curve measured before this is contaminated: what looked like a communication tax was partly the iteration count creeping up with rank count. - rejected: reading stale ghosts in the V-cycle residual
- why: caused convergence rate to depend on decomposition, contaminating prior weak-scaling measurements
Variable-viscosity rotational correction stays incremental (constant-μ) under varProps, not full pointwise¶
- area: flow
- source: multiphysics-framework-plan.md:397
- decided: undated (P4)
- status: settled
- quote: |
Incremental-rotational KEPT under varProps (user requirement: large-dt/steady-Stokes; dt=100
conv@200): Timmermans −μ∇·u* is homogeneous-μ-only (Deteix & Yakoubi AML 2018 / arXiv:1902.05643) →
rotational coeff defaults to constant χ·μ_min (stable by domination, exact uniform-μ fallback);
set_variable_rotational('min'|'full'|'off',chi); 'full' pointwise diverges at 10× contrast; full shear-rate projection ... = deferred upgrade. - rejected: 'full' pointwise variable-μ rotational correction
- why: "'full' pointwise diverges at 10× contrast"; user requirement for large-dt/steady-Stokes stability
Velocity domain BCs must be applied before divergence in project()¶
- area: flow
- source: cuda-kokkos-migration.md:332-335
- decided: undated (channel WIP session)
- status: settled
- quote: | CHANNEL DONE (2370f27, verify_channel.py: max|du|/maxu=8.9e-16 vs CUDA, outlet u_max/U_mean=1.4948): the 3rd open-BC bug was that the velocity domain BCs (outflow zero-gradient) must be applied BEFORE the divergence in project()/maxOpenDivergence() (CUDA apply_velocity_bc before diverg_open_k) -- else div(u) mis-counts the outflow and the rotational -mudiv pumps the outflow-wall corner. Fixed via fillVelGhosts(c,0) before divOpen.
- rejected: applying divergence before the velocity domain BCs
- why: "else div(u) mis-counts the outflow and the rotational -mudiv pumps the outflow-wall corner"
Velocity masking must be OFF with the cut-cell pressure operator¶
- area: flow
- source: suite-distributed-status.md:94-95
- decided: undated (Step 19, commit 30df260)
- status: settled
- quote: | velocity masking must be OFF with the cut-cell operator (masking zeros partially-open solid faces post-projection → reintroduces divergence).
- rejected: masking velocity with the cut-cell operator active
- why: masking zeros partially-open solid faces post-projection, reintroducing divergence
Velocity-diffusion MG rediscretization diverges; velocity RB-GS is the exact default¶
- area: flow
- source: suite-distributed-status.md:291-296
- decided: undated (MG Phase 2)
- status: settled
- quote: | MG Phase 2 (commit d0e6928): ... Velocity-diffusion MG rediscretization ATTEMPTED and DIVERGES (the Robust-Scaled fine stencil is row-scaled by D_rescale → inconsistent with a clean I-βL coarse op under geometric transfers; staggered velocity geom ≠ cell-face openness) → reverted to const-coeff coarse; velocity RB-GS is exact and the default (velocity is the easy non-singular operator; proper velocity MG = deferred research).
- rejected: rediscretized velocity-diffusion multigrid coarse operator
- why: the Robust-Scaled fine stencil's row scaling is inconsistent with a clean coarse operator under geometric transfers
Viscous term left plain (not epsilon-weighted) — a deliberate, documented scope choice¶
- area: flow
- source: porous-eps-conservative-momentum.md:31
- decided: 2026-07-12/13
- status: settled
- quote: |
- Viscous term left plain μ∇²u (ε-weighting is next-order for a thin gas; documented).
- rejected: epsilon-weighting the viscous term
- why: it is next-order for a thin gas
VoF execution model: no new submodule, kernels live in flow/src/vof/¶
- area: flow
- source: vof-campaign.md:491-496
- decided: 2026-08-30
- status: settled
- quote: |
Execution model (2026-08-30, plan §11): NO new submodule — VoF in
flow/src/vof/(extraction trigger: kernels → core if AMR/others consume). [...] Escalation rule: twice-failed gate = stop + findings log, never tweak numerics to pass. - rejected: a new submodule for VoF
- why: none stated (extraction to core is conditional on future multi-consumer need)
WO-H fix: CutcellMG::applyNeumannGhost; PCG selector now throws (MG-PCG is the terminal fallback)¶
- area: flow
- source: vof-campaign.md:236-251
- decided: 2026-08-31
- status: settled
- quote: |
Fix
CutcellMG::applyNeumannGhost(:601), ablationPECLET_FLOW_MG_BCGHOST=0. [...] Selector fixed too;set_pressure_pcg(False)now throws (MG-PCG is the terminal fallback). - rejected: none stated
- why: "Proof of intent:
VelocityMGalways did both halves... the pressure MG only ever got the Dirichlet half"
WY advection CFL default corrected to Weymouth's proven 3D bound 0.25¶
- area: flow
- source: vof-campaign.md:128-130
- decided: 2026-08-30
- status: settled
- quote: | Architect corrections on top: WY CFL default → Weymouth's PROVEN 3D bound 0.25 (the famous "CFL<0.5" is his 2D value, thesis eq. A.33), inclusive comparison (flow af8d6f1).
- rejected: CFL<0.5 (Weymouth's 2D value, mistakenly applied to 3D)
- why: "the famous 'CFL<0.5' is his 2D value, thesis eq. A.33"
Wall-band "1/h amplitude growth" was a unit artifact, not a real localization signal¶
- area: flow
- source: collocated-second-order-verdict.md:36-39
- decided: undated
- status: superseded
- quote: |
- Wall-band localisation: test failed 3 ways — the "1/h amplitude growth" was a UNIT ARTIFACT (cell-unit runs have ⟨u⟩∝R²; normalised, near-wall diff DECAYS ~h^0.5–1; Frank caught it); band share crosses a zero of ⟨ΔFlux⟩ at R≈16; elliptic smearing means response-localisation ≠ cause-localisation anyway.
- rejected: the "1/h amplitude growth" reading as evidence of wall-band localization
- why: it was a unit artifact of cell-unit runs (⟨u⟩∝R²); properly normalised, the near-wall difference decays
Weak efficiency must be computed from per-GPU throughput, not raw step time¶
- area: flow
- source: channel-scaling-rebenchmark.md:208-209
- decided: undated
- status: settled
- quote: |
- cells/GPU can't be constant (3-D power-of-2 refinement moves cells in 8× steps) → ±8 % → weak efficiency computed from PER-GPU THROUGHPUT, which corrects it exactly.
- rejected: computing weak efficiency directly from step time when cells/GPU varies ±8%
- why: 3-D power-of-2 refinement can't hold cells/GPU exactly constant
Weak-scaling ladder must refine a fixed physical box, not grow box length at fixed cross-section (methodology correction)¶
- area: flow
- source: channel-scaling-rebenchmark.md:156-168
- decided: 2026-08-10
- status: superseded
- quote: |
★★ ROOT CAUSE FOUND 2026-08-10 (user's question: "how do the settings compare to the example?") —
MY WEAK LADDER WAS THE WRONG ONE.
refinemode is the fix (peclet-examples 70a0704).- The production script
examples/wall-bounded-turbulence/snellius_gpu.slurmscales the DNS by REFINING A FIXED PHYSICAL BOX 4πH×2H×(4/3)πH:GNX=round(2π·GNY),GNZ=round(2π/3·GNY), Δ+=360/GNY. Its presets: GNY=240 (Δ+1.5, 182 M) 1 node, 288 (314 M) 2 nodes, 360 (614 M) 3 nodes. More GPUs buy a FINER DNS of the same channel — never a longer one. - My sweep froze the cross-section and grew GNX only → aspect x/y went 1.6 → 51 vs the physical 6.3; at np1 the box was a QUARTER of the MKM length, at np32 EIGHT TIMES it (Lx+ 18432). Pressure iters track the elongation, not the rank count...
- The production script
- rejected: growing GNX only at a fixed cross-section to weak-scale the channel (my original sweep)
- why: elongation (aspect 1.6→51 vs physical 6.3) made pressure iterations track box elongation, not rank count — an invalid weak-scaling measurement
What was disproved in the agglomerated-bottom investigation¶
- area: flow
- source: agglomerated-bottom-ibm-fix.md:29
- decided: 2026-08-13
- status: settled
- quote: |
What was disproved: solid-rhs deposits from restriction (measured exactly 0 — fine solid
residuals are identically 0 and every child of the bottom's solid cells was solid); sliver rows
from the
dc != 0.0vsac < 1e-30threshold mismatch (no tiny diags in practice — but the mismatch still exists in code); multi-component pockets on random_spheres (1 component; the per-component projector matters for other geometries/coarser bottoms). - rejected: solid-rhs deposit contamination, sliver-row threshold mismatch, and multi-component pockets as causes for THIS case
- why: each measured directly and found not to be the (or a significant) contributor on random_spheres
bcStencilPath() and implicitAdv() must agree with the actual solver in use¶
- area: flow
- source: momentum-solve-residual-stop.md:21
- decided: undated
- status: settled
- quote: |
(1)
bcStencilPath()andimplicitAdv()decide RHS treatment AND the advection scheme — they must agree with the solver used; turning the MG on used to flip advection to explicit (two 1e-11-converged solves 3e-4 apart = different equation, not a solver bug). - rejected: none stated
- why: "two 1e-11-converged solves 3e-4 apart = different equation, not a solver bug"
cylinder-vortex-street dropped from the gallery; confirmed flow bug pins the fix location¶
- area: flow
- source: peclet-examples-gallery.md:106-118
- decided: undated
- status: settled
- quote: |
cylinder-vortex-streetDROPPED — CONFIRMED peclet.flow bug: immersed solid + inflow/outflow broken 3 ways: (a)set_pressure_geometry()afterset_solid()SILENTLY WIPES the solid (order- dependent, →uniform flow no wake); (b)cutcell_pressure=Falseleaks no-slip (|u|~0.6 inside solid); (c)cutcell_pressure=True+inflow/outflow → NaN. ... NEXT session: (a) fix inflow/outflow+solid in peclet.flow (localized to setSolid openness composition, flow_ibm.hpp ~340-377); (b) ship cylinder wake via periodic body-force on GPU (D~30-40, probe wake FFT for St); (c) render BFS on GPU. - rejected: shipping a sub-resolution "steady" wake result (would misrepresent physics)
- why: three concrete bugs in immersed-solid + inflow/outflow composition; a low-res wake would misrepresent physics (real cylinder sheds by Re47)
fillPorousEpsGhosts: mirror-around-1 at inflow/outflow, zero-gradient at walls — one policy for RHS/coeffs/residual¶
- area: flow
- source: porous-cfddem-cuda-two-bugs.md:36
- decided: 2026-07-09
- status: settled
- quote: |
FIX:
fillPorousEpsGhosts()= ONE policy for RHS/coeffs/residual: mirror-around-1 at inflow/outflow (face-mean eps == 1: prescribed inflow velocity IS the superficial velocity — Kuipers/MFIX distributor contract), zero-gradient at walls; called BEFORE divergOpenEps in project() + in maxPorousResidual. - rejected: reading eps ghosts in three different states across RHS/coeffs/residual
- why: "prescribed inflow velocity IS the superficial velocity — Kuipers/MFIX distributor contract"
flow's grid is isotropic unit spacing with no wall-normal stretching¶
- area: flow
- source: channel-dns-isotropic-grid.md:14
- decided: undated
- status: settled
- quote: |
flowuses isotropic unit spacing dx=dy=dz=1, NO stretching (flow_bindings.cpp:472get_spacing"always unit on this grid"; RB example line 117 "Grid units: dx=1"). Physical scale enters only via ν, f, dt and cell counts. So you CANNOT coarsen wall-parallel independently of wall-normal → channel grid is isotropic in wall units. - rejected: wall-normal grid stretching
- why: "none stated" (stated as a hard solver constraint)
get_ox/oy/oz are openness fields, not origin aliases — NAMING.md correction¶
- area: flow
- source: suite-quality-plan-1-0-0.md:101-102
- decided: 2026-09-08
- status: superseded
- quote: |
Traps found: flow
get_ox/oy/ozare OPENNESS FIELDS, not origin aliases (NAMING had it wrong; gallery uses them as arrays). - rejected: NAMING.md's earlier classification of get_ox/oy/oz as origin aliases
- why: "gallery uses them as arrays"
poiseuille-ibm reframed: flat-wall cut-cell IBM is pointwise exact, not a 2.8% error case¶
- area: flow
- source: peclet-examples-gallery.md:48-50
- decided: undated
- status: superseded
- quote: |
poiseuille-ibm— flat-wall cut-cell IBM is POINTWISE EXACT (~1e-7, both meshes); the old "2.8% error" was a metric artifact (u.max vs continuum peak; walls at half-integer put the centre between nodes). Reframed as an exactness demo.
- rejected: the old "2.8% error" framing of flat-wall cut-cell IBM accuracy
- why: it was a metric artifact (comparing u.max vs continuum peak, with walls at half-integer nodes)
rtol rule: max(1e-8, C·eps·0.18N²·Δρ/ρ) — later re-stated as a lower bound, not an achievability guarantee¶
- area: flow
- source: vof-campaign.md:289-292, 319-327
- decided: 2026-08-31 / 2026-09-01
- status: superseded
- quote: |
κ(A) ≈ 0.18·N²·contrast (×4 wall-bounded), exactly linear in contrast, quadratic in N.
⇒ ratio 1000 @256³: κ≈1.2e7, fp64 limit ~1e-9, so a fixed rtol 1e-8 is already near the
arithmetic limit and impossible at 512³ or ratio 1e4. Use
rtol = max(1e-8, C·eps·0.18N²·Δρ/ρ). --- ⚠ κ CLOSURE HAS A VALIDITY BOUNDARY (2026-09-01): holds 192³–384³, BREAKS BY 768³. [...] CONSEQUENCE FOR US: ourrtol = max(1e-8, C·eps·0.18N²·Δρ/ρ)rule inherits this — it is a lower bound on what to DEMAND, not a prediction of what is ACHIEVABLE. Above ~384³ measure the attainable residual; never use the law to dismiss a measured floor. - rejected: treating the κ formula as a prediction of achievable residual at all N
- why: "their 768³ probe floored at 3e-4 = 3 orders above the eps·κ prediction"
sdflow accuracy resolved; default cut-cell RB-GS path matches Zick & Homsy; velocity-MG (not pressure) was the drift source¶
- area: flow
- source: suite-distributed-status.md:268-283
- decided: 2026-06-11
- status: superseded
- quote: |
(c) Accuracy: RESOLVED (2026-06-11) — both codes are CORRECT. ... sdflow's DEFAULT path (
galerkin=False, direct cut-cell RB-GS pressure) matches Z&H to <0.05% at N=128 across φ=0.064–0.343, clean grid convergence, agreeing with pnm to 4 digits. sdflow's cut-cell IBM is correct & grid-convergent. CORRECTION (2026-06-11, later same day): the "+4% drift at N=128" is the VELOCITY-diffusion multigrid, NOT the pressure. I first blamed the Galerkin pressure path, but it was a confound — every drifting run also hadset_velocity_multigridon. Isolating one solver at a time on the SC sphere at N=128 (Z&H K=4.292): velocity RB-GS + pressure-{RB-GS, const-MG, Galerkin-MG, rediscretized-MG} ALL give 4.292 ... but velocity-MG + pressure-RB-GS gives 4.4415 (+3.5%). - rejected: the earlier attribution of the +4% drift to the Galerkin pressure path
- why: isolating solvers showed the pressure coarse operator only changes iteration count; the defect was in
setDiffusionCoarse's geometry-blind const-coeff velocity coarse operator
sdflow-vs-pnm_backend "efficiency gains" correction — internal only, not a real speed gap¶
- area: flow
- source: suite-distributed-status.md:264-269
- decided: 2026-06-11
- status: superseded
- quote: |
sdflow vs pnm_backend reality-check (2026-06-11, doc
cfd-gpu/doc/sdflow_pnm_parity.md). IMPORTANT corrections: (a) the big "efficiency gains" reported during sdflow development (reductions ~96×, etc.) were sdflow-internal, NEVER vs pnm_backend — don't imply sdflow beats production from those. (b) Speed is NOT a real gap: with the SAME numerics (simple RB-GS, no MG/PCG) sdflow ≈ pnm_backend (16.1 vs 15.4 ms/step); the 1.5–3× "slowdown" I'd shown was sdflow's over-engineered defaults (Galerkin MG + PCG to rtol=1e-9 every step — absurd for a steady march). - rejected: implying sdflow's internal speedups meant it beat production pnm_backend
- why: the 1.5-3x apparent slowdown was sdflow's own over-engineered defaults, not a real numerics gap
- conflict: none
set_exact_crossings and set_openness_override remain single-rank-guarded (v1 scope exclusions unchanged)¶
- area: flow
- source: flow-ghost-projection-mpi-plan.md:17
- decided: 2026-07-23
- status: settled
- quote: |
Still single-rank-guarded (inner-sized wrap-access study inputs):
set_exact_crossings,set_openness_override. v1 exclusions unchanged (porous/varRho/domain-BC/Chebyshev). - rejected: extending this MPI landing to cover set_exact_crossings/set_openness_override or the v1-excluded modes
- why: none stated
set_face_interp(9) hybrid (aperture projection + gpCenterGrad) is the throat-safe collocated scheme¶
- area: flow
- source: flow-ghost-projection.md:115
- decided: 2026-07-18
- status: settled
- quote: | THE throat-safe collocated scheme = set_face_interp(9) "cutcell-ghost hybrid": mode-0 aperture projection verbatim (throttles, symmetric MG-PCG, no guard) + gpCenterGrad for predictor/cell-correction only. RCP monotone −13.0/−8.6/−6.2% at Ng=32/44/56 toward the stag-cutcell reference; Z&H −0.04..−0.10% band (NOT clean 2nd order ...) but 7-20× below mode-0.
- rejected: pure collocated ghost projection (mode-0 with gpCenterGrad only) for tight-throat porous media — it inherits the throat defect
- why: "collocated ghost DOES inherit the throat defect ... its raw RCP numbers look deceptively good ... CANCELLATION of throat inflation against the collocated under-shoot, don't trust it on tight throats"
set_ghost_projection(True) must be called before set_solid¶
- area: flow
- source: flow-ghost-projection.md:33
- decided: undated
- status: settled
- quote: |
API:
set_ghost_projection(True)BEFOREset_solid; guards throw for porous/varRho/domain-BC/ Chebyshev/collocated-nonzero-face-interp - rejected: none stated
- why: none stated (API contract)
set_pressure_warmstart(True) diverges on the steady Stokes march; bench default is WARMSTART=0¶
- area: flow
- source: porous-scaling-benchmark.md:106-109
- decided: undated
- status: settled
- quote: |
OPEN BUG found:
set_pressure_warmstart(True)DIVERGES on the steady Stokes march (192³ bed, k → -1.7e+120 by step 400; warmstart=0 converges in 115 steps). Bench defaults WARMSTART=0. Not investigated — flagged to Frank. - rejected: set_pressure_warmstart(True) as a benchmark default
- why: it diverges on the steady Stokes march
⚠️ UNRESOLVED — interstitial vs superficial drag normalisation¶
- area: flow
- source: ibm-accuracy-sphere-validation.md:45-50 and :110-112
- decided: undated
- status: uncertain
- quote: |
l.45 "our
K(=Zick&Homsy) is INTERSTITIAL-referenced; the literature (vdH/Tenneti/van Wachem) reports the SUPERFICIAL total drag F̄_D=(1−φ)·K" l.110 "report K normalized by SUPERFICIAL velocity (=Z&H, →1 dilute); vdH/Beetstra F is INTERSTITIAL so K_vdH=F/(1−φ)" - rejected: nothing — these two statements in the SAME note assign Zick-Homsy and vdH to OPPOSITE conventions. Both observe that the dilute limit hides the difference ("the trap").
- why: unresolved. Decides a (1−φ) factor on published permeabilities (porous-scaling study, RingBed, gallery pages). Found during the 2026-09-10 harvest. Do not rely on either reading until settled.
Published parallel-scaling iteration count is 4.0, not the earlier 12.2¶
- area: flow
- source: parallel-scaling-study.md:119 vs :158 — resolved against peclet-examples d359a00
- decided: 2026-08-10
- status: settled
- quote: | The note carries TWO blocks each marked "FINAL PAGE PUBLISHED": e861010 (2026-08-09) reporting "iters pinned 12.2", and d359a00 (2026-08-10) reporting "decomp iters FLAT 4.0 at every rank". Resolved 2026-09-10 against the published page itself: peclet-examples commit d359a00 ("final page — all results re-measured on the current solver") reports 4.0 at every rank count in the results table and in the prose.
- rejected: the e861010 figures (12.2), which predate the stale-ghost V-cycle residual fix (flow 5d77deb) that lowered iteration counts and made them decomposition-independent
- why: the later block was re-measured after the decisive fix and matches the published page
Double operator storage is the DEFAULT (SCALING_ISSUES #1 closed by decision)¶
- area: flow
- source: flow/CMakeLists.txt:48-72 — maintainer decision 2026-09-11
- decided: 2026-09-11
- status: settled
- quote: |
PECLET_FLOW_OPERATOR_DOUBLEdefaults to ON. Float operator storage silently breaks A*1 = 0 at high multigrid contrast — it fails with no error, and the porous path's default MG-PCG was reporting a non-finite preconditioner on 2 of 5 steps, deterministically (QUALITY_PLAN.md:371). P1 (2026-09-01) measured the cost of being wrong: RCP bed at rtol 1e-8 — float 24/33/CAPPED iterations, div 4.51e-06; double 14/14/28, div 9.51e-12. ~12% step time is the price of a default that cannot silently invalidate a dense-bed run. Opt out with -DPECLET_FLOW_OPERATOR_DOUBLE=OFF, which now emits a CMake WARNING. - rejected: (a) leaving float as the default and documenting the limitation — rejected because the failure is SILENT, so documentation does not protect a user who never sees it; (b) the double-DIAGONAL fallback, which remains retired — it converges to the float-face operator rather than the true one and separated 65x on divergence (P1, DIAGRESUM vs exact)
- why: a clean-break 1.0.0 must not ship a default configuration that can silently produce wrong dense-bed results
- consequence: this changes numerics in the default build. Regression state hashes and
perf_baseline.jsonmust be re-blessed, per the standing rule that changing a numerics-affecting default re-blesses bed references and is itself a decision.
The np>1 VoF colour parity gates gate conservation, not the pointwise field¶
- area: flow
- source: tests/kokkos_mpi/test_vof_bc_mpi.cpp, tests/kokkos_mpi/test_vof_collocated_mpi.cpp — work order docs/wo_vof_mpi_parity_gates.md, carried out 2026-09-14
- decided: 2026-09-14
- status: settled
-
quote: |
vof_bc_mpi_np2asked the VoF colour field to match a single-rank reference to 1e-11 after 40 coupled steps. It cannot: the colour is NOT a Lipschitz function of the velocity.mycNormal(core/vof/plic.hpp) selects between the centred and the Youngs candidate with a strict comparison,if (fabs(mm[cn][cn]) > t0) cn = 3;, and at a near-axis-aligned interface the two candidates carry the same dominant component, so that comparison is an exact tie.Traced on the failing face (packing case, cell (15,0,0), x-face, step 1): the two runs' 3x3x3 colour stencils agree to 1.4e-17 (1-2 ulp); the centred candidate scores 0.99986893026188128 against Youngs' 0.99986893026188106 at np=2 (one ulp apart, Youngs wins) and 0.99986893026188106 against 0.99986893026188106 single-rank (equal, so
>is false and the centred one wins). The two winners differ by 2.9e-06 in their transverse components; the slab volume of that face moves by 4.795e-09, and two adjacent cells part by +/-4.795e-09, equal and opposite. ONE ULP in, 4.8e-09 out.It was therefore a coin toss, not a measurement. Over 24 (ranks, threads) pairs: np=1 bitwise at 1..16 threads; np=2 flips at 2..16 threads and not at 1; np=4 flips at 2, 3, 4, 6 and 12 threads but NOT at 1, 8 and 16 — which is the only reason
vof_bc_mpi_np4passed on the 48-core box while np2 failed, and why CI (4 CPUs) saw neither.The VoF path itself is decomposition-EXACT: the new
packing-kincase drives the same composed cut-cell x open-boundary scene with a prescribed 3-D velocity and is bitwise at np=2 and 4 and at 1..16 threads. Understep()every field the solver carries agrees at the allreduce floor (u,v,w and their ghosts to 1.4e-14; p to 1.4e-13 on |p|=1.9e2).So the gates moved: pointwise colour to 1e-6 (a bound on the field going WRONG), the reduction-floor content to a new SIGNED-SUM gate at 1e-11 relative (a difference that cancels is colour moved, one that does not is colour created), and the exactness to the new bitwise kinematic case. Verified by injection: deleting the WO-F owner rule gives 1.000e+00 pointwise against 1e-06 and 5.141e+02 signed-sum against 3.886e-08.
The same reasoning retired
vof_collocated_mpi'sd(iters) <= 1lockstep gate.hydro-zis a REST STATE (converged |uf| 1.9e-13 on |P| 1.2e+03), so the pressure solve's r0 is the round-off of the hydrostatic balance, not a physical residual, and differs between decompositions by an O(1) RELATIVE amount: the V-cycle count is a random walk, not a shifted copy. Measured over 20 steps at np=2 the sequences part at ten steps, by one at nine and by two at step 11 (10 vs 12), and whether the worst gap is one or two depends on the OpenMP thread count alone (one at 1, 4, 8; two at 2, at both np=2 and np=4). The gate became the COST ENVELOPE — worst step within one V-cycle of the reference's worst, total within one V-cycle per step — with np=1 keeping its exact zero. - rejected: (a) loosening 1e-11 to a number above today's 3.174e-09 — it would have been fitted to one sample of a coin toss and would fail again at a different thread count; (b)d(iters) <= 2for the collocated gate, same objection — the random walk produces a 3 eventually; (c) adding hysteresis tomycNormal's estimator selection so the tie breaks stably — it changes numerics incorefor every VoF consumer to make a test pass, and it only MOVES the discontinuity rather than removing it; the selection discontinuity is intrinsic to MYC as published; (d) pinning the packing case's pressure rtol to 1e-11 (the work order's leading hypothesis) — measured irrelevant: the velocity and pressure fields already agree to 1.4e-14, ten thousand times tighter than the gate, and the colour difference is not proportional to any tolerance - why: a gate must assert something the code can deliver. The pointwise max-norm of a PLIC colour field is a discontinuous functional of the data; conservation, cancellation and the kinematic transport are not, and they are what a decomposition defect actually breaks - consequence: no numerics changed —git difftouchestests/kokkos_mpi/only. The suite-wide rule this establishes: do not gate a max-norm parity across decompositions on a field produced by a branch-selecting reconstruction (PLIC/MYC, slope limiters, ENO/WENO stencil choice). Gate the conservative content, and gate exactness on a kinematic case where no reduction enters.
The velocity V-cycle is the DEFAULT momentum solver, at every rank count and every block size¶
- area: flow
- source: user decision in session; benchmarks/momentum-solver (peclet-examples)
- decided: 2026-09-15
- status: superseded (same day, by the condition-number rule below — the block-size half stands; the 'whenever a solid is present' half does not)
- supersedes: "AUTO velocity MG under MPI below PECLET_FLOW_VMG_AUTO_CELLS=65536 cells/rank and np>1" (2026-09-02, momentum-solve-residual-stop.md:32)
- quote: | "Make V-cycle default for momentum. I do not care that much if numbers shift, as long as the shift is an improvement."
- rejected: (a) KEEPING the 2026-09-02 rule — red-black Gauss-Seidel by default, V-cycle only below 65536 cells/rank and only at np > 1; (b) merely RETUNING that threshold upward; (c) a global-cell floor as the small-problem guard; (d) leaving single-rank and non-MPI builds on red-black.
-
why: | The 2026-09-02 rule had the SIGN OF THE EFFECT BACKWARDS. It switched the V-cycle on where blocks were SMALL, but a pinned-solver re-run of the 1.0.0 scaling ladder (384^3 cut-cell bed, Snellius) shows the V-cycle faster at EVERY rung of BOTH ladders with its margin LARGEST at the BIGGEST blocks: Genoa 1.96x at 2.36 M cells/rank, 2.23x at 147 k, 1.66x at 74 k; H100 2.19x on one GPU at 56.6 M cells/rank, 1.81x on four. Retuning the threshold (b) cannot express "always" and would have preserved a rule whose premise is false.
Red-black was not merely slower: it ran at its iteration cap (velIters_ = 200 sweeps per component) on EVERY rung of both ladders, so it was not converging. The V-cycle takes 9.7 cycles, rank-independent. The 3.7e-07 cross-configuration disagreement in the 1.0.0 deposit's gate table is exactly this — a capped solve against a converged one.
The small-problem guard is stated in per-rank block EXTENT (16 cells on the shortest axis), not in global cells (c), because extent is what decides whether a hierarchy can be built at all: a block that cannot coarsen makes the V-cycle degenerate to its bottom smoother and add only setup. It costs one MPI_Allreduce per geometry build, never per step, so every rank reaches the same decision. - evidence: | ACCURACY IS UNCHANGED, verified rather than assumed. This register carried "sdflow accuracy resolved; ... velocity-MG (not pressure) was the drift source" (2026-06-11, +3.5 % against Zick & Homsy at N=128), marked superseded after the clean-fluid exclude mask fixed it. That flag was re-tested before the flip rather than trusted. Z&H SC drag factor, old default -> new, N=64: phi=0.125 K_ZH 4.292 4.291 (-0.02 %) -> 4.291 (-0.02 %) phi=0.343 K_ZH 15.400 15.394 (-0.04 %) -> 15.394 (-0.04 %) verify_poiseuille_flow PASS, worst node error 1.7e-08. Gates: tests/kokkos_mpi 106/106, tests/kokkos 45/45. The MPI battery also ran FASTER (chunk 1: 109 s -> 66 s). - consequence: | Numbers shift for any IBM run that was on the red-black default — by the momentum tolerance, in the direction of the converged answer, and about 2x faster. The 1.0.0 scaling deposit's figures are therefore a record of the OLD default and are labelled as such in its 2026-09-15 addendum. set_velocity_multigrid_auto(65536, 1 << 23) reproduces 1.0.0 exactly; (0) disables the V-cycle; an explicit set_velocity_multigrid() still wins over everything.
NOT CHANGED: the all-fluid domain-BC path, which never reaches this rule (it fires from set_solid). No evidence was gathered there, so it keeps red-black.
Every VoF consumer is TOLD what a pure cell is; none decides on its own¶
- area: flow
- source: this session (curvature HF collapse), flow
633a144, corekHfPureEps - decided: 2026-09-15
- status: settled
- quote: |
enable_vofruns the advector atwispEps = 1e-8(WO-R2 item 4). A cell the advector calls pure is fluxed ALGEBRAICALLY and never reconstructed back onto exactly 1.0, so the colour field legitimately carries bulk cells at1 - O(1e-9). Any consumer that judges purity at its OWN, tighter tolerance disagrees with the field it was handed — and the disagreement is silent. - rejected: |
(a) letting each consumer keep a private purity constant (what the height function did: a
hard-coded 1e-10, two orders tighter than the advector);
(b) reverting
enable_voftowispEps = 0— that restores the HF tier but reinstates the drained-open-domain failure WO-R2 exists to stop (sum C-> -inf -> NaN in three steps); (c) clipping the colour field onto exactly [0, 1] — rung V1 refuses to clip on purpose, so that conservation closes to round-off with nothing hiding the error. -
why: | Measured twice, the same mechanism both times. Phase change hit it first (
633a144): the Scriven bubble diverged at 48 % R(t) error because phase change judged interfaces at 1e-12 while the advector used 1e-8; the fix waspcEffInterfaceEps() = max(own, wispEps). The height-function cascade was the same class of consumer and was missed. On the mode-2 droplet (48^3, R = 8, mu = 0.0025, 2.5 periods) the HF tier fell from 790 cells to 134 (37 % -> 89 % PLIC fallback) as 425 bulk cells drifted into the band between 1e-10 and 1e-8, and the fitted damping rate came out 1.029e-3 against 1.460e-3 — 30 % low.It degrades SILENTLY and progressively: a fallback is a valid answer, so nothing fails, no test goes red, and the curvature simply gets worse the longer the run goes. That is why the contract is now pinned by a gate (
test_vof_curvaturegate H) rather than by a comment.The drift is BOUNDED by
wispEps— 425 of the 429 drifted cells were inside 1e-8, because a cell that drifts past it becomes mixed again and gets reconstructed. So matching the tolerance is a complete fix, not a threshold chase. - consequence: |core'shfColumnHeighttakes apureEps(floored atkHfPureEps = 1e-10, defaulted so every existing caller is bit-identical);VofCurvature::pureEpsandVofInterfaceArea::pureEpscarry it;Solver::computeVofCurvaturesets it fromvofAdv_.wispEpsAT THE POINT OF USE, becauseset_vof_wisp_epsandenable_phase_changeboth movewispEpsafterenable_vofand a cached copy would go stale.Numbers move for any surface-tension run: curvature on the affected cells switches from the PLIC-volumetric fallback to the height function, which is the more accurate branch. Runs with
wispEps = 0(the standalone advector's default, andenable_phase_change) are bit-identical.
The momentum solver is chosen by the operator's CONDITION NUMBER, and the rule names no geometry¶
- area: flow
- source: user directive in session; measured on a domain-BC channel and the 384^3 cut-cell bed
- decided: 2026-09-15
- status: settled
- supersedes: the entry above (V-cycle whenever a solid is present, 2026-09-15, hours earlier)
- quote: | "I do not like very much that schemes differ based on no-solid. IBM can also be used to sculp the geometry. Try if velocityMG also performs well for case 1. If so, make the solver more aligned."
- rejected: (a) selecting on whether an immersed solid is present — the rule as first landed; IBM is a way of SCULPTING GEOMETRY, so the same physical problem could get two different momentum solvers depending on how its walls were expressed; (b) the V-cycle unconditionally, which is a 0.86x REGRESSION at D = 0.25 where the smoother already converges in 20 sweeps; (c) keeping red-black for the no-solid path on the grounds that it had no evidence — the evidence was cheap to get, and getting it found a segfault.
-
why: | The criterion is now kappa = 1 + 4 dt mu (w_x + w_y + w_z) / rho, the Gershgorin condition number of the implicit-diffusion operator (= 1 + 12 D isotropic, metric-correct anisotropic). V-cycle at kappa >= 13, red-black below. That is the quantity that decides whether a smoother suffices, and it is a property of the TIMESTEP -- not of the mesh, and not of whether the walls came from an SDF or a boundary condition.
THE THRESHOLD IS A CORRECTNESS ONE, NOT A TUNING ONE. Channel with domain BCs, no solid: D 0.25 0.50 1.00 2.00 4.00 6.00 12.00 V-cycle x 0.86 0.87 1.04 1.34 1.88 1.84 1.55 RB-GS it 20 35 59 107 189 200 200 (velIters_ = 200 is the cap) RB-GS res 4e-11 7e-11 1e-10 8e-11 4e-10 7e-08 8e-06 (target 1e-10) Below D ~ 1 the hierarchy is pure overhead; above D ~ 4 red-black stops MEETING ITS TOLERANCE. - evidence: | Also fixed a PRE-EXISTING SEGFAULT this investigation surfaced: vmg_.init() was reachable only from set_solid, so set_velocity_multigrid(True) on a configuration with no immersed solid built a zero-level hierarchy and crashed on the first solve. Reproduced on the pre-change build, so it predates the work. The velocity-MG hierarchy needs nothing from the solid.
The decision also moved to the head of the first step(), where dt/mu/rho are final; taking it at set_solid time read a dt the caller need not have set yet, which for a condition-number criterion is a live bug rather than a latent one.
Gates: tests/kokkos_mpi 106/106, tests/kokkos 45/45, Zick & Homsy unchanged to 4 digits (4.291 at phi=0.125, 15.394 at phi=0.343), verify_poiseuille_flow PASS at 1.7e-08, and the 384^3 D=6 bed still selects the V-cycle. - consequence: | Runs at D < 1 that the earlier same-day rule would have put on the V-cycle stay on red-black, which is faster there. Runs with no immersed solid now reach the rule at all. The 1.0.0 scaling deposit is unaffected: its bed runs at D = 6.
Solid cutting an OPEN domain face is FIXED, not rejected — but a sealed inlet pocket IS rejected¶
- area: flow
- source: SCALING_ISSUES #3 work order; flow/doc/cutcell_openbc_convergence.md
- decided: 2026-09-16
- status: settled
- quote: | "Fix it, or reject the configuration with a clear error — the issue states explicitly that rejecting is legitimate, but that 'should be a decision, not the current silent stall.'"
- rejected: (a) rejecting the configuration outright, which the work order allowed; (b) fixing only the SDF ghost; (c) extending the SDF ghost by MIRROR, or by LINEAR extrapolation, rather than constant; (d) leaving the porous coefficient path on the literal 1.0 behind a runtime rejection. In detail: (a) REJECTING the configuration outright ("solid may not touch an inflow/outflow face"), which the work order allowed. A partially blocked inlet is a physically reasonable thing to want, and both defects turned out to be ordinary bugs with local fixes — an aperture computed from a ghost that had been teleported from the far side of the domain, and a boundary row that overwrote that aperture with 1.0. Neither is a limitation of the discretization, so there was nothing to reject. (b) Fixing only the SDF ghost. That makes the outlet aperture honest and thereby EXPOSES the second defect: the Dirichlet row still says 1.0, the constraint now says 0.59, and the projected divergence plateaus at 6.7e-03 instead of falling to 3e-09. The two had to land together — which is also why the outlet half looked healthy before: a wrapped ghost made the aperture come out ~1 too, so operator and constraint agreed while both were wrong about the geometry. (c) Extending the SDF ghost by MIRROR (what a free-slip face does) rather than constant. The mirror is what a symmetry plane asserts about the geometry; an inlet or an outlet asserts nothing, and the constant normal extension is the reading that makes the boundary face's aperture the fluid fraction of the boundary plane itself. LINEAR extrapolation (s_ghost = 2 s_0 - s_1) was also rejected, although it is the higher-order estimate of the SDF ON the boundary plane where ds/dn != 0: an SDF is not linear along the normal near a curved feature (for a sphere centred on the plane ds/dn is exactly 0 there, so constant is EXACT and linear is wrong), extrapolation can flip the sign across a thin gap, and the order-1 aperture model it feeds is itself first order. (d) Leaving the POROUS coefficient path on the literal 1.0 with a runtime rejection for that combination. Widening the three porous builders by one index along each axis costs three characters each and puts their Dirichlet face on the same formula as every inner face.
- why: |
Defect 1:
setSolidUploadSdffills the extended block by a PERIODIC gather and only free-slip faces were repaired afterwards, so the SDF a non-periodic boundary face sees came from the opposite side of the domain.ccFaceOpensamples AT the face, so that ghost decides the boundary-face aperture: a solid cell against the inlet (sdf = -0.73) was handed a fully OPEN inflow face, the prescribed inflow was counted into it, and its pressure row is entirely closed -- 0*p = U, an inconsistent row. No Krylov driver and no MG depth solves an inconsistent system, which is exactly why the field evidence showed FCG capping, Chebyshev NaN-ing and divergence at MGLEVELS <= 2 and why the agglomerated bottom was (correctly) exonerated. Defect 2:CutcellMG::applyBoundaryOpennessFromwrote the literal 1.0 over the Dirichlet outlet face on every level. That is right when the outlet is clear of solid and wrong when it is cut -- the operator and the divergence constraint then disagree by (1 - aperture) and the projection pushes mass out THROUGH SOLID. Silently wrong, which this project ranks worse than visibly broken. The fix for defect 2 is not new machinery: WO-R2 item 1 already solved the identical problem for the variable-density coefficient (the LOW domain face is an inner index the caller wrote, the HIGH one is a ghost index the periodic fill wraps, so save / restore / coarsen it). It is generalized here from that coefficient to the openness itself, andset_outflow_operator_coefficient(False)stays the ablation back to the literal 1.0. What IS rejected: a cell whose pressure row is entirely closed but which a prescribed inflow still feeds. That is a pocket of fluid the solid seals off from the domain, opening only onto the inlet -- a physically impossible thing to ask of an incompressible fluid whatever the discretization, socheckSealedInflowCellsthrows at set_solid time naming the face and the count. Before defect 1 was fixed this condition fired on ordinary geometry; it now takes a deliberately sealed pocket. - also: |
A THIRD defect, found while gating the fix distributed and made visible by the second fix. Ghost
indices along an axis are 0..g-1 and ext-g..ext-1, so the LOW domain face of an axis is an inner
index and the HIGH one is a ghost index; the halo wraps periodically on EVERY axis (the
decomposition is periodic by construction and the non-periodic conditions are imposed on top),
so on a rank owning that global face the openness came back carrying the OPPOSITE boundary's
aperture -- and divergOpen weights the last cell's outgoing flux by exactly that value, so the
constraint was wrong too. Measured AT np = 1: outlet-cut bed 1.8e-02 from single-rank, inlet-cut
bed 6.6e-01; 0.00e+00 on all four beds after buildOpennessHighFace re-derives the plane from the
SDF. REJECTED alternative: keeping a save/restore buffer across the exchange -- set_solid runs
per step on moving geometry, and the plane is a pure function of an SDF that is already
exchanged and already domain-extended, so re-deriving it needs no state.
The same wrap sits on fillVelGhostsTo(..., doOutflow = false), whose whole point is to preserve
the mass-conserving outflow face the projection wrote (WO-R, gate F2: it has to reach the VoF
advection). THE TWO HALVES ARE NOT SEPARABLE and the repo already had the gate that proves it:
tests/kokkos_mpi/test_vof_bc_mpi drives a packing whose last sphere CUTS THE +z OUTLET PLANE
({8.0, 8.0, NZ, 3.6}, with that comment on the line) and gates the composed conservation
identity d sum(eps_eff C) = boundary ledger absolutely. At np = 1, where there is no
decomposition at all: 1.25e-14 before any of this, 2.46e-02 with the openness half fixed ALONE,
2.89e-14 with both. An outlet whose openness is its own aperture does not pair with an outlet
velocity that is still the inlet's; the two used to wrap together and stay mutually consistent
while both were wrong about the geometry. I had first deferred this half as "a physics path
with no distributed outlet gate, do not fix blind" -- the gate existed, and the deferral would
have shipped a broken conservation identity.
COLLOCATED (2026-09-16, follow-up): no separate fix was needed, and that is a MEASURED result,
not an assumption. Every part of the fix is in the geometry, which is grid-independent, and the
pre-fix collocated inlet-cut bed reads 200 iterations / max|div| 1.000e+00 exactly as the
staggered one does -- the defect was never grid-specific. What was missing was the GATE:
test_openbc_solid now runs its cases on both grids, and test_vof_bc_mpi gained a
colo-jetconservation case, SolverColocated having had no open-boundary conservation gate at all. Two things that look like collocated gaps and are not: the collocated grid's maxOpenDivergenceProjected delegates to the clobbering diagnostic, so its number does not fall to solver tolerance at a partly blocked outlet (a property of the grid, so the collocated gate is the iteration count plus the rejection); and the collocated VoF bridge DOES re-fill the face field's ghosts after the projection corrected the outflow face, which looks like the same defect -- but the advector's boundary flux does not read that ghost index and the identity closes at 1.1e-16. I had hypothesised the opposite and built the gate first; the gate refuted it. SCOPE of the velocity half: STAGGERED only, and UNREACHABLE on the collocated grid rather than skipped -- all three callers passing doOutflow = false return or throw first. There the index in question IS the outflow face and holds exactly what bcCorrectOutflow wrote; on the COLLOCATED grid that correction lives on the FACE field while fillVelGhostsTo fills the CELL field, whose doOutflow = false means "leave the whole ghost BAND alone", so restoring one layer of two would be a third behaviour on a path no test covers (no collocated MPI test carries an outflow face). Left exactly as it was -- the lesson of the vof_bc_mpi regression applied to itself. REJECTED in the repair: restoring the plane over its FULL transverse extent. Its transverse ghost rows are legitimately the NEIGHBOUR's inner values, which the exchange has just delivered correctly; putting stale local values back over them trades one wrong plane for another. The restore covers the INNER transverse range only. The third caller, max_open_divergence_projected(), is covered by the same fix; under MPI with an outlet it used to return approximately the inlet velocity on every bed, converged or not. - evidence: | Duct 32x16x16, mu=1, dt=0.5, 3 MG levels, MG-PCG rtol 1e-10 cap 200, 8 steps, sphere R=4.3. Projected max|div| (max_open_divergence_projected, the residual of the constraint the projection actually solved): bed after before all-fluid duct 3.0e-14 3.0e-14 sphere clear of the open faces 4.4e-09 4.4e-09 (byte-identical) sphere cutting the OUTLET 3.1e-09 1.6e-09 CONVERGED, AND WRONG (see below) sphere cutting the INLET 6.1e-09 200 iters CAPPED, max|div| = 1.0 = U The outlet row before the fix CONVERGED: the wrapped ghost handed the outlet an aperture of ~1 and the Dirichlet row also said 1, so operator and constraint agreed while both were wrong about the geometry -- the mass they conserved was leaving THROUGH SOLID. Isolate defect 2 with the honest aperture in place, i.e. ablate it on the fixed build: 6.7e-03, plateaued (4.8e-03 still at 60 steps against 2.5e-09 fixed). That is the assertion the test carries. The new test run against the pre-fix build fails on four checks: the inlet bed's cap, its divergence, the ablation assertion (the pre-fix outlet is consistent-but-wrong, not inconsistent), and the sealed-pocket rejection, which did not exist. Distributed (test_openbc_solid_mpi, 32^3, 4 steps): the distributed run reproduces the single-rank one EXACTLY at np=1 (0.00e+00 relative on all four beds -- clear, outlet-cut, inlet-cut, wall-cut) and to 1e-15 at np 2/4, with the openness fields bit-identical. Gates: tests/kokkos 46/46, tests/kokkos_mpi 109/109 (np 1/2/4) -- vof_bc_mpi's composed budget among them, back at 2.89e-14 -- Zick & Homsy 4.291 / 15.394 unchanged, verify_poiseuille_flow PASS at 1.716e-08. A DIAGNOSTIC TRAP found on the way: max_open_divergence() refills the outflow ghost with the zero-gradient extrapolation BEFORE measuring, so at a partly blocked outlet it reports how far zero-gradient is from the mass-conserving face -- a property of the boundary condition, not a solver residual, and it does not decay (1.7e-02 on the outlet-cut bed). The original field A/B read that diagnostic. Read max_open_divergence_projected() instead.
- consequence: | A bed clear of the open faces is byte-identical, which is what every existing gate runs. Any bed that TOUCHES a non-periodic face changes: the aperture there is now computed from the geometry instead of from the far side of the domain. That includes wall faces, where the aperture is forced to 0 anyway but the near-boundary IBM stencils read the same ghost. POROUS runs with an OUTLET change even without solid at that face, and deliberately: the LOW outflow face already carried open_feps_fw_f (it is an inner index buildPorousCoeff writes) while the HIGH one was overwritten with the literal 1.0, so the two ends of the same axis were treated differently -- the exact asymmetry WO-R2 item 1 fixed for the density coefficient. The two are now consistent. There is no CFD-DEM outlet gate to measure the shift against; it is bounded by (1 - eps_fw_f) on the outlet plane and is zero wherever eps = 1 and the drag is slack there, which is the usual case. The gap that hid this is closed: flow/tests/kokkos/test_openbc_solid.cpp and flow/tests/kokkos_mpi/test_openbc_solid_mpi.cpp (np 1/2/4) are the first tests anywhere to call setDomainBc together with setSolid with the solid ON an open face. Four existing tests called both, but all of them keep the geometry clear of the non-periodic faces.
init_mpi precedes the geometry; the order is enforced by a raise, not by a rebuild¶
- area: flow
- source: flow
1e6f278(initMpi guard),f41be8f(test_cellforce_mpi); coupling30d1c99; found 2026-09-24 through coupling's np = 4 moving-suspension test - decided: 2026-09-24
- status: settled
- quote: |
set_pressure_geometrybuilds the pressure and velocity multigrids, the SDF ghosts and the cut-cell overlays for whatever partition exists when it runs. Called beforeinit_mpi, that is the single-rank partition, andinitMpinever rebuilt it: each rank solved its own block as an independent periodic pressure problem, every PCG converged, nothing reported an error, and the result was wrong (np = 4: max|du| 4.05e-2 at max|u| 0.358; np = 1 correct by accident).initMpinow raises if a geometry already exists (geometryBuilt_ && !distributed_);redistribute()/rebalance_by_weightsre-enter withdistributed_set and are unaffected. - rejected:
initMpirebuilding the geometry itself, asredistribute()does (a silent rebuild would zero any state set between the two calls, and flow already enforces call order where a wrong order gives a wrong answer —set_decompositionraises afterinit_mpi) - why: a wrong call order produced a silently wrong distributed answer; an error at the call is the only outcome a user cannot mistake for a result
Volumetric forces at the velocity location; surface forces as control-volume face integrals¶
- area: flow
- source: flow
1fdee46(Grid::atVelocity,tests/python/test_cell_force_placement.py); user refinement 2026-09-25 - decided: 2026-09-25
- status: settled
- quote: | VOLUMETRIC forces go where the velocity is defined: a cell-centred volumetric force (force_x/y/z — CFD-DEM drag feedback, Boussinesq, rhog — and the implicit drag coefficient drag_beta, in the RHS target AND the operator diagonal) is interpolated to the velocity location: the face mean 1/2(f(i)+f(i-s_c)) on the staggered grid, the cell value on the collocated grid. SURFACE forces (pressure, viscous stress, surface tension, the epsgrad p / buoyancy part of a fluid-particle force) take the finite-volume view: integrated over the surface of the velocity control volume as face fluxes of a stress or pressure-like quantity — never a cell-centred divergence that is then interpolated. Before 1fdee46 the constant-rho RHS used the cell value (first order on the MAC grid: steady Taylor-Green error order 1.01 -> 2.00 after) and the non-porous drag diagonal the cell beta while its variable-rho RHS target was face-averaged (max|u-U0|/U0 4.4e-2 -> 2.6e-13).
- rejected: interpolating every force the same way (a surface force is a face integral, not a cell value); choosing the placement by the density model or by porous_ (the old buildRhsForced / buildRhsVar / addDragDiagonal split)
- why: the placement is a property of where the unknown lives, not of the time term; one rule
(
Grid::atVelocity) keeps the RHS target and the drag diagonal consistent and the body force second order on both grids
Block VoF: colliding markers — debris removal with exact return, a block-only curvature clip, a gated gas–gas capillary bound¶
- area: flow
- source: flow
f4b105e(branch vof-overlap;doc/vof_overlap_design.md§5, §10–§15,doc/vof_overlap_STATE.mdgates) - decided: 2026-09-25
- status: settled
- quote: | The overlapping-marker blow-up was garbage curvature on sub-cell DEBRIS one marker's WY advection leaves inside another (|kappa| 273 and -428 per cell against a true 0.4 on channel_18), not the SUM-force / MAX-colour pairing, which is statically balanced for constant kappa. Fix: after every block advection, interfacial cells with no C > 1/2 in their 5^3 neighbourhood and sub-wispEps residue are cleared and their volume returned to the SAME marker's attached interface, weighted C(1-C), with fixed-order sums (every marker's volume to round-off; np 1/2/4 bitwise); a per-cell |kappa| <= 1/Delta_min clip on the BLOCK path only; a capillary dt bound with the gas density while markers overlap (max S > 1.01, switch diagnostics.set_vof_phantom_capillary_bound). Also fixed on the way: the block face-velocity gather copied index 0 into the LOW non-periodic domain face, so a marker with colour at a low wall leaked (bubble column: 3 % per marker by t = 84).
- rejected: union-colour force assembly (statically unnecessary; re-creates numerical coalescence through the rim crease); raising interfaceEps (moves the failure); TBFsolver's unsized fragment deletion (deletes satellites and thin sheets); discarding debris without return (percent-level loss over 20 turnovers); the clip on the structured path (fires in single-field gates)
- why: the failure was a local estimator defect on non-interface data, so the fix removes that data conservatively instead of changing the force assembly that was already balanced
Scoped constant-coefficient (Dodd–Ferrante) pressure driver: opt-in, solid-free boxes, accuracy-gated¶
- area: flow
- source: USER DECISION 2026-09-25 on
flow/doc/vof_step_performance_design.md§10 option E2(a); Snellius same-node measurement (TBFsolver 45 ms/step vs peclet 187 ms/step on 24 cores, projection 67 %) - decided: 2026-09-25
- status: superseded (2026-10-03: parked — see "Constant-coefficient pressure driver PARKED")
- quote: | The user's standing directive (on par with or better than SOTA on every case peclet handles) makes CPU parity with TBFsolver a goal, and the performance design shows no variable-coefficient path reaches it (main line: 187 -> ~80-125 ms vs 45). So peclet gets an OPT-IN constant- coefficient pressure driver (Dodd & Ferrante 2014; Cifani 2019): lap(p^{n+1})/rho0 = div(u*)/dt + div((1/rho0 - 1/rho) grad p_hat), p_hat = 2p^n - p^{n-1}, rho0 = min rho, for solid-free boxes (periodic + wall BCs, uniform grid), solved first by the existing MG-PCG on the constant operator (option (a)); the FFT solver (option (b), a new dependency) is a separate, later decision. The general variable-coefficient projection stays the DEFAULT. Adoption for a case requires its accuracy gates against the exact projection at equal resolution: static drop (spurious currents), Hysing 1 and 2, bubble-column statistics (hold-up, rise velocity, wall collection). Order: the main-line performance work first.
- rejected: making the splitting the default (its error ~ (1/rho - 1/rho0) grad(p^{n+1} - p_hat) sits where the capillary jump moves between cells — the earlier objection stands for the general path); accepting a 2x CPU gap to TBFsolver
- why: this SCOPES, not reverses, "Part III ... Dodd–Ferrante FFT not needed" and "S-ladder; Dodd–Ferrante splitting rejected": both still hold for the default driver and for pore-scale / solid cases; the opt-in driver exists only where the splitting is known to work and its error is measured per case
Staggered variable viscosity on the control volume's own faces; explicit velocity MG refuses variable mu¶
- area: flow
- source: flow
b273031(src/face_props.hppVarFaceProps::beta,fillMuGhostsedges),392bf9a(ctestmirror_symmetry),fe377a5(ctestvelocity_solver_variable_mu);doc/variable_viscosity_projection.md§2 - decided: 2026-09-25
- status: settled
- quote: | The staggered unknown u_c(i) sits on the -c face of cell i, so its viscous control volume is centred there: along c, mu is the CELL value (mu_i, mu_{i-e_c}); across c, the EDGE value 1/2[mu_f(i,j) + mu_f(i-e_c, j-e_c)] (the configured mean across the face, arithmetic along c). The old cell-face mean 1/2(mu_i + mu_j) for every component read mu half a cell towards +c: first order (manufactured variable-mu Stokes order 1.39/1.05 -> 1.99/2.00) and, in the wall-normal direction, a preference for the LOW wall — the bubble column's bubbles collected at y = 0 in the case AND its y-mirror (mirror-pair velocity difference 0.44 -> 1.4e-9 after). An explicit velocity multigrid (scalar-mu operator) with a variable viscosity RAISES; AUTO already falls back to Gauss-Seidel.
- rejected: the cell-face mean for every component (h/2 shift); letting an explicit velocity MG run silently on a constant-mu operator (results 50-70 % off Gauss-Seidel under mu(T))
- why: a coefficient must live where its flux does (same rule as volumetric forces at the velocity location, 1fdee46); a silent wrong answer is never acceptable
Collocated variable density (V8): the mass-adjoint ABC pair; the face-acceleration form is retired¶
- area: flow
- source: flow doc/collocated_varrho_forces.md; flow 26c7717 (WO-V1) .. b416144
- decided: 2026-09-25
- status: settled
- quote: | V8 (WO-T) added the pressure gradient and every force as a MAC face acceleration AFTER the implicit viscous solve (Basilisk centered.h). The velocity update was therefore non-incremental: P^n cancelled exactly in the projection, the steady state was scaled by (1 + dt mu Lambda) (TG order -0.1, drag 2.48), and the rotational term became an explicit pressure diffusion, -12 kappa dt/(rho h^2) at (pi,pi,pi), measured -12.0000. The step now puts pressure and forces inside the implicit predictor. Pressure, CSF and set_body_force act on the cell as rho_c * avg_faces[o (F_f - w G_f P)/rho_f]; per-cell forces act at the cell value times the reconstruction weight sum W. The projection reads the momentum-weighted face velocity (rho_L u_L + rho_R u_R)/(rho_L + rho_R), so M*Gamma = -(D O Pi_rho)^T for any openness. Model: spectral radius exactly 1 in ~1200 step operators (ratio to 1e6, dt to 1e4, kappa = mu and 0, immersed disk); steady state dt-independent to 1e-10 (model) and <= 1e-12 relative in the code (guard G3, every V8 path). The rotational update is unchanged.
- rejected: face acceleration after the viscous solve (non-incremental; unstable with the rotational term); kappa = 0 (Chorin); completing centered.h with a lagged g (a Rhie-Chow-type dt-dependent constraint); arithmetic centre->face with the rho-weighted force (1.018/step at ratio 1e4); the naive g - G_c P/rho_c (245x/step); force-aware reconstruction for volumetric forces (places 1/4(1,2,1) f)
- why: stability at every dt and ratio, a dt-independent steady state and cell-value placement need the incremental predictor with an adjoint pair
- conflict: conforms to suite-wide "Collocated forces stay in the implicit predictor" (entry C) and "ABC never Rhie-Chow"; keeps "rotational update must be restored" and "chi*mu_min under varProps"
The balanced-force projection is an option on both grids, default ON on collocated variable-rho (V8), OFF elsewhere¶
- area: flow
- source: flow doc/collocated_varrho_forces.md §4.6-4.8, flow 87f2da9/940d190/1b444ea/d5ed74d/e4a8ad7; USER DECISION 2026-09-25 (an option, on both grids); the V8 default ON is an orchestrator decision 2026-09-25 (the user's cost call is open)
- decided: 2026-09-25
- status: settled
- quote: | set_balanced_force_projection(enabled) (Solver and SolverColocated; default True on the collocated variable-rho/CSF path unless it has inflow/outflow faces, False elsewhere) solves, once per step and with the projection's own operator, constraint divergence and driver, D(O c w G X) = D(O c beta) (beta = the predictor's own face force: f_const + face mean of the cell forces + CSF; c = rho0/rho_f^op), then P += X - P_b, P_b = X. X is solved as the INCREMENT on P_b with the stop relative to the full right-hand side, and skipped when P_b already meets it: a static interface costs no iterations, a changing force about one extra pressure solve with its own persistent Chebyshev bounds (step ON/OFF ~1.2x for a moving ratio-1000 drop or a height-function drop at rest, ~1.0x for a static balance). P_b is restart state (field "p_balanced"): after an OFF->ON switch or a restart without it the step re-splits (P_b := X, P unchanged) and never double-counts (1.25e-2 -> 6e-14, ctest balanced_force_restart). Gradient forces (hydrostatics, constant-kappa CSF) are then balanced exactly from step 1 at every mu, dt, density ratio and openness. That includes immersed solids on the staggered grid, where it removes WO-P's mu*dt^2 residue (code: staggered drop 2.0e-5 / 1.0e-5 at ratio 10 / 1000 -> <= 2e-17; sphere across a ratio-1000 column 9.3e-4 -> 8.7e-14; sessile cap 1.0e-4 -> 2.8e-17). It is state-independent, so stability and the steady state are identical ON and OFF; OFF leaves a decaying transient (code, 3-D: V8 constant-kappa drop face 1.0e-4 at ratio 1 and 6.2e-5 at ratio 1000 after 30 steps, decaying; ON <= 2e-17). Staggered OFF is byte-identical.
- rejected: always-on (the user asked for an option); default off on V8 (loses the settled constant-kappa CSF annihilation); a new staggered predictor (the existing one reads the total P); lagged P_b (first-order for moving interfaces)
- why: exact static balance is a physics choice with a cost (one extra solve), so the user makes it; one setter, one meaning on both grids
On the collocated variable-density path set_body_force is a mean pressure gradient (face form)¶
- area: flow
- source: flow doc/collocated_varrho_forces.md §5; USER DECISION 2026-09-25
- decided: 2026-09-25
- status: settled
- quote: | A uniform per-volume drive in a two-phase fluid stands in for a mean pressure gradient, so it is a surface force and enters as rho_c * avg_faces(o f_const / rho_f), balanced exactly by a linear pressure. Per-cell force fields stay volumetric (cell value), including across density jumps (residual O(delta(f/rho)) in interface cells, accepted). On the staggered grid and at uniform density the two readings coincide bit for bit.
- rejected: f_const at the cell value (periodic hydrostatic box: cell 20.5 after 30 steps at ratio 1000, against 2.4e-13); the force-aware reconstruction for per-cell forces (places 1/4(1,2,1) f)
- why: exact balance of the only uniform drives two-phase users apply; matches the staggered semantics
Pressure MG operator stored in face form {AC, AFX, AFY, AFZ} with the band sign¶
- area: flow
- source: flow
a07d787(WO-3),doc/vof_step_performance_design.md§4.2, §5.3 + the WO-3 note;doc/vof_step_performance_log.md - decided: 2026-09-26
- status: settled
- quote: | The cut-cell pressure operator stores the diagonal AC and ONE off-diagonal coefficient per face (AFX/AFY/AFZ = the low-face coefficient, i.e. the old AW/AS/AB band values; the high-face one is the neighbour's), not seven bands: same bytes per cell as four arrays, ~45 % fewer operator bytes per V-cycle. The coefficient is stored WITH THE BAND SIGN (-ogf): the design's +ogf with a negation in every consumer was 8.8e-15 off on CUDA (nvcc contracts a different product into the FMA); the band sign keeps every consumer's expression literally the old one — bitwise on host, CUDA and float.
- rejected: seven bands (bytes); AC recomputed on the fly (not provably bitwise under FMA contraction); +o*gf storage with negated consumers (not bitwise on CUDA)
- why: a structural change must be bitwise at the old configuration; the storage sign is internal
Host launch rule: pencils on host, MDRange on device, one cell body; -ffp-contract=off on host, -march opt-in¶
- area: flow
- source: flow
363ed6e,e4fb9b2(WO-1),26d1d7a(WO-2); design §4.5, §5.1, §5.2 - decided: 2026-09-26
- status: settled
- quote: |
A hot kernel has ONE cell body (a KOKKOS_INLINE_FUNCTION of the cell index) launched as x-row
pencils (
ccFor3) on host backends and as an x-fastest MDRange on the device. Host builds always compile with -ffp-contract=off (so vectorised and scalar host builds stay bitwise); -march is opt-in via PECLET_FLOW_HOST_ARCH (site/dev builds, never the PyPI wheels). Measured: host MDRange tile loop 2.4x a flat x loop (7-point stencil, 1 thread); cc_residual 97 -> 44 ms/step at 1x8. - rejected: host MDRange in hot paths (2.4x); rebuilding the shared Kokkos prefix with Kokkos_ARCH (changes other repos' bits); -march in the wheels (portability)
- why: host performance without a second copy of every stencil, and without giving up bitwise host reproducibility
Fused periodic wrap in smoother / residual / matvec; device-resident Krylov scalars; no copies around the preconditioner¶
- area: flow
- source: flow
afc3a35(WO-4),d6ad233(WO-5a),e840912(WO-5b); design §4.3, §4.4, §5.4-§5.6 - decided: 2026-09-26
- status: settled
- quote: | On single-rank, even-dimension MG levels the periodic ghost wrap is folded into the smoother, residual and matvec index arithmetic (mg_pfill3 416 -> 19 launches/step); the single-rank MG-PCG keeps its scalars on the device and reads the host once per iteration (projection host reads 15/step at 13 iterations); the preconditioner works in place (device full-field copies 37 -> 11 per step). All bitwise (CUDA, host, float; state_hash 12 cases + np2; bubble column 50 steps). The distributed branches, FCG's scalars and BiCGStab's copies are unchanged.
- rejected: fill-then-sweep with CUDA graph capture; lagged stop checks (not bitwise)
- why: launch- and sync-bound coarse levels dominated the host projection; each change is inert in bits and removes launches, copies or host round trips
Block VoF container stages batched: one launch per stage for all markers, block-ordered force gather¶
- area: flow
- source: flow
8cfa6e0..f6dd038(WO-8a-d,035121aon main); design §4.6, §5.9 - decided: 2026-09-26
- status: settled
- quote: | The block container runs each stage (advection sweeps, debris/residue pass, box update, curvature cascade, CSF force) as ONE launch over all markers through raw-pointer job tables, with at most three host reads per container step; the per-marker CSF force is gathered into the union field in block order instead of scattered. Bitwise to the per-block path (VofBlockSet::batched = false keeps it verbatim for the A/B ctest) on the state, every marker volume and ledger field, MPI np 1/2/4/8.
- rejected: per-block CUDA streams; a pooled atlas of all blocks; atomics in the force scatter (order)
- why: per-block launches and host reads made the container launch- and sync-bound (16 markers)
predict_hierarchy(weights=) returns (rows, align) when weights are given; rows alone otherwise¶
- area: flow
- source: flow 05ce151; docs/NAMING.md change log 2026-09-27
- decided: 2026-09-27
- status: settled (USER approved 2026-09-27)
- quote: |
flow.predict_hierarchy(..., weights=None)takes the flat x-fastest per-cell list thatrebalance_by_weightstakes; with weights it returns(rows, align), align = the 2^a the rebalance would return.weights=Noneoutput is unchanged. - rejected: a new public function for the alignment (unapproved name); an extra column in every row (breaks the existing output)
- why: the name
weights=is user-approved (2026-09-25) and matches core/flow/dem; the return shape was the only open choice and changes nothing for existing callers
The porous "non-finite preconditioner" was a 0/0 in the bottom CG, not float storage (evidence correction)¶
- area: flow
- source: flow f903d27 (CutcellMG::pcgAmg breakdown guard; pcg_breakdown regression case)
- decided: 2026-09-27
- status: settled
- quote: | On a uniform periodic bed the coarse RHS is round-off on a constant (the periodic operator's null space); the AMG maps it to z = 0, so r·z = p·Ap = 0 and the unguarded CG step was 0/0 = NaN. It happens at DOUBLE storage too. Fix: stop when !(p·Ap > 0), as core's GraphAMG PCG does.
- rejected: attributing the porous non-finite preconditioner to float operator storage (the evidence cited in "Double operator storage is the DEFAULT", QUALITY_PLAN.md:371)
- why: that decision still stands on its own P1 evidence (RCP bed: float CAPPED, div 4.51e-06 vs double 9.51e-12); only this one piece of its evidence was misattributed
Steady marches are accelerated by type-II Anderson on the full march state, in a new march_to_steady — never inside step()¶
- area: flow
- source: flow
f6b89fedoc/steady_acceleration.md §1 D1, D2, D4, D7, D8, D11 - decided: 2026-10-02
- status: superseded (2026-10-03) by "Steady marches are accelerated by type-II Anderson in
march_to_steady, measured on the velocity alone; P and the collocated face field are carried (rev 1)" — the c_P-weighted pressure metric was dropped in revision 1 - quote: |
Type-II Anderson, window m = 5 (cap 8), no damping, on exactly what step() reads between steps:
velocity + accumulated pressure P (+ the collocated face field under face advection), metric
weighted c_P = h/(mu + rho h^2/dt), gauge removed. Data path C++/Kokkos (core AndersonCore + flow
AndersonAccelerator
), control path pure Python peclet.flow.march_to_steady. step() is untouched. History in double; np = 1 bit-identical to serial, np > 1 to tolerance. - rejected: velocity-only state (measured stall at 1.9e-4 error); an
enable_*switch that changesstep(); an environment variable; a pure-Python/CuPy data path (not HIP/OpenMP-portable, cannot see the face field); float history (by default) - why: the slow mode lives in P (collocated (pi,0,0) checkerboard, 0.996/step); prototype 315 -> 27 steps to 1e-4 error at N = 16, same fixed point to 4e-9
Steady state is certified by the unchanged stop instrument on PLAIN steps after acceleration¶
- area: flow
- source: flow
f6b89fedoc/steady_acceleration.md §1 D6, §7 - decided: 2026-10-02
- status: superseded (2026-10-03) by "Steady state after acceleration: velocity-residual handover, a 12-block certification budget with an early slow exit, and a stagnation fallback (rev 1 + review R1)" — the instrument and the plain-step rule stand; the handover residual, the budget, the early exit and the stagnation rule changed
- quote: | Accelerate until the residual reaches (1 - 0.997)*rtol, then certify with the unchanged study instrument (slow_rate 0.997, check_every 5, num_passes 3) on consecutive plain steps within a budget; tighten x0.1 and resume if the budget runs out; fall back to the plain march if the plain steps grow. The reported state is a plain-march state.
- rejected: applying the stop instrument to the accelerated sequence
- why: it can stop falsely where Anderson stagnates; certifying plain steps keeps the plain march's guarantee
Steady acceleration scope: staggered and collocated-ghost only; everything else refused with a named error¶
- area: flow
- source: flow
f6b89fedoc/steady_acceleration.md §1 D9, D10 - decided: 2026-10-02
- status: settled (USER approved 2026-10-02; design — implementation pending)
- quote: | Refused: gauge-exact, plain and embed collocated schemes; VoF; phase change; scalars; porous; variable properties; drag; cell forces; moving scenes; superficial velocity; pressure warm start; balanced-force projection. The Anderson history is not checkpointed.
- rejected: accelerating gauge-exact collocated (a family of fixed points); saving the history in checkpoints (~14x larger)
- why: the accelerated iteration must converge to the unique fixed point of the unaccelerated scheme
march_to_steady(accelerate=) default is set by a pre-registered measurement rule¶
- area: flow
- source: flow
f6b89fedoc/steady_acceleration.md §1 D11, §10 Q1 - decided: 2026-10-02
- status: settled (USER approved 2026-10-02; design — implementation pending)
- quote: | Measured at gate G2 on the dense random bed (production build, wall time incl. accelerator overhead): default True iff the accelerated march is >= 1.5x faster in wall time; otherwise False and acceleration is opt-in (symmetric/collocated cases).
- rejected: fixing the default now (True, the architect's proposal; or False)
- why: the large gain is the symmetric-geometry checkerboard, which random beds do not excite; the dense-bed gain (expected 2-4x, possibly ~1.5x) is unmeasured, while the cost is ~4 % per step + ~25 % memory
PV curvature fallback: one team per target cell, canonical-order accumulation (core pvFitTerm / pvFitAccum)¶
- area: flow
- source: core
cb4c7ba(released v1.3.2), flowf992f4d(WO-7b),d7d36bd(WO-7c); designdoc/vof_step_performance_design.md§4.7, §5.11 - decided: 2026-10-03
- status: settled
- quote: |
The tier-3 PV paraboloid fallback runs on the device as one warp-team per target cell: the
team's lanes compute the 125 neighbour terms (core
pvFitTerm) in parallel and ONE lane accumulates them (pvFitAccum) in the canonical offset order, so the result is bitwise the one-thread kernel. On the batched block container (WO-8) a fixed number of persistent teams, min(concurrency/32, 65536), strides over the device-counted entries — no host read. Bitwise vs main (CUDA + host, state_hash + bubble-column dumps, batteries 188/188); tier 3 on the column 14.8 -> 7.4 ms per call (shared GPU, indicative). - rejected: a warp reduction of the normal equations (changes the summation order); one team per possible cell on the batched path (~2e5 mostly empty teams); reading the fallback counts to the host (breaks WO-8's <= 3 host reads)
- why: the fallback was latency-bound (one thread fitting a 5^3 paraboloid per cell); parallel terms + serial canonical accumulation removes the latency without moving a bit
Steady marches are accelerated by type-II Anderson in march_to_steady, measured on the velocity alone; P and the collocated face field are carried (rev 1)¶
- area: flow
- source: flow doc/steady_acceleration.md "Revision 1" R1, D3 (rev 1), §1.2, §3.2; log WO-5 §1.3
- decided: 2026-10-02
- status: settled (architect rev 1; implemented flow 5538616 + core 9ff3bd2, released core v1.4.0)
- supersedes: "Steady marches are accelerated by type-II Anderson on the full march state…" — its metric clause ("weighted c_P = h/(mu + rho h^2/dt), gauge removed"); the rest is restated here unchanged
- quote: |
Type-II Anderson, window m = 5 (cap 8), no damping, in a new march_to_steady; step() is
untouched. The state stays (u, P [, u_f]); P and u_f are mixed, stored and differenced but not
measured. The residual is the relative velocity residual ||g_u - x_u|| / ||g_u||. Data path
C++/Kokkos (core AndersonCore + flow AndersonAccelerator
), control path pure Python. - rejected: the c_P-weighted, gauge-centred pressure term (rev 0); per-pocket gauge removal; the unweighted Euclidean norm of (u, P); a metric on the face gradient of P; a velocity-only STATE (the 1.9e-4 stall — P stays in the state, it is only unmeasured); an
enable_*switch that changesstep(); an environment variable; a pure-Python/CuPy data path - why: on the dense bed 99.97 % of the rev-0 residual was pressure in sealed / near-contact pockets that moves no velocity (rev-0 bed gain 1.26x in steps); a pressure error is measured by the velocity it drives. Measured on the C++ build: dense bed 3.49x (staggered) / 2.29x (collocated) in steps, 4.44x on the collocated sphere
Steady state after acceleration: velocity-residual handover, a 12-block certification budget with an early slow exit, and a stagnation fallback (rev 1 + review R1)¶
- area: flow
- source: flow doc/steady_acceleration.md "Revision 1" R2, R4, D6 (rev 1), §7; review fix R1 (2026-10-03)
- decided: 2026-10-03
- status: settled (architect rev 1, orchestrator review fix R1; implemented flow 5538616, b8a96c8). Q18 — whether converged=True must also promise that a plain march would reach the state — DECIDED by the user 2026-10-03: stationarity only, with the caveats documented in MarchResult (uniqueness is a Stokes result; with advection an unstable steady branch can be certified; slow tails under-read)
- supersedes: "Steady state is certified by the unchanged stop instrument on PLAIN steps after acceleration" — its handover residual (now the velocity residual), its budget (was num_passes + 3, no early exit), and the stagnation rule (was a halving of the best residual)
- quote: | Phase A hands over at (1 - slow_rate) rtol on the relative velocity residual; phase B certifies with the unchanged instrument on consecutive PLAIN steps within 2 (num_passes + 3) = 12 blocks; a block failing on the remainder bound with R in [slow_rate^check_every, 1) resumes acceleration at once ("slow"); budget or slow exit: target x 0.1, resume with the history. Phase A stagnates after 10·window calls without the residual falling below slow_rate^(10·window) times its best; stagnation or growth (> 10x) disables acceleration and the plain march finishes. converged=True certifies stationarity at this dt over the certification's plain steps — not that a plain march from the initial state arrives there.
- rejected: requiring plain-march reachability as well (Q18: a plain verification tail costs plain steps exactly where acceleration matters — the dense bed at dt 60 does not certify plainly in 20 000 steps); applying the stop instrument to the accelerated sequence; the 6-block budget (staggered N24 m8: 214 steps vs 135 plain; 74 at 12 blocks); no budget; handing over on the monitor's own changes along the accelerated sequence; stagnation as "no halving in 10·window calls" (review R1: a slow but real tail was declared stagnant)
- why: after an Anderson iterate the monitor's block changes start near zero and change sign or grow for up to five blocks before the slow tail emerges; certifying plain steps keeps the plain march's guarantee; in the supported scope the fixed point is unique (§2.3), so a stationarity certificate names the discrete steady solution
The Anderson steady-march accelerator has no instability guard; stability evidence comes only from plain steps (rev 2)¶
- area: flow, core
- source: flow doc/steady_acceleration.md "Revision 2", D5, §2.4, §4.6, §7; log "Revision 2" R2-1 … R2-6
- decided: 2026-10-02
- status: settled (architect rev 2; implemented core f9956ed, flow 46f94d5; released core v1.4.0)
- supersedes: the rev-0 / rev-1 Ritz guard (D5) and D5's "Rejected: no instability guard" — this entry reverses that rejection
- quote: | AndersonCore makes no stability judgement: no Ritz estimate, no status "unstable", no MarchResult reason "unstable". An unstable plain map shows only on plain steps — the certification's growth exit, R >= 1 never passes, budget/slow resume, stagnation fallback.
- rejected: a Ritz-radius guard with a higher floor, a higher threshold, a longer consecutive count; the fall-back-to-plain consequence; a diagnostic-only radius; a Ritz-pair residual test
- why: the radius is a Rayleigh-Ritz value of a non-normal map in an oblique (velocity-only) metric, bounded by the numerical range, not the spectrum. Exact STABLE synthetic maps (rho = 0.99) tripped the rev-1 guard in 12 calls with radii 1.013-1.035, above U4's true positive 1.0105; the WO-5 tight false alarms reproduce across backends. Without the guard G7a still ends not-converged at m = 3, 5, 8, both false alarms converge (2.5e-10, 6.2e-10 vs plain K), and production is unchanged (42/42 step counts)
An Anderson restart restores the last kept map output; the state never holds a rejected evaluation (review R2)¶
- area: flow, core
- source: flow doc/steady_acceleration.md §4.3 "Restart (amended 2026-10-03, review R2)"; core unit test U11
- decided: 2026-10-03
- status: settled (orchestrator review fix R2; implemented core 868938f, released core v1.4.0)
- quote: | After the broadcast (every rank acts on rank 0's decision) buf := Gprev and step 6 is skipped: Rprev/Gprev stay those of the last kept output, the rejected evaluation's rho does not enter rho_min, and residual keeps the restored output's value. The next call is a plain step from the restored output. Cost: one evaluation per restart; "too many restarts" leaves the last good one.
- rejected: "Restart keeps Rprev/Gprev (step 6 still commits)" — the fifth restart then left the state at the rejected output
- why: WO-8's G3 Δt-60 run reached 2.7e-12 and ended at 9.0e-5 on a rejected output; the K "4e-7 off" was sampled at that iterate
march_to_steady(accelerate=) defaults to True — the pre-registered rule, measured¶
- area: flow
- source: flow doc/steady_acceleration.md D11, §1.3; log WO-5 G2
- decided: 2026-10-02 (by the user's pre-registered rule, Q1 / Q13)
- status: settled
- quote: | Staggered dense bed (phi 0.6, N 64), production, window 5, wall time incl. the accelerator: plain / accelerated = 3.38-3.45x on host-openmp, 3.55-3.87x on CUDA (>= 1.5) -> True. Collocated bed beside it: 2.48-2.53x / 2.38-2.73x. Steps 325 -> 93 (staggered).
- rejected: False (opt-in acceleration)
- why: the rule's threshold is met by more than 2x on both backends
Device pressure bottom: block-tridiagonal FP32 direct factor preconditioning FP64 FCG ('direct', the GPU default)¶
- area: flow
- source: flow
9c791f7..23a3631(WO-6/WO-11 B1, §13 WO-D0..D4); designdoc/vof_step_performance_design.md§4.1, §13; logdoc/vof_step_performance_log.md - decided: 2026-10-03
- status: settled (its host clause "Host backends keep GraphAMG, bitwise" superseded 2026-10-08 by "Host backends use the 'direct' pressure bottom where eligible"; the distributed clause stands until WO-H6)
- quote: | On GPU backends the pressure multigrid's bottom (bubble column: 16x12x8 = 1536 cells) is solved on the device: an FP32 block-tridiagonal factor with explicit Schur-complement inverses, plane by plane along the slow axis, preconditions an FP64 FCG to tau = 1e-5; one factor launch per operator change, one solve launch per V-cycle. A·1 = 0, the mean projection and the stopping residual stay FP64 (a float factor inside an FP64-verified iteration is admissible; an unrefined float bottom is not). The singular operator is handled exactly by a rank-1 term per fluid component on its last-eliminated plane. Results are bitwise independent of team size. Bubble column, quiet RTX 5080: step 42.6 -> 36.1-37.4 ms, projection 23.7 -> 17.2-17.8 ms; 50-step difference to the host GraphAMG bottom 2.7e-14 with identical outer iterations; inner iterations max 2 (mean 1.02); bulk device<->host transfers in the projection 0. Misses the design target (projection <= 12.5 ms): the factor's dependent chains are L2-latency-bound (~4 us per phase vs the 1 us assumed). Host backends keep GraphAMG, bitwise; the distributed path keeps GraphAMG.
- rejected: B1's single-team V-cycle preconditioner (measured +5.3 ms/step, ~1250 dependent phases per solve; retired, code in 9c791f7/4ce0cfb); a dense 1536^2 Cholesky / inverse (memory traffic, a vendor-solver dependency); all-SM Krylov (launch-bound); Chebyshev to tau (~240 applications); a deeper / factor-3 bottom (+8.5 outer iterations); an unrefined FP32 bottom; a diagonal shift or single-cell pin for the singular operator
- why: USER DIRECTIVE: no GPU<->host transfers in the step; the plane structure makes an exact factor cheap, and keeping the convergence check in FP64 keeps the float-storage failure mode out
Constant-coefficient (Dodd–Ferrante) pressure driver PARKED: fails the static-balance gate¶
- area: flow
- source: USER DECISION 2026-10-03; flow branches
vof-e2(ab1f2bc: E2.0–E2.2 structure + log/handoff) andvof-e2-e23-stopped2(60655dd: the WO-E2.3 candidate) on origin; designdoc/vof_step_performance_design.md§12, §12.13 (on vof-e2) - decided: 2026-10-03
- status: settled
- quote: | The opt-in Dodd–Ferrante driver (E2(a): constant-coefficient pressure on the existing MG-PCG) is parked, not merged. With the corrected D-E2.8' (rotational term on div(u)) it is stable for any dt and viscosity and the bubble column runs (max|p| 577.7 vs 576.5, max|w| 16.39 vs 16.43, PCG 8.00 vs 13.32 iterations/step), but it FAILS the static-balance gate: drop at mu = 0.1, density ratio 50, 1000 steps — spurious velocity 2.6e-4 vs 5.5e-8 for the exact projection (ratio 1/50: 7.9e-6 vs 1.7e-7); the split–exact difference decays 7–20x slower than the Dodd–Ferrante model (hypotheses: a non-gradient explicit term at interface faces; feedback through the moving interface). The first design (div(u*)) was unstable with viscosity (mu_r dt kappa/rho0 > rho0/(rho_max - rho0)). Payoff while stable: ~40 % fewer pressure iterations; CPU parity with TBFsolver would still need the FFT solver (E2(b)).
- rejected: merging it as opt-in despite the failed gate; a third design round (uncertain, the payoff modest); TBFsolver's non-incremental form (dt-dependent steady state with implicit viscosity, a mu·dt·sigma·kappa spurious force)
- why: the user's approval (2026-09-25) was conditional on the accuracy gates; the measured static balance is ~5000x worse than the exact projection — exactly the register's original objection (error ~ sigma·kappa). Re-opening needs a design that addresses the non-gradient term first.
FFT pressure solver (Dodd–Ferrante option E2(b)) DROPPED — no FFT driver in flow¶
- area: flow
- source: USER DECISION 2026-10-08 ("Drop the FFT pressure driver");
flow/doc/vof_perf_STATE.md"Open"; designdoc/vof_step_performance_design.md§10 option E2(b) - decided: 2026-10-08
- status: settled
- quote: | The FFT pressure solver for the constant-coefficient (Dodd–Ferrante) splitting, option E2(b) of the VoF step-performance design and left "a separate, later decision" on 2026-09-25, is dropped. flow gets no FFT driver and no FFT dependency. CPU parity with TBFsolver on the bubble column is pursued on the exact variable-coefficient projection (the §14 host package, measured in Snellius S-1).
- rejected: an FFT Poisson solver (new dependency) behind the constant-coefficient driver
- why: an FFT solves only the constant operator, so it is useful only behind the E2(a) splitting, which is PARKED for failing the static-balance gate (~5000x the exact projection's spurious velocity); it would also be limited to solid-free uniform boxes with periodic/wall BCs. Re-opening needs a constant-coefficient driver that first passes the static-balance gate, then a new decision.
Zero-aperture staggered velocity points stay live in the momentum solve (not masked)¶
- area: flow
- source: flow doc/steady_acceleration_log.md "2026-10-03 — Pocket pressure" + "Coordinator ruling on option (iv)"; probe tests/study/pocket_mask_probe.py + pocket_mask_closed.patch
- decided: 2026-10-03
- status: settled (coordinator; measured)
- quote: | A1 bed (phi 0.6, N 64): 41 velocity points are live in the momentum mask (sdf > 0 at the face) while their projection aperture is exactly 0. Masking them: |dK/K| 2.0e-8 (tight), Z&H bit-identical, per-body reaction -1.8e-3 = exactly the body force f·h^3 on the masked points.
- rejected: masking every velocity point whose projection aperture is 0 (option iv); NaN for sealed-pocket cells in get_p() (poisons np.mean silently)
- why: the points hold real fluid in near-contact gaps; forced fluid that cannot flow passes its body force to the enclosing walls, so counting it keeps sum F = f·V_fluid; K does not move. Side effect kept: pocket pressure leaks into those points' velocity (~3e-4 of max|u| at realistic pocket pressures); reaction forces and K see <= 1e-7
Bubble-column benchmark case runs at pressure rtol 1e-8; the solver default stays 1e-10¶
- area: flow
- source: USER DECISION 2026-10-03; flow D1 tolerance study (
doc/vof_step_performance_log.mdWO-13; rerun on main a0afc9b, raw data /home/frankp/Codes/bubble_column_perf/d1_main/); peclet-examplesbenchmarks/bubble-column/scripts/run_peclet.py - decided: 2026-10-03
- status: settled
- quote: | The published bubble-column case solves the pressure to rtol 1e-8 (case script only; the solver's default stays 1e-10, per the "tolerance changes in case scripts only" default). On current main: pressure iterations 13.93 -> 10.55 per step (-24 %), total gas-volume drift 5.2e-12 per 2000 steps (2.5e-14 at 1e-10), static drop and Hysing unchanged to <= 2e-11.
- rejected: 1e-6 (iterations 7.49, but a systematic gas-volume drift 1.5e-9 per 2000 steps — ~1e-7 over a production window, against the multiple-marker container's round-off conservation); keeping 1e-10 (24 % more pressure work for differences far below the statistical uncertainty of the swarm)
- why: the swarm statistics are uncertain at the 10-15 % level from near-contact treatment; the projection tolerance only has to keep conservation near round-off, which 1e-8 does
cell_centers() vs analytic-scene placement in cell units: documented, the fix deferred to 2.0.0¶
- area: flow
- source: RELEASE_PREP §12.5 item 3; flow fc4db2c (docstrings); measured 2026-10-03: blocked-face centroid offset (0.5,0.5,0.5) cells at N 24 in cell units, 0 with extent=
- decided: 2026-10-04
- status: settled (coordinator, released with family 1.4.0; the 2.0.0 fix itself is not yet designed)
- quote: | Without an extent, cell_centers() returns i + 1/2 while set_scene places cell (i,j,k)'s centre at (i,j,k), so the same body given as an SDF sampled on cell_centers() and as a scene lands half a cell apart. With a physical domain both use origin + (i + 1/2) h. Through 1.x both conventions stay and are documented (docstrings, CHANGELOG known limitation); 2.0.0 unifies on origin + (i + 1/2) h in every entry path.
- rejected: changing either convention in a 1.x release (silently moves every cell-unit scene or SDF script: a numerics change and a semantic break the alias ladder cannot express); a runtime warning on cell-unit set_scene (noise for scripts that are correct under their own convention)
- why: QUALITY_PLAN D9 — a break costs a major; the physical-domain path, the recommended one, is consistent
Chebyshev pressure bounds re-estimated from a warm start after a coefficient rebuild (D3)¶
- area: flow
- source: flow
3819fd0, follow-ups9498c5d,110b560(WO-12); designdoc/vof_step_performance_design.md§5.13; log 2026-10-04 - decided: 2026-10-04
- status: settled
- quote: | After a variable-density coefficient rebuild the Chebyshev driver re-estimates its spectral bounds with 5 + 5 power iterations seeded from the previous NON-DEGENERATE estimate's kept v_max / v_min (masked, mean-removed, normalised) instead of a cold 15 + 15 from the right-hand side. Guard: if the solve hits the cap, or its residual after 3 iterations exceeds r0, re-estimate cold and redo (counted in diagnostics.num_pressure_chebyshev_restarts). redistribute carries the kept iterates (cross-step state). Bubble column, Chebyshev at rtol 1e-10: V-cycles/step 44.15 -> 24.05, projection 172 -> 97 ms (shared GPU); 2000 steps, 0 guard firings; static drop and Hysing equal to ~1e-12. Accepted with two literal misses of the note's gates (50-step V-cycle total +2.7 % vs ±2 %; 6/50 steps' residual > 2x main's, run maximum below main's).
- rejected: the cold 15 + 15 estimate every step (30 of 44 V-cycles per step); warm-starting from a degenerate (zero-RHS) estimate (gave bounds [0,0] and NaN pressure)
- why: the variable-density default driver spent two-thirds of its V-cycles re-estimating bounds that barely change between steps
Block-container statistics in one team-per-block launch, read back deferred (C2); host MG reductions in pencil order (B2)¶
- area: flow
- source: flow
91bd16c(WO-9),1e8ae47(WO-10); design §5.10, §5.8 - decided: 2026-10-04
- status: settled
- quote: | C2: per-block volume, area, centroid, moments and velocity in one launch (team per block); values reach the host at the next container step's read or on demand; diagnostics only (feed no state), so their summation order is the team reduction's (velocity is a difference quotient: centroid round-off ~4e-16 amplified ~5e4). 48 reduction launches + syncs per step -> 1. B2: host pressure-MG sum reductions (dot, ccReduce3, removeMean) run over (y, z) rows on ccFor3's static partition, x ascending and scalar, partials joined in thread order; never serial below the cutoff. Host results move by ~2.5e-14 (11 of 12 host hashes re-baselined); CUDA bitwise.
- rejected: C2 — three fenced per-block reductions per statistic; B2 — host MDRange reductions with an implementation-defined order, an omp-simd reduction whose order depends on the ISA
- why: launch and sync count on the container; an ISA- and thread-count-independent host order
Host backends use the 'direct' pressure bottom where eligible (reverses §13 D-5's host clause and Q-D6's default)¶
- area: flow
- source: flow
93c92ba(WO-H1); designdoc/vof_step_performance_design.md§14.3 H-1, §14.7 Q-H1, §14.8 entry 1; log "§14 WO-H0 … WO-H3" and "Snellius S-1" (jobs 27770798, 27770799; raw /home/frankp/Codes/bubble_column_perf/s1/) - decided: 2026-10-08 (design 2026-10-04; held on branch cpu14-h4 until S-1; orchestrator decision on the S-1 numbers)
- status: settled
- quote: | On host (OpenMP) backends set_pressure_bottom_solver('auto') resolves to the FP32 block-tridiagonal 'direct' factor + FP64 FCG wherever §13.4.6 admits it (single rank, <= 64 components, b <= 192, n <= 8192); otherwise GraphAMG runs, unchanged. A recorded numerics change: host G-NUM-H replaces host bitwise to GraphAMG (50-step difference 3.3e-14, pressure iterations identical, 300-step FCG max 2 / mean 1.02 / restarts 0, static drop and Hysing unchanged). Snellius genoa, bubble column, rtol 1e-8, 300 steps, ms/step direct vs algebraic: 1x24 contiguous 93.10 vs 95.81, OMP_WAIT_POLICY=active 94.86 vs 96.22, spread over 12 CCDs 83.81 vs 89.38 — faster at every single-rank layout. Revert = restore the kHostMemory line in directBottomIneligible().
- rejected: GraphAMG kept for host bitwise; GraphAMG at tau = 1e-5; GraphAMG setup reuse
- why: the host GraphAMG bottom (incl. its per-step buildAmg) cost 1.50 ms per V-cycle, ≈ 20 ms/step at 13.5 V-cycles — nearly all of the 22.4 ms/step the 2026-10 Snellius profile spent outside kernels; the plane-structured factor keeps the FP64 convergence check
- supersedes: the host clause ("Host backends keep GraphAMG, bitwise") of "Device pressure bottom: block-tridiagonal FP32 direct factor preconditioning FP64 FCG ('direct', the GPU default)"; the distributed clause stands until WO-H6
On host, the 'direct' bottom FCG's reductions are single-lane in index order (bits independent of T_host and of the thread count)¶
- area: flow
- source: flow
93c92ba(WO-H1); design §14.3 H-1, §14.8 entry 2 - decided: 2026-10-04
- status: settled
- quote: | The host bottom FCG reduces in one lane in index order, so its result is bitwise independent of T_host and of OMP_NUM_THREADS (U3 on OpenMP: factor + M and the whole FCG bitwise for T in {1, 2, 4, 8}); T_host = 8 lanes affect speed only.
- rejected: team reductions on host (order depends on the team size)
- why: a thread-count-independent host result, matching the register's host reduction-order rule (B2)
The distributed host 'direct' bottom runs redundantly on every rank, from the allgathered bottom openness (WO-H6, designed, not yet implemented)¶
- area: flow
- source: design
doc/vof_step_performance_design.md§14.3 H-6, §14.8 entry 3 - decided: 2026-10-04
- status: settled (design; until WO-H6 lands the distributed path keeps GraphAMG — S-1 6x4/3x8/8x3 ran 'algebraic')
- quote: | Under MPI the host 'direct' bottom is factored and solved redundantly on every rank from the allgathered bottom openness; no rank-0 solve and broadcast.
- rejected: a rank-0 solve plus broadcast; GraphAMG
- why: with the single-lane host reductions the redundant solve equals the single-rank 'direct' result bitwise at any thread count (given the gathered openness equals single-rank's); the gather reuses GraphAMG's global map and gatherv, and the per-V-cycle rhs allgatherv GraphAMG already pays
One-rank benchmarks run without init_mpi; init_mpi at size 1 stays on the distributed path¶
- area: flow
- source: flow
637ccfe(WO-H0:tests/study/vof_perf/run_mpi.py,bench_cpu.sh); design §14.3 H-0, §14.7 Q-H2, §14.8 entry 4 - decided: 2026-10-04
- status: settled
- quote: | A one-rank benchmark skips mpi_block / init_mpi and runs the single-rank engines (np 1 kernel listing: selfCopy 0, cc_smooth_box 0; dump bitwise to prof.py at the same rtol). init_mpi at communicator size 1 keeps the distributed engines, because the np = 1 gate needs them. The old published 24-core number (144 ms) used the distributed path at np 1 and rtol 1e-10; under H-0 and rtol 1e-8 the same 1x24 case measures 93.1 ms (S-1, with the rest of §14).
- rejected: a silent single-rank switch inside init_mpi
- why: the np = 1 distributed run is the serial<->parallel equivalence gate; the benchmark is a protocol choice, not a code path
Host list-driven batch kernels launch over exact counts with a dynamic schedule; host region kernels iterate rows¶
- area: flow
- source: flow
05332a0..7a4ba2d(WO-H4 (a)-(e)); design §14.3 H-4, §14.8 entry 5 - decided: 2026-10-04
- status: settled
- quote: | On host backends the VoF block container's list-driven batch kernels launch over the exact work count with a dynamic schedule, and its region kernels (bbox, compaction scans, movePiece copies) iterate rows with the x run inside; bitwise to the previous host kernels. Container kernels 132.9 -> 23.8 ms/step at 8 threads [workstation]. The device keeps its kernels.
- rejected: upper-bound launches under GCC's contiguous static schedule (idle threads on the short tail); per-cell div/mod index maps
- why: GCC compiles Kokkos' static RangePolicy to schedule(static) without a chunk; a block's active entries (~10 % of its upper-bound range) sit at its head and land on one thread, so with 16 blocks on 24 threads at least 8 idle; the region kernels paid three 64-bit div/mod per cell plus a job search
Benchmark layouts on Zen never straddle an L3 (CCD) with one rank¶
- area: flow
- source: design §14.2, §14.7 Q-H5/Q-H9, §14.8 entry 6; S-1 placement audit (bubble_column_perf/s1/summary_main.txt)
- decided: 2026-10-04
- status: settled
- quote: | On AMD Zen nodes every MPI rank of a benchmark layout is bound inside one L3 (CCD): 6x4, 3x8 and 8x3 per-CCD masks, audited per run. S-1 (genoa): 8x3 one rank per CCD 90.72 ms, 6x4 106.53, 3x8 107.87 (algebraic bottom). Single-rank spread placement (24 threads over 12 CCDs) is 10 % faster than contiguous (83.81 vs 93.10, memory bandwidth); the published comparison stays on contiguous cores and TBFsolver is not rerun at spread — the spread number is recorded alongside.
- rejected: 8x3 on contiguous cores 0-23 (ranks straddling CCDs)
- why: 8x3 on contiguous cores gives two ranks cores that straddle two L3s; measured on Snellius they took 122.9 and 124.4 ms kernel time against 98.0-101.7 for the others, which waited in MPI
Cut-cell scalar transport stores κ (the fluid fraction) in storage and sources¶
- area: flow
- source: flow doc/scalar_ibm_design.md §1.3–§1.4, D1, §12 entry 1; doc/scalar_ibm_log.md round 1, round 6
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Cut-cell FV with κ (fluid fraction) in storage and sources; every κ > 0 cell with an open face is an unknown Neumann, aperture FV: 2nd order with κ in storage/source; 1st order without it. unit storage (flow today) | stable, but the physical mass Σκc drifts −12 %, −5.5 %, −2.1 %
- rejected: unit storage, i.e. the legacy scalar's (first order; the Neumann eigenvalue 11 % off at 32 cells per diameter; 2–12 % mass loss under advection)
- why: κ in storage is what makes the Neumann limit second order and the advective mass exact (1e-16 in the 2-D annulus); the small cells it creates are handled by the dynamic implicit split, not by giving up κ
Every κ > 0 scalar cell with an open face is its own unknown; slivers are never merged or linked¶
- area: flow
- source: flow doc/scalar_ibm_design.md §1.3, §2.4, §12 entry 2; doc/scalar_ibm_log.md round 2
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Merging slivers costs the Neumann limit; every κ > 0 cell must be its own unknown.
- rejected: cell merging or linking of slivers (the H-k / Hf-1 link schemes: Neumann limit −4.2e-3 at order 1.0, against −6.2e-5 at order 2.0 for κ-FV with every cell its own unknown)
- why: one unknown per κ > 0 cell keeps every Robin number second order; the conditioning cost of tiny κ is carried by the probe closure (which never uses a cut cell's own tiny distance) and by the small-cell split
The scalar unknown sets follow the snapped apertures; κ = 0 ⇒ all apertures 0 by the strict-sign rule¶
- area: flow
- source: flow doc/scalar_ibm_design.md §2.2 ("Fluid" means φ > 0 strictly), §2.4, §2.6; ruling D-WO7-1 (doc/scalar_ibm_log.md, WO-7 rulings; commit bb039d1)
- decided: 2026-10-02 (fluid), 2026-10-03 (solid, D-WO7-1)
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Fluid unknown: κ > 1e-14 and Σ_f a^snap_f > 0. "Fluid" means φ > 0 strictly. A vertex with φ = 0 is solid. This makes a wall lying exactly on a face close that face, so κ = 0 never coexists with an open face. D-WO7-1 (Q-L): option (b). A solid unknown exists iff κ_s > 0 ∧ (Σ a_s > 0 ∨ the cell has a conjugate facet) — the solid mirror of the fluid's sealed rule.
- rejected: unknown sets from a κ that is not the apertures' own volume (plane-cube
plicVolume: κ = 0 next to an open face, κ > 0 behind all-closed faces, i.e. division by κ and orphan rows; the 4³cs_subsampling: slivers read κ = 0 with open faces); the design's first solid rule "κ_s > 0 and conjugate" (it admitted 95–192 uncoupled solid unknowns per rung whose transient rows m(ψ − ψⁿ) = 0 froze them, so G6's total-mass error stalled at 0.44); the strict mirror "κ_s > 0 ∧ Σ a_s > 0" (it would drop solid cells coupled only through a conjugate facet) - why: a cell is an unknown exactly when the operator couples it, so there are no orphan or frozen rows; with D-WO7-1, G6 is second order on the total mass (1.91–2.00) and the uncoupled count is 0 on every gate run
Sealed scalar cells (fluid or solid) are not unknowns; they are gated on their VOLUME, not their count¶
- area: flow
- source: flow doc/scalar_ibm_design.md §2.4, §9; rulings D-WO2-1 and D-WO7-1 (doc/scalar_ibm_log.md WO-2, WO-7 rulings)
- decided: 2026-10-03
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: |
D-WO2-1 — sealed cells. Gate (e)'s "0 sealed" is restated as: sealed volume Σκ·V over sealed cells
≤ 1e-6 of the fluid volume (the §9 threshold; measured ≤ 1.9e-9). The sealed count stays in the
census. The sealed definition (§2.4) and the snapping rule (§2.3) are NOT changed.
The excluded volume is in the census (
sealed_solid_volume, withsolid_volume= the conjugate solid κ_s V, physical) and gated ≤ 1e-6 of the solid volume (as D-WO2-1). - rejected: a "0 sealed" count gate (impossible: the corner slivers whose faces all snap to 0 at the 1e-3 floor grow like (R/h)², 0–11 / 38–46 / 154–159 at R/h 8 / 16 / 32); making sealed cells unknowns (empty rows); changing the 1e-3 snap floor (it is the pressure's measured conditioning constant, and linear exactness needs both-ends snapping)
- why: the sealed volume is ≤ 1.9e-9 of the fluid volume and ≤ 1.6e-8 of the solid volume on every gate geometry; a fluid one above 1e-6 prints a resolution warning
Scalar faces carry the plain aperture two-point flux¶
- area: flow
- source: flow doc/scalar_ibm_design.md D2, §1.4, §12 entry 3; doc/scalar_ibm_log.md round 3
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | The face-centroid flux interpolation makes no difference (JC vs JC-nofc agree to 3 digits), so it is not needed.
- rejected: Johansen–Colella face-centroid interpolation (no measurable change; costs symmetry and compactness)
- why: the 7-point part stays symmetric and compact, and the surrogate built on it stays SPD
Immersed scalar walls use probe-flux: one probe on the facet normal, the BC eliminated per facet¶
- area: flow
- source: flow doc/scalar_ibm_design.md D3, §1.4, §3, §12 entry 4; doc/scalar_ibm_log.md rounds 1–3; gates G1–G3, G7 (log WO-3/WO-4)
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Conclusion. The decisive ingredient is a wall gradient over a probe distance ≥ ~0.7 h, interpolated from neighbours. It never uses the cut cell's own value over its (possibly tiny) distance. One unknown set and one formula cover Neumann → Robin → Dirichlet at 2nd order, and the 3-D stencil is compact: 27-point plus a reach of 2.
- rejected: flow's per-cell Dirichlet mask (order 0.7–1); the Gibou linear ghost alone; Papac/Gibou symmetric Robin (order 1.4–1.7, its Dirichlet limit a mask); the aperture + link hybrid (no symmetric compact scheme is second order at all Bi); the centroid two-point / series-resistance wall flux (order 1; its own entry); the quadratic normal probe (smallest error but erratic: signs and orders jump)
- why: second order at every Bi from 0 to ∞ in 2-D and in 3-D: G1 Nu 1.93/1.97, G3a Da 0.1–100 1.93–2.00, G7 Graetz 2.01–2.02, G2 1.89/1.94
The scalar probe distance is s = 1.1·½Σ|n_a|h_a (0.55–0.95 h), the shortest that never needs the fallback¶
- area: flow
- source: flow doc/scalar_ibm_design.md D6, §3.1–§3.3, §12 entry 4; doc/scalar_ibm_log.md "probe-distance sweep" (2026-10-02);
jc.py probeN1.1 - decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | The fallback (renormalization = local constant extrapolation) is what degrades short probes. Choose the probe so that the stencil never needs it. | AMReX κ rule | 762 | 7.3e-5 | 9.3e-5 | 0.7–1.2, erratic | | 0.3 h | 762 | 1.1e-4 | 1.5e-4 | 0.4–0.9, erratic | | 0.7 h | 0 | 8.4e-6 | 3.5e-5 | 2.0, reach 2 |
- rejected: AMReX's κ-dependent short probe dx_eb = max(0.3, (κ² − ¼)/(2κ)) h with renormalized fallback (762 fallbacks, order 0.7–1.2, erratic); a 0.3 h probe (order 0.4–0.9, erratic); a constant √3/2·h probe (the worst case of S(n), needlessly long: error grows with the distance, 0.7 h 3.5e-5, 1.0 h 1.5e-4, 1.5 h 4.5e-4)
- why: s ≥ S(n) guarantees every trilinear stencil cell of a planar wall is at least cut, so R0 holds on every smooth geometry (100 % on the isotropic sphere and pipe sets) and the reach stays within the G = 2 halo; the normal-dependent distance matches the constant 0.7 h at order 2.0 (probeN1.1)
The immersed scalar wall flux is never a two-point / series-resistance flux over the cut cell's centroid distance¶
- area: flow
- source: flow doc/scalar_ibm_design.md §12 entry 4; doc/scalar_ibm_log.md round 1, round 5, round 7
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Dirichlet: Gibou linear ghost 2nd order; mask 0.7–1; cut-cell FV with a centroid wall flux 1st. Robin, series-resistance FV: 1st order. | GFM series (= Liu–Fedkiw–Kang) | 1.2e-3 / 2.4e-2 | 7.5e-5 / 5.1e-3 | 4.5e-5 / 2.6e-3 | 2.0e-5 / 2.6e-4 |
- rejected: the centroid two-point wall flux and the series-resistance (GFM, Liu–Fedkiw–Kang) wall and interface flux (first order; at the Maxwell disc, d/h = 128, k₂ = 0.01, GFM L∞ 2.4e-2 against probe-flux 1.3e-4); a surrogate built on the centroid distance (ρ(I − S⁻¹A) = 0.995, BiCGStab 59–87 iterations against 8–9 with the probe distance)
- why: a distance that shrinks with the cut makes the flux depend on the cell's own (possibly tiny) value; the probe distance keeps both the accuracy and the conditioning
Conjugate scalar transport is two fields on one grid in ψ = c/K, with the series-resistance 2×2 elimination at the fluid and solid probes (P2F)¶
- area: flow
- source: flow doc/scalar_ibm_design.md D8, §1.1, §1.3–§1.4, §12 entry 5; doc/scalar_ibm_log.md round 4, round 7 (
conj_peters.py), WO-7 (G4, G6) - decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | The one-field directional scheme imposes the jump of k ∂T/∂x along grid lines, not along the interface normal. On tilted interfaces its L∞ stays first order. P2F, with the true normal, probes and per-facet elimination, is 2–50× better in L∞ at every contrast and 3–10× better in L1, except at k₂ = 100, where DIR (tex reading) equals it. Decision. P2F remains the conjugate method. Peters' directional scheme is recorded as a measured alternative.
- rejected: Peters' directional one-field scheme (min-curvature T_ξ; Maxwell disc at d/h = 128, L1/L∞: DIRc 6.7e-5/6.0e-3, 5.7e-6/4.0e-4, 7.9e-6/4.3e-4, 1.9e-4/8.4e-3 against P2F 9.6e-6/1.3e-4, 4.1e-6/6.4e-5, 5.5e-6/9.2e-5, 2.0e-5/2.3e-4 at k₂ = 0.01 / 0.5 / 2 / 100; L∞ order ~1; no contact-resistance path; first-order Neumann limit); one field plus a side array; a mixture one-field surrogate (arithmetic coarsening across the jump); Das-style lagged partitioned coupling
- why: 3-D G4 orders 1.84–1.94 at Λ_s/Λ_f = 1e-2 … 1e3, exact (≤ 3e-11) at ratio 1, K exact, and G6 1.91–2.00 for all six (Λ_s, C_sK, K, R_c) cases; ψ = c/K keeps the coupling symmetric and the surrogate SPD; the two fields are full because the solid is 50–60 % of a packed bed
Scalar cut-cell geometry is the fan-tetrahedron PL model on the marching-squares samples; the scalar apertures are ungated and snapped at both ends¶
- area: flow
- source: flow doc/scalar_ibm_design.md D4, D5, §2.2–§2.3, §2.6, §12 entry 6; log WO-1, WO-2
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: |
The fan-tet κ is the divergence-theorem volume of the same PL surface that defines the apertures.
It is exactly consistent, bounded in [0, 1], and κ > 0 whenever any aperture is.
So the scalar keeps its own fields
sax/say/saz, andox_is never touched. - rejected: plane-cube
plicVolume(n, φ_c)for SDF sources (inconsistent with the apertures on curved walls, edges and gaps); 4³ subsampling (cs_, quantized to 1/64); the cell-centre projection as facet centroid (O(h) off the facet in corner-cut cells, hence first order); reusing the gated pressure opennessox_(up to ~0.5 wrong at every centre-solid face, an O(1) flux error on a surface set) - why: κ, apertures, facet area vector and centroid come from one polyhedron, so the PL divergence identity holds to round-off (5.6e-16) and a_f + a_s = 1 after snapping gives linear exactness; volume and area converge at order 2.00 (G-geom e, g)
ScalarMG coarsens the surrogate with rediscretized coarse FACES and variational coarse WALL terms: the plain average of the level-0 terms at the FINE probe distance (Amendment A1)¶
- area: flow
- source: flow doc/scalar_ibm_design.md §5.2 Amendment A1, §5.3, §12 entry 7; doc/scalar_ibm_log.md WO-4, WO-4 completion; commits c1e312e (RAP evidence), 03435df, 35e183d;
tests/study/scalar_ibm/mg_coarse_wall.py - decided: 2026-10-03
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Every lumped wall or interface term on a coarse level is the plain average of its level-0 value, evaluated at the fine probe distance Rescaling s by the level size divides that energy by ~2^L. The coarse problem becomes softer than any prolongated function, by 2^L, exactly where the coarse cell cannot resolve the near-wall dip. The correction then overshoots by that factor.
- rejected: wall terms at the level's own probe distance s_L, the original design (stand-alone contraction 0.79 at the finest G1 and 3.1, divergent, on a periodic box with a Dirichlet sphere; 2-D 1.5–2.7; its rationale "keep the wall-to-face ratio level-independent" is refuted); Galerkin RAP (27-point coarse operators and an 8-colour sweep, no better overall: 0.28–0.36 in 3-D, 0.33–0.42 without a solid); extending CutcellMG; VelocityMG's staircase coarse operator (Dirichlet-like coarse walls, wrong for Neumann and singular closures)
- why: after A1 every gate geometry contracts at 0.17–0.32 (box, Neumann, no solid) and 0.50–0.52 (periodic isolated sink), G1 iterations 12/14/15 → 10/10/11, MPI parity at the default rtol (np 4) 7.3e-8 → 1.5e-13; this CONFIRMS for the wall term the amr entry "Rediscretized coarse momentum operators fail" — the face part stays rediscretized because the surrogate is in the rediscretizable family
The scalar linear solve is BiCGStab + one ScalarMG V-cycle on the lumped-probe surrogate, with a max-norm relative stop (default 1e-10)¶
- area: flow
- source: flow doc/scalar_ibm_design.md D9, D11, §4.3, §5.1, §12 entry 8; doc/scalar_ibm_log.md round 5
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Conclusion. Use BiCGStab (or GMRES/FGMRES), preconditioned by an MG V-cycle on the SPD lumped-probe surrogate. The cycle is a fixed linear operator, so plain BiCGStab is correct (no FGMRES).
- rejected: fixed RB-GS sweeps (the legacy 50-sweep path); (F)GMRES (restart storage and k reductions per iteration; FGMRES unnecessary for a fixed cycle); the centroid-distance surrogate (ρ = 0.995)
- why: 8–9 BiCGStab iterations for Dirichlet and 2–5 for Robin with an exact surrogate solve, mesh-independent; in 3-D G1 10/10/11 and G-iter met on every row
Scalar small cells under advection: a dynamic implicit-FOU split relative to the bulk Courant number¶
- area: flow
- source: flow doc/scalar_ibm_design.md D12, §6.3–§6.6, §12 entry 9; doc/scalar_ibm_log.md round 6, WO-5 (G9)
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | small_i ⟺ κ_i < 1 ∧ Δt·Out_i > max(½, C_bulk) · κ_i V A cut cell stays explicit exactly when its own Courant number is no worse than the bulk's.
- rejected (for now): weighted state redistribution (needs a ~3-cell halo; its gain sits where no-slip velocities make the flux O(h); recorded for VoF and slip walls, §13 Q8); explicit κ storage (blows up at min κ 4e-5); fully implicit FOU (more diffusive, L1 1.63 against 1.50)
- why: stable at full-cell CFL, mass to 1e-16, positive; in 3-D (G9) the cut band carries ≤ 4.5 % of the FOU L1 error at R_o/h 64, so Q8 is not triggered
Scalar advection uses the projection's own constrained face flux; the scalar apertures weight diffusion only¶
- area: flow
- source: flow doc/scalar_ibm_design.md §6.1, §12 entry 10; doc/scalar_ibm_log.md WO-5 (G9b)
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: |
One predicate
Solver::scalarFaceFlux(a)returns it. It must read the same openness as the divergence kernel of the projection in force. - rejected: the scalar apertures for advection (not the projection's constraint, so Σ F ≠ 0 per cell and constants are not preserved)
- why: G9b: a constant stays exactly 1 (max|c − 1| = 0) under staggered, 'gauge-exact', 'plain' and 'embed'; wrong openness gives ~1e-2
Cut-cell scalars refuse the ghost projection (collocated 'ghost', staggered set_ghost_projection) and, for now, collocated open faces¶
- area: flow
- source: rulings D-WO5-4 (§13 Q13) and D-WO5b-1 (doc/scalar_ibm_log.md WO-5, WO-5b);
Solver::scalarCutRefusals(src/flow_ibm_scalars_cutcell.hpp) - decided: 2026-10-03
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main); the open-face refusal is reversible once the collocated
uf_outlet issue is settled - quote: | D-WO5-4. The collocated 'ghost' scheme is refused for cut-cell scalars (Q13): its face field is not divergence-free under any openness (2–4.5e-2). Collocated users select gauge-exact, plain or embed. D-WO5b-1 (Q-I). Collocated open faces stay REFUSED for cut-cell scalars (option c).
- rejected: advecting with the ghost projection's face field under its binary openness
oxb_(divergence 4.3e-2 of max|F|) or the geometricox_(2.0e-2): a constant drifts 4.1e-2 / 0.80 / 1.28 at bulk Courant 0.016 / 0.47 / 0.86; capturing the collocated high-side boundary flux afterproject()(the plane holds the opposite face's values: a constant drops 0.48 in the outlet column in one step) - why: the ghost constraint is the binary divergence PLUS the closure delta (
gpDivergDelta), so no openness-weighted flux of it is divergence-free; this is design revisit item (iii) for 'ghost', the AUTO default of SolverColocated, so a cut-cell scalar there needs an explicitset_collocated_scheme('gauge-exact' | 'plain' | 'embed')
Scalar conservation is exact in the discretization; the solver residual is reported as a defect, never fixed up¶
- area: flow
- source: flow doc/scalar_ibm_design.md D13, §6.4, §9, §12 entry 11; gates G3b, G9, G9c, G-adv (v) (log WO-3 … WO-5c)
- decided: 2026-10-02
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | Conservation is exact for the discretization; the budget reports the linear-solver defect; no mass fix-up
- rejected: a post-solve uniform mass correction
- why: the budget identity closes to round-off (≤ 2.8e-14 of d_mass per step, ≤ 1e-11 of the gross budget for steady solves, D-WO5c-1); a large defect means a loose tolerance and must stay visible
Steady scalar advection is preconditioned by a V-cycle on the ADVECTIVE surrogate (Amendment A2)¶
- area: flow
- source: flow doc/scalar_ibm_design.md §6.7 Amendment A2, §12 entry 13; doc/scalar_ibm_log.md WO-5 (Q-H), WO-5c;
tests/study/scalar_ibm/mg_advection.py; commits a61f88e, e1127ed, 7a3e5fb - decided: 2026-10-03
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | In steady mode with advection the V-cycle runs on the advective surrogate S_adv = S + the implicit-FOU couplings, on every level. Coarse advection: the summed positive parts of the sub-face fluxes. This is the piecewise-constant Galerkin value. Transient mode, and steady mode at rest, are unchanged and stay bitwise.
- rejected: the symmetric lumped surrogate for steady advection (preconditioned spread β ≈ Pe_h N/2π; 3-D 13 → 40 → 89 iterations at Pe_h 0 / 0.1 / 0.3 and no convergence at 1); advection at level 0 only (2-D 35 / 119 / 222 at Pe_h 0.1 / 1 / 10 — the failing modes are the low ones); a symmetric surrogate with the upwind diffusion sym(A) (β → N/π); pseudo-transient continuation (the same spread on every step, or O(N²) steps); GraphAMG on the assembled FOU operator (SPD design with a Chebyshev smoother, no MPI path); defect correction around the same preconditioner; VelocityMG's restricted-velocity coarse upwinding (not divergence-free on the coarse grid); downstream or line smoothers (no order on a periodic torus; not parallel on a GPU block or across MPI blocks); the net (signed-sum) coarse flux (measured identical, but drops the coarse exchange diffusion); F- or W-cycles as the default (they halve the count but change the cycle; recorded escalation §13 Q18)
- why: 3-D G-adv converges everywhere to Pe_h 10: (a) 8/13/23 and 10/17/33 iterations, closures 5/8/12 and 7/10/17, growth ≤ 1.43× per doubling, C4 contraction ≤ 0.764, the solution unchanged (9.8e-12); the transient and at-rest paths are bitwise unchanged (170/170 arrays)
The transient ScalarMG level rule switches to the full table at κ_A = 1 + 4 dt'D'Σw_a ≥ 25 (Amendment A3, ruling D-WO9-3)¶
- area: flow
- source: flow doc/scalar_ibm_design.md §5.2 Amendment A3 + its D-WO9-3 paragraph; doc/scalar_ibm_log.md D-WO9-2 and the WO-10 entry;
ScalarMG::kFullTableKappa(src/scalar_mg.hpp); commits bdc957c (13, named), b9c8ca0 (25) - decided: 2026-10-04
- status: settled (design + orchestrator rulings, branch scalar-ibm; in force when it lands on main)
- quote: | D-WO9-3 (2026-10-04) — the switch moves to κ_A = 25. The orchestrator relaxed G-iter's bound on the cold first step to ≤ 12; warm steps stay ≤ 10. κ_A = 25 is the largest measured value at which level 0 is measurably cheaper (table above); at 49 the two tie.
- rejected: κ_A = 13, the design's value, kept by D-WO9-2 only because level 0 there took 11 iterations on one cold first step against G-iter's ≤ 10; κ_A ≈ 49 (cost alone: a tie, 15 against 10 iterations, not measurably cheaper)
- why: per advance on the 128³ bed level 0 is 1.05–1.24 vs 1.93 s at κ_A 7, 1.23 vs 1.69 s at 13, 1.35–1.55 vs 1.86–2.04 s at 25; host advance ÷ projection at G-perf's condition 1.50 → 1.05; bitwise unchanged outside 13 ≤ κ_A < 25
The scalar-transport and phase-change energy API is physical under an armed extent, converted at the boundary¶
- area: flow
- source: flow
db28d0c3e249201c80540c978aeb5efd9fa(branch scalar-units);src/flow_ibm.hppUnitScales comment "Scalar transport and the phase-change energy path" - decided: 2026-10-02
- status: settled (USER DIRECTIVE 2026-10-02: "Do not leave it, but fix it.")
- quote: | A transported scalar (temperature, concentration) is never rescaled; every other scalar/energy quantity converts by its M-L-T dimension in the internal system (length hRef, time tRef, mass rhoRefhRef^3): diffusivity x tRef/hRef^2, mass flux x tRef/(rhoRef hRef), latent heat x tRef^2/hRef^2, conductivity x tRef^3/(rhoRef hRef^4), rho c_p x tRef^2/(rhoRef hRef^2), R_int x rhoRef hRef/tRef, divergence source x tRef. Constants are kept verbatim and re-derived when a scale is pinned (order-independent); the two FIELD setters (set_mass_flux, set_divergence_source) raise under an extent until set_rho/set_dt have run. Closures on rho/mu/force_* take physical parameters (inputs are unscaled fields). Getters (vof_interface_area, phase_change_diagnostics, the budget and carry ledger) report physical. The operator-flux mdot without set_phase_change_energy RAISES under an extent: its constant-D flux carries an implicit rho c_p of one internal unit.
- rejected: documenting the scalar/energy surface as internal (cell) units — "silently wrong" is the failure class U3's four missed setters already showed; converting fields set before the scales are pinned with whatever scale was current (silently wrong if set_rho/set_dt come later); interpreting the constant-D operator mdot's implicit rho c_p as "1 in the caller's units" (unit-system dependent physics)
- why: the plan's contract is one consistent physical system in and out; gates units_scalar_scale_invariance (9.6e-16), units_scalar_sine_decay (exact BE amplitude to 8e-14, order 1.98 on cubic and box cells), units_phase_change_scale_invariance (<= 1.7e-13 incl. 12 full sucking-interface steps); cell units bit-identical (12/12 state hashes)
Collocated AUTO default = ghost projection (2026-08-25), superseding "ghost as production is a dead end"¶
- area: flow
- source: flow
c672014, core7472306(commit messages); memoryarchive/collocated-attractor-campaign.md:53 - decided: 2026-08-25 (user decision; recorded in the register 2026-10-08)
- status: settled
- quote: | [flow c672014:] DEFAULT SWITCH: collocated scheme AUTO = ghost (user decision 2026-08-25) SolverColocated resolves its scheme at set_solid: AUTO (the new default) selects the GHOST fluid-only projection -- family-free, unconditionally stable, protocol-independent (doc/collocated_invisible_subspace.md; clean ladders both beds, Z&H anchor) -- and falls back to gauge-exact with a stderr notice on the configurations ghost v1 does not support (porous / variable-rho / domain-BC / Chebyshev / analytic overrides / fluid-only instruments). Any explicit set_collocated_scheme / set_face_interp / set_ghost_projection / set_fluid_only_constraint call disables AUTO, so every existing explicit configuration is unchanged. [core 7472306:] AmrFlow AUTO = ghost projection [...] with an aperture fallback + stderr notice when the finest band is too thin for the overlay.
- rejected: gauge-exact (ex mode 9) as the unconditional collocated default, chosen by the 2026-08-18 A/B; the 2026-08-20 verdict's "ghost as production" dead end
- why: the collocated-attractor campaign (flow
doc/collocated_invisible_subspace.md) found ghost family-free, unconditionally stable and protocol-independent, with clean convergence ladders on both beds and the Z&H anchor; validation: AUTO == archived ghost trajectory digit-for-digit, explicit gauge-exact == its record, regression ×3 green, 33/33 MPI ctests, AMR ctests CUDA + OpenMP - supersedes: the "ghost-as-production" part of "Mode-10 quadrature, Seo-Mittal pressure-only split, and ghost-as-production are all dead ends" (mode 10 and Seo–Mittal stand)
The host pressure-bottom factor runs a bitwise host-only schedule, with its own team constant (2)¶
- area: flow
- source: flow
62f91ef591bde4d8cd5ca(commit messages);src/mg_bottom_direct.hpp(hostAssemble / hostInvert / hostBorder),src/mac_cutcell_mg.hppkBottomHostFactorTeam;doc/vof_step_performance_log.md"A(b)" - decided: 2026-10-08 (A(b) USER DECISION: faster factor, bitwise, before factor reuse; team constant = implementer DECISION, genoa sweep pending)
- status: settled (schedule); provisional (the value 2, pending the genoa T sweep
bubble_column_perf/s1/s1_bfac.slurm) - quote: | On a host backend the factor runs a second schedule of the same arithmetic: every stored scalar is the team algorithm's expression with its operands in the same order, but the work is cut into a few barrier-free phases per tile and each phase computes many independent outputs in a vectorizable lane loop instead of one serial dot product per thread. Its factor storage (Q, Y, e, s, stat) is bitwise identical to the team algorithm's on the same backend (ctest bottom_direct U8). [kBottomHostFactorTeam = 2:] two is the fastest size that is also stable under load. Device code untouched (if constexpr on CCMem == HostSpace).
- rejected: running the device team algorithm on the host (≈170 team barriers per plane, every scalar a serial latency-bound dot: 5.6-6.5 ms per factor at its best T 4-8, 11 ms at T 1); sharing kBottomHostTeam = 8 with the FCG solve kernel (the host schedule at T 8: 1.43-1.88 ms vs 1.05-1.09 at T 2); reordering any accumulation for speed (would change bits; A(a) factor reuse is a separate, numerics-changing decision)
- why: bench_bottom_factor (walls-y 16x12x8, P 12, b 128 = the bubble column's bottom), workstation 5965WX under load ~20, ms per factor: old team T 1/2/4/8/16/24 = 11.0/7.9/5.7/5.6/6.4/6.5; host schedule T 1/2/3/4/6/8 = 1.44-1.51/1.05-1.09/0.93-1.79/0.86-1.70/1.43-1.69/1.43-1.88; production kprof 1x24 mg_bottom_factor 6.6 -> 1.5-1.8 ms/step (genoa S-1 base 16.9). Gates: factor bytes identical to main (host and CUDA, walls-y + periodic/border), U8 incl. the restart path with a negative control, host G-BIT (state_hash + np2 + 50-step 'direct' dump at OMP 1/8/24), CUDA G-BIT, host battery 231/231
Tier-3 PV fit: each interfacial cell's PLIC polygon is built once per curvature pass (per-cell cache), never once per target¶
- area: flow
- source: flow
bb95f97(WO-2a), core4806f87(v1.5.0);doc/vof_curvature_cost_design.md§2.3, §5;doc/vof_step_performance_log.md2026-10-09 - decided: 2026-10-08 (design G); landed 2026-10-09
- status: settled
- quote: | The tier-3 PV fit takes each interfacial cell's PLIC polygon from a per-cell cache filled in the planes pass (indexed by a slot map), not rebuilt for every target that uses it. Bitwise.
- rejected: rebuilding the polygon per target (15.6 builds per cell per pass, plicPolygon 50.0 of 81.2 ms of tier 3); a dense per-cell cache over the extended block
- why: harness tier 3 78.8 -> 22.2 ms with WO-2c; G-BIT host znver3 + generic + CUDA (state_hash 12 + np2, 50-step dump, census). Note:
VofCurvature::cache_is sized to listG_'s capacity, which on the structured path equals a dense cache (224 B·(n+4)³, lazy)
Tier-3 PV terms come from per-cell polygon area moments, accumulated entry-parallel on the device (recorded round-off change)¶
- area: flow
- source: flow
17ba0f9(WO-3; commit message has the state_hash table),1ee216d(WO-2b compaction);doc/vof_curvature_cost_design.md§5.4; log 2026-10-09 - decided: 2026-10-09 (USER accepted G-Q1, the round-off change; C1 amendment = coordinator ruling)
- status: settled
- quote: | Per-cell 3-D polygon area moments (PvMoments), transformed into each target's frame, replace the cached polygons; the device team body compacts the neighbours (canonical offset order kept) and sums each of the 27 lower/b entries serially in canonical order on its own lane. The moment build (curvMomentsBuild) and the team body (curvFallbackTeam) are noinline in the DEVICE pass only, so the batched and per-block kernels run one compiled body. Do NOT re-inline them on the device: inlined, nvcc contracts them differently per kernel and batched != per-block.
- rejected: per-target Green's-theorem moments (2.6× slower, FP64-heavy); a centroid point fit; a smaller support; one lane accumulating all 42 entries (FP64-issue-bound); thread per target; a warp reduction; a CUDA tolerance on the batched == per-block gate C1
- why: vs WO-2b, 50-step max rel host 2.71e-14 / CUDA 2.47e-14 (N50 2.52e-9), iterations identical, κ 3.6e-15 / 4.4e-15, census identical; only vof_droplet's hash changes. RTX 5080 tier 3 3.55 -> 0.575 ms, curvature stage 6.19 -> ~3.3 ms (shared GPU). The single-lane vs entry-parallel difference is FMA contraction (bitwise with --fmad=false), not order
- supersedes: the one-lane accumulation of "PV curvature fallback: one team per target cell, canonical-order accumulation" (team shape and canonical order stand)
Height-function columns stay 7 cells; longer (Popinet/TBFsolver-style) column walks are rejected¶
- area: flow
- source:
doc/vof_curvature_cost_design.md§2.6 (gate B sphere, harness replay) - decided: 2026-10-08 (design G)
- status: settled
- quote: | The cells only a long column closes are the steep-slope cells where height functions are inaccurate; fewer tier-3 cells via longer columns is not a saving worth its error.
- rejected: 9- and 11-cell columns to cut tier-3 work
- why: gate B sphere max-error order 1.86 -> 1.50 (9) / 0.92 (11); max error at 64³ 3.8e-3 -> 6.3e-3 / 1.4e-2
The PV normal equations accumulate only their lower triangle and diagonal¶
- area: flow
- source: flow
bb95f97; core4806f87(T3) - decided: 2026-10-08 (design G)
- status: settled
- quote: | curvSolveSym reads nothing above the diagonal, so only the lower triangle and diagonal are accumulated. Bitwise.
- rejected: the full 6×6 accumulation (dead work)
- why: core T3 0 mismatches over 1e5 cases; flow G-BIT host + CUDA