check_decomposition Namespace Reference

Functions
	halvings (d)
	child (args)
	mode_spec (tok, levels)
	main ()

Variables
	HERE = os.path.abspath(__file__)
dict	MODES = {"0": ("aligned", "0"), "aligned": ("aligned", "0"), "legacy": ("aligned", "0")}

Detailed Description

Inspect how a grid + rank count decomposes, and how deep the pressure multigrid gets on it.

Answers, without spending any GPU time, the questions that decide distributed multigrid performance:

  * how balanced is the ORB partition (max block / min block)?
  * is a direction split that must not be (e.g. the wall-normal one in a channel)?
  * how many multigrid levels does the hierarchy ACTUALLY reach, and where does each axis stop?
  * does the coarse-first decomposition help here, and at what depth?

Run it before committing a grid or a rank count to a large job. It needs only an MPI build of
`flow` (the OpenMP one is fine, no GPU), and `mpirun --oversubscribe` happily runs more ranks than
there are cores because nothing here is timed.

    # one grid, several rank counts, both decomposition modes:
    python check_decomposition.py --grid 480,80,160 --levels 6 --np 1,2,4,8,12,16,24 --mode 0,coarse

    # a whole weak-scaling ladder, ORB only (no solver allocation -> any size, instantly):
    python check_decomposition.py --grid 960,160,320 --np 1 --orb-only
    python check_decomposition.py --grid 3072,512,1024 --np 32 --orb-only

    # what the hierarchy looks like level by level:
    python check_decomposition.py --grid 1508,240,503 --levels 5 --np 4 --verbose

`--orb-only` skips constructing a Solver, so it reports the partition (balance, splits) for grids far
too large to allocate on a host — but it cannot report the achieved level count, which needs the real
MG init. Without it, the grid must fit in host memory (a few hundred M cells at most).

Requires PYTHONPATH to point at an MPI-enabled flow build, e.g.
    PYTHONPATH=$PWD/build_mpi_omp python scripts/check_decomposition.py ...

Function Documentation

halvings()

check_decomposition.halvings

(

d

)

How many times an axis can be halved: the multigrid's per-axis depth budget.

Definition at line 41 of file check_decomposition.py.

child()

check_decomposition.child

(

args

)

Definition at line 51 of file check_decomposition.py.

Referenced by main().

mode_spec()

check_decomposition.mode_spec	(		tok,
			levels
	)

'0'/'aligned' -> the legacy aligned ORB; 'coarse'/'coarse-first'/<int> -> coarse-first.

Definition at line 96 of file check_decomposition.py.

Referenced by main().

main()

check_decomposition.main

(

)

Definition at line 105 of file check_decomposition.py.

References child(), main(), and mode_spec().

Referenced by main().

Variable Documentation

HERE

check_decomposition.HERE = os.path.abspath(__file__)

Definition at line 38 of file check_decomposition.py.

MODES

dict check_decomposition.MODES = {"0": ("aligned", "0"), "aligned": ("aligned", "0"), "legacy": ("aligned", "0")}

Definition at line 93 of file check_decomposition.py.