kaschau / peregrine Goto Github PK

Should we be protecting against non-physical boundary conditions extrapolations?

i.e.

negative pressure, rho, T, Y, etc.

Maybe even summing species mass fractions to one?

If we have a single block with no communication, just 6 walls, and we enter consistify routine with only good values in the interior, the first thing that happens is communication of Q, but since we just walls, nothing happens.

The next thing we do is update the EOS for the ENTIRE block, including the potentially bogus halos for the walls, which havent been touched.

This is likely only a problem at startup or user initialization in which they may not have populated the halos with anything.

so we cant accidentally create new attributes

https://stackoverflow.com/questions/472000/usage-of-slots

metrics have divide by zeros and other invalid values being reported

See

https://arxiv.org/pdf/1712.00953.pdf

for formulation

We need to set the consistify and RHS operators in a generic manner so that neither of those two routines call specific function, but instead call generic function names that are defined in the
generate_multiblock_solver

routine. We already do this with eos, chem, but need to do it with ALL operators and routines for extend-ability. i.e. transport, dqdxyz, even diffusive and advective fluxes. Even though we only have one option for now.

If we class out consistify and RHS as can also clean up the generate_multiblock_solver routine and move the setting of all the routines to those class init()s.

We do not catch random settings for bcType, i.e. accidentally setting

bcType = "bq" instead of = "b1"

There should be a check somewhere to determine if bcType is not a known type.

See

Efficient time-stepping techniques for simulating turbulent reactive flows with stiff chemistry,
Computer Physics Communications, Volume 243, 2019, Pages 81-96,
https://doi.org/10.1016/j.cpc.2019.04.016.

The time integration in strang time stepping is causing kokkos view to hang around after finalize is called.

we need to convert the input of ints from hdf5 to python ints so yaml doesnt complain about it.

ls = []

should be replace by ls = None followed by a check if ls is None

fdOrder=2 is the default for computeMetrics() and I dont think its ever explicitly set. Like in bootstrapCase

Implement a Rosenbrock type method for chemistry integration.

Advantages: Single step, non iterative. "Looks like" RK schemes.

Disadvantages (/ really advantages) : Requires jacobian.

See

https://encyclopediaofmath.org/wiki/Rosenbrock_methods
http://www.hysafe.org/science/eAcademy/docs/ACMTransactionsOnMathematicalSoftware_v8_p93to113.pdf
https://www.sciencedirect.com/science/article/pii/S0377042715001569
https://www.epfl.ch/labs/anchp/wp-content/uploads/2018/05/part2-1.pdf

Then further implemented with the simpler operator splitting detailed in #32

Could be a massive speedup, and would be done in Kokkos spaces rather than on the python side.

adding clipping to fix.

to ensure all compute modules work, every boundary condition should set ALL primitives in ALL the cells on a face.

How do we simulate 1d or 2d simulations with periodic boundary conditions for ng > 1

mCoding says to use time.perf_counter. This is the way.

The MPI communication is causing the program to leak memory.

The grid and restart writers are reporting incorrect extents in paraview.

i and k are reversed.

Starting with 2.6+:

DeprecationWarning: GasKinetics::getFwdRateConstants: Behavior to change after Cantera 2.6;
results will no longer include third-body concentrations for three-body reactions.

see metrics.py in particular

need to divide by MW to go from moles to mass.

see

https://github.com/kaschau/PEREGRINE/blob/main/src/compute/thermo/tpg.cpp

Also using incorrect check for zeroth rank

This will make doing things like bc profiles, time dependent bcs, SEM much easier in the future.

should be changed to newtons method.

The communications buffers should be kokkos views?

It seems like the

from mpi4py import MPI

in mpiUtils is initializing MPI on peregrinepy module import (say in ipython)

This is not desirable as it makes a functional MPI a prereq just to use peregrinepy, even in preprocessing.

On HPC cluster head nodes, the MPI may not be ready to go, only on compute nodes or with an appropriate mpiexec like command.

So we need to NOT initialize upon import of MPI on the module level.

This means we need to initialize elsewhere, a good place seems to be in runPeregrine.py (just my importing MPI before importing peregrinepy.

This will in turn mess up all pytests.

There seems to be a really good pytest tool https://github.com/NOhs/pytest-easyMPI

that makes running pytest in parallel next to serial very easy. It slows it way down, but makes peregrinepy much more portable and light in terms of dependency on import.

The xmf portion of the grid/restart writer routines do not make sense in parallel when the mb container does not have ALL the blocks.

Must update the periodic communication to match fix from #10

setting the cmake option KOKKOS_BACKEND sets the ifdef, but that ifdef does not make it to the kokkos_type header file to set what packend to use at compile time.

i.e, setting Serial, or OpenMP does not set the proper variables in

the npOrKokkos method doesn't make sense on a GPU as we are wrapping the blk.array around a GPU kokkos view not a host view. so any attempt to access this blk.array will results in a segfault.

need to move checkNan to kokkos

Currently there are several issues when compiling using cuda.

1.) There are variable sized arrays that gfortran is allowing, but CUDA does not. see eos and transport computes for things like

Y[ns]

2.) Seems like the does not like accessing the thtrdat struct data, in particular std::vectors, in device code. So we need to make the thtrdat_ attributes KokkosViews.

Should look into cleaning that up

Generate halos is still taking in config as an argument.

I say that only solver blocks should have access to halos of any sort.

So we will specify a ng for a solver mb creation, but anything upstream just gets the regular old grid. This will make scripting restarts much cleaner.

Particularly bc we have bcFams for boundary conditions and there is no need to noOp a boundary condition. We can just specify it trivially. So never need to set halos in a restart and just let it be.

We can also clean up all init(XXXX)Arrays routines as well by setting a temporary ng variable.

in consistify, we are only every updating derived arrays, (qh, etc) for exterior boundary conditions. but in interior halos we never touch them. so derived arrays like qh are never set.

should use the solver object function to iterate over blocks

Change test cases with kokkos arrays to something like

def runit():
    ...
    #actually perform test
    ...

def test_xxx():
    import kokkos
    kokkos.initialize()
    runit()
    kokkos.finalize()

bc the max velocity is zero, the returned factor is zero, and you get dt/0.0

metrics should be moved to the python side to give access in restart/grid files