!=============================================================
!
! ███████╗████████╗██████╗ ███████╗ █████╗ ███╗ ███╗███████╗
! ██╔════╝╚══██╔══╝██╔══██╗██╔════╝██╔══██╗████╗ ████║██╔════╝
! ███████╗ ██║ ██████╔╝█████╗ ███████║██╔████╔██║███████╗
! ╚════██║ ██║ ██╔══██╗██╔══╝ ██╔══██║██║╚██╔╝██║╚════██║
! ███████║ ██║ ██║ ██║███████╗██║ ██║██║ ╚═╝ ██║███████║
! ╚══════╝ ╚═╝ ╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝
!
! Supersonic TuRbulEnt Accelerated navier stokes Solver
!
! input file description
!
!=============================================================
STREAmS
performs Direct Numerical Simulations of compressible turbulent flows in Cartesian geometry
solving the fully compressible Navier-Stokes equations. Currently, three canonical wall-bounded flows can be simulated:
- compressible turbulent channel flow
- compressible zero-pressure-gradient turbulent boundary layer
- supersonic oblique shock wave/turbulent boundary-layer interaction
STREAmS can be used both on local cluster and massively parallel HPC architectures.
STREAmS requires (1) a Fortran compiler and (2) an MPI library. For the GPU CUDA version, the PGI compiler is required (tested using PGI 19.4 or more recent compilers). A Makefile with predefined settings is available to facilitate compiling. Different compiling modes can be selected by changing the three variables:
COMPILE
MODE
PREC
COMPILE
currently supports these choices:
pgi-cuda
: PGI compiler Cuda Fortran asynchronous version with MPI library (tested with OpenMPI provided by PGI)pgi-cuda-sync
: PGI compiler Cuda Fortran synchronous version with MPI library (tested with OpenMPI provided by PGI)pgi
: PGI compiler CPU version with OpenMPI library (tested with OpenMPI provided by PGI)intel
: Intel compiler with MPI library (tested with IntelMPI library)gnu
: gnu compiler with MPI library (tested with OpenMPI)ibmxl
: XL IBM compiler with MPI library (tested with IBM MPI library)cray-cuda
: PGI compiler with Cray mpich library without support of CUDA-Aware MPI (currently oriented to Pitz-Daint cluster)
MODE
can be one of:
opt
: optimized compilation, for standard production runsdebug
: debugging compilation, with run-time checks enabled and backtracing when available
PREC
defines the precision of the float numbers:
double
: double precision floating point numberssingle
: single precision floating point numbers
STREAmS can be usually executed using a standard MPI launcher, e.g. mpirun
.
In addition to the executable, in the running folder you need:
input.dat
: file defining the physical and numerical setup of the simulation, to be customized according to the desired needs. Detailed description input.dat file is given below. Some examples of input.dat files are available in theexamples
folder.database_bl.dat
: only required for theboundary layer
andshock-boundary layer interaction
flows (this file does not have to be modified by the user). The file is available in the examples folder.
To run a simulation, type, e.g.:
mpirun -np 8 ./streams
or (for SLURM jobs)
srun ./streams
For CUDA versions in cluster environments, you must distribute MPI processes according to the number of GPUs available for each node. For CINECA Marconi-100 cluster -- 4 GPUS per node -- a possible submission script using 8 GPUs is:
#!/bin/bash
#SBATCH -N 2
#SBATCH --tasks-per-node 4
#SBATCH --cpus-per-task 32
#SBATCH --ntasks-per-socket 2
#SBATCH --mem=128G
#SBATCH --partition=m100_usr_prod
#SBATCH --time=00:30:00
#SBATCH --gres=gpu:4
module load profile/global hpc-sdk/20.5--binary
mpirun -np 8 ./streams
For CSCS Pitz-Daint cluster -- 1 GPU per node -- a possible submission script using 8 GPUs is:
#!/bin/bash
#SBATCH -N 8
#SBATCH --tasks-per-node 1
#SBATCH --mem=15G
#SBATCH --partition=debug
#SBATCH --time=00:30:00
#SBATCH --gres=gpu:1
#SBATCH --constraint=gpu
module swap PrgEnv-cray PrgEnv-pgi
srun ./streams
iflow
defines the type of flow. 0 = channel, 1 = boundary layer, 2 = shock/boundary-layer interaction
rlx rly rlz
real numbers defining the size of the Cartesian domain along the three directions
(x=streamwise, y=wall-normal, z=spanwise)
nxmax nymax nzmax
integer numbers defining the number of grid nodes along each direction
nymaxwr rlywr dyp_target
specify the wall-normal mesh features. dyp_target is the desired spacing at the wall in inner units, .
nymaxwr denotes the number of grid points in the wall-resolved region, ranging from y=0 (wall) up to y=rlywr, where a sinh mapping
is applied. Both nymaxwr and rlywr can be specified when iflow = 1 (turbulent boundary layer), then a geometric progression is applied from
y=rlywr up to y = rly. For iflow = 2 (shock/boundary-layer interaction), nymaxwr must be specified but rlywr is automatically computed.
nblocks(1) nblocks(3)
define the MPI decomposition along x (streamwise) and z (spanwise). These numbers
must be consistent to nxmax, nymax, and nzmax. In particular the following divisions must have zero remainder:
nxmax/nblocks(1), nzmax/nzmax. Moreover, for iflow>0 cases also nxmax/nblocks(3) and nymax/nblocks(1)
must have zero remainder. The product nblocks(1)*nblocks(3) must equal the total number of launched MPI processes.
tresduc ximp deflec
is the WENO threshold (WENO is active if the shock sensor value exceeds tresduc),
ximp is the shock abscissa and deflec is the shock angle. ximp and deflec are used only if iflow > 0.
idiski ncyc cfl nstep nprint io_type
idiski selects the start type (0=init, 1=restart, 2=collect statistics at runtime), ncyc the total number
of iterations, cfl is the CFL number. The time step is re-evaluated every nstep iterations, nprint
the residual file is printed every nprint iterations and io_type is selects the type of I/O (io_type=0 no I/O, io_type=1, serial I/O, io_type=2, MPI I/O).
rm re (friction) twall/taw visc_type s2tinf
Mach number, number, ratio between wall temperature and
adiabatic wall temperature, type of viscosity law (visc_type=1 power law, visc_type=2, Sutherland law) and ratio between
S(110.4K) and reference temperature. If s2tinf=0 s2tinf is automatically computed by the solver.
istat nstat
cumulative flow statistics are evaluate every istat iterations. nstat is the number of streamwise stations
at which flow statistics are extracted. nstat is meaningful for boundary layer flow (iflow > 0)
xstat
streamwise locations of boundary layer flow statistics (iflow > 0)
20. 30. 40. 50. 60. 70. 80. 87.
dtsave dtsave_restart plot3d vtk
dt_save is the time interval between field output, dtsave_restart is the time interval
between output of restart files, enable_plot3d>0 activates the plot3d format output,
vtk>0 activates vtk format output.
irand
if < 0 produce not reproducible random sequences based on current time, if >=0 produces random sequences which
are reproducible across different runs with the same configuration
Typical input files for canonical flow cases are available.
Plot3d binary output files -- e.g. plot3dgrid.xyz and field_0001.q -- can be read using Tecplot or Paraview
VTK Rectilinear grid files -- e.g. field_0001.vtr -- can be read using Paraview.
Other statistics files are automatically produced according to the input configuration.
When running channel flow cases (iflow
=0) the residuals file (output_streams.dat) contains seven columns:
- number of cycles,
- elapsed time,
- streamwise momentum residual,
- pressure gradient,
- bulk density (conserved to machine accuracy),
- bulk flow velocity (conserved to machine accuracy),
- bulk temperature. For instance, plotting the fourth column vs. the second allows the user to check the time history of the pressure gradient.
The file `channstat.prof` contains the mean channel flow statistics.
This file is printed at the end of each run and it contains the mean flow statistics averaged
in the homogeneous spatial directions and in time (statistics are progressively updated in time at each restart if idiski=2, or collected from scratch if idisk=1).
The file `channstat.prof` contains 15 columns:
is the wall-normal coordinate, normalized with the channel half width
the wall-normal coordinate in viscous units
the wall-normal coordinate transformed according to Trettel & Larsson in viscous units
the mean streamwise velocity averaged according to Favre, normalized with the bulk flow velocity
the mean streamwise velocity averaged according to Favre, normalized with the friction velocity
the mean streamwise velocity transformed according to van Driest, normalized with the friction velocity
the mean streamwise velocity transformed according to Trettel & Larsson in viscous units
the mean density profile, normalized with the mean wall density
the Favre streamwise Reynolds stress, normalized with the wall-shear stress
the Favre wall-normal Reynolds stress, normalized with the wall-shear stress
the Favre spanwise Reynolds stress, normalized with the wall-shear stress
the Favre shear Reynolds stress, normalized with the wall-shear stress
the mean temperature profile, normalized with the wall temperature
the density fluctuations normalized with the mean wall density
The temperature fluctuations, normalized with the wall temperature
When running boundary layer cases (iflow
=1 or 2) the residuals file (output_streams.dat) contains three columns:
- number of cycles,
- elapsed time,
- streamwise momentum residual,
The files cf_xxx.dat are ASCII files and the number xxx refers to the Cartesian streamwise MPI block to which the file belong. These files is printed at the end of each run. Statistics are progressively updated in time at each restart if idiski=2, or collected from scratch if idisk=1. The file contains 13 columns:
streamwise coordinate normalized with the inlet boundary layer thickness
skin-friction coefficient
friction Reynolds number
Compressible shape factor
Incompressible shape factor
boundary layer thickness, normalized with the inlet boundary layer thickness
Compressible displacement thickness
Compressible momentum thickness
friction velocity
- Reynolds number based on the incompressible momentum thickness
- Incompressible friction coefficient, according to van Driest II transofrmation
- Reynolds number based on the compressible momentum thickness
- Mean pressure rms at the wall normalized with the wall-shear stress
The files stat_nnn.dat are ASCII containing the mean boundary layer statistics. The files are printed at the end of each run. Statistics are progressively updated in time at each restart if idiski=2, or collected from scratch if idisk=1. The number nnn indicates the global mesh index in the streamwise direction at which statistics are printed. The files contains 10 columns:
, wall-normal coordinate, normalized with the local boundary layer thickness
wall-normal coordinate, normalized with the viscous lenght scale
the Favre averaged streamwise velocity, normalized with the friction velocity
the streamwise velocity transformed according to van Driest
the density scaled streamwise velocity rms
the density scaled wall-normal velocity rms, in viscous units
the density scaled spanwise velocity rms, in viscous units
the density scaled Reynolds shear stress, in viscous units
the square root of the mean density, normalized with the wall density
- The pressure rms, normalized with the square root of the wall-shear stress.