cwpearson / astaroth

Fork of https://bitbucket.org/jpekkila/astaroth

Astaroth - A Multi-GPU Library for Generic Stencil Computations {#mainpage}

Specification | Contributing | Licence | Repository | Issue Tracker | Wiki

Astaroth is a multi-GPU library for three-dimensional stencil computations. It is designed especially for performing high-order stencil computations in structured grids, where several coupled fields are updated each time step. Astaroth consists of a multi-GPU and single-GPU APIs and provides a domain-specific language for translating high-level descriptions of stencil computations into efficient GPU code. This makes Astaroth especially suitable for multiphysics simulations.

Astaroth is licenced under the terms of the GNU General Public Licence, version 3, or later (see LICENCE.txt). For contributing guidelines, see Contributing.

System Requirements

• An NVIDIA GPU with support for compute capability 3.0 or higher (Kepler architecture or newer)

Dependencies

Relative recent versions of

gcc cmake cuda flex bison.

Building

In the base directory, run

1. mkdir build
2. cd build
3. cmake ..
4. make -j

Optional: Documentation can be generated by running doxygen in the base directory. Generated documentation can be found in doc/doxygen.

Tip: The library is configured by passing options to CMake with -D[option]=[ON|OFF]. For example, double precision can be enabled by calling cmake -DBUILD_DOUBLE_PRECISION=ON ... See CMakeLists.txt for an up-to-date list of options.

Note: CMake will inform you if there are missing dependencies.

CMake Options

Option	Description	Default
CMAKE_BUILD_TYPE	Selects the build type. Possible values: Debug, Release, RelWithDebInfo, MinSizeRel. See (CMake documentation)[https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html] for more details.	Release
CUDA_ARCHITECTURES	Selects CUDA architecture support. Multiple architectures delimited by ;. See (CMake documentation)[https://cmake.org/cmake/help/latest/prop_tgt/CUDA_ARCHITECTURES.html] for more details.	"60;70"
DOUBLE_PRECISION	Generates double precision code.	OFF
BUILD_SAMPLES	Builds projects in samples subdirectory.	ON
MPI_ENABLED	Enables acGrid functions for carrying out computations with MPI.	OFF
MULTIGPU_ENABLED	Enables Astaroth to use multiple GPUs on a single node. Uses peer-to-peer communication instead of MPI. Affects Legacy & Node layers only.	ON
DSL_MODULE_DIR	Defines the directory to be scanned when looking for DSL files.	astaroth/acc/mhd_solver
VERBOSE	Enables various non-critical warning and status messages.	OFF

Standalone Module

Usage: ./ac_run [options]
	     --help | -h: Prints this help.
	     --test | -t: Runs autotests.
	--benchmark | -b: Runs benchmarks.
	 --simulate | -s: Runs the simulation.
	   --render | -r: Runs the real-time renderer.
	   --config | -c: Uses the config file given after this flag instead of the default.

See analysis/python/ directory of existing data visualization and analysis scripts.

Interface

• astaroth/include/astaroth.h: Astaroth main header. Contains the interface for accessing single- and multi-GPU layers.
• astaroth/include/astaroth_utils.h: Utility library header. Provides functions for performing common tasks on host, such as allocating and verifying meshes.
• <build directory>/astaroth.f90: Fortran interface to Astaroth. Generated when building the library.

FAQ

Can I use the code even if I don't make my changes public?

GPL requires only that if you release a binary based on Astaroth to the public, then you should also release the source code for it. In private you can do whatever you want (secret forks, secret collaborations, etc). Astaroth Code source files (.ac, .h) do not belong to the library and therefore are not licenced under GPL. The user who created the files holds copyright over them and can choose to distribute them under any licence.

How do I compile with MPI support?

Ensure that your MPI implementation has been built with CUDA support and invoke CMake with cmake -DMPI_ENABLED=ON -DBUILD_SAMPLES=ON ... Otherwise the build steps are the same. Assign exactly one process per GPU and run with, for example, srun --gres=gpu:v100:<ngpus per node> --ntasks-per-socket=<ngpus per node / NICs per node> -n <total number of gpus> -N <number of nodes> ./mpitest or for mpirun the command mpirun -np <ngpus per node> ./mpitest works (at least on a GPU node).

How do I contribute?

See Contributing.