Astaroth - A Multi-GPU library for generic stencil computations

Astaroth is a single-node multi-GPU library for multiphysics and other problems, which involve stencil computations in a discrete mesh. It's licenced under the terms of the GNU General Public Licence, version 3, or later (see LICENCE.txt). Astaroth ships with a domain-specific language, that can be used to translate high-level representation of the stencil computations into a heavily inlined GPU pipeline.

System requirements

NVIDIA GPU with >= 3.0 compute capability. See https://en.wikipedia.org/wiki/CUDA#GPUs_supported.

Building (3rd party libraries)

1. cd 3rdparty
2. ./setup_dependencies.sh Note: this may take some time.

Building

There are two ways to build the code as instructed below.

If you encounter issues, recheck that the 3rd party libraries were successfully built during the previous step.

Method I: In the code directory

1. cd build/
2. cmake -DDOUBLE_PRECISION=OFF -DBUILD_DEBUG=OFF .. (Use cmake -D CMAKE_C_COMPILER=icc -D CMAKE_CXX_COMPILER=icpc -DDOUBLE_PRECISION=OFF -DBUILD_DEBUG=OFF .. if compiling on TIARA)
3. ../scripts/compile_acc.sh && make -j
4. ./ac_run <options>

Edit config/astaroth.conf to change the numerical setup.

Method II: With a script in a custom build directory (RECOMMENDED)

1. source sourceme.sh to add relevant directories to the PATH
2. ac_mkbuilddir.sh -b my_build_dir/ to set up a custom build directory. There are also other options available. See ac_mkbuilddir.sh -h for more.
3. $AC_HOME/scripts/compile_acc.sh to generate kernels from the Domain Specific Language
4. cd my_build_dir/
5. make -j
6. ./ac_run <options>

Edit my_build_dir/astaroth.conf to change the numerical setup.

Available options

• -s simulation
• -b benchmark
• -t automated test

By default, the program does a real-time visualization of the simulation domain. The camera and the initial conditions can be controller by arrow keys, pgup, pgdown and spacebar.

Visualization

See analysis/python/ directory of existing data visualization and analysis scripts.

Generating documentation

Run doxygen doxyfile in astaroth_2.0 directory. The generated files can be found in doc/doxygen. The main page of the documentation will be at dox/doxygen/astaroth_doc_html/index.html.

Formatting

If you have clang-format, you may run scripts/fix_style.sh. This script will recursively fix style of all the source files down from the current working directory. The script will ask for a confirmation before making any changes.

Directory structure

Coding style.

In a nutshell

• Use K&R indentation style and 4 space tabs.
• Line width is 100 characters
• Start function names after a linebreak in source files.
• Be generous with const type qualifiers.
• When in doubt, see Google C++ Style Guide.

Header example:

// Licence notice and doxygen description here
#pragma once
#include "avoid_including_headers_here.h"

/** Doxygen comments */
void globalFunction(void);

Source example:

#include "parent_header.h"

#include <standard_library_headers.h>

#include "other_headers.h"
#include "more_headers.h"

typedef struct {
	int data;
} SomeStruct;

static inline int small_function(const SomeStruct& stuff) { return stuff.data; }

// Pass constant structs always by reference (&) and use const type qualifier.
// Modified structs are always passed as pointers (*), never as references.
// Constant parameters should be on the left-hand side, while non-consts go to the right.
static void
local_function(const SomeStruct& constant_struct, SomeStruct* modified_struct)
{
	modified_struct->data = constant_struct.data;
}

void
globalFunction(void)
{
	return;
}

TIARA cluster compilation notes

Modules used when compiling the code on TIARA cluster.

• intel/2016
• hdf5/1.8.16_openmpi_1.10.2_ic16.0
• cmake/3.9.5
• openmpi/1.10.2_ic16.0
• gcc/5.3.0
• cuda/9.0