Added Astaroth 2.0

This commit is contained in:
jpekkila
2019-06-14 14:18:35 +03:00
parent 4e4f84c8ff
commit 0e48766a68
87 changed files with 18058 additions and 1 deletions

4
doc/doxygen/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this file
!.gitignore

131
doc/manual/manual.md Normal file
View File

@@ -0,0 +1,131 @@
*Miikka Vaisala: This is just something I have astarted to write up to make sense about the Astaroth 2.0. Starting for personally important notes to understand the code. Will be refined as my understanding improves.*
#Astaroth manual
## Compilation
See the `README.md`. At the moment, let us keep certaint things in one place.
## Simulation instructions
At the moment it is only possible to build and run in the `astaroth_2.0/build/` directory. Possibility to add separate run directories will be included later.
### Choosing physics
Runtime settings can be adjusted from `astaroth_2.0/include/astaroth.h` and `astaroth_2.0/config/astaroth.conf`.
Howeve, physics switches LENTROPY, LFORCING etc. do not work at the moment. There has been an issue to get pre-processor combatible with astaroth-domain-specific language in Astaroth 2.0. Therefore, all features are online by default.
To get the switcher working now, rename `astaroth_2.0/src/core/kernels/rk3handtuned.cuh` -> `rk3.cuh`. (**MV:** Not yet tested.)
How to use?
What kind of runtime settings?
### Setting initial conditions
Where can we effectively choose the initial condition?
### Launchin a run
`./ac_run -s` assuming you are doing a normal simulation. Basic code for this invocation can be found in the source file `astaroth_2.0/src/standalone/simulation.cc`.
Please note that launching `./ac_run -t` will *fail if entropy and forcing are in use*. Test is mainly for finding paralleization bugs. (In principle if hydro stuff and induction work, so will forcing and entropy.)
### Diagnostic variables
What is calculated?
Where it is saved?
### Simulation data
Saving output binaries is not enabled yet.
**MV:** I am planning to implement HDF5 format for the data. **TOP PRIORITY**.
#### Notes about data structures
- Configuration parameters have prefix `AC_`, such as `AC_dsx`.
- All configurations are stored in the struct `AcMeshInfo`, containing tables `int_params` ja `real_params`. **NOTE:** `int_params` and `real_params` require diligence. If you call e.g. `int_params[AC_dsx]`, the result will be something unexpected. So-far error checking with this has now been possible to be automated.
- All mesh data is stored to the struct `AcMesh`, containing both configuration values and vertex data (`lnrho`, `uux`, etc.)
- All essential tructs, macros and enumerators are found in astaroth.h for better reference.
- In the case there is changes in the data layout, better use macro `AC_VTXBUF_IDX(i, j, k, mesh_info)`which transform indices from 3D to 1D. Therefore no need to start writing `i + j * mesh_info.int_params[AC_mx] + ...` which would affect the code readability.
- AcReal on generic floating point real number type used everywhere in the code. Currently can be either `float` or `double`. Possibly in the future also `half` or `long double` could become available.
Sample code:
```cpp
AcMeshInfo mesh_info;
// Loads data from astaroth.conf into the AcMeshInfo struct
load_config(&mesh_info);
// Allocates data on the host for the AcMesh struct using information found in mesh_info.
AcMesh* mesh = acmesh_create(mesh_info);
// Initializes mesh to InitType (specified in standalone/model/host_memory.h)
acmesh_init_to(INIT_TYPE_GAUSSIAN_RADIAL_EXPL, mesh);
// Allocates data on the device for the AcMesh struct
acInit(mesh_info);
acLoad(*mesh); // Loads the mesh to the device
const AcReal dt = 1.f;
// Synchronizes previous device commands
acSynchronize();
// Does a full rk3 integration step on the device
acIntegrate(dt);
acSynchronize();
// Store data from device to host mesh
acStore(mesh);
printf("nx: %d, dsx %f\n",
mesh->info.int_params[AC_nx],
double(mesh->info.real_params[AC_dsx]));
printf("First vertex of the computational domain: %f\n",
double(mesh->vertex_buffer[VTXBUF_LNRHO][AC_VTXBUF_IDX(3, 3, 3, mesh_info)]));
```
### Reading data
Depends on the output format. With HDF5 should be simple enough.
[Jupyter notebook](http://jupyter.org/) visualization?
Do we want to use [YT?](https://yt-project.org/)
### Live rendering
MV: Cool, but does not work for remote cluster so far. A GPU workstation is required.
##Multi-GPU
At the moment multi-GPU is not included in Astaroth 2.0. However, it has been implemented 1.0 (`astaroth_1.0/src/gpu/cuda/cuda_generic.cu`) could be essentially ported by copypasting to `astaroth_2.0/src/core/astaroth.cu` after we have clear idea how to run things with single GPU. Could be done overnight in principle.
## Profiling
The built-in beachmark is currently unreliable due to an unknown reason. Please use [nvprof and nvvp](https://docs.nvidia.com/cuda/profiler-users-guide/index.html) for precise profiling. Also, NVIDIA suggests their [Nsight Systems](https://developer.nvidia.com/nsight-systems).
## ETC
**Note** `auto_optimize.sh` does not currently work, but it aims to tune thread block dimensions automatically.