This is the companion repository for the talk
Controlling parallel simulations in Julia from C/C++/Fortran programs with libtrixi
Michael Schlottke-Lakemper, Benedict Geihe
University of Würzburg, HS3, 6th March 2024, 11:30 AM
(see abstract below). The talk is part of the session Cross-Platform Development with C/C++ at deRSE24 - Conference for Research Software Engineering in Germany, which takes place in Würzburg, Germany from 5th to 7th March 2024.
Besides the presentations slides (talk-2024-deRSE-libtrixi.pdf), this repository contains instructions and files that allow one to reproduce (some of) the results in the talk.
The Julia programming language aims to provide a modern approach to scientific high-performance computing by combining a high-level, accessible syntax with the runtime efficiency of traditional compiled languages. Due to its native ability to call C and Fortran functions, Julia often acts as a glue code in multi-language projects, enabling the reuse of existing libraries implemented in C/Fortran. With the software library libtrixi, we reverse this workflow: It allows one to control Trixi.jl, a complex Julia package for parallel, adaptive numerical simulations, from a main program written in C/C++/Fortran. In this talk, we will present the overall design of libtrixi, show some of the challenges we had to overcome, and discuss continuing limitations. Furthermore, we will provide some insights into the Julia C API and into the PackageCompiler.jl project for static compilation of Julia code. Besides the implications for our specific use case, these experiences can serve as a foundation for other projects that aim to integrate Julia-based libraries into existing code environments, opening up new avenues for sustainable software workflows.
The instructions are written based on a x64 machine with Ubuntu Linux 22.04, on which the results used in the talk were obtained as well.
The following standard software packages need to be made available locally before installing libtrixi:
- C compiler with support for C11 or later (e.g., GCC or Clang) (we used GCC v10.5)
- Fortran compiler with support for Fortran 2018 or later (e.g., Gfortran) (we used Gfortran v10.5)
- CMake (we used cmake v3.16.3)
- MPI (e.g., Open MPI or MPICH) (we used Open MPI v3.1)
- HDF5 (we used hdf5-openmpi v1.10)
- Julia v1.10
(it is advised to use tarballs from the official website or the
juliaupmanager, we used v1.10.0) - Paraview for visualization (we used v5.11.2)
The following software packages require manual compilation and installation. To simplify
the installation instructions, we assume that the environment variable WORKDIR was set to the
directory, where all the code is downloaded to and compiled, and PREFIX was set to an installation
prefix. For example,
export WORKDIR=$HOME/repro-talk-2024-deRSE-libtrixi
export PREFIX=$WORKDIR/installt8code is a meshing library which is used in Trixi.jl. The source code can be obtained from the official
website. Here is an example for compiling and installing it (we used v1.6.1):
cd $WORKDIR
git clone --branch 'v1.6.1' --depth 1 https://github.com/DLR-AMR/t8code.git
cd t8code
git submodule init
git submodule update
./bootstrap
./configure --prefix="${PREFIX}/t8code" \
CFLAGS="-Wall -O0 -g" \
CXXFLAGS="-Wall -O0 -g" \
--enable-mpi \
--enable-debug \
CXX=mpicxx \
CC=mpicc \
--enable-static \
--without-blas \
--without-lapack
make
make installHere is an example for compiling and installing libtrixi (we used v0.1.4):
cd $WORKDIR
git clone --branch 'v0.1.5' --depth 1 https://github.com/trixi-framework/libtrixi.git
cd libtrixi
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=${PREFIX}/libtrixi ..
make
make installA separate Julia project for use with libtrixi has to be set up:
mkdir ${PREFIX}/libtrixi-julia
cd ${PREFIX}/libtrixi-julia
${PREFIX}/libtrixi/bin/libtrixi-init-julia \
--t8code-library ${PREFIX}/t8code/lib/libt8.so \
--hdf5-library /usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.so \
${PREFIX}/libtrixiTo exactly reproduce the results presented in the talk, the provided Manifest.toml can be used to install the same package versions for all Julia dependencies:
cp Manifest.toml ${PREFIX}/libtrixi-julia/
cd ${PREFIX}/libtrixi-julia
JULIA_DEPOT_PATH=./julia-depot \
julia --project=. -e 'import Pkg; Pkg.instantiate()'Trixi2Vtk needs to be installed for later post processing (we used v0.3.12):
cd ${PREFIX}/libtrixi-julia
JULIA_DEPOT_PATH=./julia-depot \
julia --project=. -e 'import Pkg; Pkg.add(name="Trixi2Vtk", version="0.3.12")'cd ${PREFIX}/libtrixi
mpirun -n 2 ./bin/trixi_controller_simple_f \
${PREFIX}/libtrixi-julia \
${PREFIX}/libtrixi/share/libtrixi/LibTrixi.jl/examples/libelixir_tree2d_warm_bubble.jlOutput files will be written to ${PREFIX}/libtrixi/out_bubble and need to be post processed:
cd ${PREFIX}/libtrixi/out_bubble
JULIA_DEPOT_PATH=${PREFIX}/libtrixi-julia/julia-depot \
julia --project=${PREFIX}/libtrixi-julia -e 'using Trixi2Vtk; trixi2vtk("solution*.h5")'The results can now be viewed using, e.g, paraview. We used Tpotpert_contour.pvsm and Tpotpert.pvsm to generate our results.
cd ${PREFIX}/libtrixi
mpirun -n 12 ./bin/trixi_controller_simple_f \
${PREFIX}/libtrixi-julia \
${PREFIX}/libtrixi/share/libtrixi/LibTrixi.jl/examples/libelixir_p4est3d_euler_baroclinic_instability.jlOutput files will be written to ${PREFIX}/libtrixi/out_baroclinic and need to be post processed:
cd ${PREFIX}/libtrixi/out_bubble
JULIA_DEPOT_PATH=${PREFIX}/libtrixi-julia/julia-depot \
julia --project=${PREFIX}/libtrixi-julia -e 'using Trixi2Vtk; trixi2vtk("solution*.h5")'The results can now be viewed using, e.g, paraview. We used pressure.pvsm to generate our results.
This repository was initiated by Benedict Geihe and Michael Schlottke-Lakemper.
The contents of this repository are licensed under the MIT license (see LICENSE.md).