LAMMPS is a popular Molecular Dyanmics (MD) library written in C++. This repository integrates the SmartRedis C++ client into a fork (hopefully soon to be merged) of LAMMPS for online data analysis and visualization.
An example of the interactive LAMMPS visualization with ipyvolume:
The melt/ directory demonstrates how to use SmartSim to stream
atom data, in the form of DataSet objects, to a running Python process
for data visualization.
The SmartRedis client is embedded as a atom "dump style" within LAMMPS. The following line can be added to a LAMMPS input file to stream atom data to the database at a given timestep interval.
dump smart_sim all atom/smartsim 100 atoms
The source code for the integration is located in the LAMMPS submodule as a
part of this repository in lammps/src/SMARTSIM.
This demonstration uses MiniConda. Although it is not strictly required, we recommend its use to simplify the tracking of different environments.
MiniConda can be installed via wget of an appropriate installation script, such as one from
https://docs.conda.io/en/latest/miniconda.html#linux-installers
Theta users: on Theta, MiniConda is available through the module system and can be activated via the command
module load miniconda-3
We recommend creating a separate environment for SmartSim and lammps
conda create --name=smartsim-lammps python=3.8.5
conda activate smartsim-lammpsSome systems (e.g. Theta) may have a default version of CMake which is insufficient for building the SmartRedis library (v3.13 or later is required). It is recommended that CMake is installed via conda:
conda install cmakeTo get started, clone the smartsim-lamps repository
git clone --recursive https://github.com/CrayLabs/smartsim-lammpsYou will also need the SmartRedis repository. The lammps build step later assumes that SmartRedis is in a directory parallel to smartsim-lammps named SmartRedis.
git clone https://github.com/CrayLabs/SmartRedis --branch v0.3.1
cd SmartRedis
make libBefore installing additional components, Theta users must run the following commands to configure the runtime environment.
module purge
module load PrgEnv-cray
module load cray-mpich
module unload atp perftools-base cray-libsci
export CRAYPE_LINK_TYPE=dynamicSmartSim requires git-lfs. Install and configure it via the following commands
conda install git-lfs
git-lfs installThe following requirements will need to be installed in a Python environment on your system.
smartsim==0.4.1
smartredis==0.3.1
ipyvolume==0.5.2
Install all of these requirements by running
cd smartsim-lammps/melt
pip install -r requirements.txt
smart build --device cpuNext, build LAMMPS. Theta (XC users in general) users be sure to have the above modules loaded and environment setup.
cd smartsim-lammps/lammps/cmake
mkdir build
cd build
CC=cc CXX=CC cmake .. -DBUILD_MPI=yes -DPKG_SMARTSIM=yes
make -j 4 # or higher if you have more procs
export PATH=$(pwd):$PATHThe example has two batch scripts that can submit to either Cobalt (Theta) or Slurm systems in general. These can be easily adapted to other WLMs.
In each batch script are a number of parameters that configure how LAMMPS and the analysis will be executed.
launcher=cobalt # launcher for the run (slurm or cobalt)
simnodes=124 # number of nodes for LAMMPS (mind resources listed above)
simppn=64 # procs per node for LAMMPS
simsteps=10000 # number of steps for LAMMPS
simscale=4 # scale factor for LAMMPS (max = 16)
dbnodes=3 # number of DB nodes (if 3 or greater, you must change cluster flag in analysis script)
dbport=6780 # port for the DB
vis_workers=64 # number of workers to pull data for the visualization (max = nproc on single node)SmartSim will take care of writing the LAMMPS parameters into the configuration file at runtime as well as launching both lammps, the database, and the analysis out onto the system with the specified resource requirements.
The user should make sure to use their own Python environment in the batch script.
If a single database node is used, LAMMPS will need to be recompiled with the client cluster flag set to false in the initializer. By default, the case is setup to run with a 3 node database cluster.
To run the example on Theta use the following command (be sure to edit the script to point to your conda environment)
qsub run-theta.shThe point of this example is to show how SmartSim can be used to perform in-transit data analysis, however, the example can be modified to include any Python code in the analysis script.
Currently, the analysis script generates snapshots of the atom domain at every 100 iterations. An animation is also generated that shows the snapshots evolving through the integration of the simulation.