Tutorial on Applying Reinforcement Learning to the Particle Accelerator ARES

You can view the tutorial notebook as HTML slides here.

Download the repository

Get the repository with Git

You will need to have Git previously installed in your computer. To check if you have it installed, open your terminal and type:

git --version

Git installation in macOS

brew update
brew install git

Git installation in Linux

In Ubuntu/Debian

sudo apt install git

In CentOS

sudo yum install git

Downloading the repository

Once you have Git installed open your terminal, go to your desired directory, and type:

git clone https://github.com/RL4AA/rl-tutorial-ares-basic.git

Then enter the downloaded repository:

cd rl-tutorial-ares-basic

Get the repository with direct download

Open your terminal, go to your desired directory, and type:

wget https://github.com/RL4AA/rl-tutorial-ares-basic/archive/refs/heads/main.zip
unzip main.zip
cd rl-tutorial-ares-basic

Getting started

You need to install the dependencies before running the notebooks.

Install ffmpeg

Please also run these commands to install ffmpeg:

• OS X: brew install ffmpeg
• Ubuntu: sudo apt-get install ffmpeg

Using conda

If you don't have conda installed already and want to use conda for environment management, you can install the miniconda as described here.

• Create a conda env from the provided env file conda env create -f environment.yml
• Activate the environment with conda activate rl-icfa
• Additional installation steps:

python -m jupyter contrib nbextension install --user
python -m jupyter nbextension enable varInspector/main

• After the tutorial you can remove your environment with conda remove -n rl-icfa --all

Using venv only

If you do not have conda installed:

Alternatively, you can create the virtual env with venv in the standard library

python -m venv rl-icfa

and activate the env with $ source /bin/activate (bash) or C:> /Scripts/activate.bat (Windows)

Then, install the packages with pip within the activated environment

python -m pip install -r requirements.txt

Finally, install the notebook extensions if you want to see them in slide mode:

python -m jupyter contrib nbextension install --user
python -m jupyter nbextension enable varInspector/main

Now you should be able to run the provided notebook.

Running the tutorial

After installing the package

You can start the jupyter notebook in the terminal, and it will start a browser automatically

python -m jupyter notebook

Alternatively, you can use supported Editor to run the jupyter notebooks, e.g. with VS Code.

---

Citing the tutorial

This tutorial is uploaded to Zenodo. Please use the following DOI when citing this code:

@software{xu_2024_10777477,
  author       = {Xu, Chenran and
                  Santamaria Garcia, Andrea and
                  Kaiser, Jan},
  title        = {Tutorial on Applying Reinforcement Learning to the Particle Accelerator {ARES}},
  month        = {03},
  year         = {2024},
  publisher    = {Zenodo},
  version      = {v1.0.1},
  doi          = {10.5281/zenodo.10777477},
  url          = {https://doi.org/10.5281/zenodo.10777477}
}

---

Acknowledgement

This tutorial is developed by Jan Kaiser, Andrea Santamaria Garcia, and Chenran Xu.

The content is based on the tutorial given at the RL4AA'23 workshop: GitHub repository