MultiROS: ROS-Based Robot Simulation Environment for Concurrent Deep Reinforcement Learning

MultiROS is an open-source Robot Operating System (ROS)-based simulation environment designed for concurrent deep reinforcement learning. It provides a flexible and scalable framework for training and evaluating reinforcement learning agents for complex robotic tasks.

This package supports training robots in both simulation and the real world. The simulation environment is based on Gazebo, providing a realistic simulation platform for testing and developing reinforcement learning algorithms.

Additionally, MultiROS provides interfaces and tools for seamlessly transferring learned policies to real-world robotic systems.

For more information, please refer to the main UniROS repository which integrates MultiROS and RealROS.

Prerequisites

Before installing MultiROS, make sure you have the following prerequisites:

ROS Installation

MultiROS requires a working installation of ROS. If you haven't installed ROS yet, please follow the official ROS installation guide for your specific operating system. This package has been tested with ROS Noetic version, and the following instructions will guide you through the installation of ROS Noetic on Ubuntu 20.04:

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt install curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-noetic-desktop-full

echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
source ~/.bashrc

sudo apt install python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential
sudo apt install python3-rosdep
sudo rosdep init
rosdep update

Catkin Tools

MultiROS uses Catkin as the build system for ROS packages. Install Catkin Tools by running the following command:

sudo apt-get install python3-catkin-tools

Create ROS Workspace

Before using MultiROS, you need to create a ROS workspace to build and run your ROS packages. Follow these steps to create a workspace:

cd ~
source /opt/ros/noetic/setup.bash
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/
catkin build
echo "source ~/catkin_ws/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

Other Packages

MultiROS also requires the following additional packages:

• XTerm for terminal emulation:

  sudo apt install xterm

• MoveIt for motion planning:

  sudo apt install ros-noetic-moveit

• PyKDL for Kinematics:

  sudo apt install python3-pykdl

• kdl_parser_py for parsing URDF files (for Kinematics_pyrobot):

  sudo apt install ros-noetic-kdl-parser-py

• trac_ik_python for inverse kinematics (for Kinematics_pyrobot):

  # Download and install trac_ik_python
  cd ~/catkin_ws/src
  git clone https://bitbucket.org/traclabs/trac_ik.git

• pykdl_utils for kinematics (for Kinematics_pykdl):

  # download the package - modified version of the original package to support ROS Noetic
  cd ~/catkin_ws/src
  git clone https://github.com/ncbdrck/hrl-kdl.git
  
  # install the pykd_utils
  cd ~/catkin_ws/src/hrl-kdl/pykdl_utils
  python3 setup.py build
  sudo python3 setup.py install
  
  # install the hrl_geom
  cd ~/catkin_ws/src/hrl-kdl/hrl_geom
  python3 setup.py build
  sudo python3 setup.py install
  
  # Install urdf_parser and urdfdom-py
  sudo apt-get install ros-noetic-urdf-parser-plugin
  sudo apt-get install ros-noetic-urdfdom-py
  
  # Build the package
  cd ~/catkin_ws
  rosdep install --from-paths src --ignore-src -r -y
  catkin build
  source devel/setup.bash

You are now ready to proceed with the installation and usage of MultiROS.

Please note that the instructions assume you are using Ubuntu 20.04 and ROS Noetic. If you are using a different operating system or ROS version, make sure to adapt the commands accordingly.

Installation

To get started with MultiROS, follow these steps:

1. Clone the repository:

   cd ~/catkin_ws/src
   git clone -b gymnasium https://github.com/ncbdrck/multiros.git

2. MultiROS relies on several Python packages. You can install them by running the following command:

   # Install pip if you haven't already by running this command
   sudo apt-get install python3-pip
   
   # install the required Python packages for MultiROS by running
   cd ~/catkin_ws/src/multiros
   pip3 install -r requirements.txt

3. Build the ROS packages and source the environment:

   cd ~/catkin_ws/
   rosdep install --from-paths src --ignore-src -r -y
   catkin build
   source devel/setup.bash

Usage

Refer to the templates or the examples to see how MultiROS can create a simulation environment for RL applications.

The installation instructions for the examples are provided in the respective repositories.

License

MultiROS is released under the MIT License. Please see the LICENSE file for more details.

Acknowledgements

We would like to thank the following projects and communities for their valuable contributions, as well as the authors of relevant libraries and tools used in MultiROS.

• ROS (Robot Operating System)
• Gazebo
• Gymnasium
• Gymnasium-Robotics
• OpenAI Gym
• openai_ros
• frobs_rl
• gym-gazebo

Cite

If you use MultiROS in your research or work and would like to cite it, you can use the following citation:

Repository

@misc{multiros,
  author = {Kapukotuwa, Jayasekara},
  booktitle = {GitHub repository},
  publisher = {GitHub},
  title = {MultiROS: ROS-Based Robot Simulation Environment for Concurrent Deep Reinforcement Learning},
  url = {https://github.com/ncbdrck/multiros},
  year = {2022}
}

Article

@inproceedings{kapukotuwa_multiros_2022,
	title = {{MultiROS}: {ROS}-{Based} {Robot} {Simulation} {Environment} for {Concurrent} {Deep} {Reinforcement} {Learning}},
	shorttitle = {{MultiROS}},
	doi = {10.1109/CASE49997.2022.9926475},
	booktitle = {2022 {IEEE} 18th {International} {Conference} on {Automation} {Science} and {Engineering} ({CASE})},
	author = {Kapukotuwa, Jayasekara and Lee, Brian and Devine, Declan and Qiao, Yuansong},
	month = aug,
	year = {2022},
	note = {ISSN: 2161-8089},
	pages = {1098--1103},
}

Contact

For questions, suggestions, or collaborations, feel free to reach out to the project maintainer at [email protected].