cuberl

Cuberl is a C++ library containing implementations of various reinforcement learning, filtering and planning algorithms. The library documentation (under development) can be found here CubeAI The Python flavor of the library can be found at PyCubeAI.

The following is an indicative list of examples. More tutorials can be found at CubeAI.

Examples

Introductory

• CubeRl Basics
• Using PyTorch C++ API Part 1
• Using PyTorch C++ API Part 2
• Using TensorboardServer class
• Toy Markov chain

Reinforcement learning

• DummyAgent on MountainCar-v0
• Multi-armed bandit with epsilon-greedy policy
• Multi-armed bandit with soft-max policy
• Advertisement placement
• Iterative policy evaluation on FrozenLake-v0
• Policy iteration on FrozenLake-v0
• Value iteration on FrozenLake-v0
• SARSA on CliffWalking-v0
• Q-learning on CliffWalking-v0
• A2C on CartPole-v1
• DQN on CartPole-v0
• REINFORCE algorithm on CartPole-v0
• Expected SARSA on CliffWalking-v0
• Approximate Monte Carlo on MountainCar-v0
• Monte Carlo tree search on Taxi-v3
• Double Q-learning on CartPole-v0
• First visit Monte Carlo on FrozenLake-v0

Path planning

• Path planning with rapidly-exploring random trees (TODO)
• Path planning with dynamic windows (TODO)
• A* search on a road network from Open Street Map data

Installation

The cubeai library has a host of dependencies: Installation instructions and dependencies can be found here.

Enabling PyTorch and CUDA

cuberl can be complied with CUDA and/or PyTorch support. If PyTorch has been compiled using CUDA support, then you need to enable CUDA as well. In order to do so set the flag USE_CUDA in the CMakeLists.txt to ON. cuberl assumes that PyTorch is compiled with the C++11 ABI.

Documentation dependencies

There are extra dependencies if you want to generate the documentation. Namely,

• Doxygen
• Sphinx
• sphinx_rtd_theme
• breathe
• m2r2

Note that if Doxygen is not found on your system CMake will skip this. On a Ubuntu/Debian based machine, you can install Doxygen using

sudo apt-get install doxygen

Similarly, install sphinx_rtd_theme using

pip install sphinx_rtd_theme

Install breathe using

pip install breathe

Install m2r2 using

pip install m2r2

Issues

undefined reference to [email protected].

You may want to check with nvidia-msi your CUDA Version and make sure it is compatible with the PyTorch library you are linking against

TypeError: Descriptors cannot be created directly.

This issue may be occur when using the TensorBoardServer in cuberl. This issue is related an issue with protobuf. See: https://stackoverflow.com/questions/72441758/typeerror-descriptors-cannot-not-be-created-directly for possible solutions.