This is a released package of a mathematically simulated pendulum that can be flexibly adapted for applying it as an RL environment. An advantage of this pendulum simulation is the flexibility of the parameter definition, including the number of rods, the rod masses, rod inertias, and more.
It is aimed to enable more flexible RL research on the difficult control problem of the cart-pendulum environment.
Best view 📒this notebook📒 (.html file; 281 MB) and review the overall clean code to get all the information you need. Besides matplotlib animations and LaTeX math summaries, this package also supports rendering Plotly Animations, which are a lot faster to render for RL research, and easier to embed into Panel UIs. Checkout ✨this interactive animation✨ (.html file; 56 MB).
I also provide an unfinished Panel UI prototype of a planned RL experiment platform that could help others build flexible UIs for experiment configurations and applications. It is out of my scope to finish this properly, however, this could provide a lot of value for other researchers. Here is a 🎞video demonstration🎞 (.mkv file; watchable w.o download) of the current Panel UI implementation.
Credit: The very good math derivations of this notebook for the triple cart-pendulum were reused for this project and abstracted to an n-rodded cart-pendulum class.
Pendulum.Animation.Matplotlib.v5.mp4
Pendulum.Animation.Plotly.v6.mp4
You generally have to follow the setup_commands.txt to set up and launch the Panel UI.
If you only want to run the RL pendulum in a notebook, it is recommended to only copy the self-contained directory rl_pendulum/envs/cart_pendulum, while leaving the other files. The other files won't be needed, and just give functionality to run the Panel UI.
🚧 Note that the rl_pendulum/envs/rl_cart_pendulum is not fully functional, but provides an extensive feature engineering pipeline. It is planned to reimplement the environment in JAX later and to use Stoix agents to solve it without much feature engineering.