Skip to content

Codebase for project about unsupervised skill learning via variational inference and causality.

License

Notifications You must be signed in to change notification settings

tanmayshankar/CausalSkillLearning

 
 

Repository files navigation

Learning Robot Skills with Temporal Variational Inference

What is this?

This repository has code for our ICML 2020 paper on Learning Robot Skills with Temporal Variational Inference, authored by Tanmay Shankar and Abhinav Gupta.

I want a TL;DR of what this paper does.

Our paper presents a way to jointly learn robot skills and how to use them from demonstrations in an unsupervised manner. The code implements the training procedure for this across 3 different datasets, and provides tools to visualize the learnt skills.

Cool. Can I use your code?

Yes! If you would like to use our code, please cite our paper and this repository in your work. Also, be aware of the license for this repository: the Creative Commons Attribution-NonCommercial 4.0 International. Details may be viewed in the License file.

I need help, or I have brilliant ideas to make this code even better.

Great! Feel free to mail Tanmay ([email protected]), for help, suggestions, questions and feedback. You can also create issues in the repository, if you feel like the problem is pertinent to others.

How do I set up this repository?

Dependencies

You will need a few packages to be able to run the code in this repository. For Robotic environments, you will need to install Mujoco, Mujoco_Py, OpenAI Gym, and Robosuite. Here is a list of instructions on how to set these up.

You will also need some standard deep learning packages, Pytorch, Tensorflow, Tensorboard, and TensorboardX. Usually you can just pip install these packages. We recommend using a virtual environment for them.

Data

We run our model on various publicly available datasets, i.e. the MIME dataset, the Roboturk dataset, and the CMU Mocap dataset. In the case of the MIME and Roboturk datasets, we collate relevant data modalities and store them in quickly accessible formats for our code. You can find the links to these files below.

MIME Dataset Roboturk Dataset CMU Mocap Dataset

Once you have downloaded this data locally, you will want to feed the path to these datasets in the --dataset_directory command line flag when you run your code.

Tell me how to run the code already!

Here is a list of commands to run pre-training and joint skill learning on the various datasets used in our paper. The hyper-parameters specified here are used in our paper. Depending on your use case, you may want to play with these values. For a full list of the hyper-parameters, look at Experiments/Master.py.

The MIME Dataset

For the MIME dataset, to run pre-training of the low-level policy:

python Master.py --train=1 --setting=pretrain_sub --name=MIME_Pretraining --data=MIME --number_layers=8 --hidden_size=128 --kl_weight=0.01 --var_skill_length=1 --z_dimensions=64 --normalization=meanvar

This should automatically run some evaluation and visualization tools every few epochs, and you can view the results in Experimental_Logs/<Run_Name>/. Once you've run this pre-training, you can run the joint training using:

python Master.py --train=1 --setting=learntsub --name=J100 --normalization=meanvar --kl_weight=0.0001 --subpolicy_ratio=0.1 --latentpolicy_ratio=0.001 --b_probability_factor=0.01 --data=MIME --subpolicy_model=Experiment_Logs/<MIME_Pretraining>/saved_models/Model_epoch480 --latent_loss_weight=0.01 --z_dimensions=64 --traj_length=-1 --var_skill_length=1 --training_phase_size=200000  

The Roboturk Dataset

For the Roboturk dataset, to run pre-training of the low-level policy:

python Master.py --train=1 --setting=pretrain_sub --name=Roboturk_Pretraining --data=FullRoboturk --kl_weight=0.0001 --var_skill_length=1 --z_dimensions=64 --number_layers=8 --hidden_size=128

Just as in the case of the MIME dataset, you can then run the joint training using:

python Master.py --train=1 --setting=learntsub --name=RJ80 --latent_loss_weight=1. --latentpolicy_ratio=0.01 --kl_weight=0.0001 --subpolicy_ratio=0.1 --b_probability_factor=0.001 --data=Roboturk --subpolicy_model=Experiment_Logs/<Roboturk_Pretraining>/saved_models/Model_epoch20 --z_dimensions=64 --traj_length=-1 --var_skill_length=1 --number_layers=8 --hidden_size=128

Stay tuned for more!

About

Codebase for project about unsupervised skill learning via variational inference and causality.

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 74.8%
  • Jupyter Notebook 25.2%