Loading USD environment from "3DFRONT/USD-exports" (env_path)

[lyndonchan]

Hello,

I am trying to follow the documentation to run paper_simulation.py but the script is attempting to load a USD environment from "3DFRONT/USD-exports" (the env_path variable defined inside simulator/configs/config.yaml). Your repo makes no mention of any USD file input, so how am I supposed to proceed?

Thanks,
Lyndon

[eliabntt]

Hi,
first of all thanks for testing this out.

I'll make sure to improve the documentation on this. I got swamped by several things in the last few months and I still have to finish the repo.

The ideal process to reproduce our pipeline would be as follow:

1. Generate the environment following this repo. This will get you the STL, the USD and the npy file of the environment. The STL is used by the placement procedure, the npy to get rough information about the 3D limits of the environment and the USD file to get the assets.
2. Generate the human assets following this repo. This will get you the USD file of the animated asset, the STL of the animation for the placement procedure, and a pkl file with the information on the animation itself.
3. run the code you are trying to run

The goal of releasing the first batch of code was more educational/explanatory rather than full repeatability (for that you would need animation assets and environment assets that we are going to release in a while).

Now, coming back to your question you have four options:

1. Either skip that line (at which point you'll work with the empty default word, although this would require other modifications in the code)
2. convert a scene from 3DFRONT dataset yourself using our other repository
3. use one of the usd files we made available here. In that case you should set correct_paths_req as False here
4. create your own USD, stl and npy files from one of your scenes(see here for hints).

If you use one of the USD we made available, please do NOT use "Savana", since the stl file will take too long to load. You can load an usd file without the stl by setting the use_stl config parameter as False (here is where that is going to be processed). Note that without the STL our automatic placement based on MoveIT interface won't work.

Finally, my suggestion would be to start from the easier to understand script simulator_ros

If you have any fourther doubt please open new issues/pull request. It'll be of help for next people and also for me to improve the code/doc.

[eliabntt]

Note: if you don't convert AMASS or CLOTH3D data yourself you will also need to skip the human loading part, unless you have your own assets. The ETA on the data release is still unknown unfortunately.

[lyndonchan]

@eliabntt thanks for the speedy and detailed response. Yeah, it looks like your repo is more of a work in progress rather than in a plug-and-play state right now.

To clarify your latest post, you mean that I can't use the animated_human_SMPL_to_USD repo yet because it needs licensing for Cloth3D and AMASS? How do I skip the human loading part? Are there some lines in the code that I can just comment out?

[eliabntt]

Hi @lyndonchan, the system is thought to be a guidance for each and everyone else project, as we explain in the paper. Showing different use cases, scenarios, and way to adopt it without constraints in the APIs or in the SIL or in the configurations (as is the case with other programs). I'm working on this completely a
alone with some help here and there, so bear with me on quality of the code or missing Infos here and there.

I edited the post above, in the second point there was a typo and the note was not clear, sorry.

There's no issue on going in the cloth3d website or the amass website and accept their licenses and convert a bunch of assets. That repo, as well as the one of FRONT3D/blender-based-environments converter, should be ready as plug and play. Instructions are much better in those repo since they focus only a single aspect/tool.

The data will be available whenever we finished fixing some minor things like relative/absolute paths, removing unnecessary files, and organize deployment of several hundreds of GBs of data. Sorry about any inconvenience.

As to where the humans are loaded there's a "Loading humans..." Section of the code here

[eliabntt]

Hi @lyndonchan, following your suggestions I'm (re)writing documentation and refactoring some code. Now it should be clearer. Note that we're still missing some assets (planning to add them during this week). Code is (un)-tested but it's copy pasting working code so.... 🙏

Here the two brand new pages https://github.com/eliabntt/GRADE-RR/blob/main/HOWTO.md and https://github.com/eliabntt/GRADE-RR/blob/main/TipsAndTricks.md.

Also https://github.com/eliabntt/GRADE-RR/blob/main/MOVEMENT.md should be clearer now.

I'll fully test it as soon as I can free up my GPU :)