Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement mechanism for including presampled robot poses in the task instances #846

Open
cgokmen opened this issue Aug 21, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@cgokmen
Copy link
Contributor

cgokmen commented Aug 21, 2024

We include exactly one robot in a task instance in the dataset. Many times, users may want to use a robot that is not the robot included in the scene file. We currently provide a mechanism that allows not loading robots from the scene (include_robots=False) and instead loading your own robot through the config file. There are a couple problems with this:

  1. You need to specify a starting pose for the robot that satisfies the task's starting conditions. This is not a very reasonable expectation to have from the user because it's not easy to find this out and the config file would need to be different for every task.
  2. If you want the same kind of robot but different controllers, this becomes a problem: the name of the robot from the scene is the same as the config-loaded robot, so when you load state, the scene robot's state will be loaded into the config robot, but since the controllers are mismatched the controller load_state will see a state dump that is incompatible, and crash.
  3. We do NOT want to generalize saving / loading robots by default: if a user saves and loads a state they expect and should get the robot included. We just don't want to ship robots in the BEHAVIOR task init states.
  4. As a side issue, there is no easy way to randomize just the robot's starting pose while still satisfying the BEHAVIOR task conditions, which would be a rather common use case.

Suggested plan:

  1. Pre-step: make config-loaded robots get a randomly generated name that will never match the scene-loaded robot name, so that the issue in point 2 above cannot happen. This is an urgent bugfix for otherwise unusable primitives features.
  2. In the BEHAVIOR task sampler / saver, sample a pose for every kind of available robot. Save these presampled poses in the BEHAVIOR task metadata and remove the robot from the scene prior to saving so that the saved scene JSON does NOT contain robots as objects, just as metadata. We are doing this ONLY in the BEHAVIOR task presampling logic - normally if you are saving a scene you should STILL get the robot.
    Note that "every kind" doesnt necessarily just mean every class - we might need different presampled poses based on settings like default joint pos that might affect robot shape
  3. Now since the robot is loaded in the env base via the config, it will be at the wrong spot. Update the BEHAVIOR task reset such that it either resets the scene robot to the presampled pose that it finds in the metadata, or to a random pose satisfying the initial conditions, based on the state of a task config (use_presampled_robot_pose?). In the reset also assert that the initial state satisfies the task initial condition.
  4. Revert the pre-step
  5. Add assertions that make sure that if a robot is loaded through a state dump then its entire config must match the robot in the config including the name and controllers etc.
@cgokmen cgokmen added the enhancement New feature or request label Aug 21, 2024
@cgokmen cgokmen changed the title Discuss and implement mechanism to (pre-?)sample robot poses Implement mechanism for including presampled robot poses in the task instances Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants