How to align Behavior-1k to VLA, e.g. octo?

[qyx1121]

PLEASE POST QUESTIONS IN THE DISCUSSIONS TAB
Hi, I'm trying to connect Omnigibson to some VLAs, like octo. I used a pre-trained octo model that outputs 7-dimensional actions, and the Omnigibson environment can also receive 7-dimensional actions. How can I align these two actions? (For example, I found that Omnigibson's action space has a range of [-1, 1], but octo outputs values outside that range.)

[hang-yin]

Hi @qyx1121 , that's a great question.

Each controller in OmniGibson takes command_input_limits and command_output_limits during init time. You can check the documentation of these inputs at omnigibson/controllers/controller_base.py. Also attaching here for your reference:

            command_input_limits (None or "default" or Tuple[float, float] or Tuple[Array[float], Array[float]]):
                if set, is the min/max acceptable inputted command. Values outside this range will be clipped.
                If None, no clipping will be used. If "default", range will be set to (-1, 1)
            command_output_limits (None or "default" or Tuple[float, float] or Tuple[Array[float], Array[float]]):
                if set, is the min/max scaled command. If both this value and @command_input_limits is not None,
                then all inputted command values will be scaled from the input range to the output range.
                If either is None, no scaling will be used. If "default", then this range will automatically be set
                to the @control_limits entry corresponding to self.control_type

If you are specifying this through environment config, you can do something like:

    cfg = {
        "scene": {"type": "Scene"},
        "robots": [
            {
                "type": "Fetch",
                "action_normalize": False,
                "controller_config": {
                    "arm_0": {
                        "name": "InverseKinematicsController",
                        "command_input_limits": None,
                    },
                },
            }
        ],
    }

If you want to do it through a yaml config file, you can do something like:

robots:
  - type: Tiago
    obs_modalities: [rgb]
    self_collisions: true
    action_normalize: false
    action_type: continuous
    grasping_mode: sticky
    rigid_trunk: false
    default_trunk_offset: 0.15
    default_arm_pose: vertical
    sensor_config:
      VisionSensor:
        sensor_kwargs:
          image_height: 128
          image_width: 128
    controller_config:
      base:
        name: JointController
      arm_left:
        name: JointController
        motor_type: position
        command_input_limits: null
        command_output_limits: null
        use_delta_commands: false
      arm_right:
        name: JointController
        motor_type: position
        command_input_limits: null
        command_output_limits: null
        use_delta_commands: false
      gripper_left:
        name: JointController
        motor_type: position
        command_input_limits: [-1, 1]
        command_output_limits: null
        use_delta_commands: false
      gripper_right:
        name: JointController
        motor_type: position
        command_input_limits: [-1, 1]
        command_output_limits: null
        use_delta_commands: false
      camera:
        name: JointController