Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve spacemouse #561

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Improve spacemouse #561

wants to merge 8 commits into from
+195 −76

Conversation

squarefk
Copy link

This PR adds support for bimanual teleoperation using the existing SpaceMouse class.
When two space mice are connected, the system will automatically switch to bimanual mode.

To test the functionality without setting up an environment, users can connect either one or two space mice and run python -m robosuite.devices.spacemouse.

snasiriany
snasiriany reviewed Nov 15, 2024
View reviewed changes
Copy link
Contributor

@snasiriany snasiriany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Yu! Left some comments!

robosuite/devices/device.py Outdated
@@ -163,10 +163,11 @@ def input2action(self, mirror_actions=False) -> Optional[Dict]:
)
ac_dict[f"{active_arm}_abs"] = arm_action["abs"]
ac_dict[f"{active_arm}_delta"] = arm_action["delta"]
ac_dict[f"{active_arm}_gripper"] = np.array([grasp] * gripper_dof)
# ac_dict[f"{active_arm}_gripper"] = np.array([grasp] * gripper_dof)
ac_dict[f"{active_arm}_gripper"] = [1, 0.45 * grasp, grasp, grasp, grasp, grasp]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this part change?

0.45 * grasp

this seems specific to the Inspire hand. We should make it a case by case basis
Also not sure if it works for non-humanoids

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. This would impact other robots. Moved the changes into class InspireLeftHand and InspireRightHand

robosuite/controllers/config/robots/default_gr1_fixed_lower_body.json Outdated
Comment on lines 15 to 17
"robot0_torso_waist_yaw": 30.0,
"robot0_torso_waist_pitch": 100.0,
"robot0_torso_waist_roll": 100.0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes are making the robot less agile. for example reaching up inside cabinets or farther back in the counter, etc. can we try to make use of the waist only when really needed? I think this part just needs more tuning, perhaps along with the overall gains of the controller.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parameters further tuned.

robosuite/devices/spacemouse.py Outdated
@@ -101,37 +102,26 @@ def convert(b1, b2):
return scale_to_control(to_int16(b1, b2))


class SpaceMouse(Device):
class SpaceMouseSdkWrapper:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe call this SpaceMouseListener instead

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed.

robosuite/models/assets/grippers/inspire_left_hand.xml Outdated
@@ -79,12 +79,12 @@
<site name="ft_frame" pos="0 0 0" size="0.01 0.01 0.01" rgba="1 0 0 0" type="sphere" group="1"/>
<inertial diaginertia="0 0 0" mass="0" pos="0 0 0"/>
<body name="eef" pos="0 0 0" quat="0.707 0. -0.707 0.">
<site name="grip_site" pos="0 0 0" size="0.01 0.01 0.01" rgba="1 1 0 1" type="sphere" group="2"/>
<site name="grip_site" pos="0 0 0" size="0.01 0.01 0.01" rgba="1 1 0 0" type="sphere" group="2"/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reason for change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been reverted.

robosuite/models/assets/grippers/inspire_left_hand.xml
<site name="ee_x" pos="0.1 0 0" size="0.005 .1" quat="0.707105 0 0.707108 0 " rgba="1 0 0 0" type="cylinder" group="1"/>
<site name="ee_y" pos="0 0.1 0" size="0.005 .1" quat="0.707105 0.707108 0 0" rgba="0 1 0 0" type="cylinder" group="1"/>
<site name="ee_z" pos="0 0 0.1" size="0.005 .1" quat="1 0 0 0" rgba="0 0 1 0" type="cylinder" group="1"/>
<!-- This site was added for visualization. -->
<site name="grip_site_cylinder" pos="0 0 0" quat="-0.5 -0.5 -0.5 0.5" size="0.005 0.5" rgba="0 1 0 0.3" type="cylinder" group="1"/>
<site name="grip_site_cylinder" pos="0 0 0" quat="-0.5 -0.5 -0.5 0.5" size="0.005 0.5" rgba="0 1 0 0" type="cylinder" group="1"/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reason for change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the long ray invisible.

robosuite/models/assets/grippers/inspire_left_hand.xml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add some red sites for when each of the other 3 fingers are closed? in a lighter shade of red than the current red color marker of the hand

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@snasiriany
Copy link
Contributor

@squarefk also I wanted to press S and start moving the other arm (at the start). but if I do it at the start the other arm does not move. can we fix this?

@squarefk
Copy link
Author

also I wanted to press S and start moving the other arm (at the start). but if I do it at the start the other arm does not move. can we fix this?

Repost the solution here:

In the is_empty_input_spacemouse function from robocasa/robocasa/scripts/collect_demos.py , we only checked right_delta. Need to update the condition into:

    if (
        ("left_delta" not in action_dict or np.all(action_dict["left_delta"] == 0))
        and np.all(action_dict["right_delta"] == 0)
        and action_dict["base_mode"] == -1
        and np.all(action_dict["base"] == 0)
    ):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snasiriany snasiriany snasiriany left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@squarefk @snasiriany