Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Recording policies with --root argument set fails if you've recorded a policy before #534

Closed
2 tasks
apockill opened this issue Nov 30, 2024 · 4 comments · May be fixed by #539
Closed
2 tasks

Recording policies with --root argument set fails if you've recorded a policy before #534

apockill opened this issue Nov 30, 2024 · 4 comments · May be fixed by #539

Comments

@apockill
Copy link

apockill commented Nov 30, 2024 •
edited
Loading

System Info

lerobot-1  | Traceback (most recent call last):
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/scripts/control_robot.py", line 555, in <module>
lerobot-1  |     record(robot, **kwargs)
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/common/robot_devices/utils.py", line 28, in wrapper
lerobot-1  |     raise e
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/common/robot_devices/utils.py", line 24, in wrapper
lerobot-1  |     return func(robot, *args, **kwargs)
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/scripts/control_robot.py", line 249, in record
lerobot-1  |     dataset = LeRobotDataset.create(
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/common/datasets/lerobot_dataset.py", line 925, in create
lerobot-1  |     obj.meta = LeRobotDatasetMetadata.create(
lerobot-1  |   File "/tmp/ee8739f1-22d7-410a-afe3-ab4eaf2289a7/lerobot/common/datasets/lerobot_dataset.py", line 291, in create
lerobot-1  |     obj.root.mkdir(parents=True, exist_ok=False)
lerobot-1  |   File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
lerobot-1  |     self._accessor.mkdir(self, mode)
lerobot-1  | FileExistsError: [Errno 17] File exists: 'data'

Information

  • One of the scripts in the examples/ folder of LeRobot
  • My own task or dataset (give details below)

Reproduction

Run control_robot.py record --root "data" and it will fail if the data directory already exists.

Expected behavior

It should be okay to have many datasets under your root directory.

{root}/{HF_USER}/{dataset_name}
{root}/{HF_USER}/{dataset_name}
{root}/{HF_USER}/{dataset_name}

In the above example, only dataset_name should be forced to be unique.
@apockill
Copy link
Author

apockill commented Nov 30, 2024 •
edited
Loading

I'm a bit confused by lines around the codebase doing

Path(root) if root is not None else LEROBOT_HOME / repo_id

Does this imply that root should already contain the full root/user/dataset_name within it? Intuitively I would have expected it should be:

Path(root) if root is not None else LEROBOT_HOME

Without the / repo_id. I'm happy to submit a PR once I know what the intended meaning of root is within the new 2.0 system!

@apockill apockill changed the title --root directory is not allowed to exit Recording policies with --root argument set fails if you've recorded a policy before Nov 30, 2024
@apockill apockill mentioned this issue Dec 2, 2024
Open
@aliberts
Copy link
Collaborator

aliberts commented Dec 3, 2024 •
edited
Loading

Hi there,
As @Cadene mentioned in your PR, we changed the behavior of root to be directly the path where your dataset is stored.
in v1, you would do --root=data which would translate to

root = Path("data") / repo_id

Now in v2, root directly points to your dataset with no repo/id intermediate directories being created, so to get the equivalent you should now do --root=data/repo/id.

That being said, we removed references to the --root option to simplify thing in examples commands since by default, root will be at ~/.cache/huggingface/lerobot/repo/id and specifying root should not be necessary by default (but you still can if you want to).

@aliberts
Copy link
Collaborator

aliberts commented Dec 3, 2024

I'm closing this since this is intended behavior but don't hesitate to re-open if you still think there's a problem ;)

@aliberts aliberts closed this as completed Dec 3, 2024
@apockill
Copy link
Author

apockill commented Dec 3, 2024

Awesome, thanks for the clarification!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@apockill @aliberts