Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix several bugs relating to saving offline datasets when not logged in to HuggingFace #539

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

apockill
Copy link

@apockill apockill commented Dec 2, 2024

What this does

This PR fixes #534 and fixes several bugs I found that all relate to not being logged in to huggingface.

I found a few places that were broken.

  1. When --root is set, your dataset was being saved in root/DATA instead of root/hf_username/repo.
  2. If you go to control_robot record on an existing dataset, you would get an exception because it was trying to path.mkdir(exist_ok=false), when actually, it should be exist_ok=True.
  3. There are still some issues in the training pipeline, but I'm going to leave that for another PR.

How it was tested

  1. Record a few episodes
  2. Try to continue recording on the existing dataset
  3. Train a model (there's still some issues here, I ended up just logging in to huggingface to skip by them. I'll revisit this later)
  4. Visualizing the dataset

How to checkout & try? (for the reviewer)

Here's how I've been running control_robot record

record
--robot-path lerobot/configs/robot/myarm.yaml
--root "data"
--repo-id username/dataset_name
--single-task "Cool task description, wow!"
--fps 20
--push-to-hub 0
--resume 1
--local-files-only 1

@Cadene
Copy link
Collaborator

Cadene commented Dec 2, 2024

@apockill We changed the behavior of root to be more explicit.
You would need to provide --root "data/username/dataset_name".
Could you try with that?
cc @aliberts

@Cadene Cadene self-requested a review December 2, 2024 17:10
@Cadene Cadene self-assigned this Dec 2, 2024
@apockill
Copy link
Author

apockill commented Dec 2, 2024

Ahh got it. I'll try that soon and report back. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Recording policies with --root argument set fails if you've recorded a policy before
2 participants