Reward classifier and training

[ChorntonYoel]

What this does

Title	Label
Item 2 of Reward Classifier in issue #504	(Feature)

This PR is meant to add a reward classifier (used to classify if an image of a robot performing a task should get a reward or not), a training file allowing the training of the classifier (with logging + resuming), a config.yaml file that can be used to start a training, and a few tests for the training loop

How it was tested

Using 10 episodes made with the reward system of this PR: #518
Also I added a test file for the training classifier file. Lots of things are mocked but it covers the basics I believe.

How to checkout & try? (for the reviewer)

python lerobot/scripts/train_classifier.py \
    --config-name", "policy/reward_classifier.yaml",

With the wandb entity and the dataset name adapted.

I was able to reproduce 95%+ after a few epochs with facebook/convnext-base-224 as backbone and a dataset of 10 epsiodes of ~15 sec.
This branch was built on top of the branch from #518 so will need to wait for this one to be merged befre merging

[michel-aractingi]

Great work! looking forward to trying it, let me know when I can review it.

[Cadene]

What do you think of adding lerobot/common/policies/classifier/README.md with very short explanation and example commands?

Also Classifier is not a policy (it doesnt output actions). However I dont know what to do, so current implementation is good for now. Does anyone have a better idea?

(PS: waiting for the first branch to merge before approving)

[Cadene]

Nice file!

Out of curiosity, why this comment # test_train_classifier.py?

[ChorntonYoel]

No good reason haha force of habit, for some pre-commit setups you need a comment at the beginning of the file, so I start by a dummy one. will take off

[Cadene]

Nice file!

[michel-aractingi]

Nice work @ChorntonYoel ! Could you move the classifier directory to lerobot/common/policies/classifier/ to lerobot/common/policies/hilserl/classifier.

Since now we will only use the reward classifier for hil-serl then we will put everything in its directory. In the future when the classifiers are more established we can have a separate directory in lerobot/common/classifiers to host different kinds of reward recognition models.