Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mehdidc authored Aug 27, 2023
1 parent 8cace20 commit 567f01c
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,16 @@ clip_benchmark eval --pretrained_model benchmark/models.txt \

Examples are available in [benchmark/datasets.txt](benchmark/datasets.txt) and [benchmark/models.txt](benchmark/models.txt)

### Multiple checkpoints from the same model

It is also common to evaluate multiple checkpoints from the same model:

```bash
clip_benchmark eval --model ViT-B-32 --pretrained *.pt \
--dataset benchmark/datasets.txt --dataset_root "clip_benchmark_datasets/{dataset}" \
--output "{dataset}_{pretrained}_{model}_{language}_{task}.json"
```

### Model and dataset collections

We can also provide model collection names (`openai`, `openclip_base`, `openclip_multilingual`, `openclip_full` are supported) or dataset collection names (`vtab`, `vtab+`, `retrieval`, `imagenet_robustness` are supported):
Expand Down

0 comments on commit 567f01c

Please sign in to comment.