Llava-v1.5 and Llava-v1.5-hf Performes on MMMU differs a lot!

[hxhcreate]

Llava-v1.5-7b resutls

Llava-1.5-7b-hf

Is that right?
Can the official teams provide a reliable resutls for your supported model and datasets for a reference?

[sycny]

I have found that using different versions of transformers will result in different MMMU values. When I use the latest version (4.42.2), I get a low value with Llava-1.5-7b-hf. But when I switch to an older version (4.39.2), I get a reasonable value, just like the one reported by the paper. I don't know what causes this problem.