Evaluation Performance on LLaVA-Lora

[XindiWu]

I'm evaluating the LLaVA-Lora version (https://huggingface.co/liuhaotian/llava-v1.5-7b-lora/discussions), but the performance seems unusually low. Do you know if this is supported in the lmms-eval pipeline?

[kcz358]

I have never tested a Lora version of LLaVA using lmms-eval. I remember there are some discussion about this in this issue #241. For llava-v1.5, since it is already a relatively old model, I am not sure what environment or dependencies you are going to need for lmms-eval