This guide provides instructions on evaluating the pretrained GLaMM models on the downstream tasks including Grounded Conversation Generation (GCG), referring expression segmentation and region-level captioning.
Run the following instruction to evaluate GLaMM model on the GCG task
bash eval/gcg/run_evaluation.sh 'path/to/the/HF/checkpoints/path' 'path/to/the/directory/to/save/the/evaluation/results'
To evaluate provided finetuned GCG model, run,
bash eval/gcg/run_evaluation.sh 'MBZUAI/GLaMM-GCG' './results_gcg_finetuned'
This will automatically download the MBZUAI/GLaMM-GCG
from HuggingFace.
Run the following instruction to evaluate GLaMM model on the referring expression segmentation task
bash eval/referring_seg/run_evaluation.sh 'path/to/the/HF/checkpoints/path' 'path/to/the/directory/to/save/the/evaluation/results'
To evaluate provided finetuned RefSeg model, run,
bash eval/referring_seg/run_evaluation.sh 'MBZUAI/GLaMM-RefSeg' './results_refseg_finetuned'
This will automatically download the MBZUAI/GLaMM-RefSeg
from HuggingFace.
Run the following instruction to evaluate GLaMM model on the region-level captioning task
bash eval/region_captioning/run_evaluation_RefCOCOg.sh 'path/to/the/HF/checkpoints/path' 'path/to/the/directory/to/save/the/evaluation/results'
To evaluate provided finetuned RefCOCOg model, run,
bash eval/region_captioning/run_evaluation_RefCOCOg.sh 'MBZUAI/GLaMM-RegCap-RefCOCOg' './results_regcap_refcocog_finetuned'
This will automatically download the MBZUAI/GLaMM-RegCap-RefCOCOg
from HuggingFace.
bash eval/region_captioning/run_evaluation_VG.sh 'path/to/the/HF/checkpoints/path' 'path/to/the/directory/to/save/the/evaluation/results'
To evaluate provided finetuned VG model, run,
bash eval/region_captioning/run_evaluation_VG.sh 'MBZUAI/GLaMM-RegCap-VG' './results_regcap_vg_finetuned'
This will automatically download the MBZUAI/GLaMM-RegCap-VG
from HuggingFace.