PyTorch implementation of "Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment" (arXiv/AAAI), which has been accepted by AAAI-2024.
This code is modified from FAST-VQA.
For image quality assessment (IQA), please refer to IQA/demo_train_iqa_baseline.py.
For video quality assessment (VQA), please refer to VQA/demo_train.py to get the training result, and refer to VQA/demo_finetune.py to get the finetuning result. We also provide the training log for VQA.
The main idea/contribution lies in the data sampling, which can be found in IQA and VQA.
Make sure the configuration has been properly set in
- fast-sama-iqa.yml for IQA training;
- fast-SAMA-train.yml for VQA training on LSVQ;
- and fast-SAMA-tinetune.yml for VQA finetuning.
And please prepare the pretrained models of video-swin for VQA and swin-v2 for IQA.
We have provided the pretrained weights (trained on LSVQ train set): GoogleDrive / BaiDu (Code:xyns). please check the pretrained weights in ./VQA/pretrained_weights folder and put the weights in the folder.
To test on your own dataset or video files, please construct the dataset information as the examplar in ./VQA/examplar_data_labels, and set the configuration in fast-SAMA-test.yml. Run the file demo_test.py to check the details.
Different environment may induce possible fluctuation of performance.
Python 3.8.10
PyTorch 1.7.0
The installation can refer to FAST-VQA.
If you are interested in the work, or find the code helpful, please cite our work
@article{sama2024,
title={Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment},
volume={38},
number={4},
journal={Proceedings of the AAAI Conference on Artificial Intelligence},
author={Liu, Yongxu and Quan, Yinghui and Xiao, Guoyao and Li, Aobo and Wu, Jinjian},
year={2024},
month={Mar.},
pages={3792-3801},
url={https://ojs.aaai.org/index.php/AAAI/article/view/28170},
DOI={10.1609/aaai.v38i4.28170}
}
Feel free to contact me via [email protected] if any question or bug.