FineVQ: Fine-Grained User Generated Content Video Quality Assessment (CVPR 2025 Highlight💡)

If you find our database and code useful, please give a star ⭐ and citation 📝

This is the official repo of the paper FineVQ: Fine-Grained User Generated Content Video Quality Assessment: We also extend the database and hold a challenge at CVPR NTIRE.

---

🤗 FineVD Download

Download with CLI:

huggingface-cli download IntMeGroup/FineVD --repo-type dataset --local-dir ./FineVD

🏆 FineVQ Metric

⚙️ Installation

Clone the repository:

git clone https://github.com/IntMeGroup/FineVQ.git

Create and activate a conda environment:

conda create -n FineVQ python=3.9 -y
conda activate FineVQ

Install dependencies:

pip install -r requirements.txt

Install flash-attn==2.3.6 (pre-built):

pip install flash-attn==2.3.6 --no-build-isolation

Or compile from source:

git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention
git checkout v2.3.6
python setup.py install

---

🔧 Preparation

📁 Prepare dataset

huggingface-cli download IntMeGroup/FineVD data.zip --repo-type dataset --local-dir ./
unzip data.zip -d ./data

📦 Prepare model weights

huggingface-cli download OpenGVLab/InternVL2-8B --local_dir OpenGVLab/InternVL2-8B

---

🚀 Training

for stage1 training (Spatiotemporal Projection Module)

sh shell/stage1_train.sh

for stage2 training (Fine-tuning the vision encoder and LLM with LoRA)

sh shell/stage2_train.sh

🌈 Evaluation

for score evaluation

sh shell/eval.sh

🌈 Inference

📦 Download the required model weights:

huggingface-cli download IntMeGroup/FineVQ_score --local-dir ./IntMeGroup/FineVQ_score

📁 Prepare dataset

1. Refine the /data/inference.json file with the correct path:

"root": your_path_to_videos

or infer selected videos in video_names.txt 2. Refine the /data/inference2.json file with the correct path:

"root": your_path_to_videos
"video_name_txt": video_names.txt

and change the shell/infer.sh line30 to data/inference2.json

🎮 Score Inference

Refine the shell/infer.sh line27 to your_download_model_pretrained_weight_path

For Overall Score Inference

sh shell/infer.sh

For Blur Score Inference

sh shell/infer_blur.sh

For Color Score Inference

sh shell/infer_color.sh

For Noise Score Inference

sh shell/infer_noise.sh

For Artifact Score Inference

sh shell/infer_artifact.sh

For Temporal Score Inference

sh shell/infer_temporal.sh

🎮 QA Train and Eval

🚀Training

sh shell/qa_train.sh

🌈Evaluation

First Download the pretrained weights

❓ FineVQ QA (Yes/No) FineVQ_QA_yn FineVQ QA (Yes/No) focuses on evaluating binary question-answering tasks

sh shell/qa_eval.sh

🧐 FineVQ QA (Which) FineVQ_QA_which FineVQ QA (Which) focuses on which questions in FineVD

sh shell/qa_eval2.sh

🌈Inference

add questions in ./question.txt

sh shell/infer_QA.sh

📌 TODO

• ✅ Release the training code (stage1 and stage2)
• ✅ Release the evaluation code (score prediction)
• ✅ Release the FineVD database
• ✅ Release the QA code

FineVQ Datasets and Weights

This repository provides pre-trained weights for various datasets in the realm of video quality evaluation. Below, you'll find the weights corresponding to different datasets that can be used for evaluating video quality with FineVQ.

Datasets and Corresponding Weights

Dataset	Link to Weights	Dataset Overview
🏞️ KoNViD	FineVQ_KoNViD	The konstanz natural video database (konvid-1k) (QoMex)
🖥️ LIVE-VQC	FineVQ_LIVE-VQC	Large-scale study of perceptual video quality (TIP)
🎮 LSVQ	FineVQ_LSVQ	Patch-vq:’patching up’the video quality problem (CVPR)
🕹️ LIVE-YT-Gaming	FineVQ_LIVE-YT-Gaming	Subjective and objective analysis of streamed gaming videos (TOG)
📺 YouTubeUGC	FineVQ_YouTubeUGC	Youtube ugc dataset for video compression research (MMSP)
🌈 FineVQ Score	FineVQ_Score	FineVQ Score focuses on ugc video quality score prediction
❓ FineVQ QA (Yes/No)	FineVQ_QA_yn	FineVQ QA (Yes/No) focuses on evaluating binary question-answering tasks
🧐 FineVQ QA (Which)	FineVQ_QA_which	FineVQ QA (Which) focuses on which questions in FineVD

📧 Contact

If you have any inquiries, please don't hesitate to reach out via email at wangjiarui@sjtu.edu.cn

🎓Citations

If you find FineVQ is helpful, please cite:

@InProceedings{Duan_2025_CVPR,
    author    = {Duan, Huiyu and Hu, Qiang and Wang, Jiarui and Yang, Liu and Xu, Zitong and Liu, Lu and Min, Xiongkuo and Cai, Chunlei and Ye, Tianxiao and Zhang, Xiaoyun and Zhai, Guangtao},
    title     = {FineVQ: Fine-Grained User Generated Content Video Quality Assessment},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {3206-3217}
}