Coder Social home page Coder Social logo

zyf1040895256 / maniqa Goto Github PK

View Code? Open in Web Editor NEW

This project forked from iigroup/maniqa

1.0 0.0 0.0 1.12 MB

[CVPRW 2022] MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment

License: Apache License 2.0

Python 100.00%

maniqa's Introduction

MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment

Authors: Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang and Yujiu Yang

paper IIGROUP pretrained model

This is the official repository for NTIRE2022 Perceptual Image Quality Assessment Challenge Track 2 No-Reference competition. We won first place in the competition and the codes have been released now.

Abstract: No-Reference Image Quality Assessment (NR-IQA) aims to assess the perceptual quality of images in accordance with human subjective perception. Unfortunately, existing NR-IQA methods are far from meeting the needs of predicting accurate quality scores on GAN-based distortion images. To this end, we propose Multi-dimension Attention Network for no-reference Image Quality Assessment (MANIQA) to improve the performance on GAN-based distortion. We firstly extract features via ViT, then to strengthen global and local interactions, we propose the Transposed Attention Block (TAB) and the Scale Swin Transformer Block (SSTB). These two modules apply attention mechanisms across the channel and spatial dimension, respectively. In this multi-dimensional manner, the modules cooperatively increase the interaction among different regions of images globally and locally. Finally, a dual branch structure for patch-weighted quality prediction is applied to predict the final score depending on the weight of each patch's score. Experimental results demonstrate that MANIQA outperforms state-of-the-art methods on four standard datasets (LIVE, TID2013, CSIQ, and KADID-10K) by a large margin. Besides, our method ranked first place in the final testing phase of the NTIRE 2022 Perceptual Image Quality Assessment Challenge Track 2: No-Reference.

Network Architecture

image.png

Dataset

The training set is PIPAL22 and the validation dataset is PIPAL21. We have conducted experiments on LIVE, CSIQ, TID2013 and KADID-10K datasets.

NOTE:

  • Put the MOS label and the data python files into data folder.
  • The validation dataset comes from NTIRE 2021. If you want to reproduce the results on validation or test set for NTIRE 2022 NR-IQA competition, register the competition and upload the submission.zip by following the instruction on the website.

Training

Training MANIQA model:

# Modify train dataset path (PIPAL21 training dataset (PIPAL21 training dataset is same as PIPAL22)): "train_dis_path"
# Modify validation dataset path (PIPAL21 validation dataset): "val_dis_path"

python train_maniqa.py

Inference for PIPAL22 Validing and Testing

Generating the ouput file:

# Modify the path of dataset "test_dis_path"
# Modify the trained model path "model_path"

python inference.py

Results

image.png

Environments

  • Platform: PyTorch 1.8.0
  • Language: Python 3.7.9
  • Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-104-generic x86_64)
  • CUDA Version 11.2
  • GPU: NVIDIA GeForce RTX 3090 with 24GB memory

Requirements

Python requirements can installed by:

pip install -r requirements.txt

Citation

@inproceedings{yang2022maniqa,
  title={MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment},
  author={Yang, Sidi and Wu, Tianhe and Shi, Shuwei and Lao, Shanshan and Gong, Yuan and Cao, Mingdeng and Wang, Jiahao and Yang, Yujiu},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={1191--1200},
  year={2022}
}

Acknowledgment

Our codes partially borrowed from anse3832 and timm.

Related Work

NTIRE2021 IQA Full-Reference Competition

[CVPRW 2021] Region-Adaptive Deformable Network for Image Quality Assessment (4th place in FR track)

paper code

NTIRE2022 IQA Full-Reference Competition

[CVPRW 2022] Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network. (1th place in FR track)

paper code

maniqa's People

Contributors

shuweis avatar stephen0808 avatar tianhewu avatar

Stargazers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.