njuvision / qstar

We have witnessed the exponential growth of the video applications. Particularly, panoramic and omnidirectional videos and images (ODV or ODI) are becoming increasingly popular. Therefore, we have extended the QSTAR model to immersive viewing, including a viewport video subjective quality assessment database, a quality assessment method for viewport video and a quality assessment method for ODV.

Database

We first provides a new Viewport-based Omnidirectional Video Quality Assessment (VOD-VQA) database, which includes eighteen raw viewport videos covering the salient regions of omnidirectional video (ODV) contents, and corresponding 774 impaired samples generated by compressing the raw content using a variety of combinations of its Spatial (frame size, s), Temporal (frame rate, t), and Amplitude (quantization step-size, q) Resolutions (STAR). Total 160 subjects have participated to assess the processed viewport videos rendered on the head mounted display (HMD) when they stabilized their fixations to the salient areas.

Download:

MOS data

Test Viewport videos

Validation Viewport videos

Demo ODV and Saliency Map in Main Function

Model

Quality Model for Viewport Video

We have developed an analytical model to connect the perceptual quality of a compressed viewport video with its STAR variables. All four model parameters can be linearly predicted using extracted content features, making the proposed metric generalized to various contents. This model correlates well with the mean opinion scores (MOSs) of viewport videos.

Quality Model for ODV

The viewport-based quality model can be easily extended to infer the overall ODV quality by linearly weighing the saliency-aggregated qualities of salient viewports and the quality of quick-scanning (or non-salient) area. Experiments have shown that inferred model correlates with the collected MOS very well with competitive performance to the state-of-the-art algorithm, across another four independent and third-party ODV assessment datasets, including IPP_IVQD, IVQAD2017, ISTOmnidirectional and VQA-ODV.

The viewport videos we used are able to watch online in our home page.

Related Scripts

We have uploaded the relevant scripts for the model calculation. The main function is QSTAR_ODV_com.m.

Notes

• Make sure you have FFmpeg environment variables in your system.
• We used the saliency prediction method in "Saliency in VR: How Do People Explore Virtual Environments?". Other methods for ODV saliency prediction are also effective, which will have an impact on the accuracy of the final result .
• The input ODV size for the above scripts is 3840x1920. The corresponding parameters can be set in QSTAR_ODV_com.m and viewport_extract.m.

qstar's Issues

permission

The download of the database currently does not have permission, can you please provide the permission?