Multiview Contrastive Learning for Completely Blind Video Quality Assessment of User Generated Content

Overview

• This paper presents a self-supervised multiview contrastive learning framework to learn spatio-temporal quality representations for completely blind video quality assessment (VQA).
• Completely blind VQA refers to quality assessment methods that do not use any reference videos, human opinion scores, or training videos from the target database.
• The authors focus on applying this approach to user-generated content, where several deep feature extraction methods have been explored in supervised and weakly supervised settings but not in the completely blind VQA context.

Plain English Explanation

The paper discusses a new way to assess the quality of videos without using any reference videos, human ratings, or training data from the specific videos being evaluated. This is particularly useful for assessing the quality of user-generated content, where a lot of videos may be created without any reference material available.

The key idea is to use a self-supervised multiview contrastive learning framework to learn features that capture the spatial and temporal information in the videos. This involves treating different representations of the video, like frame differences and optical flow, as different "views" of the same content and learning features that capture the common information between these views.

These learned features are then compared to a collection of high-quality natural video patches to predict the quality of the distorted video. This approach does not require any human-provided quality scores or reference videos, which makes it very flexible and able to generalize well to different datasets.

Technical Explanation

The authors present a self-supervised multiview contrastive learning framework to learn spatio-temporal quality representations for completely blind VQA. They capture the common information between frame differences and frames, as well as between frame differences and optical flow, and use these shared representations to predict the quality of distorted videos.

Specifically, they treat frame differences and frames as a pair of views, and similarly obtain shared representations between frame differences and optical flow. These learned features are then compared to a corpus of pristine natural video patches to predict the quality of the distorted video.

The authors evaluate their method on multiple camera-captured VQA datasets and show that it outperforms other feature-based approaches when evaluated without using any training on human scores. This demonstrates the superior generalization of their completely blind VQA approach.

Critical Analysis

The paper presents a novel and promising approach to completely blind VQA, which is an important problem as it can enable quality assessment in scenarios where reference videos or human ratings are not available. The authors' use of self-supervised multiview contrastive learning is a well-justified and technically sound approach.

One potential limitation is that the method still requires a corpus of high-quality natural video patches, which may not always be readily available. Additionally, the paper does not explore the sensitivity of the method to the size or composition of this reference corpus.

Further research could investigate the effect of sharpness on the performance of completely blind VQA methods, as well as explore the use of recurrent memory transformers or learned scanpaths to enhance the quality representations.

Additionally, a more in-depth analysis of video quality datasets used in the evaluation could provide further insights into the strengths and limitations of the proposed approach.

Conclusion

This paper presents a novel self-supervised multiview contrastive learning framework for completely blind video quality assessment. By learning spatio-temporal quality representations without relying on reference videos, human opinion scores, or training data from the target database, the proposed method demonstrates superior generalization performance across various datasets.

The ability to perform quality assessment without any reference material is a significant advancement, as it enables the application of VQA techniques to a wider range of user-generated content scenarios. The authors' work contributes to the growing field of completely blind VQA and paves the way for further research into enhancing the robustness and versatility of video quality assessment methods.