Search

Screen content video (SCV) has drawn much more attention than ever during the COVID-19 period and has evolved from a niche to a mainstream due to the recent proliferation of remote offices, online meetings, shared-screen collaboration, and gaming live streaming. Therefore, quality assessments for screen content media are highly demanded to maintain service quality recently. Although many practical natural scene video quality assessment methods have been proposed and achieved promising results, these methods cannot be applied to the screen content video quality assessment (SCVQA) task directly since the content characteristics of SCV are substantially different from natural scene video. Besides, only one no-reference SCVQA (NR-SCVQA) method, which requires handcrafted features, has been proposed in the literature. Therefore, we propose the first deep learning approach explicitly designed for NR-SCVQA. First, a multi-channel convolutional neural network (CNN) model is used to extract spatial quality features of pictorial and textual regions separately. Since there is no human annotated quality for each screen content frame (SCF), the CNN model is pre-trained in a multi-task self-supervised fashion to extract spatial quality feature representation of SCF. Second, we propose a time-distributed CNN transformer model (TCNNT) to further process all SCF spatial quality feature representations of an SCV and learn spatial and temporal features simultaneously so that high-level spatiotemporal features of SCV can be extracted and used to assess the whole SCV quality. Experimental results demonstrate the robustness and validity of our model, which is closely related to human perception.

Compound facial expressions involve combinations of basic emotions, posing challenges to automatic facial expression recognition research. The focus of existing studies in facial expression recognition remains primarily on classifying basic or single expressions, which limits its application to compound facial expression recognition. Moreover, some compound facial expression recognition methods rely on laboratory-controlled expression data, narrowing their generalizability to real-world scenarios. In this work, we carry out the task of compound facial expression recognition in unconstrained environments. To this end, we devise a new loss function, specifically for compound facial expression recognition, called bi-center loss, which is built upon center loss. Unlike center loss that considers all categories, bi-center loss enables deep neural networks to learn compound emotion features by leveraging basic emotion centers. Additionally, we introduce a basic-center regularization term, based on the variance among the basic centers, to ensure appropriate discriminative capabilities of the learned features. Experiments conducted on unconstrained compound expression datasets demonstrate the effectiveness of the proposed method over the baselines, achieving state-of-the-art performance in compound facial expression recognition.