Search

Your search keyword '"Pantic, Maja"' showing total 275 results
Start Over Author "Pantic, Maja" Publisher ieee
275 results on '"Pantic, Maja"'

5. SynthVSR: Scaling Up Visual Speech RecognitionWith Synthetic Supervision

Author

Liu, Xubo, primary, Lakomkin, Egor, additional, Vougioukas, Konstantinos, additional, Ma, Pingchuan, additional, Chen, Honglie, additional, Xie, Ruiming, additional, Doulaty, Morrie, additional, Moritz, Niko, additional, Kolar, Jachym, additional, Petridis, Stavros, additional, Pantic, Maja, additional, and Fuegen, Christian, additional

Published
2023
Full Text
View/download PDF

6. 4DME: A Spontaneous 4D Micro-Expression Dataset With Multimodalities.

Author

Li, Xiaobai, Cheng, Shiyang, Li, Yante, Behzad, Muzammil, Shen, Jie, Zafeiriou, Stefanos, Pantic, Maja, and Zhao, Guoying

Abstract

Micro-expressions (ME) are a special form of facial expressions which may occur when people try to hide their true feelings for some reasons. MEs are important clues to reveal people's true feelings, but are difficult or impossible to be captured by ordinary persons with naked-eyes as they are very short and subtle. It is expected that robust computer vision methods can be developed to automatically analyze MEs which requires lots of ME data. The current ME datasets are insufficient, and mostly contain only one single form of 2D color videos. Researches on 4D data of ordinary facial expressions have prospered, but so far no 4D data is available in ME study. In the current study, we introduce the 4DME dataset: a new spontaneous ME dataset which includes 4D data along with three other video modalities. Both micro- and macro-expression clips are labeled out in 4DME, and 22 AU labels and five categories of emotion labels are annotated. Experiments are carried out using three 2D-based methods and one 4D-based method to provide baseline results. The results indicate that the 4D data can potentially benefit ME recognition. The 4DME dataset could be used for developing 4D-based approaches, or exploring fusion of multiple video sources (e.g., texture and depth) for the task of ME analysis in future. Besides, we also emphasize the importance of forming a clear and unified criteria of ME annotation for future ME data collection studies. Several key questions related with ME annotation are listed and discussed in depth, especially about the relationship between AUs and ME emotion categories. A preliminary AU-Emo mapping table is proposed with justified explanations and supportive experimental results. Several unsolved issues are also summarized for future work. [ABSTRACT FROM AUTHOR]

Published
2023
Full Text
View/download PDF

9. Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection

Author

Haliassos, Alexandros, Mira, Rodrigo, Petridis, Stavros, and Pantic, Maja

Subjects
FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
Abstract

One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real videos to learn, in a self-supervised cross-modal manner, temporally dense video representations that capture factors such as facial movements, expression, and identity. Second, we use these learned representations as targets to be predicted by our forgery detector along with the usual binary forgery classification task; this encourages it to base its real/fake decision on said factors. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments, and examine the factors that contribute to its performance. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors., Comment: CVPR 2022. Code: https://github.com/ahaliassos/RealForensics

Published
2022

11. Does Visual Self-Supervision Improve Learning of Speech Representations for Emotion Recognition?

Author

Shukla, Abhinav, Petridis, Stavros, and Pantic, Maja

Abstract

Self-supervised learning has attracted plenty of recent research interest. However, most works for self-supervision in speech are typically unimodal and there has been limited work that studies the interaction between audio and visual modalities for cross-modal self-supervision. This article (1) investigates visual self-supervision via face reconstruction to guide the learning of audio representations; (2) proposes an audio-only self-supervision approach for speech representation learning; (3) shows that a multi-task combination of the proposed visual and audio self-supervision is beneficial for learning richer features that are more robust in noisy conditions; (4) shows that self-supervised pretraining can outperform fully supervised training and is especially useful to prevent overfitting on smaller sized datasets. We evaluate our learned audio representations for discrete emotion recognition, continuous affect recognition and automatic speech recognition. We outperform existing self-supervised methods for all tested downstream tasks. Our results demonstrate the potential of visual self-supervision for audio feature learning and suggest that joint visual and audio self-supervision leads to more informative audio representations for speech and emotion recognition. [ABSTRACT FROM AUTHOR]

Published
2023
Full Text
View/download PDF

20. Editorial: Intelligent Signal Analysis for Contagious Virus Diseases.

Author

Schuller, Bjorn W., Eldar, Yonina, Pantic, Maja, Narayanan, Shrikanth, Virtanen, Tuomas, and Tao, Jianhua

Abstract

COVID-19 infection's recent outbreak triggered by the SARS-CoV-2 Corona virus had already led to more than two million reported infected individuals when we first addressed the community by our call – by now, the number sadly rose to roughly half a billion cases worldwide. The outbreak of COIVD-19 has also re-shaped and accelerated the scientific publication landscape in no time. One can observe a massive uprise in interest in work related to the topic of highly contagious virus diseases and potential contributions of digital health including intelligent signal processing. In addition, most publishers have reacted in one or the other way to the crises such as by opening up to pre-prints, waiving publication fees for COVID-19-related research, providing search functions and tools for COVID-19 research, and many more. Here, we gathered 13 carefully selected novel contributions across signal types such as audio, speech, image, video, or symbolic information, as well as their multimodal combination for application in the risk assessment, diagnosis, and monitoring of contagious virus diseases. [ABSTRACT FROM AUTHOR]

Published
2022
Full Text
View/download PDF

27. Towards Pose-Invariant Lip-Reading

Author

Cheng, Shiyang, primary, Ma, Pingchuan, additional, Tzimiropoulos, Georgios, additional, Petridis, Stavros, additional, Bulat, Adrian, additional, Shen, Jie, additional, and Pantic, Maja, additional

Published
2020
Full Text
View/download PDF

35. 4DFAB: A Large Scale 4D Facial Expression Database for Biometric Applications

Author

Cheng, Shiyang, Kotsia, Irene, Pantic, Maja, Zafeiriou, Stefanos, Chellappa, Rama, Zhang, Zhengyou, and Hoogs, Anthony

Abstract

The progress we are currently witnessing in many computer vision applications, including automatic face analysis, would not be made possible without tremendous efforts in collecting and annotating large scale visual databases. To this end, we propose 4DFAB, a new large scale database of dynamic high-resolution 3D faces (over 1,800,000 3D meshes). 4DFAB contains recordings of 180 subjects captured in four different sessions spanning over a five-year period. It contains 4D videos of subjects displaying both spontaneous and posed facial behaviours. The database can be used for both face and facial expression recognition, as well as behavioural biometrics. It can also be used to learn very powerful blendshapes for parametrising facial behaviour. In this paper, we conduct several experiments and demonstrate the usefulness of the database for various applications. The database will be made publicly available for research purposes.

Published
2017

37. Blind Audio–Visual Localization and Separation via Low-Rank and Sparsity.

Author

Pu, Jie, Panagakis, Yannis, Petridis, Stavros, Shen, Jie, and Pantic, Maja

Abstract

The ability to localize visual objects that are associated with an audio source and at the same time to separate the audio signal is a cornerstone in audio–visual signal-processing applications. However, available methods mainly focus on localizing only the visual objects, without audio separation abilities. Besides that, these methods often rely on either laborious preprocessing steps to segment video frames into semantic regions, or additional supervisions to guide their localization. In this paper, we aim to address the problem of visual source localization and audio separation in an unsupervised manner and avoid all preprocessing or post-processing steps. To this end, we devise a novel structured matrix decomposition method that decomposes the data matrix of each modality as a superposition of three terms: 1) a low-rank matrix capturing the background information; 2) a sparse matrix capturing the correlated components among the two modalities and, hence, uncovering the sound source in visual modality and the associated sound in audio modality; and 3) a third sparse matrix accounting for uncorrelated components, such as distracting objects in visual modality and irrelevant sound in audio modality. The generality of the proposed method is demonstrated by applying it onto three applications, namely: 1) visual localization of a sound source; 2) visually assisted audio separation; and 3) active speaker detection. Experimental results indicate the effectiveness of the proposed method on these application domains. [ABSTRACT FROM AUTHOR]

Published
2020
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results