Author: "Image Perception Interaction (LS2N - équipe IPI)" - Searchworks@Jio Institute Digital Library Search Results

Your search keyword '"Image Perception Interaction (LS2N - équipe IPI)"' showing total 61 results

Start Over Author "Image Perception Interaction (LS2N - équipe IPI)"

61 results on '"Image Perception Interaction (LS2N - équipe IPI)"'

1. Denoised CT Images Quality Assessment Through COVID-19 Pneumonia Detection Task

Author: Xia, Lumi, Jebbari, Houda, Lévêque, Lucie, Deforges, Olivier, Outtas, Meriem, Zhang, Lu, Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: COVID-19 classification, image denoising, image quality assessment, task-based quality, computed tomography, [INFO]Computer Science [cs]
Abstract: International audience; Medical images largely contribute to the diagnosis of lung diseases, especially pneumonia, an inflammation of lungs tissue. Since the emergence of COVID-19 in late 2019, medical imaging systems, notably computed tomography (CT) scans, have considerably helped in its diagnosis as well as revealing its infection severity. Serving as such an important role in clinical practice, the quality of medical images is therefore crucial for an accurate diagnosis. Denoising techniques, as a common image processing method, are being more and more used in medical imaging. However, how image denoising technique influences medical images' quality in terms of diagnostic performance still remains to be answered. In this paper, a primary study was carried out thanks to a detection task-based image quality assessment experiment, where we explored the performance of COVID-19 classifiers on both original and denoised chest CT scans. Two different denoising methods, i.e., anisotropic diffusion (AD) and total variation (TV) filters, were used. Results showed that the TV denoised model performed better than both baseline and AD denoised model, despite its less favorable mathematical image quality metrics.
Published: 2023

2. Towards guidelines for subjective Haptic quality assessment: a case study on quality assessment of compressed haptic signals

Author: Pastor, Andreas, Le Callet, Patrick, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: Haptic quality assessment human perception compression performance assessment quality of experience, compression performance assessment, human perception, [INFO]Computer Science [cs], quality of experience, Haptic quality assessment
Abstract: Modern systems are multimodal (e.g., video, audio, smell), and haptic feedback provides the user with additional entertainment and sensory immersion. Standard recommendation groups extensively studied and focused on video and audio subjective quality assessment, especially in signal transmission. In that context, subjective quality assessment and Quality of Experience (QoE) of Haptic signals is at its infant age. We propose further analyzing the collected data from a recent subjective quality assessment campaign as part of the MPEG haptic standardization group. In particular, we are addressing the following questions: 1) How the emerging field of haptic signal QoE can benefit from existing efforts of video and audio quality assessment standards? 2) How to detect possible outliers or characterize the rater's reliability? 3) How does the discriminability of haptic tests increases with the number of raters? Towards this goal, we question if traditional analysis as proposed for audio or video signal are suitable, as well as other state-of-the-art techniques. We also compare the discriminability of the haptics quality assessment tests with other modalities such as audio, video, and immersive content (360°contents). We propose recommendations on the number of raters required to meet the usual discriminability obtained for other perceptual modalities and how to process ratings to remove possible noise and biases. These results could feed future recommendations in standards such as BT500-14 or P.913 but for haptic signals.
Published: 2023

3. On Legal and Ethical Challenges of Automatic Facial Expression Recognition: An Exploratory Study

Author: Boutin, Alex, Desmoulin-Canselier, Sonia, Lévêque, Lucie, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Droit et changement social (DCS), Centre National de la Recherche Scientifique (CNRS)-Nantes Université - UFR Droit et Sciences Politiques (Nantes Univ - UFR DSP), Nantes Université - pôle Sociétés, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sociétés, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Ethics, [SHS.DROIT]Humanities and Social Sciences/Law, Emotions, Facial expression recognition (FER), [INFO]Computer Science [cs], Law, Artificial intelligence (AI)
Abstract: International audience; Automatic facial expression recognition (FER) has a lot of potential applications. However, even if it can be beneficial for some areas, e.g. security and healthcare, several legal and ethical challenges arise. In this article, we first present such challenges related to the deployment of FER. Then, we introduce the conduct of a focus group which allowed to highlight interesting points regarding the use of FER in a medical context. Particularly, transparency, data management, diagnoses, liability, best endeavours obligation, and non-discrimination principle are debated. We finally discuss on our study's limitations and directions for future work. CCS Concepts: • Applied computing → Computers in other domains; • Social and professional topics → Intellectual property.
Published: 2023

4. A Dataset of Gaze and Mouse Patterns in the Context of Facial Expression Recognition

Author: Bruckert, Alexandre, Lévêque, Lucie, Perreira Da Silva, Matthieu, Le Callet, Patrick, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: facial expression recognition (FER), eye-tracking, BubbleView, dataset, [INFO]Computer Science [cs], emotions
Abstract: International audience; Facial expression recognition is an important and challenging task for both the computer vision and affective computing communities, and even more specifically in the context of multimedia applications, where audience understanding is of particular interest. Recent data-oriented approaches have created the need for large-scale annotated datasets. However, most existing datasets present some weaknesses, because of the collecting methods used. In order to further highlight these issues, we investigate in this work how human visual attention is deployed when performing a facial expression recognition task. To do so, we carried out several complementary experiments, using the eye-tracking technology, as well as the BubbleView metaphor, both under laboratory and crowdsourcing settings. We show significant variations in gaze patterns depending on the emotion represented, but also on the difficulty of the task, i.e., whether the emotion is correctly recognised or not. Moreover, we use these results to propose recommendations on the ways to collect label data for facial expression recognition datasets.
Published: 2023

5. VSGAN: Visual Saliency guided Generative Adversarial Network for data augmentation

Author: Zhang, Jun, Xian, Chuhua, Bruckert, Alexandre, Le Callet, Patrick, Li, Guiqin, Cai, Hongmin, South China University of Technology [Guangzhou] (SCUT), Nantes Université - Ecole Polytechnique de l'Université de Nantes (Nantes Univ - EPUN), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), and Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST)
Subjects: Visual saliency, image inpainting, GAN, data augmentation, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: Deep learning approaches have allowed for a great leap in the performances of visual saliency models. However, the lack of annotated data remains the main challenge for visual saliency prediction. In this paper, we leverage image inpainting methods to synthesize augmented images, which is done by completing the weakly-salient areas, and propose a Visual Saliency guided Generative Adversarial Network (VSGAN) that contains a dual encoder to extract multiscale features and a generator equipped with visual saliency guided modulation to synthesize high fidelity and diversity results. Extensive experimental results show that our method outperforms state-of-the-art methods for image inpainting on visual saliency datasets, and demonstrate the effectiveness of VSGAN for visual saliency data augmentation both quantitatively and qualitatively. CCS CONCEPTS • Computing methodologies → Interest point and salient region detections.
Published: 2023
Full Text: View/download PDF

6. Subjective Test Environments: A Multifaceted Examination of Their Impact on Test Results

Author: Zhu, Jingwen, Ak, Ali, Dormeval, Charles, Le Callet, Patrick, Kumar, Rahul, Sethuraman, Sriram, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), CAPACITÉS SAS, Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), amazon prime video, and ACM
Subjects: HD, Subjective test HD Video Quality Assessment, Subjective test, [INFO.INFO-MM]Computer Science [cs]/Multimedia [cs.MM], Video Quality Assessment
Abstract: Quality of Experience (QoE) in video streaming scenarios is significantly affected by the viewing environment and display device. Understanding and measuring the impact of these settings on QoE can help develop viewing environment-aware metrics and improve the efficiency of video streaming services. In this ongoing work, we conducted a subjective study in both laboratory and home settings using the same content and design to measure QoE in Degradation Category Rating (DCR). We first analyzed subject inconsistency and confidence intervals of the Mean Opinion Scores (MOS) between the two settings. We then used statistical models such as ANOVA and t-test to analyze the differences in subjective tests on video quality between the two viewing environments. Additionally, we employed the Eliminated-By-Aspects (EBA) model to quantify the influence of different settings on the measured QoE. We conclude with several research questions that could be further explored to better understand the impact of the viewing environment on QoE.
Published: 2023
Full Text: View/download PDF

7. High-level cinematic knowledge to predict inter-observer visual congruency

Author: Bruckert, Alexandre, Christie, Marc, Nantes Université - Ecole Polytechnique de l'Université de Nantes (Nantes Univ - EPUN), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nous, virtuels (VIRTUS), Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Mouvement Sport Santé (M2S), Université de Rennes (UR)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-Université de Rennes (UR)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-RÉALITÉ VIRTUELLE, HUMAINS VIRTUELS, INTERACTIONS ET ROBOTIQUE (IRISA-D5), Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)
Subjects: gaze congruency, cinematography, neural networks, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: International audience; When watching the same visual stimulus, humans can exhibit a wide range of gaze behaviors. These variations can be caused by bottom-up factors (i.e. features of the stimulus itself) or top-down factors (i.e. characteristics of the observers). Inter-observer visual congruency is a measure of this range. Moreover, it has been shown that cinematic techniques, such as camera motion or shot editing, have a significant impact on this measure [17]. In this work, we first propose a metric for measuring IOC in videos, taking into account the dynamic nature of the stimuli. Then, we propose a model for predicting inter-observer visual congruency in the context of feature films, by using high-level cinematic annotation as prior information in a deep learning framework.
Published: 2023
Full Text: View/download PDF

8. Perceptual annotation of local distortions in videos: tools and datasets

Author: Pastor, Andréas, Le Callet, Patrick, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: Multimedia, Subjective methodology, [INFO.INFO-MM]Computer Science [cs]/Multimedia [cs.MM], Perception, Quality
Abstract: International audience; To assess the quality of multimedia content, create datasets, and train objective quality metrics, one needs to collect subjective opinions from annotators. Different subjective methodologies exist, from direct rating with single or double stimuli to indirect rating with pairwise comparisons. Triplet and quadruplet-based comparisons are a type of indirect rating. From these comparisons and preferences on stimuli, we can place the assessed stimuli on a perceptual scale (e.g., from low to high quality). Maximum Likelihood Difference Scaling (MLDS) solver is one of these algorithms working with triplets and quadruplets. A participant is asked to compare intervals inside pairs of stimuli: (a,b) and (c,d), where a,b,c,d are stimuli forming a quadruplet. However, one limitation is that the perceptual scales retrieved from stimuli of different contents are usually not comparable. We previously offered a solution to measure the inter-content scale of multiple contents. This paper presents an open-source python implementation of the method and demonstrates its use on three datasets collected in an in-lab environment. We compared the accuracy and effectiveness of the method using pairwise, triplet, and quadruplet for intra-content annotations. The code is available here: https:// github.com/ andreaspastor/ MLDS_inter_content_scaling.
Published: 2023
Full Text: View/download PDF

9. Could the bubbleview metaphor be used to infer visual attention on 3D graphical content ?

Author: Alexandre Bruckert, Mona Abid, Matthieu Perreira Da Silva, Patrick Le Callet, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: eye-tracking, 3D graphical objects, [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], Visual attention, visual saliency
Abstract: International audience; Understanding the deployment of human gaze on 3D graphical objects is of critical importance in order to propose rich and complex 3D environments without strong latency nor rendering constraints. However, the data needed to study this gaze deployment can be costly and difficult to obtain, especially in the context of the Covid-19 pandemic where in-lab experiments are strongly discouraged. In order to alleviate these issues, we propose to use the BubbleView metaphor as a way of crowdsourcing visual attention data on 3D graphical content. In this paper, we question the adequacy of this method to provide a reliable proxy for visual attention in the context of 3D graphical objects. Moreover, we show that data obtained in this manner can be used to train visual saliency models, with only a slight tradeoff in performances compared to the use of ground-truth eye-tracking data.
Published: 2023

10. The Salient360! Toolbox: Processing, Visualising and Comparing Gaze Data in 3D

Author: Erwan David, Jesús Gutiérrez, Melissa Le-Hoa Vo, Antoine Coutrot, Matthieu Perreira Da Silva, Patrick Le Callet, Goethe-University Frankfurt am Main, Universidad Politécnica de Madrid (UPM), Laboratoire d'InfoRmatique en Image et Systèmes d'information (LIRIS), Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL), Université de Lyon-Université de Lyon-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS), Situated Interaction, Collaboration, Adaptation and Learning (SICAL), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS)-Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL), Centre National de la Recherche Scientifique (CNRS), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: head tracking, visualisation, comparison, 360 stimuli, toolbox, processing, [INFO]Computer Science [cs], toolbox gaze tracking head tracking 360 stimuli processing comparison visualisation, gaze tracking
Abstract: International audience; Eye tracking can serve as a gateway to studying the mind. For this reason it has been adopted by a diverse range of scientific communities. With the improvement of the quality of head-mounted virtual reality devices (HMDs) over the past 10 years, eye tracking has been added to capture gaze in immersive environments. The use of HMDs with eye tracking is increasing significantly and so is the need for a toolbox enabling consensus about eye tracking methods in 3D. We present the Salient360! toolbox: it implements functions to identify saccades and fixations and output gaze characteristics (e.g., fixation duration or saccade directions), to generate saliency maps, fixation maps, and scanpath data. It also implements routines made to compare gaze data that were adapted to 3D. We hope that this toolbox will spark discussions about the methodology of 3D gaze processing, facilitate running experiments, and improve the gaze study in 3D. https:// github.com/ David-Ef/ salient360Toolbox CCS CONCEPTS • Software and its engineering → Software libraries and repositories; • Human-centered computing → Visualization toolkits; • Information systems → Extraction, transformation and loading.
Published: 2023
Full Text: View/download PDF

11. Déploiement de l’IA en situation de travail : une trop faible considération de l’expérience des employé·es ?

Author: Agossah, Alexandre, Krupa, Frédérique, Perreira Da Silva, Matthieu, Deconde, Guillaume, Le Callet, Patrick, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), École de design Nantes Atlantique, and Groupe SIGMA
Subjects: Artificial Intelligence Professional context UX Design, [INFO]Computer Science [cs]
Abstract: International audience
Published: 2023
Full Text: View/download PDF

12. VIRTUAL DATA SPHERE: INVERSE STEREOGRAPHIC PROJECTION FOR IMMERSIVE MULTI-PERSPECTIVE GEOVISUALIZATION

Author: Spur, Maxim, Tourre, Vincent, Moreau, Guillaume, Le Callet, Patrick, Centre de recherche nantais Architectures Urbanités (CRENAU ), Ambiances, Architectures, Urbanités (AAU), Ministère de la Culture et de la Communication (MCC)-Centre National de la Recherche Scientifique (CNRS)-École nationale supérieure d'architecture de Grenoble (ENSAG ), Université Grenoble Alpes (UGA)-Université Grenoble Alpes (UGA)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-NANTES UNIVERSITÉ - École nationale supérieure d'architecture de Nantes (Nantes Univ - ENSA Nantes), Nantes Université (Nantes Univ)-Ministère de la Culture et de la Communication (MCC)-Centre National de la Recherche Scientifique (CNRS)-École nationale supérieure d'architecture de Grenoble (ENSAG ), Nantes Université (Nantes Univ), École Centrale de Nantes (Nantes Univ - ECN), Equipe Immersive Natural User Interaction team (Lab-STICC_INUIT), Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance (Lab-STICC), École Nationale d'Ingénieurs de Brest (ENIB)-Université de Bretagne Sud (UBS)-Université de Brest (UBO)-École Nationale Supérieure de Techniques Avancées Bretagne (ENSTA Bretagne)-Institut Mines-Télécom [Paris] (IMT)-Centre National de la Recherche Scientifique (CNRS)-Université Bretagne Loire (UBL)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-École Nationale d'Ingénieurs de Brest (ENIB)-Université de Bretagne Sud (UBS)-Université de Brest (UBO)-École Nationale Supérieure de Techniques Avancées Bretagne (ENSTA Bretagne)-Institut Mines-Télécom [Paris] (IMT)-Centre National de la Recherche Scientifique (CNRS)-Université Bretagne Loire (UBL)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT), Département Informatique (IMT Atlantique - INFO), IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Geovisualization, Immersive Analytics, Virtual Reality, Focus+Context, Multi-Perspective Views, [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR]
Abstract: Immersive geospatial visualization finds increasing application for navigation, exploration, and analysis. Many such require the display of data at different scales, often in views with three-dimensional geometry. Multi-view solutions, such as focus+context, overview+detail, and distorted projections can show different scales at the same time, and help place an area of interest within its surroundings. By inverting the principle of stereographic projection – projecting spatial features from a map onto a virtual sphere which surrounds the viewer – we present a novel technique for immersive geospatial focus+context that aims to mitigate problems with existing solutions. This sphere can intersect the map, dividing it into two parts: the inside of the sphere, which stays unchanged, and the outside, which gets projected to the surface, resulting in an inversion of the lens metaphor by distorting the context instead of the focus. This detail-in-context visualization maximizes the amount of context that can be legibly shown by the smooth compression inherent to the stereographic projection, and by utilizing otherwise unused screen space in the sky. The projection method allows for easy control over the projection and distortion characteristics by varying only two main parameters – the sphere’s radius and its position. The omnidirectional nature of our system makes it particularly well-suited for immersive displays by accommodating typical immersive exploration and fully utilizing the additional visual space available. Applying our system to an urban environment, we were able to solicit positive reactions during feedback sessions with experts from urbanism.
Published: 2022
Full Text: View/download PDF

13. Applications of artificial intelligence and machine learning for the hip and knee surgeon: current state and implications for the future

Author: Christophe Nich, Julien Behr, Vincent Crenn, Nicolas Normand, Harold Mouchère, Gaspard d’Assignies, Centre hospitalier universitaire de Nantes (CHU Nantes), Service d'Orthopédie [CCOT, CHU Nantes], Centre hospitalier universitaire de Nantes (CHU Nantes)-Clinique Chirurgicale Orthopédique et Traumatologique - CCOT [CHU Nantes], Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Centre Hospitalier Départemental site de la Roche-sur-Yon (CHD de la Roche-sur-Yon), and Incepto Medical
Subjects: Machine Learning, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], Knee Joint, Lower Extremity, Artificial Intelligence, [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Humans, Orthopedics and Sports Medicine, Surgery, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], ComputingMilieux_MISCELLANEOUS, Forecasting
Abstract: Artificial Intelligence (AI)/Machine Learning (ML) applications have been proven efficient to improve diagnosis, to stratify risk, and to predict outcomes in many respective medical specialties, including in orthopaedics.Regarding hip and knee reconstruction surgery, AI/ML have not made it yet to clinical practice. In this review, we present sound AI/ML applications in the field of hip and knee degenerative disease and reconstruction. From osteoarthritis (OA) diagnosis and prediction of its advancement, clinical decision-making, identification of hip and knee implants to prediction of clinical outcome and complications following a reconstruction procedure of these joints, we report how AI/ML systems could facilitate data-driven personalized care for our patients.
Published: 2022
Full Text: View/download PDF

14. Computing and evaluating saliency maps for image classification: a tutorial

Author: Tristan Gomez, Harold Mouchère, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Electrical and Electronic Engineering, Atomic and Molecular Physics, and Optics, Computer Science Applications, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: International audience
Published: 2023
Full Text: View/download PDF

15. Real-time Multi-map Saliency-driven Gaze Behavior for Non-conversational Characters

Author: Ific Goude, Alexandre Bruckert, Anne-Helene Olivier, Julien Pettre, Remi Cozot, Kadi Bouatouch, Marc Christie, Ludovic Hoyet, Nous, virtuels (VIRTUS), Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Mouvement Sport Santé (M2S), Université de Rennes (UR)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-Université de Rennes (UR)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-RÉALITÉ VIRTUELLE, HUMAINS VIRTUELS, INTERACTIONS ET ROBOTIQUE (IRISA-D5), Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), Laboratoire Mouvement Sport Santé (M2S), Université de Rennes (UR)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique ), Laboratoire d'Informatique Signal et Image de la Côte d'Opale (LISIC), Université du Littoral Côte d'Opale (ULCO), Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-École normale supérieure - Rennes (ENS Rennes)-Université de Brest (UBO)-Université de Rennes 2 (UR2), Université de Rennes (UNIV-RENNES)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-RÉALITÉ VIRTUELLE, HUMAINS VIRTUELS, INTERACTIONS ET ROBOTIQUE (IRISA-D5), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Nantes Université - École Centrale de Nantes (Nantes Univ - ECN), and Université de Rennes (UNIV-RENNES)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )
Subjects: Signal Processing, Eye-tracking data, [INFO.INFO-MM]Computer Science [cs]/Multimedia [cs.MM], [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], Computer Vision and Pattern Recognition, Animation, Gaze behavior, Computer Graphics and Computer-Aided Design, [INFO.INFO-MO]Computer Science [cs]/Modeling and Simulation, Software, Simulation, Neural networks, Dataset
Abstract: International audience; Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.
Published: 2023
Full Text: View/download PDF

16. UIF: An Objective Quality Assessment for Underwater Image Enhancement

Author: Yannan Zheng, Weiling Chen, Rongfu Lin, Tiesong Zhao, Patrick Le Callet, Fuzhou University [Fuzhou], Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Image and Video Processing (eess.IV), Image Quality Assessment (IQA), Computer Science - Computer Vision and Pattern Recognition, FOS: Electrical engineering, electronic engineering, information engineering, underwater image processing, Underwater Image Enhancement (UIE), Electrical Engineering and Systems Science - Image and Video Processing, Computer Graphics and Computer-Aided Design, Software
Abstract: Due to complex and volatile lighting environment, underwater imaging can be readily impaired by light scattering, warping, and noises. To improve the visual quality, Underwater Image Enhancement (UIE) techniques have been widely studied. Recent efforts have also been contributed to evaluate and compare the UIE performances with subjective and objective methods. However, the subjective evaluation is time-consuming and uneconomic for all images, while existing objective methods have limited capabilities for the newly-developed UIE approaches based on deep learning. To fill this gap, we propose an Underwater Image Fidelity (UIF) metric for objective evaluation of enhanced underwater images. By exploiting the statistical features of these images, we present to extract naturalness-related, sharpness-related, and structure-related features. Among them, the naturalness-related and sharpness-related features evaluate visual improvement of enhanced images; the structure-related feature indicates structural similarity between images before and after UIE. Then, we employ support vector regression to fuse the above three features into a final UIF metric. In addition, we have also established a large-scale UIE database with subjective scores, namely Underwater Image Enhancement Database (UIED), which is utilized as a benchmark to compare all objective metrics. Experimental results confirm that the proposed UIF outperforms a variety of underwater and general-purpose image quality metrics., Comment: This paper was submitted to ACMMM 2021
Published: 2022
Full Text: View/download PDF

17. A methodology to compare XAI explanations on natural language processing

Author: Gaëlle Jouis, Harold Mouchère, Fabien Picarougne, Alexandre Hardouin, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Pôle emploi (France), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Data User Knowledge (LS2N - équipe DUKe), Jenny Benois-Pineau, Romain Bourqui, Dragutin Petkovic, and Georges Quénot
Subjects: [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: International audience; Recent advances in eXplainable Artificial Intelligence (XAI) led to many different methods to improve the explainability of deep learning algorithms. There are nowadays many options at hand, and adapting existing ones to new problems may be needed. Hence, one may find oneself in a struggle to choose the right method to generate explanations. Considered explanations may be a set of important features or factual and counterfactual explanations.This chapter presents two protocols to compare different XAI methods. The first is designed to be applied when no end-users are available. An objective, quantitative metric is being compared to an objective data expert's analysis. The second experiment is designed to take into account end-users feedback. It uses the quantitative metric applied in the first experiment and compares it to users' preferences. Then, the quantitative metric can be used to evaluate explanations, allowing to try multiple explanation methods and adjust them, without the cost of having a systematic end-users evaluation. The protocol can be applied to post-hoc explaining approaches as well as to self-explaining neural networks.Knowing if an explanation makes sense to end-users helps assess the explanation system but also the prediction itself. This chapter is focused on the methodology to assess such explanations. It helps fasten and reduce the cost of manual result analysis during system design. The first experiment shows that IOU can be used to focus on instances where compared explanations give different results. The second experiment shows IOU alone is not representative of end-users' preferences, and gives clues on how to improve the definition of this quantitative metric.
Published: 2023
Full Text: View/download PDF

18. Point to Segment Distance DTW for Online Handwriting Signals Matching

Author: mohamed moussa, elmokhtar, Lelore, Thibault, Mouchère, Harold, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), MyScript SAS, and Mohamed Moussa, Elmokhtar
Subjects: [INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing, [INFO.INFO-TS] Computer Science [cs]/Signal and Image Processing, Handwriting Matching, DTW Metric, Sampling invariance, Handwriting Matching DTW Metric Sampling invariance
Abstract: International audience; In this paper, we propose DTW seg , a modified DTW algorithm based on a point-to-segment distance instead of the euclidean point-to-point distance. Applying DTW seg to online handwriting matching proves to be advantageous compared to other algorithms as it is less sensitive to differences between signals sampling rates occurring due to acquisition frequencies or handwriting speed. It eliminates the need for a commonly practiced resampling that omits an important dynamic part of the ductus. Experiments on IRONOFF french words dataset and FLOWCHARTS dataset show DTW seg to be least impacted by sampling rate alterations. We also propose a new benchmark of state-of-the-art methods on offline handwriting to online conversion based on our new proposed metric.
Published: 2023
Full Text: View/download PDF

19. Objective quality assessment of medical images and videos: Review and challenges

Author: Rodrigues, Rafael, Lévêque, Lucie, Gutiérrez, Jesús, Jebbari, Houda, Outtas, Meriem, Zhang, Lu, Chetouani, Aladine, Al-Juboori, Shaymaa, Martini, Maria, Pinheiro, Antonio M. G., University of Beira Interior [Portugal] (UBI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Universidad Politécnica de Madrid (UPM), Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA), Université d'Orléans (UO), and Kingston University [London]
Subjects: [SCCO]Cognitive science, I.4, I.4.2, Image and Video Processing (eess.IV), FOS: Electrical engineering, electronic engineering, information engineering, [INFO]Computer Science [cs], Medical imaging, Electrical Engineering and Systems Science - Image and Video Processing, Task-based quality, Quality assessment, Objective metrics
Abstract: Quality assessment is a key element for the evaluation of hardware and software involved in image and video acquisition, processing, and visualization. In the medical field, user-based quality assessment is still considered more reliable than objective methods, which allow the implementation of automated and more efficient solutions. Regardless of increasing research in this topic in the last decade, defining quality standards for medical content remains a non-trivial task, as the focus should be on the diagnostic value assessed from expert viewers rather than the perceived quality from na\"{i}ve viewers, and objective quality metrics should aim at estimating the first rather than the latter. In this paper, we present a survey of methodologies used for the objective quality assessment of medical images and videos, dividing them into visual quality-based and task-based approaches. Visual quality based methods compute a quality index directly from visual attributes, while task-based methods, being increasingly explored, measure the impact of quality impairments on the performance of a specific task. A discussion on the limitations of state-of-the-art research on this topic is also provided, along with future challenges to be addressed., Comment: Submitted for peer review at Multimedia Tools and Applications
Published: 2022

20. Contrastive Self-Supervised Learning on Crohn's Disease Detection

Author: Xing, Jing, Mouchère, Harold, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and South China University of Technology [Guangzhou] (SCUT)
Subjects: Deep Learning, Self-Supervised Learning, Contrastive Learning, Medical images classification, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Crohn's Disease, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: International audience; Crohn's disease is a type of inflammatory bowel illness that is typically identified via computer-aided diagnosis (CAD), which employs images from wireless capsule endoscopy (WCE). While deep learning has recently made significant advancements in Crohn's disease detection, its performance is still constrained by limited labeled data. We suggest using contrastive self-supervised learning methods to address these difficulties which was barely used in detection of Crohn's disease. Besides, we discovered that, unlike supervised learning, it is difficult to monitor contrastive self-supervised pretraining process in real time. So we propose a method for evaluating the model during contrastive pretraining (EDCP) based on the Euclidean distance of the sample representation, so that the model can be monitored during pretraining. Our comprehensive experiment results show that with contrastive self-supervised learning, better results in Crohn's disease detection can be obtained. EDCP has also been shown to reflect the model's training progress. Furthermore, we discovered some intriguing issues with using contrastive selfsupervised learning for small dataset tasks in our experiments that merit further investigation.
Published: 2022

21. Comparing the Robustness of Humans and Deep Neural Networks on Facial Expression Recognition

Author: Lucie Lévêque, François Villoteau, Emmanuel V. B. Sampaio, Matthieu Perreira Da Silva, Patrick Le Callet, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: [SCCO]Cognitive science, Computer Networks and Communications, Hardware and Architecture, Control and Systems Engineering, Signal Processing, [INFO]Computer Science [cs], Electrical and Electronic Engineering, emotions, facial expression recognition (FER), image quality, robustness, crowdsourcing
Abstract: International audience; Emotion recognition, and more particularly facial expression recognition (FER), has been extensively used for various applications (e.g., human–computer interactions). The ability to automatically recognize facial expressions has been facilitated with recent progress in the fields of computer vision and artificial intelligence. Nonetheless, FER algorithms still seem to face difficulties with image degradations due to real-life conditions (e.g., because of image compression or transmission). In this paper, we propose to investigate the impact of different distortion configurations on a large number of images of faces on human performance, thanks to the conduct of a crowdsourcing experiment. We further compare human performance with two open-source FER algorithms. Results show that, overall, models are more sensitive to distortions than humans—even when fine-tuned. Furthermore, we broach the subject of annotation errors and bias which exist in several well-established datasets, and suggest approaches to improve the latter.
Published: 2022
Full Text: View/download PDF

22. Automatic Segmentation of Metastatic Breast Cancer Lesions on 18F-FDG PET/CT Longitudinal Acquisitions for Treatment Response Assessment

Author: Noémie Moreau, Caroline Rousseau, Constance Fourcade, Gianmarco Santini, Aislinn Brennan, Ludovic Ferrer, Marie Lacombe, Camille Guillerminet, Mathilde Colombié, Pascal Jézéquel, Mario Campone, Nicolas Normand, Mathieu Rubeaux, Keosys, Nantes Université (Nantes Univ), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut de Cancérologie de l'Ouest [Angers/Nantes] (UNICANCER/ICO), UNICANCER, and Université d'Angers (UA)
Subjects: Cancer Research, automatic segmentation, Oncology, disease monitoring, deep learning, metastatic breast cancer, imaging biomarkers, Neoplasms. Tumors. Oncology. Including cancer and carcinogens, [SDV.CAN]Life Sciences [q-bio]/Cancer, Article, RC254-282, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
Abstract: Simple Summary In the recent years, several deep learning methods for medical image segmentation have been developed for different purposes such as diagnosis, radiotherapy planning or to correlate images findings with other clinical data. However, few studies focus on longitudinal images and response assessment. To the best of our knowledge, this is the first study to date evaluating the use of automatic segmentation to obtain imaging biomarkers that can be used to assess treatment response in patients with metastatic breast cancer. Moreover, the statistical analysis of the different biomarkers shows that automatic segmentation can be successfully used for their computation, reaching similar performances compared to manual segmentation. Analysis also demonstrated the potential of the different biomarkers including novel/original ones to determine treatment response. Abstract Metastatic breast cancer patients receive lifelong medication and are regularly monitored for disease progression. The aim of this work was to (1) propose networks to segment breast cancer metastatic lesions on longitudinal whole-body PET/CT and (2) extract imaging biomarkers from the segmentations and evaluate their potential to determine treatment response. Baseline and follow-up PET/CT images of 60 patients from the EPICUREseinmeta study were used to train two deep-learning models to segment breast cancer metastatic lesions: One for baseline images and one for follow-up images. From the automatic segmentations, four imaging biomarkers were computed and evaluated: SULpeak, Total Lesion Glycolysis (TLG), PET Bone Index (PBI) and PET Liver Index (PLI). The first network obtained a mean Dice score of 0.66 on baseline acquisitions. The second network obtained a mean Dice score of 0.58 on follow-up acquisitions. SULpeak, with a 32% decrease between baseline and follow-up, was the biomarker best able to assess patients’ response (sensitivity 87%, specificity 87%), followed by TLG (43% decrease, sensitivity 73%, specificity 81%) and PBI (8% decrease, sensitivity 69%, specificity 69%). Our networks constitute promising tools for the automatic segmentation of lesions in patients with metastatic breast cancer allowing treatment response assessment with several biomarkers.
Published: 2022
Full Text: View/download PDF

23. Improving Oracle Bone Characters Recognition via A CycleGAN-based Data Augmentation Method

Author: Wang, Wei, Zhang, Ting, Jin, Xinxin, Mouchère, Harold, Yu, Xinguo, Central China Normal University [Wuhan, China], Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: Class imbalance, Data augmentation, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], CycleGAN, Characters recognition Class imbalance Data augmentation CycleGAN, Characters recognition, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE]
Abstract: International audience; Oracle bone inscription is the earliest writing system in China which contains rich information about the history of Shang dynasty. Automatically recognizing oracle bone characters is of great significance since it could promote the research on history, philology and archaeology. The proposed solutions for oracle bone characters recognition are mainly based on machine learning or deep learning algorithms which rely on a large number of supervised training data. However, the existing dataset suffers from the problem of severe class imbalance. In this work, we propose a CycleGAN-based data augmentation method to overcome the limitation. Via learning the mapping between the glyph images data domain and the real samples data domain, CycleGAN could generate oracle character images of high-quality. The quality is evaluated using the quantitative measure. Totally, 185362 samples are generated which could serve as a complementary to the existing dataset. With these generated samples, the state of the art results of recognition task on OBC306 are improved greatly in terms of mean accuracy and total accuracy.
Published: 2022

24. On the Accuracy of Open Video Quality Metrics for Local Decision in AV1 Video Codec

Author: Pastor, Andreas, Krasula, Lukas, Zhu, Xiaoqing, Li, Zhi, Callet, Patrick Le, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Netflix
Subjects: tubes, Open Video Codecs, [INFO]Computer Science [cs], Video Quality, Difference Scaling, VMAF
Abstract: International audience; VMAF is a popular objective quality metric used for video quality evaluation. The power of VMAF has been demonstrated for a wide variety of video scales and encoding processes. However, its ability to evaluate the quality of small video patches has not yet been tested, despite its importance for encoding algorithms. We applied Maximum Likelihood Difference Scaling (MLDS) methodology to estimate supra-threshold perceptual differences in localized sections in videos, also known as tubes, encoded using AV1. We further used the results to assess the performance of VMAF in this scenario and proposed a recalibration of the algorithm to improve its agreement with the subjective data.
Published: 2022
Full Text: View/download PDF

25. Towards a Calibrated 360° Stereoscopic HDR Image Dataset for Architectural Lighting Studies

Author: Michèle Atié, Toinon Vigier, François Eymond, Céline Drozd, Raphaël Labayrade, Daniel Siret, Yannick Sutter, Centre de recherche nantais Architectures Urbanités (CRENAU ), Ambiances, Architectures, Urbanités (AAU), Ministère de la Culture et de la Communication (MCC)-Centre National de la Recherche Scientifique (CNRS)-École nationale supérieure d'architecture de Grenoble (ENSAG ), Université Grenoble Alpes (UGA)-Université Grenoble Alpes (UGA)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-NANTES UNIVERSITÉ - École nationale supérieure d'architecture de Nantes (Nantes Univ - ENSA Nantes), Nantes Université (Nantes Univ)-Ministère de la Culture et de la Communication (MCC)-Centre National de la Recherche Scientifique (CNRS)-École nationale supérieure d'architecture de Grenoble (ENSAG ), Nantes Université (Nantes Univ), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Image Perception Interaction (LS2N - équipe IPI), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), École Nationale des Travaux Publics de l'État (ENTPE), École Nationale des Travaux Publics de l'État (ENTPE)-Ministère de l'Ecologie, du Développement Durable, des Transports et du Logement, and ANR-19-CE38-0010,PERCILUM,Visualisation, perception et pédagogie des ambiances lumineuses via des dispositifs numériques innovants(2019)
Subjects: [SHS.ARCHI]Humanities and Social Sciences/Architecture, space management, [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Daylight, Virtual reality VR, [SCCO.COMP]Cognitive science/Computer science, Computing technology, Ambiance lumineuse, [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR]
Abstract: International audience; Highfidelity360 images enhance user experience and offer realistic representations for architectural design studies. Specifically, VR and hyper-realistic imaging technologies can be helpful tools to study daylight in architectural places thanks to the high level of immersion and its ability to create perceptually accurate and faithful scene representations.In this paper, we present a novel method for collecting and processing physically calibrated 360◦ stereoscopic high-dynamic range images of daylit indoor places. The future dataset aims to provide a higher degree of realism, a wide range of various luminous interior spaces supplied with information on the physical characterization of the space. This paper presents the first applications of this method on different places and discusses challenges for assessing visual perception in these images. In the near future, this dataset will be publicly available for architectural as well as multimedia studies.
Published: 2022
Full Text: View/download PDF

26. Perceptual evaluation on audio-visual dataset of 360 content

Author: Fela, Randy Frans, Pastor, Andréas, Le Callet, Patrick, Zacharov, Nick, Vigier, Toinon, Forchhammer, Søren, Danmarks Tekniske Universitet = Technical University of Denmark (DTU), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), Meta AI Research [Paris], and Meta AI
Subjects: FOS: Computer and information sciences, Sound (cs.SD), Image and Video Processing (eess.IV), omnidirectional media format, ambisonic, Omnidirectional media format, Electrical Engineering and Systems Science - Image and Video Processing, Quality evaluation, Computer Science - Sound, quality evaluation, Audiovisual dataset, Multimedia (cs.MM), Ambisonics, audiovisual dataset, Audio and Speech Processing (eess.AS), 360 video, FOS: Electrical engineering, electronic engineering, information engineering, [INFO]Computer Science [cs], Computer Science - Multimedia, Electrical Engineering and Systems Science - Audio and Speech Processing
Abstract: To open up new possibilities to assess the multimodal perceptual quality of omnidirectional media formats, we proposed a novel open source 360 audiovisual (AV) quality dataset. The dataset consists of high-quality 360 video clips in equirectangular (ERP) format and higher-order ambisonic (4th order) along with the subjective scores. Three subjective quality experiments were conducted for audio, video, and AV with the procedures detailed in this paper. Using the data from subjective tests, we demonstrated that this dataset can be used to quantify perceived audio, video, and audiovisual quality. The diversity and discriminability of subjective scores were also analyzed. Finally, we investigated how our dataset correlates with various objective quality metrics of audio and video. Evidence from the results of this study implies that the proposed dataset can benefit future studies on multimodal quality evaluation of 360 content., 6 pages, 5 figures, International Conference on Multimedia and Expo 2022
Published: 2022
Full Text: View/download PDF

27. REINFORCEMENT LEARNING BASED POINT-CLOUD ACQUISITION AND RECOGNITION USING EXPLORATION-CLASSIFICATION REWARD COMBINATION

Author: Kevin Riou, Kevin Subrin, Patrick Le Callet, Nantes Université - Ecole Polytechnique de l'Université de Nantes (Nantes Univ - EPUN), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Robots and Machines for Manufacturing, Society and Services (LS2N - équipe RoMas)
Subjects: [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV], Reinforcement Learning, Active point-clouds acquisition, 3D objects recognition
Abstract: International audience; 3D points acquisitions based on robust sensors such as tactile or laser sensors are true alternatives to computer vision for 3D object recognition. In real life scenarios where robots are equipped with such sensors to acquire 3D data, only few points can be iteratively collected in a reasonable amount of time. However, existing Point-cloud classifiers are extremely sensitive to sparse points, missing parts and noise. To compensate for the sparsity of the data, some Reinforcement Learning (RL) based approaches have been proposed to learn a sparse yet efficient exploration of the target object regarding the 3D recognition objective. However, existing RL approaches only focus on classification performances to guide the training of the active acquisition-and-classification frameworks, and thus fail to dissociate poor exploration strategy (missing parts, noisy points) from actual classifier mistakes on proper data. In this study, we proposed a new RL framework that was rewarded regarding both the classification performances and the exploration quality. Our trained framework outperforms existing State-Of-The-Art models on 3D geometric objects classification. We further showed that our trained framework learnt to alternate between (1) a clean and broad exploration strategy, suitable for easily distinguishable categories, and (2) a specific local exploration strategy, facilitating the discrimination of similar categories.
Published: 2022

28. Beyond deep learning: automatic morpho-kinetic parameter extraction using spatial attention

Author: Gomez, T, Fréour, T, Mouchère, Harold, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Immunotherapy in Transplantation And Autoimmunity (Team 3 - U1064 Inserm - CRTI), Centre de Recherche en Transplantation et Immunologie - Center for Research in Transplantation and Translational Immunology (U1064 Inserm - CR2TI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Nantes Université - UFR de Médecine et des Techniques Médicales (Nantes Univ - UFR MEDECINE), Nantes Université - pôle Santé, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Santé, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Nantes Université - UFR de Médecine et des Techniques Médicales (Nantes Univ - UFR MEDECINE), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), Centre hospitalier universitaire de Nantes (CHU Nantes), and ANR-16-IDEX-0007,NExT (I-SITE),NExT (I-SITE)(2016)
Subjects: [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE]
Abstract: International audience; Abstract Study question Can interpretable deep learning Artificial Intelligence automatically learn to extract morphokinetic parameters from time-lapse embryo videos? Summary answer We trained several popular interpretable AI architectures that were successfully able to recognize most embryo developmental stages from day 1 to 5. What is known already Several deep learning AI approaches have been applied to IVF, leading to promising but heterogeneous and perfectible results. However, none of these « black box » models are interpretable, preventing users from having access to the features included in the model and thus from critically analyzing its results. Some interpretable AIs, have been recently developed and could help users visualize the regions of interest in a picture. These interpretable AI have not been adapted and tested to embryo IVF images/videos but could be extremely relevant in this field, providing more insight into AI functioning, and allowing critical analysis. Study design, size, duration In this monocentric study we used 756 randomly selected time lapse videos recorded between 2011 and 2016 (approximately 10% of all the videos generated over this period) and used half of the dataset for the training phase of the AI and the other half for the final evaluation. Participants/materials, setting, methods We used popular deep learning architectures augmented with spatial attention modules that allow the AI to focus on specific areas of the image. We used ResNet-50 as the principal model and combined it with B-CNN, BR-NPA, ProtoPNet, and ProtoTree attention modules. We also applied state-of-the-art generic explanation methods like Grad-CAM, Grad-CAM ++, or Score-CAM. We used standard training procedures and visualized the explanation map produced with a heatmap. Main results and the role of chance We trained several popular interpretable AI architectures to distinguish the 16 development phases of embryos from day 1 to 5 and obtained overall 70 % accuracy, as compared to manual annotation by experts. The inspection of the explanation maps highlighted various patterns of focus on the embryo of one particular model, BR-NPA. At the fertilization stage, this interpretable AI focused on the pro-nuclei, whereas from phase t5 to t9+ the model focused on the separate cells. At the blastocyst stage, the model was able to separate the ICM from the rest of the embryo (trophectoderm). This shows that interpretable AI can indeed be applied to time-lapse videos and focus on relevant regions of the embryo at all developmental stages, producing interpretable results. The role of chance was minimal in this study as we obtained consistent results across various AI architectures, the dataset was selected randomly among all available videos and the dataset was large enough to ensure minimal selection bias. Limitations, reasons for caution Our study suffers from the inherent limitations of monocentric studies. Indeed, lab procedures and characteristics of the population can vary from one setting to another and question the generalizability of our findings. However, deep learning is versatile and similar results can probably be obtained on data from other centers. Wider implications of the findings We postulate that this original approach will help improve the acceptability and the trust of embryologists in AI-based software, which will improve data handling and consistency, ultimately benefiting infertile patients with improved clinical success rates. Trial registration number Not applicable
Published: 2022
Full Text: View/download PDF

29. Using Knowledge Graphs and Human-Centric Artificial Intelligence for Reconfigurable Supply Chains: A Research Framework

Author: Benjamin Rolf, Nasser Mebarki, Sebastian Lang, Tobias Reggelin, Olivier Cardin, Harold Mouchère, Alexandre Dolgui, Otto-von-Guericke-Universität Magdeburg = Otto-von-Guericke University [Magdeburg] (OVGU), Nantes Université (Nantes Univ), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Conception, Pilotage, Surveillance et Supervision des systèmes (LS2N - équipe CPS3), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Fraunhofer Institute for Factory Operation and Automation (Fraunhofer IFF), Département Automatique, Productique et Informatique (IMT Atlantique - DAPI), IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Modélisation, Optimisation et DEcision pour la Logistique, l'Industrie et les Services (LS2N - équipe MODELIS), and Image Perception Interaction (LS2N - équipe IPI)
Subjects: Control and Systems Engineering, Reconfigurable supply chains, [INFO.INFO-AU]Computer Science [cs]/Automatic Control Engineering, Knowledge graphs, Open data, Supply Supply chain management, [INFO]Computer Science [cs], [INFO.INFO-RO]Computer Science [cs]/Operations Research [cs.RO], Human-centric artificial intelligence, Graph neural networks
Abstract: International audience; Reconfigurable supply chains received increasing interest from academia and industry in the past years, especially because recent events such as the COVID-19 pandemic revealed the vulnerability of present supply chains. Especially the rapid digitalization and the emergence of artificial intelligence in supply chain management create new opportunities for implementing reconfigurable supply chains. In this paper, we propose a concept that uses a knowledge graph and graph-based artificial intelligence to recommend reconfigurations of the supply chain. Due to the importance of these strategic decisions, the concept considers a human-centric approach using a recommender system. The knowledge graph exploits the inherent graph structure of supply chains to create a machine-readable and human-understandable database that also supports reasoning. Finally, we provide three research hypotheses and examine the integration of open-source knowledge graphs and news monitoring tools.
Published: 2022
Full Text: View/download PDF

30. From Olfaction to Emotions: An Interactive and Immersive Experience

Author: Le Coz, Mona, Lévêque, Lucie, Perreira da Silva, Matthieu, Le Callet, Patrick, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: immersion, multimedia, interactivity, Emotions, [SCCO.COMP]Cognitive science/Computer science, olfaction
Abstract: International audience; The causal relationship between olfactory perception and human emotions has been widely studied and accepted by various fields including, but not limited to, health, marketing, and multimedia. In this work-in-progress paper, we present an olfactive, interactive and immersive experience taking place during the World Creativity & Innovation Week in Nantes, France, in April 2022. Through this experience, we aim to observe and exploit the existing relationship between smells and emotions by measuring, in real time, participants' physiological reactions (e.g., heart rate and skin conductance) to a given olfactive stimulus. Based on the obtained signals, appropriate multimedia content, implemented in collaboration with local partners, is interactively displayed in the room with a view to further immerse the participants in their experience.
Published: 2022

31. Are Facial Expression Recognition Algorithms Reliable in the Context of Interactive Media? A New Metric to Analyse Their Performance

Author: Sampaio, Emmanuel, Lévêque, Lucie, Perreira da Silva, Matthieu, Le Callet, Patrick, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: benchmark, Emotions, facial expression recognition, [SCCO.COMP]Cognitive science/Computer science, distance metric, interactive media
Abstract: International audience; Emotions, and consequently facial expressions, play an essential role in communication-and thus in everyday life. With the increase of human-machine interactions, and more especially of multimedia applications, automatic recognition of facial expressions has emerged as a challenging task, particularly under naturalistic conditions. In the present work, a benchmark is firstly conducted using four open source deep learning solutions on four labeled image datasets. Thanks to an exhaustive analysis based on two distinct, yet complementary approaches, we show how the four models performed depending on the studied emotions. Furthermore, we present a novel metric based on the Euclidean distance between two given emotions (i.e., ground truth and predicted) to better measure the performance of said models in the context of interactive media, where human sensibility needs to be taken into consideration.
Published: 2022

32. Perception of video quality at a local spatio-temporal horizon

Author: Pastor, Andréas, Callet, Patrick Le, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: video quality, Perception, [INFO]Computer Science [cs], distortion, CODEC
Abstract: International audience; This paper contains the research proposal of Andréas Pastor that was presented at the MMSys 2022 doctoral symposium. Encoding video for streaming on Internet has become a major topic to reduce the consumption of bandwidth and latency. At the same time, the human perception of distortions has been explored in multiple research projects, especially for distortions generated by Coder-DECoder (CODEC) algorithms. These algorithms operate in a rate-distortion optimization paradigm to efficiently compress video content. This optimization can be driven by metrics that are most of the time not based on the human perception, and more importantly, not tuned to reflect the local perception of distortions by human eyes. In this doctoral study, we proposed to work on the perception of localized distortion at a small temporal and spatial horizon. We present here the fundamental research questions and challenges in the domain with a focus on methods to collect perceptual judgments in subjective studies and metrics that can help us to derive an estimate of the perception of distortions by humans.
Published: 2022
Full Text: View/download PDF

33. Morphological Difference of Closings Operator for No-Reference Quality Evaluation of DIBR-Synthesized Images

Author: Sandic-Stankovic, Dragana, Kukolj, Dragan, Le Callet, Patrick, University of Novi Sad, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), and IEEE
Subjects: No-reference image quality assessment, difference of closings operator, structural image distortion, sparse representation, [SPI.SIGNAL]Engineering Sciences [physics]/Signal and Image processing, morphological multi-resolution multi-scale image representation, [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR], granulometry
Abstract: International audience; Images synthesized using Depth-Image-Based Rendering (DIBR) techniques are characterized by complex structural distortion. Multi-resolution multi-scale sparse image representation generated using morphological Difference of Closings operator (DoC) is used to efficiently capture structure-related distortion of synthesized images in the noreference DoC-GRNN image quality assessment model. Nonlinear morphological Difference of Closings operator (DoC) with an array of line-shaped structuring elements of increasing length is used to extract perceptually important details of object structure at different scales and resolutions. The sparsity of DoC band is calculated as scalar feature. The extracted features are mapped to the quality score by general regression neural network (GRNN). We have explored the influence of the direction of an array of line-shaped structuring elements on the model's performances. The DoC-GRNN model shows high agreement with perceptual quality scores, comparable to the state-of-the-art metrics, when evaluated on the stereoscopic DIBR-synthesized images of MCL-3D dataset.
Published: 2022
Full Text: View/download PDF

34. Subjective And Objective Quality Assessment Of Mobile Gaming Video

Author: Wen, Shaoguo, Ling, Suiyi, Wang, Junle, Chen, Ximing, Jing, Yanqing, Le Callet, Patrick, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Subjective quality assessment, model distillation, gaming video, objective quality metric, [SPI.SIGNAL]Engineering Sciences [physics]/Signal and Image processing
Abstract: International audience; Nowadays, with the vigorous expansion and development of gaming video streaming techniques and services, the expectation of users, especially the mobile phone users, for higher quality of experience is also growing swiftly. As most of the existing research focuses on traditional video streaming, there is a clear lack of both subjective study and objective quality models that are tailored for quality assessment of mobile gaming content. To this end, in this study, we first present a brand new Tencent Gaming Video dataset containing 1293 mobile gaming sequences encoded with three different codecs. Second, we propose an objective quality framework, namely Efficient hard-RAnk Quality Estimator (ERAQUE), that is equipped with (1) a novel hard pairwise ranking loss, which forces the model to put more emphasis on differentiating similar pairs; (2) an adapted model distillation strategy, which could be utilized to compress the proposed model efficiently without causing significant performance drop. Extensive experiments demonstrate the efficiency and robustness of our model.
Published: 2022
Full Text: View/download PDF

35. Interstellar methanol: the challenge of reactivity under astrophysical conditions

Author: D., Comte, L., Lavy, T., Salbaing, P., Bertier, B., Farizon, M., Farizon, T.D., Märk, F., Calvo, Institut de Physique des 2 Infinis de Lyon (IP2I Lyon), Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Université de Lyon-Institut National de Physique Nucléaire et de Physique des Particules du CNRS (IN2P3)-Centre National de la Recherche Scientifique (CNRS), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), Institut für Ionenphysik und Angewandte Physik - Institute for Ion Physics and Applied Physics [Innsbruck], Leopold Franzens Universität Innsbruck - University of Innsbruck, Laboratoire Interdisciplinaire de Physique [Saint Martin d’Hères] (LIPhy ), and Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)
Subjects: [PHYS.PHYS.PHYS-ATM-PH]Physics [physics]/Physics [physics]/Atomic and Molecular Clusters [physics.atm-clus], ComputingMilieux_MISCELLANEOUS
Abstract: International audience
Published: 2022

36. Textured Mesh Quality Assessment: Large-Scale Dataset and Deep Learning-based Quality Metric

Author: Nehmé, Yana, Delanoy, Johanna, Dupont, Florent, Farrugia, Jean-Philippe, Callet, Patrick Le, Lavoué, Guillaume, Origami (Origami), Laboratoire d'InfoRmatique en Image et Systèmes d'information (LIRIS), Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL), Université de Lyon-Université de Lyon-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS)-Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), and ANR-17-CE33-0005,PISCo,Niveaux de détails perceptuels pour la visualisation distante, interactive et immersive de scènes 3D riches(2017)
Subjects: FOS: Computer and information sciences, I.3, Perceptual Metric, Computer Graphics and Computer-Aided Design, Visual Quality Assessment, Graphics (cs.GR), [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR], Deep Learning, Computer Science - Graphics, 3D Mesh, Subjective Quality Evaluation, Computer Graphics, Objective Quality Evaluation, Crowdsourcing, Perception, Texture, Dataset, ComputingMethodologies_COMPUTERGRAPHICS
Abstract: International audience; Over the past decade, three-dimensional (3D) graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which can degrade their visual quality. Thus, to ensure the best Quality of Experience, it is important to evaluate the visual quality to accurately drive the compression and find the right compromise between visual quality and data size. In this work, we focus on subjective and objective quality assessment of textured 3D meshes. We first establish a large-scale dataset, which includes 55 source models quantitatively characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of five types of compression-based distortions applied on the geometry, texture mapping, and texture image of the meshes. This dataset contains over 343k distorted stimuli. We propose an approach to select a challenging subset of 3,000 stimuli for which we collected 148,929 quality judgments from over 4,500 participants in a large-scale crowdsourced subjective experiment. Leveraging our subject-rated dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric and dataset to explore the influence of distortion interactions and content characteristics on the perceived quality of compressed textured meshes.
Published: 2022
Full Text: View/download PDF

37. Improving Semantic Segmentation with Graph-Based Structural Knowledge

Author: Jérémy Chopin, Jean-Baptiste Fasquel, Harold Mouchère, Rozenn Dahyot, Isabelle Bloch, Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS), Université d'Angers (UA), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), National University of Ireland Maynooth (Maynooth University), Learning, Fuzzy and Intelligent systems (LFI), LIP6, and Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)
Subjects: quadratic assignment problem, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], deep learning, volume segmentation, Graph matching, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], image segmentation
Abstract: International audience; Deep learning based pipelines for semantic segmentation often ignore structural information available on annotated images used for training. We propose a novel post-processing module enforcing structural knowledge about the objects of interest to improve segmentation results provided by deep learning. This module corresponds to a ``many-to-one-or-none'' inexact graph matching approach, and is formulated as a quadratic assignment problem. Using two standard measures for evaluation, we show experimentally that our pipeline for segmentation of 3D MRI data of the brain outperforms the baseline CNN (U-Net) used alone. In addition, our approach is shown to be resilient to small training datasets that often limit the performance of deep learning.
Published: 2022
Full Text: View/download PDF

38. Metrics for saliency map evaluation of deep learning explanation methods

Author: Gomez, Tristan, Fréour, Thomas, Mouchère, Harold, Mouchère, Harold, NExT (I-SITE) - - NExT (I-SITE)2016 - ANR-16-IDEX-0007 - IDEX - VALID, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Nantes Université - École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Dendritic cells and immunoregulation in transplantation and immunopathology (Team 1 - U1064 Inserm - CRTI), Centre de Recherche en Transplantation et Immunologie - Center for Research in Transplantation and Translational Immunology (U1064 Inserm - CR2TI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Nantes Université - UFR de Médecine et des Techniques Médicales (Nantes Univ - UFR MEDECINE), Nantes Université - pôle Santé, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Santé, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Nantes Université - UFR de Médecine et des Techniques Médicales (Nantes Univ - UFR MEDECINE), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ), ANR-16-IDEX-0007,NExT (I-SITE),NExT (I-SITE)(2016), and Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN)
Subjects: FOS: Computer and information sciences, Interpretable machine learning, Computer Science - Artificial Intelligence, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, [INFO.INFO-NE] Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], Objective evaluation, Artificial Intelligence (cs.AI), [INFO.INFO-TI] Computer Science [cs]/Image Processing [eess.IV], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Saliency maps
Abstract: International audience; Due to the black-box nature of deep learning models, there is a recent development of solutions for visual explanations of CNNs. Given the high cost of user studies, metrics are necessary to compare and evaluate these different methods. In this paper, we critically analyze the Deletion Area Under Curve (DAUC) and Insertion Area Under Curve (IAUC) metrics proposed by Petsiuk et al. (2018). These metrics were designed to evaluate the faithfulness of saliency maps generated by generic methods such as Grad-CAM or RISE. First, we show that the actual saliency score values given by the saliency map are ignored as only the ranking of the scores is taken into account. This shows that these metrics are insufficient by themselves, as the visual appearance of a saliency map can change significantly without the ranking of the scores being modified. Secondly, we argue that during the computation of DAUC and IAUC, the model is presented with images that are out of the training distribution which might lead to an unreliable behavior of the model being explained. To complement DAUC/IAUC, we propose new metrics that quantify the sparsity and the calibration of explanation methods, two previously unstudied properties. Finally, we give general remarks about the metrics studied in this paper and discuss how to evaluate them in a user study.
Published: 2022
Full Text: View/download PDF

39. Confusing Image Quality Assessment: Toward Better Augmented Reality Experience

Author: Huiyu Duan, Xiongkuo Min, Yucheng Zhu, Guangtao Zhai, Xiaokang Yang, Patrick Le Callet, Shanghai Jiao Tong University [Shanghai], Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Nantes Université (Nantes Univ)
Subjects: FOS: Computer and information sciences, Augmented Reality, Databases, Factual, Computer Vision and Pattern Recognition (cs.CV), Image and Video Processing (eess.IV), visual confusion, ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION, Computer Science - Computer Vision and Pattern Recognition, image quality assessment, Electrical Engineering and Systems Science - Image and Video Processing, Computer Graphics and Computer-Aided Design, quality of experience (QoE), [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR], FOS: Electrical engineering, electronic engineering, information engineering, Humans, Software, Algorithms, Augmented Reality (AR)
Abstract: International audience; With the development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary value of AR is to promote the fusion of digital contents and real-world environments, however, studies on how this fusion will influence the Quality of Experience (QoE) of these two components are lacking. To achieve better QoE of AR, whose two layers are influenced by each other, it is important to evaluate its perceptual quality first. In this paper, we consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A more general problem is first proposed, which is evaluating the perceptual quality of superimposed images, i.e., confusing image quality assessment. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. Then a subjective quality perception experiment is conducted towards attaining a better understanding of how humans perceive the confusing images. Based on the CFIQA database, several benchmark models and a specifically designed CFIQA model are proposed for solving this problem. Experimental results show that the proposed CFIQA model achieves state-of-the-art performance compared to other benchmark models. Moreover, an extended ARIQA study is further conducted based on the CFIQA study. We establish an ARIQA database to better simulate the real AR application scenarios, which contains 20 AR reference images, 20 background (BG) reference images, and 560 distorted images generated from AR and BG references, as well as the correspondingly collected subjective quality ratings. Three types of full-reference (FR) IQA benchmark variants are designed to study whether we should consider the visual confusion when designing corresponding IQA algorithms. An ARIQA metric is finally proposed for better evaluating the perceptual quality of AR images. Experimental results demonstrate the good generalization ability of the CFIQA model and the state-of-the-art performance of the ARIQA model. The databases, benchmark models, and proposed metrics are available at: https://github.com/DuanHuiyu/ARIQA.
Published: 2022
Full Text: View/download PDF

40. Improving Maximum Likelihood Difference Scaling method to measure inter content scale

Author: Andreas Pastor, Lukas Krasula, Xiaoqing Zhu, Zhi Li, Patrick Le Callet, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), and Netflix
Subjects: FOS: Computer and information sciences, Computer Science - Machine Learning, Quantitative Biology - Neurons and Cognition, FOS: Biological sciences, [INFO]Computer Science [cs], Neurons and Cognition (q-bio.NC), ComputingMilieux_MISCELLANEOUS, Machine Learning (cs.LG)
Abstract: The goal of most subjective studies is to place a set of stimuli on a perceptual scale. This is mostly done directly by rating, e.g. using single or double stimulus methodologies, or indirectly by ranking or pairwise comparison. All these methods estimate the perceptual magnitudes of the stimuli on a scale. However, procedures such as Maximum Likelihood Difference Scaling (MLDS) have shown that considering perceptual distances can bring benefits in terms of discriminatory power, observers' cognitive load, and the number of trials required. One of the disadvantages of the MLDS method is that the perceptual scales obtained for stimuli created from different source content are generally not comparable. In this paper, we propose an extension of the MLDS method that ensures inter-content comparability of the results and shows its usefulness especially in the presence of observer errors., Comment: Difference scaling, supra-threshold estimation, human perception, subjective experiment
Published: 2022
Full Text: View/download PDF

41. QAP Optimisation with Reinforcement Learning for Faster Graph Matching in Sequential Semantic Image Analysis

Author: Jérémy Chopin, Jean-Baptiste Fasquel, Harold Mouchère, Rozenn Dahyot, Isabelle Bloch, Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS), Université d'Angers (UA), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), National University of Ireland Maynooth (Maynooth University), Learning, Fuzzy and Intelligent systems (LFI), LIP6, and Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)
Subjects: Quadratic assignment problem, [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Reinforcement learning, Graph matching, [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], Semantic image analysis, Structural information
Abstract: International audience; The paper addresses the fundamental task of semantic image analysis by exploiting structural information (spatial relationshipsbetween image regions). We propose to combine a deep neural network(CNN) with graph matching where graphs encode efficiently structuralinformation related to regions segmented by the CNN. Our novel approach solves the quadratic assignment problem (QAP) sequentially formatching graphs. The optimal sequence for graph matching is conveniently defined using reinforcement-learning (RL) based on the regionmembership probabilities produced by the CNN and their structural relationships. Our RL-based strategy for solving QAP sequentially allowsus to significantly reduce the combinatorial complexity for graph matching. Preliminary experiments are performed on both a synthetic datasetand a public dataset dedicated to the semantic segmentation of face images. Results show that the proposed RL-based ordering significantlyoutperforms random ordering and that our strategy is about 386 timesfaster than a global QAP-based approach while preserving similar segmentation accuracy.
Published: 2022
Full Text: View/download PDF

42. Inflammatory optic neuropathy in granulomatosis with polyangiitis can mimick isolated idiopathic optic neuritis

Author: Michel Weber, Manon Clément, Pierre Lebranchu, Frederique Toulgoat, Mohamed Hamidou, Pascal Godmer, Antoine Néel, Pascal Hutin, Service d'ophtalmologie [Nantes], Hôtel-Dieu, Centre de Recherche en Transplantation et Immunologie (U1064 Inserm - CRTI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Nantes - UFR de Médecine et des Techniques Médicales (UFR MEDECINE), Université de Nantes (UN)-Université de Nantes (UN), Service de médecine interne [Nantes], Université de Nantes (UN)-Hôtel-Dieu-Centre hospitalier universitaire de Nantes (CHU Nantes), Service de Radiologie [Nantes], Centre hospitalier universitaire de Nantes (CHU Nantes), Service de Médecine interne - Maladies hématologiques - Maladies infectieuses [Vannes], Centre hospitalier Bretagne Atlantique (Morbihan) (CHBA), Service des Maladies Infectieuses [CHI Cornouaille, Quimper], Centre Hospitalier Intercommunal de Cornouaille [Quimper] (CHI Cornouaille [Quimper]), Image Perception Interaction (IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS), Le Bihan, Sylvie, Image Perception Interaction (LS2N - équipe IPI), and Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Adult, Male, medicine.medical_specialty, genetic structures, [SDV]Life Sciences [q-bio], Vision Disorders, Visual Acuity, Optic neuritis, vasculitis, Diagnosis, Differential, Optic neuropathy, 03 medical and health sciences, 0302 clinical medicine, Optic Nerve Diseases, Humans, Medicine, diplopia, Infusions, Intravenous, 10. No inequality, Glucocorticoids, Aged, Retrospective Studies, 030203 arthritis & rheumatology, Diplopia, business.industry, Granulomatosis with Polyangiitis, General Medicine, Middle Aged, medicine.disease, Magnetic Resonance Imaging, Dermatology, eye diseases, 3. Good health, [SDV] Life Sciences [q-bio], Ophthalmology, granulomatosis with polyangiitis (Wegener’s granulomatosis), 030221 ophthalmology & optometry, Female, sense organs, medicine.symptom, business, Vasculitis, Granulomatosis with polyangiitis, visual loss, MRI
Abstract: Objective: We describe a clinico-radiological presentation of inflammatory optic neuropathy that mimicked optic neuritis. Methods: Retrospective single-center case series and literature review of optic neuropathy without orbital pseudotumor. Results: Five local patients fulfilled the inclusion criteria. Clinical presentation revealed rapidly progressive severe unilateral visual loss, retrobulbar pain (n = 4), and paralytic strabismus (simultaneous = 2, protracted = 2) without proptosis. Optic nerve abnormality was not appreciated on initial scan review. Patients did not have any general activity of the granulomatosis with polyangiitis. Upon follow-up magnetic resonance imaging and initial imaging review, all patients revealed orbital apex anomalies. Visual acuity improved in three patients who received high-dose intravenous glucocorticosteroids immediately. Relapse was frequent and visual outcome was poor (final vision > 20/40 in two patients only). Literature review identified 16 well-documented cases of granulomatosis with polyangiitis–related isolated optic neuropathies. Magnetic resonance imaging revealed no abnormality (n = 6), optic nerve and/or sheath involvement (n = 9), apex infiltration (n = 3), and/or pachymeningitis (n = 7). Conclusion: Granulomatosis with polyangiitis is a rare yet potentially blinding cause of inflammatory optic neuropathy. Optic neuropathy in granulomatosis with polyangiitis may occur in the absence of systemic symptoms of disease activity and is challenging to distinguish from other inflammatory and non-inflammatory disorders affecting visual acuity. Several clinical and imaging clues suggest that optic neuropathy results from the development of an extravascular granulomatous process within the optic nerve sheath in the orbital apex, a place that is difficult to image. In a granulomatosis with polyangiitis patient with unexplained visual loss and a seemingly normal workup (fundoscopy, biology, and imaging), clinician should keep a high index of suspicion.
Published: 2019
Full Text: View/download PDF

43. Automatic classification of benign and malignant kidney masses using radiomics. A retrospective study exploiting the KiTS19 dataset

Author: Gianmarco Santini, Yvon Nzoughet Obame, Mathieu Rubeaux, Noémie Moreau, Constance Fourcade, Keosys, Nantes Université (Nantes Univ), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, and Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
Subjects: Computer science, business.industry, Deep learning, Feature selection, Pattern recognition, [SDV.CAN]Life Sciences [q-bio]/Cancer, Linear discriminant analysis, Convolutional neural network, Random forest, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI], Support vector machine, Feature (computer vision), Scoring algorithm, Artificial intelligence, business, ComputingMilieux_MISCELLANEOUS
Abstract: The study aims to assess the performance in differentiating benign from malignant kidney masses using a radiomics approach. For this retrospective study we worked with the scans of 210 patients from the publicly available KiTS19 dataset. Each scan had segmentations of the healthy kidney tissue, benign lesions and malignant tumors. In Phase 1 of our study, we reduced the number of radiomic features (105) extracted from the scans by using four feature selection and ranking algorithms: recursive feature elimination (RFE), fisher score, partial least square discriminant analysis (PLS-DA) and linear support vector machine (l-SVM). The features selected by each method were then used to train a series of random forest (RF) classifiers. In Phase 2, we trained a convolutional neural network (CNN) to automatically perform the segmentation of benign and malignant kidney masses. We then placed the best performing RF classifier from Phase 1 in series with the CNN to see if it corrected its prediction. The best classification performance was obtained when training a RF classifier with the 8 features selected by the RFE method (accuracy: 0.974). This RF model applied to the segmentations derived from the neural network improved the CNN’s overall results: the dice score for malignant mass went from 0.74 to 0.79 and dice score for benign mass from 0.55 to 0.80. The studied radiomics approach proved to be an accurate solution to classify benign and malignant kidney masses. A deep learning algorithm has shown to also benefit from its predictive power.
Published: 2021
Full Text: View/download PDF

44. Comparative study of the methodologies used for subjective medical image quality assessment

Author: Meriem Outtas, Hantao Liu, Lu Zhang, Lucie Leveque, Laboratoire des Sciences du Numérique de Nantes (LS2N), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Image Perception Interaction (LS2N - équipe IPI), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), Institut d'Électronique et des Technologies du numéRique (IETR), Université de Nantes (UN)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS), Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA), Cardiff University, Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Université de Nantes (UN)-Centre National de la Recherche Scientifique (CNRS)-CentraleSupélec-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES), Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES), Nantes Université (NU)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), and Institut National des Sciences Appliquées (INSA)-Université de Rennes (UNIV-RENNES)-Institut National des Sciences Appliquées (INSA)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)
Subjects: Diagnostic Imaging, Databases, Factual, Image quality, Computer science, media_common.quotation_subject, image quality assessment, medical imaging, subjective experiment, objective metrics, Video quality, 01 natural sciences, 030218 nuclear medicine & medical imaging, Task (project management), 010309 optics, 03 medical and health sciences, 0302 clinical medicine, Perception, 0103 physical sciences, Medical imaging, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, Humans, Radiology, Nuclear Medicine and imaging, Quality (business), Quality of experience, Child, media_common, Radiological and Ultrasound Technology, Data science, 3. Good health, Variety (cybernetics), task performance, Radiography, [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], Artifacts
Abstract: Healthcare professionals have been increasingly viewing medical images and videos in their \ud routine clinical practice, and this in a wide variety of environments. Both the perception and \ud interpretation of medical visual information, across all branches of practice or medical \ud specialties (e.g., diagnostic, therapeutic, or surgical medicine), career stages, and practice \ud settings (e.g., emergency care), appear to be critical for patient care. However, medical images \ud and videos are not self-explanatory and, therefore, need to be interpreted by humans, i.e., \ud medical experts. In addition, various types of degradations and artifacts may appear during \ud image acquisition or processing, and consequently affect medical imaging data. Such \ud distortions tend to impact viewers’ quality of experience, as well as their clinical practice. It is \ud accordingly essential to better understand how medical experts perceive the quality of visual \ud content. Thankfully, progress has been made in the recent literature towards such\ud understanding. In this article, we present an up-to-date state of the art of relatively recent (i.e., \ud not older than ten years old) existing studies on the subjective quality assessment of medical \ud images and videos, as well as research works using task-based approaches. Furthermore, we \ud discuss the merits and drawbacks of the methodologies used, and we provide recommendations \ud about experimental designs and statistical processes to evaluate the perception of medical \ud images and videos for future studies, which could then be used to optimise the visual experience \ud of image readers in real clinical practice. Finally, we tackle the issue of the lack of available \ud annotated medical image and video quality databases, which appear to be indispensable for the \ud development of new dedicated objective metrics.
Published: 2021
Full Text: View/download PDF

45. Considering user agreement in learning to predict the aesthetic quality

Author: Suiyi Ling, Andreas Pastor, Junle Wang, Patrick Le Callet, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), and Tencent [Shenzhen]
Subjects: FOS: Computer and information sciences, I.4.0, 68T07, Artificial Intelligence (cs.AI), Computer Science - Artificial Intelligence, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, [INFO]Computer Science [cs], ComputingMilieux_MISCELLANEOUS
Abstract: How to robustly rank the aesthetic quality of given images has been a long-standing ill-posed topic. Such challenge stems mainly from the diverse subjective opinions of different observers about the varied types of content. There is a growing interest in estimating the user agreement by considering the standard deviation of the scores, instead of only predicting the mean aesthetic opinion score. Nevertheless, when comparing a pair of contents, few studies consider how confident are we regarding the difference in the aesthetic scores. In this paper, we thus propose (1) a re-adapted multi-task attention network to predict both the mean opinion score and the standard deviation in an end-to-end manner; (2) a brand-new confidence interval ranking loss that encourages the model to focus on image-pairs that are less certain about the difference of their aesthetic scores. With such loss, the model is encouraged to learn the uncertainty of the content that is relevant to the diversity of observers' opinions, i.e., user disagreement. Extensive experiments have demonstrated that the proposed multi-task aesthetic model achieves state-of-the-art performance on two different types of aesthetic datasets, i.e., AVA and TMGA., Comment: 5 pages
Published: 2021
Full Text: View/download PDF

46. Implicitly using Human Skeleton in Self-supervised Learning: Influence on Spatio-temporal Puzzle Solving and on Video Action Recognition

Author: Patrick Le Callet, Laurent Dollé, Mathieu Riand, Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), CEA Tech Pays-de-la-Loire (DP2L), CEA Tech en régions (CEA-TECH-Reg), Direction de Recherche Technologique (CEA) (DRT (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Direction de Recherche Technologique (CEA) (DRT (CEA)), and Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)
Subjects: Self supervised learning, Computer science, business.industry, Skeleton Keypoints, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI], Siamese Network, Human skeleton, medicine.anatomical_structure, Action Recognition, medicine, Action recognition, Artificial intelligence, Few-shot Learning, business, Self-supervised Learning
Abstract: In this paper we studied the influence of adding skeleton data on top of human actions videos when performing self-supervised learning and action recognition. We show that adding this information without additional constraints actually hurts the accuracy of the network; we argue that the added skeleton is not considered by the network and seen as a noise masking part of the natural image. We bring first results on puzzle solving and video action recognition to support this hypothesis.
Published: 2021
Full Text: View/download PDF

47. A Subjective Study of Multi-Dimensional Aesthetic Assessment for Mobile Game Image

Author: Suiyi Ling, Yanqing Jing, Like Zhang, Yundi Guo, Wang Junle, Patrick Le Callet, Wenming Huang, Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Image Perception Interaction (LS2N - équipe IPI), and Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST)
Subjects: Harmony (color), Multimedia, Computer science, Image quality, media_common.quotation_subject, Colorfulness, 010103 numerical & computational mathematics, 02 engineering and technology, computer.software_genre, 01 natural sciences, Rendering (computer graphics), [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], 0202 electrical engineering, electronic engineering, information engineering, Benchmark (computing), 020201 artificial intelligence & image processing, Quality (business), 0101 mathematics, Dimension (data warehouse), Game Developer, computer, ComputingMilieux_MISCELLANEOUS, media_common
Abstract: Nowadays, mobile gaming has become one of the most rapidly developing fields boosted by fast-evolving techniques, which also gradually becomes one of the biggest parts of modern digital entertainment. With the exponential growth of users, published mobile games, and higher expectations for gaming experiences, multi-dimensional aesthetic assessment is essential in providing guidance for graphic/game developers, quality control of the overall gaming system, and achieving a better trade-off between gaming image quality and the rendering complexity (limited by the device performance). So far, most of the relative researches have been limited to only one dimension evaluation e.g., quality assessment considering streaming artifacts, which neglects other important aesthetic-related perspectives. In this paper, a comprehensive subjective study is presented considering multi-dimensional aesthetic factors (i.e., the fineness, color harmony, colorfulness, and overall quality) of mobile gaming images. Throughout extensive conducted experiments on the collected large-scale dataset, we discuss the relationships between different dimensions, and benchmark different image metrics designed for various purposes.
Published: 2020
Full Text: View/download PDF

48. Unpaired PET/CT image synthesis of liver region using CycleGAN

Author: Constance Fourcade, Mario Campone, Mathieu Rubeaux, Ludovic Ferrer, Marie Lacombe, Gianmarco Santini, Noémie Moreau, Vincent Fleury, Pascal Jézéquel, Caroline Rousseau, Keosys Medical Imaging [Saint-Herblain] (KMI), Image Perception Interaction (LS2N - équipe IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-École Centrale de Nantes (Nantes Univ - ECN), Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes université - UFR des Sciences et des Techniques (Nantes univ - UFR ST), Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Nantes Université (Nantes Univ)-Nantes Université - pôle Sciences et technologie, Nantes Université (Nantes Univ)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Nantes Université (Nantes Univ), Nuclear Oncology (CRCINA-ÉQUIPE 13), Centre de Recherche en Cancérologie et Immunologie Nantes-Angers (CRCINA), Université d'Angers (UA)-Université de Nantes (UN)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Centre hospitalier universitaire de Nantes (CHU Nantes)-Université d'Angers (UA)-Université de Nantes (UN)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Centre hospitalier universitaire de Nantes (CHU Nantes), Institut de Cancérologie de l'Ouest [Angers/Nantes] (UNICANCER/ICO), UNICANCER, Integrated Center for Oncology [Angers] (ICO), Stress Adaptation and Tumor Escape in Breast Cancer (CRCINA-ÉQUIPE 8), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Nantes Université - École Centrale de Nantes (Nantes Univ - ECN), Bernardo, Elizabeth, Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Nantes - UFR de Médecine et des Techniques Médicales (UFR MEDECINE), Université de Nantes (UN)-Université de Nantes (UN)-Centre hospitalier universitaire de Nantes (CHU Nantes)-Centre National de la Recherche Scientifique (CNRS)-Université d'Angers (UA)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Nantes - UFR de Médecine et des Techniques Médicales (UFR MEDECINE), and Université de Nantes (UN)-Université de Nantes (UN)-Centre hospitalier universitaire de Nantes (CHU Nantes)-Centre National de la Recherche Scientifique (CNRS)-Université d'Angers (UA)
Subjects: Paired Data, PET-CT, Computer science, business.industry, Pattern recognition, Image processing, [SDV.CAN]Life Sciences [q-bio]/Cancer, 030218 nuclear medicine & medical imaging, Image synthesis, 03 medical and health sciences, 0302 clinical medicine, [SDV.CAN] Life Sciences [q-bio]/Cancer, Body region, Artificial intelligence, business, 030217 neurology & neurosurgery
Abstract: International audience; Cross-modality synthesis represent nowadays a promising application in medical image processing to manage the problem of paired data scarcity. In this work we designed and trained a CycleGAN model to generate PET/CT data from 2D slices collected from the liver body region of twelve patients. The results obtained from the six test patients show how our model can outperform baseline CycleGAN framework and effectively be used for synthesizing artificial images to be used for data augmentation or dataset completion.
Published: 2020
Full Text: View/download PDF

49. Deep learning approaches for bone and bone lesion segmentation on 18FDG PET/CT imaging in the context of metastatic breast cancer

Author: Gianmarco Santini, Mario Campone, Mathilde Colombié, and Nicolas Normand, Mathieu Rubeaux, Ludovic Ferrer, Constance Fourcade, Camille Guillerminet, Marie Lacombe, Caroline Rousseau, Noémie Moreau, Image Perception Interaction (IPI), Laboratoire des Sciences du Numérique de Nantes (LS2N), Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Keosys, Institut de Cancérologie de l'Ouest [Angers/Nantes] (UNICANCER/ICO), UNICANCER, Centre de Recherche en Cancérologie et Immunologie Nantes-Angers (CRCINA), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Nantes - UFR de Médecine et des Techniques Médicales (UFR MEDECINE), Université de Nantes (UN)-Université de Nantes (UN)-Centre hospitalier universitaire de Nantes (CHU Nantes)-Centre National de la Recherche Scientifique (CNRS)-Université d'Angers (UA), Signal, IMage et Son (SIMS ), Image Perception Interaction (LS2N - équipe IPI), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Université d'Angers (UA)-Université de Nantes (UN)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Centre hospitalier universitaire de Nantes (CHU Nantes), Signal, IMage et Son (LS2N - équipe SIMS ), and Fourcade, Constance
Subjects: [INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI], medicine.medical_specialty, [INFO.INFO-IM] Computer Science [cs]/Medical Imaging, Breast Neoplasms, Context (language use), [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI], 030218 nuclear medicine & medical imaging, 03 medical and health sciences, Deep Learning, 0302 clinical medicine, Fluorodeoxyglucose F18, Positron Emission Tomography Computed Tomography, [INFO.INFO-IM]Computer Science [cs]/Medical Imaging, medicine, Humans, Segmentation, Prospective Studies, Prospective cohort study, business.industry, Deep learning, medicine.disease, 18fdg pet ct, Metastatic breast cancer, 3. Good health, Bone lesion, 030220 oncology & carcinogenesis, Radiology, Artificial intelligence, Tomography, Tomography, X-Ray Computed, business
Abstract: International audience; 18 FDG PET/CT imaging is commonly used in diagnosis and follow-up of metastatic breast cancer, but its quantitative analysis is complicated by the number and location heterogeneity of metastatic lesions. Considering that bones are the most common location among metastatic sites, this work aims to compare different approaches to segment the bones and bone metastatic lesions in breast cancer. Two deep learning methods based on U-Net were developed and trained to segment either both bones and bone lesions or bone lesions alone on PET/CT images. These methods were cross-validated on 24 patients from the prospective EPICURE seinmeta metastatic breast cancer study and were evaluated using recall and precision to measure lesion detection, as well as the Dice score to assess bones and bone lesions segmentation accuracy. Results show that taking into account bone information in the training process allows to improve the precision of the lesions detection as well as the Dice score of the segmented lesions. Moreover, using the obtained bone and bone lesion masks, we were able to compute a PET bone index (PBI) inspired by the recognized Bone Scan Index (BSI). This automatically computed PBI globally agrees with the one calculated from ground truth delineations. Clinical relevance - We propose a completely automatic deep learning based method to detect and segment bones and bone lesions on 18 FDG PET/CT in the context of metastatic breast cancer. We also introduce an automatic PET bone index which could be incorporated in the monitoring and decision process.
Published: 2020
Full Text: View/download PDF

50. Apprentissage profond appliqué à la classification d’images microscopiques embryonnaires

Author: Gomez, Tristan, Mouchère, Harold, Fréour, Thomas, Feyeux, Magalie, Laboratoire des Sciences du Numérique de Nantes (LS2N), IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS), Image Perception Interaction (IPI), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique Bretagne-Pays de la Loire (IMT Atlantique), Centre de Recherche en Transplantation et Immunologie (U1064 Inserm - CRTI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Nantes - UFR de Médecine et des Techniques Médicales (UFR MEDECINE), Université de Nantes (UN)-Université de Nantes (UN), Structure fédérative de recherche François Bonamy (SFR François Bonamy), Université de Nantes (UN)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Institut de Recherche en Santé de l'Université de Nantes (IRS-UN), DL4IVF, ANR-16-IDEX-0007,NExT (I-SITE),NExT (I-SITE)(2016), Mouchère, Harold, NExT (I-SITE) - - NExT (I-SITE)2016 - ANR-16-IDEX-0007 - IDEX - VALID, Université de Nantes - UFR des Sciences et des Techniques (UN UFR ST), Université de Nantes (UN)-Université de Nantes (UN)-École Centrale de Nantes (ECN)-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), and Image Perception Interaction (LS2N - équipe IPI)
Subjects: apprentissage profond, fécondation in vitro, modèle temporel, [INFO.INFO-NE] Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], cross validation, time model, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE], ResNet, machine learning, [INFO.INFO-TI] Computer Science [cs]/Image Processing [eess.IV], [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], apprentissage machine deep learning, 3D convolution, convolution 3D, LSTM, in vitro fertilization, CNN, validation croisée
Abstract: National audience; Dans le domaine de la fécondation in vitro (FIV), l'imagerie par time-lapse (ITL) est une technologie qui produit une vidéo montrant le développement de l'embryon pendant ses premiers jours. L'ITL permet d'annoter les instants auxquels commencent et finissent les différentes phases de développement de l'embryon. Cependant, le processus d'annotation prend du temps et nécessite des experts. Dans cet article, nous montrons que pour prédire les phases visibles dans les images issues de vidéos ITL, un ResNet-3D est meilleur qu'un équivalent 2D ou un ResNet-LSTM.
Published: 2020

Catalog

Books, media, physical & digital resources

See catalog results

Searchworks

Select search scope, currently: Articles Catalog books, media & more in Jio Institute collections Articles journal articles & other e-resources

Search

Search Constraints

Refine your results

Search Limiters

Topic

Publication Year Range

Language

Journal

Database

Publisher

61 results on '"Image Perception Interaction (LS2N - équipe IPI)"'

Search Results

Catalog

Select search scope, currently: Articles

Catalog

books, media & more in Jio Institute collections

Articles

journal articles & other e-resources