Search

The deliverable primarily covers the initial implementation of the Policies and Explainability and Interpretation Framework V1. It describes and demonstrates the tools and algorithms developed during this implementation phase. The document is structured into several chapters, starting with an introduction to the project and the deliverable. The subsequent chapters delve into the background, state of the art, solution design, implementation details, and finally, the conclusions and future works. Overall, this deliverable lays the groundwork for the project's AI models and tools, setting the stage for further development and refinement in subsequent phases.

International audience; Jacovi et al. in 2018 proposed a method for the explanation of Convolutional Neural Networks designed for text classification (Text-CNN). The problem with this method is that it only applies to Text-CNN with no hidden layer in the fully connected part. We will therefore find ourselves limited in terms of accuracy if we want to use this method. In addition, the LRP (Layer-wise Relevance Propagation) method makes it possible to compute the contributions of inputs of neural networks with multiple hidden layers, but cannot be applied directly to Text-CNNs because unlike images where each pixel has a meaning taken alone, each component of a vector representation (word embedding) of a word taken alone has no particular meaning. Therefore, we propose in this article to extend the method of Jacovi et al. by using the LRP method in order to be able to explain Text-CNN whose fully connected layer is deep. The qualitative evaluation suggests that the explanations provided by the proposed method are consistent on text classification problems such as Question-Answering (QA) and Sentiment Analysis. Moreover, the quantitative evaluation shows that the explanations provided are 100% faithful to the model.; Jacovi et al. en 2018 ont proposé une méthode pour l'explication des Réseaux de Neurones Convolutifs conçus pour la classification des textes (Text-CNN). Le problème avec cette méthode est qu'elle ne s'applique que sur les Text-CNN qui n'ont pas de couche cachée dans la partie densément connectée. On se retrouvera donc limité en terme de performance si on veut utiliser cette méthode d'explication. Par ailleurs, la méthode LRP (Layer-wise Relevance Propagation) permet de calculer les contributions des caractéristiques en entrée des réseaux de neurones avec un nombre quelconque de couches cachées, mais ne peut s'appliquer directement sur les Text-CNN car contrairement aux images où chaque pixel a un sens pris seul, chaque composante d'une représentation vectorielle (word embedding) d'un mot prise seule n'a pas de signification particulière. C'est ainsi que nous proposons dans cet article, d'étendre la méthode de Jacovi et al. en mettant à contribution la méthode LRP afin de pouvoir expliquer les Text-CNN dont la partie densément connectée est profonde (possède plusieurs couches). L'évaluation qualitative laisse percevoir que les explications fournies par la méthode proposée sont cohérentes sur les problèmes de classification des textes comme le Question-Answering (QA) ainsi que l'analyse des sentiments. Par ailleurs, l'évaluation quantitative montre que les explications fournies sont fidèles à 100% au modèle.

This paper aims at bringing some light and understanding to the field of deep learning for dynamic point cloud processing. Specifically, we focus on the hierarchical features learning aspect, with the ultimate goal of understanding which features are learned at the different stages of the process and what their meaning is. Last, we bring clarity on how hierarchical components of the network affect the learned features and their importance for a successful learning model. This study is conducted for point cloud prediction tasks, useful for predicting coding applications.

International audience; Attention maps in neural models for NLP are appealing to explain the decision made by a model, hopefully emphasizing words that justify the decision. While many empirical studies hint that attention maps can provide such justification from the analysis of sound examples, only a few assess the plausibility of explanations based on attention maps, i.e., the usefulness of attention maps for humans to understand the decision. These studies furthermore focus on text classification. In this paper, we report on a preliminary assessment of attention maps in a sentence comparison task, namely natural language inference. We compare the cross-attention weights between two RNN encoders with human-based and heuristic-based annotations on the eSNLI corpus. We show that the heuristic reasonably correlates with human annotations and can thus facilitate evaluation of plausible explanations in sentence comparison tasks. Raw attention weights however remain only loosely related to a plausible explanation.

This work aims to evaluate explainable classification methods for the detection of fish species from hydroacoustic data acquired by echo sounders at a region near the coastline of south and southeastern Brazil. Decision trees and fuzzy rule-based methods were adopted. The fitted models were evaluated by quality measures based on the performance of the classifiers and also by an expert which analyzed the usefulness of the rules on describing the schools. The models learned by the algorithms performed well for the available data and were able to represent the documented behavior of the species considered in the studied region, according to the literature. This study was supported by the Spanish Ministry of Science (TIN2016-77356-P, PID2019-108392GB-I00 AEI/10.13039/501100011033), PNPD/CAPES (464880/2019-00), CNPq (301618/2019-4), FAPERGS (19/2551-0001660), and CAPES Financial Code 001.

As already pointed out by a constantly growing literature, explainability in recommender systems field is a key aspect to increase users' satisfaction. With the increase of user generated content, tags have proven to be highly relevant when it comes to describe either users or items. A number of strategies that rely on tags have been proposed, yet, many of these algorithms exploit the frequency of user-tags interactions to gain information. We argue that a pure frequentist description might lack of specificity to grasp user's peculiar tastes. Therefore, we propose a novel approach based on game theory that tries to find the best trade-off between generality and detailing. The identified user's description can be used to keep her in the loop and allows the user to have control over system's knowledge. Additionally, we propose a user interface that embeds the proposed user's description and it can be used by the user herself to guide her catalogue's exploration toward novel and serendipitous items.

The main goal of this research report is to define a generic framework for black-box explanation methods in order to make it easier to compare and classify different approaches. We focus on two components of this framework, called respectively “Sampling” and “Generation”, which are characterized formally and used to build a taxonomy of explanation methods. We also describe precisely how each method can be expressed in the framework.; Dans ce rapport de recherche, nous proposons un cadre générique pour décrire les méthodes d’explication fonctionnant en mode “boîte noire”, l’objectif étant de faciliter la comparaison et la classification de ces méthodes. Nous définissons formellement deux composantes principales, appelées respectivement l’“Echantillonnage” et la “Génération”, qui sont ensuite utilisées pour construire une taxonomie des méthodes d’explication. Nous décrivons aussi précisément la manière dont les méthodes de la littérature s’expriment dans ce cadre.

The main goal of this research report is to define a generic framework for black-box explanation methods in order to make it easier to compare and classify different approaches. We focus on two components of this framework, called respectively “Sampling” and “Generation”, which are characterized formally and used to build a taxonomy of explanation methods. We also describe precisely how each method can be expressed in the framework.; Dans ce rapport de recherche, nous proposons un cadre générique pour décrire les méthodes d’explication fonctionnant en mode “boîte noire”, l’objectif étant de faciliter la comparaison et la classification de ces méthodes. Nous définissons formellement deux composantes principales, appelées respectivement l’“Echantillonnage” et la “Génération”, qui sont ensuite utilisées pour construire une taxonomie des méthodes d’explication. Nous décrivons aussi précisément la manière dont les méthodes de la littérature s’expriment dans ce cadre.

Este trabalho visa desenvolver um modelo explicável de sistema de classificação de organismos marinhos a partir de dados obtidos por meio de ecossondas. As ecossondas funcionam normalmente acopladas na parte de baixo de uma embarcação, enviando sinais acústicos para detectar peixes, moluscos, zooplâncton ou outros objetos na coluna de água, gerando registros acústicos que aparecem nos ecogramas. Atualmente, o método utilizado para classificar os registros acústicos é a caracterização de ecotipos, onde certos padrões nos registros acústicos são identificados visualmente por um especialista que, tendo por base informações chamadas descritores, determina uma classe ao registro. Para fazer a classificação utilizando meios computacionais, descritores foram extraídos de ecogramas obtidos de pesquisa pesqueira, que então foram tratados para filtrar as características que descreviam melhor os registros contidos nos ecogramas. Estes dados foram utilizados para o gerar os modelos a partir de três algoritmos de classificação de fácil explicabilidade, um que utiliza árvores de decisão e dois baseados em regras fuzzy. Os modelos obtidos foram avaliados por medidas de acurácia baseadas nos desempenhosdas classificações no treinamento. Os modelos foram também avaliados por um especialista da área, para análise de sua utilidade para descrever as espécies trabalhadas e o que poderia ser observado nestes resultados. Os modelos gerados pelos algoritmos tiveram um bom desempenho para os dados disponíveis, aonde os dados foram principalmente classificados por descritores geográficos e que, de acordo com um especialista da área, representam uma distribuição próxima da que realmente acontece para as espécies estudadas nessa região e época do ano trabalhadas. This work aims to develop an explainable model of classification system of marine organisms from data obtained through ecosystems. Echo sounders typically work attached to the underside of a vessel, sending acoustic signals to detect fish, mollusks, zooplankton, or other objects in the water column, generating acoustic records that appear on echograms. Currently, the method used to classify acoustic records is the characterization of ecotypes, where certain patterns in acoustic records are visually identified by a specialist who, based on information called descriptors, determines a class for the record. To make the classification using computational means, descriptors were extracted from ecograms obtained from fishing research, which were then treated to filter the characteristics that best described the records contained in the ecograms. These data were used to generate the models from three easy-to-explain classification algorithms, one using decision trees and two fuzzy rules. The obtained models were evaluated by accuracy measures based on the performance of the training classifications. The models were also evaluated by an expert in the field, to analyze their usefulness to describe the worked species and what could be observed in these results. The models generated by the algorithms performed well for the available data, where the data were mainly classified by geographic descriptors and which, according to an expert in the field, represent a close distribution of what actually happens for the species studied in this region and time of the year of the data worked.