Search

International audience; In the context of sensory or higher-level cognitive processing, we present a recurrent neural network model, similar to the popular dynamic neural field (DNF) model, for performing approximate probabilistic computations. The model is biologically plausible, avoids impractical schemes such as log-encoding and noise assumptions, and is well-suited for working in stacked hierarchies. By Lyapunov analysis, we make it very plausible that the model computes the maximum a posteriori (MAP) estimate given a certain input that may be corrupted by noise. Key points of the model are its capability to learn the required posterior distributions and represent them in its lateral weights, the interpretation of stable neural activities as MAP estimates, and of latency as the probability associated with those estimates. We demonstrate for in simple experiments that learning of posterior distributions is feasible and results in correct MAP estimates. Furthermore, a pre-activation of field sites can modify attractor states when the data model is ambiguous, effectively providing an approximate implementation of Bayesian inference.

International audience; We propose an incremental, highly parallelizable, and constant-time complexity neural learning architecture for multi-class classification (and regression) problems that remains resource-efficient even when the number of input dimensions is very high (≥ 1000). This so-called projection-prediction (PROPRE) architecture is strongly inspired by biological information processing in that it uses a prototype-based, topologically organized hidden layer trained with the SOM learning rule that updates hidden layer weights whenever an error occurs. The SOM learning adapts only the weights of localized neural sub-populations that are similar to the input, which explicitly avoids the catastrophic forgetting effect of MLPs in case new input statistics are presented. The readout layer applies linear regression to hidden layer activities subjected to a transfer function, making the whole system capable of representing strongly non-linear decision boundaries. The resource-efficiency of the algorithm stems from the fact of approximating similarity in the input space by proximity in the SOM layer due to the topological SOM projection property. This avoids the storage of inter-cluster distances (quadratic in number of hidden layer) or input space covariance matrices (quadratic in input dimensions) as K-means, RBF or LWPR would have to do. Tests on the popular MNIST handwritten digit benchmark show that the algorithm compares favorably to state-of-the-art results, and parallelizability is demonstrated by analyzing the efficiency of a parallel GPU implementation of the architecture.

International audience; PROPRE is a generic and cortically inspired framework that provides online input/output relationship learning. The input data flow is projected on a self-organizing map that provides an internal representation of the current stimulus. From this representation, the system predicts the value of the output target. A predictability measure, based on the monitoring of the prediction quality, modulates the projection learning so that to favor learning of representations that are helpful to predict the output. In this article, we study PROPRE when the input/output relationship is only defined in a small subspace of the input space, that we define as a partially learnable environment. This problem, which is not typical of the machine learning field, is however crucial for the robotic developmental field. Indeed, robots face high dimensional sensory-motor environments where large areas of these sensory-motor spaces are not learnable since a motor action cannot have a consequence on every perception each time. We show that the use of the predictability measure in PROPRE leads to an autonomous gathering of local representations where the input data are related to the output value, thus providing good classification performance as the system will learn the input/output function only where it is defined.

International audience; Self-organizing map (SOM) is a powerful paradigm that is extensively applied for clustering and visualization purpose. It is also used for regression learning, especially in robotics, thanks to its ability to provide a topological projection of high dimensional non linear data. In this case, data extracted from the SOM are usually restricted to the best matching unit (BMU), which is the usual way to use SOM for classification , where class labels are attached to individual neurons. In this article, we investigate the influence of considering more information from the SOM than just the BMU when performing regression. For this purpose , we quantitatively study several output functions for the SOM, when using these data as input of a linear regression, and find that the use of additional activities to the BMU can strongly improve regression performance. Thus, we propose an unified and generic framework that embraces a large spectrum of models from the traditional way to use SOM, with the best matching unit as output, to models related to the radial basis function network paradigm, when using local receptive field as output.

In this article, we present some preliminary work on integrating an artificial curiosity mechanism in PROPRE, a generic and modular neural architecture, to obtain online, open-ended and active learning of a sensory-motor space, where large areas can be unlearnable. PROPRE consists of the combination of the projection of the input motor flow, using a self-organizing map, with the regression of the sensory output flow from this projection representation, using a linear regression. The main feature of PROPRE is the use of a predictability module that provides an interestingness measure for the current motor stimulus depending on a simple evaluation of the sensory prediction quality. This measure modulates the projection learning so that to favor the representations that predict the output better than a local average. Especially, this leads to the learning of local representations where an input/output relationship is defined [1]. In this article, we propose an artificial curiosity mechanism based on the monitoring of learning progress, as proposed in [2], in the neighborhood of each local representation. Thus, PROPRE simultaneously learns interesting representations of the input flow (depending on their capacities to predict the output) and explores actively this input space where the learning progress is the higher. We illustrate our architecture on the learning of a direct model of an arm whose hand can only be perceived in a restricted visual space. The modulation of the projection learning leads to a better performance and the use of the curiosity mechanism provides quicker learning and even improves the final performance.

International audience; This work investigates the mass transfer of the Airborne Molecular cross Contamination (AMCs) between the Front Opening Unified Pod (FOUP) and wafer (silicon substrates) during the microelectronics devices manufacturing using dynamic boundary conditions. Such cross-contamination phenomena lead to detrimental impact on production yield in microelectronic industry and a predictive approach using modelling and computational methods is a very strong way to understand and qualify the AMCs cross contamination processes. The FOUP is made of polymeric materials and it is considered as a heterogeneous porous media, it can adsorb and desorb the contaminant, thus the modelled processes are the contamination of two-component in transient flow. Coupled diffusion and convection-diffusion model with heat effects are used to define the phenomena. The present methodology is, first using the optimization methods with the numerical solution in order to define the physical constants of various materials which have been studied experimentally and separately, and the second using the finite element methods including these physical constants and relevant interface condition in order to take into account the adsorption kinetics law. Numerical methods to solve the problem are proposed. The dynamics behaviour of the AMCs analysis was determined thanks to the switch of Dirichlet to Neumann condition. The mathematical model preserves the classical forms of the diffusion and convection diffusion equations and yields to consistent form of the Fick's law. The computed results are in correlation with the experimental measurements. Some numerical results are presented in this work.

International audience; PROPRE is a generic and modular neural learning paradigm that autonomously extracts meaningful concepts of multimodal data flows driven by predictability across modalities in an unsupervised, incremental and online way. For that purpose, PROPRE consists of the combination of projection and prediction. Firstly, each data flow is topologically projected with a self-organizing map, largely inspired from the Kohonen model. Secondly, each projection is predicted by each other map activities, by mean of linear regressions. The main originality of PROPRE is the use of a simple and generic predictability measure that compares predicted and real activities for each modal stream. This measure drives the corresponding projection learning to favor the mapping of predictable stimuli across modalities at the system level (i.e. that their predictability measure overcomes some threshold). This predictability measure acts as a self-evaluation module that tends to bias the representations extracted by the system so that to improve their correlations across modalities. We already showed that this modulation mechanism is able to bootstrap representation extraction from previously learned representations with artificial multimodal data related to basic robotic behaviors and improves performance of the system for classification of visual data within a supervised learning context. In this article, we improve the self-evaluation module of PROPRE, by introducing a sliding threshold, and apply it to the unsupervised classification of gestures caught from two time- of-flight (ToF) cameras. In this context, we illustrate that the modulation mechanism is still useful although less efficient than purely supervised learning.

International audience; PROPRE is a generic and modular unsupervised neural learning paradigm that extracts meaningful concepts of multimodal data flows based on predictability across modalities. It consists on the combination of three modules. First, a topological projection of each data flow on a self-organizing map. Second, a decentralized prediction of each projection activity from each others map activities. Third, a predictability measure that compares predicted and real activities. This measure is used to modulate the projection learning so that to favor the mapping of predictable stimuli across modalities. In this article, we use Kohonen map for the projection module, linear regression for the prediction one and we propose multiple generic predictability measures. We illustrate the properties and performances of PROPRE paradigm on a challenging supervised classification task of visual pedestrian data. The modulation of the projection learning by the predictability measure improves significantly classification performances of the system independently of the measure used. Moreover, PROPRE provides a combination of interesting functional properties, such as a dynamical adaptation to input statistic variations, that is rarely available in other machine learning algorithms.

International audience; PROPRE is a generic and semi-supervised neural learning paradigm that extracts meaningful concepts of multimodal data flows based on predictability across modalities. It consists on the combination of two computational paradigms. First, a topological projection of each data flow on a self-organizing map (SOM) to reduce input dimension. Second, each SOM activity is used to predict activities in all other SOMs. Predictability measure, that compares predicted and real activities, is used to modulate the SOM learning to favor mutually predictable stimuli. In this article, we study PROPRE applied to a classical visual pedestrian data classification task. The SOM learning modulation introduced in PROPRE improves significantly classification performance.

International audience; In this article, we present an original neural space/latency code, integrated in a multi-layered neural hierarchy, that offers a new perspective on probabilistic inference operations. Our work is based on the dynamic neural field paradigm that leads to the emergence of activity bumps, based on recurrent lateral interactions, thus providing a spatial coding of information. We propose that lateral connections represent a data model, i.e., the conditional probability of a "true" stimulus given a noisy input. We propose furthermore that the resulting attractor state encodes the most likely "true" stimulus given the data model, and that its latency expresses the confidence in this interpretation. Thus, the main feature of this network is its ability to represent, transmit and integrate probabilistic information at multiple levels so that to take near-optimal decisions when inputs are contradictory, noisy or missing. We illustrate these properties on a three-layered neural hierarchy receiving inputs from a simplified robotic object recognition task. We also compare the network dynamics to an explicit probabilistic model of the task, to verify that it indeed reproduces all relevant properties of probabilistic processing.

This paper presents an exploratory study designed for children with Autism Spectrum Disorders (ASD) that investigates children's awareness of being imitated by a robot in a play/game scenario. The Nao robot imitates all the arm movement behaviors of the child in real-time in dyadic and triadic interactions. Different behavioral criteria (i.e., eye gaze, gaze shifting, initiation and imitation of arm movements, smile/laughter) were analyzed based on the video data of the interaction. The results confirm only parts of the research hypothesis. However, these results are promising for the future directions of this work., Proceedings of the 1st International Conference on Innovative Technologies for Autism Spectrum Disorders. ASD: Tools, Trends and Testimonials (ITASD), Spain, 2012

Imitation plays an important role in development, being one of the precursors of social cognition. Even though some children with autism imitate spontaneously and other children with autism can learn to imitate, the dynamics of imitation is affected in the large majority of cases. Existing studies from the literature suggest that robots can be used to teach children with autism basic interaction skills like imitation. Based on these findings, in this study, we investigate if children with autism show more social engagement when interacting with an imitative robot (Fig 1) compared to a human partner in a motor imitation task., Proceedings of the 2nd International Conference on Innovative Research in Autism (IRIA), France, 2012

International audience; The Bio-inspired Perception Sensor (BIPS) component is a small and low power bio-inspired on-chip device which has been used in different computer vision applications (traffic analysis, driving assistance , object tracking). It caught the attention of the embedded vision community, since its specifications could help overcoming the time, size, weight and energy bottlenecks that still limits the current development of computer vision systems. For a long time, the lack of mathematical and algorithmic models of the component has prevented it to spread among the research community. But the recent formalization of BIPS basic functions and mechanisms has allowed to develop numerical models and simulators, in order to better evaluate the advantages and limitations of the concept. In this paper, we experimentally address the generalization capability of the BIPS concept, by evaluating it on the road lane detection application. This allows to illustrate how its parameters can be adapted for a specific vision task. This approach permits to automatically instantiate the main parameters, which stabilizes the system output and improves its performance. The obtained results reach the level of the caltech-lanes reference.

International audience; We present a new replay-based method of continual classification learning that we term "conditional replay" which generates samples and labels together by sampling from a distribution conditioned on the class. We compare conditional replay to another replay-based continual learning paradigm (which we term "marginal replay") that generates samples independently of their class and assigns labels in a separate step. The main improvement in conditional replay is that labels for generated samples need not be inferred, which reduces the margin for error in complex continual classification learning tasks. We demonstrate the effectiveness of this approach using novel and standard benchmarks constructed from MNIST and FashionMNIST data, and compare to the regularization-based \textit{elastic weight consolidation} (EWC) method.

International audience; Robots are progressively moving into spaces that have been primarily shaped by human agency; they collaborate with human users in different tasks that require them to understand human language so as to behave appropriately in space. To this end, a stubborn challenge that we address in this paper is inferring the syntactic structure of language, which embraces grounding parts of speech (e.g., nouns, verbs, and prepositions) through visual perception, and induction of Combinatory Categorial Grammar (CCG) in situated human-robot interaction. This could pave the way towards making a robot able to understand the syntactic relationships between words (i.e., understand phrases), and consequently the meaning of human instructions during interaction, which is a future scope of this current study. I. INTRODUCTION Creating interactive social robots able to collaborate with human users in different tasks requires high-level spatial intelligence that could make them able to discover and interact with their surroundings. Developing this spatial intelligence involves grounding language (action verbs, object characteristics (i.e., color and geometry), and spatial prepositions) and the underlying syntactic structure through sensory information so as to make a robot able to understand human instructions in the physical world. Understanding syntactic structure of language has been intensively investigated in the literature of cognitive robotics and computational linguistics. In cognitive robotics, different research studies proposed computational models for grounding nouns, verbs, adjectives, and prepositions encoding spatial relationships between objects [1, 2, 22, 26, 38]. However, they have not investigated grammar understanding at the phrase level, which constitutes a higher level than grounding words through perception. Meanwhile, in computational linguistics, recent studies presented models for inducing combinatory syntactic structure of language [5, 15]; however, they used annotated databases for grammar induction where each word has a corresponding syntactic tag (as a noun, verb, etc.). This last point illustrates the important role that cognitive robotics could play in grammar induction through grounding parts of speech in visual perception so as to allow for learning the latent syntactic structure of phrases in a developmentally plausible manner. In this study, we build on the model of Bisk and Hockenmaier [5] for grammar induction, and propose an extended probabilistic

International audience; An important increase of the number of jobs is expected in the next few years on markets related to Unmanned Aerial Vehicles (UAVs). Therefore educational curricula in this field of expertise are subject to more and more demands from students. In addition and complement to traditional courses, Massive Open Online Courses (MOOCs) are also getting a growing interest from students.

International audience; The proposal of this study is to analyze the texture pattern recognition over the local jet space looking forward to improve the texture characterization. Local jets decompose the image based on partial derivatives allowing the texture feature extraction be exploited in different levels of geometrical structures. Each local jet component evidences a different local pattern, such as, flat regions, directional variations and concavity or convexity. Subsequently , a texture descriptor is used to extract features from 0th, 1st and 2nd-derivative components. Four well-known databases (Brodatz, Vistex, Usptex and Outex) and four texture descriptors (Fourier descriptors, Gabor filters, Local Binary Pattern and Local Binary Pattern Variance) were used to validate the idea, showing in most cases an increase of the success rates.

International audience; Future intelligent robots are expected to be able to adapt continuously to their environment. For this purpose, recognizing new objects and learning new words through interactive learning with humans is fundamental. Such setup results in ambiguous teaching data which humans have been shown to address using cross-situational learning, i.e. by analyzing common factors between multiple learning situations. Moreover, they have been shown to be more efficient when actively choosing the learning samples, e.g. which object they want to learn. Implementing such abilities on robots can be performed by latent-topic learning models such as Non-Negative Matrix Factorization or Latent Dirichlet Allocation. These cross-situational learning methods tackle referential and linguistic ambiguities, and can be associated with active learning strategies. We propose two such methods: the Maximum Reconstruction Error based Selection (MRES) and Confidence Base Exploration (CBE). We present extensive experiments using these two learning algorithms through a systematic analysis on the effects of these active learning strategies in contrast with random choice. In addition, we study the factors underlying the active learning by focusing on the use of sample repetition, one of the learning behaviors that have been shown to be important for humans.

International audience; This work is based on a questioning of the quality metrics used by deep neural networks performing depth prediction from a single image, and then of the usability of recently published works on unsu-pervised learning of depth from videos. These works are all predicting depth from a single image, thus it is only known up to an undetermined scale factor, which is not sufficient for practical use cases that need an absolute depth map, i.e. the determination of the scaling factor. To overcome these limitations, we propose to learn in the same unsupervised manner a depth map inference system from monocular videos that takes a pair of images as input. This algorithm actually learns structure-from-motion from motion, and not only structure from context appearance. The scale factor issue is explicitly treated, and the absolute depth map can be estimated from camera displacement magnitude, which can be easily measured from cheap external sensors. Our solution is also much more robust with respect to domain variation and adaptation via fine tuning, because it does not rely entirely on depth from context. Two use cases are considered, unstabilized moving camera videos, and stabilized ones. This choice is motivated by the UAV (for Unmanned Aerial Vehicle) use case that generally provides reliable orientation measurement. We provide a set of experiments showing that, used in real conditions where only speed can be known, our network outperforms competitors for most depth quality measures. Results are given on the well known KITTI dataset [5], which provides robust stabilization for our second use case, but also contains moving scenes which are very typical of the in-car road context. We then present results on a synthetic dataset that we believe to be more representative of typical UAV scenes. Lastly, we present two domain adaptation use cases showing superior robustness of our method compared to single view depth algorithms, which indicates that it is better suited for highly variable visual contexts.

International audience; This paper proposes an improvement of a motion planning approach and a modified model predictive control (MPC) for solving the navigation problem of a team of dynamical wheeled mobile robots in the presence of obstacles in a realistic environment. Planning is performed by a distributed receding horizon algorithm where constrained optimization problems are numerically solved for each prediction time-horizon. This approach allows distributed motion planning for a multi-robot system with asynchronous communication while avoiding collisions and minimizing the travel time of each robot. However, the robots dynamics prevents the planned motion to be applied directly to the robots. Using unicycle-like vehicles in a dynamic simulation, we show that deviations from the planned motion caused by the robots dynamics can be overcome by modifying the optimization problem underlying the planning algorithm and by adding an MPC for trajectory tracking. Results also indicate that this approach can be used in systems subjected to real-time constraint.

International audience; Unmanned systems used for threat detection and identification are still not efficient enough for monitoring autonomously the battlefield. The limitation on size and energy makes those systems unable to use most state-of-the-art computer vision algorithms for recognition. The bio-inspired approach based on the humans peripheral and foveal visions has been reported as a way to combine recognition performance and computational efficiency. As a low resolution camera observes a large zone and detects significant changes, a second camera focuses on each event and provides a high resolution image of it. While such biomimetic existing approaches usually separate the two vision modes according to their functionality (e.g. detection, recognition) and to their basic primitives (i.e. features, algorithms), our approach uses common structures and features for both peripheral and foveal cameras, thereby decreasing the computational load with respect to the previous approaches. The proposed approach is demonstrated using simulated data. The outcome proves particularly attractive for real time embedded systems, as the primitives (features and classifier) have already proven good performances in low power embedded systems. This first result reveals the high potential of dual views fusion technique in the context of long duration unmanned video surveillance systems. It also encourages us to go further into miming the mechanisms of the human eye. In particular, it is expected that adding a retro-action of the fovea towards the peripheral vision will further enhance the quality and efficiency of the detection process.

International audience; Optical flow computation consists in recovering the apparent motion field between two images with overlapping fields of view. This paper focuses on a subset of optical flow problems, called epipolar flow, where the camera moves inside a scene containing no moving objects. Accurate solutions exist but their high computational complexities make them non suitable for a large panel of real-time applications. We propose a new epipolar flow approach with low computational complexity achieving the best error rate on the non dense KITTI optical flow 2012 benchmark and running 1000× faster than the second ranked approach. On a 4core 3GHz processor, our multi-core implementation computes a semi dense optical flow field of a 450k pixels image in 260ms. It is a significant advance in reducing the running time of accurate optical flow computation. To achieve such results we rely on the epipolar constraints and the local coherence of the optical flow not only to increase accuracy but also to reduce computational complexity. Our contribution is twofold. It is first, the acceleration and the accuracy increase of current RANSAC based visual odometry algorithms via the estimation of a robust sparse flow field, well distributed over the image domain. And then, the estimation of a semi dense flow field leveraging epipo-lar constraints and a propagation scheme to speedup the estimation and reduce error rates.

International audience; Robots are having an important growing role in human social life, which requires them to be able to behave appropriately to the context of interaction so as to create a successful long-term human-robot relationship. A major challenge in developing intelligent systems , which could enhance the interactive abilities of robots, is defining clear metrics and benchmarks for the different aspects of human-robot interaction, like human and robot skills and performances, which could facilitate comparing between systems and avoid application-biased evaluations based on particular measures. The point of evaluating robotic systems through metrics and benchmarks, in addition to some recent frameworks and technologies that could endow robots with advanced cognitive and communicative abilities, are discussed in this technical report that covers the outcome of our recent workshop on current advances in cognitive robotics: Towards Intelligent Social Robots-Current Advances in Cognitive Robotics, in conjunction with the 15th IEEE-RAS Humanoids Conference-Seoul-South Korea-2015 (https://intelligent-robots-ws.ensta-paristech.fr/). Additionally, a summary of an interactive discussion session between the workshop participants and the invited speakers about different issues related to cognitive robotics research is reported.

International audience; Future human-robot collaboration employs language in instructing a robot about specific tasks to perform in its surroundings. This requires the robot to be able to associate spatial knowledge with language to understand the details of an assigned task so as to behave appropriately in the context of interaction. In this paper, we propose a probabilistic framework for learning the meaning of language spatial concepts (spatial prepositions) and object categories based on visual cues representing spatial layouts and geometric characteristics of objects in a tabletop scene. The model investigates unsupervised Part-of-Speech (POS) tagging through a Hidden Markov Model (HMM) that infers the corresponding hidden tags to words. Spatial configurations and geometric characteristics of objects on the tabletop are described through 3D point cloud information that encodes spatial semantics and categories of referents and landmarks in the environment. The proposed model is evaluated through human user interaction with Toyota HSR robot, where the obtained results show the significant effect of the model in making the robot able to successfully engage in interaction with the user in space.

This thesis project will investigate and quantify the contribution of context-aided multimodal perception for tracking moving objects. This research study will be applied to the recognition ofrelevant objects in road traffic environments for Intelligent Vehicles (IV). The results to be obtained will allow us to transpose the proposed concept to a wide range of state-of-the-art sensors and object classes by means of an integrative system approach involving learning methods. In particular, such learning methods will investigate how the embedding into an embodied system providing a multitude of different data sources, can be harnessed to learn 1) without, or with reduced, explicit supervision by exploiting correlations 2) incrementally, by adding to existing knowledge instead of complete retraining every time new data arrive 3) collectively, each learning instance in the system being trained in a way that ensures approximately optimal fusion. Concretely, a tight coupling between object classifiers in multiple modalities as well as geometric scene context extraction will be studied, first in theory, then in the context of road traffic. The novelty of the envisioned integration approach lies in the tight coupling between system components such as object segmentation, object tracking, scene geometry estimation and object categorization based on a probabilistic inference strategy. Such a strategy characterizes systems where all perception components broadcast and receive distributions of multiple possible results together with a probabilistic belief score. In this way, each processing component can take into account the results of other components at a much earlier stage (as compared to just combining final results), thus hugely increasing its computation power, while the application of Bayesian inference techniques will ensure that implausible inputs do not cause negative effects., Cette thèse a pour but d'étudier et de quantifier la contribution de la perception multimodale assistée par le contexte pour suivre des objets en mouvement. Cette étude sera appliquée à la reconnaissance des objets pertinents dans les environnements de la circulation pour les véhicules intelligents (VI). Les résultats à obtenir devront permettre de transposer le concept proposé à un ensemble plus large de capteurs et de classes d'objets en utilisant une approche système intégrative qui implique des méthodes d'apprentissage. En particulier, ces méthodes d'apprentissage vont examiner comment l'implantation dans un système intégré, qui prévoie une multitude des sources de données différentes, peut conduire à apprendre 1) sans ou avec une supervision limitée, réduite en exploitant des corrélations 2) de façon incrémentale à la connaissance stockée au lieu de faire un entraînement complet à chaque fois qu’une nouvelle donnée arrive 3) collectivement à chaque instant d'apprentissage dans le système entraîné d'une manière qui assure approximativement une fusion optimale. Concrètement, le couplage fort entre les classifiers des objets en modalités multiples aussi bien que l'extraction du contexte de la géométrie de la scène sont à étudier: d'abord en théorie, après en application du trafic routier. La nouveauté de l'approche d'intégration envisagée se pose dans le couplage fort entre les composants du système, tels que la segmentation, le suivi des objets, l'estimation de la géométrie de la scène et la catégorisation des objets basée sur la stratégie de l'inference probabiliste.Une telle stratégie caractérise des systèmes où toutes les composants de perception émettent et reçoivent les distributions des résultats possibles avec leur score de croyance probabiliste attribué.De cette façon, chaque composant de traitement peut prendre en compte les résultats des autres composants au niveau plus bas par rapport aux combinaisons des résultats finaux. Cela diminue beaucoup le temps et les ressources pour le calcul, quand les techniques de l'application de l'inference Bayésienne garantissent que les données d'entrée peu plausible n'apportent pas des impactes négatives.

International audience; We introduce COCOTTE (COnstrained Complexity Optimization Through iTerative merging of Experts), an iterative algorithm for discovering discrete, meaningful parameterized skills and learning explicit models of them from a set of behaviour examples. We show that forward-parameterized skills can be seen as smooth components of a locally smooth function and, framing the problem as the constrained minimization of a complexity measure, we propose an iterative algorithm to discover them. This algorithm fits well in the developmental robotics framework, as it does not require any external definition of a parameterized task, but discovers skills parameterized by the action from data. An application of our method to a simulated setup featuring a robotic arm interacting with an object is shown.

International audience; Humans can learn word-object associations from ambiguous data using cross-situational learning and have been shown to be more efficient when actively choosing the learning sample order. Implementing such a capacity in robots has been performed using several models, among which are the latent-topic learning models based on Non-Negative Matrix Factorization and Latent Dirichlet Allocation. We compare these approaches on the same data in a batch and in an incremental learning scenario to analyze their strength and weaknesses and furthermore show that they can be the basis for efficient active learning strategies. The proposed modeling deals with both the referential ambiguity and the noisy linguistic descriptions and is grounding meanings of object's modal features (color and shape) and not only the object identity. The resulting active learning strategy is briefly discussed in comparison with active cross-situational learning of object names performed by humans.

International audience; Recent psychophysical evidence in humans suggests that visual attention is a highly dynamic and predictive process involving precise models of object tra-jectories. We present a proof-of-concept that such predictive spatial attention can benefit a technical system solving a challenging visual object detection task. To this end, we introduce a Bayes-like integration of so-called dynamic attention priors (DAPs) and dense detection likelihoods, which get enhanced at predicted object positions obtained by the extrapolation of trajectories. Using annotated video sequences of pedestrians in a parking lot setting, we quantitatively show that DAPs can improve detection performance significantly as compared to a baseline condition relying purely on pattern analysis.

This paper presents a series of 4 single subject experiments aimed to investigate whether children with autism show more social engagement when interacting with the Nao robot, compared to a human partner in a motor imitation task. The Nao robot imitates gross arm movements of the child in real-time. Different behavioral criteria (i.e. eye gaze, gaze shifting, free initiations and prompted initiations of arm movements, and smile/laughter) were analyzed based on the video data of the interaction. The results are mixed and suggest a high variability in reactions to the Nao robot. The results are as follows: For Child2 and Child3, the results indicate no effect of the Nao robot in any of the target variables. Child1 and Child4 showed more eye gaze and smile/laughter in the interaction with the Nao robot compared to the human partner and Child1 showed a higher frequency of motor initiations in the interaction with the Nao robot compared to the baselines, but not with respect to the human-interaction. The robot proved to be a better facilitator of shared attention only for Child1. Keywords: human-robot interaction; assistive robotics; autism

We present a novel method to perform multi-class pattern classification with neural networks and test it on a challenging 3D hand gesture recognition problem. Our method consists of a standard one-against-all (OAA) classification, followed by another network layer classifying the resulting class scores, possibly augmented by the original raw input vector. This allows the network to disambiguate hard-to-separate classes as the distribution of class scores carries considerable information as well, and is in fact often used for assessing the confidence of a decision. We show that by this approach we are able to significantly boost our results, overall as well as for particular difficult cases, on the hard 10-class gesture classification task., European Symposium on artificial neural networks (ESANN), Jun 2015, Bruges, Belgium

International audience; We present a trainable hierarchical architecture capable of detecting newness (or outliers) at all hierarchical levels. This contribution paves the way for deep neural architectures that are able to learn in an incremental fashion, for which the ability to detect newness is an indispensable prerequisite. We verify the ability to detect newness by conducting experiments on the MNIST database, where we introduce either localized changes, by adding noise to a small patch of the input, or global changes, by changing the global arrangement of local patterns which is not detectable at the local level.

International audience; We present a deep prototype-based learning architecture which achieves a performance that is competitive to a conventional, shallow prototype-based model but at a fraction of the computational cost, especially w.r.t. memory requirements. As prototype-based classification and regression methods are typically plagued by the exploding number of prototypes necessary to solve complex problems, this is an important step towards efficient prototype-based classification and regression. We demonstrate these claims by benchmarking our deep prototype-based model on the well-known MNIST dataset.

Many state of the art object classification applications require many data samples, whose collection is usually a very costly process. Performing initial model training with synthetic samples (from virtual reality tools) has been proposed as a possible solution, although the resulting classification models need to be adapted (fine-tuned) to real-world data afterwards. For this bootstrapping process, we propose to use an incremental learning algorithm from the cognitive robotics domain which is particularly suited for perceptual problems. We apply it to a pedestrian detection problem where a synthetic dataset is used for initial training, and two different real-world datasets for fine-tuning and evaluation. The proposed scheme greatly reduces the number of real-world samples required while maintaining high classification accuracy. We additionally demonstrate several innovative incremental learning schemes for object detection, the basic idea being that usually only very few background samples are actually similar to pedestrian samples. By suitable arrangement of incremental learning steps, we can keep classification models simple by representing only such “hard” background samples.

In this document, I first analyze some of the reasons why real-world environment perception is stillstrongly inferior to human perception in overall accuracy and reliability. In particular, I focus on thetask of object detection in traffic scenes and present an argument why this task is in fact a good modeltask for other, related perception problems (e.g. in robotics or surveillance). Enumerating the difficultiesencountered in this model task (and therefore, by inference, in many other detection tasks as well), I cometo the conclusion that problems in object detection can in fact be, to a significant extent, traced backto problems of the learning algorithms that are used in various forms when performing object detection.Namely, the lack of a probabilistic interpretation, the lack of incremental learning capacity, the lack oftraining samples and the inherent ambiguity of local pattern analysis are identified and used to justify aroad map for research efforts aimed at overcoming these problems. I present several of my works concerningreal-world applications of machine learning in perception, where the stated problems become very apparent.Subsequently, I describe in detail my recent research contributions and their significance in the context ofthe proposed road map: context-based object detection, generative and multi-modal learning as well as anoriginal method for incremental learning. The document is concluded by an outlook that addresses furtherwork to complete the road map, and the possibilities that are offered by such an endeavour in the field of machine perception.

International audience

International audience; We present a biologically inspired architecture for incremental learning that remains resource-efficient even in the face of very high data dimensionalities (>1000) that are typically associated with perceptual problems. In particular, we investigate how a new perceptual (object) class can be added to a trained architecture without retraining, while avoiding the well-known catastrophic forgetting effects typically associated with such scenarios. At the heart of the presented architecture lies a generative description of the perceptual space by a self-organized approach which at the same time approximates the neighbourhood relations in this space on a two-dimensional plane. This approximation , which closely imitates the topographic organization of the visual cortex, allows an efficient local update rule for incremental learning even in the face of very high dimensionalities, which we demonstrate by tests on the well-known MNIST benchmark. We complement the model by adding a biologically plausible short-term memory system, allowing it to retain excellent classification accuracy even under incremental learning in progress. The short-term memory is additionally used to reinforce new data statistics by replaying previously stored samples during dedicated " sleep " phases.

International audience; Given the success of convolutional neural networks (CNNs) during recent years in numerous object recognition tasks, it seems logical to further extend their applicability to the treatment of three-dimensional data such as point clouds provided by depth sensors. To this end, we present an approach exploiting the CNN's ability of automated feature generation and combine it with a novel 3D feature computation technique , preserving local information contained in the data. Experiments are conducted on a large data set of 600.000 samples of hand postures obtained via ToF (time-of-flight) sensors from 20 different persons, after an extensive parameter search in order to optimize network structure. Generalization performance, measured by a leave-one-person-out scheme, exceeds that of any other method presented for this specific task, bringing the error for some persons down to 1.5%.

International audience; In the context of visual object search and localization, saliency maps provide an efficient way to find object candidates in images. Unlike most approaches, we propose a way to learn saliency maps directly on a robot, by exploring the environment, discovering salient objects using geometric cues, and learning their visual aspects. More importantly, we provide an autonomous exploration strategy able to drive the robot for the task of learning saliency. For that, we describe the Reinforcement Learning-Intelligent Adaptive Curiosity algorithm (RL-IAC), a mechanism based on IAC (Intelligent Adaptive Curiosity) able to guide the robot through areas of the space where learning progress is high, while minimizing the time spent to move in its environment without learning. We demonstrate first that our saliency approach is an efficient tool to generate relevant object boxes proposal in the input image and significantly outperforms state-of-the-art algorithms. Second, we show that RL-IAC can drastically decrease the required time for learning saliency compared to random exploration.

Searching for objects in an indoor environment can be drastically improved if a task-specific visual saliency is available. We describe a method to incrementally learn such an object-based visual saliency directly on a robot, using an environment exploration mechanism. We first define saliency based on a geometrical criterion and use this definition to segment salient elements given an attentive but costly and restrictive observation of the environment. These elements are used to train a fast classifier that predicts salient objects given large-scale visual features. In order to get a better and faster learning, we use an exploration strategy based on intrinsic motivation to drive our displacement in order to get relevant observations. Our approach has been tested on a robot in indoor environments as well as on publicly available RGB-D images sequences. We demonstrate that the approach outperforms several state-of-the-art methods in the case of indoor object detection and that the exploration strategy can drastically decrease the time required for learning saliency.

International audience; In Human-Robot Interaction (HRI) scenarios , an intelligent robot should be able to synthesize an appropriate behavior adapted to human profile (i.e., personality). Recent research studies discussed the effect of personality traits on human verbal and nonverbal behaviors. The dynamic characteristics of the generated gestures and postures during the nonverbal communication can differ according to personality traits, which similarly can influence the verbal content of human speech. This research tries to map human verbal behavior to a corresponding verbal and nonverbal combined robot behavior based on the extraversion-introversion personality dimension. We explore the human-robot personality matching aspect and the similarity attraction principle, in addition to the different effects of the adapted combined robot behavior expressed through speech and gestures, and the adapted speech-only robot behavior, on interaction. Experiments with the humanoid NAO robot are reported.

International audience; Modern vehicles embed an increasing number of Advanced Driving Assistance Systems (ADAS). Whilst such systems showed their capability to improve comfort and safety, most of them provide assistance only as a last resort, that is, they alert the driver or trigger automatic braking only when collision is imminent. This limitation is mainly due to the difficulty to accurately anticipate risk situations in order to provide the driver with preventive assistance, i.e. assistance allowing for comfortable reaction. This paper presents a Bayesian framework which aims to detect risk situations sufficiently early to trigger conventional curative assistance as well as preventive assistance. By taking into consideration the context, the vehicle state, the driver actuation and the manner how the driver usually negotiates given situations, the framework allows to infer which type of assistance is the most pertinent to be provided to the driver. The principles of this framework are applied to a fundamental case study, the arrival to a stop intersection. Results obtained from data recorded under controlled conditions are presented. They show that the framework allows to coherently detect risk situations and to identify what assistance, including preventive assistance, is the most appropriate for the situation.

International audience; We present a system for performing multi-sensor fusion that learns from experience, i.e., from training data and propose that learning methods are the most appropriate approaches to real-world fusion problems, since they are largely model-free and therefore suited for a variety of tasks, even where the underlying processes are not known with sufficient precision, or are too complex to treat analytically. In order to back our claim, we apply the system to simulated fusion tasks which are representative of real-world problems and which exhibit a variety of underlying probabilistic models and noise distributions. To perform a fair comparison, we study two additional ways of performing optimal fusion for these problems: empirical estimation of joint probability distributions and direct analytical calculation using Bayesian inference. We demonstrate that near-optimal fusion can indeed be learned, and that learning is by far the most generic and resource-efficient alternative. In addition, we show that the genera-tive learning approach we use is capable of improving its performance far beyond the Bayesian optimum by detecting and rejecting outliers, and that it is capable to detect systematic changes in the input statistics.

Gated networks are networks that contain gating connections, in which the outputs of at least two neurons are multiplied. Initially, gated networks were used to learn relationships between two input sources, such as pixels from two images. More recently, they have been applied to learning activity recognition or multi-modal representations. The aims of this paper are threefold: 1) to explain the basic computations in gated networks to the non-expert, while adopting a standpoint that insists on their symmetric nature. 2) to serve as a quick reference guide to the recent literature, by providing an inventory of applications of these networks, as well as recent extensions to the basic architecture. 3) to suggest future research directions and applications., Unpublished manuscript, 17 pages

International audience; Sign languages (SLs) are visuo-gestural representations used by deaf communities. Recognition of SLs usually requires manual annotations, which are expert dependent, prone to errors and time consuming. This work introduces a method to support SL annotations based on a motion descriptor that characterizes dynamic gestures in videos. The proposed approach starts by computing local kinematic cues, represented as mixtures of Gaussians which together correspond to gestures with a semantic equivalence in the sign language corpora. At each frame, a spatial pyramid partition allows a fine-to-coarse sub-regional description of motion-cues distribution. Then for each sub-region, a histogram of motion-cues occurrence is built, forming a frame-gesture descriptor which can be used for on-line annotation. The proposed approach is evaluated using a bag-of-features framework, in which every frame-level histogram is mapped to an SVM. Experimental results show competitive results in terms of accuracy and time computation for a signing dataset.

Robots are slowly entering our houses. But for them to perform more difficult and interesting tasks, they need to have a richer knowledge of the different objects and places present in a particular household. Building and using such a complex knowledge is called semantic mapping and navigation, which requires many different capabilities such as mapping, localization and object recognition. Most of them have been thoroughly studied in the past but often separately and in a different context. We propose a complete solution to perform incremental and unsupervised semantic mapping working on a real robot. Each aspect of the solution is discussed from the software architecture to all the functionalities needed by the robot. Some already existing techniques have been derived in the effort of integration, but more importantly three different parts of the solution are original contributions. The first one is the use of 2D laser range finder to perform object recognition using multiple views of the objects. Numerous semantic mapping solutions are using this sensor for mapping and obstacle avoidance, but we fully leveraged it by developing techniques to use it in the entire process of semantic mapping, showing that it is effective at recognizing up to 20 different objects in an indoor scenario. Secondly we introduce the graph-of-views object modeling method. In our effort to extend bag-of-views techniques to perform multi-modal modeling of objects we derived an original formulation which is more adapted to partial perception of objects. We applied this approach to object recognition using both a laser range finder and a RGB-D camera, showing that it is able to take advantage of both sensor strengths. Finally, while most existing semantic mapping techniques rely on supervised learning of objects, we present an original incremental and unsupervised learning algorithm. Our technique was applied to the learning of multi-modal models of dynamic objects that are encountered as the robot navigate an indoor environment for an extended period of time, showing that it is able to produce consistent models of these objects without human intervention., Les robots entrent peu à peu dans nos maisons. Mais pour réaliser des tâches plus difficiles et intéressantes, ils doivent posséder une connaissance riche des différents objets et lieux présent dans une habitation particulière. Construire et utiliser une telle connaissance s'appelle la cartographie et navigation sémantique et nécessite de nombreuses capacités comme la cartographie, la localisation ou encore la reconnaissance d'objets. La plupart de celles-ci ont été longuement étudiées dans le passé mais souvent séparément et dans un contexte différent. Nous proposons une solution complète fonctionnant sur un robot capable d'effectuer progressivement et sans supervision la cartographie sémantique d'un lieu. Chaque aspect de cette solution est examiné depuis l'architecture logicielle jusqu'aux fonctionnalités requises par le robot. Certaines techniques déjà existantes ont été adaptées dans l'effort d'intégration et trois parties différentes de la solution sont des contributions originales. La première est l'utilisation de la télémétrie laser 2D pour effectuer la reconnaissance d'objets en utilisant des vues multiples des objets. De nombreuses solutions de cartographie sémantiques utilisent ce capteur pour la cartographie et l'évitement d'obstacle, mais nous en avons pleinement tiré profit en développant des techniques pour l'utiliser dans l'ensemble du processus de cartographie sémantique, montrant qu'il est efficace pour reconnaître jusqu'à 20 objets différents dans un scénario d'environnement intérieur. Deuxièmement, nous présentons une méthode de modélisation d'objets par graphes de vues. Dans notre effort pour étendre les techniques de sac-de-vues pour effectuer la modélisation multi-modale des objets, nous avons développé une formulation originale qui est plus adapté à la perception partielle des objets. Nous avons appliqué cette approche à la reconnaissance d'objets en utilisant à la fois un télémètre laser et une caméra RGB-D, montrant qu'elle est en mesure de tirer avantage des points forts des deux capteurs. Enfin, alors que la plupart des techniques sémantiques de cartographie existantes reposent sur l'apprentissage supervisé des objets, nous présentons un algorithme d'apprentissage incrémental et non supervisé original. Notre technique a été appliquée à l'apprentissage de modèles multi-modaux des objets dynamiques présents dans un environnement intérieur pour une période de temps prolongée, montrant qu'elle est capable de produire des modèles cohérents de ces objets sans intervention humaine.

Robots are slowly entering our houses. But for them to perform more difficult and interesting tasks, they need to have a richer knowledge of the different objects and places present in a particular household. Building and using such a complex knowledge is called semantic mapping and navigation, which requires many different capabilities such as mapping, localization and object recognition. Most of them have been thoroughly studied in the past but often separately and in a different context. We propose a complete solution to perform incremental and unsupervised semantic mapping working on a real robot. Each aspect of the solution is discussed from the software architecture to all the functionalities needed by the robot. Some already existing techniques have been derived in the effort of integration, but more importantly three different parts of the solution are original contributions. The first one is the use of 2D laser range finder to perform object recognition using multiple views of the objects. Numerous semantic mapping solutions are using this sensor for mapping and obstacle avoidance, but we fully leveraged it by developing techniques to use it in the entire process of semantic mapping, showing that it is effective at recognizing up to 20 different objects in an indoor scenario. Secondly we introduce the graph-of-views object modeling method. In our effort to extend bag-of-views techniques to perform multi-modal modeling of objects we derived an original formulation which is more adapted to partial perception of objects. We applied this approach to object recognition using both a laser range finder and a RGB-D camera, showing that it is able to take advantage of both sensor strengths. Finally, while most existing semantic mapping techniques rely on supervised learning of objects, we present an original incremental and unsupervised learning algorithm. Our technique was applied to the learning of multi-modal models of dynamic objects that are encountered as the robot navigate an indoor environment for an extended period of time, showing that it is able to produce consistent models of these objects without human intervention.; Les robots entrent peu à peu dans nos maisons. Mais pour réaliser des tâches plus difficiles et intéressantes, ils doivent posséder une connaissance riche des différents objets et lieux présent dans une habitation particulière. Construire et utiliser une telle connaissance s'appelle la cartographie et navigation sémantique et nécessite de nombreuses capacités comme la cartographie, la localisation ou encore la reconnaissance d'objets. La plupart de celles-ci ont été longuement étudiées dans le passé mais souvent séparément et dans un contexte différent. Nous proposons une solution complète fonctionnant sur un robot capable d'effectuer progressivement et sans supervision la cartographie sémantique d'un lieu. Chaque aspect de cette solution est examiné depuis l'architecture logicielle jusqu'aux fonctionnalités requises par le robot. Certaines techniques déjà existantes ont été adaptées dans l'effort d'intégration et trois parties différentes de la solution sont des contributions originales. La première est l'utilisation de la télémétrie laser 2D pour effectuer la reconnaissance d'objets en utilisant des vues multiples des objets. De nombreuses solutions de cartographie sémantiques utilisent ce capteur pour la cartographie et l'évitement d'obstacle, mais nous en avons pleinement tiré profit en développant des techniques pour l'utiliser dans l'ensemble du processus de cartographie sémantique, montrant qu'il est efficace pour reconnaître jusqu'à 20 objets différents dans un scénario d'environnement intérieur. Deuxièmement, nous présentons une méthode de modélisation d'objets par graphes de vues. Dans notre effort pour étendre les techniques de sac-de-vues pour effectuer la modélisation multi-modale des objets, nous avons développé une formulation originale qui est plus adapté à la perception partielle des objets. Nous avons appliqué cette approche à la reconnaissance d'objets en utilisant à la fois un télémètre laser et une caméra RGB-D, montrant qu'elle est en mesure de tirer avantage des points forts des deux capteurs. Enfin, alors que la plupart des techniques sémantiques de cartographie existantes reposent sur l'apprentissage supervisé des objets, nous présentons un algorithme d'apprentissage incrémental et non supervisé original. Notre technique a été appliquée à l'apprentissage de modèles multi-modaux des objets dynamiques présents dans un environnement intérieur pour une période de temps prolongée, montrant qu'elle est capable de produire des modèles cohérents de ces objets sans intervention humaine.

Mobile robots have already entered people's homes to perform simple tasks for them. For robots at home to become real assistants, they have to be able to recognize the objects in their owner's home. In this thesis, object instance recognition algorithms are designed to cope with the variations (viewing angle, lighting conditions, etc.) that occur in the context of mobile robotics experiments. Several ways to take advantage of this context are studied. First, a geometric segmentation algorithm that benefits from the structure of indoor scenes to find isolated objects is designed. Then, a neural network based object recognition process that fuses shape, color and texture information provided by color and depth cameras is presented. This step highlights the importance of carefully combining multiple features and the difficulty to use color information in robotics. Finally, an alternative approach using a nearest neighbor classifier, which is easier to train than the neural network, is detailed. It relies on physical measures available to the robot to eliminate some parameters in clustering and nearest neighbor search procedures. Also, it uses information gathered through multiple sightings of the objects to reduce the negative impact of occlusions. The algorithm can recognize 52 objects with a success rate of 80\% and runs in 500 ms on average (on an Intel Core i5 CPU with 3~GB of RAM) on a mobile robot. This work shows that identifying and taking advantage of the structure and information available is essential to handle the variations that happen in indoor mobile robot experiments.; Des robots mobiles ont déjà fait leur apparition dans nos domiciles pour accomplir des tâches simples. Pour qu'un robot devienne un véritable assistant, il doit reconnaître les objects qui appartiennent à son propriétaire. Dans cette thèse, des algorithmes de reconnaissance d'instances d'objets sont développés pour faire face aux variations (angles de vue, illumination, etc.) qui surviennent dans un contexte de robotique mobile d'intérieur. Différentes manières de tirer profit de ce contexte sont étudiées. Premièrement, un algorithme de segmentation géometrique est proposé pour profiter de la structure inhérente aux scènes d'intérieur et y détecter des objects isolés. Un réseau de neurones pour la reconnaissance d'objets qui fusionne des informations de forme, couleur et texture provenant d'images de couleur et de profondeur est ensuite présenté. Cet algorithme met en évidence l'importance de combiner plusieurs sources d'information et la difficulté à utiliser la couleur en robotique. Enfin, une solution alternative à base de recherche de plus proche voisin, plus facile à entraîner que le réseau de neurones, est détaillée. Cette approche utilise des mesures comme la distance à l'objet pour éliminer certains paramètres dans la recherche de plus proche voisin et le partitionnement de données. L'information provenant de multiples vues d'un même objet sont aussi fusionnés. L'algorithme résultant fonctionne sur un robot mobile et peut reconnaître 52 objects en 500 ms en moyenne (sur un Intel Core i5 avec 3 GB de RAM) avec un taux de succès de 80\%. Les expériences menées dans cette thèse montrent qu'il est essentiel d'identifier et d'utiliser la structure et l'information présentes dans le contexte de la robotique d'intérieur pour faire face aux grandes variations qu'il peut y survenir.

International audience

International audience; In this paper we introduce MCA-NMF, a computational model of the acquisition of multi-modal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available.

International audience; In human-human interaction, three modalities of communication (i.e., verbal, nonverbal, and paraverbal) are naturally coordinated so as to enhance the meaning of the conveyed message. In this paper, we try to create a similar coordination between these modalities of communication in order to make the robot behave as naturally as possible. The proposed system uses a group of videos in order to elicit specific target emotions in a human user, upon which interactive narratives will start (i.e., interactive discussions between the participant and the robot around each video's content). During each interaction experiment, the humanoid expressive ALICE robot engages and generates an adapted multimodal behavior to the emotional content of the projected video using speech, head-arm metaphoric gestures, and/or facial expressions. The interactive speech of the robot is synthesized using Mary-TTS (text to speech toolkit), which is used-in parallel-to generate adapted head-arm gestures [1]. This synthesized multimodal robot behavior is evaluated by the interacting human at the end of each emotion-eliciting experiment. The obtained results validate the positive effect of the generated robot behavior multimodality on interaction.