Search

Human emotions present a major challenge for artificial intelligence. Automated emotion recognition based on facial expressions is important to robotics, medicine, psychology, education, security, arts, entertainment and more. Deep learning is promising for capturing complex emotional features. However, there is no training dataset that is large and representative of the full diversity of emotional expressions in all populations and contexts. Current facial datasets are incomplete, biased, unbalanced, error-prone and have different properties. Models learn these limitations and become dependent on specific datasets, hindering their ability to generalize to new data or real-world scenarios. Our work addresses these difficulties and provides the following contributions to improve emotion recognition: 1) a methodology for merging disparate in-the-wild datasets that increases the number of images and enriches the diversity of people, gestures, and attributes of resolution, color, background, lighting and image format; 2) a balanced, unbiased, and well-labeled evaluator dataset, built with a gender, age, and ethnicity predictor and the successful Stable Diffusion model. Single- and cross-dataset experimentation show that our method increases the generalization of the FER2013, NHFI and AffectNet datasets by 13.93%, 24.17% and 7.45%, respectively; and 3) we propose the first and largest artificial emotion dataset, which can complement real datasets in tasks related to facial expression. This work has been funded by grant CIPROM/2021/017 awarded by the MEEBAI Project (Prometheus Programme for Research Groups on R&D Excellence) from Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital of Generalitat Valenciana (Spain), and partially by the grant awarded by the Central University of Ecuador through budget certification No. 34 of March 25, 2022 for the development of the research project with code DOCT-DI-2020-37.

To test the suitability of an automatic system for emotional management in the classroom following the control-value theory of achievement emotions (CVT) framework, the performance of an emotional expression recognition software of our creation is evaluated in an online synchronous context. Sixty students from the Faculty of Education at the University of Alicante participated in 16 educational activities recording close-ups of their faces and completing the AEQ emotional self-report, as well as detailed reports from the subsequent review of their videos. In addition, they completed the VCQ-36 test to measure their volitional competencies and relate their influence on their emotional response. The results indicate a high coherence between the emotional expressions detected by the automatic system and the detailed emotional self-reports, but insufficient precision to meet the CVT requirements. On the other hand, both the AEQ test results and the emotion expression recognition software suggest students' preference for participative activities as opposed to passive ones. Meanwhile, statistical analysis results indicate that volitional competencies seem to influence the emotional response of students in the educational context, although the AI system does not show sufficient sensitivity in this field. Implications and limitations of this study for future work are discussed. This work has been supported by a grant from the Redes de Investigación Docente Universitaria Program of the Instituto de Ciencias de la Educación de la Universidad de Alicante (call 2021-22). Ref: 5484. This work has been partially funded by the grant CIPROM/2021/17 funded by the Prometeo programme from Conselleria d'Innovació, Universitats, Ciència i Societat Digital of Generalitat Valenciana (Spain).

Introduction: Social robotics applied to the educational context deals with proposals that start from the present to shape the future of what training in a specific subject can be. On this occasion, the aim is to investigate the connection between the utilization of social robots and the teaching–learning processes that take place within a formal music education environment at any stage of education. Methods: To carry out this research, the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) model served as a reference to perform a systematic review of articles published in two of the most important scientific databases, Web of Science (WoS) and Scopus, since 2015. Results: A total of four articles fulfilled the inclusion criteria. Discussion: Our findings attest that social robotics still remains a practically unknown topic. Hardly any experiences have been developed in classrooms. In general, it is considered necessary for education, and more specifically for music, not to be left out of the developments in social robotics. This technology, which is increasingly present in various areas of our society, responds to the objective of defining the 21st century, and musical education is part of it. This study has been funded by the “Methodology for Emotion-Aware Education Based on Artificial Intelligence” (Programa PROMETEO 2022—CIPROM/2021/017, Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital de la Generalitat Valenciana, Spain) and the “Programa Redes 2022–23, ICE de la Universidad de Alicante (Ref. 5780).”

Human Muscular Manipulability is a metric that measures the comfort of an specific pose and it can be used for a variety of applications related to healthcare. For this reason, we introduce KIMHu: a Kinematic, Imaging and electroMyography dataset for Human muscular manipulability index prediction. The dataset is comprised of images, depth maps, skeleton tracking data, electromyography recordings and 3 different Human Muscular Manipulability indexes of 20 participants performing different physical exercises with their arm. The methodology followed to acquire and process the data is also presented for future replication. A specific analysis framework for Human Muscular Manipulability is proposed in order to provide benchmarking tools based on this dataset. Óscar G. Hernández and José M. López Castellanos expresses their gratitude to the Fundación Carolina, the National Autonomous University of Honduras and the University of Alicante, for its funding support while this work was in preparation. This work has also been supported by an University of Alicante Grant GRE-20 (2021/00710/001) and by the Spanish Government Grant PID2019-104818RB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”.

This paper presents an augmented reality application to assist with myoelectric prostheses control for people with limb amputations. For that, we use the low-cost Myo armband coupled with low-level signal processing methods specifically built to control filters’ levels and processing chain. In particular, we use deep learning techniques to process the signals and to accurately identify seven different hand gestures. From that, we have built an augmented reality projection of a hand based on AprilTag markers that displays the gesture identified by the deep learning techniques. With the aim to properly train the gesture recognition system, we have built our own dataset with nine subjects. This dataset was combined with one publicly available to work with the data of 24 subjects in total. Finally, three different deep learning architectures have been comparatively studied, achieving high accuracy values (being 95.56% the best one). This validates our hypothesis that it is possible to have an adaptive platform able to fast learn personalized hand/arm gestures while projecting a virtual hand in real-time. This can reduce the adaptation time to myoelectric prostheses and improve the acceptance levels. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been partly supported by Grant PID2019-104818RB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”; and by Generalitat Valenciana (CIGE/2021/136).

This research focused on the development of a cyber-physical robotic platform to assist speech-language pathologists who are related to articulation disorders in education environments. The first goal was the design and development of the system. The second goal was the qualitative initial validation of the platform with robotics and mobile device functionalities drawing on the participation of real patients and speech-language pathologists (SLP). The research method is based on qualitative data. The first phase was the application of engineering requirements and agile techniques to build the robotic system. The second phase was to execute test sessions of the robotic platform with speech-language pathologists supervision and analyzing the experience of real male and female patients collecting data by in-depth interviews and video recordings at Heredia Special Education Center in Costa Rica. The practical approach of the cyber-physical platform has preliminarily support. Testing with SLPs and 3 other older individuals suggests that it may become a useful tool to assist professionals in the treatment of some types of articulation disorders. The time savings and data collection possibilities should be included in future investigations of efficacy.

The most common approaches for classification rely on the inference of a specific class. However, every category could be naturally organized within a taxonomic tree, from the most general concept to the specific element, and that is how human knowledge works. This representation avoids the necessity of learning roughly the same features for a range of very similar categories, and it is easier to understand and work with and provides a classification for each abstraction level. In this paper, we carry out an exhaustive study of different methods to perform multilevel classification applied to the task of classifying wild animals and plant species. Different convolutional backbones, data setups, and ensembling techniques are explored to find the model which provides the best performance. As our experimentation remarks, in order to achieve the best performance on the datasets that are arranged in a tree-like structure, the classifier must feature an EfficientNetB5 backbone with an input size of 300 × 300 px, followed by a multilevel classifier. In addition, a Multiscale Crop data augmentation process must be carried out. Finally, the accuracy of this setup is a 62% top-1 accuracy and 88% top-5 accuracy. The architecture could benefit for an accuracy boost if it is involved in an ensemble of cascade classifiers, but the computational demand is unbearable for any real application.

Global demographic changes have resulted in a growing technological demand to meet the arisen social needs. In particular, the increasingly ageing population requires assistive technologies to stay at home for longer independently while receiving continuous healthcare. In this sense, a wide academic and industrial research is taking place, introducing these technologies in hospitals and rehabilitation centres. This paper aims at providing an overview of research projects for elderly care and assistance, focusing on cognitive and robot assistants due to their popularity in the area. More precisely, physical and/or cognitive rehabilitation are presented. This paper also discusses their limitations and the open challenges to be tackled in order to be successfully integrated in our society.

Cognitive architectures allow robots to perform their operations by drawing on a process that aims to simulate human reasoning. This paper presents an integrated semantic artificial memory system in cognitive architecture based on symbolic reasoning and a connective representation of the knowledge. This memory system attempts to simulate how humans learn to distinguish instances of particular objects within their class using a convolutional network to detect the relevant elements of an image. We use a vector with the extracted features to learn to discriminate an instance of another element from the same class. A novel feature of our approach is its autonomous learning process during the operation of the robot, integrating a deep learning embedding with a statistical classifier. The usefulness and robustness of this method are demonstrated by applying it to a social robot that learns to differentiate people. Finally, experiments are carried out to validate our approach, comparing the detection results with several alternative methods. This work has been funded by the Spanish Government TIN2016-76515-R grant for the COMBAHO project, supported with Feder funds. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243 and FPU16/00887.

The use of semantic representations to achieve place understanding has been widely studied using indoor information. This kind of data can then be used for navigation, localization, and place identification using mobile devices. Nevertheless, applying this approach to outdoor data involves certain non-trivial procedures, such as gathering the information. This problem can be solved by using map APIs which allow images to be taken from the dataset captured to add to the map of a city. In this paper, we seek to leverage such APIs that collect images of city streets to generate a semantic representation of the city, built using a clustering algorithm and semantic descriptors. The main contribution of this work is to provide a new approach to generate a map with semantic information for each area of the city. The proposed method can automatically assign a semantic label for the cluster on the map. This method can be useful in smart cities and autonomous driving approaches due to the categorization of the zones in a city. The results show the robustness of the proposed pipeline and the advantages of using Google Street View images, semantic descriptors, and machine learning algorithms to generate semantic maps of outdoor places. These maps properly encode the zones existing in the selected city and are able to provide new zones between current ones. This work has been supported by the Spanish Grant PID2019-104818RB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”. José Carlos Rangel and Edmanuel Cruz were supported by the Sistema Nacional de Investigación (SNI) of SENACYT, Panama.

Con el fin de desarrollar un sistema automático de gestión emocional en el aula durante el proceso de enseñanza-aprendizaje, se evalúa el rendimiento de un software de reconocimiento emocional creación propia en un contexto sincrónico no presencial. Sesenta alumnos del Grado en Educación Primaria impartido en la Facultad de Educación de la Universidad de Alicante participaron en 16 dinámicas educativas grabando primeros planos de sus rostros y completando autoinformes emocionales. Además, completaron el test VCQ-36 para medir sus competencias volitivas y relacionar su influencia en su respuesta emocional. Los resultados indican una alta coherencia entre las expresiones emocionales detectadas por el sistema automático y los autoinformes emocionales, aunque su implementación en un contexto presencial implica retos adicionales. Por otra parte, tanto los autoinformes emocionales como el software de reconocimiento de emociones sugieren una preferencia de los estudiantes por las dinámicas activas frente a las pasivas. Mientras tanto, según los resultados del coeficiente de correlación de Pearson, las competencias volitivas parecen influir en la respuesta emocional de los estudiantes en el contexto educativo, aunque el sistema de inteligencia artificial no muestra suficiente sensibilidad en este campo. Se discuten las implicaciones y limitaciones de este estudio para futuros trabajos.

En esta red se ha llevado a cabo una experiencia docente sobre la adaptación a la modalidad de clase invertida de una asignatura de un programa de máster. El máster en cuestión es fundamentalmente práctico y se ha impartido de forma presencial durante 6 cursos. Durante el curso académico 2020-21, tras ser aceptado por la ANECA, pasó a impartirse de forma semipresencial. Este hecho forzó la adaptación de todas sus asignaturas, para lo cual se siguieron distintas estrategias, la mayoría de ellas basadas en el aprovechamiento de los materiales existentes. En la asignatura estudiada en esta red se propusieron una serie de prácticas para la implantación de la metodología de clase invertida enfocadas en mantener la calidad y los resultados de la asignatura, remodelando completamente la planificación, los materiales existentes y creando nuevos recursos adaptados, como vídeo presentaciones o encuestas de autoevaluación, además de utilizar diferentes herramientas interactivas de Moodle. Según la experiencia llevada a cabo durante este año, si se crea una separación estricta entre contenidos teóricos y prácticos siguiendo la metodología de clase invertida, los alumnos tienden a dedicarle poco tiempo a las sesiones no presenciales, revisándolas justo antes de la sesión presencial o incluso durante la misma, buscando solo lo que les es necesario. Para evitar esto fue importante planificar bien las sesiones no presenciales, creando una guía de lo que tenían que hacer que incluya alguna actividad o cuestionario a completar. Además, para realizar un mejor seguimiento de estas sesiones y darle una mayor continuidad al curso, se planificó el máster para que intercalase las sesiones presenciales con las no presenciales y dejase el tiempo suficiente a los alumnos para su realización. Otra parte muy importante del proceso de adaptación fue la de incluir ejercicios de auto-evaluación y cuestionarios. Los ejercicios de auto-evaluación, que eran optativos y no contaban para la nota final, consistían en actividades breves para practicar con el contenido estudiado. Los cuestionarios sí que eran obligatorios y contaban un 10% de la nota final. Estos cuestionarios se tenían que completar antes de la correspondiente sesión práctica (después se cerraban automáticamente). Con esto se pretendía incentivar que los alumnos se esforzasen más en estudiar los contenidos teóricos antes de la sesión presencial. Por último, para evaluar esta experiencia se pasaron cuestionarios a los alumnos, comparando los resultados con los de otras asignaturas del máster y con los obtenidos en cursos anteriores. En todos los casos los alumnos puntuaron muy positivamente la adaptación realizada, así como los resultados obtenidos gracias a los materiales y a la metodología, destacando sobre todo el aprovechamiento del tiempo no presencial. En esta red docente se han analizado las conclusiones extraídas de este proceso y del feedback obtenido de los alumnos con el objetivo de aplicarlas tanto a la propia asignatura como al resto de asignaturas de este máster.

Convolutional Neural Networks (CNNs) have become the default paradigm for addressing classification problems, especially, but not only, in image recognition. This is mainly due to the high success rate they provide. Although there are currently approaches that apply deep learning to the 3D shape recognition problem, they are either too slow for online use or too error-prone. To fill this gap, we propose 3DSliceLeNet, a deep learning architecture for point cloud classification. Our proposal converts the input point clouds into a two-dimensional representation by performing a slicing process and projecting the points to the principal planes, thus generating images that are used by the convolutional architecture. 3DSliceLeNet successfully achieves both high accuracy and low computational cost. A dense set of experiments has been conducted to validate our system under the ModelNet challenge, a large-scale 3D Computer Aided Design (CAD) model dataset. Our proposal achieves a success rate of 94.37% and an Area Under Curve (AUC) of 0.978 on the ModelNet-10 classification task. Grant PID2019-104818RB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”.

Un problema común en la enseñanza de las Matemáticas a los estudiantes de ingeniería es el interés de estos. Esto se traduce en una elevada dificultad para aprobar la asignatura y en una alta tasa de abandono. Con el fin de superar estos problemas, se ha realizado un cambio en los métodos de enseñanza. Tras una revisión de las técnicas de aprendizaje activo, se ha usado la metodología flipped classroom para la parte práctica de la asignatura. Se proporcionaron a los alumnos varios tipos de materiales, como vídeos, apuntes de clase y ejercicios. Además, también se modificó el sistema de evaluación para adaptarlo a este método de aprendizaje. Así pues, se ha realizado un test al finalizar cada tema, acumulativos, con el objetivo de mantener los conocimientos actualizados y detectar posibles deficiencias en el proceso de aprendizaje y/o en los materiales. Posteriormente, al finalizar el semestre, se ha realizado un examen final. Esta metodología ha sido evaluada tanto por los alumnos como por los profesores. Además, se ha realizado un estudio comparativo con respecto a los resultados en cursos anteriores. Los resultados cuantitativos y cualitativos validan esta metodología.

La retroalimentación docente en el ámbito universitario es un tema poco abordado. Es difícil conseguir un equilibrio donde la evaluación se combine con una formación personalizada a través de la retroalimentación del alumnado a la hora de realizar tareas, actividades, trabajos y proyectos, entre otros. Por ello, la finalidad de esta Red ha sido abordar la retroalimentación, además en un contexto definido por una metodología activa donde se utiliza la tecnología digital. El proceso de incorporación de dicha retroalimentación a la misma metodología activa ha supuesto un enfoque innovador respecto al desarrollo anterior en las respectivas asignaturas. Los resultados han demostrado la importancia de este proceso evaluador en el aula y urge diseñar procesos de enseñanza-aprendizaje específicos que se desarrollen en el aula universitaria.

In recent years, the popularity of intelligent and autonomous vehicles has grown notably. In fact, there already exist commercial models with a high degree of autonomy as regards self-driving capabilities. A key feature for this kind of vehicle is object detection, which is commonly performed in 2D space. This has some inherent issues as an object and the depiction of such an object would be classified as the actual object, which is inadequate since urban environments are full of billboards, printed adverts and posters that would likely make these systems fail. In order to overcome this problem, a 3D sensor could be leveraged, although this would make the platform more expensive, energy inefficient and computationally complex. Thus, we propose the use of structure from motion to reconstruct the three-dimensional information of the scene from a set of images, and merge the 2D and 3D data to differentiate actual objects from depictions. As expected, our approach is able to work with a regular color camera. No 3D sensors whatsoever are required. As the experiments confirm, our approach is able to distinguish between actual pedestrians and depictions of them more than 87% of times in synthetic and real-world tests in the worst scenarios, while the accuracy is of almost 98% in the best case. This work was funded by a Spanish Government PID2019-104818RB-I00 grant, supported by Feder funds. It was also supported by Spanish grants for Ph.D. FPU16/00887. Experiments were made possible by a generous hardware donation from NVIDIA.

Environment landmarks are generally employed by visual SLAM (vSLAM) methods in the form of keypoints. However, these landmarks are unstable over time because they belong to areas that tend to change, e.g., shadows or moving objects. To solve this, some other authors have proposed the combination of keypoints and artificial markers distributed in the environment so as to facilitate the tracking process in the long run. Artificial markers are special elements (similar to beacons) that can be permanently placed in the environment to facilitate tracking. In any case, these systems keep a set of keypoints that is not likely to be reused, thus unnecessarily increasing the computing time required for tracking. This paper proposes a novel visual SLAM approach that efficiently combines keypoints and artificial markers, allowing for a substantial reduction in the computing time and memory required without noticeably degrading the tracking accuracy. In the first stage, our system creates a map of the environment using both keypoints and artificial markers, but once the map is created, the keypoints are removed and only the markers are kept. Thus, our map stores only long-lasting features of the environment (i.e., the markers). Then, for localization purposes, our algorithm uses the marker information along with temporary keypoints created just in the time of tracking, which are removed after a while. Since our algorithm keeps only a small subset of recent keypoints, it is faster than the state-of-the-art vSLAM approaches. The experimental results show that our proposed sSLAM compares favorably with ORB-SLAM2, ORB-SLAM3, OpenVSLAM and UcoSLAM in terms of speed, without statistically significant differences in accuracy. This research was funded by the project PID2019-103871GB-I00 of the Spanish Ministry of Economy, Industry and Competitiveness, FEDER, Project 1380047-F UCOFEDER-2021 of Andalusia and by the European Union–NextGeneration EU for requalification of Spanish University System 2021–2023.

This research focuses on whether the use of a cyber-physical robotic system (CPRS) to assist Speech and Language Pathologists (SLP) in a Special Education service is beneficial. The research method is based on a quasi-experiment with a 2k design and a two-way ANOVA, implemented with real high school students over 10 weeks. It was found that the use of this CPRT could improve, preliminarily and as an initial exploratory finding, therapeutic speech effectiveness up to 11.3 percentage points with a statistical confidence of 95%, when SLPs work with students with mild articulation disorder and a restricted time for therapy, but especially when the technology is used without time constraints. It is concluded that assistive CPRT could be a causal factor of improvement in specific treatments performed by SLPs, with the statistical evidence being sufficiently significant (95%) to maintain scientific and educational interest in this research line in the future. Esta investigación se centra en comprobar si el uso de un sistema robótico ciberfísico (CPRS por sus siglas en inglés) para ayudar a los patólogos del habla y el lenguaje (SLP por sus siglas en inglés) resulta beneficioso en un servicio de educación especial. El método investigador se basa en un cuasiexperimento con un diseño 2k y un test ANOVA en dos direcciones, aplicado con estudiantes de instituto reales a lo largo de 10 semanas. Se descubrió que la utilización de este CPRS podía mejorar, de forma preliminar y como hallazgo exploratorio inicial, la eficacia terapéutica sobre el habla hasta 11,3 puntos porcentuales con una confianza estadística del 95% cuando los SLP trabajan con estudiantes que padecen un trastorno articulatorio leve y con un período de terapia restringido, pero especialmente cuando se emplea la tecnología sin limitaciones de tiempo. Se llega a la conclusión de que los CPRS de apoyo podrían constituir un factor causante de una mejora en tratamientos concretos llevados a cabo por los SLP, con unas evidencias estadísticas lo bastante significativas (95%) como para mantener el interés científico y educativo por esta línea de investigación en el futuro.

Currently, state-of-the-art methods for 3D object recognition rely in a deep learning-pipeline. Nonetheless, these methods require a large amount of data that is not easy to obtain. In addition to that, the majority of them exploit features of the datasets, like the fact of being CAD models to create rendered representation which will not work in real life because the 3D sensors provide point clouds. We propose a novel global descriptor for point clouds which takes advantage of the fractal dimension of the objects. Our approach introduces many benefits, such as being agnostic to the density of points of the sample, number of points in the input cloud, sensor of choice, and noise up to a level, and it works on real life point cloud data provided by commercial sensors. We tested our descriptor for 3D object recognition using ModelNet, which is a well-known dataset for that task. Our approach achieves 92.84% accuracy on the ModelNet10, and 88.74% accuracy on the ModelNet40. This work was supported in part by the Spanish Government, with Feder funds, under Grant PID2019-104818RB-I00, and in part by the Spanish Grants for Ph.D. studies under Grant ACIF/2017/243 and Grant FPU16/00887.

In this paper, we present a novel deep learning-based architecture, which is under the scope of expert and intelligent systems, to perform accurate real-time tridimensional hand pose estimation using a single RGB frame as an input, so there is no need to use multiple cameras or points of view, or RGB-D devices. The proposed pipeline is composed of two convolutional neural network architectures. The first one is in charge of detecting the hand in the image. The second one is able to accurately infer the tridimensional position of the joints retrieving, thus, the full hand pose. To do this, we captured our own large-scale dataset composed of images of hands and the corresponding 3D joints annotations. The proposal achieved a 3D hand pose mean error of below 5 mm on both the proposed dataset and Stereo Hand Pose Tracking Benchmark, which is a public dataset. Our method also outperforms the state-of-the-art methods. We also demonstrate in this paper the application of the proposal to perform a robotic hand teleoperation with high success. This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243

Data-driven algorithms have surpassed traditional techniques in almost every aspect in robotic vision problems. Such algorithms need vast amounts of quality data to be able to work properly after their training process. Gathering and annotating that sheer amount of data in the real world is a time-consuming and error-prone task. Those problems limit scale and quality. Synthetic data generation has become increasingly popular since it is faster to generate and automatic to annotate. However, most of the current datasets and environments lack realism, interactions, and details from the real world. UnrealROX is an environment built over Unreal Engine 4 which aims to reduce that reality gap by leveraging hyperrealistic indoor scenes that are explored by robot agents which also interact with objects in a visually realistic manner in that simulated world. Photorealistic scenes and robots are rendered by Unreal Engine into a virtual reality headset which captures gaze so that a human operator can move the robot and use controllers for the robotic hands; scene information is dumped on a per-frame basis so that it can be reproduced offline to generate raw data and ground truth annotations. This virtual reality environment enables robotic vision researchers to generate realistic and visually plausible data with full ground truth for a wide variety of problems such as class and instance semantic segmentation, object detection, depth estimation, visual grasping, and navigation., Published in Virtual Reality journal

Physical traits such as the shape of the hand and face can be used for human recognition and identification in video surveillance systems and in biometric authentication smart card systems, as well as in personal health care. However, the accuracy of such systems suffers from illumination changes, unpredictability, and variability in appearance (e.g. occluded faces or hands, cluttered backgrounds, etc.). This work evaluates different statistical and chrominance models in different environments with increasingly cluttered backgrounds where changes in lighting are common and with no occlusions applied, in order to get a reliable neural network reconstruction of faces and hands, without taking into account the structural and temporal kinematics of the hands. First a statistical model is used for skin colour segmentation to roughly locate hands and faces. Then a neural network is used to reconstruct in 3D the hands and faces. For the filtering and the reconstruction we have used the growing neural gas algorithm which can preserve the topology of an object without restarting the learning process. Experiments conducted on our own database but also on four benchmark databases (Stirling’s, Alicante, Essex, and Stegmann’s) and on deaf individuals from normal 2D videos are freely available on the BSL signbank dataset. Results demonstrate the validity of our system to solve problems of face and hand segmentation and reconstruction under different environmental conditions. This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with FEDER funds, the University of Alicante Project GRE16-19, the Valencian Government Project GV-2018-022, and the UK Dunhill Medical Trust Grant RPGF1802\37.

Tradicionalmente, en la asignatura de Visión Artificial y Robótica se ha venido utilizando el software "Robot Operating System" (ROS) como herramienta para la realización de los ejercicios prácticos de la asignatura. Actualmente, ROS es la principal herramienta para la programación de robots en el ámbito profesional y académico. Sin embargo, el uso de ROS para las prácticas de la asignatura ha generado una serie de problemas, que detallaremos más adelante, principalmente relacionados con la docencia semi-presencial y la no presencial. Estos problemas se pusieron de manifiesto al impartir la asignatura en el curso 19/20 durante el cual se decretó el confinamiento de la población debido a la CoVid-19 y la docencia pasó a modalidad no-presencial.

En este trabajo se presenta la experiencia docente llevada a cabo en un programa de máster para la adaptación de una de sus asignaturas a la modalidad de clase invertida. El máster en cuestión es fundamentalmente práctico y se ha impartido de forma presencial durante los últimos 6 cursos. Durante el curso 2020-21, tras ser aceptado por la ANECA, ha pasado a impartirse de forma semipresencial. Este hecho ha forzado la adaptación de todas sus asignaturas, para lo cual se han seguido distintas estrategias, la mayoría de ellas basadas en el aprovechamiento de los materiales. A modo de proyecto piloto, en una asignatura se han propuesto una serie de prácticas para la implantación de la metodología de clase invertida. Dicha asignatura ha sido remodelada pasando de clases teórico-prácticas a separar estos contenidos por sesiones mediante la elaboración de nuevos materiales y recursos, como videotutoriales, encuestas de autoevaluación, etc. Para analizar esta experiencia se ha realizado un cuestionario que nos ha permitido conocer tanto las opiniones de los alumnos como la viabilidad de aplicar esta solución al resto de asignaturas. Los resultados obtenidos se han comparado con los de otras asignaturas del mismo máster y con los resultados de cursos anteriores.

Este proyecto tiene como objetivo estudiar la aceptación de la transición de las clases presenciales a la modalidad online por parte de los estudiantes. Para ello, se ha diseñado un estudio basado en encuestas que recogieron la opinión de varios aspectos relativos a las clases y se ha comparado los resultados académicos de los estudiantes con años anteriores. Este estudio se ha realizado en las asignaturas de Sistemas Inteligentes del Grado de Ingeniería Robótica y de Matemáticas 2 en los grados de Ingeniería Informática e Informática y Administración de Empresas. Las encuestas anónimas y voluntarias fueron respondidas por 137 estudiantes (108 de Matemáticas 2 y 29 de Sistemas Inteligentes), valorando positivamente (70,7%) la adaptación de las asignaturas a la modalidad en línea. Sin embargo, 21% y 43% de los encuestados han señalado un empeoramiento significativo con relación a la modalidad online, refiriendo que es muy difícil seguir las clases en el formato online. Por otra parte, se ha realizado un análisis en base a las calificaciones obtenidas en ambas asignaturas. El estudio pone de manifiesto que el paso a la docencia online no ha supuesto un empeoramiento (ni mejora) de las calificaciones obtenidas, pues éstas han sido equivalentes.

In this work, we introduce HaReS, a hand rehabilitation system. Our proposal integrates a series of exercises, jointly developed with a foundation for those with motor and cognitive injuries, that are aimed at improving the skills of patients and the adherence to the rehabilitation plan. Our system takes advantage of a low-cost hand-tracking device to provide a quantitative analysis of the performance of the patient. It also integrates a low-cost surface electromyography (sEMG) sensor in order to provide insight about which muscles are being activated while completing the exercises. It is also modular and can be deployed on a social robot. We tested our proposal in two different facilities for rehabilitation with high success. The therapists and patients felt more motivation while using HaReS, which improved the adherence to the rehabilitation plan. In addition, the therapists were able to provide services to more patients than when they used their traditional methodology. This work was funded by a Spanish Government PID2019-104818RB-I00 grant, supported by Feder funds. It was also supported by Spanish grants for PhD studies ACIF/2017/243 and FPU16/00887.

Along the last decades, Robotics research has taken a major turn from laboratories to factories and ordinary real-world environments. Consequently, new issues to be overcome have arisen, specially when autonomous, dexterous robots are in place. In this paper, we present this evolution in the case of robot vision for manipulation through several robot developments, by analysing their challenges and proposed solutions. This overview highlights the need of using different techniques depending on the task at hand and the scenario to work in. This work was supported in part by the Ministerio de Economía y Competitividad under Grant DPI2015-69041-R, in part by Universitat Jaume I under Grant UJI-B2018-74, and in part by Generalitat Valenciana under Grant PROMETEO/2020/034 and GV/2020/051.

Over time, the field of robotics has provided solutions to automate routine tasks in different scenarios. In particular, libraries are awakening great interest in automated tasks since they are semi-structured environments where machines coexist with humans and several repetitive operations could be automatically performed. In addition, multirotor aerial vehicles have become very popular in many applications over the past decade, however autonomous flight in confined spaces still presents a number of challenges and the use of small drones has not been reported as an automated inventory device within libraries. This paper presents the UJI aerial librarian robot that leverages computer vision techniques to autonomously self-localize and navigate in a library for automated inventory and book localization. A control strategy to navigate along the library bookcases is presented by using visual markers for self-localization during a visual inspection of bookshelves. An image-based book recognition technique is described that combines computer vision techniques to detect the tags on the book spines, followed by an optical character recognizer (OCR) to convert the book code on the tags into text. These data can be used for library inventory. Misplaced books can be automatically detected, and a particular book can be located within the library. Our quadrotor robot was tested in a real library with promising results. The problems encountered and limitation of the system are discussed, along with its relation to similar applications, such as automated inventory in warehouses. Support for the research conducted at UJI Robotic Intelligence Laboratory is provided in part by the Ministerio de Economía y Competitividad (DPI2015-69041-R), by Universitat Jaume I (UJI-B2018-74), and by Generalitat Valenciana (PROMETEO/2020/034, GV/2020/051).

Over one billion people in the world live with some form of disability. This is incessantly increasing due to aging population and chronic diseases. Among the emerging social needs, rehabilitation services are the most required. However, they are scarce and expensive what considerably limits access to them. In this paper, we propose EVA, an augmented reality platform to engage and supervise rehabilitation sessions at home using low-cost sensors. It also stores the user’s statistics and allows therapists to tailor the exercise programs according to their performance. This system has been evaluated in both qualitative and quantitative ways obtaining very promising results. This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. Edmanuel Cruz is funded by a Panamenian grant for Ph.D. studies IFARHU and SENACYT 270-2016-207. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243 and FPU16/00887. Thanks also to Nvidia for the generous donation of a Titan Xp and a Quadro P6000.

In recent years the advances in Artificial Intelligence (AI) have been seen to play an important role in human well-being, in particular enabling novel forms of human-computer interaction for people with a disability. In this paper, we propose a sEMG-controlled 3D game that leverages a deep learning-based architecture for real-time gesture recognition. The 3D game experience developed in the study is focused on rehabilitation exercises, allowing individuals with certain disabilities to use low-cost sEMG sensors to control the game experience. For this purpose, we acquired a novel dataset of seven gestures using the Myo armband device, which we utilized to train the proposed deep learning model. The signals captured were used as an input of a Conv-GRU architecture to classify the gestures. Further, we ran a live system with the participation of different individuals and analyzed the neural network’s classification for hand gestures. Finally, we also evaluated our system, testing it for 20 rounds with new participants and analyzed its results in a user study. This work was supported by the Spanish Government PID2019-104818RB-I00 grant, supported with Feder funds.

In the last years, the care of dependent people, either by disease, accident, disability, or age, is one of the current priority research topics in developed countries. Moreover, such care is intended to be at patients home, in order to minimize the cost of therapies. Patients rehabilitation will be fulfilled when their integration in society is achieved, either in the family or in a work environment. To address this challenge, we propose the development and evaluation of an assistant for people with acquired brain injury or dependents. This assistant is twofold: in the patient’s home is based on the design and use of an intelligent environment with abilities to monitor and active learning, combined with an autonomous social robot for interactive assistance and stimulation. On the other hand, it is complemented with an outdoor assistant, to help patients under disorientation or complex situations. This involves the integration of several existing technologies and provides solutions to a variety of technological challenges. Deep leaning-based techniques are proposed as core technology to solve these problems. This work has been funded by the Spanish Government TIN2016-76515-R grant for the COMBAHO project, supported with Feder funds. It has also been supported by the University of Alicante project GRE16-19 and the Valencian Government GV/2018/022. This work has also been supported by Spanish grants for PhD studies FPU15/04516 and ACIF/2017/243. Experiments were made possible by a generous hardware donation from NVIDIA.

Deep learning-based methods have proven to be the best performers when it comes to object recognition cues both in images and tridimensional data. Nonetheless, when it comes to 3D object recognition, the authors tend to convert the 3D data to images and then perform their classification. However, despite its accuracy, this approach has some issues. In this work, we present a deep learning pipeline for object recognition that takes a point cloud as input and provides the classification probabilities as output. Our proposal is trained on synthetic CAD objects and is able to perform accurately when fed with real data provided by commercial sensors. Unlike most approaches, our method is specifically trained to work on partial views of the objects rather than on a full representation, which is not the representation of the objects as captured by commercial sensors. We trained our proposal with the ModelNet10 dataset and achieved a 78.39% accuracy. We also tested it by adding noise to the dataset and against a number of datasets and real data with high success. This work has been funded by the Spanish Government TIN2016-76515-R grant for the COMBAHO project, supported with Feder funds. It has also been supported by Spanish grants for PhD studies ACIF/2017/243 and FPU16/00887.

The ability to predict, anticipate and reason about future outcomes is a key component of intelligent decision-making systems. In light of the success of deep learning in computer vision, deep-learning-based video prediction emerged as a promising research direction. Defined as a self-supervised learning task, video prediction represents a suitable framework for representation learning, as it demonstrated potential capabilities for extracting meaningful representations of the underlying patterns in natural videos. Motivated by the increasing interest in this task, we provide a review on the deep learning methods for prediction in video sequences. We firstly define the video prediction fundamentals, as well as mandatory background concepts and the most used datasets. Next, we carefully analyze existing video prediction models organized according to a proposed taxonomy, highlighting their contributions and their significance in the field. The summary of the datasets and methods is accompanied with experimental results that facilitate the assessment of the state of the art on a quantitative basis. The paper is summarized by drawing some general conclusions, identifying open research challenges and by pointing out future research directions., Submitted to TPAMI

In the past years, several works on urban object detection from the point of view of a person have been made. These works are intended to provide an enhanced understanding of the environ- ment for blind and visually challenged people. The mentioned approaches mostly rely in deep learning and machine learning methods. Nonetheless, these approaches only work with direct and bright light, namely, they will only perform correctly on daylight conditions. This is because deep learning algorithms require large amounts of data and the currently available datasets do not address this matter. In this work, we propose UrOAC, a dataset of urban objects captured in a range of different lightning conditions, from bright daylight to low and poor night-time lighting conditions. In the latter, the objects are only lit by low ambient light, street lamps and headlights of passing-by vehicles. The dataset depicts the following objects: pedestrian crosswalks, green traffic lights and red traffic lights. The annotations include the category and the bounding-box of each object. This dataset could be used for improve the performance at night-time and under low-light conditions of any vision-based method that involves urban objects. For instance, guidance and object detection devices for the visually challenged or self-driving and intelligent vehicles. This work has been supported by the Spanish Government PID2019-104818RB-I00 Grant, co-funded by EU Structural Funds.

Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we formulate the semantic segmentation problem and define the terminology of this field as well as interesting background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review common loss functions and error metrics for this problem. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques. This work has been funded by the Spanish Government TIN2016-76515-R funding for the COMBAHO project, supported with Feder funds. It has also been supported by a Spanish national grant for PhD studies FPU15/04516 (Alberto Garcia-Garcia). In addition, it was also funded by the grant Ayudas para Estudios de Master e Iniciacion a la Investigacion from the University of Alicante.

Aimed at building autonomous service robots, reasoning, perception, and action should be properly integrated. In this paper, the depth cue has been analysed as an early stage given its importance for robotic tasks. So, from neuroscience findings, a hierarchical four-level dorsal architecture has been designed and implemented. Mainly, from a stereo image pair, a set of complex Gabor filters is applied for estimating an egocentric quantitative disparity map. This map leads to a quantitative depth scene representation that provides the raw input for a qualitative approach. So, the reasoning method infers the data required to make the right decision at any time. As it will be shown, the experimental results highlight the robust performance of the biologically inspired approach presented in this paper. This paper describes research done at UJI Robotic Intelligence Laboratory. Support for this laboratory was provided in part by Ministerio de Economía y Competitividad (DPI2015-69041-R) and by Universitat Jaume I.

Pedestrian movement direction recognition is an important factor in autonomous driver assistance and security surveillance systems. Pedestrians are the most crucial and fragile moving objects in streets, roads, and events, where thousands of people may gather on a regular basis. People flow analysis on zebra crossings and in shopping centers or events such as demonstrations are a key element to improve safety and to enable autonomous cars to drive in real life environments. This paper focuses on deep learning techniques such as convolutional neural networks (CNN) to achieve a reliable detection of pedestrians moving in a particular direction. We propose a CNN-based technique that leverages current pedestrian detection techniques (histograms of oriented gradients-linSVM) to generate a sum of subtracted frames (flow estimation around the detected pedestrian), which are used as an input for the proposed modified versions of various state-of-the-art CNN networks, such as AlexNet, GoogleNet, and ResNet. Moreover, we have also created a new data set for this purpose, and analyzed the importance of training in a known data set for the neural networks to achieve reliable results. This work was supported by the Feder funds, Spanish Government through the COMBAHO Project, under Grant TIN2016-76515-R, and in part by the University of Alicante Project under Grant GRE16-19.

In the creation of new industries, products and services -- all of which are advances of the Fourth Industrial Revolution -- the human-robot interaction that includes automatic learning and computer vision are elements to consider since they promote collaborative environments between people and robots. The use of machine learning and computer vision provides the tools needed to increase productivity and minimizes delivery reaction times by assisting in the optimization of complex production planning processes. This review of the state of the art presents the main trends that seek to improve human-robot interaction in productive environments, and identifies challenges in research as well as in industrial - technological development in this topic. In addition, this review offers a proposal on the needs of use of artificial intelligence in all processes of industry 4.0 as a crucial linking element among humans, robots, intelligent and traditional machines; as well as a mechanism for quality control and occupational safety. This work has been funded by the Spanish Government [TIN2016-76515-R] grant for the COMBAHO project, supported with Feder funds.

The capacity of a robot to automatically adapt to new environments is crucial, especially in social robotics. Often, when these robots are deployed in home or office environments, they tend to fail because they lack the ability to adapt to new and continuously changing scenarios. In order to accomplish this task, robots must obtain new information from the environment, and then add it to their already learned knowledge. Deep learning techniques are often used to tackle this problem successfully. However, these approaches, complete retraining of the models, which is highly time-consuming. In this work, several strategies are tested to find the best way to include new knowledge in an already learned model in a deep learning pipeline, putting the spotlight on the time spent for this training. We tackle the localization problem in the long term with a deep learning approach and testing several retraining strategies. The results of the experiments are discussed and, finally, the best approach is deployed on a Pepper robot. The capacity of a robot to automatically adapt to new environments is crucial, especially in social robotics. Often, when these robots are deployed in home or office environments, they tend to fail because they lack the ability to adapt to new and continuously changing scenarios. In order to accomplish this task, robots must obtain new information from the environment, and then add it to their already learned knowledge. Deep learning techniques are often used to tackle this problem successfully. However, these approaches, complete retraining of the models, which is highly time-consuming. In this work, several strategies are tested to find the best way to include new knowledge in an already learned model in a deep learning pipeline, putting the spotlight on the time spent for this training. We tackle the localization problem in the long term with a deep learning approach and testing several retraining strategies. The results of the experiments are discussed and, finally, the best approach is deployed on a Pepper robot.

Pulmonary Embolism (PE) is a respiratory disease caused by blood clots lodged in the pulmonary arteries, blocking perfusion, limiting blood oxygenation, and inducing a higher load on the right ventricle. Pulmonary embolism is diagnosed using contrast enhanced Computed Tomography Pulmonary Angiography (CTPA), resulting in a 3 D image where the pulmonary arteries appear as bright structures, and emboli appear as filling defects, with these often being difficult to see, especially in the subsegmental case. In comparison to an expert panel, the average radiologist has a sensitivity of between 77% and 94 % . Computer Aided Detection (CAD) is regarded as a promising system to detect emboli, but current algorithms are hindered by a high false positive rate. In this paper, we propose a novel methodology for emboli detection. Instead of finding candidate points and characterizing them, we find emboli directly on the whole image slice. Detections across different slices are merged into a single detection volume that is post-processed to generate emboli detections. The system was evaluated on a public PE database of 80 scans. On 20 test scans, our system obtained a per-embolus sensitivity of 68% at a regime of one false positive per scan, improving on state-of-the-art methods. We therefore conclude that our multi-slice emboli segmentation CAD for PE method is a valuable alternative to the standard methods of candidate point selection and classification.

As estimated by the World Health Organization, there are millions of people who lives with some form of vision impairment. As a consequence, some of them present mobility problems in outdoor environments. With the aim of helping them, we propose in this work a system which is capable of delivering the position of potential obstacles in outdoor scenarios. Our approach is based on non-intrusive wearable devices and focuses also on being low-cost. First, a depth map of the scene is estimated from a color image, which provides 3D information of the environment. Then, an urban object detector is in charge of detecting the semantics of the objects in the scene. Finally, the three-dimensional and semantic data is summarized in a simpler representation of the potential obstacles the users have in front of them. This information is transmitted to the user through spoken or haptic feedback. Our system is able to run at about 3.8 fps and achieved a 87.99% mean accuracy in obstacle presence detection. Finally, we deployed our system in a pilot test which involved an actual person with vision impairment, who validated the effectiveness of our proposal for improving its navigation capabilities in outdoors. This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds, the University of Alicante project GRE16-19, and by the Valencian Government project GV/2018/022. Edmanuel Cruz is funded by a Panamenian grant for PhD studies IFARHU & SENACYT 270-2016-207. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243. Thanks also to Nvidia for the generous donation of a Titan Xp and a Quadro P6000.

Over one billion people in the world suffer from some form of disability. Nevertheless, according to the World Health Organization, people with disabilities are particularly vulnerable to deficiencies in services, such as health care, rehabilitation, support, and assistance. In this sense, recent technological developments can mitigate these deficiencies, offering less-expensive assistive systems to meet users&rsquo, needs. This paper reviews and summarizes the research efforts toward the development of these kinds of systems, focusing on two social groups: older adults and children with autism.

Este proyecto tiene como objetivo estudiar el uso de la competición en el proceso de enseñanza-aprendizaje. Para ello, se ha diseñado una experiencia piloto en la asignatura de “Visión por Computador” de 3º curso del grado en Ingeniería Robótica. Esta experiencia consistió en el diseño de una actividad cuya evaluación tuvo lugar a través de la competición, mientras que el resto de actividades de la asignatura siguió con una evaluación tradicional. Al finalizar la actividad basada en competición, se pidió al estudiantado que valorara el uso de la competición como herramienta de evaluación y la comparara con las metodologías tradicionales usadas en el resto de actividades. Esta encuesta anónima y voluntaria fue respondida por 31 alumnos/as, valorando positivamente la competencia en términos de conceptos de aprendizaje y admitiendo que pasaron más tiempo en la resolución debido a su compromiso. Sin embargo, el 80% de los estudiantes no recomiendan esta actividad para futuros cursos. Las razones principales son que se sintieron estresados y decepcionados por no ver reflejado adecuadamente su esfuerzo y trabajo en sus calificaciones. Este comentario resalta problemas en el diseño de la competencia que deberían resolverse para tener éxito en la aplicación de la competencia en cursos futuros.

Biomarker estimation methods from medical images have traditionally followed a segment-and-measure strategy. Deep-learning regression networks have changed such a paradigm, enabling the direct estimation of biomarkers in databases where segmentation masks are not present. While such methods achieve high performance, they operate as a black-box. In this work, we present a novel deep learning network structure that, when trained with only the value of the biomarker, can perform biomarker regression and the generation of an accurate localization mask simultaneously, thus enabling a qualitative assessment of the image locus that relates to the quantitative result. We showcase the proposed method with three different network structures and compare their performance against direct regression networks in four different problems: pectoralis muscle area (PMA), subcutaneous fat area (SFA), liver mass area in single slice computed tomography (CT), and Agatston score estimated from non-contrast thoracic CT images (CAC). Our results show that the proposed method improves the performance with respect to direct biomarker regression methods (correlation coefficient of 0.978, 0.998, and 0.950 for the proposed method in comparison to 0.971, 0.982, and 0.936 for the reference regression methods on PMA, SFA and CAC respectively) while achieving good localization (DICE coefficients of 0.875, 0.914 for PMA and SFA respectively, p < 0.05 for all pairs). We observe the same improvement in regression results comparing the proposed method with those obtained by quantify the outputs using an U-Net segmentation network (0.989 and 0.951 respectively). We, therefore, conclude that it is possible to obtain simultaneously good biomarker regression and localization when training biomarker regression networks using only the biomarker value. This work was supported in part by the National Institutes of Health (NHLBI) under Grant R01HL116931, Grant R21HL14042, and Grant R01HL149877, in part by the COPDGene Study through the NHLBI under Grant NCT00608764, Grant U01 HL089897, and Grant U01 HL089856, and in part by the COPD Foundation through contributions made to the Industry Advisory Committee comprised of AstraZeneca, Boehringer-Ingelheim, GlaxoSmithKline, Novartis, and Sunovion.

In several large retail stores, such as malls, sport or food stores, the customer often feels lost due to the difficulty in finding a product. Although these large stores usually have visual signs to guide customers toward specific products, sometimes these signs are also hard to find and are not updated. In this paper, we propose a system that jointly combines deep learning and augmented reality techniques to provide the customer with useful information. First, the proposed system learns the visual appearance of different areas in the store using a deep learning architecture. Then, customers can use their mobile devices to take a picture of the area where they are located within the store. Uploading this image to the system trained for image classification, we are able to identify the area where the customer is located. Then, using this information and novel augmented reality techniques, we provide information about the area where the customer is located: route to another area where a product is available, 3D product visualization, user location, analytics, etc. The system developed is able to successfully locate a user in an example store with 98% accuracy. The combination of deep learning systems together with augmented reality techniques shows promising results toward improving user experience in retail/commerce applications: branding, advance visualization, personalization, enhanced customer experience, etc. In several large retail stores, such as malls, sport or food stores, the customer often feels lost due to the difficulty in finding a product. Although these large stores usually have visual signs to guide customers toward specific products, sometimes these signs are also hard to find and are not updated. In this paper, we propose a system that jointly combines deep learning and augmented reality techniques to provide the customer with useful information. First, the proposed system learns the visual appearance of different areas in the store using a deep learning architecture. Then, customers can use their mobile devices to take a picture of the area where they are located within the store. Uploading this image to the system trained for image classification, we are able to identify the area where the customer is located. Then, using this information and novel augmented reality techniques, we provide information about the area where the customer is located: route to another area where a product is available, 3D product visualization, user location, analytics, etc. The system developed is able to successfully locate a user in an example store with 98% accuracy. The combination of deep learning systems together with augmented reality techniques shows promising results toward improving user experience in retail/commerce applications: branding, advance visualization, personalization, enhanced customer experience, etc.

In this paper, we propose a new dataset for outdoor depth estimation from single and stereo RGB images. The dataset was acquired from the point of view of a pedestrian. Currently, the most novel approaches take advantage of deep learning-based techniques, which have proven to outperform traditional state-of-the-art computer vision methods. Nonetheless, these methods require large amounts of reliable ground-truth data. Despite there already existing several datasets that could be used for depth estimation, almost none of them are outdoor-oriented from an egocentric point of view. Our dataset introduces a large number of high-definition pairs of color frames and corresponding depth maps from a human perspective. In addition, the proposed dataset also features human interaction and great variability of data, as shown in this work., Design Type(s)modeling and simulation objective • image creation and editing objective • database creation objectiveMeasurement Type(s)imageTechnology Type(s)digital cameraFactor Type(s)geographic locationSample Characteristic(s)Alicante • anthropogenic habitat Machine-accessible metadata file describing the reported data (ISA-Tab format)

Advances in Robotics are leading to a new generation of assistant robots working in ordinary, domestic settings. This evolution raises new challenges in the tasks to be accomplished by the robots. This is the case for object manipulation where the detect-approach-grasp loop requires a robust recovery stage, especially when the held object slides. Several proprioceptive sensors have been developed in the last decades, such as tactile sensors or contact switches, that can be used for that purpose, nevertheless, their implementation may considerably restrict the gripper’s flexibility and functionality, increasing their cost and complexity. Alternatively, vision can be used since it is an undoubtedly rich source of information, and in particular, depth vision sensors. We present an approach based on depth cameras to robustly evaluate the manipulation success, continuously reporting about any object loss and, consequently, allowing it to robustly recover from this situation. For that, a Lab-colour segmentation allows the robot to identify potential robot manipulators in the image. Then, the depth information is used to detect any edge resulting from two-object contact. The combination of those techniques allows the robot to accurately detect the presence or absence of contact points between the robot manipulator and a held object. An experimental evaluation in realistic indoor environments supports our approach.

Any robot should be provided with a proper representation of its environment in order to perform navigation and other tasks. In addition to metrical approaches, topological mapping generates graph representations in which nodes and edges correspond to locations and transitions. In this article, we present LexToMap, a topological mapping procedure that relies on image annotations. These annotations, represented in this work by lexical labels, are obtained from pre-trained deep learning models, namely CNNs, and are used to estimate image similarities. Moreover, the lexical labels contribute to the descriptive capabilities of the topological maps. The proposal has been evaluated using the KTH-IDOL 2 data-set, which consists of image sequences acquired within an indoor environment under three different lighting conditions. The generality of the procedure as well as the descriptive capabilities of the generated maps validate the proposal. Any robot should be provided with a proper representation of its environment in order to perform navigation and other tasks. In addition to metrical approaches, topological mapping generates graph representations in which nodes and edges correspond to locations and transitions. In this article, we present LexToMap, a topological mapping procedure that relies on image annotations. These annotations, represented in this work by lexical labels, are obtained from pre-trained deep learning models, namely CNNs, and are used to estimate image similarities. Moreover, the lexical labels contribute to the descriptive capabilities of the topological maps. The proposal has been evaluated using the KTH-IDOL 2 data-set, which consists of image sequences acquired within an indoor environment under three different lighting conditions. The generality of the procedure as well as the descriptive capabilities of the generated maps validate the proposal.

This work presents the design of a real-time system to model visual objects with the use of self-organising networks. The architecture of the system addresses multiple computer vision tasks such as image segmentation, optimal parameter estimation and object representation. We first develop a framework for building non-rigid shapes using the growth mechanism of the self-organising maps, and then we define an optimal number of nodes without overfitting or underfitting the network based on the knowledge obtained from information-theoretic considerations. We present experimental results for hands and faces, and we quantitatively evaluate the matching capabilities of the proposed method with the topographic product. The proposed method is easily extensible to 3D objects, as it offers similar features for efficient mesh reconstruction. This work was partially funded by the Spanish Government DPI2013-40534-R Grant.

Schaeffer's sign language consists of a reduced set of gestures designed to help children with autism or cognitive learning disabilities to develop adequate communication skills. Our automatic recognition system for Schaeffer's gesture language uses the information provided by an RGB-D camera to capture body motion and recognize gestures using dynamic time warping combined with k-nearest neighbors methods. The learning process is reinforced by the interaction with the proposed system that accelerates learning itself thus helping both children and educators. To demonstrate the validity of the system, a set of qualitative experiments with children were carried out. As a result, a system which is able to recognize a subset of 11 gestures of Schaeffer's sign language online was achieved. This work has been supported by the Spanish Government DPI2013-40534-R Grant, supported with Feder funds.