Search

Assistive robots operate in complex environments and in presence of human beings, but the interaction between them can be affected by several factors, which may lead to undesired outcomes: wrong sensor readings, unexpected environmental conditions, or algorithmic errors represent just a few examples of the possible scenarios. When the safety of the user is not only an option but must be guaranteed, a feasible solution is to rely on a human-in-the-loop approach, e.g., to monitor if the robot performs a wrong action during a task execution or environmental conditions affect safety during the human-robot interaction, and provide a feedback accordingly. The present paper proposes a human-in-the-loop framework to enable safe autonomous navigation of an electric powered and sensorized (smart) wheelchair. During the wheelchair navigation towards a desired destination in an indoor scenario, possible problems (e.g. obstacles) along the trajectory cause the generation of electroencephalography (EEG) potentials when noticed by the user. These potentials can be used as additional inputs to the navigation algorithm in order to modify the trajectory planning and preserve safety. The framework has been preliminarily tested by using a wheelchair simulator implemented in ROS and Gazebo environments: EEG signals from a benchmark known in the literature were classified, passed to a custom simulation node, and made available to the navigation stack to perform obstacle avoidance.

The development of robots, which can safely and effectively interact with people and assist them in structured environments, is an open research problem whose importance has been growing rapidly in the last years. Indeed working in shared environments with human beings, these robots require new ways to achieve human–robot interaction and cooperation. This work presents an approach for performing human–robot interaction by means of robotic manipulators. The interaction is composed by three main steps, namely the selection, the recognition and the grasping of an object. The object selection is recorded on the base of a gesture execution, realized by the user in front of a RGB-D camera and associated to each particular object. The object recognition is achieved by means of the RGB cameras mounted on the two manipulator arms, which send the workspace information to a specific classifier. With the aim of realizing the grasping step, the object position and orientation are extracted in order to correctly rotate the gripper according to the object on the desk in front of the robot. The final goal is to release the grasped object on the hand of the user standing in front of the desk. This system could support people with limited motor skills who are not able to get an object on their own, playing an important role in structured assistive and smart environments, thus promoting the human–robot interaction in activity of daily living.

Assistive robots operate in complex environments and in presence of human beings, as such they are influenced by several factors which may lead to undesired outcomes: wrong sensor readings, unexpected environmental conditions or algorithmic errors represent just few examples. When the safety of the user must be guaranteed, a possible solution is to rely on a human-in-the-loop approach, e.g. to monitor if the robot performs a wrong action or environmental conditions affect safety during the interaction, and provide a feedback accordingly. The proposed work presents a human supervised smart wheelchair, i.e. an electric powered wheelchair with semiautonomous navigation capabilities of elaborating a path planning, whose user is equipped with a Brain Computer Interface (BCI) to provide safety feedbacks. During the wheelchair navigation towards a desired destination in an indoor scenario, possible problems (e.g. obstacles) along the trajectory cause the generation of error-related potentials signals (ErrPs) when noticed by the user. These signals are captured by the interface and are used to provide a feedback to the navigation task, in order to preserve safety and avoiding possible navigation issues modifying the trajectory planning.

Background: Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015. Objectives: We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic. Methods: We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends. Results: Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency. Significance: To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

Background and objectives An Error related Potential (ErrP) can be noninvasively and directly measured from the scalp through electroencephalography (EEG), as response, when a person realizes they are making an error during a task (as a consequence of a cognitive error performed from the user). It has been shown that ErrPs can be automatically detected with time-discrete feedback tasks, which are widely applied in the Brain-Computer Interface (BCI) field for error correction or adaptation. In this work, a semi-supervised algorithm, namely the Functional Source Separation (FSS), is proposed to estimate a spatial filter for learning the ErrPs and to enhance the evoked potentials. Methods EEG data recorded on six subjects were used to evaluate the proposed method based on FFS algorithm in comparison with the xDAWN algorithm. FSS- and xDAWN-based methods were compared also to the Cz and FCz single channel. Single-trial classification was considered to evaluate the performances of the approaches. (Both the approaches were evaluated on single-trial classification of EEGs.) Results The results presented using the Bayesian Linear Discriminant Analysis (BLDA) classifier, show that FSS (accuracy 0.92, sensitivity 0.95, specificity 0.81, F1-score 0.95) overcomes the other methods (Cz - accuracy 0.72, sensitivity 0.74, specificity 0.63, F1-score 0.74; FCz - accuracy 0.72, sensitivity 0.75, specificity 0.61, F1-score 0.75; xDAWN - accuracy 0.75, sensitivity 0.79, specificity 0.61, F1-score 0.79) in terms of single-trial classification. Conclusions The proposed FSS-based method increases the single-trial detection accuracy of ErrPs with respect to both single channel (Cz, FCz) and xDAWN spatial filter.

This work proposes a real-time monitoring tool aimed to support clinicians for remote assessing exercise performances during home-based rehabilitation. The study relies on clinician indications to define kinematic features, that describe five motor tasks (i.e., the lateral tilt of the trunk, lifting of the arms, trunk rotation, pelvis rotation, squatting) usually adopted in the rehabilitation program for axial disorders. These features are extracted by the Kinect v2 skeleton tracking system and elaborated to return disaggregated scores, representing a measure of subjects performance. A bell-shaped function is used to rank the patient performances and to provide the scores. The proposed rehabilitation tool has been tested on 28 healthy subjects and on 29 patients suffering from different neurological and orthopedic diseases. The reliability of the study has been performed through a cross-sectional controlled design methodology, comparing algorithm scores with respect to blinded judgment provided by clinicians through filling a specific questionnaire. The use of task-specific features and the comparison between the clinical evaluation and the score provided by the instrumental approach constitute the novelty of the study. The proposed methodology is reliable for measuring subject’s performance and able to discriminate between the pathological and healthy condition.

In this paper, a Hidden Semi-Markov Model (HSMM) based approach is proposed to evaluate and monitor body motion during a rehabilitation training program. The approach extracts clinically relevant motion features from skeleton joint trajectories, acquired by the RGB-D camera, and provides a score for the subject's performance. The approach combines different aspects of rule and template based methods. The features have been defined by clinicians as exercise descriptors and are then assessed by a HSMM, trained upon an exemplar motion sequence. The reliability of the proposed approach is studied by evaluating its correlation with both a clinical assessment and a Dynamic Time Warping (DTW) algorithm, while healthy and neurological disabled people performed physical exercises. With respect to the discrimination between healthy and pathological conditions, the HSMM based method correlates better with the physician's score than DTW. The study supports the use of HSMMs to assess motor performance providing a quantitative feedback to physiotherapist and patients. This result is particularly appropriate and useful for a remote assessment in the home.

This paper proposes a free dataset, available at the following link, 1 named KIMORE, regarding different rehabilitation exercises collected by a RGB-D sensor. Three data inputs including RGB, depth videos, and skeleton joint positions were recorded during five physical exercises, specific for low back pain and accurately selected by physicians. For each exercise, the dataset also provides a set of features, specifically defined by the physicians, and relevant to describe its scope. These features, validated with respect to a stereophotogrammetric system, can be analyzed to compute a score for the subject’s performance. The dataset also contains an evaluation of the same performance provided by the clinicians, through a clinical questionnaire. The impact of KIMORE has been analyzed by comparing the output obtained by an example of rule and template-based approaches and the clinical score. The dataset presented is intended to be used as a benchmark for human movement assessment in a rehabilitation scenario in order to test the effectiveness and the reliability of different computational approaches. Unlike other existing datasets, the KIMORE merges a large heterogeneous population of 78 subjects, divided into 2 groups with 44 healthy subjects and 34 with motor dysfunctions. It provides the most clinically-relevant features and the clinical score for each exercise. 1 https://univpm-my.sharepoint.com/:f:/g/personal/p008099_staff_univpm_it/EiwbKIzk6N9NoJQx4J8aubIBx0o7tIa1XwclWp1NmRkA-w?e=F3jtBk

Assistive robots operate in complex environments and in presence of human beings, as such they are influenced by several factors which may lead to undesired outcomes: wrong sensor readings, unexpected environmental conditions or algorithmic errors represent just few examples. When the safety of the user must be guaranteed, a possible solution is to rely on a human-supervised approach. The proposed work presents a smart wheelchair, i.e. an electric powered wheelchair with semiautonomous navigation capabilities, whose user is equipped with a Brain Computer Interface. During the wheelchair navigation, possible problems (e.g. obstacles) along the trajectory cause the generation of error-related potentials signals when noticed by the user. These signals are captured by the interface and are used to provide a feedback to the navigation task, in order to preserve safety and avoiding possible navigation issues.

In the last years, the development of robots for assisting and collaborating with people has experienced a large growth. Applications for assistive robots include hospital service robots, factory intelligent assistants and personal homecare robots. Working in shared environments with human beings, these robots require effective ways to achieve an increasing human-robot cooperation. This work presents a possible approach for performing human-robot cooperation, namely recognition of a user selected object by means of Brain Computer Interface (BCI), followed by pick and place via a robotic arm. The object selection is achieved introducing a BCI that allows the user, after a training phase, to choose one among six different objects of common diffusion. The selection is achieved by interpreting the P300 signals generated in the brain, when the image of the object, desidered by the user, appears on a computer screen as a visual stimulus. The robot then recognizes, through a classifier, the selected object among others within its workspace, and inscribes it in a rectangle shape. Finally, the robot arm is moved in correspondence to the object position, the gripper is rotated according to the object orientation and the object grasped and moved into a different position on a desk in front of the robot. This system could support people with limited motor skills or paralysis, playing an important role in structured assistive environments in a near future.

The present study aims at providing a robotic architecture system composed by a semi-autonomous mobile robot and a manipulator robot. The user, on a smart wheelchair, is able to move in an indoor environment, by selecting, through a user-interface, the room where he/she wants to move and the object he/she would like to get. In case an object is selected, the user is automatically driven in front of a robotic workstation, where the desired object is picked and placed by a robotic arm in front of the user. The system has been tested on a simulator, and preliminary experimental results show the feasibility of the proposed architecture.

The present paper deals with the defect detection and diagnosis of induction motor, based on motor current signature analysis in a quality control scenario. In order to develop a monitoring system and improve the reliability of induction motors, Clarke-Concordia transformation and kernel density estimation are employed to estimate the probability density function of data related to healthy and faulty motors. Kullback-Leibler divergence identifies the dissimilarity between two probability distributions and it is used as an index for the automatic defects identification. Kernel density estimation is improved by fast Gaussian transform. Since these techniques achieve a remarkable computational cost reduction respect the standard kernel density estimation, the developed monitoring procedure became applicable on line, as a Quality Control method for the end of production line test.Several simulations and experimentations are carried out in order to verify the proposed methodology effectiveness: broken rotor bars and connectors are simulated, while experimentations are carried out on real motors at the end of production line. Results show that the proposed data-driven diagnosis procedure is able to detect and diagnose different induction motor faults and defects, improving the reliability of induction machines in quality control scenario.

This work proposes an approach for performing Human-Robot Cooperation (HRC) tasks by integrating a Brain Computer Interface (BCI) with a robotic manipulator. In detail, the user can select one among six different objects via BCI, which analyzes the P300 signals generated by the brain when the images of the selectable objects appear on the screen. Then, the selected object is recognized by a Support Vector Machine (SVM) classifier, which scans the data acquired by the robot camera. Finally, the recognized object is grasped by the robotic arm and moved to a different position.

Assistive technologies have the objective to improve the people quality of life of in daily living, with a special aim to those who suffer of physical disabilities or cognitive impairment, which may be caused by an accident, disease or the natural process of ageing. The present paper describes the main results of a study realized for the INTERREG IVC INNOVAGE project, where the domain target addressed are: home and building automation and assistive robotics. The project provides a quick overview of the typical needs of elderly people, describes the state-of-the-art technologies which can be adopted to satisfy these needs and presents a critical analysis of the functionalities, which present and future assistive technologies should possess. The result of this study is a detailed assessments of requirements and limits of nowadays domotics and robotics technologies aimed to improve people quality of life.

Complex systems are found in almost all field of contemporary science and are associated with a wide variety of financial, physical, biological, information and social systems. Complex systems modelling could be addressed by signal based procedures, which are able to learn the complex system dynamics from data provided by sensors, which are installed on the system in order to monitor its physical variables. In this chapter the aim of diagnosis is to detect if the electrical machine is healthy or a change is occurring due to abnormal events and, in addition, the probable causes of the abnormal events. Diagnosis is addressed by developing machine learning procedures in order to classify the probable causes of deviations from system normal events. This chapter presents two Fault Detection and Diagnosis solutions for rotating electrical machines by signal based approaches. The first one uses a current signature analysis technique based on Kernel Density Estimation and Kullback–Liebler divergence. The second one presents a vibration signature analysis technique based on Multi-Scale Principal Component Analysis. Several simulations and experimentations on real electric motors are carried out in order to verify the effectiveness of the proposed solutions. The results show that the proposed signal based diagnosis procedures are able to detect and diagnose different electric motor faults and defects, improving the reliability of electrical machines. Fault Detection and Diagnosis algorithms could be used not only with the fault diagnosis purpose but also in a Quality Control scenario. In fact, they can be integrated in test benches at the end or in the middle of the production line in order to test the machines quality. When the electric motors reach the test benches, the sensors acquire measurements and the Fault Detection and Diagnosis procedures detect if the motor is healthy or faulty, in this last case further inspections can diagnose the fault.

In the contest of household energy management, a growing interest is addressed to smart system development, able to monitor and manage resources in order to minimize wasting. One of the key factors in curbing energy consumption in the household sector is the amendment of occupant erroneous behaviours and systems malfunctioning, due to the lack of awareness of the final user. Indeed the benefits achievable with energy efficiency could be either amplified or neutralized by, respectively, good or bad practices carried out by the final users. Authors propose a diagnostic system for home energy management application able to detect faults and occupant behaviours. In particular a nonlinear monitoring method, based on Kernel Canonical Variate Analysis, is developed. To remove the assumption of normality, Upper Control Limits are derived from the estimated Probability Density Function through Kernel Density Estimation. The proposed method is applied to smart home temperature sensors to detect anomalies respect to efficient user behaviours and sensors and actuators faults. The method is tested on experimental data acquired in a real apartment.

The present paper proposes an auditory BCI paradigm for systems based on P300 signals which are generated by auditory stimuli characterized by different sound typologies and locations. A Head Related Transfer Function approach is adopted to virtualize auditory stimuli. When virtualized audio is used, the user has to focus the attention both on the type and location of the stimulus, thus generating P300 signals whose amplitude is higher than that generated without audio virtualization. Classification is performed by Support Vector Machines in which gaussian radial basis functions are used as kernel functions. The system has been validated with 14 users, who were asked to choose one among five common spoken words, previously virtualized and transmitted to stereophonic headphones. Classification results prove that the proposed auditory BCI system performed similarly to common visual BCI P300 systems, representing then an alternative to visual BCI for users with visual impairments.