Search

Human-in-the-loop approaches can greatly enhance the human–robot interaction by making the user an active part of the control loop, who can provide a feedback to the robot in order to augment its capabilities. Such feedback becomes even more important in all those situations where safety is of utmost concern, such as in assistive robotics. This study aims to realize a human-in-the-loop approach, where the human can provide a feedback to a specific robot, namely, a smart wheelchair, to augment its artificial sensory set, extending and improving its capabilities to detect and avoid obstacles. The feedback is provided by both a keyboard and a brain–computer interface: with this scope, the work has also included a protocol design phase to elicit and evoke human brain event–related potentials. The whole architecture has been validated within a simulated robotic environment, with electroencephalography signals acquired from different test subjects.

In this study, the neuromuscular control modeling of the perturbed human upright stance is assessed through piecewise affine autoregressive with exogenous input (PWARX) models. Ten healthy subjects underwent an experimental protocol where visual deprivation and cognitive load are applied to evaluate whether PWARX can be used for modeling the role of the central nervous system (CNS) in balance maintenance in different conditions. Balance maintenance is modeled as a single-link inverted pendulum; and kinematic, dynamic, and electromyography (EMG) data are used to fit the PWARX models of the CNS activity. Models are trained on 70% and tested on the 30% of unseen data belonging to the remaining dataset. The models are able to capture which factors the CNS is subjected to, showing a fitting accuracy higher than 90% for each experimental condition. The models present a switch between two different control dynamics, coherent with the physiological response to a sudden balance perturbation and mirrored by the data-driven lag selection for data time series. The outcomes of this study indicate that hybrid postural control policies, yet investigated for unperturbed stance, could be an appropriate motor control paradigm when balance maintenance undergoes external disruption.

The ability to control external devices through thought is increasingly becoming a reality. Human beings can use the electrical signals of their brain to interact or change the surrounding environment and more. The development of this technology called brain-computer interface (BCI) will increasingly allow people with motor disabilities to communicate or use assistive devices to walk, manipulate objects and communicate. Using data from the PhysioNet database, this study implemented a pattern classification system for use in a BCI on 109 healthy volunteers during real movement activities and motor imagery recorded by 64-channels electroencephalography (EEG) system. Different classifiers such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Trees (TREE) were applied on different combinations of EEG channels. Starting from two channels (C3, C4 and CP3 and CP4) positioned on the contralateral and ipsilateral sensorimotor cortex, the Region of Interest (RoI) centred on C3/Cp3 and C4/Cp4 and, finally, a data-driven automatic channels selection was tested to explore the best channel combination able to increase the classification accuracy. The results showed that the proposed automatic channels selection was able to significantly improve the performance of each classifier achieving 98% of accuracy for classification of real and imagined hand movement (sensitivity = 97%, specificity = 99%, AUC = 0.99) by SVM. While the accuracy of the classification between the imagery of hand and foot movements was 91% (sensitivity = 87%, specificity = 86%, AUC = 0.93) also with SVM. In the proposed approach, the data-driven automatic channels selection outperforms classical a priori channel selection models such as C3/C4, Cp3/Cp4, or RoIs around those channels with the utmost accuracy to help remove the boundaries of human communication and improve the quality of life of people with disabilities.

Background: Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015. Objectives: We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic. Methods: We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends. Results: Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency. Significance: To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

In this editorial, we provide an overview of the content of the Special Issue on “Smart Homes„. The aim of this Special Issue is to provide a comprehensive collection of some of the current state-of-the-art technologies in the context of smart homes, together with new advanced theoretical and technological solutions that enable smart technology diffusion into homes.

This paper presents the redundancy analysis of two cooperative manipulators, showing how they can be considered as a single redundant manipulator through the use of the relative Jacobian matrix. In this way, the kinematic redundancy can be resolved by applying the principal local optimization techniques used in the single manipulator case. We resolve the redundancy by using the Jacobian null space technique, which permits us to perform several tasks with different execution priority levels at the same time; this is a useful feature, especially when the manipulators are to be mounted on and cooperate with a mobile platform. As an illustrative example, we present a case study consisting of two planar manipulators mounted on a smart wheelchair, whose degrees of redundancy are employed to move an object along a pre-defined path, while avoiding an obstacle in the manipulator’s workspace at the same time.

Smart homes play a strategic role for improving life quality of people, enabling to monitor people at home with numerous intelligent devices. Sensors can be installed to provide a continuous assistance without limiting the resident’s daily routine, giving her/him greater comfort, well-being and safety. This paper is based on the development of domestic technological solutions to improve the life quality of citizens and monitor the users and the domestic environment, based on features extracted from the collected data. The proposed smart sensing architecture is based on an integrated sensor network to monitor the user and the environment to derive information about the user’s behavior and her/his health status. The proposed platform includes biomedical, wearable, and unobtrusive sensors for monitoring user’s physiological parameters and home automation sensors to obtain information about her/his environment. The sensor network stores the heterogeneous data both locally and remotely in Cloud, where machine learning algorithms and data mining strategies are used for user behavior identification, classification of user health conditions, classification of the smart home profile, and data analytics to implement services for the community. The proposed solution has been experimentally tested in a pilot study based on the development of both sensors and services for elderly users at home.

The integration of Wireless Sensors Networks (WSNs) into Cyber Physical Systems (CPSs) is an important research problem to solve in order to increase the performances, safety, reliability and usability of wireless automation systems. Due to the complexity of real CPSs, emulators and simulators are often used to replace the real control devices and physical connections during the development stage. The most widespread simulators are free, open source, expandable, flexible and fully integrated into mathematical modeling tools; however, the connection at a physical level and the direct interaction with the real process via the WSN are only marginally tackled; moreover, the simulated wireless sensor motes are not able to generate the analogue output typically required for control purposes. A new simulation module for the control of a wireless cyber-physical system is proposed in this paper. The module integrates the COntiki OS JAva Simulator (COOJA), a cross-level wireless sensor network simulator, and the LabVIEW system design software from National Instruments. The proposed software module has been called “GILOO” (Graphical Integration of Labview and cOOja). It allows one to develop and to debug control strategies over the WSN both using virtual or real hardware modules, such as the National Instruments Real-Time Module platform, the CompactRio, the Supervisory Control And Data Acquisition (SCADA), etc. To test the proposed solution, we decided to integrate it with one of the most popular simulators, i.e., the Contiki OS, and wireless motes, i.e., the Sky mote. As a further contribution, the Contiki Sky DAC driver and a new “Advanced Sky GUI” have been proposed and tested in the COOJA Simulator in order to provide the possibility to develop control over the WSN. To test the performances of the proposed GILOO software module, several experimental tests have been made, and interesting preliminary results are reported. The GILOO module has been applied to a smart home mock-up where a networked control has been developed for the LED lighting system.

Background: Human-Machine Interaction (HMI) has been an important field of research in recent years, since machines will continue to be embedded in many human actvities in several contexts, such as industry and healthcare. Monitoring in an ecological mannerthe cognitive workload (CW) of users, who interact with machines, is crucial to assess their level of engagement in activities and the required effort, with the goal of preventing stressful circumstances. This study provides a comprehensive analysis of the assessment of CW using wearable sensors in HMI. Methods: this narrative review explores several techniques and procedures for collecting physiological data through wearable sensors with the possibility to integrate these multiple physiological signals, providing a multimodal monitoring of the individuals’CW. Finally, it focuses on the impact of artificial intelligence methods in the physiological signals data analysis to provide models of the CW to be exploited in HMI. Results: the review provided a comprehensive evaluation of the wearables, physiological signals, and methods of data analysis for CW evaluation in HMI. Conclusion: the literature highlighted the feasibility of employing wearable sensors to collect physiological signals for an ecological CW monitoring in HMI scenarios. However, challenges remain in standardizing these measures across different populations and contexts.

In Awake Neurosurgery (AN), the assessment of the patient’s capabilities is of paramount importance to minimize the risk of post-operative cognitive, language, and motor deficits. To retain fine hand motion capabilities, in the current clinical practice, a neuropsychologist assigns hand motion tasks to the patient during AN, and these tasks are evaluated by visual observation. This study introduces an innovative and non-invasive task evaluation method that employs an infrared stereo camera for precise hand pose acquisition. The primary contribution of this work is to offer objective, quantitative and reproducible task evaluations, addressing a critical aspect of patient care during AN procedures. The proposed method focuses on dynamic hand gestures recognition and utilizes unidirectional and bidirectional Long Short-Term Memory networks to assess common motor tasks during AN. To evaluate the tasks, we employ state-of-the-art features, both distance-based and angle-based, formed by the finger bones of the human hands. These feature vectors demonstrate a promising accuracy and inter-patient portability for the tasks under consideration, with mean accuracy exceeding 80% when tested on new patients, separate from those used in training. This approach provides an efficient solution for the identification and assessment of tasks, also eliminating the need for task-specific labeling. This, in turn, enhances usability and reduces the potential for human error. The proposed method has the potential to improve clinical decision-making in AN and offers reliable classification tools for neuropsychologists.

Abstract The performances of energy management systems or electric vehicles and hybrid electric vehicles are highly dependent on the forecast of future driver torque/power request sequence that affects vehicle efficiency and economy. Since the behaviour of the driver is challenging to model/predict by first‐principles models, modern artificial intelligence algorithms would represent feasible methods for approaching this problem in real‐world automotive systems. This work provides a comparative study and analysis of performances of different data‐driven torque prediction strategies. The studied and compared torque demand prediction techniques are exponentially varying model, linear regression, shallow and deep neural networks, and least square support vector machine‐based approaches. The prediction performance and computational cost of these techniques are evaluated and reported, and the possibility of exploiting these techniques in real‐world scenarios is also discussed.

The actuator fault-tolerant control problem for a variable-pitch quadrotor is addressed under uncertain conditions. Following a model-based approach, the plant nonlinear dynamics are faced with a disturbance observer-based control and a sequential quadratic programming control allocation, where only kinematic data of the onboard inertial measurement unit are required for the fault-tolerant control, i.e., it does not require the measurement of the motor speed nor the current drawn by the actuators. In the case of almost horizontal wind, a single observer handles both faults and the external disturbance. The estimation of the wind is fed forward by the controller, while the actuator fault estimation is exploited in the control allocation layer, which copes with the variable-pitch nonlinear dynamics, thrust saturation, and rate limits. Numerical simulations in the presence of measurement noise show the capability of the scheme to handle multiple actuator faults in a windy environment.

In this paper, we propose a fault tolerant control law for a morphing quadrotor, where the considered morphing ability is that of extendable/telescopic arms. This quite recent class of systems is able to provide a good trade-off between payload capabilities, maneuverability, and space occupancy. However, such degrees of freedom require dedicated servomotors, which in turn implies more possible faults. Thus, the problem of diagnosis for the telescopic servo motors subject to a stuck fault is considered. System symmetries are exploited and used in a residual generator design, which triggers an active fault isolation/identification phase. External disturbances are also taken into account and estimated through a nonlinear disturbance observer. A classical double-loop controller closes the loop, providing an overall control system structure that follows the disturbance observer-based control paradigm. The control scheme is validated through realistic numerical simulations, and the closed-loop performances are analyzed.

Here, the authors present a low cost telerehabilitation system made up of a commercial red–green–blue depth (RGB‐D) camera and a web‐based platform. The authors goal is to monitor and assess subject movement providing acceptable and usable at‐home remote rehabilitation services without the presence of a clinician. Clinical goals, defined by physiotherapists, are firstly translated into motion analysis features. A Takagi Sugeno fuzzy inference system (FIS) is then proposed to evaluate and combine these features into scores. In this stage, the ‘collaborative design’ paradigm is used in depth and complete manner: the contribution of the clinician is not limited only to the rules definition but enters in the core of the evaluation algorithm through the definition of the fuzzy rules. A case study on low back pain rehabilitation involving 40 subjects, 5 exercises, and 4 physiotherapists is then presented to the effectiveness of the proposed system. Results of the validation of the system aimed at the assessment of the reliability of the proposed approach show high correlations between clinician evaluation and FIS scores. In this scenario, due to the high correlation, each FIS could represent a virtual alter‐ego of the physiotherapist which enable a real time and free second opinion.