Search

People with profound intellectual and multiple disabilities (PIMD) are hard to understand because they are not capable of symbolic communication. Artificial intelligence can play a key role in recognizing behavior patterns with which they express themselves. It can thus assist new caregivers that are not familiar with a PIMD person. Within the INSENSION project, we developed a behavior pattern recognition approach that classifies a person’s inner states and communication attempts based on his/her facial expressions, gestures, vocalizations, and physiological signals.

While diary studies, especially when applying ecological momentary assessment (EMA), are a great way to capture self perceptions to later use as labels for other data, they can be a burden for study participants. To increase their adherence to the study design, it is important to tailor it to their needs and take their feedback into account. This paper reports on a data collection process in a study focused on occupational stress. The data collection is briefly described and the participants’ responses are analysed in terms of adherence. Participants’ feedback was collected at the end of the study and its main themes are summarized. These experiences are compared to the ones in another study focusing on stress and burnout, with a very similar methodological design. Some general conclusions are drawn from both with suggestions on how to best carry out an EMA study.

The SHL recognition challenge 2020 was an open competition in activity recognition where the participants were tasked with recognizing eight different modes of locomotion and transportation with smartphone sensors. The main challenges were that the training data was recorded by a different person than the validation and test data, and that the smartphone location in the test data was unknown to the participants. We, team "Third time's a charm", tackled the first challenge by attempting to identify the persons with clustering, and then performed cluster/person-specific feature selection to build a separate classifier for each person. The smartphone location appears not to make much difference. We also used semi-supervised learning to classify the test data. Internal tests using this methodology yielded an accuracy of 81.01%.

Human Energy Expenditure (EE) is a valuable tool for measuring physical activity and its impact on our body in an objective way. To accurately measure the EE, there are methods such as doubly labeled water and direct and indirect calorimetry, but their cost and practical limitations make them suitable only for research and professional sports. This situation, combined with the proliferation of commercial activity monitors, has stimulated the research of EE estimation (EEE) using machine learning on multimodal data from wearable devices. The article provides an overview of existing work in this evolving field, categorizes it, and makes publicly available an EEE dataset. Such a dataset is one of the most valuable resources for the development of the field but is generally not provided by researchers due to the high cost of collection. Finally, the article highlights best practices and promising future direction for designing EEE methods. Ministerio de Educación, Cultura y Deporte CAS17/00115 Ministerio de Economía y Competitividad TIN2017-82113-C2-1-R European Commission AAL-2013-6-060 European Union's Horizon 2020 No 689660

The Sussex-Huawei Locomotion Challenge 2019 was an open competition in activity recognition where the participants were tasked with recognizing eight different modes of locomotion and transportation. The main difficulty of the challenge is that the training data was recorded with a smartphone that was placed in a different body location than the test data. Only a small validation set with all locations was provided to enable transfer learning. This paper describes our (team JSI First) approach, in which we first derived additional sensor streams from the existing ones and on them calculated a large body of features. We then used cross-location transfer learning via specialized feature selection, and performed two-step classification. Finally, we used Hidden Markov Models to alter the predictions in order to take their temporal dependencies into account. Internal tests using this methodology yielded an accuracy of 83%.

The EU-funded project WellCo1 aims to deliver a new mobile app with a virtual coach to encourage the users towards healthier behaviour choices in order to improve their physical, cognitive, mental and social well-being. Healthy nutrition can substantially contribute to health and wellbeing. We will use different techniques for dietary assessment in the WellCo project - eating detection by gesture recognition using a wrist-worn device, and estimating the quality of diet by self-reporting using a Food Frequency Questionnaire (FFQ). This paper describes the latter. We designed a short FFQ, compared it to validated questionnaires, and developed a web service and a web application to determine dietary quality score for each user by using the designed FFQ.

UPDATED---August 17, 2018. Blood pressure (BP) is the most commonly performed medical office test. We developed a system that uses exclusively wristband-collected photoplethysmogram (PPG) to estimate BP. A dataset was collected and annotated during daily activities of 22 subjects. Preprocessing was applied to remove the signal noise and artefacts. Signal was segmented into cycles and features were computed. The RReliefF algorithm was used to select a subset of relevant features. The approach was validated with a person-independent leave-one-subject-out (LOSO) experiment. The LOSO experiment was updated with personalization to improve the results. The lowest mean absolute error (MAE) was 6.70 mmHg for systolic and 4.42 for diastolic BP. Ensemble of regression trees achieved the best results, which borderline meet the requirements set by two standards for BP estimation devices.

In recent years, activity recognition (AR) has become prominent in ubiquitous systems. Following this trend, the Sussex-Huawei Locomotion-Transportation (SHL) recognition challenge provides a unique opportunity for researchers to test their AR methods against a common, real-life and large-scale benchmark. The goal of the challenge is to recognize eight everyday activities including transit. Our team, JSI-Deep, utilized an AR approach based on combining multiple machine-learning methods following the principle of multiple knowledge. We first created several base learners using classical and deep learning approaches, then integrated them into an ensemble, and finally refined the ensemble's predictions by smoothing. On the internal test data, the approach achieved 96% accuracy, which is a significant leap over the baseline 60%.

The Sussex-Huawei Locomotion-Transportation recognition challenge presents a unique opportunity to the activity-recognition community - providing a large, real-life dataset with activities different from those typically being recognized. This paper describes our submission (team JSI Classic) to the competition that was organized by the dataset authors. We used a carefully executed machine learning approach, achieving 90% accuracy classifying eight different activities (Still, Walk, Run, Bike, Car, Bus, Train, Subway). The first step was data preprocessing, including a normalization of the phone orientation. Then, a wide set of hand-crafted domain features in both frequency and time domain were computed and their quality was evaluated. Finally, the appropriate machine learning model was chosen (XGBoost) and its hyper-parameters were optimized. The recognition result for the testing dataset will be presented in the summary paper of the challenge [13].

To prevent undesirable effects of attention grabbing at times when a user is occupied with a difficult task, ubiquitous computing devices should be aware of the user's cognitive load. However, inferring cognitive load is extremely challenging, especially when performed without obtrusive, expensive, and purpose-built equipment. In this study we examine the potential for inferring one's cognitive load using merely cheap wearable sensing devices. We subject 25 volunteers to varying cognitive load using six different Primary tasks. In parallel, we collect physiological data with a cheap device, extract features, and then construct machine learning models for cognitive load prediction. As metrics for the load we use one subjective measure, the NASA Task Load Index (NASA-TLX), and two objective measures: task difficulty and reaction time. The leave-one-subject-out evaluation shows a significant influence of the task type and the chosen cognitive load metric on the prediction accuracy.

Detection of the user's context with mobile sensing systems is a common problem in ubiquitous computing. However, the typically small battery of such systems is often making continuous detection impractical. The strain on the battery can be reduced if the sensor setting is adapted to each context. We propose a method that efficiently finds near-optimal sensor settings. It uses Markov chains to simulate the behaviour of the system in different configurations, and multi-objective genetic algorithm to find a set of good non-dominated configurations.

Wearable devices are heavily used in many sports. However, the existing sports wearables are either not tennis-specific, or are limited to information on shots. We therefore added tennis-specific information to a leading commercial device. Firstly, we developed a method for classifying shot types into forehand, backhand and serve. Secondly, we used multi-objective optimization to distinguish active play from the time in-between points. By combining both parts with the general movement information already provided by the device, we get comprehensive metrics for professional players and coaches to objectively measure a player's performance and enable in-depth tactical analysis.

Affect recognition is an important task in ubiquitous computing, in particular in health and human-computer interaction. In the former, it contributes to the timely detection and treatment of emotional and mental disorders, and in the latter, it enables indigenous interaction and enhanced user experience. We present an inter-domain study for affect recognition on seven different datasets, recorded with six different sensors, three different sensor placements, 211 subjects and nearly 1000 hours of labelled data. The datasets are processed and translated into a common spectro-temporal space. The data represented in the common spectro-temporal space is used to train a deep neural network (DNN) for arousal recognition that benefits from the large amounts of data even when the data are heterogeneous (i.e., different sensors and different datasets). The DNN approach outperforms the classical machine-learning approaches in six out of seven datasets.

Modern lifestyle is largely sedentary and often stressful, giving rise to extensive research and development of solutions for the management of these two lifestyle aspects. Physical activity monitoring is a mature area of ubiquitous computing, with many devices and mobile applications available on the market. Mental stress monitoring is still a hot research topic with few commercial solutions. This demo presents technology that goes beyond the state of the art in both areas, and powers a mobile application for lifestyle management.

Poor air quality and thermal comfort at the workplace affect the productivity, satisfaction and even health of employees, often without them being aware of the reason. This is a particularly problem in buildings without automated environmental controls, which are nowadays still in the majority. In this paper we present a system that uses an affordable and easy to install consumer weather station to monitor the temperature (T), humidity (H) and CO2 concentration (C). Based on these, it estimates the number of occupants in a room and whether the windows are opened or closed. It uses this information together with knowledge stored in an ontology to recommend actions that improve the environment quality. Experimental evaluation showed that the system objectively significantly improves the T-H-C parameters, and that the occupants consider its recommendations subjectively appropriate.

With modern technology and advanced models, it is possible to rather accurately anticipate the changes of weather parameters, such as temperature or precipitation, for a couple of days in advance. On the other hand, predicting dynamics of internal parameters, such as office spaces, can be tricky, as there are many variables that influence them, and we do not have information on their status. Being able to predict how parameters are changing would allow us to recommend appropriate actions to improve their work/living environment. In this paper, we present an anticipatory system, that is built from virtual sensors, which estimates number of occupants in the room and detects the state of the windows. Together with data from real sensors ontology, as coded knowledge, outputs reasonable actions that may improve internal environment. We built models, which anticipate dynamics of internal parameters temperature (T), humidity (H) and CO2 concentration (C), regarding all combinations of actions.

Continuous exposure to stress is harmful for mental and physical health, but to combat stress, one should first detect it. In this paper we propose a method for continuous detection of stressful events using data provided from a commercial wrist device. The method consists of three machine-learning components: a laboratory stress detector that detects short-term stress every 2 minutes; an activity recognizer that continuously recognizes user's activity and thus provides context information; and a context-based stress detector that exploits the output of the laboratory stress detector and the user's context in order to provide the final decision on 20 minutes interval. The method was evaluated in a laboratory and a real-life setting. The accuracy on 55 days of real-life data, for a 2-class problem, was 92%. The method is currently being integrated in a smartphone application for managing mental health and well-being.

Monitoring human energy expenditure is important in many health and sport applications, since the energy expenditure directly reflects the level of physical activity. The actual energy expenditure is unpractical to measure; hence, the field aims at estimating it by measuring the physical activity with accelerometers and other sensors. Current advanced estimators use a context-dependent approach in which a different regression model is invoked for different activities of the user. In this paper, we go a step further and use multiple contexts corresponding to multiple sensors, resulting in an ensemble of models for energy expenditure estimation. This provides a multi-view perspective, which leads to a better estimation of the energy. The proposed method was experimentally evaluated on a comprehensive set of activities where it outperformed the current state-of-the-art.