Search

Acoustic features extracted from speech can help with the diagnosis of neurological diseases and monitoring of symptoms over time. Temporal segmentation of audio signals into individual words is an important pre-processing step needed prior to extracting acoustic features. Machine learning techniques could be used to automate speech segmentation via automatic speech recognition (ASR) and sequence to sequence alignment. While state-of-the-art ASR models achieve good performance on healthy speech, their performance significantly drops when evaluated on dysarthric speech. Fine-tuning ASR models on impaired speech can improve performance in dysarthric individuals, but it requires representative clinical data, which is difficult to collect and may raise privacy concerns. This study explores the feasibility of using two augmentation methods to increase ASR performance on dysarthric speech: 1) healthy individuals varying their speaking rate and loudness (as is often used in assessments of pathological speech); 2) synthetic speech with variations in speaking rate and accent (to ensure more diverse vocal representations and fairness). Experimental evaluations showed that fine-tuning a pre-trained ASR model with data from these two sources outperformed a model fine-tuned only on real clinical data and matched the performance of a model fine-tuned on the combination of real clinical data and synthetic speech. When evaluated on held-out acoustic data from 24 individuals with various neurological diseases, the best performing model achieved an average word error rate of 5.7% and a mean correct count accuracy of 94.4%. In segmenting the data into individual words, a mean intersection-over-union of 89.2% was obtained against manual parsing (ground truth). It can be concluded that emulated and synthetic augmentations can significantly reduce the need for real clinical data of dysarthric speech when fine-tuning ASR models and, in turn, for speech segmentation., (© 2024 The Authors.)

Generalized joint hypermobility (GJH) often leads clinicians to suspect a diagnosis of Ehlers Danlos Syndrome (EDS), but it can be difficult to objectively assess. Video-based goniometry has been proposed to objectively estimate joint range of motion in hyperextended joints. As part of an exam of joint hypermobility at a specialized EDS clinic, a mobile phone was used to record short videos of 97 adults (89 female, 35.0 ± 9.9 years old) undergoing assessment of the elbows, knees, shoulders, ankles, and fifth fingers. Five body keypoint pose-estimation libraries (AlphaPose, Detectron, MediaPipe-Body, MoveNet - Thunder, OpenPose) and two hand keypoint pose-estimation libraries (AlphaPose, MediaPipe-Hands) were used to geometrically calculate the maximum angle of hyperextension or hyperflexion of each joint. A custom domain-specific model with a MobileNet-v2 backbone finetuned on data collected as part of this study was also evaluated for the fifth finger movement. Spearman's correlation was used to analyze the angles calculated from the tracked joint positions, the angles calculated from manually annotated keypoints, and the angles measured using a goniometer. Moderate correlations between the angles estimated using pose-tracked keypoints and the goniometer measurements were identified for the elbow (rho =.722; Detectron), knee (rho =.608; MoveNet - Thunder), shoulder (rho =.632; MoveNet - Thunder), and fifth finger (rho =.786; custom model) movements. The angles estimated from keypoints predicted by open-source libraries at the ankles were not significantly correlated with the goniometer measurements. Manually annotated angles at the elbows, knees, shoulders, and fifth fingers were moderately to strongly correlated to goniometer measurements but were weakly correlated for the ankles. There was not one pose-estimation library which performed best across all joints, so the library of choice must be selected separately for each joint of interest. This work evaluates several pose-estimation models as part of a vision-based system for estimating joint angles in individuals with suspected joint hypermobility. Future applications of the proposed system could facilitate objective assessment and screening of individuals referred to specialized EDS clinics., (© 2023 The Authors.)

Falls are a leading cause of morbidity and mortality in older adults with dementia residing in long-term care. Having access to a frequently updated and accurate estimate of the likelihood of a fall over a short time frame for each resident will enable care staff to provide targeted interventions to prevent falls and resulting injuries. To this end, machine learning models to estimate and frequently update the risk of a fall within the next 4 weeks were trained on longitudinal data from 54 older adult participants with dementia. Data from each participant included baseline clinical assessments of gait, mobility, and fall risk at the time of admission, daily medication intake in three medication categories, and frequent assessments of gait performed via a computer vision-based ambient monitoring system. Systematic ablations investigated the effects of various hyperparameters and feature sets and experimentally identified differential contributions from baseline clinical assessments, ambient gait analysis, and daily medication intake. In leave-one-subject-out cross-validation, the best performing model predicts the likelihood of a fall over the next 4 weeks with a sensitivity and specificity of 72.8 and 73.2, respectively, and achieved an area under the receiver operating characteristic curve (AUROC) of 76.2. By contrast, the best model excluding ambient gait features achieved an AUROC of 56.2 with a sensitivity and specificity of 51.9 and 54.0, respectively. Future research will focus on externally validating these findings to prepare for the implementation of this technology to reduce fall and fall-related injuries in long-term care.

Background: Parkinson's disease (PD) presents with motor symptoms such as bradykinesia, rigidity, and tremor that can affect gait. To monitor changes associated with disease progression or medication use, quantitative gait assessment is often performed during clinical visits. Conversely, vision-based solutions have been proposed for monitoring gait quality in non-clinical settings., Methods: We use three 2D human pose-estimation libraries (AlphaPose, Detectron, OpenPose) and one 3D library (ROMP) to calculate gait features from color video, and correlate them with those extracted by a Zeno instrumented walkway in older adults with PD. We calculate video-based gait features using a manual and automated heel-strike detection algorithm, and compare the correlations when the participants walk towards and away from the camera separately., Results: Based on analysis of 67 bidirectional walking bouts from 25 adults with PD, moderate to strong positive correlations were identified between the number of steps, cadence, as well as the mean and coefficient of variation of step width calculated from Zeno and video using 2D pose-estimation libraries. We noted that our automated heel-strike annotation method struggled to identify short steps., Conclusion: Gait features calculated from 2D joint trajectories are more strongly correlated with the Zeno than analogous gait features calculated from ROMP. Based on our analysis, videos processed with 2D pose-estimation libraries can be used for longitudinal gait monitoring in individuals with PD. Future work will seek to improve the prediction of gait features using a comprehensive machine learning model to predict gait features directly from color video without relying on intermediate extraction of joint trajectories.

Drug-induced parkinsonism affects many older adults with dementia, often causing gait disturbances. New advances in vision-based human pose- estimation have opened possibilities for frequent and unobtrusive analysis of gait in long-term care settings. This work leverages spatial-temporal graph convolutional network (ST-GCN) architectures and training procedures to predict clinical scores of parkinsonism in gait from video of individuals with dementia. We propose a two-stage training approach consisting of a self-supervised pretraining stage that encourages the ST-GCN model to learn about gait patterns before predicting clinical scores in the finetuning stage. The proposed ST-GCN models are evaluated on joint trajectories extracted from video and are compared against traditional (ordinal, linear, random forest) regression models and temporal convolutional network baselines. Three 2D human pose-estimation libraries (OpenPose, Detectron, AlphaPose) and the Microsoft Kinect (2D and 3D) are used to extract joint trajectories of 4787 natural walking bouts from 53 older adults with dementia. A subset of 399 walks from 14 participants is annotated with scores of parkinsonism severity on the gait criteria of the Unified Parkinson's Disease Rating Scale (UPDRS) and the Simpson-Angus Scale (SAS). Our results demonstrate that ST-GCN models operating on 3D joint trajectories extracted from the Kinect consistently outperform all other models and feature sets. Prediction of parkinsonism scores in natural walking bouts of unseen participants remains a challenging task, with the best models achieving macro-averaged F1-scores of 0.53 ± 0.03 and 0.40 ± 0.02 for UPDRS-gait and SAS-gait, respectively. Pre-trained model and demo code for this work is available. 1 .

Although pain is frequent in old age, older adults are often undertreated for pain. This is especially the case for long-term care residents with moderate to severe dementia who cannot report their pain because of cognitive impairments that accompany dementia. Nursing staff acknowledge the challenges of effectively recognizing and managing pain in long-term care facilities due to lack of human resources and, sometimes, expertise to use validated pain assessment approaches on a regular basis. Vision-based ambient monitoring will allow for frequent automated assessments so care staff could be automatically notified when signs of pain are displayed. However, existing computer vision techniques for pain detection are not validated on faces of older adults or people with dementia, and this population is not represented in existing facial expression datasets of pain. We present the first fully automated vision-based technique validated on a dementia cohort. Our contributions are threefold. First, we develop a deep learning-based computer vision system for detecting painful facial expressions on a video dataset that is collected unobtrusively from older adult participants with and without dementia. Second, we introduce a pairwise comparative inference method that calibrates to each person and is sensitive to changes in facial expression while using training data more efficiently than sequence models. Third, we introduce a fast contrastive training method that improves cross-dataset performance. Our pain estimation model outperforms baselines by a wide margin, especially when evaluated on faces of people with dementia. Pre-trained model and demo code available at https://github.com/TaatiTeam/pain_detection_demo.

We present the first public dataset with videos of oro-facial gestures performed by individuals with oro-facial impairment due to neurological disorders, such as amyotrophic lateral sclerosis (ALS) and stroke. Perceptual clinical scores from trained clinicians are provided as metadata. Manual annotation of facial landmarks is also provided for a subset of over 3300 frames. Through extensive experiments with multiple facial landmark detection algorithms, including state-of-the-art convolutional neural network (CNN) models, we demonstrated the presence of bias in the landmark localization accuracy of pre-trained face alignment approaches in our participant groups. The pre-trained models produced higher errors in the two clinical groups compared to age-matched healthy control subjects. We also investigated how this bias changes when the existing models are fine-tuned using data from the target population. The release of this dataset aims to propel the development of face alignment algorithms robust to the presence of oro-facial impairment, support the automatic analysis and recognition of oro-facial gestures, enhance the automatic identification of neurological diseases, as well as the estimation of disease severity from videos and images.

Fall risk is high for older adults with dementia. Gait impairment contributes to increased fall risk, and gait changes are common in people with dementia, although the reliable assessment of gait is challenging in this population. This study aimed to develop an automated approach to performing gait assessments based on gait data that is collected frequently and unobtrusively, and analysed using computer vision methods. Recent developments in computer vision have led to the availability of open source human pose estimation algorithms, which automatically estimate the joint locations of a person in an image. In this study, a pre-existing pose estimation model was applied to 1066 walking videos collected of 31 older adults with dementia as they walked naturally in a corridor on a specialized dementia unit over a two week period. Using the tracked pose information, gait features were extracted from video recordings of gait bouts and their association with clinical mobility assessment scores and future falls data was examined. A significant association was found between extracted gait features and a clinical mobility assessment and the number of future falls, providing concurrent and predictive validation of this approach.

A reliable, accessible, and non-intrusive method for tracking respiratory and heart rate is important for improving monitoring and detection of sleep apnea. In this study, an algorithm based on motion analysis of infrared video recordings was validated in 50 adults referred for clinical overnight polysomnography (PSG). The algorithm tracks the displacements of selected feature points on each sleeping participant and extracts respiratory rate using principal component analysis and heart rate using independent component analysis. For respiratory rate estimation (mean ± standard deviation), 89.89 % ± 10.95 % of the overnight estimation was accurate within 1 breath per minute compared to the PSG-derived respiratory rate from the respiratory inductive plethysmography signal, with an average root mean square error (RMSE) of 2.10 ± 1.64 breaths per minute. For heart rate estimation, 77.97 % ± 18.91 % of the overnight estimation was within 5 beats per minute of the heart rate derived from the pulse oximetry signal from PSG, with mean RMSE of 7.47 ± 4.79 beats per minute. No significant difference in estimation of RMSE of either signal was found according to differences in body position, sleep stage, or amount of the body covered by blankets. This vision-based method may prove suitable for overnight, non-contact monitoring of respiratory rate. However, at present, heart rate monitoring is less reliable and will require further work to improve accuracy., (2168-2372 © 2019 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.)

Robotic stroke rehabilitation therapy can greatly increase the efficiency of therapy delivery. However, when left unsupervised, users often compensate for limitations in affected muscles and joints by recruiting unaffected muscles and joints, leading to undesirable rehabilitation outcomes. This paper aims to develop a computer vision system that augments robotic stroke rehabilitation therapy by automatically detecting such compensatory motions. Nine stroke survivors and ten healthy adults participated in this study. All participants completed scripted motions using a table-top rehabilitation robot. The healthy participants also simulated three types of compensatory motions. The 3-D trajectories of upper body joint positions tracked over time were used for multiclass classification of postures. A support vector machine (SVM) classifier detected lean-forward compensation from healthy participants with excellent accuracy (AUC = 0.98, F1 = 0.82), followed by trunk-rotation compensation (AUC = 0.77, F1 = 0.57). Shoulder-elevation compensation was not well detected (AUC = 0.66, F1 = 0.07). A recurrent neural network (RNN) classifier, which encodes the temporal dependency of video frames, obtained similar results. In contrast, F1-scores in stroke survivors were low for all three compensations while using RNN: lean-forward compensation (AUC = 0.77, F1 = 0.17), trunk-rotation compensation (AUC = 0.81, F1 = 0.27), and shoulder-elevation compensation (AUC = 0.27, F1 = 0.07). The result was similar while using SVM. To improve detection accuracy for stroke survivors, future work should focus on predefining the range of motion, direct camera placement, delivering exercise intensity tantamount to that of real stroke therapies, adjusting seat height, and recording full therapy sessions.

This study applies mixture-model clustering to spatiotemporal gait parameters in order to characterize the pathological gait pattern and to generate a composite measure indicative of overall gait performance. Gait data from 68 adults with stroke (age: 61.5 ± 13.6 years) and 20 healthy adults (age: 28.8 ± 7.1 years) were used in this study. Participants performed three passes across a GAITRite mat at different time points following stroke (poststroke adults only). Mixture-model clustering grouped participants' gait patterns based on their spatiotemporal gait features including symmetry, speed, and variability. Mixture-models with different covariance matrix parameterizations and numbers of clusters were examined. The selected clustering model successfully categorized participants' spatiotemporal gait data into three clinically meaningful groups. Based on the clustering results, gait speed, and variability measures varied across the three groups. Individuals in Group 1 are all symmetric and had the fastest and lowest gait velocity and variability, respectively. As expected, healthy participants were assigned to Group 1. All gait parameters were at an intermediate level in Group 2 and worse condition in Group 3. Moreover, resulting cluster centers were in line with previously published clinical studies on gait. In addition to clustering, each individual was given an indexed membership (ranged 0-1) to each of three groups. These indexed memberships were proposed as a single measure to encompass information about multiple gait parameters (symmetry, speed, and variability) and as a measure that is sensitive and responsive to improvement or deterioration and rehabilitation over time.

Individuals with obstructive sleep apnea (OSA) can experience partial or complete collapse of the upper airway during sleep. This condition affects between 10-17% of adult men and 3-9% of adult women, requiring arousal to resume regular breathing. Frequent arousals disrupt proper sleeping patterns and cause daytime sleepiness. Untreated OSA has been linked to serious medical issues including cardiovascular disease and diabetes. Unfortunately, diagnosis rates are low (∼20%) and current sleep monitoring options are expensive, time consuming, and uncomfortable. Toward the development of a convenient, noncontact OSA monitoring system, this paper presents a simple, computer vision-based method to monitor cardiopulmonary signals (respiratory and heart rates) during sleep. System testing was performed with 17 healthy participants in five different simulated sleep positions. To monitor cardiopulmonary rates, distinctive points are automatically detected and tracked in infrared image sequences. Blind source separation is applied to extract candidate signals of interest. The optimal respiratory and heart rates are determined using periodicity measures based on spectral analysis. Estimates were validated by comparison to polysomnography recordings. The system achieved a mean percentage error of 3.4% and 5.0% for respiratory rate and heart rate, respectively. This study represents an important step in building an accessible, unobtrusive solution for sleep apnea diagnosis.

The identification of different stages of cognitive impairment can allow older adults to receive timely care and plan for the level of caregiving. People with existing diagnosis of cognitive impairment go through episodic phases of dementia requiring different levels of care at different times. Monitoring the cognitive status of existing patients is, thus, critical to deciding the level of care required by older adults. In this paper, we present a system to assess the cognitive status of older adults by monitoring a common activity of daily living, namely handwashing. Specifically, we extract features from handwashing trials of participants diagnosed with different levels of dementia ranging from cognitively intact to severe cognitive impairment, as assessed by the mini-mental state exam (MMSE). Based on videos of handwashing trials, we extract two classes of features: one characterizing the occupancy of different sink regions by the participant, and the other capturing the path tortuosity of the motion trajectory of participant's hands. We perform correlation analysis to assess univariate capacity of individual features to predict MMSE scores. To assess multivariate performance, we use machine learning methods to train models that predict the cognitive status (aware, mild, moderate, severe), as well as the MMSE scores. We present results demonstrating that features derived from hand washing behavior can be potential surrogate markers of a person's dementia, which can be instrumental in developing automated tools for continuously monitoring the cognitive status of older adults.

The current diagnosis process of dementia is resulting in a high percentage of cases with delayed detection. To address this problem, in this paper, we explore the feasibility of autonomously detecting mild cognitive impairment (MCI) in the older adult population. We implement a signal processing approach equipped with a machine learning paradigm to process and analyze real-world data acquired using home-based unobtrusive sensing technologies. Using the sensor and clinical data pertaining to 97 subjects, acquired over an average period of three years, a number of measures associated with the subjects' walking speed and general activity in the home were calculated. Different time spans of these measures were used to generate feature vectors to train and test two machine learning algorithms namely support vector machines and random forests. We were able to autonomously detect MCI in older adults with an area under the ROC curve of 0.97 and an area under the precision-recall curve of 0.93 using a time window of 24 weeks. This study is of great significance since it can potentially assist in the early detection of cognitive impairment in older adults.

Patients undergoing a bone marrow stem cell transplant (BMT) face various risk factors. Analyzing data from past transplants could enhance the understanding of the factors influencing success. Records up to 120 measurements per transplant procedure from 1751 patients undergoing BMT were collected (Shariati Hospital). Collaborative filtering techniques allowed the processing of highly sparse records with 22.3% missing values. Ten-fold cross-validation was used to evaluate the performance of various classification algorithms trained on predicting the survival status. Modest accuracy levels were obtained in predicting the survival status (AUC = 0.69). More importantly, however, operations that had the highest chances of success were shown to be identifiable with high accuracy, e.g., 92% or 97% when identifying 74 or 31 recipients, respectively. Identifying the patients with the highest chances of survival has direct application in the prioritization of resources and in donor matching. For patients where high-confidence prediction is not achieved, assigning a probability to their survival odds has potential applications in probabilistic decision support systems and in combination with other sources of information.