Search

This study aimed to validate the automated temporal analysis of countermovement vertical jump (CMJ) using MMPose, a markerless pose estimation framework, by comparing it with the gold-standard 3D marker-based motion capture system. Twelve participants performed five CMJ trials, which were simultaneously recorded using the marker-based system and two smartphone cameras capturing both sides of the body. Key kinematic points, including center of mass (CoM) and toe trajectories, were analyzed to determine jump phases and temporal variables. The agreement between methods was assessed using Bland–Altman analysis, root mean square error (RMSE), and Pearson’s correlation coefficient (r), while consistency was evaluated via intraclass correlation coefficient (ICC 3,1) and two-way repeated-measures ANOVA. Cohen’s effect size (d) quantified the practical significance of differences. Results showed strong agreement (r > 0.98) with minimal bias and narrow limits of agreement for most variables. The markerless system slightly overestimated jump height and CoM vertical velocity, but ICC values (ICC > 0.91) confirmed strong reliability. Cohen’s d values were near zero, indicating trivial differences, and no variability due to recording side was observed. Overall, MMPose proved to be a reliable alternative for in-field CMJ analysis, supporting its broader application in sports and rehabilitation settings.

Pathological gait in patients with Hakim’s disease (HD, synonymous with idiopathic normal-pressure hydrocephalus; iNPH), Parkinson’s disease (PD), and cervical myelopathy (CM) has been subjectively evaluated in this study. We quantified the characteristics of upper and lower limb movements in patients with pathological gait. We analyzed 1491 measurements of 1 m diameter circular walking from 122, 12, and 93 patients with HD, PD, and CM, respectively, and 200 healthy volunteers using the Three-Dimensional Pose Tracker for Gait Test. Upper and lower limb movements of 2D coordinates projected onto body axis sections were derived from estimated 3D relative coordinates. The hip and knee joint angle ranges on the sagittal plane were significantly smaller in the following order: healthy > CM > PD > HD, whereas the shoulder and elbow joint angle ranges were significantly smaller, as follows: healthy > CM > HD > PD. The outward shift of the leg on the axial plane was significantly greater, as follows: healthy < CM < PD < HD, whereas the outward shift of the upper limb followed the order of healthy > CM > HD > PD. The strongest correlation between the upper and lower limb movements was identified in the angle ranges of the hip and elbow joints on the sagittal plane. The lower and upper limb movements during circular walking were correlated. Patients with HD and PD exhibited reduced back-and-forth swings of the upper and lower limbs.

This study examined the efficacy of an optimized DeepLabCut (DLC) model in motion capture, with a particular focus on the sit-to-stand (STS) movement, which is crucial for assessing the functional capacity in elderly and postoperative patients. This research uniquely compared the performance of this optimized DLC model, which was trained using ’filtered’ estimates from the widely used OpenPose (OP) model, thereby emphasizing computational effectiveness, motion-tracking precision, and enhanced stability in data capture. Utilizing a combination of smartphone-captured videos and specifically curated datasets, our methodological approach included data preparation, keypoint annotation, and extensive model training, with an emphasis on the flow of the optimized model. The findings demonstrate the superiority of the optimized DLC model in various aspects. It exhibited not only higher computational efficiency, with reduced processing times, but also greater precision and consistency in motion tracking thanks to the stability brought about by the meticulous selection of the OP data. This precision is vital for developing accurate biomechanical models for clinical interventions. Moreover, this study revealed that the optimized DLC maintained higher average confidence levels across datasets, indicating more reliable and accurate detection capabilities compared with standalone OP. The clinical relevance of these findings is profound. The optimized DLC model’s efficiency and enhanced point estimation stability make it an invaluable tool in rehabilitation monitoring and patient assessments, potentially streamlining clinical workflows. This study suggests future research directions, including integrating the optimized DLC model with virtual reality environments for enhanced patient engagement and leveraging its improved data quality for predictive analytics in healthcare. Overall, the optimized DLC model emerged as a transformative tool for biomechanical analysis and physical rehabilitation, promising to enhance the quality of patient care and healthcare delivery efficiency.

This paper introduces a method for measuring 3D tibiofemoral kinematics using a multi-channel A-mode ultrasound system under dynamic conditions. The proposed system consists of a multi-channel A-mode ultrasound system integrated with a conventional motion capture system (i.e., optical tracking system). This approach allows for the non-invasive and non-radiative quantification of the tibiofemoral joint’s six degrees of freedom (DOF). We demonstrated the feasibility and accuracy of this method in the cadaveric experiment. The knee joint’s motions were mimicked by manually manipulating the leg through multiple motion cycles from flexion to extension. To measure it, six custom ultrasound holders, equipped with a total of 30 A-mode ultrasound transducers and 18 optical markers, were mounted on various anatomical regions of the lower extremity of the specimen. During experiments, 3D-tracked intra-cortical bone pins were inserted into the femur and tibia to measure the ground truth of tibiofemoral kinematics. The results were compared with the tibiofemoral kinematics derived from the proposed ultrasound system. The results showed an average rotational error of 1.51 ± 1.13° and a translational error of 3.14 ± 1.72 mm for the ultrasound-derived kinematics, compared to the ground truth. In conclusion, this multi-channel A-mode ultrasound system demonstrated a great potential of effectively measuring tibiofemoral kinematics during dynamic motions. Its improved accuracy, nature of non-invasiveness, and lack of radiation exposure make this method a promising alternative to incorporate into gait analysis and prosthetic kinematic measurements later.

Archery ranks among the sports with a high incidence of upper extremity injuries, particularly affecting the drawing shoulder and elbow, as well as inducing stress on the lower back. This study seeks to bridge the gap by integrating real-time human motion with biomechanical software to enhance the ergonomics of archers. Thirteen participants were involved in four tasks, using different bows with varied draw weights and shooting distances. Through the application of advanced integrative technology, this study highlights the distinct postures adopted by both males and females, which indicate the biomechanical differences between genders. Additionally, an analysis of the correlation between exposed spinal forces and these adopted postures provides insights into injury risk assessment during the key archery movements. The findings of this study have the potential to significantly enhance the application of training methodologies and the design of assistive devices. These improvements are geared towards mitigating injury risks and enhancing the overall performance of archers.

Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder known for its significant heterogeneity and varied symptom presentation. Describing the different subtypes as predominantly inattentive (ADHD–I), combined (ADHD–C), and hyperactive–impulsive (ADHD–H) relies primarily on clinical observations, which can be subjective. To address the need for more objective diagnostic methods, this pilot study implemented a Microsoft Kinect-based Stroop Color–Word Test (KSWCT) with the objective of investigating the potential differences in executive function and motor control between different subtypes in a group of children and adolescents with ADHD. A series of linear mixture modeling were used to encompass the performance accuracy, reaction times, and extraneous movements during the tests. Our findings suggested that age plays a critical role, and older subjects showed improvements in KSWCT performance; however, no significant divergence in activity level between the subtypes (ADHD–I and ADHD–H/C) was established. Patients with ADHD–H/C showed tendencies toward deficits in motor planning and executive control, exhibited by shorter reaction times for incorrect responses and more difficulty suppressing erroneous responses. This study provides preliminary evidence of unique executive characteristics among ADHD subtypes, advances our understanding of the heterogeneity of the disorder, and lays the foundation for the development of refined and objective diagnostic tools for ADHD.

The timed up and go test (TUG) is a common clinical functional balance test often used to complement findings on sensorimotor changes due to aging or sensory/motor dysfunction. The instrumented TUG can be used to obtain objective postural and gait measures that are more sensitive to mobility changes. We investigated whether gait and body coordination during TUG is representative of walking. We examined the walking phase of the TUG and compared gait metrics (stride duration and length, walking speed, and step frequency) and head/trunk accelerations to normal walking. The latter is a key aspect of postural control and can also reveal changes in sensory and motor function. Forty participants were recruited into three groups: young adults, older adults, and older adults with visual impairment. All performed the TUG and a short walking task wearing ultra-lightweight wireless IMUs on the head, chest, and right ankle. Gait and head/trunk acceleration metrics were comparable across tasks. Further, stride length and walking speed were correlated with the participants’ age. Those with visual impairment walked significantly slower than sighted older adults. We suggest that the TUG can be a valuable tool for examining gait and stability during walking without the added time or space constraints.

Distinguishing pathological gait is challenging in neurology because of the difficulty of capturing total body movement and its analysis. We aimed to obtain a convenient recording with an iPhone and establish an algorithm based on deep learning. From May 2021 to November 2022 at Yamagata University Hospital, Shiga University, and Takahata Town, patients with idiopathic normal pressure hydrocephalus (n = 48), Parkinson’s disease (n = 21), and other neuromuscular diseases (n = 45) comprised the pathological gait group (n = 114), and the control group consisted of 160 healthy volunteers. iPhone application TDPT-GT captured the subjects walking in a circular path of about 1 meter in diameter, a markerless motion capture system, with an iPhone camera, which generated the three-axis 30 frames per second (fps) relative coordinates of 27 body points. A light gradient boosting machine (Light GBM) with stratified k-fold cross-validation (k = 5) was applied for gait collection for about 1 min per person. The median ability model tested 200 frames of each person’s data for its distinction capability, which resulted in the area under a curve of 0.719. The pathological gait captured by the iPhone could be distinguished by artificial intelligence.

Biomechanical simulation allows for in silico estimations of biomechanical parameters such as muscle, joint and ligament forces. Experimental kinematic measurements are a prerequisite for musculoskeletal simulations using the inverse kinematics approach. Marker-based optical motion capture systems are frequently used to collect this motion data. As an alternative, IMU-based motion capture systems can be used. These systems allow flexible motion collection without nearly any restriction regarding the environment. However, one limitation with these systems is that there is no universal way to transfer IMU data from arbitrary full-body IMU measurement systems into musculoskeletal simulation software such as OpenSim. Thus, the objective of this study was to enable the transfer of collected motion data, stored as a BVH file, to OpenSim 4.4 to visualize and analyse the motion using musculoskeletal models. By using the concept of virtual markers, the motion saved in the BVH file is transferred to a musculoskeletal model. An experimental study with three participants was conducted to verify our method’s performance. Results show that the present method is capable of (1) transferring body dimensions saved in the BVH file to a generic musculoskeletal model and (2) correctly transferring the motion data saved in the BVH file to a musculoskeletal model in OpenSim 4.4.

The smooth movement of hand/surgical instruments is considered an indicator of skilled, coordinated surgical performance. Jerky surgical instrument movements or hand tremors can cause unwanted damages to the surgical site. Different methods have been used in previous studies for assessing motion smoothness, causing conflicting results regarding the comparison among surgical skill levels. We recruited four attending surgeons, five surgical residents, and nine novices. The participants conducted three simulated laparoscopic tasks, including peg transfer, bimanual peg transfer, and rubber band translocation. Tooltip motion smoothness was computed using the mean tooltip motion jerk, logarithmic dimensionless tooltip motion jerk, and 95% tooltip motion frequency (originally proposed in this study) to evaluate their capability of surgical skill level differentiation. The results revealed that logarithmic dimensionless motion jerk and 95% motion frequency were capable of distinguishing skill levels, indicated by smoother tooltip movements observed in high compared to low skill levels. Contrarily, mean motion jerk was not able to distinguish the skill levels. Additionally, 95% motion frequency was less affected by the measurement noise since it did not require the calculation of motion jerk, and 95% motion frequency and logarithmic dimensionless motion jerk yielded a better motion smoothness assessment outcome in distinguishing skill levels than mean motion jerk.

To assess pathological gaits quantitatively, three-dimensional coordinates estimated with a deep learning model were converted into body axis plane projections. First, 15 healthy volunteers performed four gait patterns; that is, normal, shuffling, short-stepped, and wide-based gaits, with the Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) application. Second, gaits of 47 patients with idiopathic normal pressure hydrocephalus (iNPH) and 92 healthy elderly individuals in the Takahata cohort were assessed with the TDPT-GT. Two-dimensional relative coordinates were calculated from the three-dimensional coordinates by projecting the sagittal, coronal, and axial planes. Indices of the two-dimensional relative coordinates associated with a pathological gait were comprehensively explored. The candidate indices for the shuffling gait were the angle range of the hip joint < 30° and relative vertical amplitude of the heel < 0.1 on the sagittal projection plane. For the short-stepped gait, the angle range of the knee joint < 45° on the sagittal projection plane was a candidate index. The candidate index for the wide-based gait was the leg outward shift > 0.1 on the axial projection plane. In conclusion, the two-dimensional coordinates on the body axis projection planes calculated from the 3D relative coordinates estimated by the TDPT-GT application enabled the quantification of pathological gait features.

Human movement tracking is useful in a variety of areas, such as search-and-rescue activities. CCTV and IP cameras are popular as front-end sensors for tracking human motion; however, they are stationary and have limited applicability in hard-to-reach places, such as those where disasters have occurred. Using a drone to discover a person is challenging and requires an innovative approach. In this paper, we aim to present the design and implementation of a human motion tracking method using a Tello EDU drone. The design methodology is carried out in four steps: (1) control panel design; (2) human motion tracking algorithm; (3) notification systems; and (4) communication and distance extension. Intensive experimental results show that the drone implemented by the proposed algorithm performs well in tracking a human at a distance of 2–10 m moving at a speed of 2 m/s. In an experimental field of the size 95×35m2, the drone tracked human motion throughout a whole day, with the best tracking results observed in the morning. The drone was controlled from a laptop using a Wi-Fi router with a maximum horizontal tracking distance of 84.30 m and maximum vertical distance of 13.40 m. The experiment showed an accuracy rate for human movement detection between 96.67 and 100%.

To quantitatively assess pathological gait, we developed a novel smartphone application for full-body human motion tracking in real time from markerless video-based images using a smartphone monocular camera and deep learning. As training data for deep learning, the original three-dimensional (3D) dataset comprising more than 1 million captured images from the 3D motion of 90 humanoid characters and the two-dimensional dataset of COCO 2017 were prepared. The 3D heatmap offset data consisting of 28 × 28 × 28 blocks with three red–green–blue colors at the 24 key points of the entire body motion were learned using the convolutional neural network, modified ResNet34. At each key point, the hottest spot deviating from the center of the cell was learned using the tanh function. Our new iOS application could detect the relative tri-axial coordinates of the 24 whole-body key points centered on the navel in real time without any markers for motion capture. By using the relative coordinates, the 3D angles of the neck, lumbar, bilateral hip, knee, and ankle joints were estimated. Any human motion could be quantitatively and easily assessed using a new smartphone application named Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) without any body markers or multipoint cameras.

The paper deals with a lead-through method of programming for industrial robots. The goal is to automatically reproduce 6DoF trajectories of a tool wielded by a human operator demonstrating a motion task. We present a novel motion-tracking system built around the HTC Vive pose estimation system. Our solution allows complete automation of the robot teaching process. Specific algorithmic issues of system calibration and motion data post-processing are also discussed, constituting the paper’s theoretical contribution. The motion tracking system is successfully deployed in a pilot application of robot-assisted spray painting.

We introduce a set of input models for fusing information from ensembles of wearable sensors supporting human performance and telemedicine. Veracity is demonstrated in action classification related to sport, specifically strikes in boxing and taekwondo. Four input models, formulated to be compatible with a broad range of classifiers, are introduced and two diverse classifiers, dynamic time warping (DTW) and convolutional neural networks (CNNs) are implemented in conjunction with the input models. Seven classification models fusing information at the input-level, output-level, and a combination of both are formulated. Action classification for 18 boxing punches and 24 taekwondo kicks demonstrate our fusion classifiers outperform the best DTW and CNN uni-axial classifiers. Furthermore, although DTW is ostensibly an ideal choice for human movements experiencing non-linear variations, our results demonstrate deep learning fusion classifiers outperform DTW. This is a novel finding given that CNNs are normally designed for multi-dimensional data and do not specifically compensate for non-linear variations within signal classes. The generalized formulation enables subject-specific movement classification in a feature-blind fashion with trivial computational expense for trained CNNs. A commercial boxing system, ‘Corner’, has been produced for real-world mass-market use based on this investigation providing a basis for future telemedicine translation.

Magnetoelectric (ME) sensors with a form factor of a few millimeters offer a comparatively low magnetic noise density of a few pT/Hz in a narrow frequency band near the first bending mode. While a high resonance frequency (kHz range) and limited bandwidth present a challenge to biomagnetic measurements, they can potentially be exploited in indirect sensing of non-magnetic quantities, where artificial magnetic sources are applicable. In this paper, we present the novel concept of an active magnetic motion sensing system optimized for ME sensors. Based on the signal chain, we investigated and quantified key drivers of the signal-to-noise ratio (SNR), which is closely related to sensor noise and bandwidth. These considerations were demonstrated by corresponding measurements in a simplified one-dimensional motion setup. Accordingly, we introduced a customized filter structure that enables a flexible bandwidth selection as well as a frequency-based separation of multiple artificial sources. Both design goals target the prospective application of ME sensors in medical movement analysis, where a multitude of distributed sensors and sources might be applied.

This paper presents a wearable motion tracking system with recording and playback features. This system has been designed for gait analysis and interlimb coordination studies. It can be implemented to help reduce fall risk and to retrain gait in a rehabilitation setting. Our system consists of ten custom wearable straps, a receiver, and a central computer. Comparing with similar existing solutions, the proposed system is affordable and convenient, which can be used in both indoor and outdoor settings. In the experiment, the system calculates five gait parameters and has the potential to identify deviant gait patterns. The system can track upper body parameters such as arm swing, which has potential in the study of pathological gaits and the coordination of the limbs.

Martial arts has many benefits not only in self-defence, but also in improving physical fitness and mental well-being. In our research we focused on analyzing the velocity, impulse, momentum and impact force of the Taekwondo sine-wave punch and reverse-step punch. We evaluated these techniques in comparison with the martial arts styles of Hapkido and Shaolin Wushu and investigated the kinematic properties. We developed a sensing system which is composed of an ICSensor Model 3140 accelerometer attached to a punching bag for measuring dynamic acceleration, Kinovea motion analysis software and 2 GoPro Hero 3 cameras, one focused on the practitioner’s motion and the other focused on the punching bag’s motion. Our results verified that the motion vectors associated with a Taekwondo practitioner performing a sine-wave punch, uses a unique gravitational potential energy to optimise the impact force of the punch. We demonstrated that the sine-wave punch on average produced an impact force of 6884 N which was higher than the reverse-step punch that produced an average impact force of 5055 N. Our comparison experiment showed that the Taekwondo sine-wave punch produced the highest impact force compared to a Hapkido right cross punch and a Shaolin Wushu right cross, however the Wushu right cross had the highest force to weight ratio at 82:1. The experiments were conducted with high ranking black belt practitioners in Taekwondo, Hapkido and Shaolin Wushu.

Background: The recovery of upper limb mobility and functions is essential for people with cervical spinal cord injuries (cSCI) to maximize independence in daily activities and ensure a successful return to normality. The rehabilitative path should include a thorough neuromotor evaluation and personalized treatments aimed at recovering motor functions. Body-machine interfaces (BoMI) have been proven to be capable of harnessing residual joint motions to control objects like computer cursors and virtual or physical wheelchairs and to promote motor recovery. However, their therapeutic application has still been limited to shoulder movements. Here, we expanded the use of BoMI to promote the whole arm’s mobility, with a special focus on elbow movements. We also developed an instrumented evaluation test and a set of kinematic indicators for assessing residual abilities and recovery. Methods: Five inpatient cSCI subjects (four acute, one chronic) participated in a BoMI treatment complementary to their standard rehabilitative routine. The subjects wore a BoMI with sensors placed on both proximal and distal arm districts and practiced for 5 weeks. The BoMI was programmed to promote symmetry between right and left arms use and the forearms’ mobility while playing games. To evaluate the effectiveness of the treatment, the subjects’ kinematics were recorded while performing an evaluation test that involved functional bilateral arms movements, before, at the end, and three months after training. Results: At the end of the training, all subjects learned to efficiently use the interface despite being compelled by it to engage their most impaired movements. The subjects completed the training with bilateral symmetry in body recruitment, already present at the end of the familiarization, and they increased the forearm activity. The instrumental evaluation confirmed this. The elbow motion’s angular amplitude improved for all subjects, and other kinematic parameters showed a trend towards the normality range. Conclusion: The outcomes are preliminary evidence supporting the efficacy of the proposed BoMI as a rehabilitation tool to be considered for clinical practice. It also suggests an instrumental evaluation protocol and a set of indicators to assess and evaluate motor impairment and recovery in cSCI.

Residual motion of upper limbs in individuals who experienced cervical spinal cord injury (CSCI) is vital to achieve functional independence. Several interventions were developed to restore shoulder range of motion (ROM) in CSCI patients. However, shoulder ROM assessment in clinical practice is commonly limited to use of a simple goniometer. Conventional goniometric measurements are operator-dependent and require significant time and effort. Therefore, innovative technology for supporting medical personnel in objectively and reliably measuring the efficacy of treatments for shoulder ROM in CSCI patients would be extremely desirable. This study evaluated the validity of a customized wireless wearable sensors (Inertial Measurement Units—IMUs) system for shoulder ROM assessment in CSCI patients in clinical setting. Eight CSCI patients and eight healthy controls performed four shoulder movements (forward flexion, abduction, and internal and external rotation) with dominant arm. Every movement was evaluated with a goniometer by different testers and with the IMU system at the same time. Validity was evaluated by comparing IMUs and goniometer measurements using Intraclass Correlation Coefficient (ICC) and Limits of Agreement (LOA). inter-tester reliability of IMUs and goniometer measurements was also investigated. Preliminary results provide essential information on the accuracy of the proposed wireless wearable sensors system in acquiring objective measurements of the shoulder movements in CSCI patients.

Laparoscopic surgery demands highly skilled surgeons. Traditionally, a surgeon’s knowledge is acquired by operating under a mentor-trainee method. In recent years, laparoscopic simulators have gained ground as tools in skill acquisition. Despite the wide range of laparoscopic simulators available, few provide objective feedback to the trainee. Those systems with quantitative feedback tend to be high-end solutions with limited availability due to cost. A modular smart trainer was developed, combining tool-tracking and force-using employing commercially available sensors. Additionally, a force training system based on polydimethylsiloxane (PDMS) phantoms for sample stiffness differentiation is presented. This prototype was tested with 39 subjects, between novices (13), intermediates (13), and experts (13), evaluating execution differences among groups in training exercises. The estimated cost is USD $200 (components only), not including laparoscopic instruments. The motion system was tested for noise reduction and position validation with a mean error of 0.94 mm. Grasping force approximation showed a correlation of 0.9975. Furthermore, differences in phantoms stiffness effectively reflected user manipulation. Subject groups showed significant differences in execution time, accumulated distance, and mean and maximum applied grasping force. Accurate information was obtained regarding motion and force. The developed force-sensing tool can easily be transferred to a clinical setting. Further work will consist on a validation of the simulator on a wider range of tasks and a larger sample of volunteers.

Motion tracking techniques have been extensively studied in recent years. However, capturing movements of the upper limbs is a challenging task. This document presents the estimation of arm orientation and elbow and wrist position using wearable flexible sensors (WFSs). A study was developed to obtain the highest range of motion (ROM) of the shoulder with as few sensors as possible, and a method for estimating arm length and a calibration procedure was proposed. Performance was verified by comparing measurement of the shoulder joint angles obtained from commercial two-axis soft angular displacement sensors (sADS) from Bend Labs and from the ground truth system (GTS) OptiTrack. The global root-mean-square error (RMSE) for the shoulder angle is 2.93 degrees and 37.5 mm for the position estimation of the wrist in cyclical movements; this measure of RMSE was improved to 13.6 mm by implementing a gesture classifier.

The rapid technological advancements of Industry 4.0 have opened up new vectors for novel industrial processes that require advanced sensing solutions for their realization. Motion capture (MoCap) sensors, such as visual cameras and inertial measurement units (IMUs), are frequently adopted in industrial settings to support solutions in robotics, additive manufacturing, teleworking and human safety. This review synthesizes and evaluates studies investigating the use of MoCap technologies in industry-related research. A search was performed in the Embase, Scopus, Web of Science and Google Scholar. Only studies in English, from 2015 onwards, on primary and secondary industrial applications were considered. The quality of the articles was appraised with the AXIS tool. Studies were categorized based on type of used sensors, beneficiary industry sector, and type of application. Study characteristics, key methods and findings were also summarized. In total, 1682 records were identified, and 59 were included in this review. Twenty-one and 38 studies were assessed as being prone to medium and low risks of bias, respectively. Camera-based sensors and IMUs were used in 40% and 70% of the studies, respectively. Construction (30.5%), robotics (15.3%) and automotive (10.2%) were the most researched industry sectors, whilst health and safety (64.4%) and the improvement of industrial processes or products (17%) were the most targeted applications. Inertial sensors were the first choice for industrial MoCap applications. Camera-based MoCap systems performed better in robotic applications, but camera obstructions caused by workers and machinery was the most challenging issue. Advancements in machine learning algorithms have been shown to increase the capabilities of MoCap systems in applications such as activity and fatigue detection as well as tool condition monitoring and object recognition.

In 3D motion capture, multiple methods have been developed in order to optimize the quality of the captured data. While certain technologies, such as inertial measurement units (IMU), are mostly suitable for 3D orientation estimation at relatively high frequencies, other technologies, such as marker-based motion capture, are more suitable for 3D position estimations at a lower frequency range. In this work, we introduce a complementary filter that complements 3D motion capture data with high-frequency acceleration signals from an IMU. While the local optimization reduces the error of the motion tracking, the additional accelerations can help to detect micro-motions that are useful when dealing with high-frequency human motions or robotic applications. The combination of high-frequency accelerometers improves the accuracy of the data and helps to overcome limitations in motion capture when micro-motions are not traceable with 3D motion tracking system. In our experimental evaluation, we demonstrate the improvements of the motion capture results during translational, rotational, and combined movements.

The recognition of human activities is usually considered to be a simple procedure. Problems occur in complex scenes involving high speeds. Activity prediction using Artificial Intelligence (AI) by numerical analysis has attracted the attention of several researchers. Human activities are an important challenge in various fields. There are many great applications in this area, including smart homes, assistive robotics, human–computer interactions, and improvements in protection in several areas such as security, transport, education, and medicine through the control of falling or aiding in medication consumption for elderly people. The advanced enhancement and success of deep learning techniques in various computer vision applications encourage the use of these methods in video processing. The human presentation is an important challenge in the analysis of human behavior through activity. A person in a video sequence can be described by their motion, skeleton, and/or spatial characteristics. In this paper, we present a novel approach to human activity recognition from videos using the Recurrent Neural Network (RNN) for activity classification and the Convolutional Neural Network (CNN) with a new structure of the human skeleton to carry out feature presentation. The aims of this work are to improve the human presentation through the collection of different features and the exploitation of the new RNN structure for activities. The performance of the proposed approach is evaluated by the RGB-D sensor dataset CAD-60. The experimental results show the performance of the proposed approach through the average error rate obtained (4.5%).

Internet of Things is advancing, and the augmented role of smart navigation in automating processes is at its vanguard. Smart navigation and location tracking systems are finding increasing use in the area of the mission-critical indoor scenario, logistics, medicine, and security. A demanding emerging area is an Indoor Localization due to the increased fascination towards location-based services. Numerous inertial assessments unit-based indoor localization mechanisms have been suggested in this regard. However, these methods have many shortcomings pertaining to accuracy and consistency. In this study, we propose a novel position estimation system based on learning to the prediction model to address the above challenges. The designed system consists of two modules; learning to prediction module and position estimation using sensor fusion in an indoor environment. The prediction algorithm is attached to the learning module. Moreover, the learning module continuously controls, observes, and enhances the efficiency of the prediction algorithm by evaluating the output and taking into account the exogenous factors that may have an impact on its outcome. On top of that, we reckon a situation where the prediction algorithm can be applied to anticipate the accurate gyroscope and accelerometer reading from the noisy sensor readings. In the designed system, we consider a scenario where the learning module, based on Artificial Neural Network, and Kalman filter are used as a prediction algorithm to predict the actual accelerometer and gyroscope reading from the noisy sensor reading. Moreover, to acquire data, we use the next-generation inertial measurement unit, which contains a 3-axis accelerometer and gyroscope data. Finally, for the performance and accuracy of the proposed system, we carried out numbers of experiments, and we observed that the proposed Kalman filter with learning module performed better than the traditional Kalman filter algorithm in terms of root mean square error metric.

The assessment of risks due to biomechanical overload in manual material handling is nowadays mainly based on observational methods in which an expert rater visually inspects videos of the working activity. Currently available sensing wearable technologies for motion and muscular activity capture enables to advance the risk assessment by providing reliable, repeatable, and objective measures. However, existing solutions do not address either a full body assessment or the inclusion of measures for the evaluation of the effort. This article proposes a novel system for the assessment of biomechanical overload, capable of covering all areas of ISO 11228, that uses a sensor network composed of inertial measurement units (IMU) and electromyography (EMG) sensors. The proposed method is capable of gathering and processing data from three IMU-based motion capture systems and two EMG capture devices. Data are processed to provide both segmentation of the activity and ergonomic risk score according to the methods reported in the ISO 11228 and the TR 12295. The system has been tested on a challenging outdoor scenario such as lift-on/lift-off of containers on a cargo ship. A comparison of the traditional evaluation method and the proposed one shows the consistency of the proposed system, its time effectiveness, and its potential for deeper analyses that include intra-subject and inter-subjects variability as well as a quantitative biomechanical analysis.

Pedestrian Dead Reckoning (PDR) uses inertial measurement units (IMUs) and combines velocity and orientation estimates to determine a position. The estimation of the velocity is still challenging, as the integration of noisy acceleration and angular speed signals over a long period of time causes large drifts. Classic approaches to estimate the velocity optimize for specific applications, sensor positions, and types of movement and require extensive parameter tuning. Our novel hybrid filter combines a convolutional neural network (CNN) and a bidirectional recurrent neural network (BLSTM) (that extract spatial features from the sensor signals and track their temporal relationships) with a linear Kalman filter (LKF) that improves the velocity estimates. Our experiments show the robustness against different movement states and changes in orientation, even in highly dynamic situations. We compare the new architecture with conventional, machine, and deep learning methods and show that from a single non-calibrated IMU, our novel architecture outperforms the state-of-the-art in terms of velocity (≤0.16 m/s) and traveled distance (≤3 m/km). It also generalizes well to different and varying movement speeds and provides accurate and precise velocity estimates.

Computationally efficient 3D orientation (3DO) tracking using gyroscope angular velocity measurements enables a short execution time and low energy consumption for the computing device. These are essential requirements in today’s wearable device environments, which are characterized by limited resources and demands for high energy autonomy. We show that the computational efficiency of 3DO tracking is significantly improved by correctly interpreting each triplet of gyroscope measurements as simultaneous (using the rotation vector called the Simultaneous Orthogonal Rotation Angle, or SORA) rather than as sequential (using Euler angles) rotation. For an example rotation of 90°, depending on the change in the rotation axis, using Euler angles requires 35 to 78 times more measurement steps for comparable levels of accuracy, implying a higher sampling frequency and computational complexity. In general, the higher the demanded 3DO accuracy, the higher the computational advantage of using the SORA. Furthermore, we demonstrate that 12 to 14 times faster execution is achieved by adapting the SORA-based 3DO tracking to the architecture of the executing low-power ARM Cortex® M0+ microcontroller using only integer arithmetic, lookup tables, and the small-angle approximation. Finally, we show that the computational efficiency is further improved by choosing the appropriate 3DO computational method. Using rotation matrices is 1.85 times faster than using rotation quaternions when 3DO calculations are performed for each measurement step. On the other hand, using rotation quaternions is 1.75 times faster when only the final 3DO result of several consecutive rotations is needed. We conclude that by adopting the presented practices, the clock frequency of a processor computing the 3DO can be significantly reduced. This substantially prolongs the energy autonomy of the device and enhances its usability in day-to-day measurement scenarios.

Sensors that remotely track the displacement of a moving object have a wide range of applications from robotic control to motion capture. In this paper, we introduce a simple, small silicon integrated circuit sensor that tracks the angular displacement of an object tagged with a small light source, such as a light-emitting diode (LED). This sensor uses a new angular transduction mechanism, differential diffusion of photoelectrons generated from the light spot cast by the light tag onto a Si anode, that is described by a simple physics model using pinhole optics and carrier diffusion. Because the light spot is formed by a pinhole aperture integrated on the sensor chip, no external focusing optics are needed, reducing system complexity, size, and weight. Prototype sensors based on this model were fabricated and their basic characteristics are presented. These sensors transduce angular displacement of an LED across orthogonal latitudinal and longitudinal arcs into normalized differential photocathode currents with signal linearly proportional to LED angular position across a ± 40° field-of-view. These sensors offer potential performance and ease-of-use benefits compared to existing displacement sensor technologies.

Endoscopic autofluorescence lifetime imaging is a promising technique for making quantitative and non-invasive diagnoses of abnormal tissue. However, motion artifacts caused by vibration in the direction perpendicular to the tissue surface in a body makes clinical diagnosis difficult. Thus, this paper proposes a robust autofluorescence lifetime sensing technique with a lens tracking system based on a laser beam spot analysis. Our optical setup can be easily mounted on the head of an endoscope. The variation in distance between the optical system and the target surface is tracked by the change in the spot size of the laser beam captured by the camera, and the lens actuator is feedback-controlled to suppress motion artifacts. The experimental results show that, when using a lens tracking system, the standard deviation of fluorescence lifetime is dramatically reduced. Furthermore, the validity of the proposed method is experimentally confirmed by using a bio-mimicking phantom that replicates the shape, optical parameters, and chemical component distribution of the cancerous tissue.

Wireless sensor networks are used in a wide range of applications, many of which require real-time transmission of the measurements. Bandwidth limitations result in limitations on the sampling frequency and number of sensors. This problem can be addressed by reducing the communication load via data compression and event-based communication approaches. The present paper focuses on the class of applications in which the signals exhibit unknown and potentially time-varying cyclic patterns. We review recently proposed event-triggered learning (ETL) methods that identify and exploit these cyclic patterns, we show how these methods can be applied to the nonlinear multivariable dynamics of three-dimensional orientation data, and we propose a novel approach that uses Gaussian process models. In contrast to other approaches, all three ETL methods work in real time and assure a small upper bound on the reconstruction error. The proposed methods are compared to several conventional approaches in experimental data from human subjects walking with a wearable inertial sensor network. They are found to reduce the communication load by 60−70%, which implies that two to three times more sensor nodes could be used at the same bandwidth.

In this paper, we present a method for finding the enhanced heading and position of pedestrians by fusing the Zero velocity UPdaTe (ZUPT)-based pedestrian dead reckoning (PDR) and the kinematic constraints of the lower human body. ZUPT is a well known algorithm for PDR, and provides a sufficiently accurate position solution for short term periods, but it cannot guarantee a stable and reliable heading because it suffers from magnetic disturbance in determining heading angles, which degrades the overall position accuracy as time passes. The basic idea of the proposed algorithm is integrating the left and right foot positions obtained by ZUPTs with the heading and position information from an IMU mounted on the waist. To integrate this information, a kinematic model of the lower human body, which is calculated by using orientation sensors mounted on both thighs and calves, is adopted. We note that the position of the left and right feet cannot be apart because of the kinematic constraints of the body, so the kinematic model generates new measurements for the waist position. The Extended Kalman Filter (EKF) on the waist data that estimates and corrects error states uses these measurements and magnetic heading measurements, which enhances the heading accuracy. The updated position information is fed into the foot mounted sensors, and reupdate processes are performed to correct the position error of each foot. The proposed update-reupdate technique consequently ensures improved observability of error states and position accuracy. Moreover, the proposed method provides all the information about the lower human body, so that it can be applied more effectively to motion tracking. The effectiveness of the proposed algorithm is verified via experimental results, which show that a 1.25% Return Position Error (RPE) with respect to walking distance is achieved.

An innovative insect-based visual sensor is designed to perform active marker tracking. Without any optics and a field-of-view of about 60°, a novel miniature visual sensor is able to locate flickering markers (LEDs) with an accuracy much greater than the one dictated by the pixel pitch. With a size of only 1 cm3 and a mass of only 0.33 g, the lensless sensor, called HyperCube, is dedicated to 3D motion tracking and fits perfectly with the drastic constraints imposed by micro-aerial vehicles. Only three photosensors are placed on each side of the cubic configuration of the sensing device, making this sensor very inexpensive and light. HyperCube provides the azimuth and elevation of infrared LEDs flickering at a high frequency (>1 kHz) with a precision of 0.5°. The minimalistic design in terms of small size, low mass and low power consumption of this visual sensor makes it suitable for many applications in the field of the cooperative flight of unmanned aerial vehicles and, more generally, robotic applications requiring active beacons. Experimental results show that HyperCube provides useful angular measurements that can be used to estimate the relative position between the sensor and the flickering infrared markers.

Motion tracking systems are commonly used in virtual reality-based interventions to detect movements in the real world and transfer them to the virtual environment. There are different tracking solutions based on different physical principles, which mainly define their performance parameters. However, special requirements have to be considered for rehabilitation purposes. This paper studies and compares the accuracy and jitter of three tracking solutions (optical, electromagnetic, and skeleton tracking) in a practical scenario and analyzes the subjective perceptions of 19 healthy subjects, 22 stroke survivors, and 14 physical therapists. The optical tracking system provided the best accuracy (1.074 ± 0.417 cm) while the electromagnetic device provided the most inaccurate results (11.027 ± 2.364 cm). However, this tracking solution provided the best jitter values (0.324 ± 0.093 cm), in contrast to the skeleton tracking, which had the worst results (1.522 ± 0.858 cm). Healthy individuals and professionals preferred the skeleton tracking solution rather than the optical and electromagnetic solution (in that order). Individuals with stroke chose the optical solution over the other options. Our results show that subjective perceptions and preferences are far from being constant among different populations, thus suggesting that these considerations, together with the performance parameters, should be also taken into account when designing a rehabilitation system.

A sensor fusion method was developed for vertical channel stabilization by fusing inertial measurements from an Inertial Measurement Unit (IMU) and pressure altitude measurements from a barometric altimeter integrated in the same device (baro-IMU). An Extended Kalman Filter (EKF) estimated the quaternion from the sensor frame to the navigation frame; the sensed specific force was rotated into the navigation frame and compensated for gravity, yielding the vertical linear acceleration; finally, a complementary filter driven by the vertical linear acceleration and the measured pressure altitude produced estimates of height and vertical velocity. A method was also developed to condition the measured pressure altitude using a whitening filter, which helped to remove the short-term correlation due to environment-dependent pressure changes from raw pressure altitude. The sensor fusion method was implemented to work on-line using data from a wireless baro-IMU and tested for the capability of tracking low-frequency small-amplitude vertical human-like motions that can be critical for stand-alone inertial sensor measurements. Validation tests were performed in different experimental conditions, namely no motion, free-fall motion, forced circular motion and squatting. Accurate on-line tracking of height and vertical velocity was achieved, giving confidence to the use of the sensor fusion method for tracking typical vertical human motions: velocity Root Mean Square Error (RMSE) was in the range 0.04–0.24 m/s; height RMSE was in the range 5–68 cm, with statistically significant performance gains when the whitening filter was used by the sensor fusion method to track relatively high-frequency vertical motions.

Carpal tunnel syndrome (CTS) is typically diagnosed by physical examination or nerve conduction measurements. With these diagnostics however it is difficult to obtain anatomical information in the carpal tunnel. To further improve the diagnosis of CTS, an attempt using 30 MHz high-frequency ultrasound to noninvasively detect the local anatomical structures and the kinetic trajectory of the median nerve (MN) in the wrist was explored. Measurements were performed on the right wrist of 14 asymptomatic volunteers. The kinetic trajectory of the MN corresponding to flexion (from 0° to 90°) and extension (from 90° to 0°) movements of the fingers were detected by a cross correlation-based motion tracking technique. The average displacements of the MN according to finger movements were measured to be 3.74 and 2.04 mm for male and female subjects, respectively. Moreover, the kinetic trajectory of the MN in both the ulnar-palmar and total directions generally follows a sigmoidal curve tendency. This study has verified that the use of high-frequency ultrasound imaging and a motion tracking technique to sensitively detect the displacement and kinetic trajectory of the MN for the assessment of CTS patients is feasible.

The question whether barometric altimeters can be applied to accurately track human motions is still debated, since their measurement performance are rather poor due to either coarse resolution or drifting behavior problems. As a step toward accurate short-time tracking of changes in height (up to few minutes), we develop a stochastic model that attempts to capture some statistical properties of the barometric altimeter noise. The barometric altimeter noise is decomposed in three components with different physical origin and properties: a deterministic time-varying mean, mainly correlated with global environment changes, and a first-order Gauss-Markov (GM) random process, mainly accounting for short-term, local environment changes, the effects of which are prominent, respectively, for long-time and short-time motion tracking; an uncorrelated random process, mainly due to wideband electronic noise, including quantization noise. Autoregressive-moving average (ARMA) system identification techniques are used to capture the correlation structure of the piecewise stationary GM component, and to estimate its standard deviation, together with the standard deviation of the uncorrelated component. M-point moving average filters used alone or in combination with whitening filters learnt from ARMA model parameters are further tested in few dynamic motion experiments and discussed for their capability of short-time tracking small-amplitude, low-frequency motions.

According to neuro-rehabilitation practice, active training is effective for mild stroke patients, which means these patients are able to recovery effective when they perform the training to overcome certain resistance by themselves. Therefore, for rehabilitation devices without backdrivability, implementation of human-machine synchronization is important and a precondition to perform active training. In this paper, a method to implement this precondition is proposed and applied in a user’s performance of elbow flexions and extensions when he wore an upper limb exoskeleton rehabilitation device (ULERD), which is portable, wearable and non-backdrivable. In this method, an inertia sensor is adapted to detect the motion of the user’s forearm. In order to get a smooth value of the velocity of the user’s forearm, an adaptive weighted average filtering is applied. On the other hand, to obtain accurate tracking performance, a double close-loop control is proposed to realize real-time and stable tracking. Experiments have been conducted to prove that these methods are effective and feasible for active rehabilitation.

Here an inertial sensor-based monitoring system for measuring and analyzing upper limb movements is presented. The final goal is the integration of this motion-tracking device within a portable rehabilitation system for brain injury patients. A set of four inertial sensors mounted on a special garment worn by the patient provides the quaternions representing the patient upper limb’s orientation in space. A kinematic model is built to estimate 3D upper limb motion for accurate therapeutic evaluation. The human upper limb is represented as a kinematic chain of rigid bodies with three joints and six degrees of freedom. Validation of the system has been performed by co-registration of movements with a commercial optoelectronic tracking system. Successful results are shown that exhibit a high correlation among signals provided by both devices and obtained at the Institut Guttmann Neurorehabilitation Hospital.

Human poses are difficult to estimate due to the complicated body structure and the self-occlusion problem. In this paper, we introduce a marker-less system for human pose estimation by detecting and tracking key body parts, namely the head, hands, and feet. Given color and depth images captured by multiple red, green, blue, and depth (RGB-D) cameras, our system constructs a graph model with segmented regions from each camera and detects the key body parts as a set of extreme points based on accumulative geodesic distances in the graph. During the search process, local detection using a supervised learning model is utilized to match local body features. A final set of extreme points is selected with a voting scheme and tracked with physical constraints from the unified data received from the multiple cameras. During the tracking process, a Kalman filter-based method is introduced to reduce positional noises and to recover from a failure of tracking extremes. Our system shows an average of 87% accuracy against the commercial system, which outperforms the previous multi-Kinects system, and can be applied to recognize a human action or to synthesize a motion sequence from a few key poses using a small set of extremes as input data.

Ranging based on ultrasonic sensors can be used for tracking wearable mobile nodes accurately for a long duration and can be a cost-effective method for human movement analysis in rehabilitation clinics. In this paper, we present a Doppler-tolerant ultrasonic multiple access localization system to analyze gait parameters in human subjects. We employ multiple access methods using linear chirp wave-forms and narrow-band piezoelectric transducers. A Doppler shift compensation Technique is also incorporated without compromising on the tracking accuracy. The system developed was used for tracking the trajectory of both lower limbs of five healthy adults during a treadmill walk. An optical motion capture system was used as the reference to compare the performance. The average Root Mean Square Error values between the 3D coordinates estimated from the proposed system and the reference system while tracking both lower limbs during treadmill walk experiment by 5 subjects were found to be 16.75, 14.68 and 20.20 mm respectively along X, Y and Z-directions. Errors in the estimation of spatial and temporal parameters from the proposed system were also quantified. These promising results show that narrowband ultrasonic sensors can be utilized to accurately track more than one mobile node for human gait analysis.

We propose a motion-robust laser Doppler flowmetry (LDF) system that can be used as a non-contact blood perfusion sensor for medical diagnosis. Endoscopic LDF systems are typically limited in their usefulness in clinical contexts by the need for the natural organs to be immobilized, as serious motion artifacts due to the axial surface displacement can interfere with blood perfusion measurements. In our system, the focusing lens moves to track the motion of the target using a low-frequency reference signal in the optical data, enabling the suppression of these motion artifacts in the axial direction. This paper reports feasibility tests on a prototype of this system using a microfluidic phantom as a measurement target moving in the direction of the optical axis. The frequency spectra detected and the perfusion values calculated from those spectra show that the motion tracking system is capable of suppressing motion artifacts in perfusion readings. We compared the prototype LDF system’s measurements with and without motion feedback, and found that motion tracking improves the fidelity of the perfusion signal by as much as 87%.

One of the key technologies that lays behind the human–machine interaction and human motion diagnosis is the limbs motion tracking. To make the limbs tracking efficient, it must be able to estimate a precise and unambiguous position of each tracked human joint and resulting body part pose. In recent years, body pose estimation became very popular and broadly available for home users because of easy access to cheap tracking devices. Their robustness can be improved by different tracking modes data fusion. The paper defines the novel approach—orientation based data fusion—instead of dominating in literature position based approach, for two classes of tracking devices: depth sensors (i.e., Microsoft Kinect) and inertial measurement units (IMU). The detailed analysis of their working characteristics allowed to elaborate a new method that let fuse more precisely limbs orientation data from both devices and compensates their imprecisions. The paper presents the series of performed experiments that verified the method’s accuracy. This novel approach allowed to outperform the precision of position-based joints tracking, the methods dominating in the literature, of up to 18%.

Stair running, both ascending and descending, is a challenging aerobic exercise that many athletes, recreational runners, and soldiers perform during training. Studying biomechanics of stair running over multiple steps has been limited by the practical challenges presented while using optical-based motion tracking systems. We propose using foot-mounted inertial measurement units (IMUs) as a solution as they enable unrestricted motion capture in any environment and without need for external references. In particular, this paper presents methods for estimating foot velocity and trajectory during stair running using foot-mounted IMUs. Computational methods leverage the stationary periods occurring during the stance phase and known stair geometry to estimate foot orientation and trajectory, ultimately used to calculate stride metrics. These calculations, applied to human participant stair running data, reveal performance trends through timing, trajectory, energy, and force stride metrics. We present the results of our analysis of experimental data collected on eleven subjects. Overall, we determine that for either ascending or descending, the stance time is the strongest predictor of speed as shown by its high correlation with stride time.

Three-dimensional rotations across the human knee serve as important markers of knee health and performance in multiple contexts including human mobility, worker safety and health, athletic performance, and warfighter performance. While knee rotations can be estimated using optical motion capture, that method is largely limited to the laboratory and small capture volumes. These limitations may be overcome by deploying wearable inertial measurement units (IMUs). The objective of this study is to present a new IMU-based method for estimating 3D knee rotations and to benchmark the accuracy of the results using an instrumented mechanical linkage. The method employs data from shank- and thigh-mounted IMUs and a vector constraint for the medial-lateral axis of the knee during periods when the knee joint functions predominantly as a hinge. The method is carefully validated using data from high precision optical encoders in a mechanism that replicates 3D knee rotations spanning (1) pure flexion/extension, (2) pure internal/external rotation, (3) pure abduction/adduction, and (4) combinations of all three rotations. Regardless of the movement type, the IMU-derived estimates of 3D knee rotations replicate the truth data with high confidence (RMS error < 4 ° and correlation coefficient r ≥ 0.94 ).

Assessment of body kinematics during performance of daily life activities at home plays a significant role in medical condition monitoring of elderly people and patients with neurological disorders. The affordable and non-wearable Microsoft Kinect (“Kinect”) system has been recently used to estimate human subject kinematic features. However, the Kinect suffers from a limited range and angular coverage, distortion in skeleton joints’ estimations, and erroneous multiplexing of different subjects’ estimations to one. This study addresses these limitations by incorporating a set of features that create a unique “Kinect Signature”. The Kinect Signature enables identification of different subjects in the scene, automatically assign the kinematics feature estimations only to the subject of interest, and provide information about the quality of the Kinect-based estimations. The methods were verified by a set of experiments, which utilize real-time scenarios commonly used to assess motor functions in elderly subjects and in subjects with neurological disorders. The experiment results indicate that the skeleton based Kinect Signature features can be used to identify different subjects in high accuracy. We demonstrate how these capabilities can be used to assign the Kinect estimations to the Subject of Interest, and exclude low quality tracking features. The results of this work can help in establishing reliable kinematic features, which can assist in future to obtain objective scores for medical analysis of patient condition at home while not restricted to perform daily life activities.