Search

The research aims to lift the accuracy of table tennis trajectory prediction through advanced computer vision and deep learning techniques to achieve real-time and accurate table tennis ball position and motion trajectory tracking. The study concentrates on the innovative application of a micro-miniature fourth-generation real-time target detection algorithm with a gated loop unit to table tennis ball motion analysis by combining physical models and deep learning methods. The results show that in the comparison experiments, the improved micro-miniature fourth-generation real-time target detection algorithm outperforms the traditional target detection algorithm, with the loss value decreasing to 1.54. Its average accuracy in multi-target recognition is dramatically increased to 86.74%, which is 22.36% higher than the original model, and the ping-pong ball recognition experiments show that it has an excellent accuracy in various lighting conditions, especially in low light, with an average accuracy of 89.12%. Meanwhile, the improved model achieves a processing efficiency of 85 frames/s. In addition, compared with the traditional trajectory prediction model, the constructed model performs the best in table tennis ball trajectory prediction, with errors of 4.5 mm, 25.3 mm, and 35.58 mm. The results show that the research trajectory prediction model achieves significant results in accurately tracking table tennis ball positions and trajectories. It not only has practical application value for table tennis training and competition strategies, but also provides a useful reference for the similar techniques application in other sports. [ABSTRACT FROM AUTHOR]

The automatic detection of smoke by analyzing the video stream acquired by traditional surveillance cameras is becoming a more and more interesting problem for the scientific community thanks to the necessity to prevent fires at the very early stages. The adoption of a smart visual sensor, namely a computer vision algorithm running in real time, allows one to overcome the limitations of standard physical sensors. Nevertheless, this is a very challenging problem, due to the strong similarity of the smoke with other environmental elements like clouds, fog and dust. In addition to this challenge, data available for training deep neural networks is limited and not fully representative of real environments. Within this context, in this paper we propose a new method for smoke detection based on the combination of motion and appearance analysis with a modern convolutional neural network (CNN). Moreover, we propose a new dataset, called the MIVIA Smoke Detection Dataset (MIVIA-SDD), publicly available for research purposes; it consists of 129 videos covering about 28 h of recordings. The proposed hybrid method, trained and evaluated on the proposed dataset, demonstrated to be very effective by achieving a 94% smoke recognition rate and, at the same time, a substantially lower false positive rate if compared with fully deep learning-based approaches (14% vs. 100%). Therefore, the proposed combination of motion and appearance analysis with deep learning CNNs can be further investigated to improve the precision of fire detection approaches. [ABSTRACT FROM AUTHOR]

Recently, the offline‐trained Siamese pipeline has drawn wide attention due to its outstanding tracking performance. However, the existing Siamese trackers utilise offline training to extract 'universal' features, which is insufficient to effectively distinguish between the target and fluctuating interference in embedding the information of the two branches, leading to inaccurate classification and localisation. In addition, the Siamese trackers employ a pre‐defined scale for cropping the search candidate region based on the previous frame's result, which might easily introduce redundant background noise (clutter, similar objects etc.), affecting the tracker's robustness. To solve these problems, the authors propose two novel sub‐network spatial employed to spatial feature embedding for robust object tracking. Specifically, the proposed spatial remapping (SRM) network enhances the feature discrepancy between target and distractor categories by online remapping, and improves the discriminant ability of the tracker on the embedding space. The MAML is used to optimise the SRM network to ensure its adaptability to complex tracking scenarios. Moreover, a temporal information proposal‐guided (TPG) network that utilises a GRU model to dynamically predict the search scale based on temporal motion states to reduce potential background interference is introduced. The proposed two network is integrated into two popular trackers, namely SiamFC++ and TransT, which achieve superior performance on six challenging benchmarks, including OTB100, VOT2019, UAV123, GOT10K, TrackingNet and LaSOT, TrackingNet and LaSOT denoting them as SiamSRMC and SiamSRMT, respectively. Moreover, the proposed trackers obtain competitive tracking performance compared with the state‐of‐the‐art trackers in the attribute of background clutter and similar object, validating the effectiveness of our method. [ABSTRACT FROM AUTHOR]

In den letzten Jahren hat die digitale Revolution auch die Prävention und Rehabilitation von Sportverletzungen rasant erfasst. Für die Prävention spielt die Digitalisierung bereits durch die Integration verschiedener Technologien und Ansätze eine wichtige Rolle. Dazu gehören Wearables, Sensoren und Tracking-Plattformen, virtuelles Training und Simulation sowie Big Data und künstliche Intelligenz (KI). Die Digitalisierung ermöglicht mit diesen Methoden einen ganzheitlichen Ansatz zur Verletzungsprävention im Sport, indem Echtzeitdaten, Analysetools und innovative Technologien genutzt werden, um potenzielle Risiken zu erkennen und um umgehend präventive Maßnahmen zu ergreifen. Auch in der Rehabilitation von Sportverletzungen spielt die Digitalisierung heute eine große Rolle. Mobile Anwendungen, in die KI, Sensoren oder virtuelle Realität integriert werden können, werden in der sportmedizinischen Rehabilitation bereits klinisch eingesetzt. Diese Apps können den Rehabilitationsprozess unterstützen, indem sie Übungsprogramme vorschlagen, Feedback über Sensoren geben und den Rehabilitationsprozess mithilfe von KI an die aktuellen Fähigkeiten des Patienten anpassen. Zusammenfassend lässt sich sagen, dass die Digitalisierung zweifellos das Potenzial hat, unser bisheriges Verständnis von Prävention und Rehabilitation zu revolutionieren. Um die Risiken dieser neuen Technologien einschätzen zu können, sind jedoch weitere Forschungsarbeiten erforderlich. In recent years, the digital revolution has rapidly affected the prevention and rehabilitation of sports injuries. For prevention, digitalization plays an important role by integrating various technologies and approaches. These include wearables, sensors and tracking platforms for monitoring, motion capture for biomechanical analysis, virtual training and simulation, and big data and artificial intelligence (AI) for identifying patterns and relationships that indicate risk of injury. Digitalization therefore enables a holistic approach to injury prevention in sport by using real-time data, analysis tools and innovative technologies to identify potential risks and take appropriate measures. Digitalization also plays a major role in the rehabilitation of sports injuries today. Mobile applications in which AI, sensors or virtual reality can be integrated are already used clinically in sports medicine rehabilitation. These apps can support the rehabilitation process by suggesting exercise programs, providing feedback via sensors and using AI to adapt the rehabilitation process to the patient's current abilities. In summary, digitalization undoubtedly has the potential to revolutionize the way we have previously understood prevention and rehabilitation. However, further research is necessary to be able to assess the risks of these new technologies. [ABSTRACT FROM AUTHOR]

This article discusses the use of computer vision technology for automated surgical video analysis to improve surgical proficiency. The authors propose that automation can overcome barriers such as time and labor commitment to transfer and organize data, locate and edit critical sections, and develop systematic analyses. The authors piloted their computer vision technologies in robotic surgeries and are now adopting them for endoscopic neurosurgery. The results show high accuracy in surgical scene classification, surgical tool tracking, and camera motion estimation. The authors suggest that further studies are needed to investigate the potential for wider adaptation and usage of the data in performance tracking, enhancing surgical skill education, and individualized technical development. [Extracted from the article]

With the recent increase in interest in machine learning and computer vision, camera-based pose estimation has emerged as a promising new technology. One of the most popular libraries for camera-based pose estimation is MediaPipe Pose due to its computational efficiency, ease of use, and the fact that it is open-source. However, little work has been performed to establish how accurate the library is and whether it is suitable for usage in, for example, physical therapy. This paper aims to provide an initial assessment of this. We find that the pose estimation is highly dependent on the camera’s viewing angle as well as the performed exercise. While high accuracy can be achieved under optimal conditions, the accuracy quickly decreases when the conditions are less favourable.

This work reviews the dataset-driven advancements that have occurred in the area of lip motion analysis, particularly visual lip-reading and visual lip motion authentication, in the deep learning era. We provide an analysis of datasets and their usage, creation, and associated challenges. Future research can utilize this work as a guide for selecting appropriate datasets and as a source of insights for creating new and innovative datasets. Large and varied datasets are vital to a successful deep learning system. There have been many incredible advancements made in these fields due to larger datasets. There are indications that even larger, more varied datasets would result in further improvement upon existing systems. We highlight the datasets that brought about the progression in lip-reading systems from digit- to word-level lip-reading, and then from word- to sentence-level lip-reading. Through an in-depth analysis of lip-reading system results, we show that datasets with large amounts of diversity increase results immensely. We then discuss the next step for lip-reading systems to move from sentence- to dialogue-level lip-reading and emphasize that new datasets are required to make this transition possible. We then explore lip motion authentication datasets. While lip motion authentication has been well researched, it is not very unified on a particular implementation, and there is no benchmark dataset to compare the various methods. As was seen in the lip-reading analysis, large, diverse datasets are required to evaluate the robustness and accuracy of new methods attempted by researchers. These large datasets have pushed the work in the visual lip-reading realm. Due to the lack of large, diverse, and publicly accessible datasets, visual lip motion authentication research has struggled to validate results and real-world applications. A new benchmark dataset is required to unify the studies in this area such that they can be compared to previous methods as well as validate new methods more effectively. [ABSTRACT FROM AUTHOR]

Existing 3D single object tracking methods primarily extract features from the global coordinates of point clouds, overlooking the potential exploitation of their positional information. However, due to the unordered, sparse, and irregular nature of point clouds, effectively exploring their positional information presents a significant challenge. In this letter, the network is explicitly reformulated by introducing a point position embedding module in conjunction with a self‐attention coding module, replacing the use of global coordinate inputs. The proposed reformulation is further integrated into a top‐notch model M2‐Track, called Point Position Embedding (PPE) in this letter. Comprehensive empirical analysis are performed on the KITTI and NuScenes datasets. Experimental results show that the PPE surpasses M2‐Track by a large margin in overall performance. Especially for the challenging NuScenes dataset, the method attains the highest precision and success in all classes compared to state‐of‐the‐art methods. The code is available at https://github.com/GZHU‐DVL/PPE. [ABSTRACT FROM AUTHOR]

Postural instability is a risk factor for falls in older adults. It is possible to detect postural stability using an integrated accelerometer (ACC) sensor in a smartphone. Therefore, a novel ACC-based smartphone application running on the Android operating system called BalanceLab was created and tested. This study aimed to determine the validity and reliability of a novel ACC-based Android smartphone application for assessing balance in older adults. Using BalanceLab, 20 older adults completed three balance assessments: the Modified Clinical Test of Sensory Interaction in Balance (MCTSIB), a single-leg stance test (SLST), and a limit of stability test (LOS). The validity of this mobile application was investigated using a three-dimensional (3D) motion analysis system and the Fullerton Advanced Balance (FAB) scale. The test-retest reliability of this mobile application was determined on two separate occasions within one day, at least two hours apart. The two static balance assessments (the MCTSIB and SLST) demonstrated moderate to excellent correlation with the 3D motion analysis system (r = 0.70–0.91) and the FAB scale (r = 0.67–0.80). However, the majority of the dynamic balance tests (the LOS tests) did not demonstrate any correlation with the 3D motion analysis system or the FAB scale. This novel ACC-based application demonstrated good to excellent test-retest reliability (ICC = 0.76–0.91). A static, but not dynamic, balance assessment tool that uses a novel ACC-based application for Android can be used to measure balance in older adults. This application has moderate to excellent validity and test-retest reliability. • A new accelerometer-based app was examined for its reliability and validity. • A balance app demonstrated high test-retest reliability. • An app showed a high level of validity in static but not in dynamic balance tests. • An older adult's static balance can be evaluated with a new balance app. [ABSTRACT FROM AUTHOR]

Precisely detecting puncture times has long posed a challenge in medical education. This challenge is attributable not only to the subjective nature of human evaluation but also to the insufficiency of effective detection techniques, resulting in many medical students lacking full proficiency in injection skills upon entering clinical practice. To address this issue, we propose a novel detection method that enables automatic detection of puncture times during injection without needing wearable devices. In this study, we utilized a hardware system and the YOLOv7 algorithm to detect critical features of injection motion, including puncture time and injection depth parameters. We constructed a sample of 126 medical injection training videos of medical students, and skilled observers were employed to determine accurate puncture times. Our experimental results demonstrated that the mean puncture time of medical students was 2.264 s and the mean identification error was 0.330 s. Moreover, we confirmed that there was no significant difference (p = 0.25 with a significance level of α = 0.05) between the predicted value of the system and the ground truth, which provides a basis for the validity and reliability of the system. These results show our system's ability to automatically detect puncture times and provide a novel approach for training healthcare professionals. At the same time, it provides a key technology for the future development of injection skill assessment systems. [ABSTRACT FROM AUTHOR]

Cow lameness is a complex locomotion behaviour, and effective characteristics are the key to identifying early lameness. Machine learning methods were used to quantify the movement of the hooves and head of cows, and they showed the possibility of using the head- and back-hoof linkage features extracted from motion analysis to classify early lameness. Firstly, the key points of the body in the side-view walking videos were tracked by DeepLabCut (DLC). Based on the curves of hoof movement and head swing, the head-hoof linkage pattern was analysed. Second, the curve of cow's back was extracted and the back-hoof linkage feature was built when the hoof touched the ground. At the same time, the movement curves were analysed to extract five other gait characteristics, including asymmetry of stride length, tracking up, asymmetry in the proportion of supporting phase, landing speed, and moving speed. Multiple models were trained on a dataset containing 212 videos, and 10-fold cross-validation was used to verify the performance of the algorithm. The classification performance was improved by optimising the model and algorithm. Overall accuracy and recall were 89.2% and 90.7%, respectively. The scores of the importance of features calculated by the Chi-square test showed that the head- and back-hoof linkage features play an important role in the classification of early lame cows. The hooves and their linkage features with the head and back of cows proposed in this paper can thus effectively detect early lameness in cows. • DeepLabCut was used to track four hooves and head of walking cow. • Head- and back-hoof linkage features was analysed. • Seven lameness indicators were used to build the model. • The recall of classification for early lame cows was 90.7%. [ABSTRACT FROM AUTHOR]

Objective: Sleep‐related hypermotor epilepsy (SHE) is a focal epilepsy with seizures occurring mostly during sleep. SHE seizures present different motor characteristics ranging from dystonic posturing to hyperkinetic motor patterns, sometimes associated with affective symptoms and complex behaviors. Disorders of arousal (DOA) are sleep disorders with paroxysmal episodes that may present analogies with SHE seizures. Accurate interpretation of the different SHE patterns and their differentiation from DOA manifestations can be difficult and expensive, and can require highly skilled personnel not always available. Furthermore, it is operator dependent. Methods: Common techniques for human motion analysis, such as wearable sensors (e.g., accelerometers) and motion capture systems, have been considered to overcome these problems. Unfortunately, these systems are cumbersome and they require trained personnel for marker and sensor positioning, limiting their use in the epilepsy domain. To overcome these problems, recently significant effort has been spent in studying automatic methods based on video analysis for the characterization of human motion. Systems based on computer vision and deep learning have been exploited in many fields, but epilepsy has received limited attention. Results: In this paper, we present a pipeline composed of a set of three‐dimensional convolutional neural networks that, starting from video recordings, reached an overall accuracy of 80% in the classification of different SHE semiology patterns and DOA. Significance: The preliminary results obtained in this study highlight that our deep learning pipeline could be used by physicians as a tool to support them in the differential diagnosis of the different patterns of SHE and DOA, and encourage further investigation. [ABSTRACT FROM AUTHOR]

Goal: Conventionally, a surgeon's skill is assessed through visual observation by experts and by tracking patient outcomes. These techniques are very subjective and demands enormous time and effort. Hence, the aim of this study is to construct a framework for automated objective assessment of micro-neurosurgical skill. Methods: A mask region-based convolution neural network (RCNN) is trained to identify and localize instances of surgical instruments from the recorded neurosurgery videos. Then the tool motion and tool handling metrics are computed by tracking the detected instrument locations through time. Microscope adjustment patterns are also investigated via the proposed time based metrics.Results: This study highlights the metrics that could potentially emphasize the variance in expertise between a veteran and a novice. These variations include an expert exhibiting a lower velocity, lower acceleration, lower jerks, reduced path length, higher normalized angular displacement, increased bi-manual handling, shorter idle time and smaller inter tool-tip distances while handling tools accompanied with frequent microscope adjustments and reduced maximum and median intervals between adjustments when compared to a novice. Conclusions: The developed vision based framework has proven to be a reliable method to assess the degree of surgical skill objectively and offer prompt and precise feedback to the neurosurgeons.

Recent advancements in computing and artificial intelligence (AI) make it possible to quantitatively evaluate human movement using digital video, thereby opening the possibility of more accessible gait analysis. The Edinburgh Visual Gait Score (EVGS) is an effective tool for observational gait analysis, but human scoring of videos can take over 20 min and requires experienced observers. This research developed an algorithmic implementation of the EVGS from handheld smartphone video to enable automatic scoring. Participant walking was video recorded at 60 Hz using a smartphone, and body keypoints were identified using the OpenPose BODY25 pose estimation model. An algorithm was developed to identify foot events and strides, and EVGS parameters were determined at relevant gait events. Stride detection was accurate within two to five frames. The level of agreement between the algorithmic and human reviewer EVGS results was strong for 14 of 17 parameters, and the algorithmic EVGS results were highly correlated (r > 0.80, "r" represents the Pearson correlation coefficient) to the ground truth values for 8 of the 17 parameters. This approach could make gait analysis more accessible and cost-effective, particularly in areas without gait assessment expertise. These findings pave the way for future studies to explore the use of smartphone video and AI algorithms in remote gait analysis. [ABSTRACT FROM AUTHOR]

This paper presents a method on how to compute the position of a robotic arm in the 2D and 3D spaces. This method is slightly different from the well-known methods, such as forward or inverse kinematics. The method presented in this paper is an optical method, which uses two video cameras in stereo vision configuration to locate and compute the next move of a robotic arm in space. The method recognizes the coordinates of the markers placed at the joints of the robotic arm using the two video cameras. The coordinate points of these markers are connected with straight lines. Around certain points, circles are drawn. From the tangent to the circles, a non-Cartesian (orthogonal) coordinate system is drawn, which is enough to compute the target position of the robotic arm. All of these drawings are overlaid on the live video feed. This paper also presents another method for calculating the stereo distance using the triangulation method. An alternative method is also presented when a non-Cartesian (orthogonal) 3D coordinate system is created, which is used to compute the target position of the robotic arm in the 3D space. Because the system is in a loop, it can make micro-adjustments of the robotic arm, in order to be exactly in the desired position. In this way, there is no need to make calibrations for the robotic arm. In an industrial system, there is no need to stop the production line, which can be a really big cost saver. [ABSTRACT FROM AUTHOR]

Crowd tracking and analysis of crowd behavior is a challenging research area in computer vision. In today's crowded environment manual surveillance systems are inefficient, labor-intensive, and unwieldy. Automated video surveillance systems offer promising solutions to these problems and hence become a need for today's environment. However, challenges remain. The most important challenge is the extraction of foreground representing human pixels only, also the extraction of robust spatial and temporal descriptors along with potent classifier is an essential part for accurate behavior detection. In this paper, we present our approach to these challenges by inventing semantic segmentation for foreground extraction. Furthermore, for pedestrians counting and tracking we introduced a fusion of human motion analysis and attraction force model by the weighted averaging method that removes non-humans and non-pedestrians from the scene. The verified pedestrians are counted using a fuzzy-c-mean algorithm and tracked via Hungarian algorithm association along with dynamic template matching technique. However, for anomaly detection after silhouettes extraction, we invent robust Spatio-temporal descriptors including crowd shape deformation, silhouette slicing, particles convection, dominant motion, and energy descriptors. That we optimized using an adaptive genetic algorithm and finally, multi-fused optimal features are fed to a multilayer neuro-fuzzy classifier for decision making. The proposed system is validated via extensive experimentations and achieved an accuracy of 91.8% and 89.16% over UCSD and Mall datasets for crowd tracking. However, the mean absolute error and mean square error for pedestrian counting are 1.69 and 2.09 over UCSD dataset and 2.57 and 4.34 for Mall dataset, respectively. An accuracy of 96.5% and 94% is achieved over UMN and MED datasets for anomaly detection. [ABSTRACT FROM AUTHOR]

Regardless of the type of sport, coaches must perform the difficult task of evaluating the performance of athletes. In some cases, this task is aided using technology which provides tools for this purpose. When the sport considered is taekwondo this scenario does not apply as the athlete evaluation methods used are mostly manual. Thus, the project presented in this paper has the main objective to develop a system that can be used as a tool to evaluate the performance of taekwondo athletes in real time, with special attention to the low cost of implementation and ease of use. With the intention of meeting these requirements, the developed system comprises a 3D camera with a depth sensor (Orbbec Astra), Inertial Measurement Units (IMUs) with accelerometer and gyroscope, a computer and specific software developed for this purpose. This system allows the collection of data from the athletes' movements necessary for the creation of a dataset that is then analyzed and interpretated. The system permits the user to obtain real-time information about the speed, acceleration, and strength of the athlete's limbs during training as well as the identification of some movements and their accounting. To achieve this functionality, deep-learning architecture models were used, more specifically long short-term memory (LSTM). The intention is to provide a new training methodology through faster feedback, so that a faster evolution of the athlete's performance is possible, contributing to the technological development of the assessment practices used in taekwondo. [ABSTRACT FROM AUTHOR]

Human motion analysis has been a common thread across modern and early medicine. While medicine evolves, analysis of movement disorders is mostly based on clinical presentation and trained observers making subjective assessments using clinical rating scales. Currently, the field of computer vision has seen exponential growth and successful medical applications. While this has been the case, neurology, for the most part, has not embraced digital movement analysis. There are many reasons for this including: the limited size of labeled datasets, accuracy and nontransparent nature of neural networks, and potential legal and ethical concerns. We hypothesize that a number of opportunities are made available by advancements in computer vision that will enable digitization of human form, movements, and will represent them synthetically in 3D. Representing human movements within synthetic body models will potentially pave the way towards objective standardized digital movement disorder diagnosis and building sharable open-source datasets from such processed videos. We provide a hypothesis of this emerging field and describe how clinicians and computer scientists can navigate this new space. Such digital movement capturing methods will be important for both machine learning-based diagnosis and computer vision-aided clinical assessment. It would also supplement face-to-face clinical visits and be used for longitudinal monitoring and remote diagnosis. [ABSTRACT FROM AUTHOR]

Human pose estimation (HPE) has become a prevalent research topic in computer vision. The technology can be applied in many areas, such as video surveillance, medical assistance, and sport motion analysis. Due to higher demand for HPE, many HPE libraries have been developed in the last 20 years. In the last 5 years, more and more skeleton-based HPE algorithms have been developed and packaged into libraries to provide ease of use for researchers. Hence, the performance of these libraries is important when researchers intend to integrate them into real-world applications for video surveillance, medical assistance, and sport motion analysis. However, a comprehensive performance comparison of these libraries has yet to be conducted. Therefore, this paper aims to investigate the strengths and weaknesses of four popular state-of-the-art skeleton-based HPE libraries for human pose detection, including OpenPose, PoseNet, MoveNet, and MediaPipe Pose. A comparative analysis of these libraries based on images and videos is presented in this paper. The percentage of detected joints (PDJ) was used as the evaluation metric in all comparative experiments to reveal the performance of the HPE libraries. MoveNet showed the best performance for detecting different human poses in static images and videos. [ABSTRACT FROM AUTHOR]

The automatic recognition of pointers is of great significance for efficiently collecting measurements from industrial instruments. In this paper, an automatic pointer meter recognition system based on line scan vision is developed. A line scan camera is used to capture images of the pointer of a pointer meter. Light-spot centroid algorithm is implemented to determine the centroid position of the pointer. The captured images can record the dynamic movement of the pointer, thereby enabling condition monitoring in industrial processes. Experimental results show that the proposed pointer meter extraction method is robust against interference and that a characteristic segmentation classifier produces more accurate detection results than other approaches. [ABSTRACT FROM AUTHOR]

The authors adopted a cropping method based on the fixed four regions of the face image, to segment out the key image regions and to ignore the irrelevant regions, so as to improve the efficiency of the algorithm computation. Jian et al. presented an image recovery model via structured light projection, considering the restoration of instantaneous distorted images. Then, in a second step, the overfitting problem of the deep network due to a lack of data can be effectively mitigated, by providing the global image and segmented local image regions as training data for the integrated dual path model. [Extracted from the article]

Human pose and motion detection is an important area of computer vision research, covering the interplay of different fields such as image processing, pattern recognition, and artificial intelligence. Due to the complexity of human motion, existing 3D recognition and pose detection methods based on low-quality depth images are not very accurate and reliable. Due to the time-sensitive nature of features, a single feature cannot adapt to the dynamic changes of the scene, so it is difficult for the target tracking algorithm based on a single feature to achieve robust tracking results. If multiple features are fused and applied in the tracking algorithm, the complementarity between different features can be used to better adapt to the scene changes and achieve robust tracking results. In order to solve the problem of human pose and human motion recognition in low-quality depth images, this paper uses the Kinect somatosensory camera to obtain 20 human skeleton joint points through the Kinect skeleton tracking technology. This paper studies the typical human posture and interactive action recognition technology in daily life. On the basis of understanding the characteristics of skeletal data, this paper proposes a distance feature and angle feature model combined with human body structure. Through the experimental results, it is found that the distance feature and the angle feature value are basically not affected by the distance change. When the subjects turned 45° to the left and 45° to the right, the distance characteristics changed, which was different from the characteristic data when the subjects turned around. [ABSTRACT FROM AUTHOR]

In order to improve the effectiveness of the evaluation of student's learning status in foreign language classrooms, this paper applies machine vision to classroom teaching. Through an in-depth analysis of the relative motion relationship between the end marker points of classroom feature recognition and the center point of the machine vision system window, this paper first proposes an autonomous tracking motion algorithm of the machine vision system window based on the preset field of view parameters. Moreover, this paper realizes the motion function of the window to track the marker points autonomously, completes the simulation analysis through two sets of planned trajectories and two sets of master hands to collect the actual trajectories, and verifies the correctness and feasibility of the algorithm. The research study shows that the algorithm based on machine vision proposed in this paper can effectively judge the real-time state of students in the foreign language classroom. [ABSTRACT FROM AUTHOR]

The deep learning-based automatic recognition of the scanning or exposing region in medical imaging automation is a promising new technique, which can decrease the heavy workload of the radiographers, optimize imaging workflow and improve image quality. However, there is little related research and practice in X-ray imaging. In this paper, we focus on two key problems in X-ray imaging automation: automatic recognition of the exposure moment and the exposure region. Consequently, we propose an automatic video analysis framework based on the hybrid model, approaching real-time performance. The framework consists of three interdependent components: Body Structure Detection, Motion State Tracing, and Body Modeling. Body Structure Detection disassembles the patient to obtain the corresponding body keypoints and body Bboxes. Combining and analyzing the two different types of body structure representations is to obtain rich spatial location information about the patient body structure. Motion State Tracing focuses on the motion state analysis of the exposure region to recognize the appropriate exposure moment. The exposure region is calculated by Body Modeling when the exposure moment appears. A large-scale dataset for X-ray examination scene is built to validate the performance of the proposed method. Extensive experiments demonstrate the superiority of the proposed method in automatically recognizing the exposure moment and exposure region. This paradigm provides the first method that can enable automatically and accurately recognize the exposure region in X-ray imaging without the help of the radiographer. [ABSTRACT FROM AUTHOR]

Based on computer vision technology, this paper presents a human motion analysis and target tracking technology based on computer vision. In terms of moving target detection, the current moving target detection technology is summarized, and some experimental results of the algorithm are given. The background difference method under monocular camera is emphatically analyzed. The preliminary human contour is obtained by the background difference method. In order to obtain a smoother target contour, the mathematical morphology is used to remove the noise, and the judgment algorithm of the size of the image connected domain is added. A specific threshold is set to remove the connected domain of the noise block less than the threshold. In the aspect of human motion recognition, this paper selects human motion features, including minimum external rectangle aspect ratio, rectangularity, circularity, and moment invariant. The criteria for selecting human motion features are strong noise resistance and obvious distinction. Then, the three types of human motion images are classified and recognized. After cross-validation and parameter optimization, the recognition accuracy is significantly improved. The experimental results show that the video sequence collected in the field has a total of 376 frames, and the frame rate is 10 frames/s. Due to the small traffic, the mean shift algorithm based on adaptive feature fusion is used to track the target every 2-3 frames. And set the inverse X direction as the direction of entering the scene and the X direction as the direction of moving out of the scene so that the allowable error of the distance between the detection and tracking results is 10. The weight of each feature is dynamically updated by the similarity between the candidate model and the target model, which solves the problem that the mean shift algorithm is not robust enough when similar objects are occluded and interfered and achieves more accurate tracking. [ABSTRACT FROM AUTHOR]

Robotic harvesting research has seen significant achievements in the past decade, with breakthroughs being made in machine vision, robot manipulation, autonomous navigation and mapping. However, the missing capability of obstacle handling during the grasping process has severely reduced harvest success rate and limited the overall performance of robotic harvesting. This work focuses on leaf interference caused slip detection and handling, where solutions to robotic grasping in an unstructured environment are proposed. Through analysis of the motion and force of fruit grasping under leaf interference, the connection between object slip caused by leaf interference and inadequate harvest performance is identified for the first time in the literature. A learning-based perception and manipulation method is proposed to detect slip that causes problematic grasps of objects, allowing the robot to implement timely reaction. Our results indicate that the proposed algorithm detects grasp slip with an accuracy of 94%. The proposed sensing-based manipulation demonstrated great potential in robotic fruit harvesting, and could be extended to other pick-place applications. [ABSTRACT FROM AUTHOR]

In recent years, the visual analysis of human motion has become a frontier direction in the field of computer vision. It recognizes, identifies, and tracks people in image sequences, as well as understands and describes their actions. Using computer vision technology to study the field of image processing and pattern recognition, as well as extracting and effectively identifying human motion features from video images, has become a hot topic of concern with the rapid development and popularization of information technology. This study presents a method for analyzing the characteristics of moving human bodies based on image recognition, introduces the extraction function method, and analyzes the characteristics of the extraction function to improve the ability to analyze the characteristics of moving human bodies. To achieve fine segmentation, the hierarchical clustering algorithm is used to segment the periodic motion in each motion. Different benchmark databases and self-built data were used in the experiments. Experiments show that the algorithm can achieve good classification and recognition results while maintaining low computational complexity and extracting less feature data. It can also organically integrate the static and dynamic features of human walking. [ABSTRACT FROM AUTHOR]

Optical flow estimation is a fundamental task in computer vision and image processing. Due to the difficulty in obtaining the ground truth of flow field, unsupervised learning approaches attract more and more research interests in recent years. However, despite of their good generalization capability, unsupervised optical flow methods suffer in the scenarios with large displacement, small objects, and occlusions. In this work, we propose a novel optical flow network based on decoder with multi-scale kernels. Different from previous U-Net like or pyramidal methods, we design our network based on RAFT architecture that with a 4D correlation layer and recurrent decoder. More importantly, we incorporate three novel ideas with regard to the input, information processing and output of the update units improve the performance. Firstly, we utilize various motion-related information as input to the update units. Secondly, we propose a module of multi-scale update unit. Thirdly, for the final flow up-sampling procedure, we propose an image-guided up-sampling loss to guide the learning of up-sampling masks. Our model is trained by the occlusion-aware photometric loss, edge-aware smoothness loss, self-supervised loss, and image-guided up-sampling loss. Experimental results demonstrate that our model achieves the state-of-the-art performance on both Sintel and KITTI and outperforms other unsupervised optical flow methods remarkably. [ABSTRACT FROM AUTHOR]

Event cameras such as DAVIS can simultaneously output high temporal resolution events and low frame-rate intensity images, which own great potential in capturing scene motion, such as optical flow estimation. Most of the existing optical flow estimation methods are based on two consecutive image frames and can only estimate discrete flow at a fixed time interval. Previous work has shown that continuous flow estimation can be achieved by changing the quantities or time intervals of events. However, they are difficult to estimate reliable dense flow, especially in the regions without any triggered events. In this paper, we propose a novel deep learning-based dense and continuous optical flow estimation framework from a single image with event streams, which facilitates the accurate perception of high-speed motion. Specifically, we first propose an event-image fusion and correlation module to effectively exploit the internal motion from two different modalities of data. Then we propose an iterative update network structure with bidirectional training for optical flow prediction. Therefore, our model can estimate reliable dense flow as two-frame-based methods, as well as estimate temporal continuous flow as event-based methods. Extensive experimental results on both synthetic and real captured datasets demonstrate that our model outperforms existing event-based state-of-the-art methods and our designed baselines for accurate dense and continuous optical flow estimation. [ABSTRACT FROM AUTHOR]

Suspicious behavior is dangerous in public areas that may cause heavy causalities. There are various systems developed on the basis of video frame acquisition where or pedestrian detection occur but those systems are not intelligent enough to identify the unusual activities even at real time. It is required to recognized scamper situation at real time from video surveillance for quick and immediate management before any casualties. Proposed system focuses on recognizing suspicious activities and target to achieve a technique which is able to detect suspicious activity automatically using computer vision. Here system uses OpenCV library for classifying different kind of action at real time. The motion influence map has been used to represent the motion analysis that frequently changes the position from one place to another. System uses pixel level presentation for making it easy to understand or identify the actual situation.. [ABSTRACT FROM AUTHOR]

Deep learning became an important image classification and object detection technique more than a decade ago. It has since achieved human-like performance for many computer vision tasks. Some of them involve the analysis of human face for applications like facial recognition, expression recognition, and facial landmark detection. In recent years, researchers have generated and made publicly available many valuable datasets that allow for the development of more accurate and robust models for these important tasks. Exploiting the information contained inside these pretrained deep structures could open the door to many new applications and provide a quick path to their success. This research focuses on a unique application that analyzes short facial motion video for identity verification. Our proposed solution leverages the rich information in those deep structures to provide accurate face representation for facial motion analysis. We have developed two strategies to employ the information contained in the existing models for image-based face analysis to learn the facial motion representations for our application. Combining with those pretrained spatial feature extractors for face-related analyses, our customized sequence encoder is capable of generating accurate facial motion embedding for identity verification application. The experimental results show that the facial geometry information from those feature extractors is valuable and helps our model achieve an impressive average precision of 98.8% for identity verification using facial motion. [ABSTRACT FROM AUTHOR]

Recent advancements in computing and artificial intelligence (AI) make it possible to quantitatively evaluate human movement using digital video, thereby opening the possibility of more accessible gait analysis. The Edinburgh Visual Gait Score (EVGS) is an effective tool for observational gait analysis, but human scoring of videos can take over 20 min and requires experienced observers. This research developed an algorithmic implementation of the EVGS from handheld smartphone video to enable automatic scoring. Participant walking was video recorded at 60 Hz using a smartphone, and body keypoints were identified using the OpenPose BODY25 pose estimation model. An algorithm was developed to identify foot events and strides, and EVGS parameters were determined at relevant gait events. Stride detection was accurate within two to five frames. The level of agreement between the algorithmic and human reviewer EVGS results was strong for 14 of 17 parameters, and the algorithmic EVGS results were highly correlated (r > 0.80, “r” represents the Pearson correlation coefficient) to the ground truth values for 8 of the 17 parameters. This approach could make gait analysis more accessible and cost-effective, particularly in areas without gait assessment expertise. These findings pave the way for future studies to explore the use of smartphone video and AI algorithms in remote gait analysis.

This paper presents a method on how to compute the position of a robotic arm in the 2D and 3D spaces. This method is slightly different from the well-known methods, such as forward or inverse kinematics. The method presented in this paper is an optical method, which uses two video cameras in stereo vision configuration to locate and compute the next move of a robotic arm in space. The method recognizes the coordinates of the markers placed at the joints of the robotic arm using the two video cameras. The coordinate points of these markers are connected with straight lines. Around certain points, circles are drawn. From the tangent to the circles, a non-Cartesian (orthogonal) coordinate system is drawn, which is enough to compute the target position of the robotic arm. All of these drawings are overlaid on the live video feed. This paper also presents another method for calculating the stereo distance using the triangulation method. An alternative method is also presented when a non-Cartesian (orthogonal) 3D coordinate system is created, which is used to compute the target position of the robotic arm in the 3D space. Because the system is in a loop, it can make micro-adjustments of the robotic arm, in order to be exactly in the desired position. In this way, there is no need to make calibrations for the robotic arm. In an industrial system, there is no need to stop the production line, which can be a really big cost saver.

Human motion analysis is one of the important research directions in computer vision technology. Aiming at the problems of poor effect, accuracy, and efficiency in the current tennis serve trajectory capture process, a tennis serve trajectory capture algorithm based on wavelet multiscale decomposition is proposed. This article analyzes the related concepts and principles of wavelet transform and multiresolution analysis, uses the computer three-dimensional vision acquisition method to collect the tennis serve trajectory image, and uses the mean filter to smooth and denoise the collected tennis serve trajectory image. The wavelet multiscale decomposition algorithm is used to process the tennis serve trajectory image after smooth denoising, find the local maximum point through the modulus value and phase angle value, and obtain the initial tennis serve trajectory image. Using the local layer threshold method, the adaptive threshold is set, and the trajectory line under each scale is obtained to capture the trajectory line of tennis serve. It can be observed from the experimental and simulation results that the trajectory capture effect of the proposed algorithm is good, and it can effectively improve the accuracy and efficiency of trajectory capture. [ABSTRACT FROM AUTHOR]

Having high precision ground-truth data is a very important factor for the development and evaluation of computer vision algorithms such as digital video stabilization. However, generating this data is time consuming and cost intensive work, requiring a lot of manual effort. In this paper we both propose a way to automatically generate a large amount of accurate data for digital video stabilization verification and provide a comprehensive dataset of video sequences taken from multi-sensor imaging system with different types of disturbances. A novel method for generating verification data is based on genetic algorithm template matching. Paper provides quantitative analysis together with the visual assessment of digital video stabilization performance. [ABSTRACT FROM AUTHOR]

The computer vision community has extensively researched the area of human motion analysis, which primarily focuses on pose estimation, activity recognition, pose or gesture recognition and so on. However for many applications, like monitoring of functional rehabilitation of patients with musculo skeletal or physical impairments, the requirement is to comparatively evaluate human motion. In this survey, we capture important literature on vision-based monitoring and physical rehabilitation that focuses on comparative evaluation of human motion during the past two decades and discuss the state of current research in this area. Unlike other reviews in this area, which are written from a clinical objective, this article presents research in this area from a computer vision application perspective. We propose our own taxonomy of computer vision-based rehabilitation and assessment research which are further divided into sub-categories to capture novelties of each research. The review discusses the challenges of this domain due to the wide ranging human motion abnormalities and difficulty in automatically assessing those abnormalities. Finally, suggestions on the future direction of research are offered. [ABSTRACT FROM AUTHOR]

Video analysis of human motion has been widely used in intelligent monitoring, sports analysis, and virtual reality as a research hotspot in computer vision. It is necessary to decompose and track the movements in the process of movement in order to improve the training quality in dance training. The traditional motion tracking decomposition method, on the other hand, is unable to calculate the visual changes of adjacent key nodes, and the contour of 3D visual motion tracking remains ambiguous. This paper applies the human posture estimation algorithm in computer vision to the detection of key points of rectangular objects and obtains the heat map of key points of rectangular objects by adding a lightweight feature extraction network and a feature pyramid layer integrating multilayer semantic information, on the basis of summarizing and analyzing related research work at home and abroad. Because of the fusion of multilayer information, the network's design not only reduces the amount of calculation and parameters but also improves the accuracy of the final detection result. The test results show that the proposed algorithm's recognition accuracy has improved. [ABSTRACT FROM AUTHOR]

This work describes a novel methodology for creating exergames on an edge-native platform with the integration of multiple deep neural networks. A prototype of the platform, which includes capabilities for innovative gameplay and advanced user interactivity, has been implemented and deployed in a real-world scenario. At core of the proposed methodology is the ad hoc training of classifiers for posture classification which can be dynamically adapted to the specific requirements of the usage scenario, operational and environmental conditions allowing for real-time identification of events and advanced game control. The proposed solution is ideal for individual consumers in a home environment since is supports by-design edge platforms minimizing the cost of the system and enabling in parallel the communication with state-of-the-art hardware (i.e., GPUs, TPUs, computer boards) for real-time operation. The proposed system allows the collection and analysis of game data, which can be exploited by specialized personnel in rehabilitation centers or for other purposes in the areas of healthcare and assisted living. [ABSTRACT FROM AUTHOR]

The aims were: (1) to determine the accuracy and time-consuming of the automatic digitalization feature of 3D motion analysis systems; and (2) to determine the reliability of the 3D position when a football player performed a hard kick with spin. Four high-speed cameras (Casio ZR200 (512x384)) with a 240 Hz field rate were used. Automatic scanning of the Kinovea® and APAS® software was used. The accuracy was determined through mean absolute error, maximum absolute error, mean standard deviation of absolute error, and the variability values. The reliability was calculated with the Intra-Class Correlation. The mean absolute error (3.62 to 3.78 mm), maximum absolute error (9.09 to 11.61 mm), mean standard deviation of absolute error (0.01 to 0.59 mm), variability values (<1 mm), and the auto-digitalization time (16 to 31 minutes) were calculated and were within the values obtained in the literature. The reliability was determined with the Intra-Class Correlation, and the results were higher than 0.90, warrantied a high consistency for chosen software. The accuracy and the reliability intra-operator were determined, and the results indicated that it is possible to apply the free 3D camera system, with consistency in free-kick analysis. [ABSTRACT FROM AUTHOR]

Background and Objectives: Action recognition, as the processes of labeling an unknown action of a query video, is a challenging problem, due to the event complexity, variations in imaging conditions, and intra- and inter-individual action-variability. A number of solutions proposed to solve action recognition problem. Many of these frameworks suppose that each video sequence includes only one action class. Therefore, we need to break down a video sequence into sub-sequences, each containing only a single action class.Methods: In this paper, we develop an unsupervised action change detection method to detect the time of actions change, without classifying the actions. In this method, a silhouette-based framework will be used for action representation. This representation uses xt patterns. The xt pattern is a selected frame of xty volume. This volume is achieved by rotating the traditional space-time volume and displacing its axes. In xty volume, each frame consists of two axes (x) and time (t), and y value specifies the frame number.Results: To test the performance of the proposed method, we created 105 artificial videos using the Weizmann dataset, as well as time-continuous camera-captured video. The experiments have been conducted on this dataset. The precision of the proposed method was 98.13% and the recall was 100%.Conclusion: The proposed unsupervised approach can detect action changes with a high precision. Therefore, it can be useful in combination with an action recognition method for designing an integrated action recognition system.

Regardless of the type of sport, coaches must perform the difficult task of evaluating the performance of athletes. In some cases, this task is aided using technology which provides tools for this purpose. When the sport considered is taekwondo this scenario does not apply as the athlete evaluation methods used are mostly manual. Thus, the project presented in this paper has the main objective to develop a system that can be used as a tool to evaluate the performance of taekwondo athletes in real time, with special attention to the low cost of implementation and ease of use. With the intention of meeting these requirements, the developed system comprises a 3D camera with a depth sensor (Orbbec Astra), Inertial Measurement Units (IMUs) with accelerometer and gyroscope, a computer and specific software developed for this purpose. This system allows the collection of data from the athletes’ movements necessary for the creation of a dataset that is then analyzed and interpretated. The system permits the user to obtain real-time information about the speed, acceleration, and strength of the athlete’s limbs during training as well as the identification of some movements and their accounting. To achieve this functionality, deep-learning architecture models were used, more specifically long short-term memory (LSTM). The intention is to provide a new training methodology through faster feedback, so that a faster evolution of the athlete’s performance is possible, contributing to the technological development of the assessment practices used in taekwondo.

Human motion recognition has an important application value in scenarios such as intelligent monitoring and advanced human-computer interaction, and it is an important research direction in the field of computer vision. Traditional human motion recognition algorithms based on two-dimensional cameras are susceptible to changes in light intensity and texture. The advent of depth sensors, especially the Kinect series with good performance and low price released by Microsoft, enables extensive research based on depth information. However, to a large extent, the depth information has not overcome these problems based on two-dimensional images. This article introduces the research background and significance of human motion recognition technology based on depth information, introduces in detail the research methods of human motion recognition algorithms based on depth information at home and abroad, and analyzes their advantages and disadvantages. The public dataset is introduced. Then, based on the depth information, a method of human motion recognition is proposed and optimized. A moving human body image segmentation method based on an improved two-dimensional Otsu method is proposed to solve the problem of inaccurate and slow segmentation of moving human body images using the two-dimensional Otsu method. In the process of constructing the threshold recognition function, this algorithm not only uses the cohesion of the pixels within the class but also considers the maximum variance between the target class and the background class. Then, the quantum particle swarm algorithm is used to find the optimal threshold solution of the threshold recognition function. Finally, the optimal solution is used to achieve accurate and fast image segmentation, which increases the accuracy of human body motion tracking by more than 30%. [ABSTRACT FROM AUTHOR]

Computer vision is a significant component of human-computer interaction (HCI) processes in interactive control systems. In general, the interaction between humans and computers relies on the flexibility of the interactive visualization system. Electromyography (EMG) is a bioelectric signal used in HCI that can be captured noninvasively by placing electrodes on the human hand. Due to the impact of complex background, accurate recognition and analysis of human motion in real-time multitarget scenarios are considered challenging in HCI. Further, EMG signals of human hand motions are exceedingly nonlinear, and it is important to utilize a dynamic approach to address the noise problem in EMG signals. Hence, in this paper, the Optimized Noninvasive Human-Computer Interaction (ONIHCI) model has been proposed to predict human motion recognition. Average Intrinsic Mode Function (AIMF) has been used to reduce the noise factor in EMG signals. Furthermore, this paper introduces spatial thermographic imaging to overcome the conventional sensor problem, such as gesture recognition and human target identification in multitarget scenarios. The human motion behavior in spatial thermographic images is examined by target trajectory, and body movement kinematics is employed to classify human targets and objects. The experimental findings demonstrate that the proposed method reduces noise by 7.2% and improves accuracy by 97.2% in human motion recognition and human target identification. [ABSTRACT FROM AUTHOR]

Background: Several factors, including the aging population and the recent corona pandemic, have increased the need for cost effective, easy-to-use and reliable telerehabilitation services. Computer vision-based marker-less human pose estimation is a promising variant of telerehabilitation and is currently an intensive research topic. It has attracted significant interest for detailed motion analysis, as it does not need arrangement of external fiducials while capturing motion data from images. This is promising for rehabilitation applications, as they enable analysis and supervision of clients' exercises and reduce clients' need for visiting physiotherapists in person. However, development of a marker-less motion analysis system with precise accuracy for joint identification, joint angle measurements and advanced motion analysis is an open challenge. Objectives: The main objective of this paper is to provide a critical overview of recent computer vision-based marker-less human pose estimation systems and their applicability for rehabilitation application. An overview of some existing marker-less rehabilitation applications is also provided. Methods: This paper presents a critical review of recent computer vision-based marker-less human pose estimation systems with focus on their provided joint localization accuracy in comparison to physiotherapy requirements and ease of use. The accuracy, in terms of the capability to measure the knee angle, is analysed using simulation. Results: Current pose estimation systems use 2D, 3D, multiple and single view-based techniques. The most promising techniques from a physiotherapy point of view are 3D marker-less pose estimation based on a single view as these can perform advanced motion analysis of the human body while only requiring a single camera and a computing device. Preliminary simulations reveal that some proposed systems already provide a sufficient accuracy for 2D joint angle estimations. Conclusions: Even though test results of different applications for some proposed techniques are promising, more rigour testing is required for validating their accuracy before they can be widely adopted in advanced rehabilitation applications. [ABSTRACT FROM AUTHOR]