Search

Non-contact heart monitoring is crucial in advancing telemedicine, fitness tracking, and mass screening. Remote photoplethysmography (rPPG) is a non-contact technique to obtain information about heart pulse by analyzing the changes in the light intensity reflected or absorbed by the skin during the blood circulation cycle. However, this technique is sensitive to environmental lightning and different skin pigmentation, resulting in unreliable results. This research presents a multimodal approach to non-contact heart rate estimation by combining facial video and physical attributes, including age, gender, weight, height, and body mass index (BMI). For this purpose, we collected local datasets from 60 individuals containing a 1 min facial video and physical attributes such as age, gender, weight, and height, and we derived the BMI variable from the weight and height. We compare the performance of two machine learning models, support vector regression (SVR) and random forest regression on the multimodal dataset. The experimental results demonstrate that incorporating a multimodal approach enhances model performance, with the random forest model achieving superior results, yielding a mean absolute error (MAE) of 3.057 bpm, a root mean squared error (RMSE) of 10.532 bpm, and a mean absolute percentage error (MAPE) of 4.2% that outperforms the state-of-the-art rPPG methods. These findings highlight the potential for interpretable, non-contact, real-time heart rate measurement systems to contribute effectively to applications in telemedicine and mass screening. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) is a non-contact technology that monitors heart activity by detecting subtle color changes within the facial blood vessels. It provides an unconstrained and unconscious approach that can be widely applied to health monitoring systems. In recent years, research has been actively conducted to improve rPPG signals and to extract significant information from facial videos. However, rPPG can be vulnerable to degradation due to changes in the illumination and motion of a subject, and overcoming these challenges remains difficult. In this study, we propose a machine learning-based filter bank (MLFB) noise reduction algorithm to improve the quality of rPPG signals. The MLFB algorithm determines the optimal spectral band for extracting information on cardiovascular activity and reconstructing an rPPG signal using a support vector machine. The proposed approach was validated with an open dataset, achieving a 35.5% (i.e., resulting in a mean absolute error of 2.5 beats per minute) higher accuracy than those of conventional methods. The proposed algorithm can be integrated into various rPPG algorithms for the pre-processing of RGB signals. Moreover, its computational efficiency is expected to enable straightforward implementation in system development, making it broadly applicable across the healthcare field. [ABSTRACT FROM AUTHOR]

The quality of remote photoplethysmography (rPPG) signals presents a significant challenge in contactless optical blood pressure measurement. Feature and morphologybased approaches heavily rely on subtle changes in signal characteristics, but rPPG signals are highly susceptible to noise and interference. This study aims to evaluate rPPG signal quality by assessing correlation with a reference PPG and signal-to-noise ratio (SNR) across various color models and rPPG methods. Analyses are performed under different measurement conditions, accounting for common sources of signal artifacts. [ABSTRACT FROM AUTHOR]

Recent studies show the feasibility of using local remote photoplethysmography (rPPG) for non-contact blood perfusion assessment by creating spatial pulsatile blood perfusion maps. While global rPPG has been widely studied for its robustness, e.g. for non-contact measurement of heart rate, local analyses pose greater challenges in terms of noise suppression and thus reliability. In this paper, the effect of temperature and illumination changes on signal-to-noise ratio (SNR) perfusion maps is analysed. The results show the importance of consistent temperature and controlled illumination for improving SNR and ensuring reliable blood perfusion measurements using rPPG. This emphasises the need for standardised external conditions for accurate interpretation of the results and medical applicability. [ABSTRACT FROM AUTHOR]

Deepfake (DF) involves utilizing artificial intelligence (AI) technology to synthesize or manipulate images, voices, and other human or object data. However, recent times have seen a surge in instances of DF technology misuse, raising concerns about cybercrime and the credibility of manipulated information. The objective of this study is to devise a method that employs remote photoplethysmography (rPPG) biosignals for DF detection. The face was divided into five regions based on landmarks, with automatic extraction performed on the neck region. We conducted rPPG signal extraction from each facial area and the neck region was defined as the ground truth. The five signals extracted from the face were used as inputs to an support vector machine (SVM) model by calculating the euclidean distance between each signal and the signal extracted from the neck region, measuring rPPG signal similarity with five features. Our approach demonstrated robust performance with an area under the curve (AUC) score of 91.2% on the audio‐driven dataset and 99.7% on the face swapping generative adversarial network (FSGAN) dataset, even though we only used datasets excluding DF techniques that can be visually identified in Korean DF Detection Dataset (KoDF). Therefore, our research findings demonstrate that similarity features of rPPG signals can be utilized as key features for detecting DFs. [ABSTRACT FROM AUTHOR]

Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) is a low-cost technique to measure physiological parameters such as heart rate by analyzing videos of a person. There has been growing attention to this technique due to the increased possibilities and demand for running psychological experiments on online platforms. Technological advancements in commercially available cameras and video processing algorithms have led to significant progress in this field. However, despite these advancements, past research indicates that suboptimal video recording conditions can severely compromise the accuracy of rPPG. In this study, we aimed to develop an open-source rPPG methodology and test its performance on videos collected via an online platform, without control of the hardware of the participants and the contextual variables, such as illumination, distance, and motion. Across two experiments, we compared the results of the rPPG extraction methodology to a validated dataset used for rPPG testing. Furthermore, we then collected 231 online video recordings and compared the results of the rPPG extraction to finger pulse oximeter data acquired with a validated mobile heart rate application. Results indicated that the rPPG algorithm was highly accurate, showing a significant degree of convergence with both datasets thus providing an improved tool for recording and analyzing heart rate in online experiments. [ABSTRACT FROM AUTHOR]

Background and Objectives: Sepsis, a leading global health challenge, accounts for around 20% of deaths worldwide. The complexity of sepsis, especially the difference between bacterial and viral etiologies, requires an effective assessment of microcirculation during resuscitation. This study aimed to evaluate the impact of infusion therapy on microcirculation in patients with sepsis, focusing on bacterial- and COVID-19-associated sepsis using remote photoplethysmography (rPPG) and the automated capillary refill time (aCRT). Materials and Methods: This single-center prospective study was conducted in the ICU of Pauls Stradins Clinical University Hospital, including 20 patients with sepsis/septic shock. The patients were selected based on hemodynamic instability and divided into COVID-19 and Bacterial Septic Shock groups. Fluid responsiveness was assessed using the Passive Leg Raising Test (PLRT). Systemic hemodynamics and microcirculation were monitored through MAP CRT, rPPG, and serum lactate levels. Statistical analyses compared responses within and between the groups across different stages of the protocol. Results: The Bacterial group exhibited higher initial serum lactate levels and more pronounced microcirculatory dysfunction than the COVID-19 group. rPPG was more sensitive in detecting perfusion changes, showing significant differences between the groups. The automated CRT demonstrated greater sensitivity compared to the manual CRT, revealing significant differences during PLRT stages between bacterial- and COVID-19-associated sepsis. Both groups had a transient hemodynamic response to PLRT, with subsequent stabilization upon fluid infusion. Conclusions: When managing patients with sepsis in intensive care, monitoring microcirculation is of paramount importance in infusion therapy. Our study highlights the potential of rPPG and aCRT as tools for this purpose. These techniques can be used in conjunction with routine parameters, such as lactate levels and systemic hemodynamic parameters, to provide a comprehensive assessment of a patient's condition. [ABSTRACT FROM AUTHOR]

Audio presentation is often attributed as being capable of influencing a viewer's feeling of immersion during an audiovisual experience. However, there is limited empirical research supporting this claim. This study aimed to explore this effect by presenting a clip renowned for its immersive soundtrack to two groups of participants with either high-end or basic audio presentation. To measure immersion, a novel method is applied, which utilises a camera instead of an electroencephalogram (ECG) for acquiring a heart rate synchronisation feature. The results of the study showed no difference in the feature, or in the responses to an established immersion questionnaire, between the two groups of participants. However, the camera-sourced HR synchronicity feature correlated with the results of the immersion questionnaire. Moreover, the camera-sourced HR synchronicity feature was found to correlate with an equivalent feature sourced from synchronously recorded ECG data. Hence, this shows the viability of using a camera instead of an ECG sensor to quantify heart rate synchronisation but suggests that audio presentation alone is not capable of eliciting a measurable difference in the feeling of immersion in this context. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) is an emerging non-contact method for monitoring cardiovascular health based on facial videos. The quality of the captured videos largely determines the efficacy of rPPG in this application. Traditional rPPG techniques, while effective for heart rate (HR) estimation, often produce signals with an inadequate signal-to-noise ratio (SNR) for reliable vital sign measurement due to artifacts like head motion and measurement noise. Another pivotal factor is the overlooking of the inherent properties of signals generated by rPPG (rPPG-signals). To address these limitations, we introduce DiffPhys, a novel deep generative model particularly designed to enhance the SNR of rPPG-signals. DiffPhys leverages the conditional diffusion model to learn the distribution of rPPG-signals and uses a refined reverse process to generate rPPG-signals with a higher SNR. Experimental results demonstrate that DiffPhys elevates the SNR of rPPG-signals across within-database and cross-database scenarios, facilitating the extraction of cardiovascular metrics such as HR and HRV with greater precision. This enhancement allows for more accurate monitoring of health conditions in non-clinical settings. [ABSTRACT FROM AUTHOR]

Abstract Among the causes of the annually traffic accidents, driving fatigue is the main culprit. In consequence, it is of great practical significance to carry out the research of driving fatigue detection and early warning system. However, there are still two problems in the latest methods of driving fatigue detection: one is that a single information cannot precisely reflect the actual state of the driver in different fatigue phases, another one is the detection effect is not very well or even difficult to detect under abnormal illumination. In this paper, the multi-task cascaded convolutional networks (MTCNN) and infrared-based remote photo-plethysmography (rPPG) theory are used to extract the driver’s facial and physiological information, and the multi-modal specific fatigue information is deeply excavated, and the multi-modal feature fusion model is constructed to comprehensively analyze the driver’s fatigue variation tendency. Aiming at the matter of low detection accuracy under abnormal illumination, the multi-modal features extracted from visible light images and infrared images are fused by multi-loss reconstruction (MLR) module, and the driving fatigue detection module is established which is based on Bi-LSTM model by utilizing fatigue timing. The experiments were validated under all-weather illumination scenarios and were carried out on the datasets NTHU-DDD, UTA-RLDDD and FAHD. The results show that the multi-modal driving fatigue detection model has better performance than the single-modal model, and the accuracy is improved by 8.1%. In the abnormal illumination such as strong and weak light, the accuracy of the method can reach 91.7% at the highest and 83.6% at the lowest. Meanwhile, in the normal illumination, it can reach 93.2%.

Copyright of Telecommunication Engineering is the property of Telecommunication Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Among the causes of the annually traffic accidents, driving fatigue is the main culprit. In consequence, it is of great practical significance to carry out the research of driving fatigue detection and early warning system. However, there are still two problems in the latest methods of driving fatigue detection: one is that a single information cannot precisely reflect the actual state of the driver in different fatigue phases, another one is the detection effect is not very well or even difficult to detect under abnormal illumination. In this paper, the multi-task cascaded convolutional networks (MTCNN) and infrared-based remote photo-plethysmography (rPPG) theory are used to extract the driver’s facial and physiological information, and the multi-modal specific fatigue information is deeply excavated, and the multi-modal feature fusion model is constructed to comprehensively analyze the driver’s fatigue variation tendency. Aiming at the matter of low detection accuracy under abnormal illumination, the multi-modal features extracted from visible light images and infrared images are fused by multi-loss reconstruction (MLR) module, and the driving fatigue detection module is established which is based on Bi-LSTM model by utilizing fatigue timing. The experiments were validated under all-weather illumination scenarios and were carried out on the datasets NTHU-DDD, UTA-RLDDD and FAHD. The results show that the multi-modal driving fatigue detection model has better performance than the single-modal model, and the accuracy is improved by 8.1%. In the abnormal illumination such as strong and weak light, the accuracy of the method can reach 91.7% at the highest and 83.6% at the lowest. Meanwhile, in the normal illumination, it can reach 93.2%. [ABSTRACT FROM AUTHOR]

Deepfake-generated fake faces, commonly utilized in identity-related activities such as political propaganda, celebrity impersonations, evidence forgery, and familiar fraud, pose new societal threats. Although current deepfake generators strive for high realism in visual effects, they do not replicate biometric signals indicative of cardiac activity. Addressing this gap, many researchers have developed detection methods focusing on biometric characteristics. These methods utilize classification networks to analyze both temporal and spectral domain features of the remote photoplethysmography (rPPG) signal, resulting in high detection accuracy. However, in the spectral analysis, existing approaches often only consider the power spectral density and neglect the amplitude spectrum--both crucial for assessing cardiac activity. We introduce a novel method that extracts rPPG signals from multiple regions of interest through remote photoplethysmography and processes themusing Fast Fourier Transform (FFT). The resultant time-frequency domain signal samples are organized into matrices to create Matrix Visualization Heatmaps (MVHM), which are then utilized to train an image classification network. Additionally, we explored various combinations of time-frequency domain representations of rPPG signals and the impact of attention mechanisms. Our experimental results show that our algorithm achieves a remarkable detection accuracy of 99.22% in identifying fake videos, significantly outperforming main stream algorithms and demonstrating the effectiveness of Fourier Transform and attention mechanisms in detecting fake faces. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) technology is a non-contact physiological signal measurement method, characterized by non-invasiveness and ease of use. It has broad application potential in medical health, human factors engineering, and other fields. However, current rPPG technology is highly susceptible to variations in lighting conditions, head pose changes, and partial occlusions, posing significant challenges for its widespread application. In order to improve the accuracy of remote heart rate estimation and enhance model generalization, we propose PulseFormer, a dual-path network based on transformer. By integrating local and global information and utilizing fast and slow paths, PulseFormer effectively captures the temporal variations of key regions and spatial variations of the global area, facilitating the extraction of rPPG feature information while mitigating the impact of background noise variations. Heart rate estimation results on the popular rPPG dataset show that PulseFormer achieves state-of-the-art performance on public datasets. Additionally, we establish a dataset containing facial expressions and synchronized physiological signals in driving scenarios and test the pre-trained model from the public dataset on this collected dataset. The results indicate that PulseFormer exhibits strong generalization capabilities across different data distributions in cross-scenario settings. Therefore, this model is applicable for heart rate estimation of individuals in various scenarios. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) gains recent great interest due to its potential in contactless heart rate measurement using consumer-level cameras. This paper presents a detailed review of rPPG measurement using computer vision and deep learning techniques for heart rate estimation. Several common gaps and difficulties of rPPG development are highlighted for the feasibility study in real-world applications. Numerous computer vision and deep learning methods are reviewed to mitigate crucial issues such as motion artifact and illumination variation. In comparison, deep learning approaches are proven more accurate than conventional computer vision methods due to their adaptive pattern learning and generalization characteristics. An increasing trend of applying deep learning techniques in rPPG can improve effective heart rate estimation and artifact removal. To consider more realistic disturbances into account, additional vital signs and large training datasets are crucial to improve the accuracy of heart rate estimations. By taking the benefit of contactless and accurate estimation, the application of rPPG can be greatly adopted in real-world activities, especially in precision sports. [ABSTRACT FROM AUTHOR]

Due to its non-contact characteristics, remote photoplethysmography (rPPG) has attracted widespread attention in recent years, and has been widely applied for remote physiological measurements. However, most of the existing rPPG models are unable to estimate multiple physiological signals simultaneously, and the performance of the limited available multi-task models is also restricted due to their single-model architectures. To address the above problems, this study proposes MultiPhys, adopting a heterogeneous network fusion approach for its development. Specifically, a Convolutional Neural Network (CNN) is used to quickly extract local features in the early stage, a transformer captures global context and long-distance dependencies, and Mamba is used to compensate for the transformer’s deficiencies, reducing the computational complexity and improving the accuracy of the model. Additionally, a gate is utilized for feature selection, which classifies the features of different physiological indicators. Finally, physiological indicators are estimated after passing features to each task-related head. Experiments on three datasets show that MultiPhys has superior performance in handling multiple tasks. The results of cross-dataset and hyper-parameter sensitivity tests also verify its generalization ability and robustness, respectively. MultiPhys can be considered as an effective solution for remote physiological estimation, thus promoting the development of this field.

Remote photoplethysmography (rPPG) is a non-contact technology that monitors heart activity by detecting subtle color changes within the facial blood vessels. It provides an unconstrained and unconscious approach that can be widely applied to health monitoring systems. In recent years, research has been actively conducted to improve rPPG signals and to extract significant information from facial videos. However, rPPG can be vulnerable to degradation due to changes in the illumination and motion of a subject, and overcoming these challenges remains difficult. In this study, we propose a machine learning-based filter bank (MLFB) noise reduction algorithm to improve the quality of rPPG signals. The MLFB algorithm determines the optimal spectral band for extracting information on cardiovascular activity and reconstructing an rPPG signal using a support vector machine. The proposed approach was validated with an open dataset, achieving a 35.5% (i.e., resulting in a mean absolute error of 2.5 beats per minute) higher accuracy than those of conventional methods. The proposed algorithm can be integrated into various rPPG algorithms for the pre-processing of RGB signals. Moreover, its computational efficiency is expected to enable straightforward implementation in system development, making it broadly applicable across the healthcare field.

Non-contact heart monitoring is crucial in advancing telemedicine, fitness tracking, and mass screening. Remote photoplethysmography (rPPG) is a non-contact technique to obtain information about heart pulse by analyzing the changes in the light intensity reflected or absorbed by the skin during the blood circulation cycle. However, this technique is sensitive to environmental lightning and different skin pigmentation, resulting in unreliable results. This research presents a multimodal approach to non-contact heart rate estimation by combining facial video and physical attributes, including age, gender, weight, height, and body mass index (BMI). For this purpose, we collected local datasets from 60 individuals containing a 1 min facial video and physical attributes such as age, gender, weight, and height, and we derived the BMI variable from the weight and height. We compare the performance of two machine learning models, support vector regression (SVR) and random forest regression on the multimodal dataset. The experimental results demonstrate that incorporating a multimodal approach enhances model performance, with the random forest model achieving superior results, yielding a mean absolute error (MAE) of 3.057 bpm, a root mean squared error (RMSE) of 10.532 bpm, and a mean absolute percentage error (MAPE) of 4.2% that outperforms the state-of-the-art rPPG methods. These findings highlight the potential for interpretable, non-contact, real-time heart rate measurement systems to contribute effectively to applications in telemedicine and mass screening.

Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that heart rate (HR) and skin temperature are effective in detecting autonomic nervous system (ANS) activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote photoplethysmography (r-PPG) and automatic thermal region of interest (ROI) detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-min questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by a support vector machine (SVM). Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a random forest (RF) algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHapley Additive exPlanations (SHAP), highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging's higher predictive accuracy in unimodal analysis. [ABSTRACT FROM AUTHOR]

Deception detection is essential for protecting the public interest and maintaining social order. Its application in various fields helps to establish a safer and trustworthy social environment. This study focuses on the problem of deception detection in videos and proposes a visual deception detection method based on a capsule network (DDCapsNet). The DDCapsNet model predicts deception classification using the fusion of facial expression features and video-based heart rate feature via a channel attention mechanism. Supervised contrastive learning is further introduced to enhance the generalization ability of the DDCapsNet. The proposed model is evaluated on a self-collected dataset (physiological-assisted visual deception detection dataset, PV3D) and the public Bag-of-Lies (BOL) dataset, respectively. The results show that DDCapsNet outperforms the unimodal system and other state-of-the-art (SOTA) methods, where the ACC reaches 77.97% and the AUC reaches 78.45% on PV3D, and the ACC reaches 73.19% and the AUC reaches 72.78% on BOL dataset.

Background and Objectives: Sepsis, a leading global health challenge, accounts for around 20% of deaths worldwide. The complexity of sepsis, especially the difference between bacterial and viral etiologies, requires an effective assessment of microcirculation during resuscitation. This study aimed to evaluate the impact of infusion therapy on microcirculation in patients with sepsis, focusing on bacterial- and COVID-19-associated sepsis using remote photoplethysmography (rPPG) and the automated capillary refill time (aCRT). Materials and Methods: This single-center prospective study was conducted in the ICU of Pauls Stradins Clinical University Hospital, including 20 patients with sepsis/septic shock. The patients were selected based on hemodynamic instability and divided into COVID-19 and Bacterial Septic Shock groups. Fluid responsiveness was assessed using the Passive Leg Raising Test (PLRT). Systemic hemodynamics and microcirculation were monitored through MAP CRT, rPPG, and serum lactate levels. Statistical analyses compared responses within and between the groups across different stages of the protocol. Results: The Bacterial group exhibited higher initial serum lactate levels and more pronounced microcirculatory dysfunction than the COVID-19 group. rPPG was more sensitive in detecting perfusion changes, showing significant differences between the groups. The automated CRT demonstrated greater sensitivity compared to the manual CRT, revealing significant differences during PLRT stages between bacterial- and COVID-19-associated sepsis. Both groups had a transient hemodynamic response to PLRT, with subsequent stabilization upon fluid infusion. Conclusions: When managing patients with sepsis in intensive care, monitoring microcirculation is of paramount importance in infusion therapy. Our study highlights the potential of rPPG and aCRT as tools for this purpose. These techniques can be used in conjunction with routine parameters, such as lactate levels and systemic hemodynamic parameters, to provide a comprehensive assessment of a patient’s condition.

Audio presentation is often attributed as being capable of influencing a viewer’s feeling of immersion during an audiovisual experience. However, there is limited empirical research supporting this claim. This study aimed to explore this effect by presenting a clip renowned for its immersive soundtrack to two groups of participants with either high-end or basic audio presentation. To measure immersion, a novel method is applied, which utilises a camera instead of an electroencephalogram (ECG) for acquiring a heart rate synchronisation feature. The results of the study showed no difference in the feature, or in the responses to an established immersion questionnaire, between the two groups of participants. However, the camera-sourced HR synchronicity feature correlated with the results of the immersion questionnaire. Moreover, the camera-sourced HR synchronicity feature was found to correlate with an equivalent feature sourced from synchronously recorded ECG data. Hence, this shows the viability of using a camera instead of an ECG sensor to quantify heart rate synchronisation but suggests that audio presentation alone is not capable of eliciting a measurable difference in the feeling of immersion in this context.

Heart rate estimation from face videos is an emerging technology that offers numerous potential applications in healthcare and human–computer interaction. However, most of the existing approaches often overlook the importance of long-range spatiotemporal dependencies, which is essential for robust measurement of heart rate prediction. Additionally, they involve extensive pre-processing steps to enhance the prediction accuracy, resulting in high computational complexity. In this paper, we propose an innovative solution called LGTransPPG. This end-to-end transformer-based framework eliminates the need for pre-processing steps while achieving improved efficiency and accuracy. LGTransPPG incorporates local and global aggregation techniques to capture fine-grained facial features and contextual information. By leveraging the power of transformers, our framework can effectively model long-range dependencies and temporal dynamics, enhancing the heart rate prediction process. The proposed approach is evaluated on three publicly available datasets, demonstrating its robustness and generalizability. Furthermore, we achieved a high Pearson correlation coefficient (PCC) value of 0.88, indicating its superior efficiency and accuracy between the predicted and actual heart rate values. [ABSTRACT FROM AUTHOR]

Non-contact remote photoplethysmography can be used in a variety of medical and healthcare fields by measuring vital signs continuously and unobtrusively. Recently, end-to-end deep learning methods have been proposed to replace the existing handcrafted features. However, since the existing deep learning methods are known as black box models, the problem of interpretability has been raised, and the same problem exists in the remote photoplethysmography (rPPG) network. In this study, we propose a method to visualize temporal and spectral representations for hidden layers, deeply supervise the spectral representation of intermediate layers through the depth of networks and optimize it for a lightweight model. The optimized network improves performance and enables fast training and inference times. The proposed spectral deep supervision helps to achieve not only high performance but also fast convergence speed through the regularization of the intermediate layers. The effect of the proposed methods was confirmed through a thorough ablation study on public datasets. As a result, similar or outperforming results were obtained in comparison to state-of-the-art models. In particular, our model achieved an RMSE of 1 bpm on the PURE dataset, demonstrating its high accuracy. Moreover, it excelled on the V4V dataset with an impressive RMSE of 6.65 bpm, outperforming other methods. We observe that our model began converging from the very first epoch, a significant improvement over other models in terms of learning efficiency. Our approach is expected to be generally applicable to models that learn spectral domain information as well as to the applications of regression that require the representations of periodicity. [ABSTRACT FROM AUTHOR]

In health care applications, an evolution of electronics has made drastic advancements. There are some problems created due to this advancement. To estimate the coronary heart rate, till date some problems have been confronted. To overcome these issues, remote photoplethysmography (RPPG) technology is used to determine the heart rate (HR) and respiratory rate (RR) by using normal web cameras, without any additional hardware. Here, a high resolution camera detects the face using a face detector by means of image processing techniques. Hardware part is only used to display the heart rate and respiratory rate using sensors. The performance analysis demonstrates the practicality of the patients. Experimental results of heart rate measurement show that the proposed dynamic ROI method for RIPPG can effectively improve the RIPPG signal quality, compared with the state-of-the-art ROI methods for RIPPG. Objective performance tests show strong correlation with the ground truth values for the estimated heart rate and variation. [ABSTRACT FROM AUTHOR]

Heart rate monitoring is crucial in scientific and technical fields as it provides essential information about cardiovascular health, exercise performance, and stress levels, enabling early detection of and intervention for potential cardiac abnormalities or risks. Traditional methods for measuring heart rate often require direct contact with the body, which can be invasive and inconvenient. In this analysis, we have studied the remote photoplethysmography (rPPG) techniques for predicting heart wellness using different machine algorithms. To evaluate the effectiveness of different rPPG methods, we conducted a study with a diverse sample of 20 participants. We considered factors such as gender, skin texture (based on participants from India and Sierra Leone), and age group. By collecting data from various PPG and rPPG methods, we aimed to determine the most accurate technique for heart rate prediction. To accomplish this, we employed two machine learning algorithms: Lasso Regression and Random Forest Regression. These algorithms were trained on the collected heart rate data to predict and compare the performance of different rPPG methods. Our research findings indicate that both Random Forest Regression and Lasso Regression models exhibit promising results in predicting heart rate non-invasively and accurately. The Random Forest Regression model achieved an average mean square error of 3.193 and a coefficient of determination value of 0.885, while the Lasso Regression model achieved an average mean square error of 33.336 and a coefficient of determination, R2, value of 0.086. The relatively low Mean Squared Error (MSE) and high (R-squared) R2 values obtained from the Random Forest Regression model demonstrate its superior predictive performance compared to the Lasso Regression model. This suggests that the Random Forest algorithm is better suited for analyzing the collected heart rate prediction dataset using rPPG features. Our research findings underscore the potential of remote photoplethysmography (rPPG) and machine learning algorithms in predicting heart rate non-invasively. We have successfully analyzed the study method across different genders, regions, and skin colors. Moreover, our study emphasizes the significance of considering factors such as skin color pigments and their impact on the accuracy of heart rate predictions. By recognizing the influence of these factors, we can further refine and improve the performance of rPPG-based heart rate monitoring systems. [ABSTRACT FROM AUTHOR]

Remote photoplethysmography (rPPG) is an emerging non-contact method for monitoring cardiovascular health based on facial videos. The quality of the captured videos largely determines the efficacy of rPPG in this application. Traditional rPPG techniques, while effective for heart rate (HR) estimation, often produce signals with an inadequate signal-to-noise ratio (SNR) for reliable vital sign measurement due to artifacts like head motion and measurement noise. Another pivotal factor is the overlooking of the inherent properties of signals generated by rPPG (rPPG-signals). To address these limitations, we introduce DiffPhys, a novel deep generative model particularly designed to enhance the SNR of rPPG-signals. DiffPhys leverages the conditional diffusion model to learn the distribution of rPPG-signals and uses a refined reverse process to generate rPPG-signals with a higher SNR. Experimental results demonstrate that DiffPhys elevates the SNR of rPPG-signals across within-database and cross-database scenarios, facilitating the extraction of cardiovascular metrics such as HR and HRV with greater precision. This enhancement allows for more accurate monitoring of health conditions in non-clinical settings.

Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively. [ABSTRACT FROM AUTHOR]

Heart rate variability (HRV) is an important indicator of autonomic nervous system activity and can be used for the identification of affective states. The development of remote Photoplethysmography (rPPG) technology has made it possible to measure pulse rate variability (PRV) using a camera without any sensor-skin contact, which is highly correlated to HRV, thus, enabling contactless assessment of emotional states. In this study, we employed ten machine learning techniques to identify emotions using camera-based PRV features. Our experimental results show that the best classification model achieved a coordination correlation coefficient of 0.34 for value recognition and 0.36 for arousal recognition. The rPPG-based measurement has demonstrated promising results in detecting HAHV (high-arousal high-valence) emotions with high accuracy. Furthermore, for emotions with less noticeable variations, such as sadness, the rPPG-based measure outperformed the baseline deep network for facial expression analysis. • Features for emotion recognition were extracted from videos using rPPG. • Recognition based on rPPG shows correlation with annotation. • The rPPG-based approach is effective in detecting positive emotions. • RPPG outperforms the recognition of sad emotions based on facial expressions. [ABSTRACT FROM AUTHOR]

Complex emotion is an aggregate of two or more others which has highly variable appearances, inter-dependence, and affective dynamics.These properties make the recognition hard to handle via existing recognition techniques like action units or valence-arousal detection. In this study, we propose a bionic two-system structure for complex emotion recognition. The structure mimics the working theory of the human brain responding to problems decision-making. System I is a fast compound sensing module. System II is a slower cognitive decision module that processes data more integratively. System I contains one branch for facial expression feature representation including basic emotion, action units, and valence arousal detection and one for physiological measurement which is an image-only implementation for practicality. In System II, a decision module with segmentation is employed to ensure the chosen period including the emotion occurrence and iteratively optimize the emotion information in a given segment via reinforcement learning. The proposed method outperforms state-of-the-art on emotion recognition tasks with an accuracy of 94.15% in basic emotion recognition on the BP4D and an accuracy of 68.75% for binary valence arousal classification on the DEAP. For a subset of complex emotions, the recognition accuracy exceeds 70% on both databases, that is a significant improvement. [ABSTRACT FROM AUTHOR]

Depression is a mental illness that may be harmful to an individual's health. The detection of mental health disorders in the early stages and a precise diagnosis are critical to avoid social, physiological, or psychological side effects. This work analyzes physiological signals to observe if different depressive states have a noticeable impact on the blood volume pulse (BVP) and the heart rate variability (HRV) response. Although typically, HRV features are calculated from biosignals obtained with contact-based sensors such as wearables, we propose instead a novel scheme that directly extracts them from facial videos, just based on visual information, removing the need for any contact-based device. Our solution is based on a pipeline that is able to extract complete remote photoplethysmography signals (rPPG) in a fully unsupervised manner. We use these rPPG signals to calculate over 60 statistical, geometrical, and physiological features that are further used to train several machine learning regressors to recognize different levels of depression. Experiments on two benchmark datasets indicate that this approach offers comparable results to other audiovisual modalities based on voice or facial expression, potentially complementing them. In addition, the results achieved for the proposed method show promising and solid performance that outperforms hand-engineered methods and is comparable to deep learning-based approaches. [ABSTRACT FROM AUTHOR]

Chronic pain is a prevalent condition affecting approximately one-fifth of the global population, with significant impacts on quality of life and work productivity. Small fiber neuropathies are a common cause of chronic pain, and current diagnostic methods rely on subjective self-assessment or invasive skin biopsies, highlighting the need for objective noninvasive assessment methods. The study aims to develop a modular prototype of a contactless photoplethysmography system with three spectral bands (420, 540, and 800 nm) and evaluate its potential for assessing peripheral neuropathy patients via a skin topical heating test and spectral analyses of cutaneous flowmotions. The foot topical skin heating test was conducted on thirty volunteers, including fifteen healthy subjects and fifteen neuropathic patients. Four cutaneous nerve fiber characterizing parameters were evaluated at different wavelengths, including vasomotor response trend, flare area, flare intensity index, and the spectral power of cutaneous flowmotions. The results show that neuropathic patients had significantly lower vasomotor response (50%), flare area (63%), flare intensity index (19%), and neurogenic component (54%) of cutaneous flowmotions compared to the control group, independent of photoplethysmography spectral band. An absolute value of perfusion was 20%-30% higher in the 420 nm band. Imaging photoplethysmography shows potential as a cost-effective alternative for objective and non-invasive assessment of neuropathic patients, but further research is needed to enhance photoplethysmography signal quality and establish diagnostic criteria. [ABSTRACT FROM AUTHOR]

Blood pressure is a fundamental health metric widely employed to predict cardiac diseases and monitor overall well-being. However, conventional blood pressure measurement methods, such as the cuff method, necessitate additional equipment and can be inconvenient for regular use. This study aimed to develop a novel approach to blood pressure measurement using only an RGB camera, which promises enhanced convenience and accuracy. We employed the U-Net Basic generative model to achieve our objectives. Through rigorous experimentation and data analysis, our approach demonstrated promising results, attaining BHS (British Hypertension Society) baseline performance with grade A accuracy for diastolic blood pressure (DBP) and grade C accuracy for systolic blood pressure (SBP). The mean absolute error (MAE) achieved for DBP was 4.43 mmHg, and for SBP, it was 6.9 mmHg. Our findings indicate that blood pressure measurement using an RGB camera shows significant potential and may be utilized as an alternative or supplementary method for blood pressure monitoring. The convenience of using a commonly available RGB camera without additional specialized equipment can empower individuals to track their blood pressure regularly and proactively predict potential heart-related issues. [ABSTRACT FROM AUTHOR]