Search

We developed a system to improve the quality of telemedicine, and the test results obtained have been presented in this paper, along with the technical details of the system. The spread of COVID-19 has accelerated the need for telemedicine to effectively prevent infections. However, in traditional Japanese medicine (Kampo), where color is essential, an accurate diagnosis cannot be made without color reproduction. Because commercial smartphones cannot reproduce colors with the level of fidelity required for medical treatments, we created a color chart that includes the human skin and tongue colors to help doctors identify their colors accurately during a telemedicine examination. Further, we developed a telemedicine system that allows for automatic color correction using a mobile device, with a color chart and non-contact heart rate measurements.

Noncontact camera-based methods for measuring the pulse rate have been introduced in recent years. However, in previous studies, pulse-rate measurement experiments have been conducted only for adults and/or sleeping infants, and not for infants, who are awake. In this study, we compare the principal noncontact camera-based methods for pulse-rate measurement and identify a suitable method for infants, who are awake. We measure the pulse rate of five adults and three infants, and obtain the success rate of setting the measurement region and the root mean square error (RMSE) of the estimated pulse rate using several existing methods. Based on the obtained results, the color-based method and independent component analysis are determined as the most suitable for setting the measurement region and for signal processing, respectively, for infants, who are awake.

In this paper, we investigate the possibility of using image-based pulse transit time (iPTT) to estimate blood pressure using an RGB (red, green, blue)-camera video taken from the face only. It is generally known that there is a correlation between iPTT and blood pressure. iPTT refers to the difference between the times when a pulse wave reaches two different parts of the body. Therefore, it is possible to estimate blood pressure without contact, with iPTTs obtained from the RGB camera. However, this method requires simultaneous capture of images of the face and palm. Recording of such videos is limited by the subject’s posture. Hence, it is very difficult to achieve blood pressure estimation in a real-world environment. To solve these problems, this study examined whether it is possible to measure pulse wave propagation based on a single part of the body. The results showed that there was a time difference between the pulse waves at the forehead and the chin, thus it was possible to measure iPTT using only the face. Pulse waves were obtained from the chin and forehead, and the correlations between iPTT and blood pressure from the acquired pulse waves were examined. The results showed a negative correlation between these iPTT measures and blood pressure. The results suggest that simple, non-contact blood pressure estimation will be possible in the future.

Tongue diagnosis is an important diagnostic method in Japanese-traditional (Kampo) medicine, and the tongue is considered as a mirror reflecting the internal state of the body. Imbalance of water, heat, bloodstream, and to diagnose blood stagnation, blood deficiency, yin deficiency and so on in Kampo medicine can be diagnosed with tongue diagnosis. In this article, principal component analysis (PCA) of the hyperspectral images of tongues and analysis of correlation between its color and values of blood-biochemical data were performed to turn the empirical observation into the change in substantial factor. One hundred hyperspectral tongue images and blood samples were acquired from 50 outpatients, and the uncoated tongue region was segmented automatically. The spectral information of the uncoated tongue area was analyzed by principal component analysis (PCA). The component vector most representative of each blood-biochemical data was found by rotating the vector on a plane spanned by two arbitrary principal component vectors. Spectra of the tongue area without coating were extracted, the principal component analysis was performed, and correlation to each item of blood-chemistry was calculated with the vector rotation method. Creatinine, PDW, Chloride, MPV, FT4, RBC, γ-GTP, FBS, and AST showed large correlation to the tongue color spectra. We could determine correlations between tongue color spectra and items of blood-biochemistry. Our method would help non-invasive screening, and this tool should help non-trained people identify health status using concept of Kampo medicine.

In this study, we verified whether or not feature values of the facial skin texture can be used for facial complexion diagnosis. A total of 10 patients with fatigue were recruited and categorized under two groups: acupuncture group and blank control group. In the acupuncture group, the contact needle technique was used, while in the blank control group, the patients rested for 30 min in the supine position. We captured images of the facial skin using a digital camera before and after the intervention and then analyzed feature values from the images. The standard deviation (SD) of short-line inclination angles significantly decreased with acupuncture treatment. There was a significant difference in the SD of short-line inclination angles between the two groups. This study was approved by the Clinical Research Ethics Committee of Kanazawa University Hospital in Japan [for human study: approval no. 2018-154 (2912)] in December 2018.

To improve skin condition, it is considered to be important to make good blood circulation in the epidermis and papillary dermis. In Japan, acupuncture becomes more and more popular to improve facial skin condition. We have been used to one of the traditional Japanese acupuncture methods, Tohokai method, especially Contact Needle Therapy (CNT), for various diseases. The purpose of this study is to investigate the effect of CNT by evaluating changes in blood flow. The skin of four healthy males who have no scratches or disease on the skin of the measurement part (inner forearm) were measured with Optical Coherence Tomography (OCT) before CNT, during CNT, immediately after CNT, and one minute after CNT. OCT is an imaging technique that uses low-coherence light to capture micrometer-resolution, two- and three-dimensional images from within optical scattering media (e.g., biological tissue). CNT consists of a root intervention according to the meridian theory and branch intervention for local effects. Statistical analyses were performed using two-sample paired t test and Wilcoxon Signed-rank test. OCT blood flow signal (150–500 μm) was gradually increased during CNT, and significantly increased immediately after CNT compared with that before treatment (P

In this paper, we propose a color reproduction method using color charts to improve the color quality of a telemedicine system. Owing to the spread of COVID-19, the need for telemedicine is rapidly increasing to prevent infections more effectively. However, in practices such as traditional Japanese (Kampo) medicine, where color is used as an important examination factor, an accurate diagnosis cannot be made without adequate color reproduction. In telemedicine using a commercially available smartphone, color reproducibility may deteriorate owing to differences in the devices and lighting, which may result in a misdiagnosis. Therefore, we created a color chart that includes the colors of the human skin and tongue as a tool to help doctors identify the color of patients more accurately when conducting a telemedicine examination. Through a subjective evaluation by eight medical doctors, it was unanimously found that the proposed method is practical in terms of a color examination. The developed color chart can also be used for an automatic color correction.

In this study, we verified the effectiveness of Kampo medicine by evaluating the changes in the feature values of facial skin texture and microcirculation at two distinct tissue depths (subcutaneous 2 mm and 8 mm). A total of 80 patients who took the Kampo formula participated in this study, and the changes in the feature values of facial skin texture and microcirculation were measured before and after Kampo treatment. The treatment period lasted 6–18 months, according to the doctor’s judgment. The total area of the sulci cutis and the average thickness of the sulci cutis significantly decreased (P

The aim of this study was to propose a method to assess images of the tongue captured using a polarized light camera for diagnostic use in Kampo medicine. Glossy and non-glossy images of the tongue were captured simultaneously using a polarizing camera and a polarizing plate. Data augmentation was performed by modulating the color and gloss, resulting in an increase in the number of images from 11 to 275. To create a data set, the values for which diseases were evaluated by Kampo doctors for all tongue images were taken as the correct values and combined with the features extracted from the tongue images. Using this data set, we constructed a diagnostic support module to evaluate diseases. The resulting mean absolute error of the assessment was 0.44 for qi deficiency, 0.42 for blood deficiency, 0.33 for blood stagnation, 0.36 for yin deficiency, and 0.55 for fluid stagnation, suggesting that the diagnostic assistance module was accurate, and our proposed learning and data augmentation methods were effective.

This study is to compare the effect of contacting needle technique (CNT) and insertion needle technique (INT) on cardiovascular dynamics and visual analogue scale (VAS) in patients with shoulder pain. A total of 11 patients (9 females, 2 males, average age 32.27) were recruited and divided into two groups (CNT group and INT group). The treatment was performed once a week and a total of 4 weeks. The changes in cardiovascular circulation dynamics were detected at baseline, during the treatment and after the treatment. Pain was assessed before and after acupuncture therapy. There was significant difference in VAS within each group. There was no significant difference between the two groups on CO, SV, BPs, BPd and VAS, and had significant difference on PR (P

The change in the edema condition is visualized considering the three-dimensional shape. Continuous treatment and observation are indispensable for patients with edema. The measurement and evaluation of the three-dimensional shape of the leg are thus important in evaluating edema of the leg. Such an evaluation can confirm the therapeutic effect and assist in the planning of treatment by confirming the change in local capacity. Additionally, the depth camera of Structure Sensor used this study is feasible for use in home care systems due to its very low cost compared with other depth cameras. We obtain a point cloud of the leg and register shape models. We conducted an experiment to measure legs swathed and not swathed in bandages, with the former representing a leg with edema. In addition, for visualization of the edema condition, the change in shape was color coded according to the change obtained in the proposed analysis of the three-dimensional shape. Our experimental results show that our proposed visualization technique is effective in conveying the change in shape visually and clearly.

In the traditional Japanese (Kampo) medicine, diagnosis is based on the observation of facial color and expression. Disease states that are diagnosable from facial characteristics include blood stagnation, blood deficiency, and yin deficiency. However, these diagnoses are subjective and require lots of experience. In this paper, we aimed to quantify facial diagnosis by creating a system to evaluate the severity of disease states using feature values obtained from facial images. A Kampo physician evaluated the facial images and rated them from 1 to 5 according to severity of disease states. We verified the accuracy of this system using the mean squared error calculated from the difference between the physician evaluation scores and the system estimates. The mean squared error was close to zero, indicating that the system has high accuracy. The selection of feature values using this system corresponded with those facial regions used by the Kampo physician in diagnosing the disease states.

Cancer chemotherapy-induced peripheral neuropathy (CIPN) deteriorates the patient’s quality of life, often resulting in discontinuation of treatment with potentially effective chemotherapy. Contact needle therapy (CNT) is one of the traditional Japanese acupuncture methods based on the concept of Kampo (Japanese traditional) medicine. CNT on CIPN has been considered effective empirically, but an objective evaluation has proven difficult. We have focused on the oxygen saturation besides the symptom score. We have already proposed optical path-length matrix method (OPLM) for estimating oxygen saturation of the skin, which may help in locating peripheral tissue damage, including neuronal damage. Four patients with CIPN were treated with CNT. The severity of CIPN was evaluated using the Common Terminology Criteria for Adverse Events (CTCAE) ver. 4 and FACT/GOG-Ntx before and after CNT. We also measured oxygen saturation and hemoglobin concentration at the right toe-tip. Correlations among the values were calculated as correlation coefficients. There were significantly improvements in FACT/GOG-Ntx and hemoglobin post-CNT than in pre-CNT (p

We evaluated the effect of acupuncture treatment on the autonomic nervous system by measuring the non-contact heart rate variability (HRV). HRV shows the changes in sympathetic and parasympathetic nervous activities. Sympathetic and parasympathetic nervous activities are respectively reflected in the high-frequency- (HF) and low-frequency (LF) components of HRV. Many studies have shown that sensory stimulation affects the cardiovascular system. In particular, acupuncture stimulation often decreases the LF component, and there is a significant positive correlation between the HF component and the number of acupuncture sensations administered. These previous studies generally used an electrocardiograph or other contact plethysmograph to obtain the HRV. However, the use of electrodes and contact sensors might cause the participants to experience stress. Therefore, we used non-contact HRV measurement to evaluate the effect of acupuncture treatment. Using a remote measurement system with a digital camera enables the exclusion of external factors, and the attainment of more natural biological information. In the present study, we verified the accuracy of remote HRV measurement using a digital camera, and evaluated the effect of acupuncture on the autonomic nervous system.

In this paper, we find effective feature values for skin texture as captured by a non-contact camera to monitor piloerection on the skin to estimate emotion. Piloerection is observed as goose bumps on the skin when a person is emotionally moved or scared. This phenomenon is caused by the contraction of the arrector pili muscles with the activation of the sympathetic nervous system. Piloerection changes skin texture, because of which we think it effective to examine skin texture to estimate the subject’s emotions. Skin texture is important in the cosmetic industry to evaluate skin condition. Therefore, we thought that it will be effective to evaluate the condition of skin texture for emotion estimation. Evaluations were performed by extracting effective feature values from skin textures captured by using a high-resolution camera, where these feature values should be highly correlated with the degree of piloerection. The results showed that the feature value “the standard deviation of short-line inclination angles in texture” was satisfactorily correlated with the degree of piloerection.

In this paper, we propose a non-invasive method for measuring pulse waves corresponding to heart rate (HR) by capturing the color change on the soles of rats’ feet using a high-speed RGB camera. Remote photoplethysmography (rPPG) with a camera has been used as a non-invasive biometric method. However, the rPPG method has been challenging to apply to rats with body hair. We applied the rPPG method using a high frame rate to the sole where the skin was directly visible and successfully and accurately detected pulse waves under non-invasive, non-restraint, and non-anesthetized conditions.

In this paper, principal component analysis is applied to the distribution of pigmentation, surface reflectance, and landmarks in whole facial images to obtain feature values. The relationship between the obtained feature vectors and the age of the face is then estimated by multiple regression analysis so that facial images can be modulated for woman aged 10–70. In a previous study, we analyzed only the distribution of pigmentation, and the reproduced images appeared to be younger than the apparent age of the initial images. We believe that this happened because we did not modulate the facial structures and detailed surfaces, such as wrinkles. By considering landmarks and surface reflectance over the entire face, we were able to analyze the variation in the distributions of facial structures and fine asperity, and pigmentation. As a result, our method is able to appropriately modulate the appearance of a face so that it appears to be the correct age.

Sleep spindles are important electroencephalographic (EEG) waveforms in sleep medicine; however, it is burdensome even for experts to detect spindles, so automatic spindle detection methodologies have been investigated. Conventional methods utilize waveforms template matching or machine learning for detecting spindles. In the former approach, it is necessary to tune thresholds for individual adaptation, while the latter approach has the problem of imbalanced data because the amount of sleep spindles is small compared with the entire EEG data. The present work proposes a sleep spindle detection method that combines wavelet synchrosqueezed transform (SST) and random under-sampling boosting (RUSBoost). SST is a time-frequency analysis method suitable for extracting features of spindle waveforms. RUSBoost is a framework for coping with the imbalanced data problem. The proposed SST-RUS can deal with the imbalanced data in spindle detection and does not require threshold tuning because RUSBoost uses majority voting of weak classifiers for discrimination. The performance of SST-RUS was validated using an open-access database called the Montreal archives of sleep studies cohort 1 (MASS-C1), which showed an F-measure of 0.70 with a sensitivity of 76.9% and a positive predictive value of 61.2%. The proposed method can reduce the burden of PSG scoring.

In this paper, we propose a method to visualize blood flow of palm in different muscle tense state using RGB high-speed camera. Recently, new modalities are needed to develop a more accurate system to non-contact multi-modal affect analysis. Then, we focus on muscle tense. The muscle tense is caused by stress. Hence, the muscle tense is one of the effective modalities for non-contact multi-modal affect analysis. However, it is very difficult to measure muscle tense in the real environment because it requires a contact-type sensor. Therefore, we use iPPG to visualize the pulse wave during muscle tense from the skin video taken with the RGB video camera. As a result of this experiment, we found that it was possible to recognize the difference in pulse wave during muscle tense from the video that visualized the pulse wave. From this result, the realization of non-contact measurement of muscle tense can be expected.

In this paper, we analyze the relationship between impression of facial skin and skin pigmentation distribution by applying Canonical Coefficient Analysis (CCA) to multiple physical and psychological features obtained from facial skin. Based on the acquired relationship expression, we modulate the skin pigment features, and appearances of the face with arbitrary psychological features are reproduced. In our previous work, we applied principal component analysis (PCA) to the melanin pigment variation of the facial skin, and we obtained individual differences in it occurring over seven years. In the previous method, as the factor causing individual difference, we considered the frequency of UV care. However, actual skin appearance is thought to depend not only on melanin but also on several other factors. Therefore, in this study, we photographed the faces of women of various ages for 12 years, and at the same time obtained psychological features of appearance. As physical features, melanin and hemoglobin pigmentation distributions, shading and the frequency of UV care for 12 years were obtained. Subjective evaluation values were acquired as psychological features. As a result of CCA, it was found that the whole face can be made lighter in appearance by using UV protection every day continuously for six years or more.

In this study, we have analyzed individual differences in how the melanin component changes with time in 2003, 2010 and 2015. The frequency of use of protection against ultraviolet rays (UV) is considered to be a factor that causes individual differences in aging. Melanin is produced in human skin by exposure to UV, which promotes aging effects such as darkening and unevenness of skin color. In our previous work, we applied principal component analysis (PCA) to the distribution of melanin pigment and obtained feature values which quantitatively describe it. Furthermore, we found that the melanin component especially around the cheeks tends to increase with age. Comparing the melanin feature values for each frequency of use of UV protection, we found that the daily UV care even in winter can reduce the melanin component of face. In our previous work, we analyzed the database of facial images taken in 2003 and 2015. In this study, adding new facial data for 2010, we constructed a database of facial images captured in 2003, 2010, and 2015. It includes photographs of 60 subjects each in 3 years. The database also includes subjects’ frequency of using UV protection. Applying the same analysis as our previous study, we obtained principal component (PC) scores, which describe the changes in individuals’ faces in 2003, 2010 and 2015. From trends in these data, we found that the daily care for 12 years can suppress increasing the facial melanin component.

Volume measurement of the leg is important in the evaluation of leg edema. Recently, method for measurement by using a depth camera is proposed. However, many depth cameras are expensive. Therefore, we propose a method using Microsoft Kinect. We obtain a point cloud of the leg by Kinect Fusion technique and calculate the volume. We measured the volume of leg for three healthy students during three days. In each measurement, the increase of volume was confirmed from morning to evening. It is known that the volume of leg is increased in doing office work. Our experimental results meet this expectation.

Skin is the largest organ of human body which can easily be observed non-invasively, but objective assessment of skin status is difficult. Therefore, we have focused on the oxygen saturation, and proposed the iterative optical path-length matrix method (OPLM) for estimating oxygen saturation of skin. Oxygen saturation of skin may help finding pre-disease state. In Kampo medicine (Japanese traditional herbal medicine), inspection of skin and tongue is one of the principal methods for diagnosis of the state. Kampo medicine contains a number of concepts which are useful for preventive medicine. The Oketsu status is a pathological concept in Kampo medicine, primarily denoting blood stasis/stagnation. In this study, we focused on oxygen saturation of skin as an objective index of skin properties, and studied correlation with the blood flow and the Oketsu score. First, we acquired Oketsu score of the patients, and measured oxygen saturation, hemoglobin concentration and blood flow of the patients at two locations: fingertip of the right first finger and dorsal surface of the right hand. Then, correlation among the values was calculated. As the result, the oxygen saturation showed large correlation with the blood flow and the Oketsu score at dorsal surface of hand. The oxygen saturation of skin will reflect systemic blood flow and Oketsu status, and it would help objective diagnosis by measuring skin color spectrum.

We measured the oxygen saturation of skin at various ages by iterating our previously proposed method that can rapidly simulate skin spectral reflectance with high accuracy. Oxygen saturation is commonly measured by a pulse oximeter to evaluate oxygen delivery for monitoring the functions of heart and lungs at a specific time. On the other hand, oxygen saturation of skin is expected to assess peripheral conditions. Our previously proposed method, the optical path-length matrix method (OPLM), is based on a Monte Carlo model of steady-state light transport in multi-layered tissue (MCML), but can simulate skin spectral reflectance 27,000 times faster than MCML. In this study, we implemented an iterative OPLM with a nonlinear optimization technique such that this method can also be used for estimating hemoglobin concentration and oxygen saturation from the measured skin spectral reflectance. The iterative OPLM was over 25,000 times faster than the MCML with 95 % accuracy. Before measurements in clinic, stability of oxygen saturation among measuring pressure was confirmed. Then, the skin reflectance spectra of 72 outpatients aged between 20 and 86 years were measured by a spectrophotometer. Three points were measured for each subject: the forearm, the thenar eminence, and the intermediate phalanx. The result showed that the oxygen saturation of skin remained constant at each point as the age varied.

Kampo medicine (Japanese traditional herbal medicine) contains concepts useful for preventive medicine. For example, “Mibyou” (disease-oriented state) aims to prevent illness by early recognition. Kampo diagnosis is based on subjective examinations, such as tongue inspection, by trained specialist physicians. An objective metric of the tongue color spectrum was developed as a surrogate for subjective visual inspection. Tongue images were acquired with a hyperspectral imaging system, and the uncoated tongue region was segmented automatically. The spectral information of the uncoated tongue area was analyzed by principal component analysis (PCA). The component vector most representative of each clinical symptom was found by rotating the vector on a plane spanned by two arbitrary principal component vectors. The system was tested in human volunteers. Forty-four hyperspectral images were acquired from 30 healthy male subjects for initial testing. The Oketsu (blood stagnation) score was determined by an experienced clinician in Kampo medicine from 27 of 30 subjects. The correlation between respective principal components and Oketsu score was 0.67 at maximum, and increased to 0.73 by linear combination, while it was −0.75 by vector rotation. Significant correlations for many disorders were demonstrated, and vector rotation showed better correlation than linear combination. A PCA-based algorithm was developed to objectively evaluate patients using color images of the tongue surface. Testing showed that this method was a feasible surrogate for expert visual tongue analysis. This tool should help non-trained people identify “Mibyou” health status for individuals. The algorithm is free of empirical criteria, and it may be it applicable to many hyperspectral image types.

In this paper, principal component analysis is applied to pigmentation distributions, surface reflectance components and facial landmarks in the whole facial images to obtain feature values. Furthermore, the relationship between the obtained feature vectors and age is estimated by multiple regression analysis to modulate facial images in woman of ages 10 to 70. In our previous work, we analyzed only pigmentation distributions and the reproduced images looked younger than the reproduced age by the subjective evaluation. We considered that this happened because we did not modulate the facial structures and detailed surfaces such as wrinkles. By analyzing landmarks represented facial structures and surface reflectance components, we analyzed the variation of facial structures and fine asperity distributions as well as pigmentation distributions in the whole face. As a result, our method modulate the appearance of a face by changing age more appropriately.

In this article, we propose an effective colorprocessing algorithm to analyze the hyperspectral image of the tongue and its application to preventive medicine by the concept of Japanese traditional herbal medicine (Kampo medicine). Kampo medicine contains a number of concepts useful for preventive medicine such as "Mibyou" - disease-oriented state - signs of abnormalities. Hyperspectral images of the tongue were taken with the system with an integrating sphere, and tongue area without coating was eliminated automatically. Then, spectral information of the tongue area without coating was analyzed by principal component analysis, and the component vector best representing the clinical symptom was found by rotating the vector on a plane spanned by two arbitrary principal component vectors.