Search

Background Coronary arterial aneurysms (CAAs) associated with Kawasaki disease (KD) significantly affect prognosis. However, the clinical course of CAAs and factors associated with CAA regression have not been well analyzed. Methods and Results The cohort of the Z‐Score 2nd Project Stage study, a multicenter, retrospective, cohort study involving 44 institutions in Japan including 1006 patients with KD, was examined. CAAs were classified by the z score of their internal diameter in the acute phase: small ( z z z ≥10). The lower limit of small CAA was based on the Japanese Ministry of Health, Labour and Welfare criteria. In the right coronary artery, the CAA regression rates 10 years after diagnosis were 95.5% for small, 83.2% for medium, and 36.3% for large. In the proximal left anterior descending artery, the regression rates 10 years after diagnosis were 95.3% for small, 80.1% for medium, and 28.8% for large. Cox regression analysis showed that diagnosis under the age of 1 year and onset of KD in 2010 to 2012 for the right coronary artery and the left anterior descending artery, and female for the right coronary artery were significantly associated with a high regression rate, whereas large CAAs for the right coronary artery and the left anterior descending artery were significantly associated with a low regression rate. Conclusions The current study, the largest Japanese study of its kind, found that small aneurysm, recent onset, and diagnosis under the age of 1 year predict regression, and that even giant aneurysms could regress. These data may contribute to long‐term management of coronary aneurysms. Registration URL: https://www.umin.ac.jp/ctr/ ; Unique identifier: UMIN000010606. more...

Background The aim of this study was to determine preferences regarding transfer of patients with congenital heart disease (CHD) attending a children's hospital in Japan and related factors. Methods and Results: We conducted a self-administered questionnaire survey with CHD patients >15 years of age treated at the pediatric cardiology outpatient clinic of a children's hospital. Logistic regression analysis was used to identify factors related to patient preferences regarding the transfer. One hundred and eleven of the 122 patients given a questionnaire provided valid responses (valid response rate, 91.0%). Sixty-six subjects (64.9%) reported "not being told anything specific" by their physicians about the transfer from the children's hospital, and 72 (59.5%) stated that they "wished to continue attending the children's hospital". Visiting outpatient clinic with parents (OR, 11.00; 95% CI: 2.01-60.97), having low uncertainty about continuing to attend the children's hospital (OR, 0.95; 95% CI: 0.92-0.98), and having high uncertainty about leaving the current physician (OR, 1.04; 95% CI: 1.01-1.07) were significantly related to the patient's wish to continue to attend the children's hospital. Conclusions There is a need to improve patient education regarding the opportunities for transfer, and to develop a systematic transition program for children's hospitals and aligned specialized adult CHD centers. more...

Infantile hemangiomas undergo rapid growth during early infancy followed by gradual involution. Infantile hemangiomas sometimes impair vital functions or cause disfigurement. Thirty-two Japanese patients between the ages of 1 and 4 months with proliferating infantile hemangiomas received oral propranolol on an outpatient basis. The success rate (complete or nearly complete resolution) at week 25 was 56% (18/32). Two patients dropped out because of a personal reason and moving out. Recurrence after termination of treatment was seen in six patients. Adverse events occurred in 16 patients. There were no adverse events on day 1 (initiation of treatment at a dose of 1 mg/kg per day) and day 8 (dose increase to 2 mg/kg per day). One patient was hospitalized due to pneumonia, and suspended propranolol for 26 days. Oral propranolol at 2 mg/kg per day is effective and safe in Japanese patients with infantile hemangiomas. more...

An autonomous guide robot based on active interactions with users has been developed. Here, active interactions mean that the users can ask the robot to wait for them, explain contents around, etc. The robot can adjust its speed and moving path automatically to adapt to the motions of users so that it can provide considerate services to the users. The robot is controlled by a general control model based on integrated potential field, by which the robot can adapt to different motions of users without changing the moving rules case by case. The effectiveness of our proposed method is proven by both simulation and experiments where the users are guided by a robot under indoor environments. more...

It is difficult to track multiple people effectively for a long time in a complex environment because people’s clothes and body shapes may be similar, and their postures may be constantly changing. This paper proposes a novel method for multiple people tracking in crowded places where people can be partially or completely occluded. The people are detected by the deep learning method ConvNet from the color image first, and detection results are integrated with the depth information so that the accurate human areas can be extracted. The accurate personal color information can be extracted then without any background color information. multiple people tracking is proceeded by using particle filter based on the color information. the effectiveness of the proposed method is verified through experiments of tracking multiple children in a classroom. more...

Facial expressions are the straight link for showing human emotions. Psychologists have established the universality of six prototypic basic facial expressions of emotions which they believe are consistent among cultures and races. However, some recent cross-cultural studies have questioned and to some degree refuted this cultural universality. Therefore, in order to contribute to the theory of cultural specificity of basic expressions, from a composite viewpoint of psychology and HCI (Human Computer Interaction), this paper presents a methodical analysis of Western-Caucasian and East-Asian prototypic expressions focused on four facial regions: forehead, eyes-eyebrows, mouth and nose. Our analysis is based on facial expression recognition and visual analysis of facial expression images of two datasets composed by four standard databases CK+, JAFFE, TFEID and JACFEE. A hybrid feature extraction method based on Fourier coefficients is proposed for the recognition analysis. In addition, we present a cross-cultural human study applied to 40 subjects as a baseline, as well as one comparison of facial expression recognition performance between the previous cross-cultural tests from the literature. With this work, it is possible to clarify the prior considerations for working with multicultural facial expression recognition and contribute to identifying the specific facial expression differences between Western-Caucasian and East-Asian basic expressions of emotions. more...

The recent boom of mass media communication (such as social media and mobiles) has boosted more applications of automatic facial expression recognition (FER). Thus, human facial expressions have to be encoded and recognized through digital devices. However, this process has to be done under recurrent problems of image illumination changes and partial occlusions. Therefore, in this paper, we propose a fully automated FER system based on Local Fourier Coefficients and Facial Fourier Descriptors. The combined power of appearance and geometric features is used for describing the specific facial regions of eyes-eyebrows, nose and mouth. All based on the attributes of the Fourier Transform and Support Vector Machines. Hence, our proposal overcomes FER problems such as illumination changes, partial occlusion, image rotation, redundancy and dimensionality reduction. Several tests were performed in order to demonstrate the efficiency of our proposal, which were evaluated using three standard databases: CK+, MUG and TFEID. In addition, evaluation results showed that the average recognition rate of each database reaches higher performance than most of the state-of-the-art techniques surveyed in this paper. more...

We present a method through which domestic service robots can comprehend natural language instructions. For each action type, a variety of natural language expressions can be used, for example, the... more...

We present a framework by which the motion of an autonomous mobile guide robot is adaptively controlled. A sociable robot should adapt its speed and path to suit the users' activities, without restricting the user movement. By generating adaptive artificial potential fields for the users and the subgoal separately, and integrating them with the basic potential fields generated from obstacles, our robot can adapt to the users' activities and provide sociable tour-guide services. The robot predicts a user's moving speed and adapts to it to maintain the social distance. Moreover, with the proposed framework, users can deviate from the guided path temporarily and return to the original task afterward. Instead of waiting for the users and taking the risk of losing them, the robot deviates from its original path to follow the users and also prepares for returning to the guiding task. The robot restarts the guiding task at that place, which ensures the least cost to reach the goal. Simulation and experimental results show that our framework can automatically generate suitable motion patterns to control the robot adaptively, making it sociable while providing tour guide services. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. more...

This paper proposes a speed control method to move an autonomous mobile robot safely under the environment where blind areas exist. Sensors like a laser range finder or an optical camera cannot detect the areas in the backside of walls, pillars, or objects. When a person suddenly appears from these blind areas, the mobile robot is very likely to collide with him/her, if its speed reduction performance is limited. In this paper the safety speed of robot is carefully controlled using the distance information between the robot and the blind areas as well as obstacles. Meanwhile, the optimum path is generated to minimize the cost of moving time, considering the lengthened time cost caused by speed control process. The effectiveness of the proposed method is confirmed by both the computer simulation and the experiment using a real robot. more...

To improve the quality of childcare in nursery schools by using information technologies, a childcare assisting system is proposed to help the teachers with their daily works. The first step of the system is automatically monitoring and analyzing the behaviors of the children. In this paper, a multiple children identification and tracking method is proposed by using color and depth information from the sensors setting in the environment. The children are identified by integrating multiple personal information of color, face, moving pattern, and the identification results are used as the observation likelihoods in the particle filter based tracking process. The proposed method can continuously track multiple children under complex environment with occlusions among children and fast motions crossing with each other. The effectiveness of the proposed method is proven by the experiments conducted in a nursery school. more...

Simultaneous Localization and Mapping (SLAM) is an essential technique for autonomous mobile robot. 2D mapping has been greatly developed recently and widely used in navigation tasks since the it is easier to be generated and has fewer computation amount than 3D mapping. However, 2D information is not enough to reflect real 3D environment. This paper proposes a 2D mapping method by using immobile area grid map method. 3D objects are detected by 3D sensor and the object recognition results are reflected to the 2D map by a probability model, in which the object recognition results are used to calculate the degree of using observed information to generate the map. In this way, 3D objects are reflected to the map and potential mobile objects are deleted from the map. The effectiveness of our proposed method is proven by conducting experiments under an indoor environment, where 3D objects (e.g. desks and chairs) and human beings are all contained. With the proposed method, the generated 2D map shows the areas of 3D objects well and deletes the potential moving objects (static human beings) correctly. more...

Simultaneous Localization and Mapping (SLAM) is an important function for autonomous mobile robots. 2D or 3D maps under static or dynamic environments have been greatly developed and widely used for robot navigation and path planning. Most of the generated maps can accurately reflect the objects in the environment, but the properties of the objects have not been considered. The robot can avoid colliding with obstacles when using these kind of maps. However, the robot needs to move in a socially acceptable way like human beings. For example, human beings usually avoid moving under desks even if there are paths that can go through. Meanwhile, human beings has the ability to analyze the motion of the objects like a door and move in a considerate way without staying behind it and standing in the way. The spaces under a desk, behind a door, in front of a refrigerator etc. are not occupied by real objects but actually occupied by the objects because of their properties. These kinds of spaces are defined as potential occupied spaces in this paper and considered when generating the map. The objects in the environment are detected and reflected to the may in the same way of conventional methods. Besides, the objects are also recognized and their properties are analyzed to generated virtual areas in the map. In this way, human beings will naturally avoid entering these potentially occupied spaces and the robots can move considerately like human beings. The basic map is generated by immobile area grid map based SLAM. The objects are recognized by Single Shot multi-box Detector (SSD) and other methods, and their potential occupied spaces are generated and reflected to the map base on potential filed method. The effectiveness of the proposed method is proven by mapping under the indoor environment. more...

Humans recognize perceived continuous information by dividing it into significant segments such as words and unit motions. We believe that such unsupervised segmentation is also an important ability that robots need to learn topics such as language and motions. Hence, in this paper, we propose a method for dividing continuous time-series data into segments in an unsupervised manner. To this end, we proposed a method based on a hidden semi-Markov model with Gaussian process (GP-HSMM). If Gaussian processes, which are nonparametric models, are used, unit motion patterns can be extracted from complicated continuous motion. However, this approach requires the number of classes of segments in the time-series data in advance. To overcome this problem, in this paper, we extend GP-HSMM to a nonparametric Bayesian model by introducing a hierarchical Dirichlet process (HDP) and propose the hierarchical Dirichlet processes-Gaussian process-hidden semi-Markov model (HDP-GP-HSMM). In the nonparametric Bayesian model, an infinite number of classes is assumed and it becomes difficult to estimate the parameters naively. Instead, the parameters of the proposed HDP-GP-HSMM are estimated by applying slice sampling. In the experiments, we use various synthetic and motion-capture data to show that our proposed model can estimate a more correct number of classes and achieve more accurate segmentation than baseline methods. more...

Humans interact with one another daily and learn from their experiences via observation and interaction. To create robots that can coexist with humans, it is important that they learn how to appropriately interact in the human community. In this paper, we propose a novel model, the coupled Gaussian process hidden semi-Markov model (GP-HSMM), which enables robots to learn rules of interaction between two persons by observing them in an unsupervised manner. The continuous motions of the persons are segmented into discrete actions based on GP-HSMM, and the relationships between the actions are extracted. Moreover, all corresponding actions are not simultaneously conducted by two persons during actual interaction. Thus, the coupled GP-HSMM accounts for these lags. We conduct experiments using the motion data of interaction games. Experimental results showed that the coupled GP-HSMM can estimate actions, lags between them and their relationships. more...

Developing a childcare assisting system is highly necessary due to the lack of nursery teachers, and it will make an important progress on effective utilization of nursery teacher resources. In this paper, we proposed simultaneous children recognition and tracking system by using Kinect sensors for the childcare assisting system to provide information for the nursery teachers. Each of the children is recognized by integrating his/her personal information of color, face and motion. The tracking problem is modeled as finding the MAP solution of a posterior probability, and is solved by using Markov Chain Monte Carlo (MCMC) particle filter. Our system can recognize and robustly track each child during class activities. Trajectories, motion ranges and relative distances information can be provided for the nursery teachers to assist their childcare work. The effectiveness of our system is proved through continuous monitoring of the children in a nursery school. more...

SUMMARY Mobile robots need Simultaneous Localization and Mapping (SLAM) for autonomous movement in human living environments. The occupancy grid map used in SLAM is a conventional method which makes a map by an occupancy probability in each grid. This method renews a map based on whether an object is observed or not. In order to remove moving objects from a map, an additional method is required. However, conventional methods deal only with actually moving objects, and potential moving objects (e.g., standing humans) are mapped as static objects. Furthermore, only binary states, used or not used, are given to each object in map updating. This paper proposes the immobility area grid map to represent a map by an immobility probability in each grid. The proposed method renews a map based on the identification of observed objects by a robot's sensors, in addition to whether an object is observed or not. We introduce the map update parameter, which is set adaptively from the certainty of identification result of the object. Observed objects can take continuous states, truly static—unknown—truly moving, according to the parameter value. Potential moving objects are not mapped if the parameter takes values corresponding to moving objects. The experimental results show robust mapping in dynamic environments including potential moving objects. more...

IMPORTANCE: Few studies with sufficient statistical power have shown the association of the z score of the coronary arterial internal diameter with coronary events (CE) in patients with Kawasaki disease (KD) with coronary artery aneurysms (CAA). OBJECTIVE: To clarify the association of the z score with time-dependent CE occurrence in patients with KD with CAA. DESIGN, SETTING, AND PARTICIPANTS: This multicenter, collaborative retrospective cohort study of 44 participating institutions included 1006 patients with KD younger than 19 years who received a coronary angiography between 1992 and 2011. MAIN OUTCOMES AND MEASURES: The time-dependent occurrence of CE, including thrombosis, stenosis, obstruction, acute ischemic events, and coronary interventions, was analyzed for small (z score more...

Infantile hemangiomas are the most common tumor of childhood and undergo rapid growth during early infancy followed by gradual involution. After involution, residual lesions sometimes remain. Oral propranolol usually induces earlier involution and redness reduction of infantile hemangiomas. However, the optimal treatment duration is unknown and infantile hemangiomas sometimes recur after cessation of treatment. We report three Japanese patients with recurrent infantile hemangiomas on their cheek. These patients were a 1-month-old female baby with a superficial infantile hemangioma, a 3-month-old female baby with a mixed infantile hemangioma and a 4-month-old male baby with a mixed infantile hemangioma. Two of them also received pulsed dye laser treatment. They did not reach complete or nearly complete resolution of infantile hemangiomas at week 25. These patients experienced regrowth of their infantile hemangioma after 20 months of age and took propranolol after the age of 24 months. There were no severe adverse effects. Propranolol may not only be therapeutic but also prophylactic. Patients with infantile hemangiomas who have taken oral propranolol should be followed up at least 6 months after cessation of treatment, especially infantile hemangiomas on the cheek, and those with partial response to propranolol may require close attention in prolonged growth. more...

Recent cross-cultural studies have questioned the cultural universality of facial expressions from a psychological viewpoint. However, the automatic facial expression recognition (FER) systems are based on the assumption that facial expressions are the same for all human beings, excluding the differences that may appear between different races and cultures. Therefore, this paper presents an analysis of culturally specific facial expression recognition focused on Western and East-Asian expressive faces using an automatic FER system based on 3 different feature extraction methods (appearance-, geometric-, and a proposed hybrid-based). Our study is focused on 4 specific facial regions (eyes-eyebrows, mouth, nose and forehead/outline) and it is powered by Principal Component Analysis (PCA) which enables the visual examination of the most relevant differences among the 6 basic expressions from each racial group. Feature extraction methods are evaluated using Support Vector Machines (SVM) and 4 standard databases. In addition, our findings are compared with a cross-cultural human study applied to 40 participants from both racial groups. more...

In order to move around automatically, mobile robots usually need to recognize their working environment first. Simultaneous localization and mapping (SLAM) has become an important research field recently, by which the robot can generate a map while moving around. Both two-dimensional (2D) mapping and three-dimensional (3D) mapping methods have been developed greatly with high accuracy. However, 2D maps cannot reflect the space information of the environment and 3D mapping needs long processing time. Moreover, conventional SLAM methods based on grid maps take a long time to delete the moving objects from the map and are hard to delete the potential moving objects. In this paper, a 2D mapping method integrating with 3D information based on immobile area occupied grid maps is proposed. Objects in 3D space are recognized and their space information (e.g., shapes) and properties (moving objects or potential moving objects like people standing still) are projected to the 2D plane for updating the 2D map. By using the immobile area occupied grid map method, recognized still objects are reflected to the map quickly by updating the immobile area occupancy probability with a high coefficient. Meanwhile, recognized moving objects and potential moving objects are not used for updating the map. The unknown objects are reflected to the 2D map with a lower immobile area occupancy probability so that they can be deleted quickly once they are recognized as moving objects or start to move. The effectiveness of our method is proven by experiments of mapping under dynamic indoor environment using a mobile robot. more...

SUMMARY This paper focuses on mobile robots that can accompany a person, that is, it addresses how to control the position of the robot relative to the accompanied person according to the dynamic environment. The robot is expected to move side by side with the person in a normal situation, but a position in front or behind the person might be better if there are obstacles. The shape of the artificial potential field of the accompanied person is devised to smoothly control the robot position in a unified way. The Laplace potential method is also used subsidiarily to cope with extremely cluttered environments. Simulations and experiments in real space show the usefulness of the proposed method. more...

Darwin was the first one to assert that facial expressions are innate and universal, which are recognized across all cultures. However, recent some cross-cultural studies have questioned this assumed universality. Therefore, this paper presents an analysis of the differences between Western and East-Asian faces of the six basic expressions (anger, disgust, fear, happiness, sadness and surprise) focused on three individual facial regions of eyes-eyebrows, nose and mouth. The analysis is conducted by applying PCA for two feature extraction methods: appearance-based by using the pixel intensities of facial parts, and geometric-based by handling 125 feature points from the face. Both methods are evaluated using 4 standard databases for both racial groups and the results are compared with a cross-cultural human study applied to 20 participants. Our analysis reveals that differences between Westerns and East-Asians exist mainly on the regions of eyes-eyebrows and mouth for expressions of fear and disgust respectively. This work presents important findings for a better design of automatic facial expression recognition systems based on the difference between two racial groups. more...

Background The 1st nationwide survey by the Japanese Society of Pediatric Cardiology and Cardiac Surgery of acute or fulminant myocarditis (AMC/FMC) in children revealed that the survival rate of FMC was only 51.6%. The 2nd nationwide survey was performed to evaluate the recent outcomes of pediatric myocarditis.Methods and Results:Questionnaires regarding patients aged ≤18 years with AMC/FMC during the period from January 2006 to December 2011 were mailed. A total of 221 cases (age 6.5±5.3 years, 116 boys and 105 girls) were reported. There were 145 (65.6%) and 74 cases (33.5%) of AMC/FMC, respectively; the type of myocarditis was not reported in the remaining 2 cases (0.9%). Viruses were identified in 56 cases (25.3%), including coxsackie B in 9 and influenza A in 8. Histopathology by either endomyocardial biopsy or autopsy was obtained in 38 cases (19.2%). Intravenous immunoglobulin was effective in 49 (34.3%) of 143 cases. Steroid therapy was effective in 20 (32.8%) of 61 cases. Mechanical circulatory support was given in 54 cases (24.4%) and 94.2% of them were patients with FMC. The survival rates for the whole study population, acute myocarditis, and FMC were 75.6%, 91.0%, and 48.6%, respectively. Conclusions The survival rate of children with myocarditis was almost identical to that of 10 years ago. (Circ J 2016; 80: 2362-2368). more...

This report describes six previously healthy infants with acute mitral valve chordal rupture. Although their initial symptoms were mild respiratory distress and fever, all the infants presented with acute progressive left heart failure and required urgent surgical repair. Mitral valve annuloplasty and mitral valve replacement were performed (for three patients each). Two patients died, both of whom had cardiac arrest and required resuscitation before surgery. Histologic examination of excised mitral valve leaflets showed infiltration of gram-positive bacteria in these two patients. All the patients were 4-6 months old, which may suggest underlying congenital factors. The authors believe early diagnosis and surgery are crucial as a life-saving measure for infants with acute mitral valve chordal rupture. more...

In many real-world face recognition applications, there might be only one training image per person available. Moreover, the test images may vary in facial expressions and illuminations, or may be partially occluded. However, most classical face recognition techniques assume that multiple images per person are available for training, and they are difficult to deal with extreme expressions, illuminations and occlusions. This paper proposes a novel block-based bag of words (BBoW) method to solve those problems. In our approach, a face image is partitioned into multiple blocks, dense SIFT features are then calculated and vector quantized into different visual words on each block respectively. Finally, histograms of codeword distribution on each local block are concatenated to represent the face image. Our method is able to capture local features on each block while maintaining holistic spatial information of different facial components. Without any illumination compensation or image alignment processing, the proposed method achieves excellent face recognition results on AR and XM2VTS databases. Experimental results show that only using one neutral expression frame per person for training, our method can obtain the best performance ever on face images of AR database with extreme expressions, variant illuminations, and partial occlusions. We also test our method on the standard and darkened sets of XM2VTS database, and achieve the average rates of 100% and 96.10% on the standard and darkened sets of XM2VTS database, respectively. more...

Two people usually walk through a corridor in a parallel position. However, they spontaneously switch their positions from the parallel to the cascade position when they pass oncoming people. We propose a method for enabling a robot to switch the parallel and the cascade positions autonomously with the movement of oncoming people and obstacles when it moves with a user in a daily environment. The robot uses two laser range finders to detect walls, the user's position, as well as surrounding people and obstacles. It autonomously decides whether to go on ahead or to give way to the user. Experimental results using a robot in a daily environment show that our method was effective. more...

It is likely that in the near future, human-symbiotic robots will share people's living spaces. Since there are many objects that can cause visual occlusion in these environments, situations in which a robot cannot see an object due to occlusion whereas a user can, and vice versa, will often occur. In such situations, ideally the human-symbiotic robot should be able to interact with the user while taking into consideration the differences between their fields of view. We anticipate that such a “considerate” robot will be friendlier and provide more pleasant interactions with users. In this paper, we aim to test this anticipation. First, we propose a robot that can estimate a user's and its own fields of view and thus behave appropriately due to being aware of the difference between their perceptions. The robot estimates the orientation of the user's head and the structure of its surrounding environment using a stereo camera. By combining these results, it can inform itself of the user's and its own fields of view: that is, what they can see and what they cannot see. Next, we carry out experimental subjective evaluations of the impressions that people gain during interaction with the proposed robot. Participants who observe the interactions are asked to subjectively evaluate their impressions of the robot's behaviors. The experimental results show that the proposed robot, which can guess what the user can see and understand the differences between their different viewpoints, is perceived as more “companionable.” This ability is likely to be one of the basic requirements for achieving interactions that people interpret as “friendly” with robots and other intelligent systems. more...

Facial expression recognition has many potential applications in areas such as human-computer interaction (HCI), emotion analysis, and synthetic face animation. This paper proposes a novel framework of facial appearance and shape information extraction for facial expression recognition. For appearance information extraction, a facial-component-based bag of words method is presented. We segment face images into four component regions: forehead, eye-eyebrow, nose, and mouth. We then partition them into 4 × 4 sub-regions. Dense SIFT (scale-invariant feature transform) features are calculated over the sub-regions and vector quantized into 4 × 4 sets of codeword distributions. For shape information extraction, PHOG (pyramid histogram of orientated gradient) descriptors are computed on the four facial component regions to obtain the spatial distribution of edges. Multi-class SVM classifiers are applied to classify the six basic facial expressions using the facial-component-based bag of words and PHOG descriptors respectively. Then the appearance and shape information is fused at decision level to further improve the recognition rate. Our framework provides holistic characteristics for the local texture and shape features by enhancing the structure-based spatial information, and makes it possible to use the bag of words method and the local descriptors in facial expression recognition for the first time. The recognition rate achieved by the fusion of appearance and shape features at decision level using the Cohn-Kanade database is 96.33%, which outperforms the state-of-the-art research works. more...

Active Shape Model (ASM) is a powerful statistical tool for image interpretation, especially in face alignment. In the standard ASM, local appearances are described by intensity profiles, and the model parameter estimation is based on the assumption that the profiles follow a Gaussian distribution. It suffers from variations of poses, illumination, expressions and obstacles. In this paper, an improved ASM framework, GentleBoost based SIFT-ASM is proposed. Local appearances of landmarks are originally represented by SIFT (Scale-Invariant Feature Transform) descriptors, which are gradient orientation histograms based representations of image neighborhood. They can provide more robust and accurate guidance for search than grey-level profiles. Moreover, GentleBoost classifiers are applied to model and search the SIFT features instead of the unnecessary assumption of Gaussian distribution. Experimental results show that SIFT-ASM significantly outperforms the original ASM in aligning and localizing facial features. more...

The effectiveness of transplacental drug therapy for prenatally diagnosed isolated congenital complete atrioventricular block (CCAVB) is controversial. Nine cases of prenatal isolated CCAVB were treated from 2002 to 2007. Ritodrine was administered transplacentally to all fetuses and betamethasone to those whose mothers tested positive for maternal anti-SSA/Ro antibodies. Six of the nine patients had an anti-SSA/Ro-positive mother and received transplacental betamethasone 4 mg/day at a median gestational age of 28 weeks (range, 24-31 weeks). No patients exhibited an improvement in the degrees of complete heart block, and one patient died in utero. No serious adverse events occurred. After the mean follow-up period of 1.7 +/- 1.3 years, all five patients treated with transplacental betamethasone experienced a good cardiac function, whereas one of the three patients not treated with transplacental betamethasone experienced cardiomyopathy and died at the age of 4 months. Pacemaker implantation was required for seven of the eight live-born infants. Transplacental betamethasone therapy for the patients with isolated CCAVB neither improved the degree of atrioventricular block nor decreased the rate of patients requiring pacemaker implantation, but it probably reduced the risk for the development of myocardial disease. more...

Visual tracking is required by many vision applications such as human-computer interfaces and human-robot interactions. However, in daily living spaces where such applications are assumed to be used, stable tracking is often difficult because there are many objects which can cause the visual occlusion. While conventional tracking techniques can handle, to some extent, partial and short-term occlusion, they fail when presented with complete occlusion over long periods. They also cannot handle the case that an occluder such as a box and a bag contains and carries the tracking target inside itself, that is, the case that the target invisibly moves while being contained by the occluder. In this paper, to handle this occlusion problem, we propose a method for visual tracking by a particle filter, which switches tracking targets autonomously. In our method, if occlusion occurs during tracking, a model of the occluder is dynamically created and the tracking target is switched to this model. Thus, our method enables the tracker to indirectly track the “invisible target” by switching its target to the occluder effectively. Experimental results show the effectiveness of our method. more...

Photographs often have some people with closed eyes. A method to automatically output a best shot image in which all photographed members have open eyes is proposed. First a series of photographs (here 6 frames) is taken by using the continuous release function in commercial digital cameras. Blinking by each person is checked in each frame, and a frame in which all members have open eyes is selected. Blinking detection is carried out by calculating the normalized correlation between a 30 × 7 pixels open eyes template and input frames of reduced size. For a case where no frame exists in which all members have open eyes, the closed eye image is replaced by an open eye image obtained in a different frame. Experimental results for 18 photographs taken under various shooting conditions show the usefulness of the proposed method for automatically outputting a best shot image. more...

Recently, the requirement for the high qualified childcare schools keeps increasing, but the number of qualified nursery teachers is far from enough. Developing a childcare assisting robot is highly necessary to help the works of nursery teachers. To work like a human nursery teacher, the first challenge for the robot is to understand the behaviors of the children automatically so that the robot can give adaptive reactions to the children. In this paper, we developed a robust children behavior tracking system by using multiple Kinect sensors. Each of the child is detected and recognized by integrating his/her personal features of face, color and motion. The tracking process is realized by using the Markov Chain Monte Carlo (MCMC) particle filter. The experiments are conducted in a childcare school to show the usefulness of our system. more...

We propose a method for a robot to form various concepts. The robot uses its embodiment to obtain visual, auditory, and haptic information by grasping, shaking, and observing objects. At the same time, a user teaches the robot object features through speech. From these kinds of information, the robot can form object concepts. The information obtained by the robot is converted into Bag-of-Words representations, which are classified into categories. The proposed method is based on a stochastic model, and objects can be classified by estimating their parameters. We introduce the Chinese restaurant process into the multimodal hierarchical Dirichlet process. This model is an infinite mixture of models and enables the robot to form various concepts such as object type, color, and so on. Therefore, the robot can form not only object concepts but also various concepts that are not represented by object concepts (e.g., color). Also, because the proposed method is based on a stochastic model, it makes it possible for the robot to estimate category and unobserved information of unseen objects. We implement the proposed method on a robot and show that it can form various concepts and perform various estimations for unseen objects. more...

Recently, biologically inspired robots have been developed to acquire the capacity for directing visual attention to salient stimulus generated from the audiovisual environment. For the purpose of realizing this behavior, a general method is to calculate saliency maps to represent how much the external information attracts the robot's visual attention, where the audiovisual information and robot's motion status should be involved. In this paper, we represent a visual attention model where three modalities—audio information, visual information, and robot's motor status—are considered, because previous research has not considered all of them. First, we introduce a 2D density map, on which the value denotes how much the robot pays attention to each spatial location. Then we model the attention density using a Bayesian network where the robot's motion statuses are involved. Next, the information from both audio and visual modalities is integrated with the attention density map in integrate-fire neurons. The robot can direct its attention to the locations where the integrate-fire neurons are fired. Finally, the visual attention model is applied to make the robot select the visual information from the environment, and react to the content selected. Experimental results show that it is possible for robots to acquire the visual information related to their behaviors by using the attention model considering motion statuses. The robot can select its behaviors to adapt to the dynamic environment as well as to switch to another task according to the recognition results of visual attention. © 2006 Wiley Periodicals, Inc. Electr Eng Jpn, 158(2): 39–48, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20335 more...

A facial caricature expresses features of an individual face compactly and effectively. It is used in newspapers, magazines, home pages, and also used to represent an anthropomorphic agent and so on. This paper presents a method of synthesizing a caricature having higher flexibility in controlling individual shape features of various parts of a face, as well as features of arrangements and inclines of these parts. Firstly, an eigenspace is calculated for contour shape of each facial part such as the eyebrows, eyes, and mouth. In addition, an eigenspace for arrangements of various facial parts is obtained. Next, differences of the shapes and arrangements of the facial parts with the average face are calculated and then the orthogonal expansion to the basis vectors (eigenvectors) is carried out on these differences. Magnifying powers are determined according to the coefficients obtained by the orthogonal expansion and an emphasizing procedure is performed on eigenvectors. The difference between the slope of each facial part of the average face and that of an input face is emphasized using a quadratic function. A final caricature can be obtained by adding the average shape, average arrangement and average slope to these results. The effectiveness of the proposed caricature synthesis method is verified by subjective evaluation tests on synthesized facial caricatures. © 2004 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 88(2): 65–76, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20088 more...

This paper proposes a method to estimate sound source position by fusing auditory and visual information with Bayesian network in human-robot interaction. We firstly integrate multi-channel audio signals and a depth image about the environment to generate a likelihood map for sound source localization. However, this integration, denoted by “MICs", does not always lead to locate a sound source correctly. For correcting the failure in localization, we integrate the likelihood values generated from “MICs" and the skin-color distribution in an image according to the result of classifying audio signal into speech/non-speech categories. The audio classifier is based on support vector machine(SVM) and skin-color distribution is modeled by Gaussian mixture model (GMM). With the evidences given by MICs, SVM and GMM, we infer whether pixels in images correspond to a sound source or not according to the trained Bayesian network. Finally, experimental results are presented to show the effectiveness of the proposed method. more...

A facial caricature expresses features of an individual face summarily and has many uses for newspapers, magazines, home pages, agents, and the like. This paper presents a method of synthesizing a caricature having higher flexibility in controlling individual shape features of various parts of a face, as well as features of arrangements and inclines of these parts. In addition, it obtains eigenspaces for contours of facial parts such as the eyebrows, eyes, and mouth. It also obtains eigenspaces for arrangements of various facial parts. Next, it obtains differences of the shapes and arrangements of the facial parts with the average face and performs orthogonal expansion to the eigenspace bases (eigenvectors) on these differences. It determines magnifying power according to the coefficients obtained by orthogonal expansion and performs an emphasizing procedure on eigenvectors. The differences with the slopes of the facial parts of the average face are emphasized for the slopes of the facial parts using a second-order function. It obtains the shape of a caricature by adding the mean shape, mean arrangement, and mean slope to these results. The efficacy of the caricature synthesis method is verified by subjective evaluation tests on synthesized faces. © 2004 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 87(8): 43–54, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20088 more...

This paper proposes a speech recognition system for noisy car environments based on speaker localization using a microphone array. In this system, multiple microphones are arranged in two-dimensional (2D) space to surround the interior of a car, and the speaker location is first estimated by the proposed HE (Harmonic Enhanced) 2D MUSIC (MUltiple SIgnal Classification) technique. The proposed HE 2D MUSIC is a two-dimensional extension of the DOA (Direction Of Arrival) estimation method called MUSIC [11]. HE 2D MUSIC also uses the harmonic structure of speech signals to separate target speech signals from additive noise. Then, 2D Delay and Sum (DS) is applied to enhance the target speech. Such preprocessing makes robust speech recognition in noisy car environments possible. Moreover, in the proposed system, not only a driver, but also a fellow passenger can control car electronics by their voices regardless of where they are. The results of simulations and experiments in a real car environment are presented to confirm the validity of our proposed system. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(8): 1–12, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10415 more...

Recognizing characters on signboards in a scene would be very useful for translation, aquiring relevant information through the Internet, etc. This paper presents a method for extracting characters on signboards in arbitrary scene images that does not require any manual operations. First, we describe briefly the framework of an information handling system driven by characters on signboards in scene images. Next, a method for detecting signboard regions and individual character regions is investigated in consideration of signboards characteristics. The proposed method for detecting signboard regions is based on hierarchical clustering in the L* a* b* color space. Features of contour shape are examined to narrow the candidates of signboard regions. In addition, merging of adjacent candidate regions and rough checking of character existence are performed. Finally, each character on the signboard region is extracted by using binarization followed by horizontal and vertical projections. Experimental results from applying the method to 1000 various scene images show the effectiveness of the proposed method. more...

The ability to interact with multiple users as well as to recognize the ways in which people are interacting is essential for a robot participating in society. Forming shared attention is regarded as fundamental in improving the flexibility of human-robot interaction. This paper proposes an algorithm that enables shared attention between a robot and speakers in alternative conversational situations. We firstly present a microphone array technique and a method to combine auditory and visual information for estimating the physical location of the sound source. Secondly, the speaker is detected according to the linear combination of the result of sound source localization and the likelihood map representing the distribution of pixels having skin color. Finally, in the conversational situations, the speaker's focus of attention is alternatively detected and shared by the robot for upcoming communication based on the proposed algorithm. Several experimental results are presented that demonstrate the effectiveness of the proposed method. more...

In human-robot interaction, the ability to detect and share user’s attention is the minimal requirement for an intelligent robot, since it is very important for robot to know human’s internal state. Here we present an algorithm, which is based on face posture estimation and the spatiotemporal image processing, to calculate a saliency map in order to form share attention. After the face posture estimation, we introduce an elliptic cone to approximate the user’s visual field, whose axis is fitted to the user’s gaze line that is not necessary to be detected beforehand. A visual acuity map on the user’s retina can be yielded according to the formulization of human’s visual acuity. We calculate the saliency map in terms of recency-weighted average of visual acuity maps along time axis so that the dynamic scene (for example, the case that user’s gaze line is shifting to a new object or the gazed object is moving) can affect the saliency map calculation, as well as the moving image areas are tracked to propagate the value of the visual acuity map from the current frame to the next one. Finally, we use the saliency map to form share attention in human-robot interaction, and it is also manifested that it will be possible to detect the user’s attention by only considering face orientation even when the eyes cannot be observed clearly. more...