Search

Gait recognition is one measure of biometrics, which also includes facial, fingerprint, and retina recognition. Although most biometric methods require direct contact between a device and a subject...

Spatial change detection is a fundamental technique for finding the differences between two or more pieces of geometrical information. This technique is critical in some robotic applications, such as search and rescue, security, and surveillance. In these applications, it is desirable to find the differences quickly and robustly. The present paper proposes a fast and robust spatial change detection technique for a mobile robot using an on-board range sensors and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform. The voxels in the map and the measured data are then compared according to the features of the ND voxels. Three techniques are introduced to make the proposed system robust for noise, that is, classification of point distribution, overlapping of voxels, and voting using consecutive sensing. The present paper shows the results of indoor and outdoor experiments using an RGB-D camera and an omni-directional laser scanner mounted on a mobile robot to confirm the performance of the proposed technique.

In this paper, we describe a wearable first-person video (FPV) analysis system for evaluating the skill levels of caregivers. This is a part of our project that aims to quantize and analyze the tender-care technique known as Humanitude by using wearable sensing and AI technology devices. Using our system, caregivers can evaluate and elevate their care levels by themselves. From the FPVs of care sessions taken by wearable cameras worn by caregivers, we obtained the 3D facial distance, pose and eye-contact states between caregivers and receivers by using facial landmark detection and deep neural network (DNN)-based eye contact detection. We applied statistical analysis to these features and developed algorithms that provide scores for tender-care skill. In experiments, we first evaluated the performance of our DNN-based eye contact detection by using eye contact datasets prepared from YouTube videos and FPVs that assume conversational scenes. We then performed skill evaluations by using Humanitude training scenes involving three novice caregivers, two Humanitude experts and seven middle-level students. The results showed that our eye contact detection outperformed existing methods and that our skill evaluations can estimate the care skill levels., 優しさを伝える介護技術の習熟度をAIで評価する手法を開発 --画像認識で熟練者と初心者の違いを見つける--. 京都大学プレスリリース. 2019-07-11.

Semantic place categorization, which is one of the essential tasks for autonomous robots and vehicles, allows them to have capabilities of self-decision and navigation in unfamiliar environments. I...

This paper presents a method for reconstructing 3D image from multi-focus microscopic images captured with different focuses. We model the multi-focus imaging by a microscopy and produce the 3D image of a target object based on the model. The 3D image reconstruction is done by minimizing the difference between the observed images and the simulated images generated by the imaging model. Simulation and experimental result shows that the proposed method can generate the 3D image of a transparent object efficiently and reliably.

This paper presents a method for synthesizing multi-focus cell images by using generative adversarial networks (GANs). The proposed method, called multi-focus image GAN (MI-GAN), consists of two generators. A base image generator synthesizes a 2D base cell image from random noise. Using the generated base image, a multi-focus cell image generator produces 11 realistic multi-focus images of the cell while considering the relationships between the images acquired at successive focus points. From experimental results, MI-GAN achieves the good performance to generate realistic multi-focus cell images.

Terrain imagery collected by satellite remote sensing or by rover on-board sensors is the primary source for terrain classification used in determining terrain traversibility and mission plans for planetary rovers. Mapping models between RGB and IR for terrain classes are learned from real RGB and IR data examples in the same or similar terrain. This paper adds a new class of deep learning architectures called MU-Net (Multiple U-Net) and shows its efficiency in deriving better RGB-to-IR mapping models, improving over past work the estimation of thermal IR images from incoming RGB images and learned RGB-IR mappings.

Detection of spatial change around a robot is indispensable in several robotic applications, such as search and rescue, security, and surveillance. The present paper proposes a fast spatial change detection technique for a mobile robot using an on-board RGB-D/stereo camera and a highly precise 3D map created by a 3D laser scanner. This technique first converts point clouds in a map and measured data to grid data (ND voxels) using normal distributions transform and classifies the ND voxels into three categories. The voxels in the map and the measured data are then compared according to the category and features of the ND voxels. Overlapping and voting techniques are also introduced in order to detect the spatial changes more robustly. We conducted experiments using a mobile robot equipped with real-time range sensors to confirm the performance of the proposed real-time localization and spatial change detection techniques in indoor and outdoor environments.

In this paper we propose two novel deep learning-based terrain classification methods robust to illumination changes. The use of cameras is challenged by a variety of factors, of most importance being the changes in illumination. On the other hand, since the temperature of various types of terrains depends on the thermal characteristics of the terrain, the terrain classification can be aided by utilizing the thermal information in addition to visible information. Thus we propose 'TU-Net (Two U-Net)' based on the U-Net and 'TDeepLab (Two DeepLab)' based on DeepLab, which combine visible and thermal images and train the network robust to illumination changes implicitly. To improve the network's learning capability, we expand the proposed methods to the Siamese-based method, which explicitly trains the network to be robust to illumination changes. We also investigate multiple options to fuse the visible and thermal images at at the bottom layer, middle layer, or the top layer of the network. We evaluate the proposed methods with a challenging new dataset consisting of visible and thermal images, which were collected from 10 am till 5 pm (after sunset), and we show the effectiveness of the proposed methods.

This paper presents a new method for generating object mesh models composed of hexahedra. The proposed method is based on a self-organizing deformable model (SDM) which is a deformable surface model guided by competitive learning and an energy minimization approach. Extending the SDM, the proposed method generates a hexahedral mesh model of a target object by fitting a cuboid composed of rectangular voxels to the object. Moreover, the shape of each hexahedron in the model is corrected by dividing the hexahedron into sub-hexahedra and moving the nodes of the hexahedron. From our experimental results, the proposed method obtains the hexahedral mesh model which consists of many regular hexahedra while recovering the shape of the object.

We propose a method that increases the capability of a conventional sensor, transforming it into an enhanced virtual sensor. This paper focuses on a virtual thermal Infrared Radiation (IR) sensor based on a conventional visual (RGB) sensor. The estimation of thermal IR images can enhance the ability of terrain classification, which is crucial for autonomous navigation of rovers. The estimate in IR from visual band has inherent limitations, as these are different bands, yet correlations between visual RGB and thermal IR images exist, as different terrains, which visually may appear different, also have different thermal inertia. This paper describes the developed deep learning-based algorithm that estimates thermal IR images from RGB images of terrains, providing the feasibility of the idea with average 1.21 error [degree Celsius].

This paper presents an effective integration method of multiple modalities such as depth, color, and reflectance for place categorization. To achieve better performance with integrated multi-modalities, we introduce a novel descriptor, local N-ary patterns (LTP), which can perform robust discrimination of place categorization. In this paper, the LNP descriptor is applied to a combination of two modalities, i.e. depth and reflectance, provided by a laser range finder. However, the LNP descriptor can be easily extended to a larger number of modalities. The proposed LNP describes relationships between the multi-modal values of pixels and their neighboring pixels. Since we consider the multi-modal relationship, our proposed method clearly demonstrates more effective classification results than using individual modalities. We carried out experiments with the Kyushu University Indoor Semantic Place Dataset, which is publicly available. This data-set is composed of five indoor categories: corridors, kitchens, la...

Daily life assistance is one of the most important applications for service robots. For comfortable assistance, service robots must recognize the surrounding conditions correctly, including human motion, the position of objects, and obstacles. However, since the everyday environment is complex and unpredictable, it is almost impossible to sense all of the necessary information using only a robot and sensors attached to it. In order to realize a service robot for daily life assistance, we have been developing an informationally structured environment using distributed sensors embedded in the environment. The present paper introduces a service robot system with an informationally structured environment referred to the ROS-TMS. This system enables the integration of various data from distributed sensors, as well as storage of these data in an on-line database and the planning of the service motion of a robot using real-time information about the surroundings. In addition, we discuss experiments such as detection and fetch-and-give tasks using the developed real environment and robot. Introduction of architecture and components of the ROS-TMS.Integration of various data from distributed sensors for service robot system.Object detection system (ODS) using RGB-D camera.Motion planning for a fetch-and-give task using a wagon and a humanoid robot.Handing over an object to a human using manipulability of both a robot and a human.

We present a novel method for the outdoor scene categorization using 2D convolutional neural networks (CNNs) which take panoramic depth images obtained by a 3D laser scanner as input. We evaluate our approach in two outdoor scene datasets including six categories: coast, forest, indoor parking, outdoor parking, residential area, and urban area. Our results on both datasets (over 94%) outperform previous approaches and show the effectiveness of this approach for outdoor scene categorization using depth images. To analyze our trained networks we visualize the learned features by using two visualization methods.

This paper presents a near-future perception system named “Previewed Reality”. The system consists of an informationally structured environment (ISE), an immersive VR display, a stereo camera, an optical tracking system, and a dynamic simulator. In an ISE, a number of sensors are embedded, and information such as the position of furniture, objects, humans, and robots, is sensed and stored in a database. The position and orientation of the immersive VR display are also tracked by an optical tracking system. Therefore, we can forecast the next possible events using a dynamic simulator and synthesize virtual images of what users will see in the near future from their own viewpoint. The synthesized images, overlaid on a real scene by using augmented reality technology, are presented to the user. The proposed system can allow a human and a robot to coexist more safely by showing possible hazardous situations to the human intuitively in advance.

Mutual subspace method (MSM), which is one of image-based approaches, showed strong discrimination capability in gait recognition. In general, 2D image matrices are transformed into 1D image vectors to be used as input into MSM, and then principal component analysis (PCA) is applied to 1D vectors to generate a subspace. However, due to the high dimensionalities of 1D vectors, the evaluation accuracy of the covariance matrix in PCA is not high enough. This results in a decrease in performance, especially in case that speed difference between gallery and probe dataset is big. Thus in this paper we propose a method, which expands the MSM-based method, to recognize people with higher accuracy. The proposed method divides the human body area into multiple areas, followed by adaptive choice of areas that have high discrimination capability. Moreover, the proposed method utilizes the frieze pattern, which is one of gait features, as an additional input into MSM. The use of divided areas and the frieze pattern allows us to evaluate the covariance matrix with higher accuracy. In experiments we applied the proposed method to challenging databases with speed variations, and we show the effectiveness of the proposed method.

While laser scanners can produce a high-precision 3D shape of a real object, appearance information of the object has to be captured by an image sensor, such as a digital camera. This paper proposes a novel and simple technique for colorizing 3D geometric models based on laser reflectivity. Laser scanners capture the range data of a target object from the sensors. Simultaneously, the power of the reflected laser is obtained as a by-product of the range data. The reflectance image, which is a collection of laser reflectance depicted as a grayscale image, contains rich appearance information about the target object. The proposed technique is an alternative to texture mapping, which has been widely used to realize photo-realistic 3D modeling but requires strict alignment between range data and texture images. The proposed technique first colorizes a reflectance image based on the similarity of color and reflectance images. Then the appearance information (color and texture information) is added to a 3D model...

Conventional methods of gait analysis for person identification use features extracted from a sequence of camera images taken during one or more gait cycles. The walking direction is implicitly assumed not to change. However, with the exception of very particular cases, such as walking on a circle centered on the camera, or along a line passing through the camera, there is always some degree of orientation change, most pronounced when the person is closer to the camera. This change in the angle between the velocity vector and the position vector in respect to the camera causes a decrease in performance for conventional methods. To address this issue we propose in this paper a new method, which provides improved identification in this context of orientation change. The proposed method uses a 4D gait database consisting of multiple 3D shape models of walking people and adaptive virtual image synthesis with high accuracy. Each frame, for the duration of a gait cycle, is used to estimate the walking direction of the subject, and a virtual image corresponding to the estimated direction is synthesized from the 4D gait database. The identification uses affine moment invariants as gait features. The efficiency of the proposed method is demonstrated through experiments using a database that includes 42 subjects.

This paper describes a new method of measuring the position of everyday objects and a robot on the floor using distance and reflectance acquired by laser range finder (LRF). The information obtained by this method is important for a service robot working in a human daily life environment. Our method uses only one LRF together with a mirror installed on the wall. Moreover, since the area of sensing is limited to a LRF scanning plane parallel to the floor and just a few centimeters above the floor, the scanning covers the whole room with minimal invasion of privacy of a resident, and occlusion problem is mitigated by using mirror. We use the reflection intensity and position information obtained from the target surface. Although it is not possible to identify all objects by additionally using reflection values, it would be easier to identify unknown objects if we can eliminate easily identifiable objects by reflectance. In addition, we propose a method for measuring the robot’s pose using the tag which has the encoded reflection pattern optically identified by the LRF. Our experimental results validate the effectiveness of the proposed method.

In this paper, we propose new denoising techniques for a deteriorated range image taken by a laser scanner. Laser scanner acquires a range value from the scanner to the target by measuring the round-trip time of the emitted laser pulse. At the same time, they can obtain the strength of the reflected light as a side product of the range value. Focusing on the laser intensity, we propose two denoising techniques for a deteriorated range image utilizing the intensity image: smoothing by extended bilateral filter, and completion by belief propagation. The extended bilateral filter makes use of laser intensity in addition to the spatial and range information in order that we can smooth a range image corrupted by noises while the geometric features such as jump and roof edges are preserved. The range image completion technique with belief propagation restores a deteriorated range image using the adjacent range values and the corresponding intensity values simultaneously. We conduct simulations and experiments u...

This paper presents a new object classification technique for 3D point cloud data acquired with a laser scanner. In general, it is not straightforward to distinguish objects that have similar 3D structures but belong to different categories based only on the range data. To tackle this issue, we focus on laser reflectance obtained as a side product of range measurement by a laser scanner. Since laser reflectance contains appearance information, the proposed method classifies objects based on not only geometrical features in range data but also appearance features in reflectance data, both of which are acquired by a single laser scanner. Furthermore, we extend the conventional Histogram of Oriented Gradients (HOG) so that it couples geometrical and appearance information more tightly. Experiments show the proposed technique combining geometrical and appearance information outperforms conventional techniques.

This paper proposes a robust visual servoing method for object manipulation against temporary loss of sensory information. It is known that visual information is useful for reliable object grasping and precise manipulation. Visual information becomes unavailable, however when occlusion occurs or a grasped object disappears during manipulation. In that case, the behavior of the visual servoing system becomes unstable. Our proposed method enables an object to be grasped and manipulated stably even if visual information is temporarily unavailable during manipulation. This method is based on dynamic stable object grasping and manipulation proposed in our previous work and the concept of virtual object information. A dynamic model of the overall system is first formulated. A new controller using both actual and virtual object information is proposed next. The usefulness of this method is finally verified through both numerical simulation and experiments using a triple-fingered mechanical hand.

Global localization, which determines an accurate global position without prior knowledge, is a fundamental requirement for a mobile robot. Map-based global localization gives a precise position by comparing a provided geometric map and current sensory data. Although 3D range data is preferable for 6D global localization in terms of accuracy and reliability, comparison with large 3D data is quite timeconsuming. On the other hand, appearance-based global localization, which determines the global position by comparing a captured image with recorded ones, is simple and suitable for real-time processing. However, this technique does not work in the dark or in an environment in which the lighting conditions change remarkably. We herein propose a two-step strategy, which combines map-based global localization and appearance-based global localization. Instead of camera images, which are used for appearance-based global localization, we use reflectance images, which are captured by a laser range finder as a byproduct of range sensing. The effectiveness of the proposed technique is demonstrated through experiments in real environments.

To provide daily-life assistance appropriately by a service robot, the management of houseware's information in a room or a house is an indispensable function. Especially, the information about what and where objects are in the environment are fundamental and critical knowledge. We can track housewares with high reliability by attaching markers such as RFID tags to them, however, markerless housewares management system is still useful since it is easy-to-use and low cost. In this work, we present an object management system using an egocentric vision and a region-based convolutional neural network (R-CNN) to automatically detect and register housewares. The proposed system consists of smart glasses equipped with a wearable camera, a cloud database which manages object information, and a processing server for detecting and registering housewares to the cloud database. We perform two experiments. First, we train the R-CNN on a newly-constructed dataset to detect various housewares and configure a houseware-specific detector. All systems are composed of ROS packages. Second, we conduct experiments for automatic housewares registration using the proposed system. We demonstrate that the proposed system can detect, recognize, and register housewares approximately in real time.

This paper proposes a new method for mapping volume models of human organs onto a target volume with simple shapes. The proposed method is based on our modified Self-organizing Deformable Model (mSDM) which finds the one-to-one mapping with no foldovers between an arbitrary object surface model and a target surface. By extending mSDM to apply to organ volume models, the proposed method, called volumetric SDM (vSDM), establishes the one-to-one correspondence between the volume model and its target volume. At the same time, vSDM preserves geometrical properties of the original model before and after the mapping. In addition, vSDM allows to control the mapping of interior structures of the organ model onto specific regions inside the target volume. These characteristics of vSDM enables to easily find a reliable correspondence between different volume models via a common target volume.

We propose an object tracking system which recognizes everyday objects and estimates their positions by using distributed sensors in a room and mobile robots. The placement of objects is frequently changed according to human activities. Although a passive RFID tag is attached to each object for the object's recognition, the placement is often not uniquely determined due to the deficiency of measured data. We have already proposed a method for estimating the placement of objects by using the moving trajectories of objects. This estimation result is expressed as the probability distribution of the object placement. However intersections of trajectories cause the decrease of the estimation accuracy. So we propose a new method based on Bayesian inference to improve the estimation accuracy by using the size and the shape of an object measured by laser range finder. Then a mobile robot settles the placement with small workload by using the mounted sensor. The system successfully recognized and localized 10 objects in the experiment.

We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42% (dense) and 89.67% (sparse).

In this paper, for the purpose of monitoring people moving on the ground from a low-altitude aerial vehicle, we propose a method for stable image stabilization using planar information of the ground. In existing methods the homography-based method has been popularly used for image stabilization. However, in case that captured images consist of non-planar areas, the performance of the homography-based method decreases. The essential-based method works well in this scene, but the accuracy gets worse in case that images mainly include planar areas. Thus in the proposed method, we utilize the Geometric Robust Information Criterion (GRIC) model to choose a method from the homography-based method or the essential-based method. After the selection of the method, we extract a planar area in the scene by fitting a plane using RANSAC, followed by detection of people on the ground with high accuracy. Experimental results confirm that the effectiveness of the proposed method.

A robotic 3D scanning system using multiple robots named CPS-SLAM is presented.An automatic planning technique for a laser measurement by using CPS-SLAM is proposed.The validity is verified through simulations and experiments by two types of CPS-SLAMs. 3D modeling of real objects by a 3D laser scanner has become popular in many applications, such as reverse engineering of petrochemical plants, civil engineering and construction, and digital preservation of cultural properties. Despite the development of lightweight and high-speed laser scanners, the complicated measurement procedure and long measurement time are still heavy burdens for widespread use of laser scanning. To solve these problems, a robotic 3D scanning system using multiple robots has been proposed. This system, named CPS-SLAM, consists of a parent robot with a 3D laser scanner and child robots with target markers. A large-scale 3D model is acquired by an on-board 3D laser scanner on the parent robot from several positions determined precisely by a localization technique, named the Cooperative Positioning System (CPS), that uses multiple robots. Therefore, this system can build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a very precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique for a laser measurement by using CPS-SLAM. Planning a proper scanning strategy depending on a target structure makes it possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. The proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.

We propose a novel biometrics method based on shadows (shadow biometrics, SB) and introduce a SB-based person identification method for a vision-based surveillance system. Conventional biometric identification based on body movements, as is the case in gait recognition, uses cameras that provide a good view of entire human body. Aerial search and surveillance systems only see the human body from top view with a smaller cross-section and with less details than seen in side views, which is further aggravated by the lower resolution associated with this imagery. Shadows, i.e. body projections due to the Sun, or artificial lights at night, can offer body biometrics information that cannot be directly seen in body top view. In this paper we use SB for person identification, automatically extracting shadows in captured video images, and processing them to extract gait features, further analyzed by spherical harmonics. We demonstrate shadow-based person identification in experiments inside a building using artificial light and outside under the Sun. The introduced method using spherical harmonics outperforms methods based on Fourier transform, gait energy image, and active energy image. Furthermore, we show that the combination of body and shadow areas, as seen from an oblique camera on an upper floor of a building, has better performance than using body only or shadow only information.

This paper proposes a new smoothing technique for range images taken by a pulsed Time-of-Flight range finder. A Time-of-Flight range finder gives a range value from the sensor to the target by measuring the time until the emitted light is reflected on the target's surface and returned to the sensor. At the same time, the power of the reflected light is also measured as a side product of the range value for the most of range finders. The basic idea of the proposed technique is that the reflectance and range images are utilized for smoothing the range image corrupted by noise so that the detailed shape in the range image is preserved by applying the trilateral filter. Simulations and experiments using a laser range finder are successfully carried out for verifying the performance of the proposed technique.

For the construction of 3-D shape models of large-scale architectural structures using laser range finders, a number of range images are taken from different viewpoints around the targets. Next, the obtained images are normally aligned by post-processing procedures, such as the ICP algorithm. However, to obtain convergent results in the ICP algorithm and align these range images to their proper positions, the initial position of each range image needs to be manually aligned to roughly the correct position. This paper proposes a new measurement and modeling system using a group of multiple robots and an on-board laser range finder. Each measurement position is identified by a highly precise positioning technique called the Co-operative Positioning System (CPS), which utilizes the characteristics of the multiple-robot system. Therefore, the proposed system can construct 3-D shapes of large-scale architectural structures without any post-processing procedure or manual intervention. In addition, it is possible to register range images even if the number of measurements is few and there are only a few range images, for example, due to range images containing insufficient feature shapes or overlapping regions. Measurement experiments in unknown and large indoor/outdoor environments including a large hall, a building, an urban district, and a cultural heritage have been successfully carried out using the newly developed measurement system consisting of three mobile robots named CPS-V. Path generation experiments of the mobile robots based on the partially measured 3-D model are also presented.

People detection is a key capacity for robotics systems that have to interact with humans. This paper addresses the problem of detecting people using multiple layers of 2D laser range scans. Each layer contains a classifier able to detect a particular body part such as a head, an upper body or a leg. These classifiers are learned using a supervised approach based on AdaBoost. The final person detector is composed of a probabilistic combination of the outputs from the different classifiers. Experimental results with real data demonstrate the effectiveness of our approach to detect persons in indoor environments and its ability to deal with occlusions.

We are conducting the research project named Robot Town Project. The aim of this research is to develop a distributed sensor system such as cameras, laser range finders, and IC tags, and its management system so that autonomous robots can work with humans in an ordinary environment for daily human life. This paper presents a sensor network system consisting of distributed cameras and laser range finders for multiple objects tracking. Sensory information from cameras is processed by the Level Set Method in real time and integrated with range data obtained by laser range finders in a probabilistic manner using novel SIR/MCMC combined particle filters. Though the conventional SIR particle filter is a popular technique for object tracking, it has been pointed out that there are some drawbacks in practical applications such as its low tracking performance for multiple targets due to the degeneracy problem. In this paper, the new combined particle filters consisting of a low-resolution MCMC particle filter and a high-resolution SIR particle filter is proposed. Simultaneous tracking experiments for multiple moving targets are successfully carried out and it is verified that the combined particle filters has higher performance than the conventional particle filters in terms of the number of particles, the processing speed, and the tracking performance for multiple targets.

In medical diagnostic imaging, an X-ray CT scanner or a MRI system have been widely used to examine 3D shapes or internal structures of living organisms or bones. However, these apparatuses are generally very expensive and of large size. A prior arrangement is also required before an examination, and thus, it is not suitable for an urgent fracture diagnosis in emergency treatment. This paper proposes a method to estimate a patient-specific 3D shape of a femur from only two fluoroscopic images using a parametric femoral model. Firstly, we develop a parametric femoral model by statistical analysis of a number of 3D femoral shapes created from CT images of 51 patients. Then, the pose and shape parameters of the parametric model are estimated from two 2D fluoroscopic images using a distance map constructed by the level set method. Experiments using synthesized images and fluoroscopic images of a phantom femur are successfully carried out and the usefulness of the proposed method is verified.

This paper proposes a calibration method of a distributed vision network in a unified world coordinate system. The vision network system is conceived to support a robot working in our daily human life environment: the system provides with visual observation of the dynamically changing situation surrounding the robot. Vision cameras are rather sparsely distributed to cover a wide area such as a block of a town. Position, view direction and range of view are the camera parameters of principal importance to be estimated by the proposed method. A set of calibration data for each distributed camera is provided by a group of mobile robots having a cooperative positioning function and visually distinguishable markers mounted on the body of the robot.

This paper proposes a new method for mapping volume models of human tissues onto a target volume with simple shapes. The proposed method is based on our modified self-organizing deformable model (mSDM)1, 2 which finds the one-to-one mapping with no foldovers between arbitrary surface model and a target surface. By extending mSDM to apply to volume models, the proposed method, called volumetric SDM (vSDM), establishes the one-to-one correspondence between the tissue volume model and its target volume. At the same time, vSDM can preserve geometrical properties of the original model before and after mapping. This characteristic of vSDM makes it easy to find the correspondence between tissue models.

This paper presents an overview of our research project on digital preservation of cultural heritage objects and digital restoration of the original appearance of these objects. As an example of these objects, this project focuses on the preservation and restoration of the Great Buddhas. These are relatively large objects existing outdoors and providing various technical challenges. Geometric models of the great Buddhas are digitally achieved through a pipeline, consisting of acquiring data, aligning multiple range images, and merging these images. We have developed two alignment algorithms: a rapid simultaneous algorithm, based on graphics hardware, for quick data checking on site, and a parallel alignment algorithm, based on a PC cluster, for precise adjustment at the university. We have also designed a parallel voxel-based merging algorithm for connecting all aligned range images. On the geometric models created, we aligned texture images acquired from color cameras. We also developed two texture mapping methods. In an attempt to restore the original appearance of historical objects, we have synthesized several buildings and statues using scanned data and a literature survey with advice from experts.

This paper presents a new registration algorithm of two-dimensional (2-D) color images and 3-D geometric models for the navigation system of a surgical robot. A 2-D–3-D registration procedure is used to precisely superimpose a tumor model on an endoscopic image and is, therefore, indispensable for the surgical navigation system. Thus, the performance of the 2-D–3-D registration procedure influences directly the usability of the surgical robot operating system. One of the typical techniques that has been developed is the use of external markers. However, the accuracy of this method is reduced by the breathing or heartbeat of the patient, as well as other unknown factors. For precise registration of 3-D models and 2-D images without external markers or special measurement devices, a new registration method is proposed that utilizes the 2-D images and their distance maps as created by the Fast Marching Method. Here, we present results of fundamental experiments performed using simulated models and actual ima...

In order to construct three-dimensional shape models of large-scale architectural structures using a laser range finder, a number of range images are normally taken from various viewpoints, and these images are aligned using post-processing procedures such as the ICP algorithm. However, in general, before applying the ICP algorithm, these range images must be registered to roughly correct positions by a human operator in order to converge to precise positions. In addition, range images must be made to sufficiently overlap each other by taking dense images from close viewpoints. On the other hand, if the positions of the laser range finder at viewpoints can be identified pre-cisely, local range images can be directly converted to the global coordinate system through a simple transformation calculation. The present paper proposes a new measurement system for large-scale architectural structures using a group of multiple robots and an on-board laser range finder. Each measurement position is identified by a highly precise positioning technique called Cooperative Positioning System (CPS), which utilizes the characteristics of the multiple-robot system. The proposed system can construct 3D shapes of large-scale architectural structures without any post-processing procedure such as the ICP algorithm or dense range measurements. Measurement experiments in unknown and large indoor/outdoor environments are successfully carried out using the newly developed measurement system consisting of three mobile robots named CPS-V.

This paper proposes new features extracted from images derived from optical flow, for first-person activity recognition. Features from convolutional neural network (CNN), which is designed for 2D images, attract attention from computer vision researchers due to its powerful discrimination capability, and recently a convolutional neural network for videos, called C3D (Convolutional 3D), was proposed. Generally CNN / C3D features are extracted directly from original images / videos with pre-trained convolutional neural network, since the network was trained with images / videos. In this paper, on the other hand, we propose the use of images derived from optical flow (we call this image as "optical flow image") as input images into the pre-trained neural network, based on the following reasons; (i) optical flow images give dynamic information which is useful for activity recognition, compared with original images, which give only static information, and (ii) the pre-trained network has chance to extract features with reasonable discrimination capability, since the network was trained with huge amount of images from big categories. We carry out experiments with a dataset named "DogCentric Activity Dataset", and we show the effectiveness of the extracted features.

This paper proposes a new concept named "fourth-person sensing" for a service robot. The proposed concept combines wearable sensors (the first-person viewpoint), sensors mounted on robots (the second-person viewpoint), and sensors embedded in the environment (the third-person viewpoint), and the disadvantages in individual sensors are compensated by combining sensory information from the first, second, and third-person viewpoints. The fourth-person sensing is effective to understand a user's intention and a context of the scene, thus it enables to provide a proper service by a service robot. As one of applications of the fourth-person sensing, we develop a HCI system combining the first-person and the third-person sensing, and show the effectiveness of the proposed concept through robot service experiments.

In recent years, several low-cost 3D laser scanners are being brought to the market and 3D laser scanning is becoming widely used in many fields. For example, 3D modeling of architectural structures or digital preservation of cultural heritages are typical applications for 3D laser scanning. Despite of the development of light-weight and high-speed laser scanners, however, the complicated measurement procedure and long measurement time are still a heavy burden for the widespread use of laser scanning. We have proposed a robotic 3D scanning system using multiple robots named CPS-SLAM, which consists of parent robots with a 3D laser scanner and child robots with target markers. In this system, a large-scale 3D model can be acquired by an on-board 3D laser scanner on a parent robot from several positions determined precisely by the localization technique using multiple robots named Cooperative Positioning System, CPS. Therefore, this system enables to build a 3D model without complicated post-processing procedures such as ICP. In addition, this system is an open-loop SLAM system and a quite precise 3D model can be obtained without closed loops. This paper proposes an automatic planning technique of a laser measurement for CPS-SLAM. By planning a proper scanning strategy depending on a target structure, it is possible to perform laser scanning efficiently and accurately even for a large-scale and complex environment. Proposed technique plans an efficient scanning strategy automatically by taking account of several criteria, such as visibility between robots, error accumulation, and efficient traveling. We conducted computer simulations and outdoor experiments to verify the performance of the proposed technique.