Search

Objective : Augmented reality (AR) navigation using a position sensor in endoscopic surgeries relies on the quality of patient–image registration and hand–eye calibration. Conventional methods collect the necessary data to compute two output transformation matrices separately. However, the AR display setting during surgery generally differs from that during preoperative processes. Although conventional methods can identify optimal solutions under initial conditions, AR display errors are unavoidable during surgery owing to the inherent computational complexity of AR processes, such as error accumulation over successive matrix multiplications, and tracking errors of position sensor. Methods : We propose the simultaneous optimization of patient–image registration and hand–eye calibration in an AR environment before surgery. The relationship between the endoscope and a virtual object to overlay is first calculated using an endoscopic image, which also functions as a reference during optimization. After including the tracking information from the position sensor, patient–image registration and hand–eye calibration are optimized in terms of least-squares. Results : Experiments with synthetic data verify that the proposed method is less sensitive to computation and tracking errors. A phantom experiment with a position sensor is also conducted. The accuracy of the proposed method is significantly higher than that of the conventional method. Conclusion : The AR accuracy of the proposed method is compared with those of the conventional ones, and the superiority of the proposed method is verified. Significance : This study demonstrates that the proposed method exhibits substantial potential for improving AR navigation accuracy.

Typical visual placement inspection systems in semiconductor or electronics manufacturing industries use only highly-magnified two-dimensional imaging with controlled illumination for high accuracy. However, current semiconductor packaging technology needs more precise manufacturing processes for highly integrated semiconductor packages, in which high-precision alignment between the semiconductor die and the matched substrate is indispensable for successful electronic interconnection. To solve this problem, an advanced visual placement inspection system for 3D stacking of electronic components is proposed; it utilizes 3D profilometry and 2D telecentric optics. In addition to the introduction of the proposed system, the system calibration algorithms and the information processing algorithms for accurate positioning of the semiconductor die, as well as the implemented inspection system are described in detail. To verify the system performance, a series of real experiments on newly developed flip-chip packages for high-performance computing were performed, and the results are analyzed and discussed.

Objective Renal ischemia/reperfusion (I/R) injury is characterized by the acute deterioration of renal function during ischemia and renal inflammation. Cassia tora has various effects, including antioxidant, antidiabetic, and hypolipidemic properties. In the present study, we investigated whether C tora has a renoprotective effect on I/R-induced acute kidney injury in rats. Methods We fed Sprague-Dawley rats either C tora (100 mg/kg/d) or saline. One week later, ischemia was induced by bilateral renal pedicle occlusion for 30 minutes, followed by reperfusion. Rats were randomized into 3 major groups, which were treated as follows: 1. the sham operation group; 2. the I/R group; and 3. the I/R-C tora group. Results Compared to the sham group, the I/R group had higher levels of blood urea nitrogen and serum creatinine in serum and lower expression of catalase, superoxide dismutase, glutathione peroxidase, antioxidant, and nitric oxide. Compared to the I/R group, the I/R-C tora group had higher expression of catalase, superoxide dismutase, glutathione peroxidase, antioxidant, and nitric oxide, as well as lower levels of blood urea nitrogen and creatinine in serum. Conclusions These results suggest that C tora has significant therapeutic effects in ischemic renal injury.

The reduction in current ability accompanied by the hump phenomenon in oxide semiconductor thin-film transistors to which high DC voltages and AC drive voltages are applied has not been studied extensively, although it is a significant bottleneck in the manufacture of integrated circuits. Here, we report on the origin of the hump and current drop in reliability tests caused by the degradation in the oxide semiconductor during a circuit driving test. The hump phenomenon and current drop according to two different driving stresses were verified. Through a numerical computational simulation, we confirmed that this issue can be caused by an additional “needle”, a shallow (~0.2 eV) and narrow (

The C-arm X-ray system is a common intraoperative imaging modality used to observe the state of a fractured bone in orthopedic surgery. Using C-arm, the bone fragments are aligned during surgery, and their lengths and angles with respect to the entire bone are measured to verify the fracture reduction. Since the field-of-view of the C-arm is too narrow to visualize the entire bone, a panoramic X-ray image is utilized to enlarge it by stitching multiple images. To achieve X-ray image stitching with feature detection, the extraction of accurate and densely matched features within the overlap region between images is imperative. However, since the features are highly affected by the properties and sizes of the overlap regions in consecutive X-ray images, the accuracy and density of matched features cannot be guaranteed. To solve this problem, a heterogeneous stitching of X-ray images was proposed. This heterogeneous stitching was completed according to the overlap region based on homographic evaluation. To acquire sufficiently matched features within the limited overlap region, integrated feature detection was used to estimate a homography. The homography was then evaluated to confirm its accuracy. When the estimated homography was incorrect, local regions around the matched feature were derived from integrated feature detection and substituted to re-estimate the homography. Successful X-ray image stitching of the C-arm was achieved by estimating the optimal homography for each image. Based on phantom and ex-vivo experiments using the proposed method, we confirmed a panoramic X-ray image construction that was robust compared to the conventional methods.

Asymmetric sized single oxide diode (InZnOx/CuOx) with single resistor (Ti-doped NiO) device (1D-1R) was integrated into a crossbar array at room temperature to provide high current through the diode while keeping resistance switching localized in a small area. The Ti (0.1 wt %) doped NiO layer was fabricated inside an inverted cone-structure to localize switching and improve general device performance. We were able observe resistance switching up to around 100 cycles in the 1D-1R structure. In addition, the addition of the diode to the structure acted as an external resistance suppressing overflow current during Ti-doped NiO switching from a high resistance state to a low resistance state, thus improving switching distribution in both low (average = 3.2×10-5 Ω, standard deviation = 1.3×10-6) and high (average = 2.8×10-6 Ω, standard deviation = 6.7×10-7) resistance states.

As drones become more advanced and commercialized, crimes using drones are also on rise. For this reason, development of anti-drone systems is increasing. In this paper, CNN model is examined that is suitable for visible camera-based drone identification. The CNN models used for the validation are Alexnet, GoLeNet, Inception-v3 Vg16, Resnet-18, Resnet-50 and Squezezenet. These seven models have already been validated in the ImageNet Large Scale Visual Recognition Competition (ILSVRC). In ILSVRC, 1000 labels are classified, but in this study limits them to three drones, birds and backgrounds. Therefore, it is necessary to verify whether the three labels are the same as the ILSVRC result. In order to verify this, CNN models are learned and tested in the same environment. The experimental results show that the performance of Alexnet, Resnet and Squeeznet is relatively better then the others, unlike the performance of CNN known through ILSVRC. his result shows that a shallow network with a simple structure is more reasonable when the number of labels is small. Based on these results, the further work is to develop a neural network optimized for Drone identification.

An important and challenging research issue associated with mobile robots is simultaneous localization and map building (SLAM), which refers to the mobile robot's capability of estimating its poses in the environment without external information, and simultaneous alignment of the local maps. To solve these kinds of research problems, researchers have proposed various methods such as odometry measurement, landmark matching, laser range image matching and a scale-invariant feature transform (SIFT)–based algorithm, all of which suffer from inevitable drawbacks such as a local minimum problem and a lack of SIFT features. Our solution for these problems is a sensor fusion method that uses a Dempster Shafer algorithm to fuse both the laser range information and the SIFT features information for the SLAM. Through a series of experiments, we tested and evaluated the proposed method. By real experiments, we analyzed the parameters of the ICP and SIFT features and we checked the robustness of our algorithm.

This article deals with a novel 3-D sensor system that employs a measurement technique called the Stereo Moire technique. The general Moire method, which is well known for fast and accurate 3-D measurement, has suffered from an inherent limitation in that it cannot measure the absolute 3-D depth due to 2π ambiguity caused by regularity of fringe. In this article, to overcome this 2π ambiguity and obtain the exact depth value effectively, a novel method is proposed. The measurement system consists of a digital pattern projector and two cameras, which can indirectly detect two phase information on the fringe patterns projected onto an object's surface. These two phase values are different due to the difference in camera position. However, the depth values obtained at two different phase information must be identical because the two cameras look at the same point on the object being measured. To verify the efficiency and accuracy of the proposed method, a series of experimental tests was performed. Through t...

This paper deals with the synthesis of the 3-dimensional grasp planning for unknown objects. Previous studies have many problems, which the estimation time for finding the grasping points is much long and the analysis used the not-perfect 3-dimensional modeling. To overcome these limitations in this paper new algorithm is proposed, which algorithm is achieved by two steps. First step is to find the whole 3-dimensional geometrical modeling for unknown objects by using stereo matching. Second step is to find the optimal grasping points for unknown objects by using the neural network trained by the result of optimization using genetic algorithm. The algorithm is verified by computer simulation, comparing the result between neural network and optimization.

The growth of Cr thin films prepared by DC-magnetron sputter deposition was investigated by measuring the evolution of the grain size distribution and by computing the film growth using Monte Carlo simulation. The variation of grain size distribution with the deposition temperature is analyzed in terms of the film growth mechanisms.

Phase measuring profilometry and moire methodology have been widely applied to the three-dimensional shape measurement of target objects, because of their high measuring speed and accuracy. However, these methods suffer from inherent limitations called a correspondence problem, or 2π-ambiguity problem. Although a kind of sensing method to combine well-known stereo vision and phase measuring profilometry (PMP) technique simultaneously has been developed to overcome this problem, it still requires definite improvement for sensing speed and measurement accuracy. We propose a dynamic programming-based stereo PMP method to acquire more reliable depth information and in a relatively small time period. The proposed method efficiently fuses information from two stereo sensors in terms of phase and intensity simultaneously based on a newly defined cost function of dynamic programming. In addition, the important parameters are analyzed at the view point of the 2π-ambiguity problem and measurement accuracy. To analyze the influence of important hardware and software parameters related to the measurement performance and to verify its efficiency, accuracy, and sensing speed, a series of experimental tests were performed with various objects and sensor configurations.

A roll-type conducting polymer film was explored as a flexible organic p-type thermoelectric leg using poly(3,4-ethylenedioxythiophene) (PEDOT) doped with tosylate. The PEDOT films were prepared through solution casting polymerization and rolled up for a roll-type leg. Due to the high flexibility, the roll-type PEDOT leg enabled easy contact to both top and bottom electrodes. Simulation on the dynamic heat transfer and convective cooling for a vertically roosted rod- and roll-type PEDOT leg showed that the temperature difference (ΔT) between the hot and cold sides of the leg was much higher in the roll than that of the rod. The PEDOT legs were integrated with n-type Bi2Te3 blocks, to give a 36-couple rigid-flexible thermoelectric generator (RF-TEG). The maximum output voltage from the 36-couple RF-TEG under a ΔT of 7.9 K was determined as 36.7 mV along with a high output power of 115 nW. A wearable RF-TEG was prepared upon the combination of the 36-couple RF-TEG with an arm warmer, to afford an output voltage of 10.6 mV, which was generated constantly and steadily from human wrist heat.

Nowadays, a number of 3D shape measurement methods have been developed such as stereo vision, laser structured light, and Phase Measuring Profilometry (PMP). They have many problems such as correspondence problem, 2π-ambiguity and occlusion problem. Among them, occlusion problem is common problem for 3D shape measurement and well known as difficult to solve. To solve this problem, in our previous research, active imaging system is introduced, which can change the viewing angle without changing of field-of-view (FOV), and can do the FOV with maintaining the viewing angle. And to decide the viewing direction and position, view planning algorithm is developed. The performance of our system is checked by simulation and a series of real experiments.

The ball grid array (BGA) has become one of the most popular packaging alternatives for high I/O devices in the industry with many advantages: high interconnection density and less packaging space and so on. In these days, the size of chip becomes small and the size of ball grid also becomes small, so the process of BGA alignment becomes more important and difficult. In this paper, the BGA alignment system before the oven process step is managed. The main difficult of inspecting the BGA alignment is that the substrate is always tilted due to irregular carrier size and in-line process. In this paper, to overcome this problem, tilt angle of substrate is measured by phase measuring profilometry (PMP), and then the compensated alignment offset calc ulation algorithm is suggested. The performance of our system is checked by a series of real experiments.

Nowadays, a number of 3D shape measurement methods have been developed such as stereo vision, laser structured light, and Phase Measuring Profilometry (PMP). They have many problems such as correspondence problem, 2π-ambiguity and occlusion problem. Among them, occlusion problem is common problem for 3D shape measurement and well known as difficult to solve. To solve this problem, we suggest a new sensing system based on optical scanning system and movable mirror system, which can change the viewing angle without changing of field-of-view (FOV), and can do the FOV with maintaining the viewing angle. The performance of our system is checked by simulation and a series of real experiments.

Automatic optical inspection (AOI) for printed circuit board (PCB) assembly plays a very important role in modern electronics manufacturing industries. Well-developed inspection machines in each assembly process are required to ensure the manufacturing quality of the electronics products. However, generally almost all AOI machines are based on 2D image-analysis technology. In this paper, a 3D-measurement-method-based AOI system is proposed consisting of a phase shifting profilometer and a stereo vision system for assembled electronic components on a PCB after component mounting and the reflow process. In this system information from two visual systems is fused to extend the shape measurement range limited by 2pi phase ambiguity of the phase shifting profilometer, and finally to maintain fine measurement resolution and high accuracy of the phase shifting profilometer with the measurement range extended by the stereo vision. The main purpose is to overcome the low inspection reliability problem of 2D-based inspection machines by using 3D information of components. The 3D shape measurement results on PCB-mounted electronic components are shown and compared with results from contact and noncontact 3D measuring machines. Based on a series of experiments, the usefulness of the proposed sensor system and its fusion technique are discussed and analyzed in detail.

Nowadays, a number of 3D measurement methods have been developed such as stereo vision, laser structured light and PMP (Phase Measuring Profilometry) method. However, they have its own limitations : 2π ambiguity, correspondence problem, long estimation time. To solve these problems, in our previous researches [9,13], we introduced a novel sensing method adopting stereo vision and PMP technique (stereo PMP algorithm). One other difficult problem is occlusion problem needed to tackle by the stereo PMP algorithm which uses the principle of stereo vision and two cameras. The occlusion problem cannot be solved by using the principle of typical stereo vision, because there is no correspondence point in occlusion area. In our previous research based on stereo PMP algorithm, however, phase information related to the projector's position is additionally used which gives more additional information. By using this additional information, we can solve the occlusion problem effectively. In order to detect occlusion area, we adopt the principle of Dynamic Programming, while to measure the depth the principle of typical PMP algorithm and the geometrical relationship of detected depth area. To verify the efficiency of the proposed method, a series of experimental tests were performed.

In this paper, fusion of visual and inertial information is proposed for attitude measurement of remotely-operated mobile robots. The primary objective is to eliminate the drift error of the inertial sensors using visual information for exact and fast sensing of attitude. By this fusion each output is not only combined together but also helps the other yield better result. The environment is assumed unknown and corner features are utilized for rotation matrix calculation, for corners are better extracted in nature scene than lines. The measured information is to be used for stabilizing the image sent to the operator and for stabilizing the robot body itself in bumpy terrain. In the paper, the fusion algorithm based on Kalman filter is utilized, and its performance is evaluated by experiment. For the experiment, high performance camera is used with low-quality gyro sensors. The experimental results show that the vision algorithm well tracks the attitude change of the camera, and by fusing that with gyro sensor output, we could get high rate angle measurement without drift error.

Nowadays many parts of shipbuilding process are automated, but the painting process is not, because of the difficulty of automated on-line painting quality measurement, harsh painting environment and the difficulty of robot navigation. However, the painting automation is necessary, because it can provide consistent performance of painting film thickness. Furthermore, autonomous mobile robots are strongly required for flexible painting work. However, the main problem of autonomous mobile robot's navigation is that there are many obstacles which are not expressed in the CAD data. To overcome this problem, obstacle detection and recognition are necessary to avoid obstacles and painting work effectively. Until now many object recognition algorithms have been studied, especially 2D object recognition methods using intensity image have been widely studied. However, in our case environmental illumination does not exist, so these methods cannot be used. To overcome this, to use 3D range data must be used, but the problem of using 3D range data is high computational cost and long estimation time of recognition due to huge data base. In this paper, we propose a 3D object recognition algorithm based on PCA (Principle Component Analysis) and NN (Neural Network). In the algorithm, the novelty is that the measured 3D range data is transformed into intensity information, and then adopts the PCA and NN algorithm for transformed intensity information to reduce the processing time and make the data easy to handle which are disadvantages of previous researches of 3D object recognition. A set of experimental results are shown to verify the effectiveness of the proposed algorithm.

Intelligent autonomous mobile robots must be able to sense and recognize 3D environment for navigation and task execution where they live or work. To achieve this, a variety of techniques have been developed for the determination of the 3D geometric information such as stereo vision, laser structured light, laser range finder and so on. However, these methods have several limitations such as susceptibility to noise, low resolution, measurement error in 3D scene. In this paper to overcome these problems we introduce a new sensing method based upon stereo vision methodology which utilizes not only intensity information but also phase information. The novelty of this suggested method comes from the fact that it uses the stereo phase information from the PMP technique together with intensity information captured by stereo image. To verify the efficiency and accuracy of the proposed method, a series of experimental tests is performed. The results show that it greatly enhances correspondence matching performance and measurement resolution.

Automatic PCB (printed circuit board) assembly technique plays a very important roll in modern industry where electronics is becoming an indispensable part. Each process of assembly requires well-developed inspection technique to ensure quality of the products and prevent waste of resources. In the case of chip mounting inspection, a number of 2D inspection techniques have been developed but there are challenging problems still remaining. In this paper, a 3D measurement method based on phase shifting and stereo vision for inspection of chip mounting is proposed. The hardware system and algorithm that employ the proposed methods are developed and are described in detail. The results show that height measurement of the chip-mounted board can be achieved with satisfactory accuracy.

Nowadays, many kinds of fringe projection method for 3D depth measurements have been researched such as moire method, optical triangulation method and so on. Generally speaking these methods uses the regular vertical fringe pattern. However, by using regular vertical fringe pattern, the shape measurement results especially for symmetric objects are not accurate, because the vertical fringe pattern cannot present the shape of various objects well. In this paper to solve this problem, we introduce a new sensing methodology based on object-adapted fringe projection method. To make a flexible object-adapted fringe pattern we used the projector with spatial light modulator (SLM). Our algorithm mainly consists of three parts. The first part is to generate an object-adapted fringe pattern by applying the moire technique. The second part is image projection of moire image to the projector plane. The final part is to find an absolute depth value of the object by using the optical triangulation method. To verify the performance of our proposed sensing system we conducted a series of experiments for various simple objects. The result shows the feasibility of successful perception for some objects treated herein.

Intelligent autonomous mobile robots must be able to sense and recognize 3D environment for navigation and task execution, where they live or work. To achieve this, a variety of techniques have been developed for the determination of the 3D scene geometric information such as stereo vision, laser structured light, laser range finder and so on. But these methods have many limitations such as susceptibility to noise, low speed in 3D scene etc. To overcome these limitations we introduce a new sensing algorithm, which is based on the moire technique and stereo vision. To verify the efficiency and accuracy of our algorithm, a series of experimental tests is performed.

An important and challenging research issue associated with mobile robot navigation is simultaneous localization and map building (SLAM), which means that the mobile robot could estimate its poses in the environment without external information, and simultaneously align the local maps. Various kinds of methods such as odometry measurement, landmark matching, laser range image matching and scale invariant feature (SIFT) based algorithm have already been proposed to solve this kind of research problems, but they suffer from inevitable drawbacks. For example, range image matching may suffer from local minimum problem and thus can not get a good location estimation sometimes, and SIFT algorithm can not work if few SIFT intensity features exist in the environment. Furthermore, the map built by laser range image matching is not good for localization when the robot returns to the prior map built by itself, and the map built by SIFT could not be used for obstacle avoidance and next view generation. To solve these problems, in this paper, we propose a sensor fusion method, which makes use of Dempster Shafer algorithm to fuse both of the laser range information and SIFT features information for SLAM.. Through a series of experiments, the proposed method is tested and evaluated.

Nowadays a major research issue of mobile robots is to develop a robust 3D environment sensing for navigation and task execution. To achieve this, a variety of techniques have been developed for the determination of the 3D scene geometric information such as stereo vision, laser structured light, laser range finder and so on. But these methods have many limitations. To overcome these limitations we introduce a new sensing algorithm, which is based on the moire technique and stereo vision. To verify the performance of this sensor system we conducted a series of simulation for various simple environments. The result shows the feasibility of successful perception with several environments.

Intelligent autonomous mobile robots must be able to sense and recognize 3D indoor space where they live or work. In this paper, we propose a new 3D sensing system using the laser structured lighting method, because of its robustness on the nature of the navigation environment and the easy extraction of feature information of interest The proposed active trinocular vision system is composed of a flexible multi-stripe laser projector and two cameras arranged with a triangular shape. Based on the virtual camera concept and the trinocular epipolar constraints, the matching pairs of line features in two real camera images are established, and 3D information from one-shot image can be extracted on the patterned scene. Especially, for robust line feature matching, we propose a new correspondence matching algorithms based on line grouping and probabilistic voting. Finally, a series of experimental tests is performed to show its efficiency and accuracy.