Search

Camera calibration is an essential step for the optical measurement method used in the experimental mechanics. Most plumb line methods focus on solving lens distortions without considering camera intrinsic and extrinsic parameters. In this paper, we propose a full camera calibration method to estimate the camera parameters, including camera intrinsic parameters, extrinsic parameters and lens distortion parameters, from a single image with six or more non-coplanar lines. We parameterize the 3D lines with the intersection of two planes that allow the direct linear transformation of the lines(DLT-Lines). Based on the DLT-Lines, the projection matrix is estimated linearly, and then the camera intrinsic and extrinsic parameters are extracted from the matrix. The relationship between the distorted 2D lines and the distortion coefficients is derived, based on which the distortion coefficients can be solved linearly. In the last step, a non-linear optimization algorithm is used to jointly refine all the camera parameters, including the distortion coefficients. Both synthetic and real data are used to evaluate the performance of our method, which demonstrates that the proposed method can calibrate the cameras with radial and tangential distortions accurately. We propose a DLT-lines based camera calibration method for experimental mechanics. The proposed method can calibrate all the camera parameters from a single image.

Space manipulator has been playing an increasingly important role in space exploration due to its flexibility and versatility. This paper is to design a vision-based pose measurement system for a four-degree-of-freedom (4-DOF) lunar surface sampling manipulator relying on a monitoring camera and several fiducial markers. The system first employs double plateaus histogram equalization for the markers to improve the robustness to varying noise and illumination. The markers are then accurately extracted in sub-pixel based on template matching and curved surface fitting. Finally, given the camera parameters and 3D reference points, the pose of the manipulator end-effector is solved from the 3D-to-2D point correspondences by combining a plane-based pose estimation method with rigid-body transformation. Experiment results show that the system achieves high-precision positioning and orientation performance. The measurement error is within 3 mm in position, and 0.2° in orientation, meeting the requirements for space manipulator operations.

This paper presents a new pipeline for capturing the performance of the human body with two depth cameras in real time. Our online approach reconstructs not only the movement of the skeleton, but also the dense nonrigid 3D surface. The key to our algorithm is a coarse-to-fine strategy: large-scale deformations or rapid movements are captured by the skeleton pose tracker, with which the dense volumetric 6D motion field can be estimated robustly for dense geometry reconstruction. Shown in various sequences, our approach was able to accurately capture the 3D motion of the human body and produce increasingly denoised, detailed, and complete geometric reconstructions.

In the case of complex illumination background, the image has fuzziness and chromatic aberration, which can improve the imaging effect of the image by white balance chromatic aberration compensation. The traditional method uses the contour feature fusion adaptive matching method to compensate the white balance under the complex illumination background, which leads to the good color difference compensation effect when the image pixel is low. A white balance color compensation algorithm for fuzzy chromatic aberration based on wavelet packet decomposition is proposed. The original image was denoised and filtered. The white balance characteristics of the image were analyzed and extracted based on the wavelet packet decomposition method, and the adaptive balance design was carried out to realize the white balance and color difference compensation of the art design chromatic aberration image. By using the known pixel information of the image block to be repaired, the statistical properties of the image block to be repaired are predicted, and the matching cost of the image block to be matched that meets the restriction conditions is calculated. By introducing the objective factor, the matching cost function is improved to balance the restoration process, and the unnatural problem caused by the repeated appearance of some image details in the restoration results is solved. The simulation results show that the method can effectively balance the color difference of the image, improve the aesthetic feeling of the image, and improve the performance of the detail features of the image. The peak signal-to-noise ratio is high.

In the star images taken by the ground observatory, due to atmospheric distortion, long exposure time, and long imaging distance, some geostationary (GEO) satellite targets in the image are small in size and weak in energy. Existing methods all have troubles in dealing such cases. The paper proposes a novel GEO satellite target detection algorithm based on image transformation and energy accumulation. Firstly, the LSD line detection algorithm is used to detect the star stripe areas and eliminate them. Secondly, the star stripe is used to match the adjacent images in original images and the movement of the telescope in the vertical direction of the star stripe is eliminated via image transformation. Lastly, the energy accumulation algorithm is used to detect the location of the targets on the transformed image sequences. Experimental results on image sequences show that the proposed algorithm can effectively detect weak energy GEO satellite targets and its performance is better than the existing algorithms.

With the omnipresence of unmanned aerial vehicle (UAV), sequential aerial imagery is becoming more widely available in both civilian and military programs. Considering that the monocular imagery is sequentially ordered according to its acquisition time and a dominant ground plane is often visible in the scene, we present a novel and complete automatic Structure from Motion system based on homography constraints. The homography induced by the ground plane is used to recover the camera pose. The windowed bundle adjustment and automatic loop closure are performed in sequential reconstruction. The global bundle adjustment is performed over the complete reconstruction. The proposed SfM system has successfully been used for sequential aerial imagery taken from a UAV.

In this paper, we propose a self-calibration approach to stereo cameras with radial distortion from stereo image pairs of a common 3D scene. Based on the epipolar constraint in the stereo image pair, the intrinsic and extrinsic parameters of stereo cameras are estimated synchronously with a minimum number of nine image point correspondences. It is significant within a random sample consensus (RANSAC) scheme to cope with the outliers of feature matches efficiently and robustly. Then the inliers of the stereo image pair that have been determined after RANSAC are used to optimize the calibration parameters of stereo cameras. Furthermore, more accurate calibration results can be achieved with the joint optimization of multiple stereo image pairs. Both synthetic and real data are used to evaluate the performance of the proposed method, demonstrating that our method can calibrate stereo cameras with radial distortions efficiently and accurately.

Infrared object tracking is a key technology in many surveillance applications. General visual tracking algorithms designed for color images can not handle infrared targets very well due to their relatively low resolutions and blurred edges. This paper presents a new tracking by detection method based on online structural learning. We show how to train the classifier efficiently with dense samples through Fourier techniques and careful implementation. Furthermore, we introduce an effective feature representation for infrared objects. Finally, we demonstrate the performance of the proposed tracker on public infrared sequences with top accuracy and robustness. Meanwhile, our single thread C++ implementation of the algorithm achieves an average tracking speed of 215 FPS on a modern cpu.

In this paper, we propose two perspective-n-line (PnL)-like methods with the presence of line detection process. Compared with the traditional methods, the proposed methods use the new error models derived from the edge points and their corresponding noisy observations rather than relying on the assumption that the noises for the two endpoints are statistically independent. Meanwhile, we improve the performance of the RAPiD-like method—another type of visual tracking approach without extracting image lines by fitting the interpolated location of the corresponding edge pixel in the local region. In addition, we compare the proposed PnL-like methods with the RAPiD-like methods and find that both the types of visual tracking methods for rigid objects are fundamentally equivalent and all of them are maximum-likelihood approaches to estimate the pose parameters, given the error model for the noisy edge points. Special consideration is put into deriving a unifying probabilistic framework to express these two types of methods. Moreover, comparisons under different performance criteria, including computational efficiency, accuracy, and robustness, are also conducted.

The calibration of a camera in wide scene generally requires a large number of non-coplanar control points. This process often needs a large amount of extra auxiliary work to match the object points and the image points, and in some cases, this matching is difficult for users with insufficient experience. A feature representation and optimal priority matching method based on the angular ordered sequences (AOS) of a control point are proposed in this paper with the goal of solving the object-image auto-matching problem for control points. We also provide a strategy for dealing with the difficult situation for automatic calibration when the camera’s optical center is nearly collinear with two object points. The simulation results demonstrated that the proposed method was robust with errors for the initial value of the optical center, object points, and image points, and it could quickly achieve object-image matching for automatic calibration.

The pose of space target is a key technical parameter in navigation guidance. In launching silo, the missile pose estimation is difficult because of specific mounting location and limited field of view of cameras. To solve these problems, we design a pose measurement system which has three measurement methods based on binocular intersection, central axis and monocular camera respectively. Synthetic experiments and real image experiments show that the system is feasible and the precision of the system can meet the accuracy demand of measurement in launching silo.

PNP (Perspective-N-Point) and PNL (Perspective-N-Line) are the most commonly used methods for camera pose estimation. However, most methods are based on the hypothesis of the pinhole camera model, which ignore camera distortion’s effects. Considering the influences of camera distortion, we proposed to solve the camera pose directly by using distorted lines. In this paper, we first undistort the lines according to distortion model, then use LSD algorithm extract 2D undistorted lines from the image. Having matched 2D/3D lines, the PNL (Perspective-N-Line) was used finally to solve the camera pose. Through experiments on synthetic images and real images, it turns out that our method is effective and feasible for pose estimation.

To further extend the scope of stereo-digital image correlation (stereo-DIC) to more challenging environments, a novel Scheimpflug camera-based stereo-DIC is developed for full-field 3D deformation measurement, wherein the Scheimpflug condition, consisting of tilting the sensor plane with respect to the lens plane for the sake of larger depth of field (DOF) of the camera, is employed. The geometric imaging model of the Scheimpflug camera is described, on the basis of which a robust and effective stepwise calibration strategy is performed to calculate the intrinsic and extrinsic parameters of the stereo Scheimpflug rig. With the aid of a specially tailored stereo triangulation method and well-developed subset-based DIC algorithms, the three-dimensional shape and displacement of the specimen can be retrieved. Finally, practical experiments, including rigid motion tests and three-point bending tests, demonstrate the effectiveness and accuracy of the proposed approach.

In this paper, we propose a novel multitask learning method based on the deep convolutional network. The proposed deep network has four convolutional layers, three max-pooling layers, and two parallel fully connected layers. To adjust the deep network to multitask learning problem, we propose to learn a low-rank deep network so that the relation among different tasks can be explored. We proposed to minimize the number of independent parameter rows of one fully connected layer to explore the relations among different tasks, which is measured by the nuclear norm of the parameter of one fully connected layer, and seek a low-rank parameter matrix. Meanwhile, we also propose to regularize another fully connected layer by sparsity penalty so that the useful features learned by the lower layers can be selected. The learning problem is solved by an iterative algorithm based on gradient descent and back-propagation algorithms. The proposed algorithm is evaluated over benchmark datasets of multiple face attribute prediction, multitask natural language processing, and joint economics index predictions. The evaluation results show the advantage of the low-rank deep CNN model over multitask problems.

For the problem of 3D line reconstruction in binocular or multiple view stereo vision, when there are no corresponding points on the line, the method called Direction-then-Point (DtP) can be used, and if there are two pairs of corresponding points on the line, the method called Two Points 3D coordinates (TPS) can be used. However, when there is only one pair of corresponding points on the line, can we get the better accuracy than DtP for 3D line reconstruction? In this paper, a linear and more accurate method called Point-then-Direction (PtD) is proposed. First, we used the intersection method to obtain the 3D point&rsquo, s coordinate from its corresponding image points. Then, we used this point as a position on the line to calculate the direction of the line by minimizing the image angle residual. PtD is also suitable for multiple camera systems. The simulation results demonstrate that PtD increases the accuracy of both the direction and the position of the 3D line compared to DtP. At the same time, PtD achieves a better result in direction of the 3D line than TPS, but has a lower accuracy in the position of 3D lines than TPS.

Pose estimation is important for many robotic applications including bin picking and robotic assembly and collaboration. However, robust and accurate estimation of the poses of industrial objects is a challenging task owing to the various object shapes and complex working environments. This paper presents a method of estimating the poses of narrow and elongated industrial objects with a low-cost RGB-D (depth and color) camera to guide the process of robotic assembly. The proposed method comprises three main steps: reconstruction involved in preprocessing, pose initialization with geometric features, and tracking aided by contour cues. Pose tracking is coupled with real-time dense reconstruction, which can synthesize a smooth depth image as a substitute for the raw depth image. Because industrial objects (e.g., fork and adapter) feature mostly planar structures, primitive geometric features, such as three-dimensional planes, are extracted from the point cloud and utilized to induce a promising initial pose. For robust tracking of the adapter consisting of narrow and elongated planes, the dense surface correspondences are combined with sparse contour correspondences in the refinement scheme. This combination allows for a satisfactory tolerance to the initial guess in the pose tracking phase. The experimental results demonstrate the feasibility of the proposed method.

3D metrology of a stereovision system requires epipolar rectification to be performed before dense stereo matching. In this study, we propose an epipolar rectification method for a stereovision system with two telecentric lens-based cameras. Given the orthographic projection matrices of each camera, the new projection matrices are computed by determining the new camera coordinates system in affine space and imposing some constraints on the intrinsic parameters. Then, the transformation that maps the old image planes on to the new image planes is achieved. Experiments are performed to validate the performance of the proposed rectification method. The test results show that the perpendicular distance and 3D reconstructed deviation obtained from the rectified images is not significantly higher than the corresponding values obtained from the original images. Considering the roughness of the extracted corner points and calibrated camera parameters, we can conclude that the proposed method can provide sufficiently accurate rectification results.

This paper proposes a homograhpy estimation algorithm that uses contour model to track and locate the texture-less object in high clutter environments. The proposed method consists of a homography recognition with the correspondences between model and image lines unknown, followed by a nonlinear homography optimization based on the maximum likelihood criterion. In our approach, the initial estimation of the homography is obtained in the framework of RANSAC-like, which contains two stages: hypothesizing and verifying. The first stage generates a number of homograhpy hypotheses from the correspondences of the quadrangle-like structures. Next, the homography hypotheses are quickly ranked according to a redefined distance function. In the refinement procedure, the model sample points are projected into the image plane. After that, 1D search is utilized along the normal direction to obtain the corresponding image point for each model sample point. Finally, the optimized homography is obtained by minimizing the errors between the sample points and their corresponding image points. Experiments show that the proposed method performs robust homography recognition of texture-less planar objects and maintains accurate and stable homography estimation in the cluttered environments as well as the cases of extreme slant angles. We proposed an effective homography recognition based on quadrangle-like structures.We derived a new distance function between lines in the probabilistic criterion.We presented a robust homography optimization without image contour extraction.We solved the nonlinear homography refinement by the maximum likelihood approach.

In this study, the authors have proposed a new solution for the problem of pose estimation from a set of matched 3D model and 2D image lines. Traditional line-based pose estimation methods utilising the finite information of the observations are based on the assumption that the noises for the two endpoints of the image line segment are statistically independent. However, in this study, the authors prove that these two noises are negatively correlative when the image line segment is fitted by the least-squares technique from the noisy edge points. Moreover, the authors derive the noise model describing the probabilistic relationship between the 3D model line and their finite image observations. Based on the proposed noise model, the maximum-likelihood approach is exploited to estimate the pose parameters. The authors have carried out synthetic experiments to compare the proposed method to other pose optimisation methods in the literature. The experimental results show that the proposed methods yield a clear higher precision than the traditional methods. The authors also use real image sequences to demonstrate the performance of the proposed method.

A ship horizontal attitude estimation approach with horizon tracking has been proposed, which consists of four steps. First, the approximate parameters of the horizon in successive frames are predicted. Then the candidate edges are achieved by defining a small region of interest (ROI) around the predicted horizon location, and the final horizon is acquired according to the statistic appearance of the local region. Third, the optimal camera attitude is updated based on the horizon with the designed Kalman filter. Finally, the ship horizontal attitude is obtained according to the installation azimuth of the camera. As processing in a local region, the horizon detection process takes a short time and performs robust to the false alarms. Compared with the traditional attitude determining method based on horizon for unmanned aerial vehicle (UAV), the ship horizontal attitude is achieved with the camera's installation azimuth. In order to test the performance of the proposed approach, the experiments are performed based on the videos captured by the camera which is fixed on the ship board. The experimental results indicate that the estimation accuracy of the horizontal attitude can be achieved at 10″ (1 σ ) in pitch angle at a speed of 20 fps.

Estimating and tracking the pose of objects is a key task in the field of photogrammetry and computer vision, playing an important role in a large number of applications and is receiving significant attentions. Combining CAD model and images from monocular RGB camera to estimate and track the pose of 3D objects has lots of applications. Despite enormous approaches exist, a deep review of the literatures on 3D pose estimation and tracking is still lacking. The paper reviews the works on model-based 3D objects pose estimation and tracking during the past two decades and classifies the approaches into different categories.

Structural SVM trackers and correlation filter trackers have demonstrated dominant performance in recent object tracking benchmarks. However, structural SVM trackers naturally suffer from shortage of samples and low speed, and time-consuming adaption is need to relieve the correlation filter trackers from boundary effects. Thus, we design a jointed tracker by concatenating a high-speed SSVM method-DSLT and a multi feature CF method-STAPLE to realize advantage complementation. We show that the tracking precision and robustness can be improve by a large margin comparing to either single tracker with little sacrifice of speed.

In this paper, we exploit prior information from global positioning systems and inertial measurement units to speed up the process of large scene reconstruction from images acquired by Unmanned Aerial Vehicles. We utilize weak pose information and intrinsic parameter to obtain the projection matrix for each view. As compared to unmanned aerial vehicles' flight altitude, topographic relief can usually be ignored, we assume that the scene is flat and use weak perspective camera to get projective transformations between two views. Furthermore, we propose an overlap criterion and select potentially matching view pairs between projective transformed views. A robust global structure from motion method is used for image based reconstruction. Our real world experiments show that the approach is accurate, scalable and computationally efficient. Moreover, projective transformations between views can also be used to eliminate false matching.

As a category of special control points which can be automatically identified, artificial coded targets have been widely developed in the field of computer vision, photogrammetry, augmented reality, etc. In this paper, a new circular coded target designed by RockeTech technology Corp. Ltd is analyzed and studied, which is called circular coded diagonal target (CCDT). A novel detection and recognition method with good robustness is proposed in the paper, and implemented on Visual Studio. In this algorithm, firstly, the ellipse features of the center circle are used for rough positioning. Then, according to the characteristics of the center diagonal target, a circular frequency filter is designed to choose the correct center circle and eliminates non-target noise. The precise positioning of the coded target is done by the correlation coefficient fitting extreme value method. Finally, the coded target recognition is achieved by decoding the binary sequence in the outer ring of the extracted target. To test the proposed algorithm, this paper has carried out simulation experiments and real experiments. The results show that the CCDT recognition and accurate locating method proposed in this paper can robustly recognize and accurately locate the targets in complex and noisy background.

The bearing only tracking problem is to measure the position and velocity of moving points from a moving camera. The method in this paper mainly solves the measurement problem of a single camera or some non-synchronized cameras. This paper introduces the process model and basic method of bearing only tracking problem, and the different optimization methods based on it, then compares the advantages and disadvantages of those optimization methods. The research method of this kind of problem is discussed in the last of this paper.

The paper aims to challenge non-GPS navigation problems by using visual sensors and geo-referenced images. An area-based method is proposed to estimate full navigation parameters (FNPs), including attitude, altitude and horizontal position, for unmanned aerial vehicle (UAV) navigation. Our method is composed of three main modules: geometric transfer function, local normalized sobel energy image (LNSEI) based objective function and simplex-simulated annealing (SSA) based optimization algorithm. The adoption of relatively rich scene information and LNSEI, makes it possible to yield a solution robustly even in the presence of very noisy cases, such as multi-modal and/or multi-temporal images that differ in the type of visual sensor, season, illumination, weather, and so on, and also to handle the sparsely textured regions where features are barely detected or matched. Simulation experiments using many synthetic images clearly support noise resistance and estimation accuracy, and experimental results using 2367 real images show the maximum estimation error of 5.16 (meter) for horizontal position, 9.72 (meter) for altitude and 0.82 (degree) for attitude.

A frequency-spatial domain based model for salient region detection was proposed. The paper gives a thorough theoretical analysis of detecting salient gradients belong to salient region in the frequency domain. The model detects and highlights salient gradient changes in the frequency domain firstly; then the image is segmented into regions in the spatial domain, and the salient gradients detection and segment results are fused using a probabilistic framework to produce the full resolution, region-wise saliency map. It was demonstrated that this model has the ability to detect salient regions accurately and highlight them uniformly. A publically available benchmark was used to demonstrate that the model outperformed state-of-the-art algorithms when identifying the salient regions where greater attention should be paid by human observers.

This article elaborates on the theory of monocular measurement for a walker and uses experiments to validate this approach. This new monocular approach for measuring the movement parameters of a walker allows for the 3-D pose and position of the walker to be obtained using at least two endpoints on the target, the walker’s vertex and heel, and then other point’s position on the walker can be calculated. If the walker’s stature is not known, there is a coefficient of proportionality between the measured result and the true result. When the walker’s stature is known, the movement parameters of the walker can be measured in this way by a monocular camera, which can be used to study a person’s movement or to monitor person’s action in public places by monitor video.

A monocular method to measure the pose of a translation-only one-dimensional (1D) object containing at least two known characteristic points is presented. The position of any other point on the 1D object can be located after the pose measurement. Scene information is not needed to apply the method. If the distance between the characteristic points is unknown, there is a scale factor between the object's real positions and the measurement results. The method can also be used to measure the pose and shape parameters of a parallelogram, prism or cylinder object.

Vision navigation based on scene matching between real-time images and a reference image has many advantages over the commonly used inertial navigation system (INS), such as no cumulative measurement errors for long-endurance flight. Recent developments in vision navigation are mainly used for partial navigation parameters measurements, such as the position and the relative velocity, which cannot meet the requirements of completely autonomous navigation. We present the concept, principle and method of full-parameter vision navigation (FPVN) based on scene matching. The proposed method can obtain the three-dimensional (3D) position and attitude angles of an aircraft by the scene matching for multiple feature points instead of a single point in existing vision navigations. Thus, FPVN can achieve the geodetic position coordinates and attitude angles of the aircraft and then the velocity vector, attitude angular velocity and acceleration can be derived by the time differentials, which provide a full set of navigation parameters for aircrafts and unmanned aerial vehicles (UAVs). The method can be combined with the INS and the cumulative errors of the INS can be corrected using the measurements of FPVN rather than satellite navigation systems. The approach provides a completely autonomous and accurate navigation method for long-endurance flight without the help of satellites.

A displacement and deformation measurement method for large structures by a series-parallel connection camera network is presented. By taking the dynamic monitoring of a large-scale crane in lifting operation as an example, a series-parallel connection camera network is designed, and the displacement and deformation measurement method by using this series-parallel connection camera network is studied. The movement range of the crane body is small, and that of the crane arm is large. The displacement of the crane body, the displacement of the crane arm relative to the body and the deformation of the arm are measured. Compared with a pure series or parallel connection camera network, the designed series-parallel connection camera network can be used to measure not only the movement and displacement of a large structure but also the relative movement and deformation of some interesting parts of the large structure by a relatively simple optical measurement system.

The image Jacobian Matrix must obtain during the course of uncalibrated visual servo for classic algorithms firstly. Then the inverse of image Jacobian Matrix or pseudo-inverse of image Jacobian Matrix can be taken. But when the inverse of image Jacobian Matrix is not exist or pseudo-inverse of image Jacobian Matrix is not easy to get, the uncalibrated visual servo for robot can not realize. In this paper, a research is carried on by simulation between the classic method for uncalibrared visual servo and the strategy by computing pseudo-inverse of image Jacobian Matrix. It is conclusion that the latter not only has advantage of the performance for tracking, but also reduces computational complexity for control.

In this paper we present an affine SIFT matching method to achieve reliable correspondence points in stereo matching with large viewpoint changes. We extended the affine invariant of the conventional SIFT approach by estimating the shape of the local patch around the interest point. Since we can obtain the scale information by SIFT detector, a second moment matrix (SMM) descriptor was employed to describe the shape. Furthermore, by comparing the shapes of the potential matches, we can normalize the template of SIFT descriptor and obtain the initial affine transformation. At last, we applied the iterative based method to achieve a fine registration with the estimated initial transformation parameters. The experiment results show that the proposed method is more robust to viewpoint changes and the accuracy of registration is better than feature based methods.

The distorted (scale and rotation etc.) target images and the training images are edge extracted by Prewitt operator. The Maximum Average Correlation Height filter is synthesized with the training images. The low frequency information of the distorted target images and the filter is enhanced. Then the edge extracted distorted target image is filtered by the Maximum Average correlation Height filter. Thereby, the distortion tolerance of the Maximum Average Correlation Height filter is expanded. By this method, when the space edge extraction combines with frequency domain filtering, the scale distortion tolerance is 0.76~1.56 times; the rotation distortion tolerance can reach up to 80 degrees. For the targets exceeding the distortion tolerance range, the intensity of correlation peak becomes weaker and weaker, but some of the targets can still be identified.

In order to make the general user take vision tasks more flexibly and easily, this paper proposes a new solution for the problem of camera calibration from correspondences between model lines and their noisy image lines in multiple images. In the proposed method the common planar items in hand with the standard size and structure are utilized as the calibration objects. The proposed method consists of a closed-form solution based on homography optimization, followed by a nonlinear refinement based on the maximum likelihood approach. To automatically recover the camera parameters linearly, we present a robust homography optimization method based on the edge model by redesigning the classic 3D tracking approach. In the nonlinear refinement procedure, the uncertainty of the image line segment is encoded in the error model, taking the finite nature of the observations into account. By developing the new error model between the model line and image line segment, the problem of the camera calibration is expressed in the probabilistic formulation. Simulation data is used to compare this method with the widely used planar pattern based method. Actual image sequences are also utilized to demonstrate the effectiveness and flexibility of the proposed method.

Image Matching is often one of the first tasks in many Photogrammetry and Remote Sensing applications. This paper presents an efficient approach to automated multi-temporal and multi-sensor image matching based on local frequency information. Two new independent image representations, Local Average Phase (LAP) and Local Weighted Amplitude (LWA), are presented to emphasize the common scene information, while suppressing the non-common illumination and sensor-dependent information. In order to get the two representations, local frequency information is firstly obtained from Log-Gabor wavelet transformation, which is similar to that of the human visual system; then the outputs of odd and even symmetric filters are used to construct the LAP and LWA. The LAP and LWA emphasize on the phase and amplitude information respectively. As these two representations are both derivative-free and threshold-free, they are robust to noise and can keep as much of the image details as possible. A new Compositional Similarity Measure (CSM) is also presented to combine the LAP and LWA with the same weight for measuring the similarity of multi-temporal and multi-sensor images. The CSM can make the LAP and LWA compensate for each other and can make full use of the amplitude and phase of local frequency information. In many image matching applications, the template is usually selected without consideration of its matching robustness and accuracy. In order to overcome this problem, a local best matching point detection is presented to detect the best matching template. In the detection method, we employ self-similarity analysis to identify the template with the highest matching robustness and accuracy. Experimental results using some real images and simulation images demonstrate that the presented approach is effective for matching image pairs with significant scene and illumination changes and that it has advantages over other state-of-the-art approaches, which include: the Local Frequency Response Vectors (LFRV), Phase Congruence (PC), and Four Directional-Derivative-Energy Image (FDDEI), especially when there is a low signal-to-noise ratio (SNR). As few assumptions are made, our proposed method can foreseeably be used in a wide variety of image-matching applications.

A real-time sea-sky-line detection method under complicated sea-sky background is presented. Firstly, a black-white template is constructed and used for fast correlation matching in several searching regions which are predefined in input image, position of maximal correlation coefficient in each predefined region is hunt out, and coordinates of several candidate sea-sky-line points are made certain according to the position. Then, RANSAC algorithm is applied to preserve interior points which are really on the sea-sky-line and eliminate exterior points which are not. Finally, line parameters of the sea-sky-line can be gained by applying least squares line fitting for all interior points. The pixels of several regions in the image instead of the whole image need to be considered, so computational cost can be reduced dramatically. The experimental results show that the proposed method can detect out sea-sky-line under complicated sea-sky background effectively and has many advantages such as strong robustness and speedy calculation.

In order to fully navigate using a vision sensor, a 3D edge model based detection and tracking technique was developed. Firstly, we proposed a target detection strategy over a sequence of several images from the 3D model to initialize the tracking. The overall purpose of such approach is to robustly match each image with the model views of the target. Thus we designed a line segment detection and matching method based on the multi-scale space technology. Experiments on real images showed that our method is highly robust under various image changes. Secondly, we proposed a method based on 3D particle filter (PF) coupled with M-estimation to track and estimate the pose of the target efficiently. In the proposed approach, a similarity observation model was designed according to a new distance function of line segments. Then, based on the tracking results of PF, the pose was optimized using M-estimation. Experiments indicated that the proposed method can effectively track and accurately estimate the pose of freely moving target in unconstrained environment.

A systematic videometrics method of cooperative object pose-measurement for RVD (rendezvous and docking) is proposed in the paper. According to the method, initial values of pose parameters are calculated from binocular images respectively, and then optimized with bundle adjustment. While a certain variation of some exterior parameters of one camera are added as systematic disturbance purposely, the correct result could be calculated theoretically as the method analysis, and then also is verified in our experiment. The correct result could be converged quickly and stably in the experiment, and accurate pose-measurement results also could be obtained while initial values are provided with some certain errors. Even some amount of disturbance has been added purposely in experiment, high-precision pose results are also obtained by the binocular and bundle adjustment way.

This article presents a passive navigation method of terrain contour matching by reconstructing the 3-D terrain from the image sequence (acquired by the onboard camera). To achieve automation and simultaneity of the image sequence processing for navigation, a correspondence registration method based on control points tracking is proposed which tracks the sparse control points through the whole image sequence and uses them as correspondence in the relation geometry solution. Besides, a key frame selection method based on the images overlapping ratio and intersecting angles is explored, thereafter the requirement for the camera system configuration is provided. The proposed method also includes an optimal local homography estimating algorithm according to the control points, which helps correctly predict points to be matched and their speed corresponding. Consequently, the real-time 3-D terrain of the trajectory thus reconstructed is matched with the referenced terrain map, and the result of which provides navigating information. The digital simulation experiment and the real image based experiment have verified the proposed method.

This article intends to solve the matching problem of 2C level lunar images by Chang’E-1 (CE-1) lunar probe satellite. A line-scanner image matching method is proposed which represents deformation by the quadric function along the camera motion direction and bases on the deformation model for a relief terrain’s imaging on sensors of the satellite borne three-line scanner camera. A precise matching is carried out for the normal view, the frontward view, and the backward view images of the CE-1 by combining the proposed method with the standard correlation method. A super-resolution (SR) reconstruction algorithm based on the wavelet interpolation of non-uniformly sampled data is also adopted to realize SR reconstruction of CE-1 lunar images, which adds the recognizable targets and explores CE-1 lunar images to the full.

Videometrics is a technique for measuring displacement, deformation and motion with features of precision, multifunctional, automation and real time measurement etc. Videometrics with camera networks is a fast developed area for deformation measurements of large scale structures. Conventional camera network is parallel network where cameras are independent each other and the relations among the cameras are calibrated from their target images. In recent years, we proposed and developed two kinds of videometrics with camera series networks where cameras are connected each other in series and relations among the cameras can be relayed one by one for the deformation measurements of large and super large scale structures. In this paper, our research work in both the camera series and parallel networks for the deformation measurements of large scale structures are overviewed and some new development are introduced. First, our proposed methods of camera series networks are introduced, including the pose-relay videometrics with camera series and the displacement-relay videometrics with camera series. Then our work of large scale structure deformation measurement by camera parallel networks is overviewed. Videometrics with various types of camera networks has the broad prospect of undertaking automatic, long-term and continuous measurement for deformation in engineering projects such as wind turbine blades, ship, railroad beds, and bridges.

A simple and flexible method for non-overlapping camera rig calibration that includes camera calibration and relative poses calibration is presented. The proposed algorithm gives the solutions of the cameras parameters and the relative poses simultaneously by using nonlinear optimization. Firstly, the intrinsic and extrinsic parameters of each camera in the rig are estimated individually. Then, a linear solution derived from hand-eye calibration scheme is proposed to compute an initial estimate of the relative poses inside the camera rig. Finally, combined non-linear refinement of all parameters is performed, which optimizes the intrinsic parameters, the extrinsic parameters and relative poses of the coupled camera at the same time. We develop and test a novel approach for calibrating the parameters of non-overlapping camera rig using camera calibration and hand-eye calibration method. The method is designed inter alia for the purpose of deformation measurement using the calibrated rig. Compared the camera calibration with hand-eye calibration separately, our joint calibration is more convenient in practice application. Experimental data shows our algorithm is feasible and effective.

This paper proposes a framework for small infrared target real-time visual enhancement. The framework is consisted of three parts: energy accumulation for small infrared target enhancement, noise suppression and weighted fusion. Dynamic programming based track-before-detection algorithm is adopted in the energy accumulation to detect the target accurately and enhance the target’s intensity notably. In the noise suppression, the target region is weighted by a Gaussian mask according to the target’s Gaussian shape. In order to fuse the processed target region and unprocessed background smoothly, the intensity in the target region is treated as weight in the fusion. Experiments on real small infrared target images indicate that the framework proposed in this paper can enhances the small infrared target markedly and improves the image’s visual quality notably. The proposed framework outperforms tradition algorithms in enhancing the small infrared target, especially for image in which the target is hardly visible.

We propose a monocular trajectory intersection method to solve the problem that a monocular moving camera cannot be used for three-dimensional reconstruction of a moving object point. The necessary and sufficient condition of when this method has the unique solution is provided. An extended application of the method is to not only achieve the reconstruction of the 3D trajectory, but also to capture the orientation of the moving object, which would not be obtained by PnP problem methods due to lack of features. It is a breakthrough improvement that develops the intersection measurement from the traditional “point intersection” to “trajectory intersection” in videometrics. The trajectory of the object point can be obtained by using only linear equations without any initial value or iteration, the orientation of the object with poor conditions can also be calculated. The required condition for the existence of definite solution of this method is derived from equivalence relations of the orders of the moving trajectory equations of the object, which specifies the applicable conditions of the method. Simulation and experimental results show that it not only applies to objects moving along a straight line, or a conic and another simple trajectory, but also provides good result for more complicated trajectories, making it widely applicable.

The automatic detection of visually salient information from abundant video imagery is crucial, as it plays an important role in surveillance and reconnaissance tasks for Unmanned Aerial Vehicle (UAV). A real-time approach for the detection of salient objects on road, e.g. stationary and moving vehicle or people, is proposed, which is based on region segmentation and saliency detection within related domains. Generally, the traditional method specifically depends upon additional scene information and auxiliary thermal or IR sensing for secondary confirmation. However, this proposed approach can detect the interesting objects directly from video imagery captured by optical camera fixed on the small level UAV platform. To validate this proposed salient object detection approach, the 25 Hz video data from our low speed small UAV are tested. The results have demonstrated the proposed approach performs excellently in isolated rural environments.