Search

Video surveillance camera (VSC) is an important source of information during investigations especially if used as a tool for the extraction of verified and reliable forensic measurements. In this study, some aspects of human height extraction from VSC video frames are analyzed with the aim of identifying and mitigating error sources that can strongly affect the measurement. More specifically, those introduced by lens distortion are present in wide-field-of-view lens such as VSCs. A weak model, which is not able to properly describe and correct the lens distortion, could introduce systematic errors. This study focuses on the aspect of camera calibration to verify human height extraction by Amped FIVE software, which is adopted by the Forensic science laboratories of Carabinieri Force (RaCIS), Italy. A stable and reliable approach of camera calibration is needed since investigators have to deal with different cameras while inspecting the crime scene. The performance of the software in correcting distorted images is compared with a technique of single view self-calibration. Both approaches were applied to several frames acquired by a fish-eye camera and then measuring the height of five different people. Moreover, two actual cases, both characterized by common low-resolution and distorted images, were also analyzed. The height of four known persons was measured and used as reference value for validation. Results show no significant difference between the two calibration approaches working with fish-eye camera in test field, while evidence of differences was found in the measurement on the actual cases.

Fisheye cameras prove a convenient means in surveillance and automotive applications as they provide a very wide field of view for capturing their surroundings. Contrary to typical rectilinear imagery, however, fisheye video sequences follow a different mapping from the world coordinates to the image plane which is not considered in standard video processing techniques. In this paper, we present a motion estimation method for real-world fisheye videos by combining perspective projection with knowledge about the underlying fisheye projection. The latter is obtained by camera calibration since actual lenses rarely follow exact models. Furthermore, we introduce a re-mapping for ultra-wide angles which would otherwise lead to wrong motion compensation results for the fisheye boundary. Both concepts extend an existing hybrid motion estimation method for equisolid fisheye video sequences that decides between traditional and fisheye block matching in a block-based manner. Compared to that method, the proposed calibration and re-mapping extensions yield gains of up to 0.58 dB in luminance PSNR for real-world fisheye video sequences. Overall gains amount to up to 3.32 dB compared to traditional block matching.

Existing projector-camera calibration methods typically warp keypoints from a camera image to a projector image using estimated homographies and often suffer from errors in camera parameters and noises due to imperfect planarity of the calibration target. This article proposes a practical and robust projector-camera calibration system that explicitly deals with these challenges. First, a graph-theory-based correspondence algorithm is built on top of a color-coded spatial structured light (SL) pattern. Such SL correspondences are then used for a coarse projector-camera calibration. To gain more robustness against noises from an imperfect planar calibration board, we develop a bundle adjustment algorithm to jointly optimize the estimated projector-camera parameters and the correspondences’ coordinates. Moreover, our system requires only one shot of an SL pattern for each calibration board pose, which is much more practical than multishot solutions. Comprehensive experimental validation is conducted on both synthetic and real data sets, and our method clearly outperforms the existing methods in all experiments. For the benefit of the society, a practical open-source software with graphical user interface (GUI) of the developed system is publicly available at https://github.com/bingyaohuang/single-shot-pro-cam-calib . Note to Practitioners —The proposed method is motivated by two challenges in industrial structured light (SL) system calibration: 1) robustness against imperfect planarity of the calibration target and 2) the number of SL projections per pose. In many industrial SL-based 3-D reconstruction systems, the calibration accuracy greatly affects the reconstruction reliability. Our SL calibration system explicitly deals with calibration target’s imperfect planarity and thus outperforms the existing methods in terms of system accuracy. Another advantage of our SL calibration system is single-shot-per-pose, allowing fast recalibration and reducing the decoding error due to slight pattern misalignment in multishot methods [37] . In addition, we release the open-source calibration software with a graphical user interface (GUI), with which calibration and sparse 3-D reconstruction can be easily performed without any further instructions. Moreover, considering the complex calibration environment and setup, we make the camera and projector imaging parameters, such as exposure, brightness, and contrast, adjustable through widgets and preview. Finally, a limitation of our color-coded SL system is its sensitivity to environment lighting and target texture. This problem may be solved by projector photometric compensation [16] , [18] , [19] , [39] .

Camera calibration is an essential step for the optical measurement method used in the experimental mechanics. Most plumb line methods focus on solving lens distortions without considering camera intrinsic and extrinsic parameters. In this paper, we propose a full camera calibration method to estimate the camera parameters, including camera intrinsic parameters, extrinsic parameters and lens distortion parameters, from a single image with six or more non-coplanar lines. We parameterize the 3D lines with the intersection of two planes that allow the direct linear transformation of the lines(DLT-Lines). Based on the DLT-Lines, the projection matrix is estimated linearly, and then the camera intrinsic and extrinsic parameters are extracted from the matrix. The relationship between the distorted 2D lines and the distortion coefficients is derived, based on which the distortion coefficients can be solved linearly. In the last step, a non-linear optimization algorithm is used to jointly refine all the camera parameters, including the distortion coefficients. Both synthetic and real data are used to evaluate the performance of our method, which demonstrates that the proposed method can calibrate the cameras with radial and tangential distortions accurately. We propose a DLT-lines based camera calibration method for experimental mechanics. The proposed method can calibrate all the camera parameters from a single image.

For optical 3D measurements in close-range and UAV applications, the modelling of interior orientation is of superior importance in order to subsequently allow for high precision and accuracy in geometric 3D reconstruction. Nowadays, modern camera systems are often used for optical 3D measurements due to UAV payloads and economic purposes. They are constructed of aspheric and spherical lens combinations and include image pre-processing like low-pass filtering or internal distortion corrections that may lead to effects in image space not being considered with the standard interior orientation models. With a variety of structure-from-motion (SfM) data sets, four typical systematic patterns of residuals could be observed. These investigations focus on the evaluation of interior orientation modelling with respect to minimising systematics given in image space after bundle adjustment. The influences are evaluated with respect to interior and exterior orientation parameter changes and their correlations as well as the impact in object space. With the variety of data sets, camera/lens/platform configurations and pre-processing influences, these investigations indicate a number of different behaviours. Some specific advices in the usage of extended interior orientation models, like Fourier series, could be derived for a selection of the data sets. Significant reductions of image space systematics are achieved. Even though increasing standard deviations and correlations for the interior orientation parameters are a consequence, improvements in object space precision and image space reliability could be reached.

Current research of binocular vision systems mainly need to resolve the camera’s intrinsic parameters before the reconstruction of three-dimensional (3D) objects. The classical Zhang’ calibration is hardly to calculate all errors caused by perspective distortion and lens distortion. Also, the image-matching algorithm of the binocular vision system still needs to be improved to accelerate the reconstruction speed of welding pool surfaces. In this paper, a preset coordinate system was utilized for camera calibration instead of Zhang’ calibration. The binocular vision system was modified to capture images of welding pool surfaces by suppressing the strong arc interference during gas metal arc welding. Combining and improving the algorithms of speeded up robust features, binary robust invariant scalable keypoints, and KAZE, the feature information of points (i.e., RGB values, pixel coordinates) was extracted as the feature vector of the welding pool surface. Based on the characteristics of the welding images, a mismatch-elimination algorithm was developed to increase the accuracy of image-matching algorithms. The world coordinates of matching feature points were calculated to reconstruct the 3D shape of the welding pool surface. The effectiveness and accuracy of the reconstruction of welding pool surfaces were verified by experimental results. This research proposes the development of binocular vision algorithms that can reconstruct the surface of welding pools accurately to realize intelligent welding control systems in the future.

Previous camera calibration methods often use a checkerboard to capture images and estimate the camera parameters from the correspondences between images and the checkerboard. The corner points in the checkerboard images are used as useful features for correspondence matching. Therefore, it is essential to precisely find the corner points in the checkerboard images. In many previous works, the corner points are extracted assuming that the checkerboard images are not distorted by its lens. Instead, image blurring and Gaussian noise on the images are usually considered, but other cases are not dealt with. However, the captured checkerboard images are often corrupted by lens distortions and compression artifacts, which leads to performance degradation of corner point extraction. Moreover, the corner points are extracted individually in the previous methods without considering their geometric relations. To better handle the corner point extraction problem under lens distortions, in our corner point extraction optimization, the distorted locations of the pixels on checkerboard images are corrected with the camera parameters, and the structural constraints for checkerboard image grids are then applied under the line-to-line mapping. Also, to robustly find the blurred edges between corner points due to JPEG compression, an edge surface model is newly proposed that models the transitions with over- and under-shoots around the blurred edges. Extensive experimental results show that our method significantly outperforms the state-of-the-art method with average 88.3% and 54.3% reduction in RMSE for corner point reprojection and camera parameter estimation, respectively under compression and lens distortions for synthetic and real data.

The laser-based 3-D reconstruction is crucial for autonomous manipulation and resource exploration in both the air and underwater scenarios, due to its accurate precision and robustness to disturbances. However, most current laser-based 3-D reconstruction sensors cannot be applied across different media without recalibration directly, e.g., air, air $\rightarrow $ glass $\rightarrow $ air, and air $\rightarrow $ glass $\rightarrow $ water, which is called killing three birds with one stone. This is because the variation of the medium density could change the sensor calibration parameters and further make the sensor suffer from a systematic geometric bias. To address these challenges, a unified laser-based 3-D reconstruction method is proposed in order that the laser-based scanner could be used across different media without recalibration, where we first explicitly model the refraction of the underwater vision systems and transform it across different media into a unified sensor reference frame. More specifically, an underwater refractive camera calibration model is designed to calibrate the orientation and position of the refractive interface, which can improve the accuracy of underwater reconstruction for the laser-based scanner, and we then present a refractive pose estimation model with a unified sensor reference frame, which can help the sensor to be applied across different scenarios directly. For the experiments, we validate the performance of our method on our underwater 3-D scanner prototype. Several reconstruction results on different objects and scenarios demonstrate the effectiveness of our proposed method and the practicality of our designed sensor.

Photogrammetric processes such as camera calibration, feature and target detection and referencing are assumed to strongly depend on the quality of the images that are provided for the process. Consequently, motion and optically blurred images are usually excluded from photogrammetric processes to supress their negative influence. To evaluate how much optical blur is acceptable and how large the influence of optical blur is on photogrammetric procedures a variety of test environments were established. These were based upon previous motion blur research and included test fields for the analysis of camera calibration. For the evaluation, a DSLR camera as well as Lytro Illum light field camera were used. The results show that optical blur has a negative influence on photogrammetric procedures, mostly automatic target detection. With the intervention of an experienced operator and the use of semi-automatic tools, acceptable results can be established.

Technological improvement of Unmanned Aerial Vehicles (UAVs) and computer vision algorithms, such as Structured-from-Motion (SfM) and Multi-view Stereo (MVS) have provided the possibility for high-resolution mapping and high-density point cloud generation using low-cost equipment and sensors. Orthomosaics and Digital Terrain Model (DTM) are the main digital products considering mapping purposes. Their quality is directly related to the sensors boarded on the UAV and data processing. Ground Control Points (GCPs) are used in the process of indirect georeferencing and also to model the lens distortions. The number of GCPs used in this process affects the positional accuracy of the final products. This study aims to determine the optimum number of GCPs to achieve high accuracy orthomosaics and DTM. To obtain this optimum number, an area of 3.85 ha was mapped with a low-cost UAV DJI Phantom 4 Advanced at 31 m flying height, lateral and longitudinal overlap of 90% and 80%, respectively, and using 22 checkpoints for quality assessment. For the experiments, different configuration were used both for the number of GCPs and for the use of self-calibration process or pre-calibrated camera IOP (Interior Orientation Parameters). The results show that for the flight configuration used in this work and for the mentioned UAV, a total of 5 GCPs, with pre-calibrated camera IOP, yields an accuracy of 0.023 m for X, 0.031 m for Y and 0.033 m for Z.

Traditional methods of geometric camera calibration (GCC) are based on angle measurements (AM) or diffractive optical elements (DOE). However, the AM-based approach has the low accuracy and reliability because the vibration of rotating mechanisms, i.e., turntables and angle measuring accuracy easily influence the calibration accuracy. The DOE failure easily occurs in the calibration process when some micro-apertures are blocked by dust particles. In this paper, a new method for the GCC by means of single pixel illumination in a deep neural network (DNN) is presented. A closed-loop calibration link is composed of a single stimulus input produced by a single pixel generator and a collimator, an uncalibrated camera, and a DNN. The dynamic single-pixel illumination forms the different stimulus input of the DNN at different stimulus time. The synaptic weights (the camera interior parameters) of multilayer perceptron are adjusted until the cost function is minimized. This method is able to avoid the aforementioned shortcoming of conventional calibration methods. This method can be especially used for on-ground calibration of remote sensing cameras but in principle also suitable for on-orbit GCC and other cameras.

Situational awareness of the crew is critical for the safety of the air flight. Head-up display allows providing all required flight information in front of the pilot over the cockpit view visible through the cockpit’s front window. This device has been created for solving the problem of informational overload during piloting of an aircraft. While computer graphics such as scales and digital terrain model can be easily presented on the such display, errors in the Head-up display alignment for correct presenting of sensor data pose challenges. The main problem arises from the parallax between the pilot’s eyes and the position of the camera. This paper is focused on the development of an online calibration algorithm for conform projection of the 3D terrain and runway models on the pilot’s head-up display. The aim of our algorithm is to align the objects visible through the cockpit glass with their projections on the Head-up display. To improve the projection accuracy, we use an additional optical sensor installed on the aircraft. We combine classical photogrammetric techniques with modern deep learning approaches. Specifically, we use an object detection neural network model to find the runway area and align runway projection with its actual location. Secondly, we re-project the sensor’s image onto the 3D model of the terrain to eliminate errors caused by the parallax. We developed an environment simulator to evaluate our algorithm. Using the simulator we prepared a large training dataset. The dataset includes 2000 images of video sequences representing aircraft’s motion during takeoff, landing and taxi. The results of the evaluation are encouraging and demonstrate both qualitatively and quantitatively that the proposed algorithm is capable of precise alignment of the 3D models projected on a Head-up display.

This paper presents an approach for Structure from Motion (SfM) for unorganized complex image sets. To achieve high accuracy and robustness, image triplets are employed and an (approximate) internal camera calibration is assumed to be known. The complexity of an image set is determined by the camera configurations which may include wide as well as weak baselines. Wide baselines occur for instance when terrestrial images and images from small Unmanned Aerial Systems (UAS) are combined. The resulting large (geometric/radiometric) distortions between images make image matching difficult possibly leading to an incomplete result. Weak baselines mean an insufficient distance between cameras compared to the distance of the observed scene and give rise to critical camera configurations. Inappropriate handling of such configurations may lead to various problems in triangulation-based SfM up to total failure. The focus of our approach lies on a complete linking of images even in case of wide or weak baselines. We do not rely on any additional information such as camera configurations, Global Positioning System (GPS) or an Inertial Navigation System (INS). As basis for generating suitable triplets to link the images, an iterative graph-based method is employed formulating image linking as the search for a terminal Steiner minimum tree in the line graph. SIFT (Lowe, 2004) descriptors are embedded into Hamming space for fast image similarity ranking. This is employed to limit the number of pairs to be geometrically verified by a computationally and more complex wide baseline matching method (Mayer et al., 2012). Critical camera configurations which are not suitable for geometric verification are detected by means of classification (Michelini and Mayer, 2019). Additionally, we propose a graph-based approach for the optimization of the hierarchical merging of triplets to efficiently generate larger image subsets. By this means, a complete, 3D reconstruction of the scene is obtained. Experiments demonstrate that the approach is able to produce reliable orientation for large image sets comprising wide as well as weak baseline configurations.

Camera calibration is an avoidable process for computational vision applications, such as 3D reverse engineering, industrial robot calibration, optic-pattern recognition, simultaneous localization and mapping, autonomous visual-driving and photogrammetric vision. The camera calibration problem is too complex, nonlinear and multimodal. Traditional camera calibration methods using gradient-based optimization often trap to one of the many local solutions available. Accurate computation ability of traditional camera calibration methods is limited since they use gradient-based optimization methods. Since evolutionary computing algorithms can avoid local solutions of numerical problems, they have the potential to accurately compute the required camera calibration parameters for high-precision computational vision applications. In this paper, the camera calibration parameters are computed by using 11 evolutionary computing algorithms, i.e., WDE, ABC, PSO, COBIDE, DE, CS, GWO, TLBO, MVMO, FOA and LSHADE. In order to make unbiased evaluation of the camera calibration results provided by the related evolutionary computing algorithms, two gradient-based traditional camera calibration methods, i.e., Zhang and Bouguet, have been used in the conducted experiments in this paper. The camera calibration results of the related methods were used to model a 3D physical test scene by using Structure from Motion photogrammetry method. The reference data set of the related 3D physical scene has been captured by using a 3D terrestrial laser scanner. Statistical comparison of the camera calibration results exposed that WDE supplies statistically better results than other comparison algorithms.

Camera calibration methods usually consist of capturing images of known calibration patterns and using the detected correspondences to optimize the parameters of the assumed camera model. A meaningful evaluation of these methods relies on the availability of realistic synthetic data. In previous works concerned with conventional cameras the synthetic data was mainly created by rendering perfect images with a pinhole camera and subsequently adding distortions and aberrations to the renderings and correspondences according to the assumed camera model. This method can bias the evaluation since not every camera perfectly complies with an assumed model. Furthermore, in the field of plenoptic camera calibration there is no synthetic ground truth data available at all. We address these problems by proposing a method based on backward ray tracing to create realistic ground truth data that can be used for an unbiased evaluation of calibration methods for both types of cameras., 9 pages, 8 figures. Accepted at 3DV 2019

The existing lens correction methods deal with the distortion correction by one or more specific image distortion models. However, distortion determination may fail when an unsuitable model is used. So, methods based on the distortion model would have some drawbacks. A model-free lens distortion correction based on the phase analysis of fringe-patterns is proposed in this paper. Firstly, the mathematical relationship of the distortion displacement and the modulated phase of the sinusoidal fringe-pattern are established in theory. By the phase demodulation analysis of the fringe-pattern, the distortion displacement map can be determined point by point for the whole distorted image. So, the image correction is achieved according to the distortion displacement map by a model-free approach. Furthermore, the distortion center, which is important in obtaining an optimal result, is measured by the instantaneous frequency distribution according to the character of distortion automatically. Numerical simulation and experiments performed by a wide-angle lens are carried out to validate the method.

Lens distortion can introduce deviations in visual measurement and positioning. The distortion can be minimized by optimizing the lens and selecting high-quality optical glass, but it cannot be completely eliminated. Most existing correction methods are based on accurate distortion models and stable image characteristics. However, the distortion is usually a mixture of the radial distortion and the tangential distortion of the lens group, which makes it difficult for the mathematical model to accurately fit the non-uniform distortion. This paper proposes a new model-independent lens complex distortion correction method. Taking the horizontal and vertical stripe pattern as the calibration target, the sub-pixel value distribution visualizes the image distortion, and the correction parameters are directly obtained from the pixel distribution. A quantitative evaluation method suitable for model-independent methods is proposed. The method only calculates the error based on the characteristic points of the corrected picture itself. Experiments show that this method can accurately correct distortion with only 8 pictures, with an error of 0.39 pixels, which provides a simple method for complex lens distortion correction.

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector's focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size.

A method based on 3D digital image correlation (DIC) to measure the shape of specular surface is proposed. The proposed active speckle deflectometry (ASD) utilizes a stereo-camera system to monitor the liquid crystal display (LCD), which is deliberately moved during the measurement. Another testing camera (TC) is used to capture the single-shot speckle pattern displayed on the LCD screen after reflection by the test surface. With this proposal, the movement of the LCD screen can be arbitrary as long as the TC can capture the reflection of speckle pattern. The distance as well as the direction of the movement is not required to be known. The coordinates of the point source are determined by applying the 3D DIC technique with the monitoring stereo-cameras (MSC) before and after the movement of the LCD screen, then the slope and surface shape are obtained. The measurement accuracy of this method is evaluated by measuring a flat glass with a diameter of about 80 mm, compared with the measurement results of interferometer, the shape measurement difference is 0.278um in root mean square (RMS). The shape of two wafers is also measured, and the measurement results are compared to that of the traditional phase measuring deflectometry (PMD). ASD has the advantages of fast measurement, low cost, arbitrary LCD movement, tolerance for the out-of-plane shape of the LCD screen. It provides a new method for specular surface measurement.

The key problems in a structured-light 3D scanner using a projector and camera are its geometric and radiometric calibration. A projector can be effectively represented using the familiar pin-hole model for cameras. However, since we cannot directly observe the projector plane, it cannot be calibrated like a camera. The first contribution of our paper is to develop a geometric calibration and 3D estimation method that utilises the projective geometric relationships available in a projector---camera pair, i.e., homography induced by a plane and invariance of cross-ratios. The low-dimensional parametric form of the homography averages out individual errors, resulting in a geometric calibration approach that is both simple to use and highly accurate. We present an extensive set of results to demonstrate the effectiveness of our approach and also characterise its accuracy. Second, we present a method for correcting systematic errors introduced due to the radiometric non-linearities present in commercial projectors. These errors are pronounced when only a few phase shifts are used in the sinusoidal coding scheme for structured-light scanners. We develop a cubic spline-based method to model and remove the effects due to these non-linearities. The efficacy of our model is demonstrated on real datasets.

Fringe projection is a widely used technique for measurement of out-of-plane deformation of diffusely reflecting objects. Among the various fringe patterns employed for projection, usage of linear fringes is most common. However, the phase function of linear fringes has 2π periodicity, thereby leading to ambiguity in determining whole-field out-of-plane deformation of dynamic objects. This paper presents a new method of measurement of whole-field out-of-plane deformation of targets by projecting a circular fringe pattern. New technique of fringe analysis is introduced to estimate the underlying phase distribution of the fringe pattern from a single image. In this technique, the fringe pattern is coordinate transformed into polar coordinates, and then Fourier fringe analysis technique is employed to obtain its phase. Subsequent to unwrapping, the phase map is re-transformed to Cartesian coordinates. The phase map obtained after coordinate transformation is converted to out-of-plane deformation map using camera calibration parameters. Simulations are carried out to establish the analysis method. Experimental validation is performed with the development of circular grating based fringe projection system.

Consumer grade digital cameras are widely used in many applications including Photogrammetric mapping and 3D modelling. One common limitation found in such cameras is radial lens distortions. To produce wide angle lenses camera manufacturers reduce the amount of barrel distortion by minimizing both the central and edge distortion profiles, resulting in a mixture of pincushion and barrel distortions for a single lens. These lenses also lack symmetry, making some of the existing distortion models almost ineffective. The mostly used model for radial distortion corrections is the polynomial model which is difficult to solve analytically especially when the model possesses many quadratic terms. Suggestions were made for division models but such models are not suitable when the lens field of view exceeds 180 degrees and exhibit some instabilities when dealing with large magnitude of distortion coefficients. Moreover mathematical formulations of some models cannot handle negative distortion coefficients. Attempts to improve the division models were made with proposals for rational models which present the advantage of handling larger distortion magnitude with fewer terms. However some of these models do not account for all the distortion coefficients in their solutions, limiting the potential of the techniques. This study presents an irrational distortion model with analytical solutions. The proposed model was tested with imagery captured by wide angle lenses and the experimental results reveal that the technique produced the best estimates of radial distortion coefficients. The proposed model was also able to capture image distortions originating from projection errors by the wide angles lenses used in this study.

When information derived from the imagery is used for metric purposes and not as communication tools the impact of small imaging errors can be significant on the accuracy of derived information. As the number of photogrammetry applications grows and the technology advances, camera calibration becomes more complex. In fact, most lenses suffer from at least one kind of distortion profile such as barrel, pincushion or ‘moustache’ profiles. Commonly observed on wide angle lenses, barrel distortions curve straight lines inwards to give the image shape of barrel, however most lenses are prone to more than one distortion profile. Proposals for lens correction have been made using line-based approaches as well as division and rational models. The limitations of some of the techniques include the very limited number of distortion profiles they can handle due to their mathematical formulations. For example some can only handle distortion with positive coefficients and would exhibit instabilities when dealing with barrel distortions describe by negative distortion coefficients. In this paper we propose a new class of hyperbolic radial distortion model which handles several distortion profiles. The mathematical formulation of the new approach offers stability of the model since it can handle both positive and negative distortion coefficients -an improvement on some of the current techniques. The calibration results show that the approach produced the best distortion coefficients k1 and k2. The model can handle distortions from panorama imagery as well as ‘moustache ’profile produced by wide angle lenses.

This paper is focused on the finite-element (FE) method of camera calibration. The FE method enables the modelling of systematic error effects, including those which cannot be recovered by standard modelling procedures, e.g., those based on Brown’s distortion model. The FE approach to camera modelling has been previously published a number of times; however, some important aspects were not sufficiently addressed in this earlier research work. In addition, the computing power was too low to test the finite-element method with high-resolution FE grid. The proposed FE implementation is fully independent of any polynomial model and includes correction of the distance-dependent distortion effect. Besides modelling the effects such as lens distortion and sensor unflatness, the approach also accommodates the calibration of non-perspective lenses such as fisheye lenses. In addition to introducing the proposed FE calibration method, this paper addresses the related issues of sufficient target density, correction pattern smoothness and FE grid size. It also reports on experimental testing of the new FE implementation using the acceptance test procedure of the German VDI guideline 2634. Two different cameras were calibrated within the acceptance tests to analyse the impacts of the sensor size and field of view of the lens. For comparison with the FE method, both data sets were also processed using standard photogrammetric software (AICON 3D Studio). The results have proven the ability of the proposed FE modification to recover any systematic effects and to model ultra-wide field-of-view lenses, while achieving highly accurate measurements. The method is able to model the distance-dependent distortion effect, but requires a very large number of observations, which may be expensive and difficult to establish in practise. The proposed method, which can be characterised by utilising a high-resolution grid, is mostly intended for laboratory calibration of highly stable camera systems and not for on-the-job type calibration, where the target density would likely not be sufficiently large.

The traditional calibration paradigms fail to give reliable and accurate results in case of low-quality 2D planar calibration plates. In this paper, an active method is proposed by employing an LCD panel for camera calibration. This method automatically generates a sequence of virtual patterns in different views by pre-defined transforms without manually manipulation or other equipment’s help to move the patterns. Then, the projections of virtual patterns are captured by a camera. The homography between projective patterns in virtual world coordinate and their images is calculated directly to obtain the camera parameters. Experimental results show that the calibration error is 0.018 pixel in terms of mean re-projection error by using 18 virtual patterns, which is significantly less than the state-of-the-art methods. The proposed scheme makes camera calibration flexible and easy to use.

Research on wheeled soccer robots has been carried out by several researchers. This is due to the existence of national and international competitions. Previous research was to create a ball position transformation system with a modified method of neural network architecture. This research was developed by building an intelligent transformation system with the Tensorflow library. This transformation system aims to be able to directly measure the distance of objects in real terms without first changing the environmental image from an omni field to a flat plane with conventional camera calibration techniques. This process can replace manual calibration with a variety of field size changes The system can transform with mean error 0.0000026 on epoch 10000 using “conda-tensorflowneural network” libraries. It can transform the position of the ball from the omni space to the cartesian space. This system was implemented on wheeled soccer robot as keeper.

Current state-of-the-art in speed measurement technologies includes magnetic inductive loop detectors, Doppler radar, infrared sensors, and laser sensors. Many of these systems rely on intrusive methods that require intricate installation and maintenance processes that hinder traffic while leading to high acquisition and maintenance costs. Speed measurement from monocular videos appears as an alternative in this context. However, most of these systems present as a drawback the requirement of camera calibration - a fundamental step to convert the vehicle speed from pixels per frame to some real-world unit of measurement (e.g. km/h). Considering that, we propose a speed measurement system based on monocular cameras with no need for calibration. Our proposed system was trained from a synthetic data set containing 12,290 instances of vehicle speeds. We extract the motion information of the vehicles that pass in a specific region of the image by using dense optical flow, using it as input to a regressor based on a customized VGG-16 network. The performance of our method was evaluated over the Luvizon's data set, which contains real-world scenarios with 7,766 vehicle speeds, ground-truthed by a high precision system based on properly calibrated and approved inductive loop detectors. Our proposed system was able to measure 85.4% of the speed instances within an error range of [-3, + 2] km/h, which is ideally defined by the regulatory authorities in several countries. Our proposed system does not rely on any distance measurements in the real world as input, eliminating the need for camera calibration.

Optical metrology techniques such as fringe projection technology and phase-measuring deflectometry, which use cameras as image sensors for 3D coordinate measurement and inspection, are becoming increasingly demanding. The necessary precise and at the same time robust calibration of image sensors of these systems is offered by so called vision ray calibration. This generic approach is a model-free description of a vision ray and are determined for each imaging sensor pixel. Numerous images of the sinusoidal fringe images displayed on the active calibration target (LCD) are captured in different poses. This generates a large amount of data and consequently leads to time-consuming evaluation for the camera calibration. In this paper, we propose a new optimization method that uses “vision threads" instead of vision rays to enhance the computation efficiency associated with conventional vision ray calibration. We present MATLAB simulation results in to validate our novel approach.

Vision Ray Calibration provides a description of imaging properties of cameras by identification of an independent vision ray for each sensor pixel. Due to this approach, no model parameters of any cameras are determined in the non-linear optimization procedure of the calibration. Therefore, a setup of multiple cameras can be considered as one imaging system. This enables simultaneous and holistic calibration of an arbitrary number of cameras. Vision Ray Calibration utilizes Liquid Crystal Displays as calibration targets since these can provide the required continuous spatial coding of their surfaces by means of Phase Shifting Technique. However, displays employed as calibration targets exhibit some unfavorable properties such as flatness deviations of the surface. It is known that extending the Vision Ray Calibration by a polynomial parameterization of the display surface increases the calibration accuracy. This work investigates the influence of the order of polynomial terms employed for display surface parameterization. The stereoscopic Fringe Projection enables the evaluation of the calibration accuracy since Vision Ray Calibration of a stereo camera setup provides full system calibration for this technique. Our results confirm the significant improvement of calibration accuracy due to parameterization of flatness deviations of the display. Best calibration results were obtained by limiting the maximum order of polynomial terms to three. Our results indicate that terms of higher orders do not contribute true surface shape features.

Objetive:To realize real-time measurement and monitoring of helicopter motion state, based on fully considering the rotor measurement environment, power supply, information storage, processing, and information transmission, a set of rotor motion parameter test system based on vision is designed. The system adopts a non-contact measurement method, which can realize the real-time measurement and monitoring of the rotor state. Method: First, the overall design of the rotor motion state test scheme is given, and the working principle and flow chart are provided. Then combined with the actual measurement environment, it presents the concrete method of camera calibration and data processing. Result: The measurement accuracy under this type of selection and layout is evaluated. After satisfying the design accuracy, the test environment is set up in the laboratory, and the measurement accuracy is assessed through the actual test. Experimental results indicate that the measurement accuracy of the system is better than 1cm for the rotor of 8m, when the measurement baseline is 350mm, and the image recognition is better than 0.5 pixels. Conclusion: It can satisfy the system requirements of online, state accuracy measurement.

The evaluation of industrial robot path accuracy is significant for improving its dynamic operating performance. In this paper, a monocular stereo vision (MSV) based method is proposed for measuring the path error of robot end-effectors. Firstly, the structure parameters of the MSV system are analyzed to determine appropriate imaging parameters of the equivalent virtual binocular vision (VBV). Secondly, to save cost and measurement time, an all-in-one high-accuracy artifact, which integrates path representation and camera calibration, is designed. And accordingly, an improved decoding method is proposed to distinguish different tags on the artifact. After calibrating depth of field (DoF) dependent lens distortion, the path accuracy can be assessed with high-accuracy by the difference between path measured by the vision and the nominal one. Experimental results show that the MSV system with an average measurement accuracy of 0.012 mm can evaluate the path performance of the robot, and the maximum circular path error at 1.5 m/min is 0.252 mm.

We propose an off-axis deflectometric microscope system for microscopic surface testing with both high measurement accuracy and a large slope dynamic range. A high-luminance liquid crystal display directly illuminates the tested sample with coded fringes, and then the reflected fringes passing through a microscope objective are captured by a pinhole camera, from which the deflectometric microscopic testing with a large slope range can be achieved. The accuracy of the proposed system is validated numerically and experimentally, and a large measurable slope dynamic range is also demonstrated. The proposed system provides a feasible way with the slope range in the order of sub-radians and sag resolution better than 0.05 nm.

Stereo vision can provide surgeons with 3D images and reduce the difficulty of operation in robot-assisted surgery. In natural orifice transluminal endoscopic surgery, distortions of the stereoscopic images could be induced at different observation depths. This would increase the risk of surgery. We proposed a novel camera to solve this problem. This study integrated the camera calibration matrix and the geometric model of stereoscopic system to find the cause of distortion. It was found that image distortions were caused by inappropriate disparity, and this could be avoided by changing the camera baseline. We found the relationship between camera baseline and observation depth with the model. A variable baseline stereoscopic camera with deployable structure was designed to achieve this requirement. The baseline could be adjusted to provide appropriate disparity. Three controlled experiments were conducted to verify the stereo vision of the proposed camera at different observation depths. No significant difference was observed in the completion time. At the observation depths of 30 mm and 90 mm, the number of errors apparently decreased by 62.90% and 51.06%, respectively. The significant decrease in number of errors shows that the proposed camera has a better stereo vision than a regular camera at both small and large observation depths. It can produce more accurate stereoscopic images at any depth. This will further improve the safety of robot-assisted surgery.

In order to accurately and efficiently measure the inner and outer diameters of bearing seal rings, a monocular measurement instrument was developed. The mathematical principles of camera calibration and distortion correction were studied. After the figures of rings were distortion corrected and preprocessed, we calculate the inner and outer diameters based on Otsu threshold segmentation and RANSAC algorithm. Operating the monocular measuring instrument to measure eight different types of sealing rings. According to the measurement results, it is found that the average time to measure the two diameters of a sealing ring is 0.79 seconds and the relative error of the measurement is 0.12%, which proves that the monocular measuring instrument can meet actual needs for sampling measurement of rings in the factory laboratory.

To computer-generate high-quality holographic stereograms, a huge number of images must be provided: several hundred for a horizontal parallax and the square of this number for a full parallax. In this paper, we propose to drastically reduce this number to four input images with depth maps (or equivalently, four groups of neighboring images used to compute a depth map) in any pose, in order to create the missing images with depth image-based rendering. We evaluate the view synthesis method objectively before providing visual results of the corresponding holographic stereograms. We believe this method outperforms shearlet-based approaches in objective view synthesis quality metrics and in the number of required input images (7×7)., info:eu-repo/semantics/published

Purpose: Emerging deep-brain stimulation (DBS) procedures require a high degree of accuracy in placement of neuroelectrodes, even in the presence of deformation due to cerebrospinal fluid (CSF) egress during surgical access. We are developing ventriculoscope and hand-eye calibration methods for a robot-assisted guidance system to augment accurate electrode placement through transventricular approach. Methods: The ventriculoscope camera was modelled and calibrated for lens distortion using three different checkerboards, followed by evaluation on a separate board. The experimental system employed a benchtop UR3e robot (Universal Robots, Denmark) and ventriculoscope (Karl Storz, Tuttlingen, Germany) affixed to the end effector – referred to as the robotassisted ventriculoscopy (RAV) platform. Performance was evaluated in terms of three error metrics (RPE, FCE and PDE). Experiments were conducted to estimate the camera frame of reference using hand-eye calibration methods, and evaluated using a ChAruco board, using five different solvers and residual calibration error as the metric. Results: Camera calibration demonstrated subpixel (0.81 ± 0.11) px reprojection error and projection distance error (PDE)

Purpose: Pelvic fracture fixation is a challenging procedure that commonly relies on 2D fluoroscopic guidance to place guidewires within complex bone corridors. Prior work reported on a video-on-drill navigation system for guidewire insertion as a potential solution. Here, we assess performance across a range of hardware components to help guide the design of future system prototypes with respect to clinical requirements. Methods: The video-on-drill system uses a camera rigidly mounted on the drill and multimodality fiducial markers (optical and radio-opaque) to provide real-time trajectory visualization. This work reports on the selection of a new camera+lens configuration. Configurations were assessed across two cameras (referred to as the ArduCam and ELP) and five lens options (A45-A90). Clinical requirements were specified by an orthopaedic-surgeon in terms of the nominal drill operating distance (𝐷) and operating area (𝐴). Performance was evaluated in terms of the accuracy of fiducial marker pose estimation (δ𝐷), and the field of view (FOV). Results: At matched FOV, the accuracy for the ELP camera was significantly better (p < 0.01) with median δ𝐷 of 1.26 mm (1.1 mm IQR) compared to the ArduCam [median δ𝐷 of 1.85 mm (2.0 mm IQR)]. The accuracy of the A45 and A55 lens was found to be suitable (δ𝐷 < 2 mm) while providing sufficient FOV at nominal drill operating conditions. Conclusion: With respect to application requirements, the camera+lens combination (ELP+Ardu55) was identified to provide the best performance, serving as an important precursor to future design iterations of the video-on-drill system.

High precision camera calibration is the premise of establishing 3D surface measurement system with excellent performance. Therefore, it is necessary to conduct systematic research on camera calibration method to improve measurement efficiency and accuracy. The original Tsai Two-Step method had two main disadvantages: firstly, different points on different planes should be adopted for camera calibration; Secondly, only radial distortion of the lens is considered. Many scholars optimized the method, but most of them optimized one of its shortcomings. In this paper, comprehensive optimization was considered to improve the two shortcomings together. At the same time, Taylor series expansion method is used to process the data and improve the calibration speed and accuracy. The improved method is proved to be feasible by comparing experiments with the Zhang's calibration method.

[EN] Camera calibration is a crucial step in robotics and computer vision. Accurate camera parameters are necessary to achieve robust applications. Nowadays, camera calibration process consists of adjusting a set of data to a pin-hole model, assuming that with a reprojection error close to zero, camera parameters are correct. Since all camera parameters are unknown, computed results are considered true. However, the pin-hole model does not represent the camera behavior accurately if the autofocus is considered. Real cameras with autofocus lenses change the focal length slightly to obtain sharp objects in the image, and this feature skews the calibration result if a unique pin-hole model is computed with a constant focal length. In this article, a deep analysis of the camera calibration process is done to detect and strengthen its weaknesses when autofocus lenses are used. To demonstrate that significant errors exist in computed extrinsic parameters, the camera is mounted in a robot arm to know true extrinsic camera parameters with an accuracy under 1 mm. It is also demonstrated that errors in extrinsic camera parameters are compensated with bias in intrinsic camera parameters. Since significant errors exist with autofocus lenses, a modification of the widely accepted camera calibration method using images of a planar template is presented. A pin-hole model with distance-dependent focal length is proposed to improve the calibration process substantially.

We describe a novel projector design based on interferometric fringe projection. The key component in the projector, a movable micro-mirror, has undergone initial vibration, TVAC and radiation testing, with no observed component degradation. We use multi-frame active stereo as basis for 3D reconstruction, giving 3D measurement standard deviation of 0.2 mm, with 12.5 Hz 3D image capture rate and 3D image resolution of 500 x 500 pixels. The system's low power, small size and component longevity makes it well suitable for space applications.

Welding robots have been widely used in manufacturing process to substitute for human welders. In an effort to enhance their flexibility as well as accomplishing more efficient industrial welding, this paper proposes a novel calibrated strategy based on visual guidance for smart welding, which includes the camera calibration, the calibration of the laser plane, and the calibration of the hand-eye. The corresponding experimental results have demonstrated the effectiveness, dependability and efficiency of the calibration strategy.

Light field cameras capture spatial and angular information simultaneously. A scene point in the 3D space appears many times on the raw image, bringing challenges to light field camera calibration. This paper proposes a novel calibration method for standard plenoptic cameras by using corner features from raw images. We select appropriate micro-lens images on raw images and detect corner features on them. During calibration, we first build the relationship of corner features and points in object space by using a few intrinsic parameters and then perform a linear calculation of these parameters, which are further refined via a non-linear optimization. Experiments on Lytro and Lytro Illum cameras demonstrate that the accuracy and efficiency of the proposed method are superior to the state-of-the-art methods based on features of raw images.

LAMOST, as the astronomical telescope with the highest spectrum acquisition efficiency in the world, requires high positioning accuracy, the maximum allowable positioning error is only 40μm. Due to various aberrations, the general photogrammetry system cannot meet the requirements of detecting high-precision optical fiber positioning errors. In this study, we proposed a double telecentric measurement system to detect accurately the actual position of fiber positioner by taking advantage of ultra-low distortion and ultra-wide depth of field of telecentric lens. In this paper, the main sources of telecentric lens distortion were analyzed, and the calibration method of polynomial calibration model and dot matrix calibration target was adopted, and the validity of the five parameters obtained by calibration is verified by experiments. The experimental results showed that all optical fibers reached the target after two steps of approximation, and the errors met the positioning requirements.

The mobile manipulator with vision sensor is of great significance to the future factory. How to make the mobile platform part participate in the process of trajectory planning efficiently, and how to reduce the camera calibration error and the influence of external factors on the visual servo accuracy are some problems that need to be solved. Uncalibrated Vision Servo (UVS) based on image has great flexibility and adaptability, therefore, it will solve many problems when it is applied to omni-directional mobile manipulator. A complete constrained mobile manipulator model is established. In this paper, we design a visual servo controller based on the centralized control strategy, and estimate the mapping relationship from the image feature space to the manipulator joint and the moving space of the mobile platform by online identification of the composite Jacobian matrix. Finally, the simulation experiment of Moving plane servo control in space is carried out.

Structured light is an optical 3D surface measurement technique with the merits of high speed and high robustness. However, the huge size of traditional digital light processing (DLP) projectors limits its convenience in numerous applications. In this paper, a one-axis MEMS mirror is used as the structured light projector in 3D modeling systems, and has the advantages of small volume and low cost. Limited by the inability to project orthogonal patterns and projection distortion, it is difficult for the one-axis MEMS mirror based 3D modeling system to obtain high accuracy through existing calibration methods. This paper proposed a calibration method for structured light 3D modeling systems that can only project stripes in one direction with projection distortion. A curved surface equation called curved light surface model was proposed to replace the ideal plane equation as the mathematic model of the projected structured light stripes. Experiment results verified that this method can significantly reduce the effect of projection distortion and an accuracy of 0.11 mm was achieved when measuring a standard dumbbell-shaped object with 201.10 mm center-to-center distance.

Computer vision as an important branch of computer science and artificial intelligence has made rapid progress in the past thirty years. Binocular stereo vision is one of the most important parts in computer vision. Binocular stereo vision can well simulate human eyes stereoscopic perception of three-dimensional objects, and has been widely used in various fields of industrial automation production and practical life. Binocular stereo vision technology is a comprehensive technology, whose knowledge includes optics, physics, image processing, computer, artificial intelligence and electronic technology, and other fields of content. Camera calibration and feature point matching are difficult and key points in binocular stereo vision technology. In this paper, the 3D reconstruction of binocular stereo vision based on feature point matching method is discussed. The main research includes the following contents: binocular stereo vision system principle, feature matching algorithm research and 3D reconstruction system implementation. This project puts forward a set of feasible algorithms and finally can get a good three-dimensional reconstruction effect based on the analysis and research of a series of algorithms.

Optical remote sensing satellite usually adopts the method of photographing calibration field to achieve calibration of internal and external orientation elements of optical camera. However, as time goes on, or as the latitude changes, the accuracy of satellite positioning after calibration will decline. The traditional calibration method requires satellite passes calibration field and is difficult to achieve globally, so it's hard to be realized that keeping the global positioning accuracy high. The purpose of this study is to explore a more feasibility calibration method of interior orientation elements in optical satellites. That is, by photographing stars, the stars are used as control points to calibrate the camera's internal orientation elements. In this study, a camera calibration method based on multi-frame star map was proposed. The star maps were captured by the star camera of Ziyuan-3 02 (ZY-3 02). Final result proved to be efficient to offer sufficient information for interior orientation elements calibration purpose.

The novel approach to physical security based on visible light communication (VLC) using an informative object-pointing and simultaneous recognition by high-framerate (HFR) vision systems is presented in this study. In the proposed approach, a convolutional neural network (CNN) based object detection method is used to detect the environmental objects that assist a spatiotemporal-modulated-pattern (SMP) based imperceptible projection mapping for pointing the desired objects. The distantly located HFR vision systems that operate at hundreds of frames per second (fps) can recognize and localize the pointed objects in real-time. The prototype of an artificial intelligence-enabled camera-projector (AiCP) system is used as a transmitter that detects the multiple objects in real-time at 30 fps and simultaneously projects the detection results by means of the encoded-480-Hz-SMP masks on to the objects. The multiple 480-fps HFR vision systems as receivers can recognize the pointed objects by decoding pixel-brightness variations in HFR sequences without any camera calibration or complex recognition methods. Several experiments were conducted to demonstrate our proposed method’s usefulness using miniature and real-world objects under various conditions.