Search

In this paper, we aim to obtain a dense piecewise planar reconstruction of the scene from multiple image frames based on a factorization framework. Integrating all the relevant constraints in a global objective function, we are able to effectively leverage on the scene smoothness prior afforded by the dense formulation, as well as imposing the necessary algebraic constraints required by the shape matrix. These constraints also help to robustly decompose the measurement matrix into the underlying low-rank subspace and the sparse outlier part. Numerically, we achieve the constrained factorization and decomposition via modifying a recently proposed proximal alternating robust subspace minimization algorithm. The results show that our algorithm is effective in handling real life sequences, and outperforms other algorithms in recovering motions and dense scene estimate.

This paper presents an algorithm-independent geometrical analysis of the behavior of differential Structure from Motion (SFM) algorithms when there are errors in intrinsic parameters of the camera. We demonstrate both analytically and in simulation how uncertainty in the calibration parameters gets propagated to motion estimates in a differential setting. In particular, we studied how erroneous focal length and principal point estimates affect the behavior of the bas-relief ambiguity and introduce additional biasing to the translation estimate in a non-simple manner not revealed by previous analyses. Our formulation allows us to characterize the influence of various factors such as different scene-motion configurations and field of views in an analytically tractable manner. Guidelines are given as to whether one should err on the low or the high side in the estimation of the focal length depending on various operating conditions such as the feature density and the noise level. Simulations with synthetic data and real images were conducted to support our findings.

This paper presents an original algorithm to automatically acquire accurate camera calibration from broadcast tennis video (BTV) as well as demonstrates two of its many applications. Accurate camera calibration from BTV is challenging because the frame-data of BTV is often heavily distorted and full of errors, resulting in wildly fluctuating camera parameters. To meet this challenge, we propose a frame grouping technique, which is based on the observation that many frames in BTV possess the same camera viewpoint. Leveraging on this fact, our algorithm groups frames according to the camera viewpoints. We then perform a group-wise data analysis to obtain a more stable estimate of the camera parameters. Recognizing the fact that some of these parameters do vary somewhat even if they have similar camera viewpoint, we further employ a Hough-like search to tune such parameters, minimizing the reprojection disparity. This two-tiered process gains stability in the estimates of the camera parameters, and yet ensures good match between the model and the reprojected camera view via the tuning step. To demonstrate the utility of such stable calibration, we apply the camera matrix acquired to two applications: (a) 3D virtual content insertion; and (b) tennis-ball detection and tracking. The experimental results show that our algorithm is able to acquire accurate camera matrix and the two applications have very good performances.

There have been relatively little works to shed light on the effects of errors in the intrinsic parameters on motion estimation and scene reconstruction. Given that the estimation of the extrinsic and intrinsic parameters apts to be imprecise, it is important to study the resulting distortion on the recovered structure. By making use of the iso-distortion framework, we explicitly characterize the geometry of the distorted space recovered from 3D motion with freely varying focal length. This characterization allows us: (1) to investigate the effectiveness of the visibility constraint in disambiguating errors in calibration parameters by studying the negative distortion regions and (2) to make explicit those ambiguous error situations under which the visibility constraint is not effective. An important finding is that under these ambiguous situations, the direction of heading can be accurately recovered and the structure recovered experienced a well-behaved distortion. The distortion is given by a relief transformation which preserves ordinal depth relations. Thus in the case where the change in focal length is not well estimated, structure information in the form of depth relief can be obtained. Experiments were presented to support the use of the visibility constraint in obtaining such partial motion and structure solutions.

A key step for managing a large video database is to partition the video sequences into shots. Past approaches to this problem tend to confuse gradual shot changes with changes caused by smooth camera motions. This is in part due to the fact that camera motion has not been dealt with in a more fundamental way. We propose an approach that is based on a physical constraint used in optical flow analysis, namely, the total brightness of a scene point across two frames should remain constant if the change across two frames is a result of smooth camera motion. Since the brightness constraint would be violated across a shot change, the detection can be based on detecting the violation of this constraint. It is robust because it uses only the qualitative aspect of the brightness constraint?detecting a scene change rather than estimating the scene itself. Moreover, by tapping on the significant know-how in using this constraint, the algorithm's robustness is further enhanced. Experimental results are presented to demonstrate the performance of various algorithms. It is shown that our algorithm is less likely to interpret gradual camera motions as shot changes, resulting in a better precision performance than most other algorithms. However, its performance deteriorates under large camera or object motions. A twin-threshold scheme is proposed to improve its robustness.

A sequence of images acquired by a moving sensor contains information about the three-dimensional motion of the sensor and the shape of the imaged scene. Interesting research during the past few years has attempted to characterize the errors that arise in computing 3D motion (egomotion estimation) as well as the errors that result in the estimation of the scene's structure (structure from motion). Previous research is characterized by the use of optic flow or correspondence of features in the analysis as well as by the employment of particular algorithms and models of the scene in recovering expressions for the resulting errors. This paper presents a geometric framework that characterizes the relationship between 3D motion and shape in the presence of errors. We examine how the three-dimensional space recovered by a moving monocular observer, whose 3D motion is estimated with some error, is distorted. We characterize the space of distortions by its level sets, that is, we characterize the systematic distortion via a family of iso-distortion surfaces, which describes the locus over which the depths of points in the scene in view are distorted by the same multiplicative factor. The framework introduced in this way has a number of applications: Since the visible surfaces have positive depth (visibility constraint), by analyzing the geometry of the regions where the distortion factor is negative, that is, where the visibility constraint is violated, we make explicit situations which are likely to give rise to ambiguities in motion estimation, independent of the algorithm used. We provide a uniqueness analysis for 3D motion analysis from normal flow. We study the constraints on egomotion, object motion, and depth for an independently moving object to be detectable by a moving observer, and we offer a quantitative account of the precision needed in an inertial sensor for accurate estimation of 3D motion.