Search

Occlusions and complex backgrounds are common factors that hinder many computer vision applications. In a street scene, the challenge of accurately predicting pedestrian trajectories comes from the complexity of human behavior and the diversity of the external environment. It is difficult, if not impossible, to extract relevant information to accurately predict pedestrian trajectories in dynamic scenes. Synthetic aperture imaging (SAI) uses an array of cameras to mimic a camera with a large virtual convex lens by projecting images of a scene from different views onto a virtual focal plane. It is commonly used to reconstruct occluded objects, and in a street scene, can provide observation of pedestrians occluded by other objects and pedestrians. In this paper, we propose a joint prediction method based on autofocusing of SAI to predict pedestrian trajectories in dynamic scenes. The main contributions of this paper include: 1) The task of pedestrian trajectory prediction in dynamic scenarios is redefined as pedestrian trajectory prediction and SAI autofocusing from a practical but more challenging perspective. 2) The proposed method is based on an existing SAI-based method to extract information in heavily occluded views, which can obtain more accurate results but with less computational cost and without using other sensors such as LiDAR or depth cameras. 3) A new pedestrian trajectory prediction model, an attention-based trajectory prediction variational autoencoder (ATP-VAE), is proposed to extract complex human behavior and social interactions in dynamic scenes through a new Intention Attention Unit. The experimental results on multiple public datasets show that the proposed method achieves state-of-the-art results in the first-person perspective and in aerial view.

The key to crossview geolocalization is to match images of the same target from different viewpoints, e.g., images from drones and satellites. It is a challenging problem due to the changing appearance of objects from variable viewpoints. Most existing methods focus mainly on extracting global features or on segmenting feature maps, causing the loss of information contained in the images. To address the above issues, we propose a new ConvNeXt-based method called MCCG, which stands for Multiple Classifier for Cross-view Geolocalization. The proposed method captures rich discriminative information by cross-dimension interaction and acquires multiple feature representations, realizing a comprehensive feature representation. Additionally, the robustness of the model is improved crediting the multiple feature representations exploiting more contextual information despite position shifting or scale variations. Extensive experiments on the widely used public benchmarks University-1652 and SUES-200 demonstrate that the proposed method achieves state-of-the-art performance in both drone-view target localization and drone navigation applications by over 3% compared to existing methods. Our code and model are available at https://github.com/mode-str/crossview.

Conventional deconvolution methods utilize hand-crafted image priors to constrain the optimization. While deep-learning-based methods have simplified the optimization by end-to-end training, they fail to generalize well to blurs unseen in the training dataset. Thus, training image-specific models is important for higher generalization. Deep image prior (DIP) provides an approach to optimize the weights of a randomly initialized network with a single degraded image by maximum a posteriori (MAP), which shows that the architecture of a network can serve as the hand-crafted image prior. Unlike conventional hand-crafted image priors, which are obtained through statistical methods, finding a suitable network architecture is challenging due to the unclear relationship between images and their corresponding architectures. As a result, the network architecture cannot provide enough constraint for the latent sharp image. This paper proposes a new variational deep image prior (VDIP) for blind image deconvolution, which exploits additive hand-crafted image priors on latent sharp images and approximates a distribution for each pixel to avoid suboptimal solutions. Our mathematical analysis shows that the proposed method can better constrain the optimization. The experimental results further demonstrate that the generated images have better quality than that of the original DIP on benchmark datasets.

AbstractBackgroundEvidence of cerebral degeneration is not apparent on routine brain MRI in amyotrophic lateral sclerosis (ALS). Texture analysis can detect change in images based on the statistical properties of voxel intensities. Our objective was to test the utility of texture analysis in detecting cerebral degeneration in ALS. A secondary objective was to determine whether the performance of texture analysis is dependent on image resolution.MethodsHigh-resolution (0.5×0.5 mm2in-plane) coronal T2-weighted MRI of the brain were acquired from 12 patients with ALS and 19 healthy controls on a 4.7 Tesla MRI system. Image data sets at lower resolutions were created by down-sampling to 1×1, 2×2, 3×3, and 4×4 mm2. Texture features were extracted from a slice encompassing the corticospinal tract at the different resolutions and tested for their discriminatory power and correlations with clinical measures. Subjects were also classified by visual assessment by expert reviewers.ResultsTexture features were different between ALS patients and healthy controls at 1×1, 2×2, and 3×3 mm2resolutions. Texture features correlated with measures of upper motor neuron function and disability. Optimal classification performance was achieved when best-performing texture features were combined with visual assessment at 2×2 mm2resolution (0.851 area under the curve, 83% sensitivity, 79% specificity).ConclusionsTexture analysis can detect subtle abnormalities in MRI of ALS patients. The clinical yield of the method is dependent on image resolution. Texture analysis holds promise as a potential source of neuroimaging biomarkers in ALS.

A novel unsupervised approach to automatically constructing multilevel image clusters from unordered images is proposed in this paper. The whole input image collection is represented as an imaging sample space (ISS) consisting of globally indexed image features extracted by a new efficient multi-view image feature matching method. By making an analogy between image capturing and observation of ISS, each image is represented as a binary sequence, in which each bit indicates the visibility of a corresponding feature. Based on information theory-inspired image popularity and dissimilarity measures, we show that the image content and distance can be quantitatively described, guided by which an input image collection is organized into multilevel clusters automatically. The effectiveness and the efficiency of the proposed approach are demonstrated using three real image collections and promising results were obtained from both qualitative and quantitative evaluation.

In an underwater imaging system, a perspective camera is often placed outside a tank or in waterproof housing with a flat glass window. The refraction of light occurs when a light ray passes through the water–glass and air–glass interface, rendering the conventional multiple view geometry based on the single viewpoint (SVP) camera model invalid. While most recent underwater vision studies mainly focus on the challenging topic of calibrating such systems, no previous work has systematically studied the influence of refraction on underwater three-dimensional (3D) reconstruction. This paper demonstrates the possibility of using the SVP camera model in underwater 3D reconstruction through theoretical analysis of refractive distortion and simulations. Then, the performance of the SVP camera model in multiview underwater 3D reconstruction is quantitatively evaluated. The experimental results reveal a rather surprising and useful yet overlooked fact that the SVP camera model with radial distortion correction and focal length adjustment can compensate for refraction and achieve high accuracy in multiview underwater 3D reconstruction (within 0.7 mm for an object of dimension 200 mm) compared with the results of land-based systems. Such an observation justifies the use of the SVP camera model in underwater application for reconstructing reliable 3D scenes. Our results can be used to guide the selection of system parameters in the design of an underwater 3D imaging setup.

In this paper, a new genetic-based stereo algorithm is presented. Our motivation is to improve the accuracy of the disparity map by removing the mismatches caused by both occlusions and false targets. In our approach, the stereo matching problem is considered as an optimization problem. The algorithm first takes advantage of multi-view stereo images to detect occlusions, and therefore, removes mismatches caused by visibility problems. By optimizing the compatibility between corresponding points and the continuity of the disparity map using a genetic algorithm, mismatches caused by false targets are removed. The quadtree structure is used to implement the multi-resolution framework. Since nodes at different level of the quadtree cover different number of pixels, selecting nodes at different levels gives a similar effect as adjusting the window size at different locations of the image. The experimental results show that our approach can generate more accurate disparity maps than two existing approaches. In addition, we introduce a new disparity map evaluation technique, which is developed based on a similar technique employed in the image segmentation area. Comparing with two existing evaluation approaches, the new technique can evaluate the disparity maps generated without additional knowledge of the scene, such as the correct depth information or novel views.

Procedural texturing has been an active research area in computer graphics with some open problems still unsolved (D. S. Ebert, F. K. Musgrave, K. P. Peachey, K. Perlin, and S. Worley, 1998, “Texturing and Modeling: A Procedural Approach,” Academic Press, San Diego). One major problem is on how to estimate or recover the parameter values for a given procedural texture using the input texture image if the original parameter values are not available. In this paper, we propose a solution to this problem and present a genetic-based multiresolution parameter estimation approach. The key idea of our approach is to use an efficient search method (a genetic-based search algorithm is used in this paper) to find appropriate values of the parameters for the given procedure. During the search process, for each set of parameter values, we generate a temporary texture image using the given texturing procedure; then we compare the temporary texture image with the given target texture image to check if they match. The comparison between two texture images is done by using a multiresolution MRF texture model. The search process stops when a match is found. The estimated values of the parameters for a given procedure are the values of the parameters to the procedure to generate a texture image that matches the target texture image. Experimental results are presented to demonstrate the success of our approach. Application of our parameter estimation approach to texture synthesis is also discussed in the paper. [Copyright &y& Elsevier]

Image-morphing techniques are often employed to generate fluid transitions between two reference images. Many algorithms have been proposed in this area. Existing warp functions used in morphing algorithms are one-to-one. Consequently, when some features are visible in only one of the reference images, “ghosting” problems will appear in some in-between images. In addition, since the control features have global effects, there is no way to control different objects in the scene separately. In this paper, two new morphing approaches are introduced. In the first approach, called region-based morphing, regions are specified and morphed separately. In the second approach, called layer-based morphing, regions are defined in different layers. Hidden information is recovered and is used to solve the visibility problems.

Explosions are among the most common special effects employed in the entertainment industry. Producing real explosions, however, is costly and potentially dangerous and special effects technicians have been turning to software to generate explosion sequences. Software tools typically allow artists to re-create the appearance of an explosion but ignore physical processes. More accurate techniques, used in scientific research, produce physically correct results but demand substantial computational resources and do not provide support for artistic expression. This paper discusses an explosion animation system, X-Sim, that incorporates physical principles relevant to the generation of an explosion in a compressible medium. Diverse explosion effects can be created automatically while still allowing the animator to adjust the processes involved. Computational issues concerning X-Sim are addressed and a multiresolution scheme is introduced that focuses computation on turbulent regions of the environment. Results generated with X-Sim, evaluated qualitatively and quantitatively, indicate that the system can generate explosion phenomena with reasonable accuracy. Animations produced with this system also point to its flexibility and utility.

Continuum mechanics and the finite element method are used as basic tools to facilitate the animation of deformable objects. Two unique features in this paper are: (1) introduction of a framework for 3-D finite interface elements to deal with object interaction and (2) a simple approach to handle fracture propagation inside objects. Animation sequences have been constructed, and they indicate that the system is able to provide a realistic depiction of dynamic collision processes between deformable objects and a simple presentation of fracture propagation inside objects.

In this paper we present an algorithm that integrates spatial and temporal information for the tracking of moving nonrigid objects. In addition, we obtain outlines of the moving objects.

Humans appear to have the ability to infer the three-dimensional shape of an object from Rs outline or occluding contour. In computer vision, an occluding contour is an important clue in the shape recovery process. So far, only a small family of shapes, in particular, shapes generated by generalized cylinders, and shapes of polyhedrons, has been investigated in computer vision. However, there are many instances where these types of surfaces are not applicable, e.g., a solid with holes or a solid with no planar surfaces. This paper proposes a new concept called the default shape theory which includes the solid of revolution as a special case. A default shape is a function whose domain is a closed contour and whose range is a vector representation of a three-dimensional solid. The application of this new concept to recover the direction of the light source is demonstrated in this paper. Experimental results show that the default-shape-based method has comparable performance as the Pentland light source determination algorithm. This paper also proves the relationship between one particular type of default shape with the solid of revolution. Other possible definitions of default shape are also discussed. Copyright 1994, 1999 Academic Press

Most of the previously proposed image-based rendering approaches rely on a large number of samples, accurate depth information, or 3D geometric proxies. It is, therefore, a challenge to apply them to render novel views for dynamic scenes as the required information is difficult to obtain in real-time. The proposed approach requires only sparely sampled data. Two new interpolation techniques are presented in this paper, the combination of which can produce reasonable rendering results for sparsely sampled real scenes. The first one, which is a color-matching based interpolation, searches for a possible physical point along the testing ray using color information in nearby reference images. The second technique, which is a disparity-matching based interpolation, tries to find the closest intersection between the testing ray and the disparity surfaces defined by nearby reference images. Both approaches are designed as backward rendering techniques and can be combined to produce robust results. Our experiments suggest that the proposed approach is capable of handling complex dynamic real scenes offline. [Copyright &y& Elsevier]