Search

Many classical Computer Vision problems, such as essential matrix computation and pose estimation from 3D to 2D correspondences, can be tackled by solving a linear least-square problem, which can be done by finding the eigenvector corresponding to the smallest, or zero, eigenvalue of a matrix representing a linear system. Incorporating this in deep learning frameworks would allow us to explicitly encode known notions of geometry, instead of having the network implicitly learn them from data. However, performing eigendecomposition within a network requires the ability to differentiate this operation. While theoretically doable, this introduces numerical instability in the optimization process in practice. In this paper, we introduce an eigendecomposition-free approach to training a deep network whose loss depends on the eigenvector corresponding to a zero eigenvalue of a matrix predicted by the network. We demonstrate that our approach is much more robust than explicit differentiation of the eigendecomposition using two general tasks, outlier rejection and denoising, with several practical examples including wide-baseline stereo, the perspective-n-point problem, and ellipse fitting. Empirically, our method has better convergence properties and yields state-of-the-art results. [ABSTRACT FROM AUTHOR]

Seismic source locations are fundamental to many fields of Earth and planetary sciences, such as seismology, volcanology and tectonics. However, seismic source detection and location are challenging when events cluster closely in space and time with signals tangling together at observing stations, such as they often do in major aftershock sequences. Though emerging algorithms and artificial intelligence (AI) models have made processing high volumes of seismic data easier, their performance is still limited, especially for complex aftershock sequences. In this study, we propose a novel approach that utilizes three‐dimensional image segmentation—a computer vision technique—to detect and locate seismic sources, and develop this into a complete workflow, Source Untangler Guided by Artificial intelligence image Recognition (SUGAR). In our synthetic and real data tests, SUGAR can handle complex, energetic earthquake sequences in near real time better than skillful analysts and other AI and non‐AI based algorithms. We apply SUGAR to the 2016 Kaikōura, New Zealand sequence and obtain five times more events than the analyst‐based GeoNet catalog. The improved aftershock distribution illuminates a continuous fault system with extensive fracture zones beneath the segmented, discontinuous surface ruptures. Our method has broader applicability to non‐earthquake sources and other time series image data sets. Plain Language Summary: Detecting and locating earthquakes is fundamental to seismology, volcanology, and tectonics. A number of emerging algorithms, including some based upon artificial intelligence (AI), have made processing large volumes of seismic data much easier. However, their performance is still limited, especially in clustered aftershock sequences whose signals overlap at observing seismographs. We propose a new, AI computer vision‐based approach to this problem, and develop it into a complete earthquake detection and location workflow, named SUGAR. Tests on synthetic and real earthquake data sets show that SUGAR characterizes complex earthquake sequences better than other AI and non‐AI algorithms or professional analysts. We apply SUGAR to the complex aftershock sequence of the 2016 Mw 7.8 Kaikōura, New Zealand earthquake, detecting five times more events than the analyst‐based GeoNet catalog. Whereas surface breaks of the Kaikōura earthquake are highly discontinuous, our improved aftershock distribution supports a continuous fault system surrounded by extensive fracture zones at depth. Our method has broader potential for other types of seismic sources and image series. Key Points: We propose a new seismic source detection and location approach based on the source‐scanning algorithm and 3D image segmentationThis approach outperforms human analysts and popular artificial intelligence (AI) and non‐AI based methods in characterizing intense aftershock sequencesThe resulting catalog of the 2016 Kaikōura earthquake sequence suggests a continuous fault system surrounded by extensive fracturing [ABSTRACT FROM AUTHOR]

Purpose: When investigating new radiation therapy techniques in the treatment planning stage, it can be extremely time consuming to locate multiple patient scans that match the desired characteristics for the treatment. With the help of machine learning, we propose to bypass the difficulty in finding patient computed tomography (CT) scans that match the treatment requirements. Furthermore, we aim to provide the developed method as a tool that is easily accessible to interested researchers. Methods: We propose a generative adversarial network (GAN) to edit individual volumes of interest (VOIs) in pre‐existing CT scans, translating features of the healthy VOIs into features of cancerous volumes. Training and testing was done using VOIs from a dataset of 460 diagnostic and lung cancer screening CT scans. Agreement between real tumors and those produced by the editor was tested by comparing the distributions of several histogram parameters and second‐order statistics as well as using qualitative analysis. Results: After training, the network was successfully able to map healthy CT segments to realistic looking cancerous volumes. Based on visual inspection, tumors produced by the editor were found to be both realistic and visually consistent with the surrounding anatomy when placed back into the original CT scan. Furthermore, the network was found to be able to extrapolate well beyond the upper size limit of the training set. Lastly, a graphical user interface (GUI) was developed to easily interact with the resulting network. Conclusion: The trained network and associated GUI can serve as a tool to develop an abundance of lung cancer patient data to be used in treatment planning. In addition, this method can be extended to a variety of cancer types if given an appropriate baseline dataset. The GUI and instructions on how to utilize the tool have been made publicly available at https://github.com/teaghan/CT_Editor. [ABSTRACT FROM AUTHOR]

In this paper, a tracking method based on sequential Bayesian inference is proposed. The proposed method focuses on solving both the problem of tracking under partial occlusions and the problem of non-rigid object tracking in real-time on a desktop personal computer (PC). The proposed method is mainly composed of two parts: (1) modeling the target object using elastic structure of local patches for robust performance; and (2) efficient hierarchical diffusion method to perform the tracking procedure in real-time. The elastic structure of local patches allows the proposed method to handle partial occlusions and non-rigid deformations through the relationship among neighboring patches. The proposed hierarchical diffusion method generates samples from the region where the posterior is concentrated to reduce computation time. The method is extensively tested on a number of challenging image sequences with occlusion and non-rigid deformation. The experimental results show the real-time capability and the robustness of the proposed method under various situations. [ABSTRACT FROM AUTHOR]

A relatively new concern for the forthcoming massive spectroscopic sky surveys is the impact of contamination from low earth orbit satellites. Several hundred thousand of these satellites are licensed for launch in the next few years and it has been estimated that, in some cases, up to a few per cent of spectra could be contaminated when using wide field, multifibre spectrographs. In this paper, a multistaged approach is used to assess the practicality and limitations of identifying and minimizing the impact of satellite contamination in a WEAVE-like stellar spectral survey. We develop a series of convolutional-network-based architectures to attempt identification, stellar parameter and chemical abundances recovery, and source separation of stellar spectra that we artificially contaminate with satellite (i.e. solar-like) spectra. Our results show that we are able to flag 67 per cent of all contaminated sources at a precision level of 80 per cent for low-resolution spectra and 96 per cent for high-resolution spectra. Additionally, we are able to remove the contamination from the spectra and recover the clean spectra with a <1 per cent reconstruction error. The errors in stellar parameter predictions reduce by up to a factor of 2–3 when either including contamination as an augmentation to a training set or by removing the contamination from the spectra, with overall better performance in the former case. The presented methods illustrate several machine learning mitigation strategies that can be implemented to improve stellar parameters for contaminated spectra in the WEAVE stellar spectroscopic survey and others like it. [ABSTRACT FROM AUTHOR]

Synchrotron X-rays can be used to obtain highly detailed images of parts of the lung. However, micro-motion artifacts induced by such as cardiac motion impede quantitative visualization of the alveoli in the lungs. This paper proposes a method that applies a neural network for synchrotron X-ray Computed Tomography (CT) data to reconstruct the high-quality 3D structure of alveoli in intact mouse lungs at expiration, without needing ground-truth data. Our approach reconstructs the spatial sequence of CT images by using a deep-image prior with interpolated input latent variables, and in this way significantly enhances the images of alveolar structure compared with the prior art. The approach successfully visualizes 3D alveolar units of intact mouse lungs at expiration and enables us to measure the diameter of the alveoli. We believe that our approach helps to accurately visualize other living organs hampered by micro-motion. [ABSTRACT FROM AUTHOR]

• The core research question of this paper is how can we find the image part described by a user, such that the output image crop will represent and preserve the caption information meanwhile result in an aesthetically pleasing output? • We have proposed a caption and aesthetics guided framework for cropping images according to the user's intention. Our framework is the first to account for the user's intention directly from the provided image caption. • We argue that the currently available image cropping and caption grounding datasets are not suitable for our description-based image cropping task. Therefore, we proposed a novel dataset with multiple ground truth bounding box annotations for each caption. • The experiments in Section 4.2 show that we can achieve better performance than the baseline methods for caption-based image cropping by re-proposing existing deep networks. We propose a novel optimization framework that crops a given image based on user description and aesthetics. Unlike existing image cropping methods, where one typically trains a deep network to regress to crop parameters or cropping actions, we propose to directly optimize for the cropping parameters by repurposing pre-trained networks on image captioning and aesthetic tasks, without any fine-tuning, thereby avoiding training a separate network. Specifically, we search for the best crop parameters that minimize a combined loss of the initial objectives of these networks. To make the optimization stable, we propose three strategies: (i) multi-scale bilinear sampling, (ii) annealing the scale of the crop region, therefore effectively reducing the parameter space, (iii) aggregation of multiple optimization results. Through various quantitative and qualitative evaluations, we show that our framework can produce crops that are well-aligned to intended user descriptions and aesthetically pleasing. [ABSTRACT FROM AUTHOR]

The Canada–France Imaging Survey (CFIS) will consist of deep, high-resolution r -band imaging over ∼5000 deg2 of the sky, representing a first-rate opportunity to identify recently merged galaxies. Because of the large number of galaxies in CFIS, we investigate the use of a convolutional neural network (CNN) for automated merger classification. Training samples of post-merger and isolated galaxy images are generated from the IllustrisTNG simulation processed with the observational realism code RealSim. The CNN's overall classification accuracy is 88 per cent, remaining stable over a wide range of intrinsic and environmental parameters. We generate a mock galaxy survey from IllustrisTNG in order to explore the expected purity of post-merger samples identified by the CNN. Despite the CNN's good performance in training, the intrinsic rarity of post-mergers leads to a sample that is only ∼6 per cent pure when the default decision threshold is used. We investigate trade-offs in purity and completeness with a variable decision threshold and find that we recover the statistical distribution of merger-induced star formation rate enhancements. Finally, the performance of the CNN is compared with both traditional automated methods and human classifiers. The CNN is shown to outperform Gini–M20 and asymmetry methods by an order of magnitude in post-merger sample purity on the mock survey data. Although the CNN outperforms the human classifiers on sample completeness, the purity of the post-merger sample identified by humans is frequently higher, indicating that a hybrid approach to classifications may be an effective solution to merger classifications in large surveys. [ABSTRACT FROM AUTHOR]

We introduce a comprehensive benchmark for local features and robust estimation algorithms, focusing on the downstream task—the accuracy of the reconstructed camera pose—as our primary metric. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods and heuristics. This is demonstrated by embedding dozens of popular algorithms and evaluating them, from seminal works to the cutting edge of machine learning research. We show that with proper settings, classical solutions may still outperform the perceived state of the art. Besides establishing the actual state of the art, the conducted experiments reveal unexpected properties of structure from motion pipelines that can help improve their performance, for both algorithmic and learned methods. Data and code are online (https://github.com/ubc-vision/image-matching-benchmark), providing an easy-to-use and flexible framework for the benchmarking of local features and robust estimation methods, both alongside and against top-performing methods. This work provides a basis for the Image Matching Challenge (https://image-matching-challenge.github.io). [ABSTRACT FROM AUTHOR]

In the current era of stellar spectroscopic surveys, synthetic spectral libraries are the basis for the derivation of stellar parameters and chemical abundances. In this paper, we compare the stellar parameters determined using five popular synthetic spectral grids (INTRIGOSS, FERRE, AMBRE, PHOENIX, and MPIA/1DNLTE) with our convolutional neural network (CNN, StarNet). The stellar parameters are determined for six physical properties (effective temperature, surface gravity, metallicity, [ α /Fe], radial velocity, and rotational velocity) given the spectral resolution, signal-to-noise ratio, and wavelength range of optical FLAMES-UVES spectra from the Gaia-ESO Survey. Both CNN modelling and epistemic uncertainties are incorporated through training an ensemble of networks. StarNet training was also adapted to mitigate differences between the synthetic grids and observed spectra by augmenting with realistic observational signatures (i.e. resolution matching, wavelength sampling, Gaussian noise, zeroing flux values, rotational and radial velocities, continuum removal, and masking telluric regions). Using the FLAMES-UVES spectra for FGK-type dwarfs and giants as a test set, we quantify the accuracy and precision of the stellar label predictions from StarNet. We find excellent results over a wide range of parameters when StarNet is trained on the MPIA/1DNLTE synthetic grid, and acceptable results over smaller parameter ranges when trained on the 1DLTE grids. These tests also show that our CNN pipeline is highly adaptable to multiple simulation grids. [ABSTRACT FROM AUTHOR]

We present an algorithm for estimating the pose of a rigid object in real-time under challenging conditions. Our method effectively handles poorly textured objects in cluttered, changing environments, even when their appearance is corrupted by large occlusions, and it relies on grayscale images to handle metallic environments on which depth cameras would fail. As a result, our method is suitable for practical Augmented Reality applications including industrial environments. At the core of our approach is a novel representation for the 3D pose of object parts: We predict the 3D pose of each part in the form of the 2D projections of a few control points. The advantages of this representation is three-fold: We can predict the 3D pose of the object even when only one part is visible; when several parts are visible, we can easily combine them to compute a better pose of the object; the 3D pose we obtain is usually very accurate, even when only few parts are visible. We show how to use this representation in a robust 3D tracking framework. In addition to extensive comparisons with the state-of-the-art, we demonstrate our method on a practical Augmented Reality application for maintenance assistance in the ATLAS particle detector at CERN. [ABSTRACT FROM PUBLISHER]