Search

This paper reviews the challenge on constrained high dynamic range (HDR) imaging that was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop, held in conjunction with CVPR 2022. This manuscript focuses on the competition set-up, datasets, the proposed methods and their results. The challenge aims at estimating an HDR image from multiple respective low dynamic range (LDR) observations, which might suffer from under- or over-exposed regions and different sources of noise. The challenge is composed of two tracks with an emphasis on fidelity and complexity constraints: In Track 1, participants are asked to optimize objective fidelity scores while imposing a low-complexity constraint (i.e. solutions can not exceed a given number of operations). In Track 2, participants are asked to minimize the complexity of their solutions while imposing a constraint on fidelity scores (i.e. solutions are required to obtain a higher fidelity score than the prescribed baseline). Both tracks use the same data and metrics: Fidelity is measured by means of PSNR with respect to a ground-truth HDR image (computed both directly and with a canonical tonemapping operation), while complexity metrics include the number of Multiply-Accumulate (MAC) operations and runtime (in seconds)., Comment: CVPR Workshops 2022. 15 pages, 21 figures, 2 tables

Self-supervised learning (SlfSL), aiming at learning feature representations through ingeniously designed pretext tasks without human annotation, has achieved compelling progress in the past few years. Very recently, SlfSL has also been identified as a promising solution for semi-supervised learning (SemSL) since it offers a new paradigm to utilize unlabeled data. This work further explores this direction by proposing to couple SlfSL with SemSL. Our insight is that the prediction target in SemSL can be modeled as the latent factor in the predictor for the SlfSL target. Marginalizing over the latent factor naturally derives a new formulation which marries the prediction targets of these two learning processes. By implementing this idea through a simple-but-effective SlfSL approach -- rotation angle prediction, we create a new SemSL approach called Conditional Rotation Angle Estimation (CRAE). Specifically, CRAE is featured by adopting a module which predicts the image rotation angle conditioned on the candidate image class. Through experimental evaluation, we show that CRAE achieves superior performance over the other existing ways of combining SlfSL and SemSL. To further boost CRAE, we propose two extensions to strengthen the coupling between SemSL target and SlfSL target in basic CRAE. We show that this leads to an improved CRAE method which can achieve the state-of-the-art SemSL performance.

In this paper, we propose a hybrid Structure-from-Motion scheme which combines the strength of both global and local incremental SfM methods to get a drift-free and accurate estimation with lower time consumption. More specifically, we propose to construct a robust maximum leaf spanning tree (RMLST) from the initial scene graph and further expand it to a robust graph (RG) to grasp the global picture of camera distribution and scene structure. Then the views in the robust graph are solved in global manner as an initial estimation. After that, the remaining views are estimated with the proposed community-based local incremental approach to guarantee local accuracy and scalability. Bundle adjustment is conducted to optimize the estimation. Experiments show that our method is robust and free from the scene drift as global SfM, and shows much better efficiency than incremental approaches. Besides, our algorithm achieves higher accuracy compared with the state-of-the-art methods.

Ghosting artifacts caused by moving objects or misalignments is a key challenge in high dynamic range (HDR) imaging for dynamic scenes. Previous methods first register the input low dynamic range (LDR) images using optical flow before merging them, which are error-prone and cause ghosts in results. A very recent work tries to bypass optical flows via a deep network with skip-connections, however, which still suffers from ghosting artifacts for severe movement. To avoid the ghosting from the source, we propose a novel attention-guided end-to-end deep neural network (AHDRNet) to produce high-quality ghost-free HDR images. Unlike previous methods directly stacking the LDR images or features for merging, we use attention modules to guide the merging according to the reference image. The attention modules automatically suppress undesired components caused by misalignments and saturation and enhance desirable fine details in the non-reference images. In addition to the attention model, we use dilated residual dense block (DRDB) to make full use of the hierarchical features and increase the receptive field for hallucinating the missing details. The proposed AHDRNet is a non-flow-based method, which can also avoid the artifacts generated by optical-flow estimation error. Experiments on different datasets show that the proposed AHDRNet can achieve state-of-the-art quantitative and qualitative results., Comment: Accepted to appear at CVPR 2019

Generating a high dynamic range (HDR) image from a set of sequential exposures is a challenging task for dynamic scenes. The most common approaches are aligning the input images to a reference image before merging them into an HDR image, but artifacts often appear in cases of large scene motion. The state-of-the-art method using deep learning can solve this problem effectively. In this paper, we propose a novel deep convolutional neural network to generate HDR, which attempts to produce more vivid images. The key idea of our method is using the coarse-to-fine scheme to gradually reconstruct the HDR image with the multi-scale architecture and residual network. By learning the relative changes of inputs and ground truth, our method can produce not only artificial free image but also restore missing information. Furthermore, we compare to existing methods for HDR reconstruction, and show high-quality results from a set of low dynamic range (LDR) images. We evaluate the results in qualitative and quantitative experiments, our method consistently produces excellent results than existing state-of-the-art approaches in challenging scenes.

The challenge in blind image deblurring is to remove the effects of blur with limited prior information about the nature of the blur process. Existing methods often assume that the blur image is produced by linear convolution with additive Gaussian noise. However, including even a small number of outliers to this model in the kernel estimation process can significantly reduce the resulting image quality. Previous methods mainly rely on some simple but unreliable heuristics to identify outliers for kernel estimation. Rather than attempt to identify outliers to the model a priori, we instead propose to sequentially identify inliers, and gradually incorporate them into the estimation process. The selfpaced kernel estimation scheme we propose represents a generalization of existing self-paced learning approaches, in which we gradually detect and include reliable inlier pixel sets in a blurred image for kernel estimation. Moreover, we automatically activate a subset of significant gradients w.r.t. the reliable inlier pixels, and then update the intermediate sharp image and the kernel accordingly. Experiments on both synthetic data and real-world images with various kinds of outliers demonstrate the effectiveness and robustness of the proposed method compared to the stateof- the-art methods.

Removing pixel-wise heterogeneous motion blur is challenging due to the ill-posed nature of the problem. The predominant solution is to estimate the blur kernel by adding a prior, but extensive literature on the subject indicates the difficulty in identifying a prior which is suitably informative, and general. Rather than imposing a prior based on theory, we propose instead to learn one from the data. Learning a prior over the latent image would require modeling all possible image content. The critical observation underpinning our approach, however, is that learning the motion flow instead allows the model to focus on the cause of the blur, irrespective of the image content. This is a much easier learning task, but it also avoids the iterative process through which latent image priors are typically applied. Our approach directly estimates the motion flow from the blurred image through a fully-convolutional deep neural network (FCN) and recovers the unblurred image from the estimated motion flow. Our FCN is the first universal end-to-end mapping from the blurred image to the dense motion flow. To train the FCN, we simulate motion flows to generate synthetic blurred-image-motion-flow pairs thus avoiding the need for human labeling. Extensive experiments on challenging realistic blurred images demonstrate that the proposed method outperforms the state-of-the-art.

Blind image deconvolution is an ill-posed inverse problem which is often addressed through the application of appropriate prior. Although some priors are informative in general, many images do not strictly conform to this, leading to degraded performance in the kernel estimation. More critically, real images may be contaminated by nonuniform noise such as saturation and outliers. Methods for removing specific image areas based on some priors have been proposed, but they operate either manually or by defining fixed criteria. We show here that a subset of the image gradients are adequate to estimate the blur kernel robustly, no matter the gradient image is sparse or not. We thus introduce a gradient activation method to automatically select a subset of gradients of the latent image in a cutting-plane-based optimization scheme for kernel estimation. No extra assumption is used in our model, which greatly improves the accuracy and flexibility. More importantly, the proposed method affords great convenience for handling noise and outliers. Experiments on both synthetic data and real-world images demonstrate the effectiveness and robustness of the proposed method in comparison with the state-of-the-art methods.

Motion blurred images are widely existing when using a hand-held camera especially under the dim lighting conditions. Since edge information contained in the noisy image may be blurred by the motion blur, a blurred/noisy image pair captured under different exposure time can help to restore a sharp image. In the traditional deblurring methods based on blurred/noisy image pair, the deblurring process is in series with the denoising process, so that restoration result is sensitive to the denoised result. In this paper, we propose a robust algorithm to obtain the sharp image by fusing the blurred image and noisy image. By joint modeling the deblurring model and denoising model, the restoration result can be optimized via estimating the sharp image and blur kernel alternately in the proposed methods, and it is not sensitive to the denoised result benefited by the joint model. Experimental results demonstrated that the proposed method can achieve better performance compared with the state-of-the-art single image denoising methods, single image deblurring methods and blurred/noisy pair deblurring methods.

In this paper, we propose a novel neighbor combination framework for the reconstruction of the atmospheric turbulence degenerated image sequence. To utilize the spatial and temporal redundancy, a neighbor vector sampling strategy in spatial and temporal domain is conducted relying on the modeling of the registered sequence. Then, a combinator of neighbor vectors is developed based on a resampling maximum likelihood model and a relative approximation. Relying on the neighbor combination and spatial-invariant deconvolution, a clear image is reconstructed. Experiments on real data sets demonstrate the effectiveness of this framework.

Expatiating on the important functions of the joint operation model in military emulation, discussing the causations of military operations based on UML, bringing forward dynamic frame class and the rule of the modeling, taking the eliminating fraise action in the attack campaign as the example to illuminating.

In ultrasonic pulse-echo NDT of structures, the transducers usually operate in its harmonic frequencies in order to maximize the amplitude response. However, there exist some unwanted effects in such mode. For instance, the generated/detected signals will have long ringing, which means the time-domain resolution will be reduced. Presented here is a digital filtering method to compensate the unwanted frequency response characteristics of transducers. The first step is to establish a discrete transfer function model of transducer system based on time-domain system identification algorithms. Then the established model can be realized by a digital compensation filter to reduce the ringing effects of the measured signals. Experimental verification of the proposed method is carried out. After calibrating and modeling for transducers by water-immersion test, a compensation model is established and used to the thickness measurement of a concrete specimen. The compensation results show that the ring of transducer can be greatly reduced and the reflection from bottom of the specimen can be extracted from the original overlapped signals. To further validate the effect of the proposed method, its application in B-scan imaging of a concrete element with embedded anomaly is also given.