Search

Backlit images are usually taken when the light source is opposite to the camera. The uneven exposure (e.g., underexposure on the foreground and overexposure on the background) makes the backlit images more challenging than general image enhancement tasks that only need to increase or decrease the exposure on the whole images. Compared to traditional approaches, Convolutional Neural Networks perform well in enhancing images due to the abilities of exploiting contextual features. However, the lack of large benchmark datasets and specially designed models impedes the development of backlit image enhancement. In this paper, we build the first large-scale BAcklit Image Dataset (BAID), which contains 3000 backlit images and the corresponding ground truth manually adjusted by trained photographers. It covers a broad range of categories under different backlit conditions in both indoor and outdoor scenes. Furthermore, we propose a saliency guided backlit image enhancement network, namely BacklitNet, for robust and natural restoration of backlit images. In particular, our model innovatively combines a nested U-structure with bilateral grids, which enables fully extracting multi-scale saliency information and rapidly enhancing arbitrary resolution images. Moreover, a carefully designed loss function based on prior knowledge of brightness distribution of backlit images is proposed to enforce the network to focus more on backlit regions during the training phase. We evaluate the proposed method on the BAID dataset and two public small-scale backlit image datasets. Experimental results demonstrate that our method performs favorably against the state-of-the-art approaches. • We build the first large-scale backlit image dataset (BAID) which contains 3000 backlit images and the corresponding high-quality ground truth. The constructed dataset makes end-to-end learning of robust backlit enhancers possible and promotes the application of neural networks in backlit image enhancement. • We design a novel backlit image enhancement framework, which efficiently restores ill-exposed regions of backlit images based on subject salience and brightness distribution prior knowledge without damaging the well-exposed regions. • We evaluate the proposed method on the BAID dataset and two public small-scale backlit image datasets. The results show that our model achieves state-of-the-art performance both on visual effects and commonly-used metrics. [ABSTRACT FROM AUTHOR]

Correlation filters are playing an important role in state-of-the-art visual tracking. To achieve superior speed and great discriminative power, correlation filter based trackers are equipped with circulant matrix and fast Fourier transform (FFT), which implicitly use a large amount of samples to efficiently train the correlation filters and contribute significantly to model complexity. However, the number of training samples is significantly reduced when a target's size is small. Consequently, the performance of the resulting correlation filter based trackers may be inhibited. Moreover, how to cope with the scale variations and the tracker drift is still an open problem. In this paper, to address the above problems, we propose a coarse-to-fine tracker which integrates a kernelized correlation filter (KCF) based tracker with detection proposal and a multi-expert based tracker via a simple yet effective Peak to Sidelobe Ratio (PSR) and scale driven schema. Specifically, in the coarse level, the KCF based tracker with detection proposal is used to estimate a target's state in each frame. Then, the PSR and scale variations are analyzed according to the tracking results obtained by the simple KCF based tracker with detection proposal. In the fine level, the complicated multi-expert based tracker will be started for tracking when PSR or scale is lower than a threshold. Based on the simple yet effective PSR and scale driven expert-switching scheme, the proposed coarse-to-fine tracker can select the most reliable tracker in the tracking process. Extensive experimental results on the OTB-50 benchmark demonstrate the efficiency and effectiveness of the proposed tracking method. [ABSTRACT FROM AUTHOR]

One of the key limitations of the many existing visual tracking method is that they are built upon low-level visual features and have limited predictability power of data semantics. To effectively fill the semantic gap of visual data in visual tracking with little supervision, we propose a tracking method which constructs a robust object appearance model via learning and transferring mid-level image representations using a deep network, i.e., Network in Network (NIN). First, we design a simple yet effective method to transfer the mid-level features learned from NIN on the source tasks with large scale training data to the tracking tasks with limited training data. Then, to address the drifting problem, we simultaneously utilize the samples collected in the initial and most previous frames. Finally, a heuristic schema is used to judge whether updating the object appearance model or not. Extensive experiments show the robustness of our method. [ABSTRACT FROM AUTHOR]

Current transformer-based trackers typically represent targets using bounding boxes. However, bounding boxes do not accurately describe the target and uncontrollably contain most background pixels. This paper proposes a Segment Anything Model (SAM)-Assisted Temporal-Location Enhanced Transformer Segmentation for Object Tracking with Online Motion Inference. First, a novel transformer-based temporal-location enhanced segmentation method is proposed. The target temporal features are clustered into foreground–background tokens utilizing a mask to capture discriminative information distribution. Then, the suitable positional prompts are learned in the proposed mask prediction head to establish the mapping between target features and localization, which enhances the specific foreground weights for precise mask generation. Second, a temporal-based motion inference module is proposed. It fully utilizes the target temporal state to construct an online displacement model inferring motion relationships of the target between frames and providing robust position prompts for the segmentation process. We also introduce SAM for initial mask generation. Precise pixel-level object tracking is achieved by combining segmentation and localization within a unified process. Experimental results demonstrate that the proposed method yields competitive performance compared to existing approaches. [ABSTRACT FROM AUTHOR]

Video dehazing is of crucial importance to the outdoor surveillance, and its main challenges mainly come from the spatial-temporal coherence and computational efficiency. This paper presents an efficient real-time video dehazing approach via incremental transmission learning and spatial-temporally coherent regularization, while explicitly suppressing the possible visual artifacts. First, we propose to initialize the transmission map frame-by-frame by a boundary constrained open dark channel model. Then, supposing a scene point yields highly correlated transmission values between adjacent frames, we impose a temporally coherent term to maintain the temporal consistency of consecutive transmission values, and simultaneously derive an incremental transmission adjusting term to adapt the abrupt scene depth changes between the adjacent frames. Accordingly, the highly correlated and spatial-temporally coherent transmission maps can be optimized for each video frame dehazing, whereby the flickering artifacts can be well reduced. Finally, we group each haze/haze-free images in pair and further utilize the guided joint bilateral filter to suppress the visual artifacts that possibly amplified during the dehazing process. The experimental results show that the proposed video dehazing approach is able to well preserve the spatial-temporal coherence, runs sufficiently fast, and also performs favorably compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

To adaptively identify the transient modal parameters for linear weakly damped structures with slow time-varying characteristics under unmeasured stationary random ambient loads, this paper proposes a novel operational modal analysis (OMA) method based on the frozen-in coefficient method and limited memory recursive principal component analysis (LMRPCA). In the modal coordinate, the random vibration response signals of mechanical weakly damped structures can be decomposed into the inner product of modal shapes and modal responses, from which the natural frequencies and damping ratios can be well acquired by single-degree-of-freedom (SDOF) identification approach such as FFT. Hence, for the OMA method based on principal component analysis (PCA), it becomes very crucial to examine the relation between the transformational matrix and the modal shapes matrix, to find the association between the principal components (PCs) matrix and the modal responses matrix, and to turn the operational modal parameter identification problem into PCA of the stationary random vibration response signals of weakly damped mechanical structures. Based on the theory of “time-freezing”, the method of frozen-in coefficient, and the assumption of “short time invariant” and “quasistationary”, the non-stationary random response signals of the weakly damped and slow linear time-varying structures (LTV) can approximately be seen as the stationary random response time series of weakly damped and linear time invariant structures (LTI) in a short interval. Thus, the adaptive identification of time-varying operational modal parameters is turned into decompositing the PCs of stationary random vibration response signals subsection of weakly damped mechanical structures after choosing an appropriate limited memory window. Finally, a three-degree-of-freedom (DOF) structure with weakly damped and slow time-varying mass is presented to illustrate this method of identification. Results show that the LMRPCA algorithm, which is robustness to Gauss measurement noise disturbances, can well identify the transient modal parameters (transient natural frequencies & transient modal shapes) for weakly damped and slow LTV structures online only from non-stationary random response signals. [ABSTRACT FROM AUTHOR]

Object tracking has been typically formulated as an online learning problem, in which a target appearance is updated adaptively using the single-shot images tracked from the previous 2D frames. The traditional methods have the following limitations: (1) useful depth information is ignored; (2) each training and testing example is a single image, so it can be easily corrupted. In this paper, we propose a novel 3D object tracking method using image sets and depth-based occlusion detection, in which each training and testing example contains a set of image instances of an object and covers large variations in the object׳s appearance. To do so, we first represent each image set as its natural second-order statistic. Then, we use kernel partial least squares to adaptively learn low-dimensional discriminative feature subspace for object representation. Third, to prevent improper appearance model updating during occlusions, we exploit depth information obtained from binocular video data to detect occlusion. Finally, to alleviate the tracker drifting problem caused by model update, we exploit both the ground truth appearance information of the object labeled in the initial frames and the image observations obtained online. Extensive experiments on challenging video sequences demonstrate the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Vision transformer (ViT) has become a leading tool in various computer vision tasks, owing to its unique self-attention mechanism that learns visual representations explicitly through cross-patch information interactions. Despite having good success, the literature seldom explores the explainability of ViT, and there is no clear picture of how the attention mechanism with respect to the correlation across comprehensive patches will impact the performance and what is the further potential. In this work, we propose a novel explainable visualization approach to analyze and interpret the crucial attention interactions among patches for ViT. Specifically, we first introduce a quantification indicator to measure the impact of patch interaction and verify such quantification on attention window design and indiscriminative patches removal. Then, we exploit the effective responsive field of each patch in ViT and devise a window-free transformer (WinfT) architecture accordingly. Extensive experiments on ImageNet demonstrate that the exquisitely designed quantitative method is shown able to facilitate ViT model learning, leading the top-1 accuracy by 4.28% at most. More remarkably, the results on downstream fine-grained recognition tasks further validate the generalization of our proposal.

A class-agnostic tracker typically consists of three key components, i.e., its motion model, its target appearance model, and its updating strategy. However, most recent topperforming trackers mainly focus on constructing complicated appearance models and updating strategies, while using comparatively simple and heuristic motion models that may result in an inefficient search and degrade the tracking performance. To address this issue, we propose a hierarchical tracker that learns to move and track based on the combination of data-driven search at the coarse level, and coarse-to-fine verification at the fine level. At the coarse level, a data-driven motion model learned from deep recurrent reinforcement learning provides our tracker with coarse localization of an object. By formulating motion search as an action-decision problem in reinforcement learning, our tracker utilizes a recurrent convolutional neural network based deep Q-network to effectively learn data-driven searching policies. The learned motion model cannot only significantly reduce the search space, but also provide more reliable interested regions for further verifying. At the fine level, a kernelized correlation filter (KCF) based appearance model is adopted to densely yet efficiently verify a local region centered on the predicted location from the motion model. Through using of circulant matrices and fast Fourier transformation, a large number of candidate samples in the local region can be efficiently and effectively evaluated by the KCF based appearance model. Finally, a simple yet robust estimator is designed to analyze possible tracking failure. The experiments on OTB50 and OTB100 illustrate that our tracker achieves better performance than the state-of-the-art trackers.