Search

Accurate target prediction, especially bounding box estimation, is a key problem in visual tracking. Many recently proposed trackers adopt the refinement module called IoU predictor by designing a high‐level modulation vector to achieve bounding box estimation. However, due to the lack of spatial information that is important for precise box estimation, this simple one‐dimensional modulation vector has limited refinement representation capability. In this study, a novel IoU predictor (IoUNet++) is designed to achieve more accurate bounding box estimation by investigating spatial matching with a spatial cross‐layer interaction model. Rather than using a one‐dimensional modulation vector to generate representations of the candidate bounding box for overlap prediction, this paper first extracts and fuses multi‐level features of the target to generate template kernel with spatial description capability. Then, when aggregating the features of the template and the search region, the depthwise separable convolution correlation is adopted to preserve the spatial matching between the target feature and candidate feature, which makes their IoUNet++ network have better template representation and better fusion than the original network. The proposed IoUNet++ method with a plug‐and‐play style is applied to a series of strengthened trackers including DiMP++, SuperDiMP++ and SuperDIMP_AR++, which achieve consistent performance gain. Finally, experiments conducted on six popular tracking benchmarks show that their trackers outperformed the state‐of‐the‐art trackers with significantly fewer training epochs. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new hyperspectral image (HSI) denoising model with the group sparsity regularized hybrid spatio-spectral total variation (GHSSTV) and low-rank tensor decomposition, which is based on the analysis of structural sparsity of HSIs. First, the global correlations among all modes are explored by the Tucker decomposition, which applies low-rank constraints to the clean HSIs. To avoid over-smoothing, we propose GHSSTV regularization to ensure the group sparsity not only in the first-order gradient domain but also in the second-order ones along the spatio-spectral dimensions. Then, the sparse noise in HSI can be detected by the ℓ 1 norm. Furthermore, strong Gaussian noise is simulated by the Frobenius norm. The alternating direction multiplier method (ADMM) algorithm is employed to effectively solve the GHSSTV model. Finally, experimental results from a series of simulations and real-world data suggest a superior performance of the GHSSTV method in HSIs denoising. [ABSTRACT FROM AUTHOR]

Hyperspectral images (HSIs) are usually corrupted by various noises, e.g., Gaussian noise, impulse noise, stripes, dead lines, and many others. In this article, motivated by the good performance of the L1−2 nonconvex metric in image sparse structure exploitation, we first develop a 3-D L1−2 spatial–spectral total variation (L1−2 SSTV) regularization to globally represent the sparse prior in the gradient domain of HSIs. Then, we divide HSIs into local overlapping 3-D patches, and low-rank tensor recovery (LTR) is locally used to effectively separate the low-rank clean HSI patches from complex noise. The patchwise LTR can not only adapt to the local low-rank property of HSIs well but also significantly reduce the information loss caused by the global LTR. Finally, integrating the advantages of both the global L1−2 SSTV regularization and local LTR model, we propose a L1−2 SSTV regularized local LTR model for hyperspectral restoration. In the framework of the alternating direction method of multipliers, the difference of convex algorithm, the split Bregman iteration method, and tensor singular value decomposition method are adopted to solve the proposed model efficiently. Simulated and real HSI experiments show that the proposed model can reduce the dependence on noise independent and identical distribution hypotheses, and simultaneously remove various types of noise, even structure-related noise. [ABSTRACT FROM AUTHOR]

The main problem of stitching unmanned aerial vehicle (UAV) thermal infrared remote sensing (TIRS) images lies in that the cumulative error caused by the inaccurate alignment of image matching pairs easily leads to deformation and even failure. Few studies in the literature are reported in stitching TIRS images. For the first time, we propose a simple and robust TIRS image stitching method by exploring prior information during flight. First, according to the position and orientation system information and parameter of camera, the overlap ratio of adjacent images is estimated, and the image pairs with high overlap ratio and high matching confidence in different directions are selected. Then, they are added into the alignment term of global similarity prior (GSP) model. Therefore, each image has more matching pairs constraints compared to the traditional construction method of matching pairs, which greatly improves the local registration capability of GSP and then prevents it from converging to the local optimal solution. The proposed method was extensively evaluated on a dataset including to 24 groups of large-scale farmland TIRS images collected in four experimental areas under different crop growth periods and meteorological conditions. Compared with two commercial tools and two representative stitching algorithms, the proposed method significantly improves the local alignment ability and overall stitching quality on both qualitative and quantitative evaluation. Besides, when the front overlap ratio is reduced from 85% to 70%, the proposed method still shows obvious advantages over the related methods and commercial tools, which improves the acquisition efficiency of UAV TIRS images. [ABSTRACT FROM AUTHOR]

This paper presents a novel object tracking framework by joint registration and active contour segmentation (JRACS), which can robustly deal with the non-rigid shape changes of the target. The target region, which includes both foreground and background pixels, is implicitly represented by a level set. A Bhattacharyya similarity based metric is proposed to locate the region whose foreground and background distributions best match those of the tracked target. Based on this metric, a tracking framework that consists of a registration stage and a segmentation stage is then established. The registration step roughly locates the target object by modeling its motion as an affine transformation, and the segmentation step refines the registration result and computes the true contour of the target. The robust tracking performance of the proposed JRACS method is demonstrated by real video sequences where the objects have clear non-rigid shape changes. [ABSTRACT FROM PUBLISHER]

Crop classification in agriculture is one of important applications for polarimetric synthetic aperture radar (PolSAR) data. For agricultural crop discrimination, compared with single-temporal data, multi-temporal data can dramatically increase crop classification accuracies since the same crop shows different external phenomena as it grows up. In practice, the utilization of multi-temporal data encounters a serious problem known as a "dimension disaster". Aiming to solve this problem and raise the classification accuracy, this study developed a feature dimension reduction method using stacked sparse auto-encoders (S-SAEs) for crop classification. First, various incoherent scattering decomposition algorithms were employed to extract a variety of detailed and quantitative parameters from multi-temporal PolSAR data. Second, based on analyzing the configuration and main parameters for constructing an S-SAE, a three-hidden-layer S-SAE network was built to reduce the dimensionality and extract effective features to manage the "dimension disaster" caused by excessive scattering parameters, especially for multi-temporal, quad-pol SAR images. Third, a convolutional neural network (CNN) was constructed and employed to further enhance the crop classification performance. Finally, the performances of the proposed strategy were assessed with the simulated multi-temporal Sentinel-1 data for two experimental sites established by the European Space Agency (ESA). The experimental results showed that the overall accuracy with the proposed method was raised by at least 17% compared with the long short-term memory (LSTM) method in the case of a 1% training ratio. Meanwhile, for a CNN classifier, the overall accuracy was almost 4% higher than those of the principle component analysis (PCA) and locally linear embedded (LLE) methods. The comparison studies clearly demonstrated the advantage of the proposed multi-temporal crop classification methodology in terms of classification accuracy, even with small training ratios. [ABSTRACT FROM AUTHOR]

With increasing consumption, plastic mulch benefits agriculture by promoting crop quality and yield, but the environmental and soil pollution is becoming increasingly serious. Therefore, research on the monitoring of plastic mulched farmland (PMF) has received increasing attention. Plastic mulched farmland in unmanned aerial vehicle (UAV) remote images due to the high resolution, shows a prominent spatial pattern, which brings difficulties to the task of monitoring PMF. In this paper, through a comparison between two deep semantic segmentation methods, SegNet and fully convolutional networks (FCN), and a traditional classification method, Support Vector Machine (SVM), we propose an end-to-end deep-learning method aimed at accurately recognizing PMF for UAV remote sensing images from Hetao Irrigation District, Inner Mongolia, China. After experiments with single-band, three-band and six-band image data, we found that deep semantic segmentation models built via single-band data which only use the texture pattern of PMF can identify it well; for example, SegNet reaching the highest accuracy of 88.68% in a 900 nm band. Furthermore, with three visual bands and six-band data (3 visible bands and 3 near-infrared bands), deep semantic segmentation models combining the texture and spectral features further improve the accuracy of PMF identification, whereas six-band data obtains an optimal performance for FCN and SegNet. In addition, deep semantic segmentation methods, FCN and SegNet, due to their strong feature extraction capability and direct pixel classification, clearly outperform the traditional SVM method in precision and speed. Among three classification methods, SegNet model built on three-band and six-band data obtains the optimal average accuracy of 89.62% and 90.6%, respectively. Therefore, the proposed deep semantic segmentation model, when tested against the traditional classification method, provides a promising path for mapping PMF in UAV remote sensing images. [ABSTRACT FROM AUTHOR]

The objective of this study was to explore the potential of near infrared hyperspectral imaging combined with statistical regression models and neural networks for nondestructive prediction of protein content of wheat kernels. Seventy-nine samples from 11 breeds of wheat kernels were collected. The protein percentage of each sample measured by semimicro-Kjeldahl method was taken as the reference value. After comparing the prediction models of principal components regression (PCR) and partial least squares regression (PLSR) with various pretreatment methods, PLSR preprocessed by zero mean normalization (z score) function of MATLAB was found to obtain better prediction results than other regression models. Based on 10 latent variables of PLSR, the radial basis function (RBF) neural network was applied to improve the prediction, in which the coefficients of determination (R2) were greater than 0.92 for both the calibration set and validation set, while the corresponding RMSE values were 0.3496 and 0.4005, respectively. Therefore, hyperspectral imaging can provide a fast and non-destructive method for predicting the wheat kernels' protein content. [ABSTRACT FROM AUTHOR]

The aim of this study was to demonstrate the capability of hyperspectral imaging in predicting anthocyanin content changes in wine grapes during ripening. One hundred twenty groups of Cabernet Sauvignon grapes were collected periodically after veraison. The hyperspectral images were recorded by a hyperspectral imaging system with a spectral range from 900 to 1700 nm. The anthocyanin content was measured by the pH differential method. A quantitative model was developed using partial least squares regression (PLSR) or support vector regression (SVR) for calculating the anthocyanin content. The best model was obtained using SVR, yielding a coefficient of validation ( P - R 2 ) of 0.9414 and a root mean square error of prediction ( RMSEP ) of 0.0046, higher than the PLSR model, which had a P - R 2 of 0.8407 and a RMSEP of 0.0129. Therefore, hyperspectral imaging can be a fast and non-destructive method for predicting the anthocyanin content of wine grapes during ripening. [ABSTRACT FROM AUTHOR]

Phenolics contents in wine grapes are key indicators for assessing ripeness. Near-infrared hyperspectral images during ripening have been explored to achieve an effective method for predicting phenolics contents. Principal component regression (PCR), partial least squares regression (PLSR) and support vector regression (SVR) models were built, respectively. The results show that SVR behaves globally better than PLSR and PCR, except in predicting tannins content of seeds. For the best prediction results, the squared correlation coefficient and root mean square error reached 0.8960 and 0.1069 g/L (+)-catechin equivalents (CE), respectively, for tannins in skins, 0.9065 and 0.1776 (g/L CE) for total iron-reactive phenolics (TIRP) in skins, 0.8789 and 0.1442 (g/L M3G) for anthocyanins in skins, 0.9243 and 0.2401 (g/L CE) for tannins in seeds, and 0.8790 and 0.5190 (g/L CE) for TIRP in seeds. Our results indicated that NIR hyperspectral imaging has good prospects for evaluation of phenolics in wine grapes. [ABSTRACT FROM AUTHOR]

Timely and accurate detection of crop water stress is vital for precision irrigation. Whether the accuracy of the prevailing diagnosis of crop water stress using vegetation indices (VIs) and spectral reflectance can be improved still remains to be investigated. The crop surface characteristics such as grayscale or color vary under different water stress, so in this study one more variable, image texture, was utilized together to diagnose water stress. For this end, the canopy image of winter wheat in bloom was obtained by unmanned aerial vehicle (UAV) equipped with multispectral sensor, and the effect of soil background was eliminated using vegetation index threshold method. On this basis, Grey level co-occurrence matrix (GLCM) was used to calculate the mean (MEA), variance (VAR), homogeneity (HOM), contrast (CON), dissimilarity (DIS), entropy (ENT), second moment (SEC) and correlation (COR) of the image texture under different spatial resolutions (0.008 m, 0.01 m, 0.02 m, 0.05 m, 0.1 m and 0.2 m). Next, the canopy vegetation indices were obtained by mathematical transformation of canopy reflectance, and then sensitive image texture and vegetation indices by full subset regression method. Finally, Cubist, BPNN (Back Propagation Neural Network) and ELM (Extreme Learning Machine) methods were adopted to build the estimation models of the stomatal conductance (Gs) of winter wheat (between the sensitive image texture and Gs, and between vegetation index and Gs), and the water stress map was plotted based on the optimal Gs estimation model. The result showed: (i) the image texture obtained from the high-resolution multispectral image had a high correlation with Gs, and the image texture (VAR, HOM, CON, DIS, ENT and SEC) at 550 nm had the most significant correlation; (ii) the higher the ground resolution, the higher the correlation between the Gs and the image texture, the vegetation indices, respectively. The image texture with a ground resolution of 0.008 m combined with VIs and Gs had the highest correlation, and combining image texture and vegetation index can significantly improve the estimation accuracy of winter wheat Gs; (iii) Among the three estimation models, the BPNN model constructed by combining the image texture and VIs (MEA, VAR, ENT, DWSI and EXG) had the best estimation performance (Calibration: R c 2 = 0.899, RMSE c = 0.01, MAE c = 0.006; Validation: R c 2 = 0.834, RMSE v =;0.018, MAE v = 0.014), and an accurate estimation could even be achieved at a lower Gs value. Compared with the BPNN model solely based on VIs or image texture, the R c 2 of the BPNN model based on the combined variables increased by 24% and 22.48%, respectively. Therefore, combining UAV multispectral image texture and VIs to estimate Gs provides a feasible and accurate method for water stress diagnosis of winter wheat. • The incorporation of texture improved the accuracy of water stress diagnoses with VIs. • BPNN model with VIs and texture provided the highest accuracy of Gs estimation. • UAV provides a feasible and accurate method for water stress diagnosis of winter wheat. • Removing soil background and screening methods improve the quality of remote sensing data. [ABSTRACT FROM AUTHOR]

Recently, one-stream trackers have achieved parallel feature extraction and relation modeling through the exploitation of Transformer-based architectures. This design greatly improves the performance of trackers. However, as one-stream trackers often overlook crucial tracking cues beyond the template, they prone to give unsatisfactory results against complex tracking scenarios. To tackle these challenges, we propose a multi-cue single-stream tracker, dubbed MCTrack here, which seamlessly integrates template information, historical trajectory, historical frame, and the search region for synchronized feature extraction and relation modeling. To achieve this, we employ two types of encoders to convert the template, historical frames, search region, and historical trajectory into tokens, which are then collectively fed into a Transformer architecture. To distill temporal and spatial cues, we introduce a novel adaptive update mechanism, which incorporates a thresholding component and a local multi-peak component to filter out less accurate and overly disturbed tracking cues. Empirically, MCTrack achieves leading performance on mainstream benchmark datasets, surpassing the most advanced SeqTrack by 2.0% in terms of the AO metric on GOT-10k. The code is available at https://github.com/wsumel/MCTrack.

An essential need for accurate visual object tracking is to capture better correlations between the tracking target and the search region. However, the dominant Siamese-based trackers are limited to producing dense similarity maps at once via a cross-correlations operation, ignoring to remedy the contamination caused by erroneous or ambiguous matches. In this paper, we propose a novel tracker, termed neighborhood consensus constraint-based siamese tracker (NCSiam), which takes the idea of neighborhood consensus constraint to refine the produced correlation maps. The intuition behind our approach is that we can support the nearby erroneous or ambiguous matches by analyzing a larger context of the scene that contains a unique match. Specifically, we devise a 4D convolution-based multi-level similarity refinement (MLSR) strategy. Taking the primary similarity maps obtained from a cross-correlation as input, MLSR acquires reliable matches by analyzing neighborhood consensus patterns in 4D space, thus enhancing the discriminability between the tracking target and the distractors. Besides, traditional Siamese-based trackers directly perform classification and regression on similarity response maps which discard appearance or semantic information. Therefore, an appearance affinity decoder (AAD) is developed to take full advantage of the semantic information of the search region. To further improve performance, we design a task-specific disentanglement (TSD) module to decouple the learned representations into classification-specific and regression-specific embeddings. Extensive experiments are conducted on six challenging benchmarks, including GOT-10k, TrackingNet, LaSOT, UAV123, OTB2015, and VOT2020. The results demonstrate the effectiveness of our method. The code will be available at https://github.com/laybebe/NCSiam.

Structural support vector machine (SSVM) is popular in the visual tracking field as it provides a consistent target representation for both learning and detection. However, the spatial distribution of feature is not considered in standard SSVM-based trackers, therefore leading to limited performance. To obtain a robust discriminative classifier, this paper proposes a novel tracking framework that spatially regularizes SSVM, which yields a new spatially regularized SSVM (SRSSVM). We utilize the spatial regularization prior to penalize the learning classifier with the same size as the target region. The location of classifier spatially located far from the center of region is assigned large weight and vice versa. Then, it is introduced into the SSVM model as a regularization factor to learn the robust discriminative model. Furthermore, an optimizing algorithm with dual coordination descent is presented to efficiently solve the SRSSVM tracking model. Our proposed SRSSVM tracking method has low computational cost like the traditional linear SSVM tracker while can significantly improve the robustness of the discriminative classifier. The experimental results on three popular tracking benchmark data sets show that the proposed SRSSVM tracking method performs favorably against the state-of-the-art trackers.