Search

Manifold learning has been demonstrated as an effective way to represent intrinsic geometrical structure of samples. In this paper, a new manifold learning approach, named Local Coordinates Alignment (LCA), is developed based on the alignment technique. LCA first obtains local coordinates as representations of local neighborhood by preserving proximity relations on a patch, which is Euclidean. Then, these extracted local coordinates are aligned to yield the global embeddings. To solve the out of sample problem, linearization of LCA (LLCA) is proposed. In addition, in order to solve the non-Euclidean problem in real world data when building the locality, kernel techniques are utilized to represent similarity of the pairwise points on a local patch. Empirical studies on both synthetic data and face image sets show effectiveness of the developed approaches. [ABSTRACT FROM AUTHOR]

Abstract: This paper aims to bridge human hearing and vision from the viewpoint of database search for images or music. The semantic content of an image can be illustrated with music or conversely images can be associated with a piece of music. The theoretical basis of the bridge is synaesthesia, a property of human perception. A prototype cross-media retrieval system is built, using principles established in the neuroscientific study of synaesthesia. [Copyright &y& Elsevier]

Abstract: This paper proposes Tensor Rank One Discriminant Analysis (TR1DA) in which general tensors are input for pattern classification. TR1DA is based on Differential Scatter Discriminant Criterion (DSDC) and Tensor Rank One Analysis (TR1A). DSDC is a generalization of the Fisher discriminant criterion. It ensures convergence during training stage. TR1A is a method for adapting general tensors as input to DSDC. The benefits of TR1DA include: (1) a natural way of representing data without losing structure information, i.e., the information about the relative positions of pixels or regions; (2) a reduction in the small sample size problem which occurs in conventional discriminant learning because the number of training samples is much less than the dimensionality of the feature space; (3) a better convergence during the training procedure. We use a graph-embedding framework to generalize TR1DA in manifold learning-based feature selection algorithms, such as locally linear embedding, ISOMAP, and the Laplace eigenmap. We also kernelize TR1DA to nonlinear problems. TR1DA is then demonstrated to outperform traditional subspace methods, such as principal component analysis and linear discriminant analysis. [Copyright &y& Elsevier]

We present a new method for simultaneously determining three-dimensional (3D) motion and structure of a nonrigid object from its uncalibrated two-dimensional (2D) data with Gaussian or non-Gaussian distributions. A nonrigid motion can be treated as a combination of a rigid component and a nonrigid deformation. To reduce the high dimensionality of the deformable structure or shape, we estimate the probability distribution function (PDF) of the structure through random sampling, integrating an established probabilistic model. The fitting between the observations and the estimated 3D structure will be evaluated using the pooled variance estimator. The recovered structure is only available when the 2D feature points have been properly corresponded over two image frames. Applications of the proposed method to both synthetic and real image sequences are demonstrated with promising results. [ABSTRACT FROM AUTHOR]

Abstract: Multimodal biometric system utilizes two or more individual modalities, e.g., face, gait, and fingerprint, to improve the recognition accuracy of conventional unimodal methods. However, existing multimodal biometric methods neglect interactions of different modalities during the subspace selection procedure, i.e., the underlying assumption is the independence of different modalities. In this paper, by breaking this assumption, we propose a Geometry Preserving Projections (GPP) approach for subspace selection, which is capable of discriminating different classes and preserving the intra-modal geometry of samples within an identical class. With GPP, we can project all raw biometric data from different identities and modalities onto a unified subspace, on which classification can be performed. Furthermore, the training stage is carried out once and we have a unified transformation matrix to project different modalities. Unlike existing multimodal biometric systems, the new system works well when some modalities are not available. Experimental results demonstrate the effectiveness of the proposed GPP for individual recognition tasks. [Copyright &y& Elsevier]

Similarity metric is a key component in query-by-example image searching with visual features. After extraction of image visual features, the scheme of computing their similarities can affect the system performance dramatically — the image searching results are normally displayed in decreasing order of similarity (alternatively, increasing order of distance) on the graphical interface for end users. Unfortunately, conventional similarity metrics, in image searching with visual features, usually encounter several difficulties, namely, lighting, background, and viewpoint problems. From the signal processing point of view, this paper introduces a novel similarity metric and therefore reduces the above three problems to some extent. The effectiveness of this newly developed similarity metric is demonstrated by a set of experiments upon a small image ground truth. [ABSTRACT FROM AUTHOR]

Human tastes in art motivate the need for effective means to build a visual mosaic picture which is made up of many small tiles. In some previous work researchers have tried to translate pictures' styles.1 In this paper, a new approach to image processing in arts is presented in the domain of generating a series of artistic mosaic pictures. An arbitrary image is first translated into a mosaic-based one by dividing it into a number of sub-blocks each with the mean value of the pixels in it. Each of these small fragments can be assigned a new location within this image so that a new mosaic picture is generated. By this mean, images with similar color features can be used to create a series of mosaic pictures. The mosaic pictures consist of the same elements as each other, but might be extremely different from the semantic contents. A basic algorithm is presented, followed by some further improvements. Some preliminary experimental results are then given to show the impact of the proposed special techniques. [ABSTRACT FROM AUTHOR]

Abstract: In video segmentation, an intrinsic problem is that the moving cast shadows are always misclassified as part of the moving objects. This paper presents a novel moving cast shadow detection algorithm. The experiments demonstrate shadow region can be removed quite well and thus good video segmentation results can be obtained. [Copyright &y& Elsevier]

Picture in Picture news video is one typical kind of video sequences, where some regions undergo the moving camera and the others undergo the static camera. Actually it is a very difficult for computer to distinguish whether the motion is caused by the moving objects themselves or the moving camera, thus extracting anchorperson from this kind of video becomes a challenging vision task. To our knowledge, it is the first time to address this kind of application. Change detection is implemented to achieve fast, automatic and robust anchorperson extraction with the aid of a simple anchorperson model, which is initialized once in the first frame. The experiment results demonstrate our performance. [Copyright &y& Elsevier]

A novel adaptive color quantization algorithm is presented based on perceptive information carried by image edges. A few colors can be selected automatically to protect the semantic regions and represent the original image. Experimental results show that the proposed methods perform well in term of perception and speed. [Copyright &y& Elsevier]

This letter proposes a novel security mechanism for the traditional content-based image retrieval (CBIR). A watermark-based algorithm is developed as a prototype for hierarchical queries with different authorization on a large image collection. [Copyright &y& Elsevier]

A novel algorithm based on running sub-blocks with different similarity weights is proposed for object-based image retrieval. By splitting the entire image into certain sub-blocks, we use color region information and similarity matrix analysis to retrieval images under the query of special object. [Copyright &y& Elsevier]

• We established the elastic-plastic-brittle mutation rockburst model of coal rock with structural surface. • We established the volume element-region-system model of rockburst. • We researched on the relationship between energy accumulation and dissipation during the coal rock dynamic rupture process. Rockburst is a complex coal and rock dynamic disaster, which poses a serious threat to safe and efficient mining activities. With the increase in the mining depth and strength, rockburst becomes more and more serious. Coal rock contains different types and sizes of structural surface, and its bucking failure is the result of crack propagation and coalescence. In this paper, based on the elastic–plastic mechanics and damage mechanics, we established the elastic–plastic-brittle mutation rockburst model of coal rock with structural surface, introduced the concept of the coal rock volume energy potential function, and researched the relationship between energy accumulation and dissipation during the coal rock dynamic deformation and rupture process to obtain the rockburst energy condition. In addition, the volume element-region-system model of rockburst was established to explain the rockburst evolution process very well. In the external disturbance or high stress state, the coal rock system volume element is destroyed and quickly releases energy to the adjacent volume element. When the adjacent volume element cannot absorb this energy, it is destroyed, and the energy is released quickly. This energy release forms a chain reaction, and the local coal rock is unstable. Eventually, the entire coal rock system is unstable, and rockburst happens. Based on the above theory, the microseismic (MS) and electromagnetic radiation (EMR) characteristics of coal rock in the rockburst evolution process were analyzed. The significance of the study lies in both theory and practice for deeply understanding the rockburst mechanism, as well as for accurate monitoring and early warning. [ABSTRACT FROM AUTHOR]

In recent years, object counting has been investigated and has made significant progress under a surveillance-view. However, there exists only a few works focusing on the remote sensing object density estimation, and the performance of existing methods is not promising. On the one hand, due to the imbalance distribution of targets in remote sensing images, the model might collapse, leading a severe degradation. On the other hand, the scale of targets in remote sensing images actually varies in real scenarios, which remains a challenge for counting objects accurately. To remedy the above problems, we propose an approach named "SwinCounter" for object counting in remote sensing. Moreover, we introduce a Balanced MSE Loss to pay more attention to the fewer samples, which alleviates the problem of imbalanced object labels. In addition, the attention mechanism in our SwinCounter can precisely capture multi-scale information. Thus, the model is aware of different scales of objects, which capture small and dense targetes more precisely. We build experiments on the RSOC dataset, achieving MAEs of 7.2, 151.5, 14.38, and 52.88 and MSEs of 10.1, 436.0, 22.7, and 74.82 on the Building, Small-Vehicle, Large-Vehicle, and Ship sub-datasets, which demonstrates the competitiveness and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a finite-time convergent continuous action iterated dilemma (CAID) with topological optimization is proposed to overcome the limitations of traditional methods. Asymptotic stability in traditional CAID does not provide information about the rate of convergence or the dynamics of the system in the finite time. There are no effective methods to analyze its convergence time in previous works. We made some efforts to solve these problems. Firstly, CAID is proposed by enriching the players' strategies as continuous, which means the player can choose an intermediate state between cooperation and defection. And discount rate is considered to imitate that players cannot learn accurately based on strategic differences. Then, to analyze the convergence time of CAID, a finite-time convergent analysis based on the Lyapunov function is introduced. Furthermore, the optimal communication topology generation method based on the Deep Q-learning (DQN) is proposed to explore a better game structure. At last, the simulation shows the effectiveness of the proposed method. • The dynamic model of Continuous Action Iterated Dilemma (CAID) with continuous strategy is more realistic. • The convergence time of CAID is analyzed by proposed finite-time convergent analysis method based on the Lyapunov function. • The optimal communication topology generation method based on DQN is proposed to enhance the game structure. [ABSTRACT FROM AUTHOR]

Sparse unmixing (SU) algorithms use the existing spectral library as prior knowledge to analyze the endmembers and estimate abundance maps. The majority of SU algorithms use loss functions based on the $L_{2,1}$ -norm or $F$ -norm to minimize reconstruction error. They have different advantages and shortcomings. In short, $F$ -norm has a differentiable characteristic, and it is easy to minimize as a loss function. However, it is very sensitive to heavy noise and outliers. While the $L_{2,1}$ -norm emphasizes the reconstruction error on each band and is robust to noise with different intensities in different bands. But the $L_{2,1}$ -norm is nondifferentiable at zero-point. This article introduces an adaptive loss function based on the $\sigma $ -norm for SU, which combines the advantages of the $L_{2,1}$ -norm and $F$ -norm. The adaptive loss function is related to a nonnegative parameter $\sigma $. By adjusting the parameter $\sigma $ , the adaptive loss function can approach the $F$ -norm or $L_{2,1}$ -norm. To the best of our knowledge, it is the first time to apply an adaptive loss function to SU. Moreover, the adaptive loss function is globally differentiable, and we propose an optimization algorithm for the adaptive loss function and verify its convergence. Experiments on real-world and synthetic HSIs show that the adaptive loss function effectively enhances the performance of the SU algorithms. [ABSTRACT FROM AUTHOR]

Appearance similarity is of great importance for the association between objects and candidates. Recurrent models and similarity vector are two ways widely used by trackers for calculating similarities between objects and candidates. Recurrent models, like Long Short Term Memory network (LSTM), are capable of modeling the continuous change of object's appearance in trajectory. But it is prone to identity (ID) switch when only employ recurrent models as appearance model. The similarity vector way is able to maintain correct IDs for objects when they reappear. But association fails easily when the object is partially occluded and similarity vector is used as the only appearance model. To obtain more accurate and robust appearance similarity, in this paper, we propose an online association by continuous-discrete appearance similarity measurement, OA-CDASM, for multi-object tracking. For continuous perspective, the concept of "smoothness" is proposed to explicitly model and use the continuous and smooth change of object's appearance in trajectory. For discrete perspective, similarity vector is employed. By taking both continuous smoothness and discrete similarity vector into consideration, we can get the continuous-discrete appearance similarity measurement, CDASM, and further perform online association based on CDASM. Experimental results on three public benchmarks demonstrate the effectiveness of our work. [ABSTRACT FROM AUTHOR]

Sparse unmixing (SU) is a semisupervised learning problem, which performs abundance estimation when a spectral library is given. In this way, the essence of SU is to select the most suitable subset from the spectral library for representing all mixed pixels. Many SU methods adopt joint-sparse and low-rank constraints to guide the abundance estimation. However, the spatial correlation learning in these algorithms is not accurate enough, which seriously affects the unmixing performance. Besides, most pruning-based unmixing methods suffer from complicated pruning strategies and ignore the relationship between the spectral library and mixed pixels. This article proposes a reweighted low-rank and joint-sparse unmixing approach, which combines an effective pruning strategy (RLSU-LP). The RLSU-LP approach consists of rough unmixing stage, library pruning, and fine-tuning unmixing stage. First, the proposed method utilizes image segmentation to obtain different homogeneous regions, i.e., superpixels. A confidence index is introduced to describe the superpixel homogeneity, which is conducive to learning the meticulous spatial correlation. The RLSU-LP method reasonably relaxes or tightens the sparse and low-rank constraints of the abundance matrix by using the confidence index. Furthermore, a supervised library pruning strategy is proposed, which aims to eliminate the inactive endmembers by considering the contribution of representing mixed pixels. Experiments on the synthesized dataset and authentic hyperspectral images verify the effectiveness of our proposed algorithm. [ABSTRACT FROM AUTHOR]

Object detection possesses extremely significant applications in the field of optical remote sensing images. A great many works have achieved remarkable results in this task. However, some common problems, such as scale, illumination, and image quality, are still unresolved. Inspired by the mechanism of cascade attention eagle-eye fovea, we propose a new attention mechanism network named the eagle-eye fovea network (EFNet) which contains two foveae for remote sensing object detection. The EFNet consists of two eagle-eye fovea modules: front central fovea (FCF) and rear central fovea (RCF). The FCF is mainly used to learn the candidate object knowledge based on the channel attention and the spatial attention, while the RCF mainly aims to predict the refined objects with two subnetworks without anchors. Three remote sensing object-detection datasets, namely DIOR, HRRSD, and AIBD, are utilized in the comparative experiments. The best results of the proposed EFNet are obtained on the HRRSD with a 0.622 A P score and a 0.907 A P 50 score. The experimental results demonstrate the effectiveness of the proposed EFNet for both multi-category datasets and single category datasets. [ABSTRACT FROM AUTHOR]

There are many successful spectral based unsupervised dimensionality reduction methods, including Laplacian Eigenmap (LE), Locality Preserving Projection (LPP), Spectral Regression (SR), etc. We find that LPP and SR are equivalent if the symmetric similarity matrix is doubly stochastic, Positive Semi-Definite (PSD) and with rank $p$ p , where $p$ p is the reduced dimension. Since solving SR is believed faster than solving LPP based on some related literature, the discovery promotes us to seek to construct such specific similarity matrix to speed up LPP solving procedures. We then propose an unsupervised linear method called Unsupervised Large Graph Embedding (ULGE). ULGE starts with a similar idea as LPP but adopts an efficient approach to construct anchor-based similarity matrix and then performs spectral analysis on it. Moreover, since conventional anchor generation strategies suffer kinds of problems, we propose an efficient and effective anchor generation strategy, called Balanced $K$ K -means based Hierarchical $K$ K -means (BHKH). The computational complexity of ULGE can reduce to $O(ndm)$ O (n d m) , which is a significant improvement compared to conventional methods need $O(n^2d)$ O (n 2 d) at least, where $n$ n , $d$ d and $m$ m are the number of samples, dimensions, and anchors, respectively. Extensive experiments on several publicly available datasets demonstrate the efficiency and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Hyperspectral anomaly detection (HAD) as a special target detection can automatically locate anomaly objects whose spectral information are quite different from their surroundings, without any prior information about background and anomaly. In recent years, HAD methods based on the low rank representation (LRR) model have caught much attention, and achieved good results. However, LRR is a global structure model, which inevitably ignores the local geometrical information of hyperspectral image. Furthermore, most of these methods need to construct dictionaries with clustering algorithm in advance, and they are carried out stage by stage. In this paper, we introduce a locality constrained term inspired by manifold learning topreserve the local geometrical structure during the LRR process, and incorporate the dictionary learning into the optimization process of the LRR. Our proposed method is an one-stage algorithm, which can obtain the low rank representation coefficient matrix, the dictionary matrix, and the residual matrix referring to anomaly simultaneously. One simulated and three real hyperspectral images are used as test datasets. Three metrics, including the ROC curve, AUC value, and box plot, are used to evaluate the detection performance. The visualized results demonstrate convincingly that our method can not only detect anomalies accurately, but also suppress the background information and noises effectively. The three evaluation metrics also prove that our method is superior to other typical methods. [ABSTRACT FROM AUTHOR]

Graph clustering has achieved promising performance in various real-world applications, and attracted sufficient attention in machine learning. Generally, it encodes the samples' relationship with an affinity graph, and then conducts graph-theoretic optimization to partition the samples into clusters. The performance of the graph clustering methods may be affected by many factors, i.e., the graph quality, the loss measurement and the ad hoc post-processing. In this paper, a new Robust Doubly Stochastic graph clustering method (RDS) is presented, which has the following advantages: (1) it learns a doubly stochastic graph with the self-expression strategy automatically, and does not need the graph normalization step to improve the graph quality; (2) it utilizes a new loss function to calculate the approximation error, which is robust to the outliers that far from the normal samples; (3) it generates the cluster indicator according to the learned graph directly, such that the uncertainty caused by the post-processing procedure can be avoided. Extensive experiments on eight benchmarks demonstrate the effectiveness of RDS on data clustering, and show its advantages over the previous graph clustering methods are also verified. [ABSTRACT FROM AUTHOR]

Matrix completion, aiming at restoring a low-rank matrix from observed entries, indicates the connection with the subspace clustering due to the low-rank property. However, it is expensive to incorporate subspace learning into the pervasive surrogate of matrix completion, the nuclear norm. In this paper, we design an orthogonal subspace exploration model for matrix completion, which can be easily integrated due to the succinct formulation. Then, we propose a non-convex surrogate with tractable solutions for low-rank matrix completion, so that the subspace exploration can be performed simultaneously. Compared with the existing surrogates (e.g. , nuclear norm, Schatten- p norm, max norm, etc.), the proposed surrogate is differential such that the optimization is still simple even after the subspace exploration is incorporated. Although the surrogate is non-convex, a parameter-free algorithm that is proved to converge into the global optimum is developed. The optimization consists of closed-form solutions so that the orthogonal subspace exploration will not distinctly bring additional costs and the algorithm empirically converges within dozens of iterations. Experiments illustrate the efficiency and superiority of our model. • A tractable orthogonal subspace exploration is designed for matrix completion. • A smooth non-convex surrogate is proposed to replace the classic nuclear norm. • The optimization is proved to converge into the global minimum. • The optimization of surrogate usually converges within dozens of iterations. • The surrogate is expandable so it is still fast with additionl subspace exploration. [ABSTRACT FROM AUTHOR]

In this work, aero-optical imaging deviation is investigated for a blunt-headed front-window aircraft, and the deflection angle is introduced to calculate the imaging deviation. This work mainly analyzes the influence of optical sensor position and line of sight (LOS) angle changes on imaging deviation. The results show that the imaging deviation monotonically increases with the increase of LOS angle when the sensor position is in front, and the value of imaging deviation is large. When the sensor position is at the back, the imaging deviation increases with the increase of LOS angle, then decreases, and then increases in the reverse direction, and the value of the imaging deviation is small. [ABSTRACT FROM AUTHOR]

A decrease in the viscosity of a solution of sodium carboxymethyl cellulose (CMC-Na) is a commonly encountered problem in coalbed methane (CBM) pressure measurement with an apron CMC-Na sealer and capsule CMC-Na sealer. Studies have shown that the factors that can reduce the CMC-Na solution viscosity include the hydrogen ion concentration index (pH), the environmental temperature, the drilling and sealing time, and the high-pressure CBM gas environment. In this study, a homemade instrument for measuring the viscosity of a CMC-Na solution under high pressure was used to determine the change laws of the viscosity under different conditions, and comparative tests were performed to investigate various factors affecting the viscosity. The best solution for stabilizing the viscosity was determined and effectively applied at a coal mine site, where an overall improvement in the viscosity behaviour was observed. [ABSTRACT FROM AUTHOR]

Although video summarization has achieved tremendous success benefiting from Recurrent Neural Networks (RNN), RNN-based methods neglect the global dependencies and multi-hop relationships among video frames, which limits the performance. Transformer is an effective model to deal with this problem, and surpasses RNN-based methods in several sequence modeling tasks, such as machine translation, video captioning, etc. Motivated by the great success of transformer and the natural structure of video (frame-shot-video), a hierarchical transformer is developed for video summarization, which can capture the dependencies among frame and shots, and summarize the video by exploiting the scene information formed by shots. Furthermore, we argue that both the audio and visual information are essential for the video summarization task. To integrate the two kinds of information, they are encoded in a two-stream scheme, and a multimodal fusion mechanism is developed based on the hierarchical transformer. In this paper, the proposed method is denoted as Hierarchical Multimodal Transformer (HMT). Practically, extensive experiments show that HMT achieves (F-measure: 0.441, Kendall's τ : 0.079, Spearman's ρ : 0.080) and (F-measure: 0.601, Kendall's τ : 0.096, Spearman's ρ : 0.107) on SumMe and TVsum, respectively. It surpasses most of the traditional, RNN-based and attention-based video summarization methods. [ABSTRACT FROM AUTHOR]

Deep learning-based hyperspectral image superresolution methods have achieved great success recently. However, most methods utilize 2D or 3D convolution to explore features, and rarely combine the two types of convolution to design networks. Moreover, when the model only contains 3D convolution, almost all the methods take all the bands of hyperspectral image as input to analyze, which requires more memory footprint. To address these issues, we explore a new structure for hyperspectral image superresolution using spectrum and feature context. Inspired by the high similarity among adjacent bands, we design a dual-channel network through 2D and 3D convolution to jointly exploit the information from both single band and adjacent bands, which is different from previous works. Under the connection of depth split, it can effectively share spatial information so as to improve the learning ability of 2D spatial domain. Besides, our method introduces the features extracted from previous band, which contributes to the complementarity of information and simplifies the network structure. Through feature context fusion, it significantly enhances the performance of the algorithm. Extensive evaluations and comparisons on three public datasets demonstrate that our approach produces the state-of-the-art results over the existing approaches. [ABSTRACT FROM AUTHOR]

Multi-view clustering, which utilizes the ample information provided by multiple sources to obtain better performance, has attracted much attention. However, existing clustering algorithms either have no ability to offer confidence for each assignment or suffer from the disturbance of outliers. To address these problems, in this paper, we propose a novel multi-view fuzzy clustering method via transferring softmin to fuzzy models. To obtain fuzzy assignments, we utilize the softmin with temperature and further develop an efficient algorithm to solve the non-convex problem approximately. We also show another explanation for the algorithm from the aspect of the prior distribution of various views. Besides, we design a scalable robust loss function, which interpolates between ℓ 2 -norm and the squared ℓ 2 -norm, to enhance the robustness to outliers. Extensive experiments show the superiority of our model under different clustering metrics. [ABSTRACT FROM AUTHOR]

Hyperspectral image super-resolution (SR) methods based on deep learning have achieved significant progress recently. However, previous methods lack the joint analysis between spectrum and horizontal or vertical direction. Besides, when both 2D and 3D convolution are in the network, the existing models cannot effectively combine the two. To address these issues, in this article, we propose a novel hyperspectral image SR method by exploring the relationship between 2D/3D convolution (ERCSR). Our method alternately employs 2D and 3D units to solve the problem of structural redundancy by sharing spatial information during reconstruction for existing model, which can enhance the learning ability of 2D spatial domain. Importantly, compared with the network using 3D units, i.e., 2D units are replaced by 3D units, it can not only reduce the size of the model but also improve the performance of the model. Furthermore, to exploit the spectrum fully, the split adjacent spatial and spectral convolution (SAEC) is designed to parallelly explore information between spectrum and horizontal or vertical direction in space. Experiments on widely used benchmark datasets demonstrate that the proposed approach outperforms state-of-the-art SR algorithms across different scales in terms of quantitative and qualitative analysis. [ABSTRACT FROM AUTHOR]

License plate detection and recognition (LPDR) has attracted considerable attention in recent years, and many algorithms have presented the competitive performance on several datasets. However, there are still three significant issues to be addressed in this field. Firstly, most methods have poor detection performance in unconstrained scenarios with moving vehicles and highly distracting background objects. Secondly, existing systems generally focus on single image-based algorithms, yet traffic video sequences provide more effective information than individual frames for LPDR tasks. Thirdly, images and videos captured in complex environments may be adversely affected by distortions and low resolution, causing sensitive recognition performance and reduced robustness. To remedy these issues, we propose to automatically perform license plate detection, tracking, and recognition in real-world traffic videos and integrate them into a unified end-to-end framework via deep learning. The contributions of this paper are threefold: 1) A deep flow-guided spatiotemporal license plate detector is proposed to model the video contextual information by introducing optical flow and a novel spatiotemporal attention mechanism; 2) An online license plate tracker is developed to bridge video-based detection and recognition which utilizes both motion and deep appearance information, and innovatively, it can be end-to-end trained with the detector via multi-task learning; 3) The efficient quality-guided license plate recommender and recognizer are proposed to jointly perform stream recognition. The former recommends high-quality frames from video streams while the latter generates recognition results. We evaluate the proposed method on three traffic video-based license plate datasets, and ablation studies have been presented to verify the effectiveness of each component mentioned above. Moreover, extensive experiments are conducted for comparison with other approaches in different scenarios, and the results have demonstrated that our method achieves state-of-the-art performance on all datasets. [ABSTRACT FROM AUTHOR]

Hyperspectral images can provide dozens to hundreds of continuous spectral bands, so the richness of information has been greatly improved. However, these bands lead to increasing complexity of data processing, and the redundancy of adjacent bands is large. Recently, although many band selection methods have been proposed, this task is rarely handled through the context information of the whole spectral bands. Moreover, the scholars mainly focus on the different numbers of selected bands to explain the influence by accuracy measures, neglecting how many bands to choose is appropriate. To tackle these issues, we propose a fast neighborhood grouping method for hyperspectral band selection (FNGBS). The hyperspectral image cube in space is partitioned into several groups using coarse-fine strategy. By doing so, it effectively mines the context information in a large spectrum range. Compared with most algorithms, the proposed method can obtain the most relevant and informative bands simultaneously as subset in accordance with two factors, such as local density and information entropy. In addition, our method can also automatically determine the minimum number of recommended bands by determinantal point process. Extensive experimental results on benchmark data sets demonstrate the proposed FNGBS achieves satisfactory performance against state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Multi-Task Learning attempts to explore and mine the sufficient information within multiple related tasks for the better solutions. However, the performance of the existing multi-task approaches would largely degenerate when dealing with the polluted data, i.e., outliers. In this paper, we propose a novel robust multi-task model by incorporating a flexible manifold constraint (FMC-MTL) and a robust loss. Specifically speaking, multi-task subspace is embedded with a relaxed and generalized Stiefel Manifold for considering point-wise correlation and preserving the data structure simultaneously. In addition, a robust loss function is developed to ensure the robustness to outliers by smoothly interpolating between ℓ2,1-norm and squared Frobenius norm. Equipped with an efficient algorithm, FMC-MTL serves as a robust solution to tackling the severely polluted data. Moreover, extensive experiments are conducted to verify the superiority of our model. Compared to the state-of-the-art multi-task models, the proposed FMC-MTL model demonstrates remarkable robustness to the contaminated data. [ABSTRACT FROM AUTHOR]

Many representation methods were built for gray image textures. However, they are not effective for color textures in general. To alleviate this problem, in this paper we propose a novel Compact Interchannel Sampling Difference Descriptor (CISDD) for color texture classification. In particular, considering sampling-based method can capture more directional information, we first use a heavy-tailed distribution, t-distribution to generate sample points in the image patch to calculate the micro-block difference. Then we model the interchannel relationship of color texture image by using dense micro-block differences. Furthermore, we utilize principal component analysis (PCA) to reduce the dimensions of the features encoded by the Fisher vector, and construct a Compact Interchannel Sampling Difference Descriptor (CISDD) for representing color texture image. Finally, experimental results on five published standard texture datasets (KTH-TIPS, VisTex, CUReT, USPTex and Colored Brodatz) reveal that CISDD is effective and outperforms thirteen representative color texture classification methods. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) change detection plays an important role in remote sensing applications, and considerable research has been done focused on improving change detection performance. However, the high dimension of hyperspectral data makes it hard to extract discriminative features for hyperspectral processing tasks. Though deep convolutional neural networks (CNN) have superior capability in high-level semantic feature learning, it is difficult to employ CNN for change detection tasks. As a ground truth map is usually used for the evaluation of change detection algorithms, it cannot be directly used for supervised learning. In order to better extract discriminative CNN features, a novel noise modeling-based unsupervised fully convolutional network (FCN) framework is presented for HSI change detection in this paper. Specifically, the proposed method utilizes the change detection maps of existing unsupervised change detection methods to train the deep CNN, and then removes the noise during the end-to-end training process. The main contributions of this paper are threefold: (1) A new end-to-end FCN-based deep network architecture for HSI change detection is presented with powerful learning features; (2) An unsupervised noise modeling method is introduced for the robust training of the proposed deep network; (3) Experimental results on three datasets confirm the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and ℓ2,p-norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

As a fundamental and challenging task in bridging language and vision domains, Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality, and its key challenge is to measure the semantic similarity across different modalities. Although significant progress has been achieved, existing approaches typically suffer from two major limitations: (1) It hurts the accuracy of the representation by directly exploiting the bottom-up attention based region-level features where each region is equally treated. (2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism. To address these limitations, we propose a Unified Semantic Enhancement Momentum Contrastive Learning (USER) method for ITR. Specifically, we delicately design two simple but effective Global representation based Semantic Enhancement (GSE) modules. One learns the global representation via the self-attention algorithm, noted as Self-Guided Enhancement (SGE) module. The other module benefits from the pre-trained CLIP module, which provides a novel scheme to exploit and transfer the knowledge from an off-the-shelf model, noted as CLIP-Guided Enhancement (CGE) module. Moreover, we incorporate the training mechanism of MoCo into ITR, in which two dynamic queues are employed to enrich and enlarge the scale of negative sample pairs. Meanwhile, a Unified Training Objective (UTO) is developed to learn from mini-batch based and dynamic queue based samples. Extensive experiments on the benchmark MSCOCO and Flickr30K datasets demonstrate the superiority of both retrieval accuracy and inference efficiency. For instance, compared with the existing best method NAAF, the metric R@1 of our USER on the MSCOCO 5K Testing set is improved by 5% and 2.4% on caption retrieval and image retrieval without any external knowledge or pre-trained model while enjoying over 60 times faster inference speed. Our source code will be released at https://github.com/zhangy0822/USER. [ABSTRACT FROM AUTHOR]

Fuzzy K-Means (FKM) based on fuzzy theory is a classic method to effectively handle overlapping regions between clusters. However, redundant features and noises brought by increasing data dimensions affect the effectiveness of FKM. To cope with this issue, we propose a Discriminative Projection Fuzzy K-Means with adaptive neighbors (DPFKM) model, which embeds a discriminative subspace into FKM to facilitate learning of global structure and the most discriminative information. Firstly, a novel projection space with uncorrelated constraints are adopted to promote statistical independence among the data in the subspace as well as to enhance the ability of FKM to discern and utilize discriminative information. Secondly, the Frobenius norm is introduced as the regularization term to eliminate discrete solutions, while preserving the fuzziness and enhancing the sparsity of FKM. Finally, we propose a novel optimization method to finetune the model, with a particular focus on adaptive adjustment of the regularization parameter based on the proximity relationship between the samples and clusters. Comprehensive experiments are conducted on multiple data sets, and the results can prove the superiority of the proposed model. • The discriminative projection fuzzy K-means (DPFKM) is proposed to handle subspace data clustering. • DPFKM utilizes uncorrelated constraints to maintain discriminative information and global structure in subspace. • DPFKM adaptively determines the regularization parameter using neighbor relationship of samples and clusters. • Experiments on nine data sets show that our method realize a competitive performance in clustering results. [ABSTRACT FROM AUTHOR]

Over the past few years making use of deep networks, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), classifying hyperspectral images has progressed significantly and gained increasing attention. In spite of being successful, these networks need an adequate supply of labeled training instances for supervised learning, which, however, is quite costly to collect. On the other hand, unlabeled data can be accessed in almost arbitrary amounts. Hence it would be conceptually of great interest to explore networks that are able to exploit labeled and unlabeled data simultaneously for hyperspectral image classification. In this article, we propose a novel graph-based semisupervised network called nonlocal graph convolutional network (nonlocal GCN). Unlike existing CNNs and RNNs that receive pixels or patches of a hyperspectral image as inputs, this network takes the whole image (including both labeled and unlabeled data) in. More specifically, a nonlocal graph is first calculated. Given this graph representation, a couple of graph convolutional layers are used to extract features. Finally, the semisupervised learning of the network is done by using a cross-entropy error over all labeled instances. Note that the nonlocal GCN is end-to-end trainable. We demonstrate in extensive experiments that compared with state-of-the-art spectral classifiers and spectral–spatial classification networks, the nonlocal GCN is able to offer competitive results and high-quality classification maps (with fine boundaries and without noisy scattered points of misclassification). [ABSTRACT FROM AUTHOR]

Band selection is one of the most important technique in the reduction of hyperspectral image (HSI). Different from traditional feature selection problem, an important characteristic of it is that there is usually strong correlation between neighboring bands, that is, bands with close indexes. Aiming to fully exploit this prior information, a novel band selection method called optimal neighborhood reconstruction (ONR) is proposed. In ONR, band selection is considered as a combinatorial optimization problem. It evaluates a band combination by assessing its ability to reconstruct the original data, and applies a noise reducer to minimize the influence of noisy bands. Instead of using some approximate algorithms, ONR exploits a recurrence relation that underlies the optimization target to obtain the optimal solution in an efficient way. Besides, we develop a parameter selection approach to automatically determine the parameter of ONR, ensuring it is adaptable to different data sets. In experiments, ONR is compared with some state-of-the-art methods on six HSI data sets. The results demonstrate that ONR is more effective and robust than the others in most of the cases. [ABSTRACT FROM AUTHOR]

Deep neural-network has been widely used in natural image denoising. However, due to the lack of label of real ultrasound (US) B-mode image for de-speckling, the deep neural network is greatly restricted in US image de-speckling. In this paper, we propose to use transfer learning and two types of prior knowledge to construct a hybrid neural network structure for de-speckling. Firstly, based on a given US image model, the speckle noise is similar to Gaussian distribution in the logarithmic transformation domain, called Gaussian prior knowledge. The distribution parameters are estimated in the logarithmic transformation domain based on four typical traditional US image de-speckling methods with maximum likelihood estimation. Secondly, depending on the prior parameters, a transferable denoising network is trained with clean natural image dataset. Finally, a VGGNet is used to extract the structure boundaries before and after US image de-speckling based on the transfer network, and we call it structural prior knowledge. The structural boundaries of a US image should be unchanged after the de-speckling, and hence we use this constraint to fine-tune the transfer network. The proposed de-speckling framework is verified on artificially generated phantom (AGP) images and real US images, and the results demonstrate its effectiveness. [ABSTRACT FROM AUTHOR]

Recently, deep-learning based lane detection methods effectively boost the development of Advanced Driver Assistance Systems (ADAS) and Self-Driving Systems. However, these methods only detect lane lines with sketchy bounding boxes while ignore the shape of specific curved lanes. To address the above problems, this paper introduces deep reinforcement learning into cursory lane detection models for accurate lane detection and localization. This model consists of two stages, namely the bounding box detector and landmark point localizer. To be specific, a bounding box level convolution neural network lane detector outputs the preliminary location of lanes in the form of bounding boxes. Then, a reinforcement based Deep Q-Learning Localizer (DQLL) accurately localizes the lanes as a group of landmarks to achieve better representation of curved lanes. Moreover, a pixel-level lane detection dataset named NWPU Lanes Dataset is constructed and released. It contains a variety of real traffic scenes and accurate masks of the lane lines. This approach achieves competitive performance in the released dataset and TuSimple Lane dataset. Furthermore, the codes and dataset will be released on https://github.com/tuzixini/DQLL. [ABSTRACT FROM AUTHOR]

Crowd counting from a single image is a challenging task due to high appearance similarity, perspective changes, and severe congestion. Many methods only focus on the local appearance features and they cannot handle the aforementioned challenges. In order to tackle them, we propose a perspective crowd counting network (PCC Net), which consists of three parts: 1) density map estimation (DME) focuses on learning very local features of density map estimation; 2) random high-level density classification (R-HDC) extracts global features to predict the coarse density labels of random patches in images; and 3) fore-/background segmentation (FBS) encodes mid-level features to segments the foreground and background. Besides, the Down, Up, Left, and Right (DULR) module is embedded in PCC Net to encode the perspective changes on four directions (DULR). The proposed PCC Net is verified on five mainstream datasets, which achieves the state-of-the-art performance on the one and attains the competitive results on the other four datasets. The source code is available at https://github.com/gjy3035/PCC-Net. [ABSTRACT FROM AUTHOR]

Kernel trick is widely applied to Support Vector Machine (SVM) to deal with linearly inseparable data which is known as kernel SVM. However, kernel SVM always has high computational cost in practice which makes it unsuitable to handle large scale data. Moreover, kernel SVM always brings hyper-parameters, e.g. bandwidth in Gaussian kernel. Since the hyper-parameters have a significant influence on the final performance of kernel SVM and are pretty hard to tune especially for large scale data, one may need to put lots of effort into finding good enough parameters, and improper settings of the hyper-parameters often make the classification performance even lower than that of linear SVM. Inspired by recent progresses on linear SVM for dealing with large scale data, we propose a well-designed classifier to efficiently handle large scale linearly inseparable data, i.e., Decision Tree SVM (DTSVM). DTSVM has much lower computational cost compared with kernel SVM, and it brings almost no hyper-parameters except a few thresholds which can be fixed in practice. Comprehensive experiments on large scale datasets demonstrate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]