Search

Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost of acquisition equipment, thereby providing a feasible way to improve the quality of remote sensing images. Clearly, image SR is a severe ill-posed problem. With the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. Since the texture information of various remote sensing images are totally different from each other, in this article, we proposed a network based on generative adversarial network (GAN) to achieve high-resolution remote sensing images, named multiattention GAN (MA-GAN). The main body of the generator in MA-GAN contains three blocks: pyramid convolutional residual dense (PCRD) block, attention-based upsampling (AUP) block, and attention-based fusion (AF) block. Specifically, the developed attention pyramid convolutional (AttPConv) operator in the PCRD block combines multiscale convolution and channel attention (CA) to automatically learn and adjust the scale of residuals for better representation. The established AUP block uses pixel attention (PA) to perform arbitrary scales of upsampling. The AF block uses branch attention (BA) to integrate upsampled low-resolution images with high-level features. Besides, the loss function takes both adversarial loss and feature loss into consideration to guide the learning procedure of the generator. We have compared our MA-GAN approach with several state-of-the-art methods on a number of remote sensing scenes, and the experimental results consistently demonstrate the effectiveness of the proposed MA-GAN. For study replication, the source code will be released at: https://github.com/ZhihaoWang1997/MA-GAN. [ABSTRACT FROM AUTHOR]

The rapid development of remote sensing sensors makes the acquisition, analysis, and application of hyperspectral images (HSIs) more and more extensive. However, the limited sample sets, high-dimensional features, highly correlated bands, and mixing spectral information make the classification of HSIs a great challenge. In this article, an unsupervised multiscale and diverse feature learning (UMsDFL) method is proposed for HSI classification, which deeply considers the spatial–spectral features via convolutional neural networks (CNNs). Specifically, after employing the simple noniterative clustering (SNIC) algorithm with the heuristic calculation of superpixel size, the HSIs are segmented into superpixels for feature learning. The unsupervised network is designed with the convolutional encoder and decoder, the additional clustering branch, and the multilayer feature fusion to enhance the distinguishability of feature learning and the reusability of feature maps. Then, the spatial relationships and object attributes in large- and small-scale contexts are learned collaboratively through the unsupervised network to utilize the complementary multiscale characteristics. Moreover, the diverse features of hyperspectral information and nonsubsampled contourlet transform (NSCT) textures are learned simultaneously via the unsupervised network to alleviate the insufficiency of geometric representation. Finally, the random forest (RF) is adopted as the comprehensive classifier for land cover mapping based on the UMsDFL, and superpixel regularization is adopted to optimize the classification results. A series of experiments are performed on three real-world HSI datasets to demonstrate the effectiveness of our UMsDFL approach. The experimental results show that the proposed UMsDFL can achieve the overall accuracy of 79.23%, 96.49%, and 77.26% for Houston, Pavia, and Dioni datasets, respectively, when there are only five samples per class for training. [ABSTRACT FROM AUTHOR]

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available at http://jiasen.tech/papers/. [ABSTRACT FROM AUTHOR]

As an important medium of human-computer interaction, flexible tactile sensors have a vital influence on the intelligence and safety of a system by accurately identifying the contact position and force. For now, most of the existing flexible capacitive pressure sensors adopt an array structure, which still cannot meet the requirements including simple manufacturing, simple wiring, and low cost. This paper innovatively proposes a flexible array-less capacitive tactile sensor, a 30 mm $\times30$ mm sensor prototype, consisting of flexible electrodes and a dielectric layer of a polydimethylsiloxane (PDMS) and multiwalled carbon nanotube (MWCNT) composite. Under the action of a contact force, the contact area is deformed, resulting in a capacitance change in the flexible sensor. First, Poisson’s equation is used to determine the relationship between the electric potential and position. Then, elastic contact mechanics is adopted to describe the relationship between the electric charges and force. Finally, the contact position and force are recognized by measuring the change in capacitance in real time, based on the classical principle of capacitance. Simulation and experimental results verified the effectiveness of the proposed recognition method, showing that the recognition resolution of contact position can reach $500\,\,\mu \text{m}\,\,\times 100\,\,\mu \text{m}$ , and the accuracy of contact force is 95.85%. Theoretically, the proposed flexible array-less sensor in this paper has infinite spatial resolution, and most importantly, it does not rely on complex manufacturing, redundant circuits, or sensor cell arrays. Therefore, it has broad application prospects in related fields such as electronic skin and wearable electronic devices. [ABSTRACT FROM AUTHOR]

Hyperspectral images encompass abundant information and provide unique characteristics for material classification. However, the labeling of training samples can be challenging in hyperspectral image classification. To address this problem, this study proposes a framework named flexible Gabor-based superpixel-level unsupervised linear discriminant analysis (FG-Su ULDA) to extract the most informative and discriminating features for classification. First, a number of 3-D flexible Gabor filters are rigorously designed using an asymmetric sinusoidal wave to sufficiently characterize the spatial–spectral structure in hyperspectral images. Then, an unsupervised linear discriminant analysis strategy guided by the entropy rate superpixel (ERS) segmentation algorithm, called Su ULDA, is skillfully introduced to reduce the extracted large amount of FG features. The Su ULDA method not only boosts the classification capability but also increases the peculiarity of features, with the aid of superpixel information. Finally, the achieved features are imported to the popular support vector machine classifier. The proposed FG-Su ULDA framework is applied to four real hyperspectral image data sets, and the experiments constantly prove that our FG-Su ULDA is superior to several state-of-the-art methods in both classification performance and computational efficiency, especially with scarce training samples. The codes of this work are available at http://jiasen.tech/papers/ for the sake of reproducibility. [ABSTRACT FROM AUTHOR]

Deep learning methods have achieved attractive performance in dynamic MR cine imaging. However, most of these methods are driven only by the sparse prior of MR images, while the important low-rank (LR) prior of dynamic MR cine images is not explored, which may limit further improvements in dynamic MR reconstruction. In this paper, a learned singular value thresholding (Learned-SVT) operator is proposed to explore low-rank priors in dynamic MR imaging to obtain improved reconstruction results. In particular, we put forward a model-based unrolling sparse and low-rank network for dynamic MR imaging, dubbed as SLR-Net. SLR-Net is defined over a deep network flow graph, which is unrolled from the iterative procedures in the iterative shrinkage-thresholding algorithm (ISTA) for optimizing a sparse and LR-based dynamic MRI model. Experimental results on a single-coil scenario show that the proposed SLR-Net can further improve the state-of-the-art compressed sensing (CS) methods and sparsity-driven deep learning-based methods with strong robustness to different undersampling patterns, both qualitatively and quantitatively. Besides, SLR-Net has been extended to a multi-coil scenario, and achieved excellent reconstruction results compared with a sparsity-driven multi-coil deep learning-based method under a high acceleration. Prospective reconstruction results on an open real-time dataset further demonstrate the capability and flexibility of the proposed method on real-time scenarios. [ABSTRACT FROM AUTHOR]

Due to the importance in many military and civilian applications, hyperspectral anomaly detection has attracted remarkable interest. Low-rank representation (LRR)-based anomaly detectors use the low-rank property to represent background pixels, and pixels that cannot be well represented are detected as anomalies. The ability of an LRR-based detector to separate background pixels and anomalous pixels depends on the dictionary representation ability, which usually can be enhanced by designing a proper prior for dictionary representation coefficients and constructing a better dictionary. However, it is not easy to handcraft effective and meaningful regularizers for dictionary coefficients. In this article, we propose a novel anomaly detection algorithm that uses a plug-and-play prior for representation coefficients and constructs a new dictionary based on clustering. Instead of cumbersomely handcrafting a regularizer for representation coefficients, we propose solving the anomaly detection problem using the plug-and-play framework, which enables us to plug state-of-the-art priors for representation coefficients. An effective convolutional neural network (CNN) denoiser is plugged into our framework to fully exploit the spatial correlation of representation coefficients. We also propose a modified background dictionary construction method, which carefully includes background pixels and excludes anomalous pixels from clustering results. We refer to the proposed anomaly detection method as plug-and-play denoising CNN regularized anomaly detection (DeCNN-AD) method. Extensive experiments were performed on five data sets in a comparison with eight state-of-the-art anomaly detection methods. The experimental results suggest that the proposed method is effective in anomaly detection and can produce better anomaly detection results than that of the comparison methods. The codes of this work will be available at https://github.com/FxyPd for the sake of reproducibility. [ABSTRACT FROM AUTHOR]

In the hyperspectral image, each pixel corresponds to a small area on the Earth’s surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial–spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial–spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples. [ABSTRACT FROM AUTHOR]

The rapid increase in the number of remote sensing sensors makes it possible to develop multisource feature extraction and fusion techniques to improve the classification accuracy of surface materials. It has been reported that light detection and ranging (LiDAR) data can contribute complementary information to hyperspectral images (HSIs). In this article, a multiple feature-based superpixel-level decision fusion (MFSuDF) method is proposed for HSIs and LiDAR data classification. Specifically, superpixel-guided kernel principal component analysis (KPCA) is first designed and applied to HSIs to both reduce the dimensions and compress the noise impact. Next, 2-D and 3-D Gabor filters are, respectively, employed on the KPCA-reduced HSIs and LiDAR data to obtain discriminative Gabor features, and the magnitude and phase information are both taken into account. Three different modules, including the raw data-based feature cube (concatenated KPCA-reduced HSIs and LiDAR data), the Gabor magnitude feature cube, and the Gabor phase feature cube (concatenation of the corresponding Gabor features extracted from the KPCA-reduced HSIs and LiDAR data), can be, thus, achieved. After that, random forest (RF) classifier and quadrant bit coding (QBC) are introduced to separately accomplish the classification task on the aforementioned three extracted feature cubes. Alternatively, two superpixel maps are generated by utilizing the multichannel simple noniterative clustering (SNIC) and entropy rate superpixel segmentation (ERS) algorithms on the combined HSIs and LiDAR data, which are then used to regularize the three classification maps. Finally, a weighted majority voting-based decision fusion strategy is incorporated to effectively enhance the joint use of the multisource data. The proposed approach is, thus, named MFSuDF. A series of experiments are conducted on three real-world data sets to demonstrate the effectiveness of the proposed MFSuDF approach. The experimental results show that our MFSuDF can achieve the overall accuracy of 73.64%, 93.88%, and 74.11% for Houston, Trento, and Missouri University and University of Florida (MUUFL) Gulport data sets, respectively, when there are only three samples per class for training. [ABSTRACT FROM AUTHOR]

Objective: The purpose of this article was to build a radio frequency (RF) coil system to achieve high vessel wall image quality with coverage extending from the aortic arch to the intracranial vessels. Methods: A 48-channel coil system was built and characterized at a 3 tesla (T) Magnetic Resonance Imaging (MRI) scanner (uMR 790, Shanghai United Imaging Healthcare, Shanghai, China). The coil's performance was compared with a commercially available 36-channel coil system. By human studies, signal-to-noise ratio (SNR) units were evaluated and g-factors were calculated in the transverse planes of the brain and neck regions. Results: The SNR was increased by at least 28% in the brain region and up to fourfold in the neck region. The average g-factor with the acceleration factor, R = 3, was lowered by 21% in the transverse plane of the neck region. Intracranial and carotid arterial wall images with an isotropic spatial resolution of 0.63 mm were acquired within 7.7 minutes and thoracic aorta wall images with an isotropic spatial resolution of 1.1 mm were acquired within 2.7 minutes with the 48-channel coil system. The vessel wall can be more clearly visualized with the 48-channel coil system compared with the 36-channel coil system. Conclusion: A 48-channel coil system was developed and demonstrated superior performance for vessel wall imaging at the intracranial and cervical carotid arteries compared with a commercial 36-channel coil. Significance: The 48-channel coil system is potentially useful for clinical diagnostics, especially when attempting to diagnose ischemic stroke. [ABSTRACT FROM AUTHOR]

As a typical graph-based semisupervised learning technique, the label propagation (LP) approach has gained much attention in recent years. The key to LP algorithms is the propagation capability and efficiency of the similarity matrix, which describes the similarity between two data points. Concerning hyperspectral image which often contains hundreds of thousands of pixels, the corresponding similarity matrix is particularly huge and thus the LP procedure is intractable. Fortunately, superpixel, which can effectively characterize the spatial semantic information of surface objects, provides a reasonable way to solve this problem. In this article, we propose an elaborate superpixel-based weighted LP approach, abbreviated as SuWLP, for hyperspectral image classification. First, the hyperspectral image is oversegmented by the entropy rate segmentation (ERS) method, and the internal consistency of each superpixel can be achieved. Second, a new similarity measure is designed to estimate the similarity between two superpixels, and a superpixel-based similarity matrix can be thus established. Third, after the training samples have been expanded based on the superpixel distribution, a weighted LP technique is designed to propagate the sample label at the superpixel level without any parameter tuning. Finally, the label of each superpixel maps back to the contained pixels. We compared our proposed SuWLP method with several state-of-the-art ones, and experimental results on three real hyperspectral data sets certify the effectiveness and efficiency of the superpixel-level LP strategy. [ABSTRACT FROM AUTHOR]

Learning from a limited number of labeled samples (pixels) remains a key challenge in the hyperspectral image (HSI) classification. To address this issue, we propose a deep metric learning-based feature embedding model, which can meet the tasks both for same- and cross-scene HSI classifications. In the first task, when only a few labeled samples are available, we employ ideas from metric learning based on deep embedding features and make a similarity learning between pairs of samples. In this case, the proposed model can learn well to compare whether two samples belong to the same class. In another task, when an HSI image (target scene) that needs to be classified is not labeled at all, the embedding model can learn from another similar HSI image (source scene) with sufficient labeled samples and then transfer to the target model by using an unsupervised domain adaptation technique, which not only employs the adversarial approach to make the embedding features from the source and target samples indistinguishable but also encourages the target scene’s embeddings to form similar clusters with the source scene one. After the domain adaptation between the HSIs of the two scenes is finished, any traditional HSI classifier can be used. In a simple manner, the nearest neighbor (NN) algorithm is selected as the classifier for the classification tasks throughout this article. The experimental results from a series of popular HSIs demonstrate the advantages of the proposed model both in the same- and cross-scene classification tasks. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing imagery provides valuable and rich information to distinguish the characteristics of materials. However, this advantage of hyperspectral imagery often encounters the problem of a limited amount of training samples, which is caused by the difficulty of manually labeling. Fortunately, the spatial distribution of surface objects can be integrated with the spectral signature to improve the discriminative ability. In this paper, a 3-D Gaussian–Gabor feature extraction and selection framework has been proposed for hyperspectral image classification. First, a bank of 3-D Gaussian–Gabor filters are convolved with the concatenated data of both extended multi-attribute profile (EMAP) features and raw hyperspectral data. Second, an improved fast density peak clustering (IFDPC) method is introduced to select the most representative features from each extracted 3-D Gaussian–Gabor feature cube. Finally, the retained features are combined together to accomplish the classification task. The proposed method is thus named as GG-IFDPC. Three real hyperspectral imagery data sets have been utilized, and the experiments demonstrate the advantages of the proposed GG-IFDPC approach over the compared ones. [ABSTRACT FROM AUTHOR]

In virtue of the spatial structural characteristic of surface materials, the performance of the hyperspectral image classification can be boosted by incorporating texture information. Normally, the spatial structure can be extracted by predefined operators, including the popular extended multiattribute profiles (EMAPs) and the Gabor filters. Recently, superpixel segmentation, which reflects the homogeneous regularity of objects, has drawn much attention in the field. In this paper, a collaborative representation-based multiscale superpixel fusion (CRMSF) approach has been proposed for the hyperspectral image classification. First, after obtaining the EMAPs from the raw hyperspectral image, a group of predesigned 3-D Gabor wavelet filters is convolved with the EMAP features, and the EMAP-Gabor features can, thus, be achieved. Second, the collaborative representation-based classification (CRC) is employed to fully and efficiently make use of the huge amount of extracted EMAP-Gabor features. Third, multiscale superpixel maps are generated from the EMAP features that are utilized to regularize the classification map obtained by CRC. A heuristic strategy has been specially devised to automatically decide the number of extracted superpixels in multiple scales, which can be perfectly compatible with hyperspectral images having various spatial sizes and spatial resolutions. This is the most important contribution of the developed CRMSF approach. Finally, the classification task is accomplished by fusing the multiple regularized classification maps. The CRMSF approach has been evaluated on four popular hyperspectral image data sets, and the experimental results show the advantages of CRMSF, particularly for a hyperspectral image with high spatial resolution. [ABSTRACT FROM AUTHOR]

Since the spatial distribution of surface materials is usually regular and locally continuous, it is reasonable to utilize the spectral and spatial information for the hyperspectral image classification. In this paper, a spectral–spatial Gabor surface feature (GSF) fusion approach has been proposed for hyperspectral image classification. First, Gabor magnitude pictures (GMPs) are extracted by applying a set of predefined 2-D Gabor filters to hyperspectral images. Second, the GSF has been extended to the spectral–spatial domains to comply with the 3-D structure of hyperspectral imagery, called 3-DGSF, which utilizes the first-order derivative of GMPs. Meanwhile, a classic superpixel segmentation method, called simple linear iterative clustering (SLIC), is adopted to divide the original hyperspectral image into disjoint superpixels. Third, principal component analysis is adopted to reduce the dimensionality of each extracted 3-DGSF feature cube. Next, a support vector machine classifier is applied on each reduced 3-DGSF features, and the majority voting strategy is used to obtain the classification results. Finally, the superpixel map obtained by SLIC is used to regularize the classification map, and thus, the proposed approach is named as S3-DGSF. Extensive experiments on three real hyperspectral data sets have demonstrated the higher performance of the proposed S3-DGSF approach over several state-of-the-art methods in the literature. [ABSTRACT FROM AUTHOR]

This paper presents an advanced urban traffic density estimation solution using the latest deep learning techniques to intelligently process ultrahigh-resolution traffic videos taken from an unmanned aerial vehicle (UAV). We first capture nearly an hour-long ultrahigh-resolution traffic video at five busy road intersections of a modern megacity by flying a UAV during the rush hours. We then randomly sampled over 17 K 512×512 pixel image patches from the video frames and manually annotated over 64 K vehicles to form a dataset for this paper, which will also be made available to the research community for research purposes. Our innovative urban traffics analysis solution consists of an advanced deep neural network (DNN) based vehicle detection and localization, type (car, bus, and truck) recognition, tracking, and vehicle counting over time. We will present extensive experimental results to demonstrate the effectiveness of our solution. We will show that our enhanced single shot multibox detector (Enhanced-SSD) outperforms other DNN-based techniques and that deep learning techniques are more effective than traditional computer vision techniques in traffic video analysis. We will also show that ultrahigh-resolution video provides more information that enables more accurate vehicle detection and recognition than lower resolution contents. This paper not only demonstrates the advantages of using the latest technological advancements (ultrahigh-resolution video and UAV), but also provides an advanced DNN-based solution for exploiting these technological advancements for urban traffic density estimation. [ABSTRACT FROM AUTHOR]

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial–spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial–spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial–spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3 approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training. [ABSTRACT FROM AUTHOR]