Search

In the realm of urban development, the precise classification and identification of land types are crucial for improving land use efficiency. This article proposes a land recognition and classification method based on data sparsity and improved Soft Hirschfeld-Gebelein-Rényi (Soft-HGR) under multimodal conditions. First, a sparse information processing module is designed to enhance information accuracy and quickly obtain data sample features. Then, to solve the problem of information independence in single mode and lack of fusion in multimodal mode, an improved SoftHGR module is developed. This module incorporates covariance and trace constraints, enhances machine learning efficiency by stabilizing output and addressing dimensionality and variance issues in HGR, and speeds up land classification by cross-fusing multimodal features to deepen the understanding of diverse information interconnections. Based on this, a multimodal MI-SoftHGR fusion network is constructed, which can achieve cross-correlation sharing and collaborative extraction of feature information, thereby realizing accurate remote sensing image recognition and classification under multimodal conditions. Finally, empirical evaluations were conducted on Berlin, Augsburg, and MUUFL datasets, and the proposed method was compared with state-of-the-art algorithms. The results fully validate the efficacy and significant superiority of the proposed method.

This paper investigates remote sensing data recognition and classification with multimodal data fusion. Aiming at the problems of low recognition and classification accuracy and the difficulty in integrating multimodal features in existing methods, a multimodal remote sensing data recognition and classification model based on a heatmap and Hirschfeld–Gebelein–Rényi (HGR) correlation pooling fusion operation is proposed. A novel HGR correlation pooling fusion algorithm is developed by combining a feature fusion method and an HGR maximum correlation algorithm. This method enables the restoration of the original signal without changing the value of transmitted information by performing reverse operations on the sample data. This enhances feature learning for images and improves performance in specific tasks of interpretation by efficiently using multi-modal information with varying degrees of relevance. Ship recognition experiments conducted on the QXS-SROPT dataset demonstrate that the proposed method surpasses existing remote sensing data recognition methods. Furthermore, land cover classification experiments conducted on the Houston 2013 and MUUFL datasets confirm the generalizability of the proposed method. The experimental results fully validate the effectiveness and significant superiority of the proposed method in the recognition and classification of multimodal remote sensing data.

While non-linear activation functions play vital roles in artificial neural networks, it is generally unclear how the non-linearity can improve the quality of function approximations. In this paper, we present a theoretical framework to rigorously analyze the performance gain of using non-linear activation functions for a class of residual neural networks (ResNets). In particular, we show that when the input features for the ResNet are uniformly chosen and orthogonal to each other, using non-linear activation functions to generate the ResNet output averagely outperforms using linear activation functions, and the performance gain can be explicitly computed. Moreover, we show that when the activation functions are chosen as polynomials with the degree much less than the dimension of the input features, the optimal activation functions can be precisely expressed in the form of Hermite polynomials. This demonstrates the role of Hermite polynomials in function approximations of ResNets.

Distributed hypothesis testing (DHT) has emerged as a significant research area, but the information-theoretic optimality of coding strategies is often typically hard to address. This paper studies the DHT problems under the type-based setting, which is requested from the popular federated learning methods. Specifically, two communication models are considered: (i) DHT problem over noiseless channels, where each node observes i.i.d. samples and sends a one-dimensional statistic of observed samples to the decision center for decision making; and (ii) DHT problem over AWGN channels, where the distributed nodes are restricted to transmit functions of the empirical distributions of the observed data sequences due to practical computational constraints. For both of these problems, we present the optimal error exponent by providing both the achievability and converse results. In addition, we offer corresponding coding strategies and decision rules. Our results not only offer coding guidance for distributed systems, but also have the potential to be applied to more complex problems, enhancing the understanding and application of DHT in various domains.

As pioneering information technology, the Internet of Things (IoT) targets at building an infrastructure of embedded devices and networks of connected objects, to offer omnipresent ecosystem and interaction across billions of smart devices, sensors, and actuators. The deployment of IoT calls for decentralized power supplies, self-powered sensors, and wireless transmission technologies, which have brought both opportunities and challenges to the existing solutions, especially when the network scales up. The Triboelectric Nanogenerators (TENGs), recently developed for mechanical energy harvesting and mechanical-to-electrical signal conversion, have the natural properties of energy and information, which have demonstrated high potentials in various applications of IoT. This context provides a comprehensive review of TENG enabled IoT and discusses the most popular and significant divisions. Firstly, the basic principle of TENG is re-examined in this article. Subsequently, a comprehensive and detailed review is given to the concept of IoT, followed by the scientific development of the TENG enabled IoT. Finally, the future of this evolving area is addressed.

In this paper, we propose a novel regression analysis approach, called maximal correlation regression, by exploiting the ideas from the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation. We show that in supervised learning problems, the optimal weights in maximal correlation regression can be expressed analytically with the relationships to the HGR maximal correlation functions, which reveals theoretical insights for our approach. In addition, we apply the maximal correlation regression to deep learning, in which efficient training algorithms are proposed for learning the weights in hidden layers. Furthermore, we illustrate that the maximal correlation regression is deeply connected to several existing approaches in information theory and machine learning, including the universal feature selection problem, linear discriminant analysis, and the softmax regression. Finally, experiments on real datasets demonstrate that our approach can obtain performance comparable to the widely used softmax regression based-method.

The Oja's algorithm is widely applied for computing principal eigenvectors in real problems, and it is practically useful to understand the theoretical relationships between the learning rate, convergence rate, and generalization error of this algorithm for noisy samples. In this paper, we show that under mild assumptions of sampling noise, both the generalization error and the convergence rate reveal linear relationships with the learning rate in the large sample size and small learning rate regime. In addition, when the algorithm nearly converges, we provide a refined characterization of the generalization error, which suggests the optimal design for the learning rate for data with noise. Moreover, we investigate the minibatch variation of Oja's algorithm and demonstrate that the learning rate of minibatch training is decayed by a factor characterized by the batch size, which provides theoretical insights and guidance for designing the learning rate in minibatch training algorithms. Finally, our theoretical results are validated by experiments on both synthesized data and the MNIST dataset.

It was hypothesized already more than 40 years ago that photoelectron spectroscopy should in principle be able to image atomic orbitals. If this can be made to work for orbitals in crystalline solids, one would have literally a different view on the electronic structure of a wide range of quantum materials. Here, we demonstrate how hard x-ray photoelectron spectroscopy can make direct images of the orbitals making up the band structure of our model system, ReO_{3}. The images are energy specific and enable us to unveil the role of each of those orbitals for the chemical bonding and the Fermi surface topology. The orbital image information is complementary to that from angle-resolved photoemission and thus completes the determination of the electronic structure of materials.

Audio-visual emotion recognition is the research of identifying human emotional states by combining the audio modality and the visual modality simultaneously, which plays an important role in intelligent human-machine interactions. With the help of deep learning, previous works have made great progress for audio-visual emotion recognition. However, these deep learning methods often require a large amount of data for training. In reality, data acquisition is difficult and expensive, especially for the multimodal data with different modalities. As a result, the training data may be in the low-data regime, which cannot be effectively used for deep learning. In addition, class imbalance may occur in the emotional data, which can further degrade the performance of audio-visual emotion recognition. To address these problems, we propose an efficient data augmentation framework by designing a multimodal conditional generative adversarial network (GAN) for audio-visual emotion recognition. Specifically, we design generators and discriminators for audio and visual modalities. The category information is used as their shared input to make sure our GAN can generate fake data of different categories. In addition, the high dependence between the audio modality and the visual modality in the generated multimodal data is modeled based on Hirschfeld-Gebelein-Rényi (HGR) maximal correlation. In this way, we relate different modalities in the generated data to approximate the real data. Then, the generated data are used to augment our data manifold. We further apply our approach to deal with the problem of class imbalance. To the best of our knowledge, this is the first work to propose a data augmentation strategy with a multimodal conditional GAN for audio-visual emotion recognition. We conduct a series of experiments on three public multimodal datasets, including eNTERFACE’05, RAVDESS, and CMEW. The results indicate that our multimodal conditional GAN has high effectiveness for data augmentation of audio-visual emotion recognition.

With the unprecedented performance achieved by deep learning, it is commonly believed that deep neural networks (DNNs) attempt to extract informative features for learning tasks. To formalize this intuition, we apply the local information geometric analysis and establish an information-theoretic framework for feature selection, which demonstrates the information-theoretic optimality of DNN features. Moreover, we conduct a quantitative analysis to characterize the impact of network structure on the feature extraction process of DNNs. Our investigation naturally leads to a performance metric for evaluating the effectiveness of extracted features, called the H-score, which illustrates the connection between the practical training process of DNNs and the information-theoretic framework. Finally, we validate our theoretical results by experimental designs on synthesized data and the ImageNet dataset.

In this paper, we study the phase transition property of an Ising model defined on a special random graph—the stochastic block model (SBM). Based on the Ising model, we propose a stochastic estimator to achieve the exact recovery for the SBM. The stochastic algorithm can be transformed into an optimization problem, which includes the special case of maximum likelihood and maximum modularity. Additionally, we give an unbiased convergent estimator for the model parameters of the SBM, which can be computed in constant time. Finally, we use metropolis sampling to realize the stochastic estimator and verify the phase transition phenomenon thfough experiments.

Audio-visual emotion recognition aims to distinguish human emotional states by integrating the audio and visual data acquired in the expression of emotions. It is crucial for facilitating the affect-related human-machine interaction system by enabling machines to intelligently respond to human emotions. One challenge of this problem is how to efficiently extract feature representations from audio and visual modalities. Although progresses have been made by previous works, most of them ignore common information between audio and visual data during the feature learning process, which may limit the performance since these two modalities are highly correlated in terms of their emotional information. To address this issue, we propose a deep learning approach in order to efficiently utilize common information for audio-visual emotion recognition by correlation analysis. Specifically, we design an audio network and a visual network to extract the feature representations from audio and visual data respectively, and then employ a fusion network to combine the extracted features for emotion prediction. These neural networks are trained by a joint loss, combining: (i) the correlation loss based on Hirschfeld-Gebelein-Rényi (HGR) maximal correlation, which extracts common information between audio data, visual data, and the corresponding emotion labels, and (ii) the classification loss, which extracts discriminative information from each modality for emotion prediction. We further generalize our architecture to the semi-supervised learning scenario. The experimental results on the eNTERFACE’05 dataset, BAUM-1s dataset, and RAVDESS dataset show that common information can significantly enhance the stability of features learned from different modalities, and improve the emotion recognition performance.

The miniaturization of spectrometer can broaden the application area of spectrometry, which has huge academic and industrial value. Among various miniaturization approaches, filter-based miniaturization is a promising implementation by utilizing broadband filters with distinct transmission functions. Mathematically, filter-based spectral reconstruction can be modeled as solving a system of linear equations. In this paper, we propose an algorithm of spectral reconstruction based on sparse optimization and dictionary learning. To verify the feasibility of the reconstruction algorithm, we design and implement a simple prototype of a filter-based miniature spectrometer. The experimental results demonstrate that sparse optimization is well applicable to spectral reconstruction whether the spectra are directly sparse or not. As for the non-directly sparse spectra, their sparsity can be enhanced by dictionary learning. In conclusion, the proposed approach has a bright application prospect in fabricating a practical miniature spectrometer.