Search

As pioneering information technology, the Internet of Things (loT) targets at building an infrastructure of embedded devices and networks of connected objects, to offer omnipresent ecosystem and interaction across billions of smart devices, sensors, and actuators. The deployment of IoT calls for decentralized power supplies, self-powered sensors, and wireless transmission technologies, which have brought both opportunities and challenges to the existing solutions, especially when the network scales up. The Triboelectric Nanogenerators (TENGs), recently developed for mechanical energy harvesting and mechanical-to-electrical signal conversion, have the natural properties of energy and information, which have demonstrated high potentials in various applications of IoT. This context provides a comprehensive review of TENG enabled IoT and discusses the most popular and significant divisions. Firstly, the basic principle of TENG is re-examined in this article. Subsequently, a comprehensive and detailed review is given to the concept of IoT, followed by the scientific development of the TENG enabled IoT. Finally, the future of this evolving area is addressed.

In the time of Big Data, training complex models on large-scale data sets is challenging, making it appealing to reduce data volume for saving computation resources by subsampling. Most previous works in subsampling are weighted methods designed to help the performance of subset-model approach the full-set-model, hence the weighted methods have no chance to acquire a subset-model that is better than the full-set-model. However, we question that how can we achieve better model with less data? In this work, we propose a novel Unweighted Influence Data Subsampling (UIDS) method, and prove that the subset-model acquired through our method can outperform the full-set-model. Besides, we show that overly confident on a given test set for sampling is common in Influence-based subsampling methods, which can eventually cause our subset-model's failure in out-of-sample test. To mitigate it, we develop a probabilistic sampling scheme to control the worst-case risk over all distributions close to the empirical distribution. The experiment results demonstrate our methods superiority over existed subsampling methods in diverse tasks, such as text classification, image classification, click-through prediction, etc., Comment: AAAI 2020

In this paper, we propose a novel regression analysis approach, called maximal correlation regression, by exploiting the ideas from the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation. We show that in supervised learning problems, the optimal weights in maximal correlation regression can be expressed analytically with the relationships to the HGR maximal correlation functions, which reveals theoretical insights for our approach. In addition, we apply the maximal correlation regression to deep learning, in which efficient training algorithms are proposed for learning the weights in hidden layers. Furthermore, we illustrate that the maximal correlation regression is deeply connected to several existing approaches in information theory and machine learning, including the universal feature selection problem, linear discriminant analysis, and the softmax regression. Finally, experiments on real datasets demonstrate that our approach can obtain performance comparable to the widely used softmax regression based-method.

The Oja's algorithm is widely applied for computing principal eigenvectors in real problems, and it is practically useful to understand the theoretical relationships between the learning rate, convergence rate, and generalization error of this algorithm for noisy samples. In this paper, we show that under mild assumptions of sampling noise, both the generalization error and the convergence rate reveal linear relationships with the learning rate in the large sample size and small learning rate regime. In addition, when the algorithm nearly converges, we provide a refined characterization of the generalization error, which suggests the optimal design for the learning rate for data with noise. Moreover, we investigate the minibatch variation of Oja's algorithm and demonstrate that the learning rate of minibatch training is decayed by a factor characterized by the batch size, which provides theoretical insights and guidance for designing the learning rate in minibatch training algorithms. Finally, our theoretical results are validated by experiments on both synthesized data and the MNIST dataset.

Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks. An important question is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task. Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning. In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems using statistical and information theoretic principles. Experiments on real image data show that our metric is not only consistent with the empirical transferability measurement, but also useful to practitioners in applications such as source model selection and task transfer curriculum learning.

Cross-modal retrieval is a classic task in the multimedia community, which aims to search for semantically similar results from different modalities. The core of cross-modal retrieval is to learn the most correlated features in a common feature space for the multi-modal data so that the similarity can be directly measured. In this paper, we propose a novel model using optimal transport for bridging the heterogeneity gap in cross-modal retrieval tasks. Specifically, we calculate the optimal transport plans between feature distributions of different modalities and then minimize the transport cost by optimizing the feature embedding functions. In this way, the feature distributions of multi-modal data can be well aligned in the common feature space. In addition, our model combines the complementary losses in different levels: 1) semantic level, 2) distributional level, and 3) pairwise level for improving cross-modal retrieval performance. In extensive experiments, our method outperforms many other cross-modal retrieval methods, which proves the efficacy of using optimal transport in cross-modal retrieval tasks.

Accurately and continuously measuring and collecting data on human gait is critical for human activity recognition and individual identification, enabling various applications in smart homes/buildings, including security authentication, personal healthcare, and intelligent automation. Many sensing technologies have been investigated by researchers recently, such as camera-based, laser-based, and mobile approaches, which have limitations in particular sensing situations, such as environments with fewer privacy concerns, line-of-sight, and the use of wearables, etc. On the other hand, the floor with the embedded sensor is stable and robust to different circumstances, enabling non-intrusive gait recognition and human identification. Therefore, a triboelectric nanogenerator (TENG)-based gait sensor system installed on the floor is proposed in this paper. Our approach has many advantages in comparison to the existing gait recognition systems, including low cost, simple fabrication, lightweight, and high durability. The TENG-based sensors can be simply embedded into a smart carpet to discern mechanical motions through electrical signals. Furthermore, a deep learning model, deep residual bidirectional long short-term memory network with dense layers (Residual Dense-BiLSTM), is proposed for multichannel floor-based gait recognition. By utilizing this model to analyze the electrical outputs, our system can accurately detect various human activities and distinguish different individuals’ walking patterns, with a recognition rate over 98% and 97%, respectively. We conclude that the proposed deep learning enabled triboelectric gait sensor system has broad applications in security and healthcare.

Distributed learning is recently an important research topic, while the information theoretic optimality of the distributed learning algorithms is often not sufficiently addressed. This paper studies the distributed learning problems such that each node observes i.i.d. samples and sends a feature function of observed samples to the central machine for decision making. Both the binary hypothesis testing in information theory and the classification problems in machine learning are considered, and the optimal error exponent and the set of optimal features are characterized. By exploiting an information theoretic framework, we show that these two problems share the same set of optimal features, from which the information theoretic optimality of some machine learning algorithms can be established. Finally, we generalize our analyses to $M$ -ary distributed hypothesis testing and classification problems. A full version of this paper is accessible at: https://xiangxiangxu.com/media/documents/isit2021.pdf

Audio-visual emotion recognition is important for human-machine interaction systems by combining the information of audio and visual modalities. Although great progress has been made by previous works using multimodal learning compared with unimodal learning, they still cannot effectively deal with two key challenges. Firstly, it is difficult or expensive to acquire labeled emotional data, which results in a large amount of data with missing labels. Secondly, emotional data often has missing modalities. To address these problems, we propose a unified deep learning framework to efficiently handle missing labels and missing modalities for audio-visual emotion recognition through correlation analysis. Specifically, we consider four types of emotional data during the training stage: complete, label missing, visual missing, and audio missing. We propose a correlation loss based on Hirschfeld-Gebelein-Ŕenyi (HGR) maximal correlation to effectively capture the common information in different types of training data for emotion prediction. Experiments on the eNTERFACE’05 and RAVDESS datasets show that our deep learning approach has high effectiveness for audio-visual emotion recognition.

Missing data is a common problem in multimodal and multi-view learning. It raises a critical challenge for most multimodal algorithms, which are unable to deal with incomplete datasets. Rather than discarding entries with missing modalities, this paper aims to reconstruct the complete image-based multimodal data by imputing missing modalities. We solve the imputation problem as an image translation task, which transforms images in one domain to other domains. Existing image translation techniques either can not fully utilize the information contained in partially complete entries or are limited to the bimodal situation. We propose a semi-supervised algorithm for multimodal learning with missing data, namely Cyclic Autoencoder (CycAE). Specifically, a novel cyclical structure, as well as the correlation among modalities, is integrated to leverage infoπnation from complete entries to incomplete ones. Experiments on two multimodal datasets show that our model outperforms state-of-the-art models. Downstream tasks can also benefit from the completed datasets.

City management plays an important role in the era of urbanization. Understanding city regions and urban mobility patterns are two vital aspects of city management. Numerous studies have been conducted on these two aspects respectively. However, few work has considered combining city region partition and mobility pattern mining together while these two problems are closely related. In this paper, we propose region-aware mobility pattern mining framework, which jointly finds the precise origin and destination region partitions while extracting mobility patterns. We formulate it as an optimization problem of maximizing OD’s correlations with spatial constraints. Kernelized ACE, is proposed to solve the problem by learning feature representations that guarantee both objectives. Evaluation results using Beijing’s taxi data show that the extracted features are appropriate for this problem and our approach outperforms all the other methods with ∼ 0.3% spatial overlap and 86.43% OD correlation. Our case studies on New York City’s urban dynamics and Beijing’s three-year consecutive analysis also yield insightful findings that reveal city-scale mobility patterns and propose potential improvement for city management.

The training efficiency of distributed learning systems is vulnerable to stragglers, namely, those slow worker nodes. A naive strategy is performing the distributed learning by incor-porating the fastest K workers and ignoring these stragglers, which may induce high deviation for non-IID data. To tackle this, we develop a Live Gradient Compensation (LGC) strategy to incorporate the one-step delayed gradients from stragglers, aiming to accelerate learning process and utilize the stragglers simultaneously. In LGC framework, mini-batch data are divided into smaller blocks and processed separately, which makes the gradient computed based on partial work accessible. In addition, we provide theoretical convergence analysis of our algorithm for non-convex optimization problem under non-IID training data to show that LGC-SGD has almost the same convergence error as full synchronous SGD. The theoretical results also allow us to quantify a novel tradeoff in minimizing training time and error by selecting the optimal straggler threshold. Finally, extensive simulation experiments of image classification on CIFAR-10 dataset are conducted, and the numerical results demonstrate the effectiveness of our proposed strategy.

Multimodal person recognition is a common task in video analysis and public surveillance, where information from multiple modalities, such as images and audio extracted from videos, are used to jointly determine the identity of a person. Previous person recognition techniques either use only uni-modal data or only consider shared representations between different input modalities, while leaving the extraction of their relationship with identity information to downstream tasks. Furthermore, real-world data often contain noise, which makes recognition more challenging practical situations. In our work, we propose a novel correlation-based multimodal person recognition framework that is relatively simple but can efficaciously learn supervised information in multimodal data fusion and resist noise. Specifically, our framework learns a discriminative embeddings of persons by joint learning visual features and audio features while maximizing HGR maximal correlation among multimodal input and persons' identities. Experiments are done on a subset of Voxceleb2. Compared with state-of-the-art methods, the proposed method demonstrates an improvement of accuracy and robustness to noise.

Transfer learning across heterogeneous data distributions (a.k.a. domains) and distinct tasks is a more general and challenging problem than conventional transfer learning, where either domains or tasks are assumed to be the same. While neural network based feature transfer is widely used in transfer learning applications, finding the optimal transfer strategy still requires time-consuming experiments and domain knowledge. We propose a transferability metric called Optimal Transport based Conditional Entropy (OTCE), to analytically predict the transfer performance for supervised classification tasks in such cross-domain and cross-task feature transfer settings. Our OTCE score characterizes transferability as a combination of domain difference and task difference, and explicitly evaluates them from data in a unified framework. Specifically, we use optimal transport to estimate domain difference and the optimal coupling between source and target distributions, which is then used to derive the conditional entropy of the target task (task difference). Experiments on the largest cross-domain dataset DomainNet and Office31 demonstrate that OTCE shows an average of 21% gain in the correlation with the ground truth transfer accuracy compared to state-of-the-art methods. We also investigate two applications of the OTCE score including source model selection and multi-source feature fusion., Comment: 13 pages, accepted by CVPR2021

Current deep learning based disease diagnosis systems usually fall short in catastrophic forgetting, i.e., directly fine-tuning the disease diagnosis model on new tasks usually leads to abrupt decay of performance on previous tasks. What is worse, the trained diagnosis system would be fixed once deployed but collecting training data that covers enough diseases is infeasible, which inspires us to develop a lifelong learning diagnosis system. In this work, we propose to adopt attention to combine medical entities and context, embedding episodic memory and consolidation to retain knowledge, such that the learned model is capable of adapting to sequential disease-diagnosis tasks. Moreover, we establish a new benchmark, named Jarvis-40, which contains clinical notes collected from various hospitals. Experiments show that the proposed method can achieve state-of-the-art performance on the proposed benchmark. Code is available at https://github.com/yifyang/LifelongLearningDiseaseDiagnosis.

Audio-visual emotion recognition aims to distinguish human emotional states by integrating the audio and visual data acquired in the expression of emotions. It is crucial for facilitating the affect-related human-machine interaction system by enabling machines to intelligently respond to human emotions. One challenge of this problem is how to efficiently extract feature representations from audio and visual modalities. Although progresses have been made by previous works, most of them ignore common information between audio and visual data during the feature learning process, which may limit the performance since these two modalities are highly correlated in terms of their emotional information. To address this issue, we propose a deep learning approach in order to efficiently utilize common information for audio-visual emotion recognition by correlation analysis. Specifically, we design an audio network and a visual network to extract the feature representations from audio and visual data respectively, and then employ a fusion network to combine the extracted features for emotion prediction. These neural networks are trained by a joint loss, combining: (i) the correlation loss based on Hirschfeld-Gebelein-Re&acute, nyi (HGR) maximal correlation, which extracts common information between audio data, visual data, and the corresponding emotion labels, and (ii) the classification loss, which extracts discriminative information from each modality for emotion prediction. We further generalize our architecture to the semi-supervised learning scenario. The experimental results on the eNTERFACE&rsquo, 05 dataset, BAUM-1s dataset, and RAVDESS dataset show that common information can significantly enhance the stability of features learned from different modalities, and improve the emotion recognition performance.

With the rapid growth of multimedia data, the cross-modal retrieval problem has attracted a lot of interest in both research and industry in recent years. However, the inconsistency of data distribution from different modalities makes such task challenging. In this paper, we propose Semantically Supervised Maximal Correlation (S2MC) method for cross-modal retrieval by incorporating semantic label information into the traditional maximal correlation framework. Combining with maximal correlation based method for extracting unsupervised pairing information, our method effectively exploits supervised semantic information on both common feature space and label space. Extensive experiments show that our method outperforms other current state-of-the-art methods on cross-modal retrieval tasks on three widely used datasets.

We propose a Healthcare Graph Convolutional Network (HealGCN) to offer disease self-diagnosis service for online users based on Electronic Healthcare Records (EHRs). Two main challenges are focused in this paper for online disease diagnosis: (1) serving cold-start users via graph convolutional networks and (2) handling scarce clinical description via a symptom retrieval system. To this end, we first organize the EHR data into a heterogeneous graph that is capable of modeling complex interactions among users, symptoms and diseases, and tailor the graph representation learning towards disease diagnosis with an inductive learning paradigm. Then, we build a disease self-diagnosis system with a corresponding EHR Graph-based Symptom Retrieval System (GraphRet) that can search and provide a list of relevant alternative symptoms by tracing the predefined meta-paths. GraphRet helps enrich the seed symptom set through the EHR graph when confronting users with scarce descriptions, hence yield better diagnosis accuracy. At last, we validate the superiority of our model on a large-scale EHR dataset.

While the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation and the Wyner common information share similar information processing purposes of extracting common knowledge structures between random variables, the relationships between these approaches are generally unclear. In this paper, we demonstrate such relationships by considering the Wyner common information in the weakly dependent regime, called ϵ-common information. We show that the HGR maximal correlation functions coincide with the relative likelihood functions of estimating the auxiliary random variables in ϵ-common information, which establishes the fundamental connections these approaches. Moreover, we extend the ϵ-common information to multiple random variables, and derive a novel algorithm for extracting feature functions of data variables regarding their common information. Our approach is validated by the MNIST problem, and can potentially be useful in multi-modal data analyses.

Anomaly detection is an important area for both research and real-world applications. In the surface mounting technology (SMT) process, the defectives of solder paste printing need to be detected immediately or it may cause great effort for recycling and slow down the whole process. In this paper, we propose a novel model, MM-DNN, for anomaly detection with multi-modal data. We collect a multi-modal dataset from different sensors in the factory. Our method efficiently extracts both predictive features for classification and correlative features between multi-modal data to achieve a higher detection rate. As shown in the experiment, our method can further reduce 77% false alarm rate of the detection result in the factory while keeping 95% of real defectives be correctly detected.

Multilabel learning, the problem of mapping each data instance to a subset of labels, appears frequently in many real-world applications. However, obtaining complete label annotation for every instance requires tremendous efforts, especially when the label set is large. As a result, multilabel learning with missing labels remains as a common challenge. Existing works either cannot handle missing labels or lack nonlinear expressiveness and scalability to large label set. In this paper, we present a novel end-to-end solution for multilabel learning with missing labels. Our algorithm, Maximal Correlation Embedding Network learns a low dimensional label embedding using an encoder-decoder architecture. It exploits label similarity through a maximal correlation regularization in the embedded label space to reduce the classification bias due to missing labels. A series of experiments on popular multilabel datasets demonstrate that our approach outperforms state of the art, both in complete data and partially observed data.

It is commonly believed that the hidden layers of deep neural networks (DNNs) attempt to extract informative features for learning tasks. In this paper, we formalize this intuition by showing that the features extracted by DNN coincide with the result of an optimization problem, which we call the `universal feature selection' problem, in a local analysis regime. We interpret the weights training in DNN as the projection of feature functions between feature spaces, specified by the network structure. Our formulation has direct operational meaning in terms of the performance for inference tasks, and gives interpretations to the internal computation results of DNNs. Results of numerical experiments are provided to support the analysis., Accepted to ISIT 2019

Multimodal emotion recognition is important for facilitating efficient interaction between humans and machines. To better detect emotional states from multimodal data, we need to effectively extract both the common information that captures dependencies among different modalities, and the private information that characterizes variations in each modality. However, existing works are mostly designed to pursue either one of these objectives but not both. In our work, we propose an end-to-end learning approach to simultaneously extract the common and private information for multimodal emotion recognition. Specifically, we use a correlation loss based on Hirschfeld-Gebelein-Renyi (HGR) maximal correlation and a reconstruction loss based on autoencoders to preserve the common and private information, respectively. Experimental results on eNTERFACE'05 database and RML database demonstrate the effectiveness of our proposed approach.

Unsupervised anomaly detection is a fundamental problem in various research areas and application domains, namely the discrimination of abnormal samples from normal samples where training data are only composed of one class (normal) while testing data contains both among which the majority are normal samples. However, previous works can not effectively fit the distribution of high dimensional data and suffers from low AUC scores which measures the classification performance of imbalanced data. To solve these problems, we propose an unsupervised anomaly detection model based on GAN, i.e., UAD-GAN. Specifically, we adopt transfer learning to extract visual features with pre-trained Inception-v3 model and use the discriminator to detect anomalies. UAD-GAN can fit the data distribution and detect anomalies efficiently. Extensive experiments show that UAD-GAN achieves state-of-the-art performance compared to other approaches.

Outlier detection is one of major task in unsupervised learning. We propose a cluster analysis based outlier detection method called Info-Detection. Info-Detection determines the number of outliers automatically and captures the global property of the provided data. To implement Info-Detection and overcome the global computational complexity, we use principal sequence of partition, which we improve one order of magnitude faster than the original version. Experiments show that compared with other outlier detection methods, Info-Detection achieves better accuracy with an affordable time overhead.

The reproducibility of scientific experiment is vital for the advancement of disciplines based on previous work. To achieve this goal, many researchers focus on complex methodology and self-invented tools which have difficulty in practical usage. In this article, we introduce the Cloud DevOps infrastructure from software engineering community and shows how it can be used effectively for heterogeneous agents to reproduce experiments for computer science related disciplines. DevOps can be enabled using freely available cloud computing machines for medium-sized experiment and self-hosted computing engines for large-scale computing, thus powering researchers to share their experiment result with others in a more reliable way.

In this paper, we extend the information theoretic framework that was developed in earlier work to multi-hop network settings. For a given network, we construct a novel deterministic model that quantifies the ability of the network in transmitting private and common messages across users. Based on this model, we formulate a linear optimization problem that explores the throughput of a multi-layer network, thereby offering the optimal strategy as to what kind of common messages should be generated in the network to maximize the throughput. With this deterministic model, we also investigate the role of feedback for multi-layer networks, from which we identify a variety of scenarios in which feedback can improve transmission efficiency. Our results provide fundamental guidelines as to how to coordinate cooperation between users to enable efficient information exchanges across them., to appear in the IEEE Transactions on Information Theory

Mobility pattern mining answers the fundamental question of where people are likely to go from a given location. It plays an important role in city planning, public transport management and location-based mobile applications. Among these applications, many concern the mobility pattern over contiguous spatial regions as a whole. Traditional ways of mobility pattern mining either result in trip clusters with overlapped origin and destination regions, or require an extra step to partition the city into discrete regions, which may not be optimal for mobility pattern extraction. In this paper, we present a region-aware mobility pattern mining framework to jointly extract trip clusters while maintaining non-overlapping partitions of trip origins and destinations. We developed kernelized ACE, a novel extension to a classic algorithm in statistics to compute the optimal mobility clusters under spatial constraints. Experimental results using Beijing taxi trip data show that our approach outperforms other methods with only ~ 0.3% spatial overlap and 86.43% origin-destination correlation. Our case studies on New York City's and Beijing's taxi datasets also yield insightful findings that reveal city-scale mobility patterns and propose potential improvement for public transportation.

Speech unlocks the huge potentials in emotion recognition. High accurate and real-time understanding of human emotion via speech assists Human-Computer Interaction. Previous works are often limited in either coarse-grained emotion learning tasks or the low precisions on the emotion recognition. To solve these problems, we construct a real-world large-scale corpus composed of 4 common emotions (i.e., anger, happiness, neutral and sadness). We also propose a multi-task attention-based DNN model (i.e., MT-A-DNN) on the emotion learning. MT-A-DNN efficiently learns the high-order dependency and non-linear correlations underlying in the audio data. Extensive experiments show that MT-A-DNN outperforms conventional methods on the emotion recognition. It could take one step further on the real-time acoustic emotion recognition in many smart audio-devices.

Emotion recognition technologies have been widely used in numerous areas including advertising, healthcare and online education. Previous works usually recognize the emotion from either the acoustic or the visual signal, yielding unsatisfied performances and limited applications. To improve the inference capability, we present a multimodal emotion recognition model, EMOdal. Apart from learning the audio and visual data respectively, EMOdal efficiently learns the common and modality-specific information underlying the two kinds of signals, and therefore improves the inference ability. The model has been evaluated on our large-scale emotional data set. The comprehensive evaluations demonstrate that our model outperforms traditional approaches.

Time series classification is a critical problem in the machine learning field, which spawns numerous research works on it. In this work, we propose AttLSTM-CNNs, an attention-based LSTM network and convolution network that jointly extracts the underlying pattern among the time-series for the classification. The attention-based LSTM automatically captures the long-term temporal dependency among the series, and the CNN describes the spatial sparsity and heterogeneity in the data. The extensive experiments show that the proposed model outperforms the other methods for time-series classification.

In this paper, we formulate the generalized softmax learning (GSL) problem, as a symmetric extension of the softmax regression problem. We further study the geometric structure of GSL and demonstrate the equivalence of GSL and the original softmax regression problem. Besides, this geometric structure indicates the symmetry between a neural network and its reverse network, and the symmetric roles of the weights and feature in a neural network. Finally, we present a numerical simulation to verify these symmetry properties in neural networks.

The miniaturization of spectrometer can broaden the application area of spectrometry, which has huge academic and industrial value. Among various miniaturization approaches, filter-based miniaturization is a promising implementation by utilizing broadband filters with distinct transmission functions. Mathematically, filter-based spectral reconstruction can be modeled as solving a system of linear equations. In this paper, we propose an algorithm of spectral reconstruction based on sparse optimization and dictionary learning. To verify the feasibility of the reconstruction algorithm, we design and implement a simple prototype of a filter-based miniature spectrometer. The experimental results demonstrate that sparse optimization is well applicable to spectral reconstruction whether the spectra are directly sparse or not. As for the non-directly sparse spectra, their sparsity can be enhanced by dictionary learning. In conclusion, the proposed approach has a bright application prospect in fabricating a practical miniature spectrometer.

Widespread use of the Internet and social networks invokes the generation of big data, which is proving to be useful in a number of applications. To deal with explosively growing amounts of data, data analytics has emerged as a critical technology related to computing, signal processing, and information networking. In this paper, a formalism is considered in which data is modeled as a generalized social network and communication theory and information theory are thereby extended to data analytics. First, the creation of an equalizer to optimize information transfer between two data variables is considered, and financial data is used to demonstrate the advantages. Then, an information coupling approach based on information geometry is applied for dimensionality reduction, with a pattern recognition example to illustrate the effectiveness. These initial trials suggest the potential of communication theoretic data analytics for a wide range of applications., Published in IEEE Journal on Selected Areas in Communications, Jan. 2015

In this paper, we propose an information-theoretic approach to design the functional representations to extract the hidden common structure shared by a set of random variables. The main idea is to measure the common information between the random variables by Watanabe's total correlation, and then find the hidden attributes of these random variables such that the common information is reduced the most given these attributes. We show that these attributes can be characterized by an exponential family specified by the eigen-decomposition of some pairwise joint distribution matrix. Then, we adopt the log-likelihood functions for estimating these attributes as the desired functional representations of the random variables, and show that such representations are informative to describe the common structure. Moreover, we design both the multivariate alternating conditional expectation (MACE) algorithm to compute the proposed functional representations for discrete data, and a novel neural network training approach for continuous or high-dimensional data. Furthermore, we show that our approach has deep connections to existing techniques, such as Hirschfeld-Gebelein-R\'{e}nyi (HGR) maximal correlation, linear principal component analysis (PCA), and consistent functional map, which establishes insightful connections between information theory and machine learning. Finally, the performances of our algorithms are validated by numerical simulations., Comment: 35 pages; Submitted to IEEE Journal on Selected Areas in Information Theory (JSAIT)

We develop an information theoretic framework for addressing feature selection in applications where the inference task is not specified in advance and the data is from a large alphabet. We introduce a natural notion of universality for such problems, and show that locally optimal solutions are straight forward to obtain, admit natural interpretations via information geometry, have computationally efficient implementations, and represent a practically useful learning methodology. Our development also reveals the key role of Hirschfeld-Gebelein-Renyi maximal correlation and the alternating conditional expectations (ACE) algorithm in such problems.

Understanding human drivers' behavior is critical for the self-driving cars, and has been intensively studied in the past decade. We exploit the widely available camera and motion sensor data from car recorders, and propose a hybrid method of recognizing driving events based on the random forest approach. The classification results are analyzed by comparing different features, classifiers and filters. A high accuracy of 98.1% on driving behavior classification is obtained and the robustness is verified on a dataset including 2400 driving events.

The accelerating urbanization procedure is putting increasing pressure on the management of cities. The administrative zones by which a city is managed are setup based on historical or political reasons, while the dynamics of people is hardly considered in the context. We exploit the widely available mobility data to divide the urban areas into zones by the joint K-mean clustering in origin and destination spaces. The method is evaluated with the New York City and Shenzhen taxi data, and the created zones are compared with the current static zoning plans of the city to evaluate the effectiveness.

Statistical learning has attracted considerable recent research interest due to the wide-ranging demands of big data analytics. The recent introduction of communication theory and information coupling theory into this area suggests a new perspective on statistical learning and inference for data analytics. This paper investigates inference of one data variable from heterogeneous data variables, a problem that plays an increasingly important role in the emerging applications of big data analytics. To generalize the existing conceptual approach, information coupling filtering under hidden data structure or unknown knowledge of interactions among data variables is developed. A least-mean-squares (LMS) filtering approach for non-stationary data similar to an equalizer is suggested, while the training data gives the depth of the filter analogously to model selection in learning theory. The information combining in diversity communication is extended to fuse more data variables for even greater precision of inference. Extending from multiuser detection, an algorithm based on Multiple Signal Classification (MUSIC) is demonstrated to identify useful data variables for inference, as a novel solution to knowledge discovery. A series of examples illustrate the effectiveness of this framework, suggesting that statistical communication theory and statistical signal processing can substantially contribute to statistical learning theory.

In this paper, we investigate how to efficiently extract informative features of high-dimensional data through noisy channels. Specifically, we decompose the feature space of the data into a sequence of score functions with decreasing information volumes, such that different scores are uncorrelated. From this decomposition, the features of the data become a sequence of score functions such that the most informative lowdimensional feature can be selected as the first few scores. This greatly simplifies the feature selection problem. In addition, we apply this spectrum decomposition to data with high-dimensional structures, i.e., the hidden Markov model (HMM). We show that in HMM, it is desirable to consider a particular class of score functions called as the node scores, which allows us to efficiently extract informative features of the hidden variables by applying the spectrum decomposition approach. Finally, we develop efficient algorithms to extract such features from node scores, and present an example to illustrate the performance of the node scores.

Systematically analyzing the dynamic behaviors of social networks is one of the central topic in understanding the structure of large networks. In particular, the information cascade [1] introduced by Banerjee provides great insights in characterizing the opinion exchanging between network agents. Traditionally studies of information cascades focus on the Bayesian models, which are often difficult to model real world situations. In this paper, we attempt to study the information cascades from a non-Bayesian point of view. In particular, we consider a sequential decision model but with an arbitrary decision rule. We show that the fraction of agents in a network making any specific decision will converge. Thus, the agents in the network reach a sort of consensus with high probability, which allows us to predict the herd behaviors. In addition, we also apply our non-Bayesian model to different network structures, such as ER model and network with communities, in which the affect of information cascades are quantified. Finally, we simulate the decision process for multiple communities, which justifies our proposed model to comprehend real world complex user behaviors and dynamics.

With the boom of big data, traditional source coding techniques face the common obstacle to decode only a small portion of information efficiently. In this paper, we aim to resolve this difficulty by introducing a specific type of source coding scheme called locally decodable source coding (LDSC). Rigorously, LDSC is capable of recovering an arbitrary bit of the unencoded message from its encoded version, by only feeding a small number of the encoded message to the decoder, and we call the decoder t-local if only t encoded symbols are required.We consider both almost lossless (block error) and lossy (bit error) cases for LDSC. First, we show that using linear encoder and a decoder with bounded locality, the reliable compress rate can not be less than one. More importantly, we show that even with a general encoder and 2-local decoders (t = 2), the rate of LDSC is still one. On the contrary, the achievability bounds for almost lossless and lossy compressions with excess distortion suggest that optimal compression rate is achievable when O(log n) encoded symbols is queried by the decoder with block-length n. We also show that, rate distortion is achievable when the number of queries is scaled over n with a bound on the rate in finite-length regime. Although the achievability bounds are simply based on the concatenation of code blocks, they outperform the existing bounds in succinct data structures literature.