Search

By using the traditional active E2E (end-to-end) detection method to locate the congested link in IP network, we can only judge whether the network is congestion or not, and the congested cause cannot be effectively known. However, the passive detection method needs to acquire the syslog of all routers in the managed network, which requires a large amount of data. In this paper, a kind of multimodal spatiotemporal correlation fuzzy inference method for network health monitoring method was proposed through combining with the method of active E2E detection and passive detection. First, the set of network-congested links is obtained based on the Bayesian theorem through E2E path performance detection; then, the router syslog associated with the congested links will be acquired for a period of time, and the abnormal events in the syslog are detected based on the principle of mutual information (MI), so as to find the causes of network congestion. Based on the multimodal spatiotemporal correlation fuzzy paradigms combining with the active end-to-end detection and passive detection technology, the amount of data analysis will be greatly reduced and the operation speed will be improved. Simulation experiments and actual experiments verified the accuracy of the method this paper proposed. [ABSTRACT FROM AUTHOR]

The real world data are complex in nature and addition to that a large number of features add more value to the complexity. However, the features associated with the data may be redundant and erroneous in nature. To deal with such type of features, feature selection plays a vital role in computational learning. The reduction in the dimensionality of the dataset not only reduces the computational time required for classification but also enhances the classification accuracy by removing the misleading features. This paper presents a Filter Approach using Elitism based Multi-objective Differential Evolution algorithm for feature selection (FAEMODE) and the novelty lies in the objective formulation, where both linear and nonlinear dependency among features have been considered to handle the redundant and unwanted features of a dataset. Finally, the selected feature subsets of 23 benchmark datasets are tested using 10-fold cross validation with four well-known classifiers to endorse the result. A comparative analysis of the proposed approach with seven filter approaches and two conventional as well as three metaheuristic based wrapper approaches have been carried out for validation. The result reveals that the proposed approach can be considered as a powerful filter method for feature selection in various fields. [ABSTRACT FROM AUTHOR]

The space weather effects at the Earth's magnetosphere are mostly driven by the solar wind that carries the interplanetary magnetic field (IMF). In this paper, we use 2 years of data in the solar wind from lunar orbiting ARTEMIS and MMS spacecraft upstream of the Earth's bow shock to study the structure of the IMF. We determine the lag times of IMF structures and their dependence on spacecraft positions by conducting an information theory analysis and comparing it with two traditional analysis methods: cross‐correlation (CC) analysis and minimum variance of magnetic field analysis (MVAB). For the events with long time intervals (i.e., >4 hr) and with small‐spatial separation between the MMS and ARTEMIS along the yGSM‐direction (i.e., <40Re, where Re is the Earth's radius), the lag times based on the CC and the mutual information (MI) analyses statistically agree with each other, with p‐values of 1.675 × 10−7 and 4.833 × 10−9, with the confidence of 95%. Both the results based on MI and CC have a large deviation from the results from MVAB. For some of the events, such a deviation could be improved by taking the fast mode speed into account; however, p‐tests showed that they were not statistically significant to the 95% confidence level. Plain Language Summary: Typically, single‐point measurements from the Earth‐Sun Lagrange Point‐1 are used to forecast space weather. However, it is important to understand that multi‐point measurements are required to further understand and predict geo‐effective solar wind structures. Three analysis methods using multi‐point measurements from the MMS and THEMIS missions of large solar wind structures as they move from regions around the Moon to regions around Earth's bow shock, are presented and compared in this paper. Information theory analysis (specifically Mutual Information) determines dependence between two variables by quantifying the entropy (uncertainty) in the two variables. Minimum variance analysis calculates the direction of least change in a magnetic field and assumes the solar wind is propagating in that direction. Lag time is calculated by taking the dot products of this direction with both the displacement vector between the two spacecraft, to get a distance, and the ion velocity (from ARTEMIS) to get a speed. The distance is divided by the speed to get a lag time. Cross‐Correlation analysis is a traditional method for measuring similarity between two variables. This paper shows that information theory analysis along with cross‐correlation analysis could be used successfully for space weather forecasting. Key Points: The solar wind data by MMS and THEMIS are analyzed by using information theory to understand the propagation of solar wind structuresMutual information, linear cross‐correlation, and minimum variance analyses sometimes give different lag‐timesInformation theory analysis along with the traditional method of cross‐correlation analysis could be used successfully for space weather forecasting [ABSTRACT FROM AUTHOR]

Time series data have characteristics such as high dimensionality, excessive noise, data imbalance, etc. In the data preprocessing process, feature selection plays an important role in the quantitative analysis of multidimensional time series data. Aiming at the problem of feature selection of multidimensional time series data, a feature selection method for time series based on mutual information (MI) is proposed. One of the difficulties of traditional MI methods is in searching for a suitable target variable. To address this issue, the main innovation of this paper is the hybridization of principal component analysis (PCA) and kernel regression (KR) methods based on MI. Firstly, based on historical operational data, quantifiable system operability is constructed using PCA and KR. The next step is to use the constructed system operability as the target variable for MI analysis to extract the most useful features for the system data analysis. In order to verify the effectiveness of the method, an experiment is conducted on the CMAPSS engine dataset, and the effectiveness of condition recognition is tested based on the extracted features. The results indicate that the proposed method can effectively achieve feature extraction of high-dimensional monitoring data. [ABSTRACT FROM AUTHOR]

Emotion recognition using brain wave signals involves using high dimensional electroencephalogram (EEG) data. In this paper, a window selection method based on mutual information is introduced to select an appropriate signal window to reduce the length of the signals. The motivation of the windowing method comes from EEG emotion recognition being computationally costly and the data having low signal-to-noise ratio. The aim of the windowing method is to find a reduced signal where the emotions are strongest. In this paper, it is suggested, that using only the signal section which best describes emotions improves the classification of emotions. This is achieved by iteratively comparing different-length EEG signals at different time locations using the mutual information between the reduced signal and emotion labels as criterion. The reduced signal with the highest mutual information is used for extracting the features for emotion classification. In addition, a viable framework for emotion recognition is introduced. Experimental results on publicly available datasets, DEAP and MAHNOB-HCI, show significant improvement in emotion recognition accuracy. [ABSTRACT FROM AUTHOR]

SiamFC++ only extracts the object feature of the first frame as a tracking template, and only uses the highest level feature maps in both the classification branch and the regression branch, so that the respective characteristics of the two branches are not fully utilized. In view of this, the present paper proposes an object tracking algorithm based on SiamFC++. The algorithm uses the multi-layer features of the Siamese network to update template. First, FPN is used to extract feature maps from different layers of Backbone for classification branch and regression branch. Second, 3D convolution is used to update the tracking template of the object tracking algorithm. Next, a template update judgment condition is proposed based on mutual information. Finally, AlexNet is used as the backbone and GOT-10K as training set. Compared with SiamFC++, our algorithm obtains improved results on OTB100, VOT2016, VOT2018 and GOT-10k data sets, and the tracking process is real time. [ABSTRACT FROM AUTHOR]

Background: Construction of co-occurrence networks in metagenomic data often employs correlation to infer pairwise relationships between microbes. However, biological systems are complex and often display qualities non-linear in nature. Therefore, the reliance on correlation alone may overlook important relationships and fail to capture the full breadth of intricacies presented in underlying interaction networks. It is of interest to incorporate metrics that are not only robust in detecting linear relationships, but non-linear ones as well. Results: In this paper, we explore the use of various mutual information (MI) estimation approaches for quantifying pairwise relationships in biological data and compare their performances against two traditional measures–Pearson's correlation coefficient, r, and Spearman's rank correlation coefficient, ρ. Metrics are tested on both simulated data designed to mimic pairwise relationships that may be found in ecological systems and real data from a previous study on C. diff infection. The results demonstrate that, in the case of asymmetric relationships, mutual information estimators can provide better detection ability than Pearson's or Spearman's correlation coefficients. Specifically, we find that these estimators have elevated performances in the detection of exploitative relationships, demonstrating the potential benefit of including them in future metagenomic studies. Conclusions: Mutual information (MI) can uncover complex pairwise relationships in biological data that may be missed by traditional measures of association. The inclusion of such relationships when constructing co-occurrence networks can result in a more comprehensive analysis than the use of correlation alone. [ABSTRACT FROM AUTHOR]

The Averaged One-Dependence Estimators (AODE) is a popular and effective method of Bayesian classification. In AODE, selecting the optimal sub-model based on a cross-validated risk minimization strategy can further enhance classification performance. However, existing cross-validation risk minimization strategies do not consider the differences in attributes in classification decisions. Consequently, this paper introduces an algorithm for Model Selection-based Weighted AODE (SWAODE). To express the differences in attributes in classification decisions, the ODE corresponding to attributes are weighted, with mutual information commonly used in the field of machine learning adopted as weights. Then, these weighted sub-models are evaluated and selected using leave-one-out cross-validation (LOOCV) to determine the best model. The new method can improve the accuracy and robustness of the model and better adapt to different data features, thereby enhancing the performance of the classification algorithm. Experimental results indicate that the algorithm merges the benefits of weighting with model selection, markedly enhancing the classification efficiency of the AODE algorithm. [ABSTRACT FROM AUTHOR]

Feature selection is a common data preprocessing technique that aims to construct better models by selecting the most predictive features. Existing particle swarm optimization-based feature selection algorithms encounter two challenges when dealing with high-dimensional problems: easy to fall into local optimum and high computational cost. Therefore, this paper proposes an evolutionary dual-task feature selection method based on probability-based initialization particle swarm optimization (PPSO-EDT), which aims to find optimal solutions by transferring knowledge between two related tasks. Firstly, a probability-based initialization strategy is designed to accelerate population convergence by fully utilizing the correlation between labels and features. Secondly, a task generation strategy based on feature correlation was designed, which constructs the main task and auxiliary task by selecting feature subsets with highly correlated values and feature subsets without redundancy, respectively. Finally, an multi-task transfer mechanism is used to transfer knowledge and find optimal solutions. The results on 12 high-dimensional datasets indicate that the proposed method achieves high classification performance with a small feature subset in a relatively short amount of time. [ABSTRACT FROM AUTHOR]

The complexity in stock index futures markets, influenced by the intricate interplay of human behavior, is characterized as nonlinearity and dynamism, contributing to significant uncertainty in long-term price forecasting. While machine learning models have demonstrated their efficacy in stock price forecasting, they rely solely on historical price data, which, given the inherent volatility and dynamic nature of financial markets, are insufficient to address the complexity and uncertainty in long-term forecasting due to the limited connection between historical and forecasting prices. This paper introduces a pioneering approach that integrates financial theory with advanced deep learning methods to enhance predictive accuracy and risk management in China's stock index futures market. The SF-Transformer model, combining spot-forward parity and the Transformer model, is proposed to improve forecasting accuracy across short and long-term horizons. Formulated upon the arbitrage-free futures pricing model, the spot-forward parity model offers variables such as stock index price, risk-free rate, and stock index dividend yield for forecasting. Our insight is that the mutual information generated by these variables has the potential to significantly reduce uncertainty in long-term forecasting. A case study on predicting major stock index futures prices in China demonstrates the superiority of the SF-Transformer model over models based on LSTM, MLP, and the stock index futures arbitrage-free pricing model, covering both short and long-term forecasting up to 28 days. Unlike existing machine learning models, the Transformer processes entire time series concurrently, leveraging its attention mechanism to discern intricate dependencies and capture long-range relationships, thereby offering a holistic understanding of time series data. An enhancement of mutual information is observed after introducing spot-forward parity in the forecasting. The variation of mutual information and ablation study results highlights the significant contributions of spot-forward parity, particularly to the long-term forecasting. Overall, these findings highlight the SF-Transformer model's efficacy in leveraging spot-forward parity for reducing uncertainty and advancing robust and comprehensive approaches in long-term stock index futures price forecasting. [ABSTRACT FROM AUTHOR]

In this paper, we want to investigate classical-quantum multiple access wiretap channels (CQ-MA-WTC) under one-shot setting. In this regard, we analyze the CQ-MA-WTC using a simultaneous position-based decoder for reliable decoding and using a newly introduced technique to decode securely. Also, for the sake of comparison, we analyze the CQ-MA-WTC using Sen's one-shot joint typicality lemma for reliable decoding. The simultaneous position-based decoder tends to a multiple hypothesis testing problem. Also, using convex splitting to analyze the privacy criteria in a simultaneous scenario becomes problematic. To overcome both problems, we first introduce a new channel that can be considered as a dual to the CQ-MA-WTC. This channel is called a point-to-point quantum wiretap channel with multiple messages (PP-QWTC). In the following, as a strategy to solve the problem, we also investigate and analyze quantum broadcast channels (QBC) in the one-shot regime. [ABSTRACT FROM AUTHOR]

As an auxiliary loss function, the mutual information constraint is widely used in various deep learning tasks, such as deep reinforcement learning and representation learning. However, the degree of mutual information constraint is not concerned in many tasks. This paper discusses a situation in which mutual information must be constrained within an appropriate range. In this situation, it is challenging to calculate mutual information accurately. There is still an error between the obtained and ideal features when using mutual information constraints. Therefore, how to reduce this error is a problem worthy of study. This paper proposes a new method to utilize mutual information constraints to more precisely constrain features. This method, called the Adversarial Training-based Mutual Information Constraint (ATMIC) method, can be constructed using different mutual information estimation methods and widely applied in various tasks. ATMIC extracts noise from the obtained feature to attack the original mutual information constraint. The noise can force the encoder to reduce the error between the obtained and ideal features. Experiments show that our ATMIC method achieves better results than other mutual information constraint methods in permutation-invariant MNIST, fair learning, multimodal sentiment analysis, and GLUE tasks. [ABSTRACT FROM AUTHOR]

Multi-label learning is a machine learning subclass that aims to assign more than one label simultaneously for each instance. Many real-world tasks include high-dimensional data which reduces the performance of machine learning methods. To solve this issue, a filter and multi-label feature selection is proposed in this paper. The main idea of the proposed method is to choose highly relevant and non-redundant features with the lowest information loss. The proposed method first uses a novel graph-based density peaks clustering to group similar features to reach this goal. It then uses the ant colony optimization search process to rank features based on their relevancy to a set of labels and also their redundancy with the other features. A graph first represents the feature space, and then a novel density peaks clustering is used to group similar features. Then, the ants are searched through the graph to select a set of non-similar features by remaining in the clusters with a low probability and jumping among the clusters with a high probability. Moreover, in this paper, to evaluate the solutions found by the ants, a novel criterion based onmutual informationwas used to assign a high pheromone value to highly relevant and non-redundant features. Finally, the final features are chosen based on their pheromone values. The results of experiments on a set of real-world datasets show the superiority of the proposed method over a set of baseline and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Retinal image registration is a critical task in the diagnosis and treatment of various eye diseases. And as a relatively new imaging method, optical coherence tomography (OCT) has been widely used in the diagnosis of retinal diseases. This paper is devoted to retinal OCT image registration methods and their clinical applications. Registration methods including volumetric transformation-based registration methods and image features-based registration methods are systematically reviewed. Furthermore, to better understanding these methods, their applications in correcting scanning artifacts, reducing speckle noise, fusing and splicing images and evaluating longitudinal disease progression are studied as well. At the end of this paper, registration of retina with serious pathology and registration with deep learning technique are also discussed. [ABSTRACT FROM AUTHOR]

Deep clustering is an approach that uses deep learning to cluster data, since it involves training a neural network model to become familiar with a data representation that is suitable for clustering. Deep clustering has been applied to a wide range of data types, including images, texts, time series and has the advantage of being able to automatically learn features from the data, which can be more effective than using hand-crafted features. It is also able to handle high-dimensional data, such as time series with many variables, which can be challenging for traditional clustering techniques. In this paper, we introduce a novel deep neural network type to improve the performance of the auto-encoder part by ignoring the unnecessary extra-noises and labelling the input data. Our approach is helpful when just a limited amount of labelled data is available, but labelling a big amount of data would be costly or time-consuming. It also applies for the data in high-dimensional and difficult to define a good set of features for clustering. [ABSTRACT FROM AUTHOR]

Label Distribution Learning (LDL) is a fine-grained learning paradigm that addresses label ambiguity, yet it confronts the curse of dimensionality. Feature selection is an effective method for dimensionality reduction, and several algorithms have been proposed for LDL that tackle the problem from different views. In this paper, we propose a novel feature selection method for LDL. First, an effective LDL model is trained through a classical LDL loss function, which is composed of the maximum entropy model and KL divergence. Then, to select common and label-specific features, their weights are enhanced by l 21 -norm and label correlation, respectively. Considering that direct constraint on the parameter by label correlation will make the label-specific features between relevant labels tend to be the same, we adopt the strategy of constraining the maximum entropy output model. Finally, the proposed method will introduce Mutual Information (MI) for the first time in the optimization model for LDL feature selection, which distinguishes similar features thus reducing the influence of redundant features. Experimental results on twelve datasets validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Feature selection can be seen as a multi-objective task, where the goal is to select a subset of features that exhibit minimal correlation among themselves while maximizing their correlation with the target label. Multi-objective particle swarm optimization algorithm (MOPSO) has been extensively utilized for feature selection and has achieved good performance. However, most MOPSO-based feature selection methods are random and lack knowledge guidance in the initialization process, ignoring certain valuable prior information in the feature data, which may lead to the generated initial population being far from the true Pareto front (PF) and influence the population’s rate of convergence. Additionally, MOPSO has a propensity to become stuck in local optima during the later iterations. In this paper, a novel feature selection method (fMOPSO-FS) is proposed. Firstly, with the aim of improving the initial solution quality and fostering the interpretability of the selected features, a novel initialization strategy that incorporates prior information during the initialization process of the particle swarm is proposed. Furthermore, an adaptive hybrid mutation strategy is proposed to avoid the particle swarm from getting stuck in local optima and to further leverage prior information. The experimental results demonstrate the superior performance of the proposed algorithm compared to the comparison algorithms. It yields a superior feature subset on nine UCI benchmark datasets and six gene expression profile datasets. [ABSTRACT FROM AUTHOR]

Recent advancements in information thermodynamics have revealed that information can be directly converted into mechanical work. Specifically, RNA transcription and nanopore sequencing serve as prime examples of this conversion, by reading information from a DNA template. This paper introduces an information thermodynamic model in which these molecular motors can move along the DNA template by converting the information read from the template DNA into their own motion. This process is a stochastic one, characterized by significant fluctuations in forward movement and is described by the Fokker–Planck equation, based on drift velocity and diffusion coefficients. In the current study, it is hypothesized that by utilizing the sequence information of the template DNA as mutual information, the fluctuations can be reduced, thereby biasing the forward movement on DNA and, consequently, reducing reading errors. Further research into the conversion of biological information by molecular motors could unveil new applications, insights, and important findings regarding the characteristics of information processing in biology. [ABSTRACT FROM AUTHOR]

Ransomware attacks have emerged as a significant threat to critical data and systems, extending beyond traditional computers to mobile and IoT/Cyber–Physical Systems. This study addresses the need to detect early ransomware behavior when only limited data are available. A major step for training such a detection model is choosing a set of relevant and non-redundant features, which is challenging when data are scarce. Therefore, this paper proposes an incremental mutual information-selection technique as a method for selecting the relevant features at the early stages of ransomware attacks. It introduces an adaptive feature-selection technique that processes data in smaller, manageable batches. This approach lessens the computational load and enhances the system's ability to quickly adapt to new data arrival, making it particularly suitable for ongoing attacks during the initial phases of the attack. The experimental results emphasize the importance of the proposed technique in estimating feature significance in limited data scenarios. Such results underscore the significance of the incremental approach as a proactive measure in addressing the escalating challenges posed by ransomware. [ABSTRACT FROM AUTHOR]

The Inner Product Masking (IPM) scheme has been shown to provide higher theoretical security guarantees than the BooleanMasking (BM). This scheme aims to increase the algebraic complexity of the coding to achieve a higher level of security. Some previous work unfolds when certain (adversarial and implementation) conditions are met, and we seek to complement these investigations by understanding what happens when these conditions deviate from their expected behaviour. In this paper, we investigate the security characteristics of IPM under different conditions. In adversarial condition, the security properties of first-order IPMs obtained through parametric characterization are preserved in the face of univariate and bivariate attacks. In implementation condition, we construct two new polynomial leakage functions to observe the nonlinear leakage of the IPM and connect the security order amplification to the nonlinear function. We observe that the security of IPMis affected by the degree and the linear component in the leakage function. In addition, the comparison experiments from the coefficients, signal-to-noise ratio (SNR) and the public parameter show that the security properties of the IPM are highly implementation-dependent. [ABSTRACT FROM AUTHOR]

Localization performance in wireless networks has traditionally been benchmarked using the Cramér–Rao lower bound (CRLB), given a fixed geometry of anchor nodes and a target. However, by endowing the target and anchor locations with distributions, this paper recasts this traditional scalar benchmark as a random variable. The goal of this paper is to derive an analytical expression for the distribution of this now random CRLB, in the context of Time-of-Arrival-based positioning. To derive this distribution, this paper first analyzes how the CRLB is affected by the order statistics of the angles between consecutive participating anchors (i.e., internodal angles). This analysis reveals an intimate connection between the second largest internodal angle and the CRLB, which leads to an accurate approximation of the CRLB. Using this approximation, a closed-form expression for the distribution of the CRLB, conditioned on the number of participating anchors, is obtained. Next, this conditioning is eliminated to derive an analytical expression for the marginal CRLB distribution. Since this marginal distribution accounts for all target and anchor positions, across all numbers of participating anchors, it therefore statistically characterizes localization error throughout an entire wireless network. This paper concludes with a comprehensive analysis of this new network-wide-CRLB paradigm. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the convergence speed of the Arimoto-Blahut algorithm. For many channel matrices, the convergence speed is exponential, but for some channel matrices it is slower than exponential. By analyzing the Taylor expansion of the defining function of the Arimoto-Blahut algorithm, we will make the conditions clear for the exponential or slower convergence. The analysis of the slow convergence in this paper is new. Based on this analysis, we will compare the convergence speeds of the Arimoto-Blahut algorithm numerically with the values obtained in our theorems for several channel matrices. The purpose of this paper is to obtain a complete understanding of the convergence speed of the Arimoto-Blahut algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we provide geometric insights with visualization into the multivariate Gaussian distribution and its entropy and mutual information. In order to develop the multivariate Gaussian distribution with entropy and mutual information, several significant methodologies are presented through the discussion, supported by illustrations, both technically and statistically. The paper examines broad measurements of structure for the Gaussian distributions, which show that they can be described in terms of the information theory between the given covariance matrix and correlated random variables (in terms of relative entropy). The content obtained allows readers to better perceive concepts, comprehend techniques, and properly execute software programs for future study on the topic's science and implementations. It also helps readers grasp the themes' fundamental concepts to study the application of multivariate sets of data in Gaussian distribution. The simulation results also convey the behavior of different elliptical interpretations based on the multivariate Gaussian distribution with entropy for real-world applications in our daily lives, including information coding, nonlinear signal detection, etc. Involving the relative entropy and mutual information as well as the potential correlated covariance analysis, a wide range of information is addressed, including basic application concerns as well as clinical diagnostics to detect the multi-disease effects. [ABSTRACT FROM AUTHOR]

Material identification is playing an increasingly important role in various sectors such as industry, petrochemical, mining, and in our daily lives. In recent years, material identification has been utilized for security checks, waste sorting, etc. However, current methods for identifying materials require direct contact with the target and specialized equipment that can be costly, bulky, and not easily portable. Past proposals for addressing this limitation relied on non-contact material identification methods, such as Wi-Fi-based and radar-based material identification methods, which can identify materials with high accuracy without physical contact; however, they are not easily integrated into portable devices. This paper introduces a novel non-contact material identification based on acoustic signals. Different from previous work, our design leverages the built-in microphone and speaker of smartphones as the transceiver to identify target materials. The fundamental idea of our design is that acoustic signals, when propagated through different materials, reach the receiver via multiple paths, producing distinct multipath profiles. These profiles can serve as fingerprints for material identification. We captured and extracted them using acoustic signals, calculated channel impulse response (CIR) measurements, and then extracted image features from the time–frequency domain feature graphs, including histogram of oriented gradient (HOG) and gray-level co-occurrence matrix (GLCM) image features. Furthermore, we adopted the error-correcting output code (ECOC) learning method combined with the majority voting method to identify target materials. We built a prototype for this paper using three mobile phones based on the Android platform. The results from three different solid and liquid materials in varied multipath environments reveal that our design can achieve average identification accuracies of 90% and 97%. [ABSTRACT FROM AUTHOR]

Photonic neural networks are among the most promising recently proposed neuromorphic solutions for providing fast and energy efficient Deep Learning (DL) implementations. However, photonic networks need to be first appropriately trained to support the underlying hardware implementation (e.g., activation functions), since it is not possible to directly deploy an already trained DL model into a neuromorphic platform that uses different transfer functions compared to regular networks. The main hypothesis examined in this paper is whether we can employ a neural network distillation approach to transfer the knowledge from a regular DL model into a photonic neural network reducing in this way the accuracy gap between regular and photonic models. Furthermore, we propose a novel fully differentiable formulation of mutual information that can be used for efficiently transferring the knowledge between layers of a regular neural networks and photonic neural networks. In this way, we further increase the effectiveness of distillation by also taking into account the behavior of intermediate layers of DL models. Experiments conducted on two image datasets demonstrate the effectiveness of the proposed approach, further closing the accuracy gap between photonic and regular DL models. [ABSTRACT FROM AUTHOR]

Leaf segmentation is crucial for plant recognition, especially for tree species identification. In natural environments, leaf segmentation can be very challenging due to the lack of prior information about leaves and the variability of backgrounds. In typical applications, supervised algorithms often require pixel-level annotation of regions, which can be labour-intensive and limited to identifying plant species using pre-labelled samples. On the other hand, traditional unsupervised image segmentation algorithms require specialised parameter tuning for leaf images to achieve optimal results. Therefore, this paper proposes an unsupervised leaf segmentation method that combines mutual information with neural networks to better generalise to unknown samples and adapt to variations in leaf shape and appearance to distinguish and identify different tree species. First, a model combining a Variational Autoencoder (VAE) and a segmentation network is used as a pre-segmenter to obtain dynamic masks. Secondly, the dynamic masks are combined with the segmentation masks generated by the mask generator module to construct the initial mask. Then, the patcher module uses the Mutual Information Minimum (MIM) loss as an optimisation objective to reconstruct independent regions based on this initial mask. The process of obtaining dynamic masks through pre-segmentation is unsupervised, and the entire experimental process does not involve any label information. The experimental method was performed on tree leaf images with a naturally complex background using the publicly available Pl@ntLeaves dataset. The results of the experiment showed that compared to existing excellent methods on this dataset, the IoU (Intersection over Union) index increased by 3.9%. [ABSTRACT FROM AUTHOR]

This paper proposed the use of mutual information (MI) decomposition as a novel approach to identifying indispensable variables and their interactions for contingency table analysis. The MI analysis identified subsets of associative variables based on multinomial distributions and validated parsimonious log-linear and logistic models. The proposed approach was assessed using two real-world datasets dealing with ischemic stroke (with 6 risk factors) and banking credit (with 21 discrete attributes in a sparse table). This paper also provided an empirical comparison of MI analysis versus two state-of-the-art methods in terms of variable and model selections. The proposed MI analysis scheme can be used in the construction of parsimonious log-linear and logistic models with a concise interpretation of discrete multivariate data. [ABSTRACT FROM AUTHOR]

In recent times, there has been a swift advancement in the field of cryptocurrency. The advent of cryptocurrency has provided us with convenience and prosperity, but has also given rise to certain illicit and unlawful activities. Unlike classical currency, cryptocurrency conceals the activities of criminals and exposes their behavioral patterns, allowing us to determine whether present cryptocurrency transactions are legitimate by analyzing their behavioral patterns. There are two issues to consider when determining whether cryptocurrency transactions are legitimate. One is that most cryptocurrency transactions comply with laws and regulations, but only a small portion of them are used for illegal activities, which is related to the sample imbalance problem. The other issue concerns the excessive volume of data, and there are some unknown illegal transactions, so the data set contains an abundance of unlabeled data. As a result, it is critical to accurately distinguish between which transactions among the plethora of cryptocurrency transactions are legitimate and which are illegal. This presents quite a difficult challenge. Consequently, this paper combines mutual information and self-supervised learning to create a self-supervised model on the basis of mutual information that is used to improve the massive amount of untagged data that exist in the data set. Simultaneously, by merging the conventional cross-entropy loss function with mutual information, a novel loss function is created. It is employed to address the issue of sample imbalance in data sets. The F1-Score results obtained from our experimentation demonstrate that the novel loss function in the GCN method improves the performance of cryptocurrency illegal behavior detection by four points compared with the traditional loss function of cross-entropy; use of the self-supervised network that relies on mutual information improves the performance by three points compared with the original GCN method; using both together improves the performance by six points. [ABSTRACT FROM AUTHOR]

Optimal data detection of data transmitted over a linear channel can always be implemented through the Viterbi algorithm (VA). However, in many cases of interest the memory of the channel prohibits direct application of the VA. A popular and conceptually simple method in this case, studied since the early 1970s, is to first filter the received signal in order to shorten the memory of the channel, and then to apply a VA that operates with the shorter memory. We shall refer to this general concept as a channel shortening (CS) receiver. Although studied for almost four decades, an information theoretic understanding of what such a simple receiver solution is actually doing is not available. In this paper, we show that an optimized CS receiver has a direct correspondance to the chain rule of mutual information. Furthermore, we show that the tools for analyzing the ensuing achievable rates from an optimized CS receiver are precisely the same as those used for analyzing the achievable rates of a minimum mean-square-error (MMSE) receiver. [ABSTRACT FROM AUTHOR]

This paper investigates the hybrid precoding design for millimeter wave (mmWave) multiple-input-multiple-output (MIMO) systems with finite-alphabet inputs. The mmWave MIMO system employs partially-connected hybrid precoding architecture with dynamic subarrays, where each radio frequency (RF) chain is connected to a dynamic subset of antennas. We consider the design of analog and digital precoders utilizing statistical and/or mixed channel state information (CSI), which involve solving an extremely difficult problem in theory: First, designing the optimal partition of antennas over RF chains is a combinatorial optimization problem, whose optimal solution requires an exhaustive search over all antenna partitioning solutions; Second, the average mutual information under mmWave MIMO channels lacks closed-form expression and involves prohibitive computational burden; and Third, the hybrid precoding problem with given partition of antennas is nonconvex with respect to the analog and digital precoders. To address these issues, this paper first presents a simple criterion and the corresponding low complexity algorithm to design the optimal partition of antennas using statistical CSI. Then, it derives the lower bound and its approximation for the average mutual information, in which the computational complexity is greatly reduced compared to calculating the average mutual information directly. In addition, it also shows that the lower bound with a constant shift offers a very accurate approximation to the average mutual information. This paper further proposes utilizing the lower bound approximation as a low-complexity and accurate alternative for developing a manifold-based gradient ascent algorithm to find near-optimal analog and digital precoders. Several numerical results are provided to show that our proposed algorithm outperforms the existing hybrid precoding algorithms. [ABSTRACT FROM AUTHOR]

This paper studies the capacity of an $n$ -dimensional vector Gaussian noise channel subject to the constraint that an input must lie in the ball of radius $R$ centered at the origin. It is known that in this setting, the optimizing input distribution is supported on a finite number of concentric spheres. However, the number, the positions, and the probabilities of the spheres are generally unknown. This paper characterizes necessary and sufficient conditions on the constraint $R$ , such that the input distribution supported on a single sphere is optimal. The maximum $\bar {R}_{n}$ , such that using only a single sphere is optimal, is shown to be a solution of an integral equation. Moreover, it is shown that $\bar {R}_{n}$ scales as $\sqrt {n}$ and the exact limit of $\frac {\bar {R}_{n}}{\sqrt {n}}$ is found. [ABSTRACT FROM AUTHOR]

The performance of a communication link can be improved by maximizing the mutual information between the input and output signals. This paper considers this maximization problem in a molecular communication link where both the transmitter and the receiver are molecular circuit. This general optimization is hard to solve. We simplify the problem by limiting to reactions with linear reaction rates and molecular circuits with a limited number of species. We derive an expression of mutual information and use it for numerical maximization. We show that our parameterized transmitter circuit is able to give mutual information that is close to upper bound obtained in our earlier work. [ABSTRACT FROM AUTHOR]

In this paper, we introduce spatial modulation (SM) in the layered division multiplexing (LDM) system for digital TV scenarios. To analyze the achievable spectral efficiency (SE) performance of the proposed SM-LDM system, we calculate its mutual information (MI) through the Monte Carlo method. Considering both Gaussian and finite alphabet inputs, we derive the MI’s computational-efficient lower bounds as well as their asymptotic values. Simulation results confirm both the advantages and our SE analysis of the proposed SM-LDM system. [ABSTRACT FROM AUTHOR]

This paper considers structure learning from data with $n$ samples of $p$ variables, assuming that the structure is a forest, using the Chow–Liu algorithm. Specifically, for incomplete data, we construct two model selection algorithms that complete in $O(p^{2})$ steps: one obtains a forest with the maximum posterior probability given the data and the other obtains a forest that converges to the true one as $n$ increases. We show that the two forests are generally different when some values are missing. In addition, we present estimations for benchmark data sets to demonstrate that both algorithms work in realistic situations. Moreover, we derive the conditional entropy provided that no value is missing, and we evaluate the per-sample expected redundancy for the universal coding of incomplete data in terms of the number of non-missing samples. [ABSTRACT FROM AUTHOR]

It is well known that the mutual information between two random variables can be expressed as the difference of two relative entropies that depend on an auxiliary distribution, a relation sometimes referred to as the golden formula. This paper is concerned with a finite-blocklength extension of this relation. This extension consists of two elements: 1) a finite-blocklength channel-coding converse bound by Polyanskiy and Verdú, which involves the ratio of two Neyman–Pearson $\beta $ functions (beta–beta converse bound); and 2) a novel beta–beta channel-coding achievability bound, expressed again as the ratio of two Neyman–Pearson $\beta $ functions. To demonstrate the usefulness of this finite-blocklength extension of the golden formula, the beta–beta achievability and converse bounds are used to obtain a finite-blocklength extension of Verdú’s wideband-slope approximation. The proof parallels the derivation of the latter, with the beta–beta bounds used in place of the golden formula. The beta–beta (achievability) bound is also shown to be useful in cases where the capacity-achieving output distribution is not a product distribution due to, e.g., a cost constraint or structural constraints on the codebook, such as orthogonality or constant composition. As an example, the bound is used to characterize the channel dispersion of the additive exponential-noise channel and to obtain a finite-blocklength achievability bound (the tightest to date) for multiple-input multiple-output Rayleigh-fading channels with perfect channel state information at the receiver. [ABSTRACT FROM AUTHOR]

Community structures are critical towards understanding not only the network topology but also how the network functions. However, how to evaluate the quality of detected community structures is still challenging and remains unsolved. The most widely used metric, normalized mutual information (NMI), was proven to have finite size effect, and its improved form relative normalized mutual information (rNMI) has reverse finite size effect. Corrected normalized mutual information (cNMI) was thus proposed and has neither finite size effect nor reverse finite size effect. However, in this paper, we show that cNMI violates the so-called proportionality assumption. In addition, NMI-type metrics have the problem of ignoring importance of small communities. Finally, they cannot be used to evaluate a single community of interest. In this paper, we map the computed community labels to the ground-truth ones through integer linear programming, and then use kappa index and F-score to evaluate the detected community structures. Experimental results demonstrate the advantages of our method. [ABSTRACT FROM AUTHOR]