Search

Pattern selection is crucial for optimizing the calibration process of optical proximity correction (OPC) models in computational lithography. However, it remains a challenge to achieve a balance between representative coverage and computational efficiency. This work presents a comprehensive evaluation of the feature vectors' (FVs') effectiveness in pattern selection for OPC model calibration, leveraging key performance indicators (KPIs) based on Kullback–Leibler divergence and distance ranking. Through the construction of autoencoder-based FVs and fast Fourier transform (FFT)-based FVs, we compare their efficacy in capturing critical pattern features. Validation experimental results indicate that autoencoder-based FVs, particularly augmented with the lithography domain knowledge, outperform FFT-based counterparts in identifying anomalies and enhancing lithography model performance. These results also underscore the importance of adaptive pattern representation methods in calibrating the OPC model with evolving complexities. [ABSTRACT FROM AUTHOR]

The Kullback–Leibler (KL) divergence is a widely used measure for comparing probability distributions, but it faces limitations such as its unbounded nature and the lack of comparability between distributions with different quantum values (the discrete unit of probability). This study addresses these challenges by introducing the concept of quantized distributions, which are probability distributions formed by distributing a given discrete quantity or quantum. This study establishes an upper bound for the KL divergence between two quantized distributions, enabling the development of a normalized KL divergence that ranges between 0 and 1. The theoretical findings are supported by empirical evaluations, demonstrating the distinct behavior of the normalized KL divergence compared to other commonly used measures. The results highlight the importance of considering the quantum value when applying the KL divergence, offering insights for future advancements in divergence measures. [ABSTRACT FROM AUTHOR]

Two typical fixed-length random number generation problems in information theory are considered for general sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the optimum achievable rate with respect to the given approximation measure is one of our main concerns and has been characterized using two different information quantities: the information spectrum and the smooth Rényi entropy. Recently, optimum achievable rates with respect to f-divergences have been characterized using the information spectrum quantity. The f-divergence is a general non-negative measure between two probability distributions on the basis of a convex function f. The class of f-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to f-divergences. However, optimum achievable rates with respect to f-divergences using the smooth Rényi entropy have not been clarified yet in both problems. In this paper, we try to analyze the optimum achievable rates using the smooth Rényi entropy and to extend the class of f-divergence. To do so, we first derive general formulas of the first-order optimum achievable rates with respect to f-divergences in both problems under the same conditions as imposed by previous studies. Next, we relax the conditions on f-divergence and generalize the obtained general formulas. Then, we particularize our general formulas to several specified functions f. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. Furthermore, a kind of duality between the resolvability and the intrinsic randomness is revealed in terms of the smooth Rényi entropy. Second-order optimum achievable rates and optimistic achievable rates are also investigated. [ABSTRACT FROM AUTHOR]

At present, the early diagnosis of diabetic retinopathy (DR), a possible complication of diabetes due to elevated glucose concentrations in the blood, is usually performed by specialists using a manual inspection of high-resolution fundus images based on lesion screening, leading to problems such as high work-intensity and accessibility only in specialized health centers. To support the diagnosis of DR, we propose a deep learning-based (DL) DR lesion classification method through a knowledge distillation (KD) strategy. First, we use the pre-trained DL architecture, Inception-v3, as a teacher model to distill the dataset. Then, a student model, also using the Inception-v3 model, is trained on the distilled dataset to match the performance of the teacher model. In addition, a new combination of Kullback–Leibler (KL) divergence and categorical cross-entropy (CCE) loss is used to measure the difference between the teacher and student models. This combined metric encourages the student model to mimic the predictions of the teacher model. Finally, the trained student model is evaluated on a validation dataset to assess its performance and compare it with both the teacher model and another competitive DL model. Experiments are conducted on the two datasets, corresponding to an imbalanced and a balanced dataset. Two baseline models (Inception-v3 and YOLOv8) are evaluated for reference, obtaining a maximum training accuracy of 66.75% and 90.90%, respectively, and a maximum validation accuracy of 35.94% and 81.52%, both for the imbalanced dataset. On the other hand, the proposed DR classification model achieves an average training accuracy of 99.01% and an average validation accuracy of 97.30%, overcoming the baseline models and other state-of-the-art works. Experimental results show that the proposed model achieves competitive results in DR lesion detection and classification tasks, assisting in the early diagnosis of diabetic retinopathy. [ABSTRACT FROM AUTHOR]

The increasing proliferation of cyber-attacks threatening the security of computer networks has driven the development of more effective methods for identifying malicious network flows. The inclusion of statistical laws, such as Benford's Law, and distance functions, applied to the first digits of network flow metadata, such as IP addresses or packet sizes, facilitates the detection of abnormal patterns in the digits. These techniques also allow for quantifying discrepancies between expected and suspicious flows, significantly enhancing the accuracy and speed of threat detection. This paper introduces a novel method for identifying and analyzing anomalies within computer networks. It integrates Benford's Law into the analysis process and incorporates a range of distance functions, namely the Mean Absolute Deviation (MAD), the Kolmogorov–Smirnov test (KS), and the Kullback–Leibler divergence (KL), which serve as dispersion measures for quantifying the extent of anomalies detected in network flows. Benford's Law is recognized for its effectiveness in identifying anomalous patterns, especially in detecting irregularities in the first digit of the data. In addition, Bayes' Theorem was implemented in conjunction with the distance functions to enhance the detection of malicious traffic flows. Bayes' Theorem provides a probabilistic perspective on whether a traffic flow is malicious or benign. This approach is characterized by its flexibility in incorporating new evidence, allowing the model to adapt to emerging malicious behavior patterns as they arise. Meanwhile, the distance functions offer a quantitative assessment, measuring specific differences between traffic flows, such as frequency, packet size, time between packets, and other relevant metadata. Integrating these techniques has increased the model's sensitivity in detecting malicious flows, reducing the number of false positives and negatives, and enhancing the resolution and effectiveness of traffic analysis. Furthermore, these techniques expedite decisions regarding the nature of traffic flows based on a solid statistical foundation and provide a better understanding of the characteristics that define these flows, contributing to the comprehension of attack vectors and aiding in preventing future intrusions. The effectiveness and applicability of this joint method have been demonstrated through experiments with the CICIDS2017 public dataset, which was explicitly designed to simulate real scenarios and provide valuable information to security professionals when analyzing computer networks. The proposed methodology opens up new perspectives in investigating and detecting anomalies and intrusions in computer networks, which are often attributed to cyber-attacks. This development culminates in creating a promising model that stands out for its effectiveness and speed, accurately identifying possible intrusions with an F1 of nearly 80 % , a recall of 99.42 % , and an accuracy of 65.84 % . [ABSTRACT FROM AUTHOR]

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O (m n) operations and can be trivially parallelized. Moreover, the strong ℓ 1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers. [ABSTRACT FROM AUTHOR]

Dimension reduction is a technique used to transform data from a high-dimensional space into a lower-dimensional space, aiming to retain as much of the original information as possible. This approach is crucial in many disciplines like engineering, biology, astronomy, and economics. In this paper, we consider the following dimensionality reduction instance: Given an n-dimensional probability distribution p and an integer m < n , we aim to find the m-dimensional probability distribution q that is the closest to p, using the Kullback–Leibler divergence as the measure of closeness. We prove that the problem is strongly NP-hard, and we present an approximation algorithm for it. [ABSTRACT FROM AUTHOR]

Recently, discriminative correlation filters (DCF) have been successfully applied for visual tracking. However, traditional DCF trackers tend to separately solve boundary effect and temporal degradation problems in the tracking process. In this paper, a variational online learning correlation filter (VOLCF) is proposed for visual tracking to improve the robustness and accuracy of the tracking process. Unlike previous methods, which use only first-order temporal constraints, this approach leads to overfitting and filter degradation. First, beyond the standard filter training requirement, our proposed VOLCF method introduces a model confidence term, which leverages the temporal information of adjacent frames during filter training. Second, to ensure the consistency of the temporal and spatial characteristics of the video sequence, the model introduces Kullback–Leibler (KL) divergence to obtain the second-order information of the filter. In contrast to traditional target tracking models that rely solely on first-order feature information, this approach facilitates the acquisition of a generalized connection between the previous and current filters. As a result, it incorporates joint-regulated filter updating. Through quantitative and qualitative analyses of the experiment, it proves that the VOLCF model has excellent tracking performance. [ABSTRACT FROM AUTHOR]

Cross-entropy loss is crucial in training many deep neural networks. In this context, we show a number of novel and strong correlations among various related divergence functions. In particular, we demonstrate that, in some circumstances, (a) cross-entropy is almost perfectly correlated with the little-known triangular divergence, and (b) cross-entropy is strongly correlated with the Euclidean distance over the logits from which the softmax is derived. The consequences of these observations are as follows. First, triangular divergence may be used as a cheaper alternative to cross-entropy. Second, logits can be used as features in a Euclidean space which is strongly synergistic with the classification process. This justifies the use of Euclidean distance over logits as a measure of similarity, in cases where the network is trained using softmax and cross-entropy. We establish these correlations via empirical observation, supported by a mathematical explanation encompassing a number of strongly related divergence functions. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel approach for the optimal camera selection in video games. The new approach explores the use of information theoretic metrics f-divergences, to measure the correlation between the objects as viewed in camera frustum and the ideal or target view. The f-divergences considered are the Kullback–Leibler divergence or relative entropy, the total variation and the χ 2 divergence. Shannon entropy is also used for comparison purposes. The visibility is measured using the differential form factors from the camera to objects and is computed by casting rays with importance sampling Monte Carlo. Our method allows a very fast dynamic selection of the best viewpoints, which can take into account changes in the scene, in the ideal or target view, and in the objectives of the game. Our prototype is implemented in Unity engine, and our results show an efficient selection of the camera and an improved visual quality. The most discriminating results are obtained with the use of Kullback–Leibler divergence. [ABSTRACT FROM AUTHOR]

A measurement performed on a quantum system is an act of gaining information about its state. However, in the foundations of quantum theory, the concept of information is multiply defined, particularly in the area of quantum reconstruction, and its conceptual foundations remain surprisingly under-explored. In this paper, we investigate the gain of information in quantum measurements from an operational viewpoint in the special case of a two-outcome probabilistic source. We show that the continuous extension of the Shannon entropy naturally admits two distinct measures of information gain, differential information gain and relative information gain, and that these have radically different characteristics. In particular, while differential information gain can increase or decrease as additional data are acquired, relative information gain consistently grows and, moreover, exhibits asymptotic indifference to the data or choice of Bayesian prior. In order to make a principled choice between these measures, we articulate a Principle of Information Increase, which incorporates a proposal due to Summhammer that more data from measurements leads to more knowledge about the system, and also takes into consideration black swan events. This principle favours differential information gain as the more relevant metric and guides the selection of priors for these information measures. Finally, we show that, of the symmetric beta distribution priors, the Jeffreys binomial prior is the prior that ensures maximal robustness of information gain for the particular data sequence obtained in a run of experiments. [ABSTRACT FROM AUTHOR]

Using information-theoretic quantities in practical applications with continuous data is often hindered by the fact that probability density functions need to be estimated in higher dimensions, which can become unreliable or even computationally unfeasible. To make these useful quantities more accessible, alternative approaches such as binned frequencies using histograms and k-nearest neighbors (k-NN) have been proposed. However, a systematic comparison of the applicability of these methods has been lacking. We wish to fill this gap by comparing kernel-density-based estimation (KDE) with these two alternatives in carefully designed synthetic test cases. Specifically, we wish to estimate the information-theoretic quantities: entropy, Kullback–Leibler divergence, and mutual information, from sample data. As a reference, the results are compared to closed-form solutions or numerical integrals. We generate samples from distributions of various shapes in dimensions ranging from one to ten. We evaluate the estimators' performance as a function of sample size, distribution characteristics, and chosen hyperparameters. We further compare the required computation time and specific implementation challenges. Notably, k-NN estimation tends to outperform other methods, considering algorithmic implementation, computational efficiency, and estimation accuracy, especially with sufficient data. This study provides valuable insights into the strengths and limitations of the different estimation methods for information-theoretic quantities. It also highlights the significance of considering the characteristics of the data, as well as the targeted information-theoretic quantity when selecting an appropriate estimation technique. These findings will assist scientists and practitioners in choosing the most suitable method, considering their specific application and available data. We have collected the compared estimation methods in a ready-to-use open-source Python 3 toolbox and, thereby, hope to promote the use of information-theoretic quantities by researchers and practitioners to evaluate the information in data and models in various disciplines. [ABSTRACT FROM AUTHOR]

Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide them into a sparse collection of smaller ones, underlies Judea Pearl's causality, and determines their explainability and interpretability. Despite their popularity, there are almost no resources in the literature on how to compute Shannon's entropy and the Kullback–Leibler (KL) divergence for BNs under their most common distributional assumptions. In this paper, we provide computationally efficient algorithms for both by leveraging BNs' graphical structure, and we illustrate them with a complete set of numerical examples. In the process, we show it is possible to reduce the computational complexity of KL from cubic to quadratic for Gaussian BNs. [ABSTRACT FROM AUTHOR]

As part of the research in the recently ended project SANET II, we were trying to create a new machine-learning system without a teacher. This system was designed to recognize DDoS attacks in real time, based on adaptation to real-time arbitrary traffic and with the ability to be embedded into the hardware implementation of network probes. The reason for considering this goal was our hands-on experience with the high-speed SANET network, which interconnects Slovak universities and high schools and also provides a connection to the Internet. Similar to any other public-facing infrastructure, it is often the target of DDoS attacks. In this article, we are extending our previous research, mainly by dealing with the use of various statistical parameters for DDoS attack detection. We tested the coefficients of Variation, Kurtosis, Skewness, Autoregression, Correlation, Hurst exponent, and Kullback–Leibler Divergence estimates on traffic captures of different types of DDoS attacks. For early machine recognition of the attack, we have proposed several detection functions that use the response of the investigated statistical parameters to the start of a DDoS attack. The proposed detection methods are easily implementable for monitoring actual IP traffic. [ABSTRACT FROM AUTHOR]

Cranes are pivotal heavy equipment used in the construction of transmission line scenarios. Accurately identifying these cranes and monitoring their status is pressing. The rapid development of computer vision brings new ideas to solve these challenges. Since cranes have a high aspect ratio, conventional horizontal bounding boxes contain a large number of redundant objects, which deteriorates the accuracy of object detection. In this study, we use a rotating target detection paradigm to detect cranes. We propose the YOLOv8-Crane model, where YOLOv8 serves as a detection network for rotating targets, and we incorporate Transformers in the backbone to improve global context modeling. The Kullback–Leibler divergence (KLD) with excellent scale invariance is used as a loss function to measure the distance between predicted and true distribution. Finally, we validate the superiority of YOLOv8-Crane on 1405 real-scene data collected by ourselves. Our approach demonstrates a significant improvement in crane detection and offers a new solution for enhancing safety monitoring. [ABSTRACT FROM AUTHOR]

Traffic sensors play a pivotal role in monitoring and assessing network-wide traffic conditions. However, the substantial costs associated with deploying an extensive sensor network across real-world highway systems can often prove prohibitive. Thus, the strategic selection of optimal sensor locations within budget and resource constraints becomes imperative, leading to the well-known Traffic Sensor Location Problem (TSLP). In this study, we introduce a novel framework to address the TSLP for large-scale highway networks, focusing on maximizing information gain in a joint vector space that comprehensively captures both network topology and segment-level features. To solve this optimization problem, we devised a genetic algorithm (GA) with penalty handling. Additionally, we developed a physics-guided random walk algorithm, which not only significantly reduces the search space but offers remarkable flexibility in striking a practical balance between computational load and the confidence of achieving global optimality. For illustration purposes, the proposed framework was applied to the Savannah highway network in Georgia. The results from our GA method align well with those from exhaustive research, but with significantly reduced computational time. By leveraging information theory and maximizing information gain in a low-dimensional vector space, the proposed framework permits parallel, scalable computation and offers considerable potential in the strategic planning and deployment of various sensors for expansive, real-world highway networks. [ABSTRACT FROM AUTHOR]

We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles of information bottleneck and bottleneck-based networks in the auto-encoding setting and identifying their inherent limitations, which become more prominent for data with multiple relevant, physics-related representations. The TURBO framework is then introduced, providing a comprehensive derivation of its core concept consisting of the maximisation of mutual information between various data representations expressed in two directions reflecting the information flows. We illustrate that numerous prevalent neural network models are encompassed within this framework. The paper underscores the insufficiency of the information bottleneck concept in elucidating all such models, thereby establishing TURBO as a preferable theoretical reference. The introduction of TURBO contributes to a richer understanding of data representation and the structure of neural network models, enabling more efficient and versatile applications. [ABSTRACT FROM AUTHOR]

We introduce a quantile-based multivariate log-normal distribution, providing a new multivariate skewed distribution with positive support. The parameters of this distribution are interpretable in terms of quantiles of marginal distributions and associations between pairs of variables, a desirable feature for statistical modeling purposes. We derive statistical properties of the quantile-based multivariate log-normal distribution involving the transformations, closed-form expressions for the mixed moments, expected value, covariance matrix, mode, Shannon entropy, and Kullback–Leibler divergence. We also present results on marginalization, conditioning, and independence. Additionally, we discuss parameter estimation and verify its performance through simulation studies. We evaluate the model fitting based on Mahalanobis-type distances. An application to children data is presented. [ABSTRACT FROM AUTHOR]

The variational Bayesian method solves nonlinear estimation problems by iteratively computing the integral of the marginal density. Many researchers have demonstrated the fact its performance depends on the linear approximation in the computation of the variational density in the iteration and the degree of nonlinearity of the underlying scenario. In this paper, two methods for computing the variational density, namely, the natural gradient method and the simultaneous perturbation stochastic method, are used to implement a variational Bayesian Kalman filter for maneuvering target tracking using Doppler measurements. The latter are collected from a set of sensors subject to single-hop network constraints. We propose a distributed fusion variational Bayesian Kalman filter for a networked maneuvering target tracking scenario and both of the evidence lower bound and the posterior Cramér–Rao lower bound of the proposed methods are presented. The simulation results are compared with centralized fusion in terms of posterior Cramér–Rao lower bounds, root-mean-squared errors and the 3 σ bound. [ABSTRACT FROM AUTHOR]

This paper introduces a unified Bayesian approach for testing various hypotheses related to multinomial distributions. The method calculates the Kullback–Leibler divergence between two specified multinomial distributions, followed by comparing the change in distance from the prior to the posterior through the relative belief ratio. A prior elicitation algorithm is used to specify the prior distributions. To demonstrate the effectiveness and practical application of this approach, it has been applied to several examples. [ABSTRACT FROM AUTHOR]

In the field of human–computer interaction (HCI), text entry methods can be evaluated through controlled user experiments or predictive modeling techniques. While the modeling approach requires a language model, the empirical approach necessitates representative text phrases for the experimental stimuli. In this context, finding a phrase set with the best language representativeness belongs to the class of optimization problems in which a solution is sought in a large search space. We propose a genetic algorithm (GA)-based method for extracting a target phrase set from the available text corpus, optimizing its language representativeness. Kullback–Leibler divergence is utilized to evaluate candidates, considering the digram probability distributions of both the source corpus and the target sample. The proposed method is highly customizable, outperforms typical random sampling, and exhibits language independence. The representative phrase sets generated by the proposed solution facilitate a more valid comparison of the results from different text entry studies. The open source implementation enables the easy customization of the GA-based sampling method, promotes its immediate utilization, and facilitates the reproducibility of this study. In addition, we provide heuristic guidelines for preparing the text entry experiments, which consider the experiment's intended design and the phrase set to be generated with the proposed solution. [ABSTRACT FROM AUTHOR]

(1) Objective: Whether abnormal sleep-wake rhythms were associated with depressive symptoms in patents with epilepsy had remained unclear. Our study aimed to establish relative entropy for the assessment of sleep-wake patterns and to explore the relationship between this index and the severity of depressive symptoms in patients with epilepsy. (2) Methods: We recorded long-term scalp electroencephalograms (EEGs) and Hamilton Depression Rating Scale-17 (HAMD-17) questionnaire scores from 64 patients with epilepsy. Patients with HAMD-17 scores of 0–7 points were defined as the non-depressive group, while patients with scores of 8 or higher were defined as the depression group. Sleep stages were firstly classified based on EEG data. We then quantified sleep-wake rhythm variations in brain activity using the Kullback–Leibler divergence (KLD) of daytime wakefulness and nighttime sleep. The KLD at different frequency bands in each brain region was analyzed between the depression and non-depression groups. (3) Results: Of the 64 patients with epilepsy included in our study, 32 had depressive symptoms. It was found that patients with depression had significantly decreased KLD for high-frequency oscillations in most brain areas, especially the frontal lobe. A detailed analysis was conducted in the right frontal region (F4) because of the significant difference in the high-frequency band. We found that the KLDs at the gamma bands were significantly decreased in the depression groups compared to the non-depression group (KLDD = 0.35 ± 0.05, KLDND = 0.57 ± 0.05, p = 0.009). A negative correlation was displayed between the KLD of gamma band oscillation and HAMD-17 score (r = −0.29, p = 0.02). (4) Conclusions: Sleep-wake rhythms can be assessed using the KLD index calculated from long-term scalp EEGs. Moreover, the KLD of high-frequency bands had a negative correlation with HAMD-17 scores in patients with epilepsy, which indicates a close relationship between abnormal sleep-wake patterns and depressive symptoms in patients with epilepsy. [ABSTRACT FROM AUTHOR]

Due to the complexity of electromechanical equipment and the difficulties in obtaining large-scale health monitoring datasets, as well as the long-tailed distribution of data, existing methods ignore certain characteristics of health monitoring data. In order to solve these problems, this paper proposes a method for the fault diagnosis of rolling bearings in electromechanical equipment based on an improved prototypical network—the weight prototypical networks (WPorNet). The main contributions of this paper are as follows: (1) the prototypical networks, which perform well on small-sample classification tasks, were improved by calculating the different levels of influence of support sample distributions in order to achieve the prototypical calculation. The change in sample influence was calculated using the Kullback–Leibler divergence of the sample distribution. The influence change in a specific sample can be measured by assessing how much the distribution changes in the absence of that sample; and (2) The Gramian Angular Field (GAF) algorithm was used to transform one-dimensional time series into two-dimensional vibration images, which greatly improved the application effect of the 2D convolutional neural network (CNN). Through experiments on MAFAULDA and CWRU bearing datasets, it was shown that this network effectively solves the shortcomings of a small number of valid samples and a long-tail distribution in health monitoring data, it enhances the dependency between the samples and the global data, it improves the model's feature extraction ability, and it enhances the accuracy of model classification. Compared with the prototypical network, the improved network model increased the performance of the 2-way 10-shot, 2-way 20-shot, and 2-way 50-shot classification tasks by 5.23%, 5.74%, and 4.37%, respectively, and it increased the performance of the 4-way 10-shot, 4-way 20-shot, and 4-way 50-shot classification tasks by 12.02%, 10.47%, and 4.66%, respectively. Experimental results show that the improved prototypical network model has higher sample classification accuracy and stronger anti-interference ability compared with traditional small-sample classification models. [ABSTRACT FROM AUTHOR]

Canonical Correlation Analysis (CCA) infers a pairwise linear relationship between two groups of random variables, X and Y. In this paper, we present a new procedure based on Rényi's pseudodistances (RP) aiming to detect linear and non-linear relationships between the two groups. RP canonical analysis (RPCCA) finds canonical coefficient vectors, a and b , by maximizing an RP-based measure. This new family includes the Information Canonical Correlation Analysis (ICCA) as a particular case and extends the method for distances inherently robust against outliers. We provide estimating techniques for RPCCA and show the consistency of the proposed estimated canonical vectors. Further, a permutation test for determining the number of significant pairs of canonical variables is described. The robustness properties of the RPCCA are examined theoretically and empirically through a simulation study, concluding that the RPCCA presents a competitive alternative to ICCA with an added advantage in terms of robustness against outliers and data contamination. [ABSTRACT FROM AUTHOR]

Embodied PointGoal navigation is a fundamental task for embodied agents. Recent works have shown that the performance of the embodied navigation agent degrades significantly in the presence of visual corruption, including Spatter, Speckle Noise, and Defocus Blur, showing the weak robustness of the agent. To improve the robustness of embodied navigation agents to various visual corruptions, we propose a navigation framework called Regularized Denoising Masked AutoEncoders Navigation (RDMAE-Nav). In a nutshell, RDMAE-Nav mainly consists of two modules: a visual module and a policy module. In the visual module, a self-supervised pretraining method, dubbed Regularized Denoising Masked AutoEncoders (RDMAE), is designed to enable the Vision Transformers (ViT)-based visual encoder to learn robust representations. The bidirectional Kullback–Leibler divergence is introduced in RDMAE as the regularization term for a denoising masked modeling task. Specifically, RDMAE mitigates the gap between clean and noisy image representations by minimizing the bidirectional Kullback–Leibler divergence. Then, the visual encoder is pretrained by RDMAE. In contrast to existing works, RDMAE-Nav applies denoising masked visual pretraining for PointGoal navigation to improve robustness to various visual corruptions. Finally, the pretrained visual encoder with frozen weights is applied to extract robust visual representations for policy learning in the RDMAE-Nav. Extensive experiments show that RDMAE-Nav performs competitively compared with state of the arts (SOTAs) on various visual corruptions. In detail, RDMAE-Nav performs the absolute improvement: 28.2% in SR and 23.68% in SPL under Spatter; 2.28% in SR and 6.41% in SPL under Speckle Noise; and 9.46% in SR and 9.55% in SPL under Defocus Blur. [ABSTRACT FROM AUTHOR]

We propose a new model selection method, named the posterior averaging information criterion, for Bayesian model assessment to minimize the risk of predicting independent future observations. The theoretical foundation is built on the Kullback–Leibler divergence to quantify the similarity between the proposed candidate model and the underlying true model. From a Bayesian perspective, our method evaluates the candidate models over the entire posterior distribution in terms of predicting a future independent observation. Without assuming that the true distribution is contained in the candidate models, the new criterion is developed by correcting the asymptotic bias of the posterior mean of the in-sample log-likelihood against out-of-sample log-likelihood, and can be generally applied even for Bayesian models with degenerate non-informative priors. Simulations in both normal and binomial settings demonstrate superior small sample performance. [ABSTRACT FROM AUTHOR]

Image style transfer has become a key technique in modern photo-editing applications. Although significant progress has been made to blend content from one image with style from another image, the synthesized image may have a hallucinatory effect when the texture from the style image is rich when processing high-resolution image style transfer tasks. In this paper, we propose a novel attention mechanism, named compositional attention, to design a compositional transformer-based autoencoder (CTA) to solve this above-mentioned issue. With the support from this module, our model is capable of generating high-quality images when transferring from texture-riched style images to content images with semantics. Additionally, we embed region-based consistency terms in our loss function for ensuring internal structure semantic preservation in our synthesized image. Moreover, information theory-based CTA is discussed and Kullback–Leibler divergence loss is introduced to preserve more brightness information for photo-realistic style transfer. Extensive experimental results based on three benchmark datasets, namely Churches, Flickr Landscapes, and Flickr Faces HQ, confirmed excellent performance when compared to several state-of-the-art methods. Based on a user study assessment, the majority number of users, ranging from 61% to 66%, gave high scores on the transfer effects of our method compared to 9% users who supported the second best method. Further, for the questions of realism and style transfer quality, we achieved the best score, i.e., an average of 4.5 out of 5 compared to other style transfer methods. [ABSTRACT FROM AUTHOR]

Practitioners often face the situation of comparing any set of k distributions, which may follow neither normality nor equality of variances. We propose a semiparametric model to compare those distributions using an exponential tilt method. This extends the classical analysis of variance models when all distributions are unknown by relaxing its assumptions. The proposed model is optimal when one of the distributions is known. Large-sample estimates of the model parameters are derived, and the hypotheses for the equality of the distributions are tested for one-at-a-time and simultaneous comparison cases. Real data examples from NASA meteorology experiments and social credit card limits are analyzed to illustrate our approach. The proposed approach is shown to be preferable in a simulated power comparison with existing parametric and nonparametric methods. [ABSTRACT FROM AUTHOR]

The Gaussian law reigns supreme in the information theory of analog random variables. This paper showcases a number of information theoretic results which find elegant counterparts for Cauchy distributions. New concepts such as that of equivalent pairs of probability measures and the strength of real-valued random variables are introduced here and shown to be of particular relevance to Cauchy distributions. [ABSTRACT FROM AUTHOR]

Queuing networks (QNs) are essential models in operations research, with applications in cloud computing and healthcare systems. However, few studies have analyzed the cell's biological signal transduction using QN theory. This study entailed the modeling of signal transduction as an open Jackson's QN (JQN) to theoretically determine cell signal transduction, under the assumption that the signal mediator queues in the cytoplasm, and the mediator is exchanged from one signaling molecule to another through interactions between the signaling molecules. Each signaling molecule was regarded as a network node in the JQN. The JQN Kullback–Leibler divergence (KLD) was defined using the ratio of the queuing time (λ) to the exchange time (μ), λ/μ. The mitogen-activated protein kinase (MAPK) signal-cascade model was applied, and the KLD rate per signal-transduction-period was shown to be conserved when the KLD was maximized. Our experimental study on MAPK cascade supported this conclusion. This result is similar to the entropy-rate conservation of chemical kinetics and entropy coding reported in our previous studies. Thus, JQN can be used as a novel framework to analyze signal transduction. [ABSTRACT FROM AUTHOR]

The precise separation of rice particles is an important step in rice processing. In this paper, discrete element simulations of the motion of rice particles of different integrity in an indented cylinder separator were carried out using numerical simulation methods. The effects of single factors (cylinder rotation rate, cylinder axial inclination angle, and collection trough inclination angle) on the motion trajectories of particles are investigated and the probability distribution functions of particles are obtained. The statistical method of Kullback-Leibler divergence is used to quantitatively evaluate the differences in the probability distribution functions of the escape angles of particles of different degrees of integrity. The purpose of this paper is to determine the optimum parameters for an indent cylinder separator by understanding the material cylinder separating process from particle scale and to provide a basis for the numerical design of a grain particle cylinder separators. [ABSTRACT FROM AUTHOR]

To derive a latent trait (for instance ability) in a computer adaptive testing (CAT) framework, the obtained results from a model must have a direct relationship to the examinees' response to a set of items presented. The set of items is previously calibrated to decide which item to present to the examinee in the next evaluation question. Some useful models are more naturally based on conditional probability in order to involve previously obtained hits/misses. In this paper, we integrate an experimental part, obtaining the information related to the examinee's academic performance, with a theoretical contribution of maximum entropy. Some academic performance index functions are built to support the experimental part and then explain under what conditions one can use constrained prior distributions. Additionally, we highlight that heuristic prior distributions might not properly work in all likely cases, and when to use personalized prior distributions instead. Finally, the inclusion of the performance index functions, arising from current experimental studies and historical records, are integrated into a theoretical part based on entropy maximization and its relationship with a CAT process. [ABSTRACT FROM AUTHOR]

In recent years, the Poisson lognormal mixed model has been frequently used in modeling count data because it can accommodate both the over-dispersion of the data and the existence of within-subject correlation. Since the likelihood function of this model is expressed in terms of an intractable integral, estimating the parameters and obtaining inference for the parameters are challenging problems. Some approximation procedures have been proposed in the literature; however, they are computationally intensive. Moreover, the existing studies of approximate parameter inference using the Gaussian variational approximation method are usually restricted to models with only one predictor. In this paper, we consider the Poisson lognormal mixed model with more than one predictor. By extending the Gaussian variational approximation method, we derive explicit forms for the estimators of the parameters and examine their properties, including the asymptotic distributions of the estimators of the parameters. Accurate inference for the parameters is also obtained. A real-life example demonstrates the applicability of the proposed method, and simulation studies illustrate the accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

This work studies the power of adversarial attacks against machine learning algorithms that use differentially private mechanisms as their weapon. In our setting, the adversary aims to modify the content of a statistical dataset via insertion of additional data without being detected by using the differential privacy to her/his own benefit. The goal of this study is to evaluate how easy it is to detect such attacks (anomalies) when the adversary makes use of Gaussian and Laplacian perturbation using both statistical and information-theoretic tools. To this end, firstly via hypothesis testing, we characterize statistical thresholds for the adversary in various settings, which balances the privacy budget and the impact of the attack (the modification applied on the original data) in order to avoid being detected. In addition, we establish the privacy-distortion trade-off in the sense of the well-known rate-distortion function for the Gaussian mechanism by using an information-theoretic approach. Accordingly, we derive an upper bound on the variance of the attacker's additional data as a function of the sensitivity and the original data's second-order statistics. Lastly, we introduce a new privacy metric based on Chernoff information for anomaly detection under differential privacy as a stronger alternative for the (ϵ , δ) -differential privacy in Gaussian mechanisms. Analytical results are supported by numerical evaluations. [ABSTRACT FROM AUTHOR]

This paper deals with a new Bayesian approach to the one-sample test for proportion. More specifically, let x = (x 1 , ... , x n) be an independent random sample of size n from a Bernoulli distribution with an unknown parameter θ. For a fixed value θ 0 , the goal is to test the null hypothesis H 0 : θ = θ 0 against all possible alternatives. The proposed approach is based on using the well-known formula of the Kullback–Leibler divergence between two binomial distributions chosen in a certain way. Then, the difference of the distance from a priori to a posteriori is compared through the relative belief ratio (a measure of evidence). Some theoretical properties of the method are developed. Examples and simulation results are included. [ABSTRACT FROM AUTHOR]

In this paper, the design of low probability of intercept (LPI) radar waveforms considers not only the performance of passive interception systems (PISs), but also radar detection and resolution performance. Waveform design is an important considerations for the LPI ability of radar. Since information theory has a powerful performance-bound description ability from the perspective of information flow, LPI waveforms are designed in this paper within the constraints of the detection performance metrics of radar and PISs, both of which are measured by the Kullback–Leibler divergence, and the resolution performance metric, which is measured by joint entropy. The designed optimization model of LPI waveforms can be solved using the sequential quadratic programming (SQP) method. Simulation results verify that the designed LPI waveforms not only have satisfactory target-detecting and resolution performance, but also have a superior low interception performance against PISs. [ABSTRACT FROM AUTHOR]

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback–Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions. [ABSTRACT FROM AUTHOR]

This paper aims at certain theoretical studies and additional computational analysis on symmetry and its lack in Kullback-Leibler and Jeffreys probabilistic divergences related to some engineering applications. As it is known, the Kullback-Leibler distance in between two different uncertainty sources exhibits a lack of symmetry, while the Jeffreys model represents its symmetrization. The basic probabilistic computational implementation has been delivered in the computer algebra system MAPLE 2019®, whereas engineering illustrations have been prepared with the use of the Finite Element Method systems Autodesk ROBOT® & ABAQUS®. Determination of the first two probabilistic moments fundamental in the calculation of both relative entropies has been made (i) analytically, using a semi-analytical approach (based upon the series of the FEM experiments), and (ii) the iterative generalized stochastic perturbation technique, where some reference solutions have been delivered using (iii) Monte-Carlo simulation. Numerical analysis proves the fundamental role of computer algebra systems in probabilistic entropy determination and shows remarkable differences obtained with the two aforementioned relative entropy models, which, in some specific cases, may be neglected. As it is demonstrated in this work, a lack of symmetry in probabilistic divergence may have a decisive role in engineering reliability, where extreme and admissible responses cannot be simply replaced with each other in any case. [ABSTRACT FROM AUTHOR]

Among the existing bearing faults, ball ones are known to be the most difficult to detect and classify. In this work, we propose a diagnosis methodology for these incipient faults' classification using time series of vibration signals and their decomposition. Firstly, the vibration signals were decomposed using empirical mode decomposition (EMD). Time series of intrinsic mode functions (IMFs) were then obtained. Through analysing the energy content and the components' sensitivity to the operating point variation, only the most relevant IMFs were retained. Secondly, a statistical analysis based on statistical moments and the Kullback–Leibler divergence (KLD) was computed allowing the extraction of the most relevant and sensitive features for the fault information. Thirdly, these features were used as inputs for the statistical clustering techniques to perform the classification. In the framework of this paper, the efficiency of several family of techniques were investigated and compared including linear, kernel-based nonlinear, systematic deterministic tree-based, and probabilistic techniques. The methodology's performance was evaluated through the training accuracy rate (TrA), testing accuracy rate (TsA), training time (Trt) and testing time (Tst). The diagnosis methodology has been applied to the Case Western Reserve University (CWRU) dataset. Using our proposed method, the initial EMD decomposition into eighteen IMFs was reduced to four and the most relevant features identified via the IMFs' variance and the KLD were extracted. Classification results showed that the linear classifiers were inefficient, and that kernel or data-mining classifiers achieved 100 % classification rates through the feature fusion. For comparison purposes, our proposed method demonstrated a certain superiority over the multiscale permutation entropy. Finally, the results also showed that the training and testing times for all the classifiers were lower than 2 s, and 0.2 s, respectively, and thus compatible with real-time applications. [ABSTRACT FROM AUTHOR]

Multiple Importance Sampling (MIS) combines the probability density functions (pdf) of several sampling techniques. The combination weights depend on the proportion of samples used for the particular techniques. Weights can be found by optimization of the variance, but this approach is costly and numerically unstable. We show in this paper that MIS can be represented as a divergence problem between the integrand and the pdf, which leads to simpler computations and more robust solutions. The proposed idea is validated with 1D numerical examples and with the illumination problem of computer graphics. [ABSTRACT FROM AUTHOR]

This paper investigates the degree of efficiency for the Moscow Stock Exchange. A market is called efficient if prices of its assets fully reflect all available information. We show that the degree of market efficiency is significantly low for most of the months from 2012 to 2021. We calculate the degree of market efficiency by (i) filtering out regularities in financial data and (ii) computing the Shannon entropy of the filtered return time series. We developed a simple method for estimating volatility and price staleness in empirical data in order to filter out such regularity patterns from return time series. The resulting financial time series of stock returns are then clustered into different groups according to some entropy measures. In particular, we use the Kullback–Leibler distance and a novel entropy metric capturing the co-movements between pairs of stocks. By using Monte Carlo simulations, we are then able to identify the time periods of market inefficiency for a group of 18 stocks. The inefficiency of the Moscow Stock Exchange that we have detected is a signal of the possibility of devising profitable strategies, net of transaction costs. The deviation from the efficient behavior for a stock strongly depends on the industrial sector that it belongs to. [ABSTRACT FROM AUTHOR]

Evaluation of driving behaviour is helpful for policy development, and for designing infrastructure and an intelligent safety system for a car. This study focused on a quantitative evaluation method of driving behaviour based on the shared-electrical car. The data were obtained from the OBD interface via CAN bus and transferred to a server by 4G network. Eleven types of NDS data were selected as the indexes for driving behaviour evaluation. Kullback–Leibler divergence was calculated to confirm the minimum data quantity and ensure the effectiveness of the analysis. The distribution of the main driving behaviour parameters was compared and the change trend of the parameters was analysed in conjunction with car speed to identify the threshold for recognition of aberrant driving behaviour. The weights of indexes were confirmed by combining the analytic hierarchy process and entropy weight method. The scoring rule was confirmed according to the distribution of the indexes. A score-based evaluation method was proposed and verified by the driving behaviour data collected from randomly chosen drivers. [ABSTRACT FROM AUTHOR]

This paper studies the agent identity privacy problem in the scalar linear quadratic Gaussian (LQG) control system. The agent identity is a binary hypothesis: Agent A or Agent B. An eavesdropper is assumed to make a hypothesis testing the agent identity based on the intercepted environment state sequence. The privacy risk is measured by the Kullback–Leibler divergence between the probability distributions of state sequences under two hypotheses. By taking into account both the accumulative control reward and privacy risk, an optimization problem of the policy of Agent B is formulated. This paper shows that the optimal deterministic privacy-preserving LQG policy of Agent B is a linear mapping. A sufficient condition is given to guarantee that the optimal deterministic privacy-preserving policy is time-invariant in the asymptotic regime. It is also shown that adding an independent Gaussian random process noise to the linear mapping of the optimal deterministic privacy-preserving policy cannot improve the performance of Agent B. The numerical experiments justify the theoretic results and illustrate the reward–privacy trade-off. [ABSTRACT FROM AUTHOR]

This paper proposes a new technique for the construction of a concrete-beam health indicator based on the Kullback–Leibler divergence (KLD) and deep learning. Health indicator (HI) construction is a vital part of remaining useful lifetime (RUL) approaches for monitoring the health of concrete structures. Through the construction of a HI, the deterioration process can be processed and portrayed so that it can be forwarded to a prediction module for RUL prognosis. The degradation progression and failure can be identified by predicting the RUL based on the situation of the current specimen; as a result, maintenance can be planned to reduce safety risks, reduce financial costs, and prolong the specimen's useful lifetime. The portrayal of deterioration through HI construction from raw acoustic emission (AE) data is performed using a deep neural network (DNN), whose parameters are obtained by pretraining and fine tuning using a stack autoencoder (SAE). Kullback–Leibler divergence, which is calculated between a reference normal-conditioned signal and a current unknown signal, was used to represent the deterioration process of concrete structures, which has not been investigated for the concrete beams so far. The DNN-based constructor then learns to generate HI from raw data with KLD values as the training label. The HI construction result was evaluated with run-to-fail test data of concrete specimens with two measurements: fitness analysis of the construction result and RUL prognosis. The results confirm the reliability of KLD in portraying the deterioration process, showing a large improvement in comparison to other methods. In addition, this method requires no adept knowledge of the nature of the AE or the system fault, which is more favorable than model-based approaches where this level of expertise is compulsory. Furthermore, AE offers in-service monitoring, allowing the RUL prognosis task to be performed without disrupting the specimen's work. [ABSTRACT FROM AUTHOR]

Cancer is a complex disease caused by genomic and epigenetic alterations; hence, identifying meaningful cancer drivers is an important and challenging task. Most studies have detected cancer drivers with mutated traits, while few studies consider multiple omics characteristics as important factors. In this study, we present a framework to analyze the effects of multi-omics characteristics on the identification of driver genes. We utilize four machine learning algorithms within this framework to detect cancer driver genes in pan-cancer data, including 75 characteristics among 19,636 genes. The 75 features are divided into four types and analyzed using Kullback–Leibler divergence based on CGC genes and non-CGC genes. We detect cancer driver genes in two different ways. One is to detect driver genes from a single feature type, while the other is from the top N features. The first analysis denotes that the mutational features are the best characteristics. The second analysis reveals that the top 45 features are the most effective feature combinations and superior to the mutational features. The top 45 features not only contain mutational features but also three other types of features. Therefore, our study extends the detection of cancer driver genes and provides a more comprehensive understanding of cancer mechanisms. [ABSTRACT FROM AUTHOR]

The complex nature of rubbing faults makes it difficult to use traditional signal analysis methods for feature extraction. Various time-frequency analysis approaches based on signal decomposition, such as empirical mode decomposition (EMD) and ensemble EMD (EEMD), have been widely utilized recently to analyze rub-impact faults. However, traditional EMD suffers from “mode-mixing”, and in both EMD and EEMD the relevance of the extracted components to rubbing processes must be determined. In this paper, we introduce a new informative intrinsic mode function (IMF) selection method for EEMD and a hybrid feature model for diagnosing rub-impact faults of various intensities. Our method uses a novel selection procedure that combines the degree-of-presence ratio of rub impact and a Kullback–Leibler divergence-based similarity measure into an IMF quality metric with adaptive threshold-based selection to pick the meaningful signal-dominant modes. Signals reconstructed using the selected IMFs contained explicit information about the rubbing faults and are used for hybrid feature extraction. Experimental results demonstrated that the proposed approach effectively defines meaningful IMFs for rubbing processes, and the presented hybrid feature model allows for the classification of rub-impact faults of various intensities with good accuracy. [ABSTRACT FROM AUTHOR]

We demonstrate that questions of convergence and divergence regarding shapes of distributions can be carried out in a location- and scale-free environment. This environment is the class of probability density quantiles (pdQs), obtained by normalizing the composition of the density with the associated quantile function. It has earlier been shown that the pdQ is representative of a location-scale family and carries essential information regarding shape and tail behavior of the family. The class of pdQs are densities of continuous distributions with common domain, the unit interval, facilitating metric and semi-metric comparisons. The Kullback-Leibler divergences from uniformity of these pdQs are mapped to illustrate their relative positions with respect to uniformity. To gain more insight into the information that is conserved under the pdQ mapping, we repeatedly apply the pdQ mapping and find that further applications of it are quite generally entropy increasing so convergence to the uniform distribution is investigated. New fixed point theorems are established with elementary probabilistic arguments and illustrated by examples. [ABSTRACT FROM AUTHOR]

In this paper, we examine two widely-used approaches, the polynomial chaos expansion (PCE) and Gaussian process (GP) regression, for the development of surrogate models. The theoretical differences between the PCE and GP approximations are discussed. A state-of-the-art PCE approach is constructed based on high precision quadrature points; however, the need for truncation may result in potential precision loss; the GP approach performs well on small datasets and allows a fine and precise trade-off between fitting the data and smoothing, but its overall performance depends largely on the training dataset. The reproducing kernel Hilbert space (RKHS) and Mercer's theorem are introduced to form a linkage between the two methods. The theorem has proven that the two surrogates can be embedded in two isomorphic RKHS, by which we propose a novel method named Gaussian process on polynomial chaos basis (GPCB) that incorporates the PCE and GP. A theoretical comparison is made between the PCE and GPCB with the help of the Kullback-Leibler divergence. We present that the GPCB is as stable and accurate as the PCE method. Furthermore, the GPCB is a one-step Bayesian method that chooses the best subset of RKHS in which the true function should lie, while the PCE method requires an adaptive procedure. Simulations of 1D and 2D benchmark functions show that GPCB outperforms both the PCE and classical GP methods. In order to solve high dimensional problems, a random sample scheme with a constructive design (i.e., tensor product of quadrature points) is proposed to generate a valid training dataset for the GPCB method. This approach utilizes the nature of the high numerical accuracy underlying the quadrature points while ensuring the computational feasibility. Finally, the experimental results show that our sample strategy has a higher accuracy than classical experimental designs; meanwhile, it is suitable for solving high dimensional problems. [ABSTRACT FROM AUTHOR]

A fully adaptive particle filtering algorithm is proposed in this paper which is capable of updating both state process models and measurement models separately and simultaneously. The approach is a significant step toward more realistic online monitoring or tracking damage. The majority of the existing methods for Bayes filtering are based on predefined and fixed state process and measurement models. Simultaneous estimation of both state and model parameters has gained attention in recent literature. Some works have been done on updating the state process model. However, not many studies exist regarding an update of the measurement model. In most of the real-world applications, the correlation between measurements and the hidden state of damage is not defined in advance and, therefore, presuming an offline fixed measurement model is not promising. The proposed approach is based on optimizing relative entropy or Kullback-Leibler divergence through a particle filtering algorithm. The proposed algorithm is successfully applied to a case study of online fatigue damage estimation in composite materials. [ABSTRACT FROM AUTHOR]

In this paper, a distributed Bayesian filter design was studied for nonlinear dynamics and measurement mapping based on Kullback-Leibler divergence. In a distributed structure, the nonlinear filter becomes a challenging problem, since each sensor cannot access the global measurement likelihood function over the whole network, and some sensors have weak observability of the state. To solve the problem in a sensor network, the distributed Bayesian filter problem was converted into an optimization problem by maximizing a posterior method. The global cost function over the whole network was decomposed into the sum of the local cost function, where the local cost function can be solved by each sensor. With the help of the Kullback-Leibler divergence, the global estimate was approximated in each sensor by communicating with its neighbors. Based on the proposed distributed Bayesian filter structure, a distributed cubature Kalman filter (DCKF) was proposed. Finally, a cooperative space object tracking problem was studied for illustration. The simulation results demonstrated that the proposed algorithm can solve the issues of varying communication topology and weak observability of some sensors. [ABSTRACT FROM AUTHOR]