Search

In this article, we propose to estimate the probability mass function (pmf) of a discrete supported random variable by a semiparametric bias corrected method using discrete associated kernels. This method consists in applying a two-stage multiplicative bias correction (MBC) approach for the initial parametric model in order to improve the accuracy of the estimator measured in terms of the vanishing bias. Various properties of the resulting semiparametric MBC discrete associated kernel estimator are provided (bias, variance, and mean integrated squared error). The common cross-validation technique and the Kullback–Leibler divergence are adapted for bandwidth selection. Monte Carlo simulations and a real-data application for count data illustrate the performance of the semiparametric-MBC estimator. [ABSTRACT FROM AUTHOR]

AbstractThis article introduces a methodology for quantifying the proportion of information that sets of consecutive moments contain. The proposed approach relates a sequence of moments, up to a given order, to certain points that are deemed representative of the target distribution. It involves the use of Fritsch-Carlson monotonic interpolants in conjunction with the Kullback-Leibler measure of divergence. Four illustrative examples are presented. [ABSTRACT FROM AUTHOR]

Learning from a nonstationary data stream is challenging, as a data stream is generally considered to be endless, and the learning model is required to be constantly amended for adapting the shifting data distributions. When it meets multi-label data, the challenge would be further intensified. In this study, an adaptive online weighted multi-label ensemble learning algorithm called MLDME (multi-label learning with distribution matching ensemble) is proposed. It simultaneously calculates both the feature matching level and label matching level between any one reserved data block and the new received data block, further providing an adaptive decision weight assignment for ensemble classifiers based on their distribution similarities. Specifically, MLDME abandons the most commonly used but not totally correct underlying hypothesis that in a data stream, each data block always has the most approximate distribution with that emerging after it; thus, MLDME could provide a just-in-time decision for the new received data block. In addition, to avoid an infinite extension of ensemble classifiers, we use a fixed-size buffer to store them and design three different dynamic classifier updating rules. Experimental results for nine synthetic and three real-world multi-label nonstationary data streams indicate that the proposed MLDME algorithm is superior to some popular and state-of-the-art online learning paradigms and algorithms, including two specifically designed ones for classifying a nonstationary multi-label data stream. [ABSTRACT FROM AUTHOR]

This work investigates the attack strategy design problem of the false data injection attack against the cyber-physical system. Distinct from the relevant results which utilise the current intercepted data or additionally consider side information, a more universal attack model is proposed which combines historical data and side information with the current intercepted data to synergistically deteriorate the system estimation performance. In order to quantify the impact resulting from the proposed attack strategy, the optimisation objective is characterised by deriving the error covariance matrix under the attacks, which becomes more intricate since the proposed attack model introduces more decision variables and coupled terms. Take the stealthiness which is characterised by Kullback-Leibler divergence as the constraints, the problem investigated in this work is equivalently transformed into the constrained multi-variable non-convex optimisation problems, which are not able to be solved directly by the methods in the relevant results. By utilising the Lagrange multiplier method, the structural characteristic of the optimal mean and the optimal covariance which only related to the Lagrange multiplier are derived, such that the optimal distribution of the modified innovation is able to be obtained by a simple search procedure. Following that, the design of the optimal attack strategy is completed by using semi-definite programming to derive the optimal attack matrices. Finally, the simulation examples are given such that the validity of the results is verified. [ABSTRACT FROM AUTHOR]

We study entanglement in a simple model comprising two coupled linear harmonic oscillators of the same natural frequency. The system is separable in the center of mass (COM) and relative coordinates into two oscillators of frequency ωc and ωr. We compute standard entanglement measures (subsystem linear entropy and subsystem von Neumann entropy) as well as several tomographic entanglement indicators (Bhattacharyya distance, Kullback–Leibler divergence, and inverse participation ratio) as functions of the frequency ratio η = ωc/ωr, keeping the COM oscillator in its ground state. We demonstrate that, overall, the entanglement indicators reflect quite faithfully the variations in the standard measures. The entanglement is shown to be minimum at η = 1 and maximum at η → 0 or ∞. [ABSTRACT FROM AUTHOR]

In this paper, we focus on model selection problem for conditional density function when response variables are right-censored with censoring indicators missing at random. We propose three model selection criteria based on the Kullback–Leibler divergence and define corresponding parametric estimators. At the same time, we construct Wald test statistics of hypothesis testing on the parameters. Under suitable conditions, the consistency of proposed criteria as well as asymptotic distributions of estimators and test statistics are established. Also, simulation study and real data analysis are conducted to evaluate finite-sample performance of the proposed methods. [ABSTRACT FROM AUTHOR]

The Averaged One-Dependence Estimators (AODE) algorithm is an improvement of the naive Bayes algorithm, which allows all the attributes dependent on one common attribute, called parent attribute, thus forming One-Dependence Estimators (ODE). The classification probability is estimated by averaging the conditional probability of the ODE. When there is a dependency relationship between attributes, the AODE algorithm can better capture these relationships, thus improving classification performance. The AODE algorithm treats the parent and child attribute values equally in different ODEs. However, the parent attribute value and the child attribute value in different ODEs have different importance for classification. In this paper, two attribute value weighted AODE based on Kullback–Leibler divergence are proposed, one is parent attribute value weighted AODE, and the other is child attribute value weighted AODE. Comparative experiments were carried out with 30 datasets in the UCI database, and the experiments indicate that the performance of the parent attribute value weighted AODE algorithm with Kullback–Leibler divergence is significantly better than the original AODE algorithm, and its performance is also better than the mutual information weighted AODE algorithm. [ABSTRACT FROM AUTHOR]

Dementia, including Alzheimer's disease, is a neurodegenerative illness characterized by the progressive impairment of cognitive functions, posing a significant global health threat. Physical exercise is widely recognized for its preventive role, providing benefits for both the body composition and brain health. This study aimed to explore the relationship between physical exercise, the body composition, and the progression of dementia. The analysis used clinical and neuroradiology data from 42 patients enrolled in the Alzheimer's Disease Neuroimaging Initiative (ADNI). Our study mainly focused on crucial parameters such as the body mass index (BMI), skeletal muscle index (SMI), and MRI biomarkers, including the hippocampal volume and white matter integrity. We grouped the participants according to the similarities of their body compositions through clustering techniques. Then, atrophy-related changes in the brain structures were computed using the Kullback–Leibler divergence. Our findings suggest that a higher BMI and greater muscle mass may slow down brain atrophy, suggesting a protective effect on the brain. Based on these results, preserving muscle mass and metabolic health through resistance and aerobic exercise appears crucial in reducing the risk of dementia. Body composition interventions may slow neurodegenerative changes and promote brain health. This is an essential piece of information about prevention strategies, especially for individuals at risk of dementia who may benefit from following structured physical activity strategies. [ABSTRACT FROM AUTHOR]

Finding closed-form solutions in Bayesian data analysis can be critical and time-saving, as it eliminates the need for computationally expensive techniques like MCMC methods. This paper explores Bayesian analysis with closed-form solutions of the bivariate gamma distribution. We present predictive density estimations under the Kullback-Leibler divergence, utilizing three well-known (non-) informative prior distributions, all analyzable in closed form. We compare these methods through simulation studies and a real-world example, applying them to hydrological Food data. [ABSTRACT FROM AUTHOR]

For subject i, we monitor an event that can occur multiple times over a random observation window [0, τ i ). At each recurrence, p concomitant variables, x i , associated to the event recurrence are recorded—a subset ( q ≤ p ) of which is measured with errors. To circumvent the problem of bias and consistency associated with parameter estimation in the presence of measurement errors, we propose inference for corrected estimating equations with well-behaved roots under an additive measurement errors model. We show that estimation is essentially unbiased under the corrected profile likelihood for recurrent events, in comparison to biased estimations under a likelihood function that ignores correction. We propose methods for obtaining estimators of error variance and discuss the properties of the estimators. We further investigate the case of misspecified error models and show that the resulting estimators under misspecification converge to a value different from that of the true parameter—thereby providing a basis for bias assessment. We demonstrate the foregoing correction methods on an open-source rhDNase dataset gathered in a clinical setting. [ABSTRACT FROM AUTHOR]

The surface defect of hot-rolled strip is a significant factor that impacts the performance of strip products. In recent years, convolutional neural networks (CNNs) have been extensively used in strip surface defect recognition to ensure product quality. However, the existing CNNs-based methods confront the challenges of high complexity, difficult deployment and slow inference speed. Accordingly, this work proposes a soft optimization knowledge distillation (SOKD) scheme to distill the ResNet-152 large model and extract a compact strip surface recognition model. The SOKD scheme utilizes Kullback-Leibler (KL) divergence to minimize the error between the soft probability distributions of the student network and the teacher network, and gradually reduces the weight of "Hard loss" during the training process. The operation significantly reduces the learning constraints that the prior knowledge of the teacher network on the student network in the original KD, which improves the recognition performance of the model. Additionally, SOKD is applicable to most CNNs for identify surface defect of hot-rolled strip. The experimental results on NEU-CLS dataset show that the SOKD outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose the approximate Bregman proximal gradient algorithm (ABPG) for solving composite nonconvex optimization problems. ABPG employs a new distance that approximates the Bregman distance, making the subproblem of ABPG simpler to solve compared to existing Bregman-type algorithms. The subproblem of ABPG is often expressed in a closed form. Similarly to existing Bregman-type algorithms, ABPG does not require the global Lipschitz continuity for the gradient of the smooth part. Instead, assuming the smooth adaptable property, we establish the global subsequential convergence under standard assumptions. Additionally, assuming that the Kurdyka–Łojasiewicz property holds, we prove the global convergence for a special case. Our numerical experiments on the ℓ p regularized least squares problem, the ℓ p loss problem, and the nonnegative linear system show that ABPG outperforms existing algorithms especially when the gradient of the smooth part is not globally Lipschitz or even locally Lipschitz continuous. [ABSTRACT FROM AUTHOR]

In a data based learning process, training data set is utilized to provide a hypothesis that can be generalized to explain all data points from a domain set. The hypothesis is chosen from classes with potentially different complexities. Linear regression modeling is an important category of learning algorithms. The practical uncertainty of the label samples in the training data set has a major effect in the generalization ability of the learned model. Failing to choose a proper model or hypothesis class can lead to serious issues such as underfitting or overfitting. These issues have been addressed mostly by alternating modeling cost functions or by utilizing cross-validation methods. Drawbacks of these methods include introducing new hyperparameters with their own new challenges and uncertainties, potential increase of the computational complexity or requiring large set of training data sets. On the other hand, the theory of probably approximately correct (PAC) aims at defining learnability based on probabilistic settings. Despite its theoretical value, PAC bounds can’t be utilized in practical regression learning applications with only available training data sets. This work is motivated by practical issues in regression learning generalization and is inspired by the foundations of the theory of statistical learning. The proposed approach, denoted by $\epsilon $ -Confidence Approximately Correct ( $\epsilon $ -CoAC), utilizes the conventional Kullback-Leibler divergence (relative entropy) and defines new related typical sets to develop a unique method of probabilistic statistical learning for practical regression learning and generalization. $\epsilon $ -CoAC learnability is able to validate the learning process as a function of training data sample size, as well as a function of the hypothesis class complexity order. Consequently, it enables the learner to automatically compare hypothesis classes of different complexity orders and to choose among them the optimum class with the minimum $\epsilon $ in the $\epsilon $ -CoAC framework. The $\epsilon $ -CoAC learnability overcomes the issues of overfitting and underfitting. In addition, it shows advantages over the well-known cross-validation method in the sense of accuracy and data length requirements for convergence. Simulation results, for both synthetic and real data, confirm not only strength and capability of $\epsilon $ -CoAC in providing learning measurements as a function of data length and/or hypothesis complexity, but also superiority of the method over the existing approaches in hypothesis complexity and model selection.

If P 1 , ... , P n and Q 1 , ... , Q n are probability measures on R d and P 1 ∗ ⋯ ∗ P n and Q 1 ∗ ⋯ ∗ Q n are their respective convolutions, the Rényi divergence D λ of order λ ∈ (0 , 1 ] satisfies D λ (P 1 ∗ ⋯ ∗ P n | | Q 1 ∗ ⋯ ∗ Q n) ≤ ∑ i = 1 n D λ (P i | | Q i). When P i belongs to the natural exponential family generated by Q i , with the same natural parameter θ for any i = 1 , ... , n , the equality sign holds. The present note tackles the inverse problem, namely "does the equality D λ (P 1 ∗ ⋯ ∗ P n | | Q 1 ∗ ⋯ ∗ Q n) = ∑ i = 1 n D λ (P i | | Q i) imply that P i belongs to the natural exponential family generated by Q i for every i = 1 , ... , n ?" The answer is not always positive and depends on the set of solutions of a generalization of the celebrated Cauchy functional equation. We discuss in particular the case P 1 = ⋯ = P n = P and Q 1 = ⋯ = Q n = Q , with n = 2 and n = ∞ , the latter meaning that the equality holds for all n. Our analysis is mainly devoted to P and Q concentrated on non-negative integers, and P and Q with densities with respect to the Lebesgue measure. The results cover the Kullback–Leibler divergence (KL), this being the Rényi divergence for λ = 1 . We also show that the only f-divergences such that D f (P ∗ 2 | | Q ∗ 2 ) = 2 D f (P | | Q) , for P and Q in the same exponential family, are mixtures of KL divergence and its dual. [ABSTRACT FROM AUTHOR]

In high-dimensional regression models, variable selection becomes challenging from a computational and theoretical perspective. Bayesian regularized regression via shrinkage priors like the Laplace or spike-and-slab prior are effective methods for variable selection in p > n scenarios provided the shrinkage priors are configured adequately. We propose an empirical Bayes configuration using checks for prior-data conflict: tests that assess whether there is disagreement in parameter information provided by the prior and data. We apply our proposed method to the Bayesian LASSO and spike-and-slab shrinkage priors in the linear regression model and assess the variable selection performance of our prior configurations through a high-dimensional simulation study. Additionally, we apply our method to proteomic data collected from patients admitted to the Albany Medical Center in Albany NY in April of 2020 with COVID-like respiratory issues. Simulation results suggest our proposed configurations may outperform competing models when the true regression effects are small. [ABSTRACT FROM AUTHOR]

Introduction. When managing traffic at the packet level in modern telecommunication networks, it is proposed to use methods that transform a self-similar stochastic packet flow into a quasi-deterministic one. To do this, it is required to apply complex probabilistic laws of distribution of self-similar flows. From the literature, methods of balancing the network load are known, which, with the problem indicated above, contribute to increasing the efficiency of telecommunication systems. However, there is no strictly mathematical solution to find out the optimal probabilistic characteristics of the output flow, based on the input flow. The presented research is intended to fill this gap. Its objective is to create a method for determining the optimal probabilistic characteristics of the packet flow, using the minimum value of the proximity measure of the self-similar input and quasi-deterministic output flows.Materials and Methods. To solve the research problem, the parameters of the output flow distribution were selected so that the approximation function was close to 𝛿𝛿-function. The Kullback-Leibler divergence was used as a proximity measure of the input and output distributions of time intervals. Methods of set theory, metric spaces, multidimensional optimization, and teletraffic were used. The solution algorithm included minimization of the Kullback-Leibler divergence and the limit passage to 𝛿𝛿-function.Results. A probability distribution is shown — an approximation of 𝛿𝛿-function, which maintains the equality of time intervals of a quasi-deterministic output packet flow. A method for transforming a self-similar input flow into a quasideterministic output flow is presented. The Kullback–Leibler divergence was used as a measure of their proximity. The minimum of the Kullback-Leibler divergence between the input and output flows with a normal distribution was achieved in the case of equality of the mathematical expectations of these flows. Using the passage to the limit, it has been established that time interval T between packets of the quasi-deterministic output flow must be equal to the mathematical expectation of the time intervals between packets of the input self-similar flow. To obtain a quasi-deterministic flow, the passage to the limit is performed for the found value of the mathematical expectation at σ → 0.Discussion and Conclusion. The application of this method will reduce the negative impact of self-similarity of network traffic on the efficiency of the telecommunication network. The use of quasi-deterministic flows makes it possible to predict the load of network resources, which can be the basis for improving the quality of user service. Two difficulties associated with calculations and practical implementation of the solution are eliminated. Firstly, it is difficult to use the delta function as a function of the output flow distribution density. Secondly, there are no ideal deterministic flows in the operation of telecommunication networks. The proposed method has great potential in the design and optimization of communication networks.

For any pair of probability measures defined on a common space, their relative information spectra——specifically, the distribution functions of the loglikelihood ratio under either probability measure——fully encapsulate all that is relevant for distinguishing them. This paper explores the properties of the relative information spectra and their connections to various measures of discrepancy including total variation distance, relative entropy, Rényi divergence, and general $ f $-divergences. A simple definition of sufficient statistics, termed $ I $-sufficiency, is introduced and shown to coincide with longstanding notions under the assumptions that the data model is dominated and the observation space is standard. Additionally, a new measure of discrepancy between probability measures, the NP-divergence, is proposed and shown to determine the area of the error probability pairs achieved by the Neyman-Pearson binary hypothesis tests. For independent identically distributed data models, that area is shown to approach 1 at a rate governed by the Bhattacharyya distance.

Abstract Background The standard approach to local inconsistency assessment typically relies on testing the conflict between the direct and indirect evidence in selected treatment comparisons. However, statistical tests for inconsistency have low power and are subject to misinterpreting a p-value above the significance threshold as evidence of consistency. Methods We propose a simple framework to interpret local inconsistency based on the average Kullback–Leibler divergence (KLD) from approximating the direct with the corresponding indirect estimate and vice versa. Our framework uses directly the mean and standard error (or posterior mean and standard deviation) of the direct and indirect estimates obtained from a local inconsistency method to calculate the average KLD measure for selected comparisons. The average KLD values are compared with a semi-objective threshold to judge the inconsistency as acceptably low or material. We exemplify our novel interpretation approach using three networks with multiple treatments and multi-arm studies. Results Almost all selected comparisons in the networks were not associated with statistically significant inconsistency at a significance level of 5%. The proposed interpretation framework indicated 14%, 66%, and 75% of the selected comparisons with an acceptably low inconsistency in the corresponding networks. Overall, information loss was more notable when approximating the posterior density of the indirect estimates with that of the direct estimates, attributed to indirect estimates being more imprecise. Conclusions Using the concept of information loss between two distributions alongside a semi-objectively defined threshold helped distinguish target comparisons with acceptably low inconsistency from those with material inconsistency when statistical tests for inconsistency were inconclusive.

Copyright of Baghdad Science Journal is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A method for evaluating Kullback–Leibler (KL) divergence and Squared Hellinger (SH) distance between empirical data and a model distribution is proposed. This method exclusively utilises the empirical Cumulative Distribution Function (CDF) of the data and the CDF of the model, avoiding data processing such as histogram binning. The proposed method converges almost surely, with the proof based on the use of exponentially distributed waiting times. An example demonstrates convergence of the KL divergence and SH distance to their true values when utilising the Generalised Pareto (GP) distribution as empirical data and the K distribution as the model. Another example illustrates the goodness of fit of these (GP and K-distribution) models to real sea clutter data from the widely used Intelligent PIxel processing X-band (IPIX) measurements. The proposed method can be applied to assess the goodness of fit of various models (not limited to GP or K distribution) to clutter measurement data such as those from the Adriatic Sea. Distinctive features of this small and immature sea, like the presence of over 1300 islands that affect local wind and wave patterns, are likely to result in an amplitude distribution of sea clutter returns that differs from predictions of models designed for oceans or open seas. However, to the author's knowledge, no data on this specific topic are currently available in the open literature, and such measurements have yet to be conducted. [ABSTRACT FROM AUTHOR]

The symmetric Kullback–Leibler centroid, also called the Jeffreys centroid, of a set of mutually absolutely continuous probability distributions on a measure space provides a notion of centrality which has proven useful in many tasks, including information retrieval, information fusion, and clustering. However, the Jeffreys centroid is not available in closed form for sets of categorical or multivariate normal distributions, two widely used statistical models, and thus needs to be approximated numerically in practice. In this paper, we first propose the new Jeffreys–Fisher–Rao center defined as the Fisher–Rao midpoint of the sided Kullback–Leibler centroids as a plug-in replacement of the Jeffreys centroid. This Jeffreys–Fisher–Rao center admits a generic formula for uni-parameter exponential family distributions and a closed-form formula for categorical and multivariate normal distributions; it matches exactly the Jeffreys centroid for same-mean normal distributions and is experimentally observed in practice to be close to the Jeffreys centroid. Second, we define a new type of inductive center generalizing the principle of the Gauss arithmetic–geometric double sequence mean for pairs of densities of any given exponential family. This new Gauss–Bregman center is shown experimentally to approximate very well the Jeffreys centroid and is suggested to be used as a replacement for the Jeffreys centroid when the Jeffreys–Fisher–Rao center is not available in closed form. Furthermore, this inductive center always converges and matches the Jeffreys centroid for sets of same-mean normal distributions. We report on our experiments, which first demonstrate how well the closed-form formula of the Jeffreys–Fisher–Rao center for categorical distributions approximates the costly numerical Jeffreys centroid, which relies on the Lambert W function, and second show the fast convergence of the Gauss–Bregman double sequences, which can approximate closely the Jeffreys centroid when truncated to a first few iterations. Finally, we conclude this work by reinterpreting these fast proxy Jeffreys–Fisher–Rao and Gauss–Bregman centers of Jeffreys centroids under the lens of dually flat spaces in information geometry. [ABSTRACT FROM AUTHOR]

Simple Summary: The evolutionary process can be seen as a process of acquisition, storage, and updating of information by a population about the environment in which it lives. In this paper, I propose a model that starts with the distribution of mating that occurs according to mutual mating fitness and ends with the distribution of viable adult genotypes obtained after this mating. The result of the evolutionary dynamics associated with each stage of the model can be described in terms of information. This informational description facilitates the connection between cause and effect, as well as the development of statistics to test the null model of zero information, i.e., random mating and/or no effect of natural selection. Incorporating the informational perspective into the mathematical formalism of population genetics/genomics contributes to clarifying, expanding, and deepening the mathematical description of evolutionary theory. Population genetics describes evolutionary processes, focusing on the variation within and between species and the forces shaping this diversity. Evolution reflects information accumulated in genomes, enhancing organisms' adaptation to their environment. In this paper, I propose a model that begins with the distribution of mating based on mutual fitness and progresses to viable adult genotype distribution. At each stage, the changes result in different measures of information. The evolutionary dynamics at each stage of the model correspond to certain aspects of interest, such as the type of mating, the distribution of genotypes in regard to mating, and the distribution of genotypes and haplotypes in the next generation. Changes to these distributions are caused by variations in fitness and result in Jeffrey's divergence values other than zero. As an example, a model of hybrid sterility is developed of a biallelic locus, comparing the information indices associated with each stage of the evolutionary process. In conclusion, the informational perspective seems to facilitate the connection between cause and effect and allows the development of statistical tests to perform hypothesis testing against zero-information null models (random mating, no selection, etc.). The informational perspective could contribute to clarify, deepen, and expand the mathematical foundations of evolutionary theory. [ABSTRACT FROM AUTHOR]

The ability to compute the exact divergence between two high-dimensional distributions is useful in many applications, but doing so naively is intractable. Computing the α β -divergence—a family of divergences that includes the Kullback–Leibler divergence and Hellinger distance—between the joint distribution of two decomposable models, i.e., chordal Markov networks, can be done in time exponential in the treewidth of these models. Extending this result, we propose an approach to compute the exact α β -divergence between any marginal or conditional distribution of two decomposable models. In order to do so tractably, we provide a decomposition over the marginal and conditional distributions of decomposable models. We then show how our method can be used to analyze distributional changes by first applying it to the benchmark image dataset QMNIST and a dataset containing observations from various areas at the Roosevelt Nation Forest and their cover type. Finally, based on our framework, we propose a novel way to quantify the error in contemporary superconducting quantum computers. [ABSTRACT FROM AUTHOR]

Gaussian processes and the Kullback–Leibler divergence have been deeply studied in statistics and machine learning. This paper marries these two concepts and introduce the local Kullback–Leibler divergence to learn about intervals where two Gaussian processes differ the most. We address subtleties entailed in the estimation of local divergences and the corresponding interval of local maximum divergence as well. The estimation performance and the numerical efficiency of the proposed method are showcased via a Monte Carlo simulation study. In a medical research context, we assess the potential of the devised tools in the analysis of electrocardiogram signals. [ABSTRACT FROM AUTHOR]

In ball-type robust universal hypothesis testing (UHT), the null hypothesis is a set of probability distributions constrained by a ball of radius r > 0 denoted B (P 0 , r) based on the cumulative density function of the nominal distribution P 0 . A major limitation is that this method is originally designed only for one-dimensional distributions. To overcome this limitation, this paper proposes a new method to deal with multidimensional samples. For this purpose, first of all, new bounds are defined in the multidimensional domain. Later, a new mathematical programming model based on the transformed region of B (P 0 , r) , namely empirical multidimensional robust UHT problem based on Kullback–Leibler divergence is proposed for ball-type robust UHT. The power of the new testing method combined with different types of bounds was then demonstrated by a computational study. This method fills the research gap by enabling ball-type robust UHT for multidimensional samples and is flexible in that it can be used with different type of bounds. [ABSTRACT FROM AUTHOR]

For any pair of probability measures defined on a common space, their relative information spectra——specifically, the distribution functions of the loglikelihood ratio under either probability measure——fully encapsulate all that is relevant for distinguishing them. This paper explores the properties of the relative information spectra and their connections to various measures of discrepancy including total variation distance, relative entropy, Rényi divergence, and general f -divergences. A simple definition of sufficient statistics, termed I -sufficiency, is introduced and shown to coincide with longstanding notions under the assumptions that the data model is dominated and the observation space is standard. Additionally, a new measure of discrepancy between probability measures, the NP-divergence, is proposed and shown to determine the area of the error probability pairs achieved by the Neyman-Pearson binary hypothesis tests. For independent identically distributed data models, that area is shown to approach 1 at a rate governed by the Bhattacharyya distance. [ABSTRACT FROM AUTHOR]

We study a reinsurer who faces multiple sources of model uncertainty. The reinsurer offers contracts to n insurers whose claims follow compound Poisson processes representing both idiosyncratic and systemic sources of loss. As the reinsurer is uncertain about the insurers' claim severity distributions and frequencies, they design reinsurance contracts that maximise their expected wealth subject to an entropy penalty. Insurers meanwhile seek to maximise their expected utility without ambiguity. We solve this continuous-time Stackelberg game for general reinsurance contracts and find that the reinsurer prices under a distortion of the barycentre of the insurers' models. We apply our results to proportional reinsurance and excess-of-loss reinsurance contracts, and illustrate the solutions numerically. Furthermore, we solve the related problem where the reinsurer maximises, still under ambiguity, their expected utility and compare the solutions. [ABSTRACT FROM AUTHOR]

Good lattice point (GLP) sets are a type of number-theoretic method widely utilized across various fields. Their space-filling property can be further improved, especially with large numbers of runs and factors. In this paper, Kullback-Leibler (KL) divergence is used to measure GLP sets. The generalized good lattice point (GGLP) sets obtained from linear-level permutations of GLP sets have demonstrated that the permutation does not reduce the criterion maximin distance. This paper confirms that linear-level permutation may lead to greater mixture discrepancy. Nevertheless, GGLP sets can still enhance the space-filling property of GLP sets under various criteria. For small-sized cases, the KL divergence from the uniform distribution of GGLP sets is lower than that of the initial GLP sets, and there is nearly no difference for large-sized points, indicating the similarity of their distributions. This paper incorporates a threshold-accepting algorithm in the construction of GGLP sets and adopts Frobenius distance as the space-filling criterion for large-sized cases. The initial GLP sets have been included in many monographs and are widely utilized. The corresponding GGLP sets are partially included in this paper and will be further calculated and posted online in the future. The performance of GGLP sets is evaluated in two applications: computer experiments and representative points, compared to the initial GLP sets. It shows that GGLP sets perform better in many cases. [ABSTRACT FROM AUTHOR]

For the analysis of square contingency tables with ordinal classifications, the marginal homogeneity (MH) model is the one of important models. Some measures for analyzing the degree of departure from the MH model have been proposed. This study proposes a new measure using the continuation odds. Continuation odds may be considered as discrete time hazard. The proposed measure is expressed in the form of Cressie-Read's power-divergence (including the Kullback-Leibler divergence) or Patil and Taillie's diversity index (including Shannon entropy). This study derives a plug-in estimator of the proposed measure and an approximate confidence interval for the proposed measure. Through numerical examples, we evaluate the performances of them. Additionally. the usefulness of the proposed measure is demonstrated by applying it to real data that the row and column variables are the discrete survival time variables. [ABSTRACT FROM AUTHOR]

AbstractModel checking evaluates the appropriateness of a statistical model based on the observed data, and it is essential to make valid statistical analyses. In this paper, a new procedure for model checking type II censored data is proposed. This procedure combines the Kullback-Leibler divergence, the Dirichlet process, and the relative belief ratio. The method is implemented via a computational algorithm and is explained through several examples. [ABSTRACT FROM AUTHOR]

Background: The standard approach to local inconsistency assessment typically relies on testing the conflict between the direct and indirect evidence in selected treatment comparisons. However, statistical tests for inconsistency have low power and are subject to misinterpreting a p-value above the significance threshold as evidence of consistency. Methods: We propose a simple framework to interpret local inconsistency based on the average Kullback–Leibler divergence (KLD) from approximating the direct with the corresponding indirect estimate and vice versa. Our framework uses directly the mean and standard error (or posterior mean and standard deviation) of the direct and indirect estimates obtained from a local inconsistency method to calculate the average KLD measure for selected comparisons. The average KLD values are compared with a semi-objective threshold to judge the inconsistency as acceptably low or material. We exemplify our novel interpretation approach using three networks with multiple treatments and multi-arm studies. Results: Almost all selected comparisons in the networks were not associated with statistically significant inconsistency at a significance level of 5%. The proposed interpretation framework indicated 14%, 66%, and 75% of the selected comparisons with an acceptably low inconsistency in the corresponding networks. Overall, information loss was more notable when approximating the posterior density of the indirect estimates with that of the direct estimates, attributed to indirect estimates being more imprecise. Conclusions: Using the concept of information loss between two distributions alongside a semi-objectively defined threshold helped distinguish target comparisons with acceptably low inconsistency from those with material inconsistency when statistical tests for inconsistency were inconclusive. [ABSTRACT FROM AUTHOR]

Anomaly detection of three-dimensional (3D) topographic data is a challenging problem in spatial data analysis. In this paper, we investigate spatial patterns of 3D surface data that exhibit multiple in-control modes. In complex manufacturing processes, surfaces of final products could contain different topographic features from one in-control surface to another, thus making it difficult to monitor the surface with existing approaches, which rely on the assumption of the presence of single mode surface topography. We propose a novel anomaly detection approach for monitoring local topographic variations in the presence of multimode surface topography. We present a binarization model to capture the generic behavior of the multimode surfaces and enhance the representation of the surface. To systematically monitor the surface, we introduce a new probabilistic distance measure (PDM) that quantifies the similarity of spatial patterns between two binarized surfaces. The proposed PDM takes advantage of identifying local variations by utilizing the order neighbor statistics, which captures the local property on the surface. Experimental results with numerical simulation data and real-life paper surface data are provided to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

The multivariate one-sample problem considers an independent random sample from a multivariate normal distribution with mean μ and unknown variance Σ. For a given real vector μ 1 , the interest is to assess the hypothesis H 0 : μ = μ 1. This paper proposes a new Bayesian approach to this problem based on comparing the change in the Kullback-Leibler divergence from a priori to a posteriori via the relative belief ratio. Eliciting the prior is also considered. The use of the approach is illustrated through several examples. [ABSTRACT FROM AUTHOR]

Pattern selection is crucial for optimizing the calibration process of optical proximity correction (OPC) models in computational lithography. However, it remains a challenge to achieve a balance between representative coverage and computational efficiency. This work presents a comprehensive evaluation of the feature vectors' (FVs') effectiveness in pattern selection for OPC model calibration, leveraging key performance indicators (KPIs) based on Kullback–Leibler divergence and distance ranking. Through the construction of autoencoder-based FVs and fast Fourier transform (FFT)-based FVs, we compare their efficacy in capturing critical pattern features. Validation experimental results indicate that autoencoder-based FVs, particularly augmented with the lithography domain knowledge, outperform FFT-based counterparts in identifying anomalies and enhancing lithography model performance. These results also underscore the importance of adaptive pattern representation methods in calibrating the OPC model with evolving complexities. [ABSTRACT FROM AUTHOR]

In this introduction to the special issue celebrating CLLT's 20th anniversary, we look back and forward in time. To look back, we present the results of a (tongue-in-cheek) corpus-linguistic analysis of about 10 years worth of data of research published in CLLT, IJCL, and Corpora in order to distill the "essence" of CLLT for the reader. As an added bonus, we use the opportunity to discuss ways to improve established ways of performing keyness analyses. To look forward, we asked six (teams of) researchers who all have shaped corpus linguistics and thus the journal to give us their take on what the most significant developments in the field have been, and where they see the most impactful opportunities and challenges arise. This introduction briefly summarizes their contributions. [ABSTRACT FROM AUTHOR]

The Kullback–Leibler (KL) divergence is frequently used in data science. For discrete distributions on large state spaces, approximations of probability vectors may result in a few small negative entries, rendering the KL divergence undefined. We address this problem by introducing a parameterized family of substitute divergence measures, the shifted KL (sKL) divergence measures. Our approach is generic and does not increase the computational overhead. We show that the sKL divergence shares important theoretical properties with the KL divergence and discuss how its shift parameters should be chosen. If Gaussian noise is added to a probability vector, we prove that the average sKL divergence converges to the KL divergence for small enough noise. We also show that our method solves the problem of negative entries in an application from computational oncology, the optimization of Mutual Hazard Networks for cancer progression using tensor-train approximations. [ABSTRACT FROM AUTHOR]

In crowdsourcing scenarios, we can obtain each instance’s multiple noisy labels from different crowd workers and then infer its integrated label via label aggregation. In spite of the effectiveness of label aggregation methods, there still remains a certain level of noise in the integrated labels. Thus, some noise correction methods have been proposed to reduce the impact of noise in recent years. However, to the best of our knowledge, existing methods rarely consider an instance’s information from both its features and multiple noisy labels simultaneously when identifying a noise instance. In this study, we argue that the more distinguishable an instance’s features but the noisier its multiple noisy labels, the more likely it is a noise instance. Based on this premise, we propose a label distribution similarity-based noise correction (LDSNC) method. To measure whether an instance’s features are distinguishable, we obtain each instance’s predicted label distribution by building multiple classifiers using instances’ features and their integrated labels. To measure whether an instance’s multiple noisy labels are noisy, we obtain each instance’s multiple noisy label distribution using its multiple noisy labels. Then, we use the Kullback-Leibler (KL) divergence to calculate the similarity between the predicted label distribution and multiple noisy label distribution and define the instance with the lower similarity as a noise instance. The extensive experimental results on 34 simulated and four real-world crowdsourced datasets validate the effectiveness of our method. [ABSTRACT FROM AUTHOR]