Search

This paper presents a comparative study of entropy estimation in a large-alphabet regime. A variety of entropy estimators have been proposed over the years, where each estimator is designed for a different setup with its own strengths and caveats. As a consequence, no estimator is known to be universally better than the others. This work addresses this gap by comparing twenty-one entropy estimators in the studied regime, starting with the simplest plug-in estimator and leading up to the most recent neural network-based and polynomial approximate estimators. Our findings show that the estimators' performance highly depends on the underlying distribution. Specifically, we distinguish between three types of distributions, ranging from uniform to degenerate distributions. For each class of distribution, we recommend the most suitable estimator. Further, we propose a sample-dependent approach, which again considers three classes of distribution, and report the top-performing estimators in each class. This approach provides a data-dependent framework for choosing the desired estimator in practical setups. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of distinguishing the leading agents in the group. The problem is considered in the framework of classification problems, where the agents in the group select the items with respect to certain properties. The suggested method of distinguishing the leading agents utilizes the connectivity between the agents and the Rokhlin distance between the subgroups of the agents. The method is illustrated by numerical examples. The method can be useful in considering the division of labor in swarm dynamics and in the analysis of the data fusion in the tasks based on the wisdom of the crowd techniques. [ABSTRACT FROM AUTHOR]

In this paper, we suggest a potential use of data compression measures, such as the Entropy, and the Huffman Coding, to assess the effects of noise factors on the reliability of tested systems. In particular, we extend the Taguchi method for robust design by computing the entropy of the percent contribution values of the noise factors. The new measures are computed already at the parameter-design stage, and together with the traditional S/N ratios enable the specification of a robust design. Assuming that (some of) the noise factors should be naturalized, the entropy of a design reflects the potential efforts that will be required in the tolerance-design stage to reach a more reliable system. Using a small example, we illustrate the contribution of the new measure that might alter the designer decision in comparison with the traditional Taguchi method, and ultimately obtain a system with a lower quality loss. Assuming that the percent contribution values can reflect the probability of a noise factor to trigger a disturbance in the system response, a series of probabilistic algorithms can be applied to the robust design problem. We focus on the Huffman coding algorithm, and show how to implement this algorithm such that the designer obtains the minimal expected number of tests in order to find the disturbing noise factor. The entropy measure, in this case, provides the lower bound on the algorithm's performance. [ABSTRACT FROM AUTHOR]