Search

This paper presents a comparative study of entropy estimation in a large-alphabet regime. A variety of entropy estimators have been proposed over the years, where each estimator is designed for a different setup with its own strengths and caveats. As a consequence, no estimator is known to be universally better than the others. This work addresses this gap by comparing twenty-one entropy estimators in the studied regime, starting with the simplest plug-in estimator and leading up to the most recent neural network-based and polynomial approximate estimators. Our findings show that the estimators' performance highly depends on the underlying distribution. Specifically, we distinguish between three types of distributions, ranging from uniform to degenerate distributions. For each class of distribution, we recommend the most suitable estimator. Further, we propose a sample-dependent approach, which again considers three classes of distribution, and report the top-performing estimators in each class. This approach provides a data-dependent framework for choosing the desired estimator in practical setups. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of distinguishing the leading agents in the group. The problem is considered in the framework of classification problems, where the agents in the group select the items with respect to certain properties. The suggested method of distinguishing the leading agents utilizes the connectivity between the agents and the Rokhlin distance between the subgroups of the agents. The method is illustrated by numerical examples. The method can be useful in considering the division of labor in swarm dynamics and in the analysis of the data fusion in the tasks based on the wisdom of the crowd techniques. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of detecting multiple static and mobile targets by an autonomous mobile agent acting under uncertainty. It is assumed that the agent is able to detect targets at different distances and that the detection includes errors of the first and second types. The goal of the agent is to plan and follow a trajectory that results in the detection of the targets in a minimal time. The suggested solution implements the approach of deep Q-learning applied to maximize the cumulative information gain regarding the targets' locations and minimize the trajectory length on the map with a predefined detection probability. The Q-learning process is based on a neural network that receives the agent location and current probability map and results in the preferred move of the agent. The presented procedure is compared with the previously developed techniques of sequential decision making, and it is demonstrated that the suggested novel algorithm strongly outperforms the existing methods. [ABSTRACT FROM AUTHOR]

Current global COVID-19 booster scheduling strategies mainly focus on vaccinating high-risk populations at predetermined intervals. However, these strategies overlook key data: the direct insights into individual immunity levels from active serological testing and the indirect information available either through sample-based sero-surveillance, or vital demographic, location, and epidemiological factors. Our research, employing an age-, risk-, and region-structured mathematical model of disease transmission—based on COVID-19 incidence and vaccination data from Israel between 15 May 2020 and 25 October 2021—reveals that a more comprehensive strategy integrating these elements can significantly reduce COVID-19 hospitalizations without increasing existing booster coverage. Notably, the effective use of indirect information alone can considerably decrease COVID-19 cases and hospitalizations, without the need for additional vaccine doses. This approach may also be applicable in optimizing vaccination strategies for other infectious diseases, including influenza. [ABSTRACT FROM AUTHOR]

Current global COVID-19 booster scheduling strategies mainly focus on vaccinating high-risk populations at predetermined intervals. However, these strategies overlook key data: the direct insights into individual immunity levels from active serological testing and the indirect information available either through sample-based sero-surveillance, or vital demographic, location, and epidemiological factors. Our research, employing an age-, risk-, and region-structured mathematical model of disease transmission—based on COVID-19 incidence and vaccination data from Israel between 15 May 2020 and 25 October 2021—reveals that a more comprehensive strategy integrating these elements can significantly reduce COVID-19 hospitalizations without increasing existing booster coverage. Notably, the effective use of indirect information alone can considerably decrease COVID-19 cases and hospitalizations, without the need for additional vaccine doses. This approach may also be applicable in optimizing vaccination strategies for other infectious diseases, including influenza. [ABSTRACT FROM AUTHOR]

In this paper, we suggest a potential use of data compression measures, such as the Entropy, and the Huffman Coding, to assess the effects of noise factors on the reliability of tested systems. In particular, we extend the Taguchi method for robust design by computing the entropy of the percent contribution values of the noise factors. The new measures are computed already at the parameter-design stage, and together with the traditional S/N ratios enable the specification of a robust design. Assuming that (some of) the noise factors should be naturalized, the entropy of a design reflects the potential efforts that will be required in the tolerance-design stage to reach a more reliable system. Using a small example, we illustrate the contribution of the new measure that might alter the designer decision in comparison with the traditional Taguchi method, and ultimately obtain a system with a lower quality loss. Assuming that the percent contribution values can reflect the probability of a noise factor to trigger a disturbance in the system response, a series of probabilistic algorithms can be applied to the robust design problem. We focus on the Huffman coding algorithm, and show how to implement this algorithm such that the designer obtains the minimal expected number of tests in order to find the disturbing noise factor. The entropy measure, in this case, provides the lower bound on the algorithm's performance. [ABSTRACT FROM AUTHOR]

Most statistical process control (SPC) methods are not suitable for monitoring nonlinear and state-dependent processes. This article introduces the context-based SPC (CSPC) methodology for state-dependent data generated by a finite-memory source. The key idea of the CSPC is to monitor the statistical attributes of a process by comparing two-context at any monitoring period of time. The first is a reference tree that represents the "in control" reference behavior of the process; the second is a monitored tree, generated periodically from a simple of sequenced observations, that represents the behavior of the process at that period. The Kullback-Leibler (KL) statistic is used to measure the relative "distance" between these two trees, and an analytic distribution of this statistic is derived. Monitoring the KL statistic ample of buffer-level monitoring in a production system demonstrates the viability of the new method with respect to conventional methods. [ABSTRACT FROM AUTHOR]

The history of information theory, as a mathematical principle for analyzing data transmission and information communication, was formalized in 1948 with the publication of Claude Shannon's famous paper "A Mathematical Theory of Communication" [[1]]. This underling uncertainty also appears in the Von Neumann entropy, which is an extension of the classical Gibbs entropy concepts to the field of quantum mechanics. However, the origins of this theory were incubated approximately seventy years earlier, when Willard Gibbs [[2]] and Ludwig Boltzmann [[3]] published their definition of entropy with a similar form and meaning to Shannon's entropy. [Extracted from the article]

The paper considers the detection of multiple targets by a group of mobile robots that perform under uncertainty. The agents are equipped with sensors with positive and non-negligible probabilities of detecting the targets at different distances. The goal is to define the trajectories of the agents that can lead to the detection of the targets in minimal time. The suggested solution follows the classical Koopman's approach applied to an occupancy grid, while the decision-making and control schemes are conducted based on information-theoretic criteria. Sensor fusion in each agent and over the agents is implemented using a general Bayesian scheme. The presented procedures follow the expected information gain approach utilizing the "center of view" and the "center of gravity" algorithms. These methods are compared with a simulated learning method. The activity of the procedures is analyzed using numerical simulations. [ABSTRACT FROM AUTHOR]

We propose a new algorithm called the context-based predictive information (CBPI) for estimating the predictive information (PI) between time series, by utilizing a lossy compression algorithm. The advantage of this approach over existing methods resides in the case of sparse predictive information (SPI) conditions, where the ratio between the number of informative sequences to uninformative sequences is small. It is shown that the CBPI achieves a better PI estimation than benchmark methods by ignoring uninformative sequences while improving explainability by identifying the informative sequences. We also provide an implementation of the CBPI algorithm on a real dataset of large banks' stock prices in the U.S. In the last part of this paper, we show how the CBPI algorithm is related to the well-known information bottleneck in its deterministic version. [ABSTRACT FROM AUTHOR]

This paper presents a method for unsupervised classification of entities by a group of agents with unknown domains and levels of expertise. In contrast to the existing methods based on majority voting ("wisdom of the crowd") and their extensions by expectation-maximization procedures, the suggested method first determines the levels of the agents' expertise and then weights their opinions by their expertise level. In particular, we assume that agents will have relatively closer classifications in their field of expertise. Therefore, the expert agents are recognized by using a weighted Hamming distance between their classifications, and then the final classification of the group is determined from the agents' classifications by expectation-maximization techniques, with preference to the recognized experts. The algorithm was verified and tested on simulated and real-world datasets and benchmarked against known existing algorithms. We show that such a method reduces incorrect classifications and effectively solves the problem of unsupervised collaborative classification under uncertainty, while outperforming other known methods. [ABSTRACT FROM AUTHOR]

Existing viral-marketing network models commonly assume a preliminary phase in which a marketer actively infects a subset of social network’s users, represented by nodes, followed by a passive viral process, in which nodes infect other nodes without external intervention. However, in real-world commercial scenarios, substantial efforts are often invested by companies to promote their products, suggesting that the adoption of products is rarely the consequence of a viral spread alone. Under this observation, this paper proposes a new diffusion model, named Active Viral Marketing, which better fits real-world marketing scenarios, where adoption of products relies on continuous active promotion efforts by the marketer. In the proposed model, the success of a marketing attempt to infect a potential customer (uninfected node), depends on the number of adopting friends (infected neighbors) of this user, assuming a user is more likely to adopt a product if more of his/her friends have already adopted it, while taking into account that social influence diminishes over time due to a memory-loss effect. The paper further proposes a set of heuristics to schedule the marketing attempts. The main idea behind these heuristics is to utilize the information on the dynamic adoption-states of neighbor nodes, in addition to the static social network topology, when choosing the next node to seed. An extensive experimentation demonstrates how the proposed seeding heuristics improve the adoption rate of products by 30%–75% in comparison to existing state-of-the-art methods that mainly rely on the network topology. [ABSTRACT FROM AUTHOR]

Background: Contact mixing plays a key role in the spread of COVID-19. Thus, mobility restrictions of varying degrees up to and including nationwide lockdowns have been implemented in over 200 countries. To appropriately target the timing, location, and severity of measures intended to encourage social distancing at a country level, it is essential to predict when and where outbreaks will occur, and how widespread they will be. Methods: We analyze aggregated, anonymized health data and cell phone mobility data from Israel. We develop predictive models for daily new cases and the test positivity rate over the next 7 days for different geographic regions in Israel. We evaluate model goodness of fit using root mean squared error (RMSE). We use these predictions in a five-tier categorization scheme to predict the severity of COVID-19 in each region over the next week. We measure magnitude accuracy (MA), the extent to which the correct severity tier is predicted. Results: Models using mobility data outperformed models that did not use mobility data, reducing RMSE by 17.3% when predicting new cases and by 10.2% when predicting the test positivity rate. The best set of predictors for new cases consisted of 1-day lag of past 7-day average new cases, along with a measure of internal movement within a region. The best set of predictors for the test positivity rate consisted of 3-days lag of past 7-day average test positivity rate, along with the same measure of internal movement. Using these predictors, RMSE was 4.812 cases per 100,000 people when predicting new cases and 0.79% when predicting the test positivity rate. MA in predicting new cases was 0.775, and accuracy of prediction to within one tier was 1.0. MA in predicting the test positivity rate was 0.820, and accuracy to within one tier was 0.998. Conclusions: Using anonymized, macro-level data human mobility data along with health data aids predictions of when and where COVID-19 outbreaks are likely to occur. Our method provides a useful tool for government decision makers, particularly in the post-vaccination era, when focused interventions are needed to contain COVID-19 outbreaks while mitigating the collateral damage from more global restrictions. [ABSTRACT FROM AUTHOR]

• We define an ordinal oriented information metric, based on weighted entropy. • We propose an ordinal oriented decision-tree, using the new information metric. • The new decision-tree method is effective for ordinal classification problems. • The new tree outperforms C4.5 on most datasets with an ordinal target. • The new tree outperforms Random Forest on several datasets with an ordinal target. This paper proposes an ordinal decision-tree model, which applies a new weighted information-gain ratio (WIGR) measure for selecting the classifying attributes in the tree. The proposed measure utilizes a weighted entropy function that is defined proportionally to the value deviation of different classes and thus reflects the consequences of the magnitude of potential classification errors. The WIGR can be used to select the classifying attributes in decision trees in a manner that reduces risks. The proposed ordinal decision tree is found effective for classification problems in which the class variable exhibits some form of ordinal ordering, and where dependencies between the attributes and the class value can be non-monotonic. In a series of experiments based on publicly-known datasets, it is shown that the proposed ordinal decision tree outperforms its non-ordinal counterparts that utilize traditional entropy measures. The proposed model can be used as a part of an expert system for ordinal classification applications, such as health-state monitoring, portfolio investments classification and performance evaluation of service systems. [ABSTRACT FROM AUTHOR]

Projects are rarely executed exactly as planned. Often, the actual duration of a project's activities differ from the planned duration, resulting in costs stemming from the inaccurate estimation of the activity's completion date. While monitoring a project at various inspection points is pricy, it can lead to a better estimation of the project completion time, hence saving costs. Nonetheless, identifying the optimal inspection points is a difficult task, as it requires evaluating a large number of the project's path options, even for small-scale projects. This paper proposes an analytical method for identifying the optimal project inspection points by using information theory measures. We search for monitoring (inspection) points that can maximize the information about the project's estimated duration or completion time. The proposed methodology is based on a simulation-optimization scheme using a Monte Carlo engine that simulates potential activities' durations. An exhaustive search is performed of all possible monitoring points to find those with the highest expected information gain on the project duration. The proposed algorithm's complexity is little affected by the number of activities, and the algorithm can address large projects with hundreds or thousands of activities. Numerical experimentation and an analysis of various parameters are presented. [ABSTRACT FROM AUTHOR]

Wireless body area networks (WBANs) have strong potential in the field of health monitoring. However, the energy consumption required for accurate monitoring determines the time between battery charges of the wearable sensors, which is a key performance factor (and can be critical in the case of implantable devices). In this paper, we study the inherent trade-off between the power consumption of the sensors and the probability of misclassifying a patient's health state. We formulate this trade-off as a dynamic problem, in which at each step, we can choose to activate a subset of sensors that provide noisy measurements of the patient's health state. We assume that the (unknown) health state follows a Markov chain, so our problem is formulated as a partially observable Markov decision problem (POMDP). We show that all the past measurements can be summarized as a belief state on the true health state of the patient, which allows tackling the POMDP problem as an MDP on the belief state. Then, we empirically study the performance of a greedy one-step look-ahead policy compared to the optimal policy obtained by solving the dynamic program. For that purpose, we use an open-source Continuous Glucose Monitoring (CGM) dataset of 232 patients over six months and extract the transition matrix and sensor accuracies from the data. We find that the greedy policy saves ≈50% of the energy costs while reducing the misclassification costs by less than 2% compared to the most accurate policy possible that always activates all sensors. Our sensitivity analysis reveals that the greedy policy remains nearly optimal across different cost parameters and a varying number of sensors. The results also have practical importance, because while the optimal policy is too complicated, a greedy one-step look-ahead policy can be easily implemented in WBAN systems. [ABSTRACT FROM AUTHOR]