Search

$H$-mutual information ($H$-MI) is a wide class of information leakage measures, where $H=(\eta, F)$ is a pair of monotonically increasing function $\eta$ and a concave function $F$, which is a generalization of Shannon entropy. $H$-MI is defined as the difference between the generalized entropy $H$ and its conditional version, including Shannon mutual information (MI), Arimoto MI of order $\alpha$, $g$-leakage, and expected value of sample information. This study presents a variational characterization of $H$-MI via statistical decision theory. Based on the characterization, we propose an alternating optimization algorithm for computing $H$-capacity.

The problem of computing $\alpha$-capacity for $\alpha>1$ is equivalent to that of computing the correct decoding exponent. Various algorithms for computing them have been proposed, such as Arimoto and Jitsumatsu--Oohama algorithm. In this study, we propose a novel alternating optimization algorithm for computing the $\alpha$-capacity for $\alpha>1$ based on a variational characterization of the Augustin--Csisz{\'a}r mutual information. A comparison of the convergence performance of these algorithms is demonstrated through numerical examples.

The Augustin--Csisz{\' a}r mutual information (MI) and Lapidoth--Pfister MI are well-known generalizations of the Shannon MI, but do not have known closed-form expressions, so they need to be calculated by solving optimization problems. In this study, we propose alternating optimization algorithms for computing these types of MI and present proofs of their global convergence properties. We also provide a novel variational characterization of the Augustin--Csisz{\' a}r MI that is similar to that of the Sibson MI.

The Sibson and Arimoto capacity, which are based on the Sibson and Arimoto mutual information (MI) of order {\alpha}, respectively, are well-known generalizations of the channel capacity C. In this study, we derive novel alternating optimization algorithms for computing these capacities by providing new variational characterizations of the Sibson and Arimoto MI. Moreover, we prove that all iterative algorithms for computing these capacities are equivalent under appropriate conditions imposed on their initial distributions.

In the field of decision trees, most previous studies have difficulty ensuring the statistical optimality of a prediction of new data and suffer from overfitting because trees are usually used only to represent prediction functions to be constructed from given data. In contrast, some studies, including this paper, used the trees to represent stochastic data observation processes behind given data. Moreover, they derived the statistically optimal prediction, which is robust against overfitting, based on the Bayesian decision theory by assuming a prior distribution for the trees. However, these studies still have a problem in computing this Bayes optimal prediction because it involves an infeasible summation for all division patterns of a feature space, which is represented by the trees and some parameters. In particular, an open problem is a summation with respect to combinations of division axes, i.e., the assignment of features to inner nodes of the tree. We solve this by a Markov chain Monte Carlo method, whose step size is adaptively tuned according to a posterior distribution for the trees.

We propose an algorithm for computing Stratonovich's value of information (VoI) that can be regarded as an analogue of the distortion-rate function. We construct an alternating optimization algorithm for VoI under a general information leakage constraint and derive a convergence condition. Furthermore, we discuss algorithms for computing VoI under specific information leakage constraints, such as Shannon's mutual information (MI), $f$-leakage, Arimoto's MI, Sibson's MI, and Csiszar's MI.