Back to Search
Start Over
Lossless Transformations and Excess Risk Bounds in Statistical Inference.
- Source :
-
Entropy . Oct2023, Vol. 25 Issue 10, p1394. 25p. - Publication Year :
- 2023
-
Abstract
- We study the excess minimum risk in statistical inference, defined as the difference between the minimum expected loss when estimating a random variable from an observed feature vector and the minimum expected loss when estimating the same random variable from a transformation (statistic) of the feature vector. After characterizing lossless transformations, i.e., transformations for which the excess risk is zero for all loss functions, we construct a partitioning test statistic for the hypothesis that a given transformation is lossless, and we show that for i.i.d. data the test is strongly consistent. More generally, we develop information-theoretic upper bounds on the excess risk that uniformly hold over fairly general classes of loss functions. Based on these bounds, we introduce the notion of a δ -lossless transformation and give sufficient conditions for a given transformation to be universally δ -lossless. Applications to classification, nonparametric regression, portfolio strategies, information bottlenecks, and deep learning are also surveyed. [ABSTRACT FROM AUTHOR]
- Subjects :
- *INFERENTIAL statistics
*DEEP learning
*RANDOM variables
Subjects
Details
- Language :
- English
- ISSN :
- 10994300
- Volume :
- 25
- Issue :
- 10
- Database :
- Academic Search Index
- Journal :
- Entropy
- Publication Type :
- Academic Journal
- Accession number :
- 173267321
- Full Text :
- https://doi.org/10.3390/e25101394