Search

We consider the problem of estimating the mean of a symmetric log-concave distribution under the constraint that only a single bit per sample from this distribution is available to the estimator. We study the mean squared error as a function of the sample size (and hence the number of bits). We consider three settings: first, a centralized setting, where an encoder may release $n$ bits given a sample of size $n$ , and for which there is no asymptotic penalty for quantization; second, an adaptive setting in which each bit is a function of the current observation and previously recorded bits, where we show that the optimal relative efficiency compared to the sample mean is precisely the efficiency of the median; lastly, we show that in a distributed setting where each bit is only a function of a local sample, no estimator can achieve optimal efficiency uniformly over the parameter space. We additionally complement our results in the adaptive setting by showing that one round of adaptivity is sufficient to achieve optimal mean-square error. [ABSTRACT FROM AUTHOR]

We consider the distributional connection between the lossy compressed representation of a high-dimensional signal X using a random spherical code and the observation of X under an additive white Gaussian noise (AWGN). We show that the Wasserstein distance between a bitrate-R compressed version of X and its observation under an AWGN-channel of signal-to-noise ratio 22R−1 is bounded in the problem dimension. We utilize this fact to connect the risk of an estimator based on the compressed version of X to the risk attained by the same estimator when fed the AWGN-corrupted version of X. We demonstrate the usefulness of this connection by deriving various novel results for inference problems under compression constraints, including minimax estimation, sparse regression, compressed sensing, and universality of linear estimation in remote source coding. [ABSTRACT FROM AUTHOR]

Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon’s rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the RD risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon’s RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions are simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data. [ABSTRACT FROM AUTHOR]

We consider the recovery of a continuous-time Wiener process from a quantized or a lossy compressed version of its uniform samples under limited bitrate and sampling rate. We derive a closed-form expression for the optimal tradeoff among sampling rate, bitrate, and quadratic distortion in this setting. This expression is given in terms of a reverse waterfilling formula over the asymptotic spectral distribution of a sequence of finite-rank operators associated with the optimal estimator of the Wiener process from its samples. We show that the ratio between this expression and the standard distortion rate function of the Wiener process, describing the optimal tradeoff between bitrate and distortion without a sampling constraint, is only a function of the number of bits per sample. We also consider a sub-optimal lossy compression scheme in which the continuous-time process is estimated from the output of an encoder that is optimal with respect to the discrete-time samples. We show that the latter is strictly greater than the distortion under optimal encoding but only by at most 3%. We, therefore, conclude that near optimal performance is attained even if the encoder is unaware of the continuous-time origin of the samples. [ABSTRACT FROM AUTHOR]

Representing a continuous-time signal by a set of samples is a classical problem in signal processing. We study this problem under the additional constraint that the samples are quantized or compressed in a lossy manner under a limited bitrate budget. To this end, we consider a combined sampling and source coding problem in which an analog stationary Gaussian signal is reconstructed from its encoded samples. These samples are obtained by a set of bounded linear functionals of the continuous-time path, with a limitation on the average number of samples per unit time given in this setting. We provide a full characterization of the minimal distortion in terms of the sampling frequency, the bitrate, and the signal’s spectrum. Assuming that the signal’s energy is not uniformly distributed over its spectral support, we show that for each compression bitrate there exists a critical sampling frequency smaller than the Nyquist rate, such that the distortion in signal reconstruction when sampling at this frequency is minimal. Our results can be seen as an extension of the classical sampling theorem for bandlimited random processes in the sense that they describe the minimal amount of excess distortion in the reconstruction due to lossy compression of the samples and provide the minimal sampling frequency required in order to achieve this distortion. Finally, we compare the fundamental limits in the combined source coding and sampling problem to the performance of pulse code modulation, where each sample is quantized by a scalar quantizer using a fixed number of bits. [ABSTRACT FROM AUTHOR]

A general expression for the quadratic distortion rate function (DRF) of cyclostationary Gaussian processes in terms of their spectral properties is derived. This expression can be seen as the result of orthogonalization over the different components in the polyphase decomposition of the process. We use this expression to derive, in a closed form, the DRF of several cyclostationary processes arising in practice. We first consider the DRF of a combined sampling and source coding problem. It is known that the optimal coding strategy for this problem involves source coding applied to a signal with the same structure as one resulting from pulse amplitude modulation (PAM). Since a PAM-modulated signal is cyclostationary, our DRF expression can be used to solve for the minimal distortion in the combined sampling and source coding problem. We also analyze in more detail the DRF of a source with the same structure as a PAM-modulated signal, and show that it is obtained by reverse waterfilling over an expression that depends on the energy of the pulse and the baseband process modulated to obtain the PAM signal. This result is then used to explore the effect of the symbol rate in PAM on the DRF of its output. In addition, we also study the DRF of sources with an amplitude-modulation structure, and show that the DRF of a narrow-band Gaussian stationary process modulated by either a deterministic or a random phase sine-wave equals the DRF of the baseband process. [ABSTRACT FROM PUBLISHER]

The amount of information lost in sub-Nyquist sampling of a continuous-time Gaussian stationary process is quantified. We consider a combined source coding and sub-Nyquist reconstruction problem in which the input to the encoder is a noisy sub-Nyquist sampled version of the analog source. We first derive an expression for the mean squared error in the reconstruction of the process from a noisy and information rate-limited version of its samples. This expression is a function of the sampling frequency and the average number of bits describing each sample. It is given as the sum of two terms: minimum mean square error in estimating the source from its noisy but otherwise fully observed sub-Nyquist samples, and a second term obtained by reverse waterfilling over an average of spectral densities associated with the polyphase components of the source. We extend this result to multi-branch uniform sampling, where the samples are available through a set of parallel channels with a uniform sampler and a pre-sampling filter in each branch. Further optimization to reduce distortion is then performed over the pre-sampling filters, and an optimal set of pre-sampling filters associated with the statistics of the input signal and the sampling frequency is found. This results in an expression for the minimal possible distortion achievable under any analog-to-digital conversion scheme involving uniform sampling and linear filtering. These results thus unify the Shannon–Whittaker–Kotelnikov sampling theorem and Shannon rate-distortion theory for Gaussian sources. [ABSTRACT FROM PUBLISHER]

The chaos expansion of a general non-linear function of a Gaussian stationary increment process conditioned on its past realizations is derived. This work combines the Wiener chaos expansion approach to study the dynamics of a stochastic system with the classical problem of the prediction of a Gaussian process based on a realization of its past. This is done by considering special bases for the Gaussian space 𝒢 generated by the process, which allows us to obtain an orthogonal basis for the Fock space of 𝒢 such that each basis element is either measurable or independent with respect to the given samples. This allows us to easily derive the chaos expansion of a random variable conditioned on part of the sample path. We provide a general method for the construction of such basis when the underlying process is Gaussian with stationary increment. We evaluate the basis elements in the case of the fractional Brownian motion, which leads to a prediction formula for this process. [ABSTRACT FROM PUBLISHER]

Given a Gaussian stationary increment processes, we show that a Skorokhod-Hitsuda stochastic integral with respect to this process, which obeys the Wick-Itô calculus rules, can be naturally defined using ideas taken from Hida's white noise space theory. We use the Bochner-Minlos theorem to associate a probability space to the process, and define the counterpart of the S-transform in this space. We then use this transform to define the stochastic integral and prove an associated Itô formula. [ABSTRACT FROM AUTHOR]