Search

A new approach to $L_{2}$ -consistent estimation of a general density functional using $k$ -nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$ -nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f$. Some instantiations of the proposed estimator recover existing $k$ -nearest neighbor based estimators of Shannon and Rényi entropies and Kullback–Leibler and Rényi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_{2}$ -consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities. [ABSTRACT FROM AUTHOR]

We consider the problem of transmitting data at rate R over a state-dependent channel p(y|x, s) with state information available at the sender and at the same time conveying the information about the channel state itself to the receiver. The amount of state information that can be learned at the receiver is captured by the mutual information I([S.sup.n]; [Y.sup.n]) between the state sequence [S.sup.n] and the channel output [Y.sup.n]. The optimal tradeoff is characterized between the information transmission rate R and the state uncertainty reduction rate [DELTA], when the state information is either causally or noncausally available at the sender. In particular, when state transmission is the only goal, the maximum uncertainty reduction rate is given by [[DELTA].sup.*] = [max.sub.p(x|s)] I(X, S; Y). This result is closely related and in a sense dual to a recent study by Merhav and Shamai, which solves the problem of masking the state information from the receiver rather than conveying it. Index Terms--Capacity, causal state information, channels with state information, joint source--channel coding, noncausal state information, state amplification, state uncertainty reduction, writing on dirty paper.

A coding theorem is proved for a class of stationary channels with feedback in which the output [Y.sub.n] = f ([X.sup.n.sub.n-m], [Z.sup.n.sub.n-m]) is the function of the current and past m symbols from the channel input [X.sub.n] and the stationary ergodic channel noise [Z.sub.n]. In particular, it is shown that the feedback capacity is equal to [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] where I ([X.sup.n] [right arrow] [Y.sup.n]) = [[summation of].sup.n.sub.i=1] ([X.sup.i]; [Y.sub.i]|[Y.sup.i-1]) denotes the Massey directed information from the channel input to the output, and the supremum is taken over all causally conditioned distributions p([x.sup.n][parallel][y.sup.n-1]) = [[PI].sup.n.sub.i=1] p([x.sub.i]|[x.sup.i=1], [y.sup.i-1]). The main ideas of the proof are a classical application of the Shannon strategy for coding with side information and a new elementary coding technique for the given channel model without feedback, which is in a sense dual to Gallager's lossy coding of stationary ergodic sources. A similar approach gives a simple alternative proof of coding theorems for finite state channels by Yang-Kavcic--Tatikonda, Chen--Berger, and Permuter--Weissman--Goldsmith. Index Terms--Capacity, channels with memory, coding theorem, directed information, ergodic decomposition, feedback, feedback capacity, Shannon strategy.

Despite numerous bounds and partial results, the feedback capacity of the stationary nonwhite Gaussian additive noise channel has been open, even for the simplest cases such as the first-order autoregressive Gaussian channel studied by Butman, Tiernan and Schalkwijk, Wolfowitz, Ozarow, and more recently, Yang, Kavcic, and Tatikonda. Here we consider another simple special case of the stationary first-order moving average additive Gaussian noise channel and find the feedback capacity in closed form. Specifically, the channel is given by [Y.sub.i] = [X.sub.i] + [Z.sub.i], i = 1, 2, ..., where the input {[X.sub.i]} satisfies a power constraint and the noise {[Z.sub.i]} is a first-order moving average Gaussian process defined by [Z.sub.i] = [alpha] [U.sub.i]-1 + [U.sub.i], |[alpha]| [less than or equal to] 1, with white Gaussian innovations [U.sub.i], i = 0, 1,.... We show that the feedback capacity of this channel is [C.sub.FB] = -log [x.sub.o] where [x.sub.o] is the unique positive root of the equation [rho][x.sup.2] = (i - [ x.sup.2])[(i - |[alpha]|x).sup.2] and [rho] is the ratio of the average input power per transmission to the variance of the noise innovation [U.sub.i]. The optimal coding scheme parallels the simple linear signaling scheme by Schalkwijk and Kailath for the additive white Gaussian noise channel--the transmitter sends a real-valued information-bearing signal at the beginning of communication and subsequently refines the receiver's knowledge by processing the feedback noise signal through a linear stationary first-order autoregressive filter. The resulting error probability of the maximum likelihood decoding decays doubly exponentially in the duration of the communication. Refreshingly, this feedback capacity of the first-order moving average Gaussian channel is very similar in form to the best known achievable rate for the first-order autoregressive Gaussian noise channel given by Butman. Index Terms--Additive Gaussian noise channels, feedback capacity, first-order moving average, Gaussian feedback capacity, linear feedback, Schalkwijk--Kailath coding scheme.

The dependence of the Gaussian input information rate on the line-of-sight (LOS) matrix in multiple-input multiple-output (MIMO) coherent Rician fading channels is explored. It is proved that the outage probability and the mutual information induced by a multivariate circularly symmetric Gaussian input with any covariance matrix are monotonic in the LOS matrix D, or more precisely, monotonic in Dt D in the sense of the Loewner partial order. Conversely, it is also demonstrated that this ordering on the LOS matrices is a necessary condition for the uniform monotonicity over all input covariance matrices. This result is subsequently applied to prove the monotonicity of the isotropic Gaussian input information rate and channel capacity in the singular values of the LOS matrix. Extensions to multiple-access channels (MAC) are also provided. Index Terms--Capacity, line of sight, multiple-access channels (MAC), multiple-input multiple-output (MIMO), outage, partial order, preorder, Rician fading.

We formulate a problem of state information transmission over a state-dependent channel with states known at the transmitter. In particular, we solve a problem of minimizing the mean-squared channel state estimation error E[parallel][S.sup.n] - [[bar.S].sup.n][parallel] for a state-dependent additive Gaussian channel [Y.sup.n] = [X.sup.n] + [S.sup.n] + [Z.sup.n] with an independent and identically distributed (i.i.d.) Gaussian state sequence [S.sup.n] = ([S.sub.1], ..., [S.sub.n]) known at the transmitter and an unknown i.i.d, additive Gaussian noise [Z.sup.]. We show that a simple technique of direct state amplification (i.e., [X.sup.n] = [alpha][S.sup.n]), where the transmitter uses its entire power budget to amplify the channel state, yields the minimum mean-squared state estimation error. This same channel can also be used to send additional independent information at the expense of a higher channel state estimation error. We characterize the optimal tradeoff between the rate R of the independent information that can be reliably transmitted and the mean-squared state estimation error D. We show that any optimal (R, D) tradeoff pair can be achieved via a simple power-sharing technique, whereby the transmitter power is appropriately allocated between pure information transmission and state amplification. Index Terms--Additive Gaussian noise channels, channels with state information, joint source-channel coding, state amplification, state estimation.

The capacity of a class of deterministic relay channels with transmitter input X, receiver output Y, relay output [Y.sub.1] = f(X, Y), and separate noiseless communication link of capacity [R.sub.0] from the relay to the receiver, is shown to be C([R.sub.o]) = sup min{I(X:Y) + [R.sub.0], I(X:[Y.sub.1])}/p(x) Roughly speaking, every hit from the relay is worth one bit to the receiver until saturation at capacity. Index Terms--Capacity, deterministic relay channel, primitive relay channel.

We provide a counterexample to Cover's conjecture that the feedback capacity [C.sub.FB] of an additive Gaussian noise channel under power constraint P be no greater than the nonfeedback capacity C of the same channel under power constraint 2P, i.e., [C.sub.FB](P) [less than or equal to] C(2P). Index Terms--Additive Gaussian noise channels, channel capacity, conjecture, counterexample, feedback.

The optimal rate region for interference networks is characterized when encoding is restricted to random code ensembles with superposition coding and time sharing. A simple simultaneous nonunique decoding rule, under which each receiver decodes for the intended message as well as the interfering messages, is shown to achieve this optimal rate region regardless of the relative strengths of signal, interference, and noise. This result implies that the Han-Kobayashi bound, the best known inner bound on the capacity region of the two-user pair interference channel, cannot be improved merely by using the optimal maximum likelihood decoder. [ABSTRACT FROM AUTHOR]

Optimal coding over the additive white Gaussian noise channel under the peak energy constraint is studied when there is noisy feedback over an orthogonal additive white Gaussian noise channel. As shown by Pinsker, under the peak energy constraint, the best error exponent for communicating an M-ary message, M\geq 3, with noise-free feedback is strictly larger than the one without feedback. This paper extends Pinsker's result and shows that if the noise power in the feedback link is sufficiently small, the best error exponent for communicating an M-ary message can be strictly larger than the one without feedback. The proof involves two feedback coding schemes. One is motivated by a two-stage noisy feedback coding scheme of Burnashev and Yamamoto for binary symmetric channels, while the other is a linear noisy feedback coding scheme that extends Pinsker's noise-free feedback coding scheme. When the feedback noise power \alpha is sufficiently small, the linear coding scheme outperforms the two-stage (nonlinear) coding scheme, and is asymptotically optimal as \alpha tends to zero. By contrast, when \alpha is relatively larger, the two-stage coding scheme performs better. [ABSTRACT FROM AUTHOR]

A notion of directed information between two continuous-time processes is proposed. A key component in the definition is taking an infimum over all possible partitions of the time interval, which plays a role no less significant than the supremum over “space” partitions inherent in the definition of mutual information. Properties and operational interpretations in estimation and communication are then established for the proposed notion of directed information. For the continuous-time additive white Gaussian noise channel, it is shown that Duncan's classical relationship between causal estimation error and mutual information continues to hold in the presence of feedback upon replacing mutual information by directed information. A parallel result is established for the Poisson channel. The utility of this relationship is demonstrated in computing the directed information rate between the input and output processes of a continuous-time Poisson channel with feedback, where the channel input process is constrained to be constant between events at the channel output. Finally, the capacity of a wide class of continuous-time channels with feedback is established via directed information, characterizing the fundamental limit on reliable communication. [ABSTRACT FROM AUTHOR]