Search

We investigate the control-coding (CC) capacity of general dynamical decision models (DMs) that involve nonlinear filtering, which is absent in the specific DMs investigated in [C. K. Kourtellaris and C. D. Charalambous, IEEE Trans. Inform. Theory, 64 (2018), pp. 4962--4992]. We derive characterizations of CC capacity and we show their equivalence to extremum problems of maximizing the information theoretic measure of directed information from the input process to the output process of the DM over randomized strategies. Due to the generality of the DMs, the CC capacity is shown to be equivalent to partially observable Markov decision problems, contrary to the DMs in the above mentioned paper, which give rise to fully observable Markov decision problems. Subsequently, the CC capacity is transformed, using nonlinear filtering theory, to fully observable Markov decision problems. For the application example of a Gaussian DM with past dependence on inputs and outputs, we prove a decentralized separation principle that states optimal inputs are Gaussian and consist of (i) a control, (ii) an estimation, and (iii) an information transmission part, which interact in a specific order. The optimal control and estimation parts are related to linear-quadratic Gaussian stochastic optimal control problems with partial information. Various degenerated cases are discussed, including examples from the above mentioned paper, which do not involve estimation. [ABSTRACT FROM AUTHOR]

Motivated by the demand of reliable and finite blocklength communications, we employ tools from information theory, stochastic processes and queueing theory, in order to provide a comprehensive framework regarding the analysis of a Time Division Multiple Access (TDMA) network with bursty traffic, in the finite blocklength regime. Specifically, we re-examine the stability conditions of a non-cooperative TDMA multiple access channel, evaluate the optimal throughput, and identify the optimal data packet size, $k$ , for fixed codeword of blocklength, $n$. The evaluation is performed both numerically and via the proposed approximations, which result in closed form expressions and provide insight on how the optimal data size, $k^{*}$ , relates to the information metrics of channel capacity and channel dispersion in the finite blocklength regime. Then, we examine the stability conditions and the performance of the Multiple Access Relay Channel with TDMA scheduling, subject to finite blocklength constraints, by applying a cognitive cooperation protocol that assumes relaying is enabled when sources are idle. Finally, we propose the novel Batch-And-Forward (BAF) strategy, a mechanism that allows terminals to send batches of data packets instead on individual data packets, and evaluate the stability conditions and the optimal throughput. Numerical evaluation of the proposed strategy indicates that the performance of the cooperative network in the finite blocklength regime, in terms of throughput, can be significantly enhanced. Moreover, it reduces the requirement in control signals (metadata) , since, it avoids the unnecessary repetition of metadata (e.g. address of the source terminal and the destination). The BAF strategy is quite versatile, thus, it can be embedded in existing cooperative protocols, without imposing additional complexity on the overall scheme. [ABSTRACT FROM AUTHOR]

The control-coding (CC) capacity of dynamical decision models (DMs) is defined as the maximum amount of information transfer per unit time from its inputs to its outputs, called CC rate R in bits/second, which is operational with the aid of a controller-encoder and a decoder, as in Shannon's mathematical theory of communication over noisy channels, with the encoder replaced by a controller-encoder [C. Kourtellaris and C. D. Charalambous, IEEE Trans. Inform. Theory, 64 (2018), pp. 4962-4992]. In the first part of the paper, data processing inequalities and information structures of optimal controllers-encoders are derived for controlled and uncontrolled information processes. Further, the ergodic theory of a Markov decision is applied to establish direct and converse CC theorems. In the second part of the paper, the problem of signalling information via a Gaussian decision model (G-DM) subject to a quadratic cost constraint to another Gaussian control system (G-CS) with a quadratic pay-off is investigated. A hierarchical decomposition and decentralized optimality of linear quadruple of strategies, fcontroller-encoder, decoder, controllerg, is shown by construction as follows: (i) the controller-encoder simultaneously controls the output of the G-DM and encodes the state of the G-CS, and operates at the CC capacity of the G-DM; (ii) the decoder is optimal with respect to a mean square error (MSE) criterion; and (iii) the controller of the G-CS which acts on the decoder's information is optimal. [ABSTRACT FROM AUTHOR]

The control-coding (CC) capacity of dynamical decision models (DMs) is defined as the maximum amount of information transfer per unit time from its inputs to its outputs, called CC rate R in bits/second, which is operational with the aid of a controller-encoder and a decoder, as in Shannon's mathematical theory of communication over noisy channels, with the encoder replaced by a controller-encoder [C. Kourtellaris and C. D. Charalambous, IEEE Trans. Inform. Theory, 64 (2018), pp. 4962-4992]. In the first part of the paper, data processing inequalities and information structures of optimal controllers-encoders are derived for controlled and uncontrolled information processes. Further, the ergodic theory of a Markov decision is applied to establish direct and converse CC theorems. In the second part of the paper, the problem of signalling information via a Gaussian decision model (G-DM) subject to a quadratic cost constraint to another Gaussian control system (G-CS) with a quadratic pay-off is investigated. A hierarchical decomposition and decentralized optimality of linear quadruple of strategies, {controller-encoder, decoder, controller}, is shown by construction as follows: (i) the controller-encoder simultaneously controls the output of the G-DM and encodes the state of the G-CS, and operates at the CC capacity of the G-DM; (ii) the decoder is optimal with respect to a mean square error (MSE) criterion; and (iii) the controller of the G-CS which acts on the decoder's information is optimal. [ABSTRACT FROM AUTHOR]

A method is developed to realize optimal channel input conditional distributions, which maximize the finite transmission feedback information (FTFI) capacity, often called $n$ -block length feedback capacity, by information lossless randomized strategies. The method is applied to compute closed form expressions for the FTFI capacity and feedback capacity, of nonstationary, nonergodic, unstable, multiple input multiple output Gaussian channels with memory on past channel outputs, subject to average transmission cost constraints of quadratic form in the channel inputs and outputs. It is shown that randomized strategies decompose into two orthogonal parts-an deterministic part, which controls the channel output process, and an innovation part, which transmits new information over the channel. Then a separation principle is shown between the computation of the optimal deterministic part and the random part of the optimal randomized strategies. Finally, the ergodic theory of linear-quadratic-Gaussian stochastic optimal control theory, is applied to identify sufficient conditions, expressed in terms of solutions to matrix difference and algebraic Riccati equations, so that the optimal control part of randomized strategies induces asymptotic stationarity and ergodicity, and feedback capacity is characterized by the per unit time limit of the FTFI capacity. The method reveals an interaction of the control and the information transmission parts of the optimal randomized strategies, and that whether feedback increases capacity, is directly related to the channel parameters and the transmission cost function, through the solutions of the matrix Riccati equations. For unstable channels, it is shown that feedback capacity exists and it is strictly positive, provided the power exceeds a critical threshold. [ABSTRACT FROM AUTHOR]

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions ${\mathbf{P}}_{B_{i}|B^{i-1}, A_{i}}$ and ${\mathbf{P}}_{B_{i}|B_{i-M}^{i-1}, A_{i}}$ , where $M$ is the memory of the channel, $B^{i} \stackrel {\triangle }{=}\{B^{-1},B_{0}, \ldots, B_{i}\}$ are the channel outputs, and $A^{i} \stackrel {\triangle }{=}\{A_{0}, A_{1}, \ldots, A_{i}\}$ , are the channel inputs, for $i=0, \ldots, n$. The characterizations of FTFI capacity are obtained by first identifying the information structures of the optimal channel input conditional distributions ${\mathscr P}_{[0, n]} \stackrel {\triangle }{=}\big \{ {\mathbf{P}}_{A_{i}|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big \}$ , which maximize directed information $C_{A^{n} \rightarrow B^{n}}^{FB} \stackrel {\triangle }{=}\sup _{\mathscr P_{[0, n]} } I(A^{n} \rightarrow B^{n}), \hspace {.2in} I(A^{n} \rightarrow B^{n}) \stackrel {\triangle }{=}\sum _{i=0}^{n} I(A^{i};B_{i}|B^{i-1})$. The main theorem states that, for any channel with memory $M$ , the optimal channel input conditional distributions occur in the subset satisfying conditional independence $\stackrel {\circ }{\mathscr P}_{[0, n]} \stackrel {\triangle }{=}\big \{ {\mathbf{P}}_{A_{i}|A^{i-1}, B^{i-1}}= {\mathbf{P}}_{A_{i}|B_{i-M}^{i-1}}: i=0, \ldots, n\big \}$ , and the characterization of FTFI capacity is given by $C_{A^{n} \rightarrow B^{n}}^{FB, M} \stackrel {\triangle }{=}\sup _{ \stackrel {\circ }{\mathscr P}_{[0, n]} } \sum _{i=0}^{n} I(A_{i}; B_{i}|B_{i-M}^{i-1})$. Similar conclusions are derived for problems with average cost constraints of the form $\frac {1}{n+1} {\mathbf{E}}\Big \{c_{0,n}(A^{n}, B^{n-1})\Big \} \leq \kappa,\,\, \kappa \geq 0$ , for specific functions $c_{0,n}(a^{n},b^{n-1})$. The feedback capacity is addressed by investigating $\lim _{n \longrightarrow \infty } \frac {1}{n+1}C_{A^{n} \rightarrow B^{n}}^{FB, M} $. The methodology utilizes stochastic optimal control theory, to identify the control process, the controlled process, and often a variational equality of directed information, to derive upper bounds on $I(A^{n} \rightarrow B^{n})$ , which are achievable over specific subsets of channel input conditional distributions ${\mathscr P}_{[0, n]}$ , which are characterized by conditional independence. The main results illustrate a direct analogy, in terms of conditional independence, of the characterizations of FTFI capacity and Shannon’s capacity formulae of memoryless channels. An example is presented to illustrate the role of optimal channel input process in the derivations of the direct and converse coding theorems. [ABSTRACT FROM AUTHOR]

This paper derives the optimal test channel distribution and the complete characterization of the classical Gorbunov and Pinsker (1973), Gorbunov and Pinsker (1974) nonanticipatory epsilon entropy of multivariate Gaussian Markov sources with square-error fidelity, which remained an open problem since 1974. The paper also formulates a state dependent nonanticipatory epsilon entropy, in which past reproductions are available to the decoder and not to the encoder, the test channel is specified with respect to an auxiliary (state) random process, and the reproduction process is a causal function of past reproduction and the auxiliary random process. This variation is analogous to the Wyner and Ziv (1976) and Wyner (1978) rate distortion function (RDF), of memoryless sources. It is shown that the operational rate of zero-delay codes, with past reproductions available to the decoder but not to the encoder is bounded below by the state dependent nonanticipatory epsilon entropy rate. For the case of multivariate Gaussian Markov sources with square-error fidelity, the optimal test channel distribution and the complete characterization of the state dependent of nonanticipatory epsilon entropy are derived, and also shown that that the two nonanticipatory epsilon entropies coincide. The derivations are new; they are based on structural properties of the stochastic realizations of the reproduction process that induce the optimal test channel distributions. They are derived using, achievable lower bounds on information theoretic measures, properties of mean-square estimation theory, Hadamard’s inequality, and canonical correlation coefficients of a tuple of multivariate jointly Gaussian random processes. Applications of the nonanticipatory epsilon entropy and its state dependent variation are discussed to the areas of control of unstable Gaussian systems over limited memory channels, design of causal estimators for Gaussian Markov sources with a fidelity criterion, computation of the rate loss of causal and zero-delay codes of Gaussian Markov sources with respect to non-causal codes. [ABSTRACT FROM AUTHOR]

We derive sequential necessary and sufficient conditions for any channel input conditional distribution \mathcal P0,n\triangleq \{PXt|X^{t-1,Y^{t-1}}:t=0,\ldots ,n\} to maximize the finite-time horizon directed information defined by C^\mathrm FBX^{n \rightarrow Y^{n}} \triangleq \sup \cal P0,n I(X^{n}\rightarrow ~{Y^{n}}) , where I(X^n \rightarrow Y^n) =\sum t=0^n{I}(X^{t};Yt|Y^{t-1}) , for channel distributions \PYt|Y^{t-1,Xt:t=0,\ldots ,n\} and \PYt|Yt-M^{t-1,Xt:~t=0,\ldots ,n\} , where Y^t\triangleq \Y^-1, Y0,\ldots ,Y_t\ and X^t\triangleq \X0,\ldots ,X_t\ are the channel input and output random processes, and $M$ is a finite non-negative integer. We apply the necessary and sufficient conditions to application examples of time-varying channels with memory to derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Furthermore, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit C_{X^\infty \rightarrow Y^\infty }^{\mathrm{ FB}} \triangleq \lim _{ n \longrightarrow \infty } ({1}/({n+1})) C_{X^{n} \rightarrow Y^{n}}^{\mathrm{ FB}} without any ´ a priori assumptions, such as stationarity, ergodicity, or irreducibility of the channel distribution. The framework based on sequential necessary and sufficient conditions can be easily applied to a variety of channels with memory, beyond the ones considered in this paper. [ABSTRACT FROM PUBLISHER]

This paper provides a connection between the shot-noise analysis of Rice and the statistical analysis of multipath fading wireless channels when the received signals are a low-pass signal and a bandpass signal. Under certain conditions, explicit expressions are obtained for autocorrelation functions, power spectral densities, and moment-generating functions. In addition, a central limit theorem is derived identifying the mean and covariance of the received signals, which is a generalization of Campbell's theorem. The results are easily applicable to transmitted signals which are random and to CDMA signals. [ABSTRACT FROM AUTHOR]