Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels With Memory and Feedback.

We derive sequential necessary and sufficient conditions for any channel input conditional distribution \mathcal P0,n\triangleq \{PXt|X^{t-1,Y^{t-1}}:t=0,\ldots ,n\} to maximize the finite-time horizon directed information defined by C^\mathrm FBX^{n \rightarrow Y^{n}} \triangleq \sup \cal P0,n I(X^{n}\rightarrow ~{Y^{n}}) , where I(X^n \rightarrow Y^n) =\sum t=0^n{I}(X^{t};Yt|Y^{t-1}) , for channel distributions \PYt|Y^{t-1,Xt:t=0,\ldots ,n\} and \PYt|Yt-M^{t-1,Xt:~t=0,\ldots ,n\} , where Y^t\triangleq \Y^-1, Y0,\ldots ,Y_t\ and X^t\triangleq \X0,\ldots ,X_t\ are the channel input and output random processes, and $M$ is a finite non-negative integer. We apply the necessary and sufficient conditions to application examples of time-varying channels with memory to derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Furthermore, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit C_{X^\infty \rightarrow Y^\infty }^{\mathrm{ FB}} \triangleq \lim _{ n \longrightarrow \infty } ({1}/({n+1})) C_{X^{n} \rightarrow Y^{n}}^{\mathrm{ FB}} without any ´ a priori assumptions, such as stationarity, ergodicity, or irreducibility of the channel distribution. The framework based on sequential necessary and sufficient conditions can be easily applied to a variety of channels with memory, beyond the ones considered in this paper. [ABSTRACT FROM PUBLISHER]