Search

Joint source–channel coding (JSCC) based on deep learning has shown significant advancements in image transmission tasks. However, previous channel-adaptive JSCC methods often rely on the signal-to-noise ratio (SNR) of the current channel for encoding, which overlooks the neural network's self-adaptive capability across varying SNRs. This paper investigates the self-adaptive capability of deep learning-based JSCC models to dynamically changing channels and introduces a novel method named Channel-Blind JSCC (CBJSCC). CBJSCC leverages the intrinsic learning capability of neural networks to self-adapt to dynamic channels and diverse SNRs without relying on external SNR information. This approach is advantageous, as it is not affected by channel estimation errors and can be applied to one-to-many wireless communication scenarios. To enhance the performance of JSCC tasks, the CBJSCC model employs a specially designed encoder–decoder. Experimental results show that CBJSCC outperforms existing channel-adaptive JSCC methods that depend on SNR estimation and feedback, both in additive white Gaussian noise environments and under slow Rayleigh fading channel conditions. Through a comprehensive analysis of the model's performance, we further validate the robustness and adaptability of this strategy across different application scenarios, with the experimental results providing strong evidence to support this claim. [ABSTRACT FROM AUTHOR]

We consider the problem of learned speech transmission. Existing methods have exploited joint source–channel coding (JSCC) to encode speech directly to transmitted symbols to improve the robustness over noisy channels. However, the fundamental limit of these methods is the failure of identification of content diversity across speech frames, leading to inefficient transmission. In this paper, we propose a novel neural speech transmission framework named NST. It can be optimized for superior rate–distortion–perception (RDP) performance toward the goal of high-fidelity semantic communication. Particularly, a learned entropy model assesses latent speech features to quantify the semantic content complexity, which facilitates the adaptive transmission rate allocation. NST enables a seamless integration of the source content with channel state information through variable-length joint source–channel coding, which maximizes the coding gain. Furthermore, we present a streaming variant of NST, which adopts causal coding based on sliding windows. Experimental results verify that NST outperforms existing speech transmission methods including separation-based and JSCC solutions in terms of RDP performance. Streaming NST achieves low-latency transmission with a slight quality degradation, which is tailored for real-time speech communication. [ABSTRACT FROM AUTHOR]

In recent years, semantic communication has received significant attention from both academia and industry, driven by the growing demands for ultra-low latency and high-throughput capabilities in emerging intelligent services. Nonetheless, a comprehensive and effective theoretical framework for semantic communication has yet to be established. In particular, finding the fundamental limits of semantic communication, exploring the capabilities of semantic-aware networks, or utilizing theoretical guidance for deep learning in semantic communication are very important yet still unresolved issues. In general, the mathematical theory of semantic communication and the mathematical representation of semantics are referred to as semantic information theory. In this paper, we introduce the pertinent advancements in semantic information theory. Grounded in the foundational work of Claude Shannon, we present the latest developments in semantic entropy, semantic rate-distortion, and semantic channel capacity. Additionally, we analyze some open problems in semantic information measurement and semantic coding, providing a theoretical basis for the design of a semantic communication system. Furthermore, we carefully review several mathematical theories and tools and evaluate their applicability in the context of semantic communication. Finally, we shed light on the challenges encountered in both semantic communication and semantic information theory. [ABSTRACT FROM AUTHOR]

Joint source–channel coding (JSCC) based on deep learning has shown significant advancements in image transmission tasks. However, previous channel-adaptive JSCC methods often rely on the signal-to-noise ratio (SNR) of the current channel for encoding, which overlooks the neural network’s self-adaptive capability across varying SNRs. This paper investigates the self-adaptive capability of deep learning-based JSCC models to dynamically changing channels and introduces a novel method named Channel-Blind JSCC (CBJSCC). CBJSCC leverages the intrinsic learning capability of neural networks to self-adapt to dynamic channels and diverse SNRs without relying on external SNR information. This approach is advantageous, as it is not affected by channel estimation errors and can be applied to one-to-many wireless communication scenarios. To enhance the performance of JSCC tasks, the CBJSCC model employs a specially designed encoder–decoder. Experimental results show that CBJSCC outperforms existing channel-adaptive JSCC methods that depend on SNR estimation and feedback, both in additive white Gaussian noise environments and under slow Rayleigh fading channel conditions. Through a comprehensive analysis of the model’s performance, we further validate the robustness and adaptability of this strategy across different application scenarios, with the experimental results providing strong evidence to support this claim.

We consider the problem of transmitting a Gaussian source with minimum mean square error distortion over an infinite-bandwidth additive white Gaussian noise channel with an unknown noise level and under an input energy constraint. We construct a universal joint source–channel coding scheme with respect to the noise level, that uses modulo-lattice modulation with multiple layers. For each layer, we employ either analog linear modulation or analog pulse-position modulation (PPM). We show that the designed scheme with linear layers requires less energy compared to existing solutions to achieve the same quadratically increasing distortion profile with the noise level; replacing the linear layers with PPM layers offers an additional improvement. [ABSTRACT FROM AUTHOR]

We consider the problem of learned speech transmission. Existing methods have exploited joint source–channel coding (JSCC) to encode speech directly to transmitted symbols to improve the robustness over noisy channels. However, the fundamental limit of these methods is the failure of identification of content diversity across speech frames, leading to inefficient transmission. In this paper, we propose a novel neural speech transmission framework named NST. It can be optimized for superior rate–distortion–perception (RDP) performance toward the goal of high-fidelity semantic communication. Particularly, a learned entropy model assesses latent speech features to quantify the semantic content complexity, which facilitates the adaptive transmission rate allocation. NST enables a seamless integration of the source content with channel state information through variable-length joint source–channel coding, which maximizes the coding gain. Furthermore, we present a streaming variant of NST, which adopts causal coding based on sliding windows. Experimental results verify that NST outperforms existing speech transmission methods including separation-based and JSCC solutions in terms of RDP performance. Streaming NST achieves low-latency transmission with a slight quality degradation, which is tailored for real-time speech communication.

In this article, a graph-theoretic method (taking advantage of constraints among sets associated with the corresponding parity-check matrices) is applied for the construction of a double low-density parity-check (D-LDPC) code (also known as LDPC code pair) in a joint source-channel coding (JSCC) system. Specifically, we pre-set the girth of the parity-check matrix for the LDPC code pair when jointly designing the two LDPC codes, which are constructed by following the set constraints. The constructed parity-check matrices for channel codes comprise an identity submatrix and an additional submatrix, whose column weights can be pre-set to be any positive integer numbers. Simulation results illustrate that the constructed D-LDPC codes exhibit significant performance improvement and enhanced flexible frame length (i.e., adaptability under various channel conditions) compared with the benchmark code pair. [ABSTRACT FROM AUTHOR]

Although extensive optimization of encoding and decoding schemes for joint source-channel coding (JSCC) systems has been conducted, efficient optimization schemes are still required for designing and optimizing the linking matrix between variable nodes of the source code and check nodes of the channel code. A scheme has been proposed for design and optimization of linking matrix with multi-edges by analyzing the performance of the JSCC system using the joint protograph extrinsic information transfer algorithm to calculate decoding thresholds. The proposed scheme incorporates structural constraints and is effective in designing and optimizing the multi-edges in linking matrix for the JSCC system. Experimental results have demonstrated that the designed and optimized linking matrix significantly improves the performance of the JSCC system. Furthermore, the proposed scheme reduces the complexity of the solution space for the optimized example. [ABSTRACT FROM AUTHOR]

Despite the remarkable achievements of modern communication systems based on Shannon's theory, there is still considerable room for exploration in information transmission capacity, and semantic communication technology has emerged as a promising approach in this regard. Nonetheless, the benefits of semantic communication remain elusive, and the absence of a unified system model has hindered practical implementation. In this context, we contend that semantic communication can benefit from data distortion and the incorporation of natural language modeling information, such that source coding with semantic modeling information does not compromise the performance of semantic communication systems. To fortify our stance, a novel Separated Data-Semantic Coding (SDSC) system is proposed, which disentangles the source coding and semantic coding. Furthermore, relevant experiments are conducted to validate the contention and the SDSC system. By illuminating the superiority of semantic communication, the research not only contributes to the advancement of semantic communication technologies but also facilitates the development of more practical communication systems. [ABSTRACT FROM AUTHOR]

Index assignment (IA) is a low-complexity joint source-channel coding technique that has the potential for use in low-latency and low-power applications, such as wireless sensor networks (WSNs). Though binary IA has been extensively studied for assigning binary indices to quantized codewords (or symbols) under the assumption of binary symmetric channels (BSCs), real-world scenarios often use $M$ -ary modulations. Directly applying binary IAs designed for BSCs to $M$ -ary modulations results in suboptimal performance. In this paper, we investigate the $M$ -ary IA, which assigns $M$ -ary labels to quantized codewords (or symbols), assuming the use of a equiprobable lattice quantizer. For such a system, we derive a tight performance bound and propose a near-optimal IA scheme based on a two-step design. In addition, we propose explicit IA constructions for practical modulation schemes, including PAM, QAM, and PSK. Our proposed IA design is rigorously proven to be optimal for 3-PSK and QPSK, whereas for larger modulation orders, the proposed IA constructions approach the bounds within small gaps. Our simulations show that the constructed IA scheme can achieve significant energy savings compared to the conventional binary IA scheme. Specifically, in some WSN scenarios, the proposed IA for 16-QAM is shown to achieve significant reductions in energy consumption relative to the conventional binary counterpart.

The error probability of block codes sent under a non-uniform input distribution over the memoryless binary symmetric channel (BSC) and decoded via the maximum a posteriori (MAP) decoding rule is investigated. It is proved that the ratio of the probability of MAP decoder ties to the probability of error grows most linearly in blocklength when no MAP decoding ties occur, thus showing that decoder ties do not affect the code's error exponent. This result generalizes a similar recent result shown for the case of block codes transmitted over the BSC under a uniform input distribution. [ABSTRACT FROM AUTHOR]

Joint communications and sensing functionalities integrated into the same communication network have become increasingly relevant due to the large bandwidth requirements of next-generation wireless communication systems and the impending spectral shortage. While there exist system-level guidelines and waveform design specifications for such systems, an information-theoretic analysis of the absolute performance capabilities of joint sensing and communication systems that take into account practical limitations such as fading has not been addressed in the literature. Motivated by this, we undertake a network information-theoretic analysis of a typical joint communications and sensing system in this paper. Towards this end, we consider a state-dependent fading Gaussian multiple access channel (GMAC) setup with an additive state. The state process is assumed to be independent and identically distributed (i.i.d.) Gaussian, and non-causally available to all the transmitting nodes. The fading gains on the respective links are assumed to be stationary and ergodic and available only at the receiver. In this setting, with no knowledge of fading gains at the transmitters, we are interested in joint message communication and estimation of the state at the receiver to meet a target distortion in the mean-squared error sense. Our main contribution here is a complete characterization of the distortion-rate trade-off region between the communication rates and the state estimation distortion for a two-sender GMAC. Our results show that the optimal strategy is based on static power allocation and involves uncoded transmissions to amplify the state, along with the superposition of the digital message streams using appropriate Gaussian codebooks and dirty paper coding (DPC). This acts as a design directive for realistic systems using joint sensing and transmission in next-generation wireless standards and points to the relative benefits of uncoded communications and joint source-channel coding in such systems. [ABSTRACT FROM AUTHOR]

In recent years, semantic communication has received significant attention from both academia and industry, driven by the growing demands for ultra-low latency and high-throughput capabilities in emerging intelligent services. Nonetheless, a comprehensive and effective theoretical framework for semantic communication has yet to be established. In particular, finding the fundamental limits of semantic communication, exploring the capabilities of semantic-aware networks, or utilizing theoretical guidance for deep learning in semantic communication are very important yet still unresolved issues. In general, the mathematical theory of semantic communication and the mathematical representation of semantics are referred to as semantic information theory. In this paper, we introduce the pertinent advancements in semantic information theory. Grounded in the foundational work of Claude Shannon, we present the latest developments in semantic entropy, semantic rate-distortion, and semantic channel capacity. Additionally, we analyze some open problems in semantic information measurement and semantic coding, providing a theoretical basis for the design of a semantic communication system. Furthermore, we carefully review several mathematical theories and tools and evaluate their applicability in the context of semantic communication. Finally, we shed light on the challenges encountered in both semantic communication and semantic information theory.

A key component of the joint source-channel coding (JSCC) scheme based on double low-density parity-check (D-LDPC) codes is the introduction of a linking matrix between the source LDPC code and channel LDPC code, by which the decoding information including the source redundancy and channel state information can be transferred iteratively. However, the linking matrix is a fixed one-to-one mapping, i.e., an identity matrix in a conventional D-LDPC code system, which may not take full advantage of the decoding information. Therefore, this paper introduces a general linking matrix, i.e., a non-identity linking matrix, connecting the check nodes (CNs) of the source LDPC code and the variable nodes (VNs) of the channel LDPC code. Further, the encoding and decoding algorithms of the proposed D-LDPC coding system are generalized. A joint extrinsic information transfer (JEXIT) algorithm is derived for calculating the decoding threshold of the proposed system with a general linking matrix. In addition, several general linking matrices are optimized with the aid of the JEXIT algorithm. Finally, the simulation results demonstrate the superiority of the proposed D-LDPC coding system with general linking matrices. [ABSTRACT FROM AUTHOR]

In this paper, a joint group shuffled scheduling decoding (JGSSD) algorithm for a joint source-channel coding (JSCC) scheme based on double low-density parity-check (D-LDPC) codes is presented. The proposed algorithm considers the D-LDPC coding structure as a whole and applies shuffled scheduling to each group; the grouping relies on the types or the length of the variable nodes (VNs). By comparison, the conventional shuffled scheduling decoding algorithm can be regarded as a special case of this proposed algorithm. A novel joint extrinsic information transfer (JEXIT) algorithm for the D-LDPC codes system with the JGSSD algorithm is proposed, by which the source and channel decoding are calculated with different grouping strategies to analyze the effects of the grouping strategy. Simulation results and comparisons verify the superiority of the JGSSD algorithm, which can adaptively trade off the decoding performance, complexity and latency. [ABSTRACT FROM AUTHOR]

A two-terminal distributed binary hypothesis testing problem over a noisy channel is studied. The two terminals, called the observer and the decision maker, each has access to n independent and identically distributed samples, denoted by U and V , respectively. The observer communicates to the decision maker over a discrete memoryless channel, and the decision maker performs a binary hypothesis test on the joint probability distribution of (U , V) based on V and the noisy information received from the observer. The trade-off between the exponents of the type I and type II error probabilities is investigated. Two inner bounds are obtained, one using a separation-based scheme that involves type-based compression and unequal error-protection channel coding, and the other using a joint scheme that incorporates type-based hybrid coding. The separation-based scheme is shown to recover the inner bound obtained by Han and Kobayashi for the special case of a rate-limited noiseless channel, and also the one obtained by the authors previously for a corner point of the trade-off. Finally, we show via an example that the joint scheme achieves a strictly tighter bound than the separation-based scheme for some points of the error-exponents trade-off. [ABSTRACT FROM AUTHOR]

Minimum energy required to achieve a distortion-noise profile, i.e., a function indicating the maximum allowed distortion value for each channel noise level, is studied for transmission of Gaussian sources over Gaussian channels. A general coding scheme is proposed to upper bound the minimum energy needed for any distortion-noise profile. Conversely, utilizing a family of lower bounds originally derived for broadcast channels with power constraints, the minimum required energy is lower bounded for any profile. As examples, linear, exponential, square-law, and staircase profiles are studied. It is shown that for the linear profile, as well as for any concave profile, uncoded transmission is optimal. As a negative result, it is shown that exponential profiles are not achievable with finite energy. For square-law and staircase profiles, upper bounds and lower bounds are derived and the gaps between them are studied. [ABSTRACT FROM AUTHOR]

We consider the problem of transmitting a Gaussian source with minimum mean square error distortion over an infinite-bandwidth additive white Gaussian noise channel with an unknown noise level and under an input energy constraint. We construct a universal joint source–channel coding scheme with respect to the noise level, that uses modulo-lattice modulation with multiple layers. For each layer, we employ either analog linear modulation or analog pulse-position modulation (PPM). We show that the designed scheme with linear layers requires less energy compared to existing solutions to achieve the same quadratically increasing distortion profile with the noise level; replacing the linear layers with PPM layers offers an additional improvement.

It is well known that independent (separate) encoding of $K$ correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where it is the same source that is encoded in each description. We observe that under mild conditions about the source and distortion measure, the sum-rate of $K$ separately encoded individually good descriptions tends to the rate-distortion function of the joint decoder in the limit of vanishing small coding rates of the descriptions. Moreover, we then propose to successively encode the source into $K$ independent descriptions in each round in order to achieve a final distortion $D$ after $M$ rounds. We provide two examples – a Gaussian source with mean-squared error and an exponential source with one-sided error – for which the excess rate vanishes in the limit as the number of rounds $M$ goes to infinity, for any fixed $D$ and $K$. This result has an interesting interpretation for a multi-round variant of the multiple descriptions problem, where after each round the encoder gets a (block) feedback regarding which of the descriptions arrived: In the limit as the number of rounds $M$ goes to infinity (i.e., many incremental rounds), the total rate of received descriptions approaches the rate-distortion function. We provide theoretical and experimental evidence showing that this phenomenon is in fact more general than in the two examples above. [ABSTRACT FROM AUTHOR]

We present DeepWiVe, the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our DNN decoder predicts residuals without distortion feedback, which improves the video quality by accounting for occlusion/disocclusion and camera movements. We simultaneously train different bandwidth allocation networks for the frames to allow variable bandwidth transmission. Then, we train a bandwidth allocation network using reinforcement learning (RL) that optimizes the allocation of limited available channel bandwidth among video frames to maximize the overall visual quality. Our results show that DeepWiVe can overcome the cliff-effect, which is prevalent in conventional separation-based digital communication schemes, and achieve graceful degradation with the mismatch between the estimated and actual channel qualities. DeepWiVe outperforms H.264 video compression followed by low-density parity check (LDPC) codes in all channel conditions by up to 0.0485 in terms of the multi-scale structural similarity index measure (MS-SSIM), and H.265+ LDPC by up to 0.0069 on average. We also illustrate the importance of optimizing bandwidth allocation in JSCC video transmission by showing that our optimal bandwidth allocation policy is superior to uniform allocation as well as a heuristic policy benchmark. [ABSTRACT FROM AUTHOR]

In this article, a graph-theoretic method (taking advantage of constraints among sets associated with the corresponding parity-check matrices) is applied for the construction of a double low-density parity-check (D-LDPC) code (also known as LDPC code pair) in a joint source-channel coding (JSCC) system. Specifically, we pre-set the girth of the parity-check matrix for the LDPC code pair when jointly designing the two LDPC codes, which are constructed by following the set constraints. The constructed parity-check matrices for channel codes comprise an identity submatrix and an additional submatrix, whose column weights can be pre-set to be any positive integer numbers. Simulation results illustrate that the constructed D-LDPC codes exhibit significant performance improvement and enhanced flexible frame length (i.e., adaptability under various channel conditions) compared with the benchmark code pair.

Although extensive optimization of encoding and decoding schemes for joint source-channel coding (JSCC) systems has been conducted, efficient optimization schemes are still required for designing and optimizing the linking matrix between variable nodes of the source code and check nodes of the channel code. A scheme has been proposed for design and optimization of linking matrix with multi-edges by analyzing the performance of the JSCC system using the joint protograph extrinsic information transfer algorithm to calculate decoding thresholds. The proposed scheme incorporates structural constraints and is effective in designing and optimizing the multi-edges in linking matrix for the JSCC system. Experimental results have demonstrated that the designed and optimized linking matrix significantly improves the performance of the JSCC system. Furthermore, the proposed scheme reduces the complexity of the solution space for the optimized example.

To reduce the waste of energy in communications, unequal error protection (UEP) is used to provide asymmetric protection for messages with different levels of importance. This paper proposes new efficient strategies of UEP based on bilayer protograph-based low-density parity check (PLDPC) codes in decoding-and-forward (DF) relay systems. In particular, we jointly utilize source coding and channel coding to design UEP strategies and then save transmission energy. According to the different levels of importance of discrete cosine transform (DCT) coefficients of image and variance statistical characteristics of image sub-blocks, bilayer-lengthened PLDPC codes are exploited to protect the transmitted image information with different importance levels at the half-duplex relay system. In the end, the simulation result shows that the proposed UEP schemes achieve excellent performance gains compared to conventional equal error protection (EEP) scheme. Additionally, the complexity analysis of the UEP strategies is given. [ABSTRACT FROM AUTHOR]

Recently, semantic communication has been brought to the forefront because deep learning (DL)-based methods, such as Transformer, have achieved great success in semantic extraction. Although semantic communication has been successfully applied in sentence transmission to reduce semantic errors, the existing architecture is usually fixed in terms of codeword length and inefficient and inflexible for varying sentence lengths. In this study, we exploit hybrid automatic repeat request (HARQ) to reduce the semantic transmission error further. We combine semantic coding (SC) with Reed-Solomon (RS) channel coding and HARQ (called SC-RS-HARQ). SC-RS-HARQ exploits the superiority of SC and the reliability of conventional methods successfully. Although SC-RS-HARQ can be easily applied in existing HARQ systems, we also develop an end-to-end architecture called SCHARQ to pursue enhanced performance. Numerical results demonstrate that SCHARQ significantly reduces the required number of bits for semantic sentence transmission and the sentence error rate. We also attempt to replace error detection from cyclic redundancy check to a similarity detection network called Sim32 to allow the receiver to reserve wrong sentences with similar semantic information and conserve transmission resources. [ABSTRACT FROM AUTHOR]

We study the problem of transmitting a source sample with minimum distortion over an infinite-bandwidth additive white Gaussian noise channel under an energy constraint. To that end, we construct a joint source–channel coding scheme using analog pulse position modulation (PPM) and bound its quadratic distortion. We show that this scheme outperforms existing techniques, since its quadratic distortion attains both the exponential and polynomial decay orders of Burnashev’s outer bound. We supplement our theoretical results with numerical simulations and comparison to existing schemes. [ABSTRACT FROM AUTHOR]

In this paper, we propose a class of high-efficiency deep joint source-channel coding methods that can closely adapt to the source distribution under the nonlinear transform, it can be collected under the name nonlinear transform source-channel coding (NTSCC). In the considered model, the transmitter first learns a nonlinear analysis transform to map the source data into latent space, then transmits the latent representation to the receiver via deep joint source-channel coding. Our model incorporates the nonlinear transform as a strong prior to effectively extract the source semantic features and provide side information for source-channel coding. Unlike existing conventional deep joint source-channel coding methods, the proposed NTSCC essentially learns both the source latent representation and an entropy model as the prior on the latent representation. Accordingly, novel adaptive rate transmission and hyperprior-aided codec refinement mechanisms are developed to upgrade deep joint source-channel coding. The whole system design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under established perceptual quality metrics. Across test image sources with various resolutions, we find that the proposed NTSCC transmission method generally outperforms both the analog transmission using the standard deep joint source-channel coding and the classical separation-based digital transmission. Notably, the proposed NTSCC method can potentially support future semantic communications due to its content-aware ability and perceptual optimization goal. [ABSTRACT FROM AUTHOR]

The error probability of block codes sent under a non-uniform input distribution over the memoryless binary symmetric channel (BSC) and decoded via the maximum a posteriori (MAP) decoding rule is investigated. It is proved that the ratio of the probability of MAP decoder ties to the probability of error grows most linearly in blocklength when no MAP decoding ties occur, thus showing that decoder ties do not affect the code’s error exponent. This result generalizes a similar recent result shown for the case of block codes transmitted over the BSC under a uniform input distribution.

Joint communications and sensing functionalities integrated into the same communication network have become increasingly relevant due to the large bandwidth requirements of next-generation wireless communication systems and the impending spectral shortage. While there exist system-level guidelines and waveform design specifications for such systems, an information-theoretic analysis of the absolute performance capabilities of joint sensing and communication systems that take into account practical limitations such as fading has not been addressed in the literature. Motivated by this, we undertake a network information-theoretic analysis of a typical joint communications and sensing system in this paper. Towards this end, we consider a state-dependent fading Gaussian multiple access channel (GMAC) setup with an additive state. The state process is assumed to be independent and identically distributed (i.i.d.) Gaussian, and non-causally available to all the transmitting nodes. The fading gains on the respective links are assumed to be stationary and ergodic and available only at the receiver. In this setting, with no knowledge of fading gains at the transmitters, we are interested in joint message communication and estimation of the state at the receiver to meet a target distortion in the mean-squared error sense. Our main contribution here is a complete characterization of the distortion-rate trade-off region between the communication rates and the state estimation distortion for a two-sender GMAC. Our results show that the optimal strategy is based on static power allocation and involves uncoded transmissions to amplify the state, along with the superposition of the digital message streams using appropriate Gaussian codebooks and dirty paper coding (DPC). This acts as a design directive for realistic systems using joint sensing and transmission in next-generation wireless standards and points to the relative benefits of uncoded communications and joint source-channel coding in such systems.

We study the following semi–deterministic setting of the joint source–channel coding problem: a deterministic source sequence (a.k.a. individual sequence) is transmitted via a memoryless channel, using delay-limited encoder and decoder, which are both implementable by periodically–varying finite-state machines, and the decoder is granted with access to side information, which is a noisy version of the source sequence. We first derive a lower bound on the achievable expected distortion in terms of the empirical statistics of the source sequence, the number of states of the encoder, the number of states of the decoder, their period, and the overall delay. The bound is shown to be asymptotically achievable by universal block codes in the limit of long blocks. We also derive a lower bound to the best achievable excess–distortion probability and discuss situations where it is achievable. Here, of course, source coding and channel coding cannot be completely separated without loss of optimality. Finally, we outline a few extensions of the model considered, such as: (i) incorporating a common reconstruction constraint, (ii) availability of side information at both ends, and (iii) extension to the Shannon channel with causal state information at the encoder. This work both extends and improves on earlier work of the same flavor (Ziv 1980, Merhav 2014), which focused only on the expected distortion, without side information at either end, and without the above mentioned additional ingredients. [ABSTRACT FROM AUTHOR]

In this work, we consider the transmissions of structured data such as text over a noisy channel and correlated texts over a broadcast channel. As the separate source-channel coding principle no longer holds in such scenarios, we propose a joint source-channel coding scheme which is based on deep learning architecture. In order to enhance the convergence speed, we adopt the bidirectional gated recurrent unit at the encoder. For the decoder, to improve the recovery quality, we propose the following two types of strategies: (1) After a unidirectional neural network based decoder is used, a generative adversarial network is applied to train the whole joint source-channel coding framework and pointwise mutual information is added to the objective function of beam search process; (2) Rather than using a unidirectional neural network-based decoder, we develop a bidirectional neural network based and bidirectional attention mechanism integrated decoder to utilize past and future information. Experiments under different types of channels show that our schemes are superior to the existing deep learning joint source-channel coding method and in the case of low bit budget, long sentence length and small channel signal to noise ratio, our models are significantly superior to those of separate source-channel coding. In addition, we extend the proposed unidirectional and bidirectional decoders to the broadcast channel. Additionally, to improve the performance of unidirectional decoding, we utilize not only the correlation between adjacent words in the same text but also the correlation between words in different languages with the same meaning in the beam search process. [ABSTRACT FROM AUTHOR]

A high-quality data transmission scheme in wireless body area networks is implemented by a joint source-channel coding (JSCC) system with $M$ -ary differential chaos shift keying modulation over an on-body channel. Current DP-LDPC code pairs, which have shown good bit-error ratio performance over additive white Gaussian noise channel and Rayleigh fading channel, cannot perform well over on-body channel. In this paper, the DP-LDPC codes are redesigned for the JSCC system with new methods depending on the on-body channel characteristics. A protomatrix dimension-restrictive searching algorithm for DP-LDPC codes in the JSCC system is proposed according to the on-body channel bandwith and source statistic, which is effective to predesign the dimension of the code pair for a new JSCC system. In this way, the source-channel code rates are appropriately allocated to leave more source redundancy for improving the error floor. Moreover, the joint protomatrix is reconstructed into two kinds of compact structures by simplifying the edge linking relationships and modifying joint protograph extrinsic information transfer algorithm, which decreases the induction of searching entries to further accelerate the code searching speed. The searched code pairs show better bit-error ratio performances compared with existing DP-LDPC codes. The power consumption in the new joint coding system is lower than the separate coding system in WBAN on the physical layer.

Analog-digital hybrid precoding and combining schemes constitute an interesting approach to millimeter-wave (mmWave) multiple-input multiple-output (MIMO) systems due to the low hardware complexity and/or low power required for its deployment. However, the design of the hybrid precoders and combiners of a wideband multiuser (MU) mmWave MIMO system is challenging because the signal processing in the analog domain is constrained to be frequency flat. Furthermore, the number of radio frequency (RF) chains limits the number of individual streams that a common base station (BS) can simultaneously serve. This work jointly addresses the user scheduling, the user precoder design, and the BS hybrid combining design for the uplink of wideband MU mmWave MIMO systems. On the one hand, user precoding and BS hybrid combining are jointly designed to minimize the impact of having frequency-flat RF components. On the other hand, a number of users larger than the number of RF chains are served at the BS by employing a distributed quantizer linear coding (DQLC)-based non-orthogonal multiple access (NOMA) scheme. The use of this encoding strategy also allows exploiting the spatial correlation between the source information. Simulation results show remarkable performance gains of the proposed approaches for wideband mmWave MIMO hardware-constrained systems.

The joint shuffled scheduling decoding (JSSD) algorithm can reduce the decoding complexity of the joint source-channel coding system (JSCC) based on double protograph low-density parity-check (P-LDPC) codes. However, the JSSD algorithm will not work when the linking matrix between check nodes (CNs) of the source P-LDPC and variable nodes (VNs) of the channel P-LDPC is adopted in such a system, and this linking matrix has a significant influence on the system performance. In this paper, a generalized joint shuffled scheduling decoding (GJSSD) algorithm is designed to work for the system, and the JSSD algorithm can be regarded as a special case of this algorithm. Simulations show that the proposed GJSSD algorithm can reduce the decoding complexity with performance improvement when compared with the joint belief-propagation (JBP) algorithm.

In this paper, a method for joint source-channel coding (JSCC) based on concatenated spatially coupled low-density parity-check (SC-LDPC) codes is investigated. A construction consisting of two SC-LDPC codes is proposed: one for source coding and the other for channel coding, with a joint belief propagation-based decoder. Also, a novel windowed decoding (WD) scheme is presented with significantly reduced latency and complexity requirements. The asymptotic behavior for various graph node degrees is analyzed using a protograph-based Extrinsic Information Transfer (EXIT) chart analysis for both LDPC block codes with block decoding and for SC-LDPC codes with the WD scheme, showing robust performance for concatenated SC-LDPC codes. Simulation results show a notable performance improvement compared to existing state-of-the-art JSCC schemes based on LDPC codes with comparable latency and complexity constraints. [ABSTRACT FROM AUTHOR]

Although there are many studies on code optimization of the joint source-channel coding (JSCC) system based on double protograph low-density parity-check codes with the joint belief propagation (JBP) algorithm, but it is still unknown whether the source code and channel code (as a code pair) can perform well when the joint shuffled scheduling decoding (JSSD) algorithm is adopted. In this letter, two decoding threshold analysis algorithms, including joint shuffled protograph extrinsic information transfer (PEXIT) and source shuffled PEXIT algorithm, are proposed to calculate joint/source decoding thresholds for this system with the JSSD algorithm. With the proposed algorithms, it is found that the optimized code pairs for this system with the JBP algorithm may not perform well with the JSSD algorithm, implying that code pair with the JSSD algorithm needs to be redesigned. Then, a two-stage optimized framework is proposed to design the code pair for this system with the JSSD algorithm. Simulations and decoding threshold analysis both show that the proposed code pair for this system with the JSSD algorithm can obtain lower error floor and better waterfall performance than the existing code pairs. [ABSTRACT FROM AUTHOR]

We propose deep learning based communication methods for adaptive-bandwidth transmission of images over wireless channels. We consider the scenario in which images are transmitted progressively in layers over time or frequency, and such layers can be aggregated by receivers in order to increase the quality of their reconstructions. We investigate two scenarios, one in which the layers are sent sequentially, and incrementally contribute to the refinement of a reconstruction, and another in which the layers are independent and can be retrieved in any order. Those scenarios correspond to the well known problems of successive refinement and multiple descriptions, respectively, in the context of joint source-channel coding (JSCC). We propose DeepJSCC- $l$ , an innovative solution that uses convolutional autoencoders, and present three architectures with different complexity trade-offs. To the best of our knowledge, this is the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. Numerical results show that DeepJSCC- $l$ can learn to transmit the source progressively with negligible losses in the end-to-end performance compared with a single transmission. Moreover, DeepJSCC- $l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes, with the additional advantage of graceful degradation with channel SNR. [ABSTRACT FROM AUTHOR]

Hybrid analog/digital schemes for precoding/combining have proved to be a low-complexity and/or low-power strategy to obtain reasonable beamforming gains in multiuser millimeter-wave (mmWave) multiple-input multiple-output (MIMO) systems. Hybrid precoding/combining performs jointly baseband processing and analog processing in the radio frequency (RF) domain. In these systems, the number of RF chains limits the maximum number of streams simultaneously handled by the transceivers. In the uplink of a multiuser mmWave MIMO system, the hardware reduction based on hybrid transceivers is limited by the number of data streams that must be simultaneously served by the centralized node. Most works approach hybrid transceiver design by considering more RF chains than data streams, an unrealistic assumption when the number of nodes is large. On the other hand, statistically independent information is conventionally assumed in multiuser mmWave systems. This assumption does not hold in scenarios like wireless sensor networks (WSNs), where the sources produce correlated information. In this paper, by enabling inter-user correlation exploitation, we propose a grouping approach to handle a high number of individual sources with a limited number of RF chains through distributed quantizer linear coding (DQLC) mappings. The allocation of the users per group and the hybrid design of the combiner at the common central node to serve the grouped users is also analyzed. We also propose a hybrid minimum mean square error (MMSE) combining design in order to exploit the spatial correlation between the sources in a conventional uncoded mmWave uplink. Simulation results show the performance advantages of the proposed approaches in various hardware-constrained system settings.

A two-terminal distributed binary hypothesis testing problem over a noisy channel is studied. The two terminals, called the observer and the decision maker, each has access to n independent and identically distributed samples, denoted by U and V, respectively. The observer communicates to the decision maker over a discrete memoryless channel, and the decision maker performs a binary hypothesis test on the joint probability distribution of (U,V) based on V and the noisy information received from the observer. The trade-off between the exponents of the type I and type II error probabilities is investigated. Two inner bounds are obtained, one using a separation-based scheme that involves type-based compression and unequal error-protection channel coding, and the other using a joint scheme that incorporates type-based hybrid coding. The separation-based scheme is shown to recover the inner bound obtained by Han and Kobayashi for the special case of a rate-limited noiseless channel, and also the one obtained by the authors previously for a corner point of the trade-off. Finally, we show via an example that the joint scheme achieves a strictly tighter bound than the separation-based scheme for some points of the error-exponents trade-off.

In this paper, a joint group shuffled scheduling decoding (JGSSD) algorithm for a joint source-channel coding (JSCC) scheme based on double low-density parity-check (D-LDPC) codes is presented. The proposed algorithm considers the D-LDPC coding structure as a whole and applies shuffled scheduling to each group; the grouping relies on the types or the length of the variable nodes (VNs). By comparison, the conventional shuffled scheduling decoding algorithm can be regarded as a special case of this proposed algorithm. A novel joint extrinsic information transfer (JEXIT) algorithm for the D-LDPC codes system with the JGSSD algorithm is proposed, by which the source and channel decoding are calculated with different grouping strategies to analyze the effects of the grouping strategy. Simulation results and comparisons verify the superiority of the JGSSD algorithm, which can adaptively trade off the decoding performance, complexity and latency.

A key component of the joint source-channel coding (JSCC) scheme based on double low-density parity-check (D-LDPC) codes is the introduction of a linking matrix between the source LDPC code and channel LDPC code, by which the decoding information including the source redundancy and channel state information can be transferred iteratively. However, the linking matrix is a fixed one-to-one mapping, i.e., an identity matrix in a conventional D-LDPC code system, which may not take full advantage of the decoding information. Therefore, this paper introduces a general linking matrix, i.e., a non-identity linking matrix, connecting the check nodes (CNs) of the source LDPC code and the variable nodes (VNs) of the channel LDPC code. Further, the encoding and decoding algorithms of the proposed D-LDPC coding system are generalized. A joint extrinsic information transfer (JEXIT) algorithm is derived for calculating the decoding threshold of the proposed system with a general linking matrix. In addition, several general linking matrices are optimized with the aid of the JEXIT algorithm. Finally, the simulation results demonstrate the superiority of the proposed D-LDPC coding system with general linking matrices.

We propose an adaptive lossy joint source-channel coding (JSCC) scheme for sending correlated sources over two-terminal discrete-memoryless two-way channels (DM-TWCs). The main idea is to couple the independent operations of the terminals via an adaptive coding mechanism, which can mitigate cross-interference resulting from simultaneous channel transmissions and concurrently exploit the sources’ correlation to reduce the end-to-end reconstruction distortions. Our adaptive JSCC scheme not only subsumes existing lossy coding methods for two-way simultaneous communication but also improves their performance. Furthermore, we derive outer bounds for our two-way lossy transmission problem and establish complete JSCC theorems in some special settings. In these special cases, a non-adaptive separate source-channel coding (SSCC) scheme achieves the optimal performance, thus simplifying the design of the source-channel communication system. [ABSTRACT FROM AUTHOR]

We propose a joint source-channel coding (JSCC) scheme based on model-predictive control (MPC) for encoding an analog discrete-time source over a memoryless communication channel. The decoder is assumed to be fixed and given, and the MPC encoder is set to minimize the frequency-weighted mean-squared error (FWMSE) while satisfying a channel input constraint. We derive analytic expressions for the associated finite-horizon optimization MPC problem for an asymptotic (in time) average or sum channel input constraint. For the case of complex additive white Gaussian noise channel and an M - quadrature-amplitude modulation (M -QAM) decoder, these expressions can be written explicitly, which allows for an efficient numerical solution. For the case M =4 and considering a frequency weighting criterion that models the sensitivity of the human hearing, we show by simulations that our proposed scheme yields an FWMSE significantly smaller than that yielded by the corresponding non-JSCC scheme (a 2-bit uniform quantizer in tandem with a 4-QAM channel encoder and decoder). [ABSTRACT FROM AUTHOR]

We propose a novel formulation of the “effectiveness problem” in communications, put forth by Shannon and Weaver in their seminal work “The Mathematical Theory of Communication”, by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment, can also communicate with each other over a noisy communication channel. The noisy communication channel is considered explicitly as part of the dynamics of the environment, and the message each agent sends is part of the action that the agent can take. As a result, the agents learn not only to collaborate with each other but also to communicate “effectively” over a noisy channel. This framework generalizes both the traditional communication problem, where the main goal is to convey a message reliably over a noisy channel, and the “learning to communicate” framework that has received recent attention in the MARL literature, where the underlying communication channels are assumed to be error-free. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately from the underlying MA-POMDP. This is a very powerful framework, which has many real world applications, from autonomous vehicle planning to drone swarm control, and opens up the rich toolbox of deep reinforcement learning for the design of multi-user communication systems. [ABSTRACT FROM AUTHOR]

Diversity “multiple description” (MD) source coding promises graceful degradation in the presence of a priori unknown number of erased packets in the channel. A simple coding scheme for the case of two packets consists of oversampling the source by a factor of two and delta-sigma quantization. This approach was applied successfully to JPEG-based image coding over a lossy packet network, where the interpolation and splitting into two descriptions are done in the discrete cosine transform (DCT) domain. Moreover, unlike the classical source-channel separation approach - which is designed for a predetermined number of erasures (say, $K$ out of $N$), hence its distortion does not improve when the channel behaves better than expected - an MD coding scheme aims to achieve a better reconstruction quality when more or all the $N$ descriptions are received at the decoder side. The extension to a larger number of descriptions, however, suffers from noise amplification whenever the received descriptions form a non-uniform sampling pattern. In this work, we examine inter- and intra-block interpolation methods, and show how noise amplification can be reduced by redesigning the interpolation filter at the encoder. Specifically, for a given total coding rate, we demonstrate that an “irregular” interpolation filter is robust to the pattern of received packets over all ($K$ out of $N$) patterns, with some degradation relative to low-pass (LP) interpolation in the case where all $N$ packets arrived. We provide experimental results comparing LP and irregular interpolation filters, and examine the effect of noise shaping on the trade-off between the central distortion (receiving all packets) and side distortion (receiving $K$ packets). [ABSTRACT FROM AUTHOR]

Asymptotic energy-distortion performance of zero- and low-delay transmission of Gaussian sources over energy-limited Gaussian channels is studied. A lower bound for the leading term in the negative logarithm of the distortion, termed the energy-distortion exponent, is derived through an achievable scheme based on high-resolution quantization coupled with orthogonal signaling. The higher-order term in the negative logarithm of the distortion, termed the energy-distortion dispersion, is optimized while keeping the leading term, the energy-distortion exponent, at its optimal (respectively, the best known) value for the zero-delay (respectively, low-delay) regime. In contrast with the decaying dispersion previously reported in the literature, the proposed coding scheme achieves a constant dispersion. When the scheme is optimized, this constant can be improved with respect to its naïve value, i.e., that achieved by optimizing purely the source coding performance instead of the end-to-end distortion. Lastly, a tradeoff of achievable energy-distortion exponents is derived for broadcast scenarios by extending the point-to-point scheme to include a successive refinement source coder coupled with two rounds of orthogonal signaling. A simple parametric computation algorithm is derived for the tradeoff. [ABSTRACT FROM AUTHOR]

In this paper, we present a learning scheme for Joint Source-Channel Coding (JSCC) over analog independent additive noise channels. We formulate the learning problem by showing that the minimization loss function from rate-distortion theory, is upper bounded by the loss function of the Variational Autoencoder (VAE). We show that when the source dimension is greater than the channel dimension, the encoding of two source samples in the neighborhood of each other need not be near each other. Such discontinuous projection needs to be accounted for by using multiple encoders and selecting an encoder to encode samples on a particular side of the discontinuity. We explore two selection methodologies, one based on an intuitive rule and the other where it is posed as a learning task in a Mixture-of-Experts (MoE) setup. We analyze the gradients of these methods and reason why the latter is better at avoiding local optima. We show the efficacy of the proposed methodology by simulating the performance of the system for JSCC of Gaussian sources over AWGN channels and showing that the learned solutions are close to or better than the ones proposed earlier. The proposed methodology is also naturally capable of generalizing to other source distributions which we showcase by simulating for Laplace sources. The learned systems are also robust to changes in channel conditions. Further, a single system can be trained to generalize over a range of channel conditions provided the channel conditions are known at both the transmitter and the receiver. Finally, we evaluate our proposed methodology on three different image datasets and showcase consistent improvement over existing methods due to the VAE formulation. [ABSTRACT FROM AUTHOR]

We present the second part of our work on communicating correlated sources over multiple access (MAC) and interference channels (IC). Specifically, we undertake a Shannon-theoretic study of the above scenarios and focus on characterizing sufficient conditions for lossless recoverability of the sources at the decoder(s). We enhance the fixed block-length (B-L) coding technique by incorporating the technique of inducing source correlation onto channel inputs, originally discovered by Cover, El Gamal and Salehi. In contrast to the first part, performance analysis of a joint source-channel decoder poses new challenges. We enhance the earlier developed suite of coding and analytical tools to overcome these challenges and derive (simplified) single-letter characterizations for a new set of sufficient conditions for both scenarios. For both the MAC and IC problems, the derived sufficient conditions are (i) subsumed in the current known tightest, and (ii) strictly weaker for identified examples. Lastly, we propose simple ‘plug-in’ approaches that can further weaken the derived sufficient conditions. Our findings enable us to subsume Dueck’s findings (1981) and go even further for the example considered therein. [ABSTRACT FROM AUTHOR]

The widely deployed Internet-of-Things (IoT) devices provide intelligent services with its cognition capability. Since the IoT data are usually transmitted to the server for recognition (e.g., image classification) due to low computational capability and limited power supply, achieving recognition accuracy under limited bandwidth and noisy channel of wireless networks is a crucial but challenging task. In this paper, we propose a deep learning-constructed joint transmission-recognition scheme for the IoT devices to effectively transmit data wirelessly to the server for recognition, jointly considering transmission bandwidth, transmission reliability, complexity, and recognition accuracy. Compared with other schemes that may be deployed on the IoT devices, i.e., a scheme based on JPEG compression and two compressed sensing-based schemes, the proposed deep neural network-based scheme has much higher recognition accuracy under various transmission scenarios at all signal-to-noise ratios (SNRs). In particular, the proposed scheme maintains good performance at the very low SNR. Moreover, the complexity of the proposed scheme is low, making it suitable for IoT applications. Finally, a transfer learning-based training method is proposed to effectively mitigate the computing burden and reduce the overhead of online training.

To reduce the waste of energy in communications, unequal error protection (UEP) is used to provide asymmetric protection for messages with different levels of importance. This paper proposes new efficient strategies of UEP based on bilayer protograph-based low-density parity check (PLDPC) codes in decoding-and-forward (DF) relay systems. In particular, we jointly utilize source coding and channel coding to design UEP strategies and then save transmission energy. According to the different levels of importance of discrete cosine transform (DCT) coefficients of image and variance statistical characteristics of image sub-blocks, bilayer-lengthened PLDPC codes are exploited to protect the transmitted image information with different importance levels at the half-duplex relay system. In the end, the simulation result shows that the proposed UEP schemes achieve excellent performance gains compared to conventional equal error protection (EEP) scheme. Additionally, the complexity analysis of the UEP strategies is given.