Search

Sequential Monte Carlo (SMC), or particle filtering, is widely used in nonlinear state-space systems, but its performance often suffers from poorly approximated proposal and state-transition distributions. This work introduces a differentiable particle filter that leverages the unsupervised variational SMC objective to parameterize the proposal and transition distributions with a neural network, designed to learn from high-dimensional observations. Experimental results demonstrate that our approach outperforms established baselines in tracking the challenging Lorenz attractor from high-dimensional and partial observations. Furthermore, an evidence lower bound based evaluation indicates that our method offers a more accurate representation of the posterior distribution.

A broad range of technologies rely on remote inference, wherein data acquired is conveyed over a communication channel for inference in a remote server. Communication between the participating entities is often carried out over rate-limited channels, necessitating data compression for reducing latency. While deep learning facilitates joint design of the compression mapping along with encoding and inference rules, existing learned compression mechanisms are static, and struggle in adapting their resolution to changes in channel conditions and to dynamic links. To address this, we propose Adaptive Rate Task-Oriented Vector Quantization (ARTOVeQ), a learned compression mechanism that is tailored for remote inference over dynamic links. ARTOVeQ is based on designing nested codebooks along with a learning algorithm employing progressive learning. We show that ARTOVeQ extends to support low-latency inference that is gradually refined via successive refinement principles, and that it enables the simultaneous usage of multiple resolutions when conveying high-dimensional data. Numerical results demonstrate that the proposed scheme yields remote deep inference that operates with multiple rates, supports a broad range of bit budgets, and facilitates rapid inference that gradually improves with more bits exchanged, while approaching the performance of single-rate deep quantization methods., Comment: 13 pages, 12 figures

In this paper, we propose a low-complexity and fast hybrid beamforming design for joint communications and sensing (JCAS) based on deep unfolding. We first derive closed-form expressions for the gradients of the communications sum rate and sensing beampattern error with respect to the analog and digital precoders. Building on this, we develop a deep neural network as an unfolded version of the projected gradient ascent algorithm, which we refer to as UPGANet. This approach efficiently optimizes the communication-sensing performance tradeoff with fast convergence, enabled by the learned step sizes. UPGANet preserves the interpretability and flexibility of the conventional PGA optimizer while enhancing performance through data training. Our simulations show that UPGANet achieves up to a 33.5% higher communications sum rate and 2.5 dB lower beampattern error compared to conventional designs based on successive convex approximation and Riemannian manifold optimization. Additionally, it reduces runtime and computational complexity by up to 65% compared to PGA without unfolding., Comment: arXiv admin note: substantial text overlap with arXiv:2307.04376

The Kalman filter (KF) and its variants are among the most celebrated algorithms in signal processing. These methods are used for state estimation of dynamic systems by relying on mathematical representations in the form of simple state-space (SS) models, which may be crude and inaccurate descriptions of the underlying dynamics. Emerging data-centric artificial intelligence (AI) techniques tackle these tasks using deep neural networks (DNNs), which are model-agnostic. Recent developments illustrate the possibility of fusing DNNs with classic Kalman-type filtering, obtaining systems that learn to track in partially known dynamics. This article provides a tutorial-style overview of design approaches for incorporating AI in aiding KF-type algorithms. We review both generic and dedicated DNN architectures suitable for state estimation, and provide a systematic presentation of techniques for fusing AI tools with KFs and for leveraging partial SS modeling and data, categorizing design approaches into task-oriented and SS model-oriented. The usefulness of each approach in preserving the individual strengths of model-based KFs and data-driven DNNs is investigated in a qualitative and quantitative study, whose code is publicly available, illustrating the gains of hybrid model-based/data-driven designs. We also discuss existing challenges and future research directions that arise from fusing AI and Kalman-type algorithms., Comment: Submitted to IEEE Signal Processing Magazine

Joint communications and sensing (JCAS) is expected to be a crucial technology for future wireless systems. This paper investigates beamforming design for a multi-user multi-target JCAS system to ensure fairness and balance between communications and sensing performance. We jointly optimize the transmit and receive beamformers to maximize the weighted sum of the minimum communications rate and sensing mutual information. The formulated problem is highly challenging due to its non-smooth and non-convex nature. To overcome the challenges, we reformulate the problem into an equivalent but more tractable form. We first solve this problem by alternating optimization (AO) and then propose a machine learning algorithm based on the AO approach. Numerical results show that our algorithm scales effectively with the number of the communications users and provides better performance with shorter run time compared to conventional optimization approaches., Comment: 5 pages, 5 figures

The ability to process signals in digital form depends on analog-to-digital converters (ADCs). Traditionally, ADCs are designed to ensure that the digital representation closely matches the analog signal. However, recent studies have shown that significant power and memory savings can be achieved through task-based acquisition, where the acquisition process is tailored to the downstream processing task. An emerging technology for task-based acquisition involves the use of memristors, which are considered key enablers for neuromorphic computing. Memristors can implement ADCs with tunable mappings, allowing adaptation to specific system tasks or power constraints. In this work, we study task-based acquisition for a generic classification task using memristive ADCs. We consider the unique characteristics of this such neuromorphic ADCs, including their power consumption and noisy read-write behavior, and propose a physically compliant model based on resistive successive approximation register ADCs integrated with memristor components, enabling the adjustment of quantization regions. To optimize performance, we introduce a data-driven algorithm that jointly tunes task-based memristive ADCs alongside both digital and analog processing. Our design addresses the inherent stochasticity of memristors through power-aware distillation, complemented by a specialized learning algorithm that adapts to their unique analog-to-digital mapping. The proposed approach is shown to enhance accuracy by up to 27% and reduce power consumption by up to 66% compared to uniform ADCs. Even under noisy conditions, our method achieves substantial gains, with accuracy improvements of up to 19% and power reductions of up to 57%. These results highlight the effectiveness of our power-aware neuromorphic ADCs in improving system performance across diverse tasks., Comment: Under review in the IEEE

A leading family of algorithms for state estimation in dynamic systems with multiple sub-states is based on particle filters (PFs). PFs often struggle when operating under complex or approximated modelling (necessitating many particles) with low latency requirements (limiting the number of particles), as is typically the case in multi target tracking (MTT). In this work, we introduce a deep neural network (DNN) augmentation for PFs termed learning flock (LF). LF learns to correct a particles-weights set, which we coin flock, based on the relationships between all sub-particles in the set itself, while disregarding the set acquisition procedure. Our proposed LF, which can be readily incorporated into different PFs flow, is designed to facilitate rapid operation by maintaining accuracy with a reduced number of particles. We introduce a dedicated training algorithm, allowing both supervised and unsupervised training, and yielding a module that supports a varying number of sub-states and particles without necessitating re-training. We experimentally show the improvements in performance, robustness, and latency of LF augmentation for radar multi-target tracking, as well its ability to mitigate the effect of a mismatched observation modelling. We also compare and illustrate the advantages of LF over a state-of-the-art DNN-aided PF, and demonstrate that LF enhances both classic PFs as well as DNN-based filters., Comment: Under review for publication in the IEEE

Deep neural networks (DNNs) were shown to facilitate the operation of uplink multiple-input multiple-output (MIMO) receivers, with emerging architectures augmenting modules of classic receiver processing. Current designs consider static DNNs, whose architecture is fixed and weights are pre-trained. This induces a notable challenge, as the resulting MIMO receiver is suitable for a given configuration, i.e., channel distribution and number of users, while in practice these parameters change frequently with network variations and users leaving and joining the network. In this work, we tackle this core challenge of DNN-aided MIMO receivers. We build upon the concept of hypernetworks, augmenting the receiver with a pre-trained deep model whose purpose is to update the weights of the DNN-aided receiver upon instantaneous channel variations. We design our hypernetwork to augment modular deep receivers, leveraging their modularity to have the hypernetwork adapt not only the weights, but also the architecture. Our modular hypernetwork leads to a DNN-aided receiver whose architecture and resulting complexity adapts to the number of users, in addition to channel variations, without retraining. Our numerical studies demonstrate superior error-rate performance of modular hypernetworks in time-varying channels compared to static pre-trained receivers, while providing rapid adaptivity and scalability to network variations.

We study iterative blind symbol detection for block-fading linear inter-symbol interference channels. Based on the factor graph framework, we design a joint channel estimation and detection scheme that combines the expectation maximization (EM) algorithm and the ubiquitous belief propagation (BP) algorithm. Interweaving the iterations of both schemes significantly reduces the EM algorithm's computational burden while retaining its excellent performance. To this end, we apply simple yet effective model-based learning methods to find a suitable parameter update schedule by introducing momentum in both the EM parameter updates as well as in the BP message passing. Numerical simulations verify that the proposed method can learn efficient schedules that generalize well and even outperform coherent BP detection in high signal-to-noise scenarios., Comment: Accepted for presentation at Asilomar Conference on Signals, Systems, and Computers 2024

Multiple-input multiple-output (MIMO) systems play a key role in wireless communication technologies. A widely considered approach to realize scalable MIMO systems involves architectures comprised of multiple separate modules, each with its own beamforming capability. Such models accommodate cell-free massive MIMO and partially connected hybrid MIMO architectures. A core issue with the implementation of modular MIMO arises from the need to rapidly set the beampatterns of the modules, while maintaining their power efficiency. This leads to challenging constrained optimization that should be repeatedly solved on each coherence duration. In this work, we propose a power-oriented optimization algorithm for beamforming in uplink modular hybrid MIMO systems, which learns from data to operate rapidly. We derive our learned optimizer by tackling the rate maximization objective using projected gradient ascent steps with momentum. We then leverage data to tune the hyperparameters of the optimizer, allowing it to operate reliably in a fixed and small number of iterations while completely preserving its interpretable operation. We show how power efficient beamforming can be encouraged by the learned optimizer, via boosting architectures with low-resolution phase shifts and with deactivated analog components. Numerical results show that our learn-to-optimize method notably reduces the number of iterations and computation latency required to reliably tune modular MIMO receivers, and that it allows obtaining desirable balances between power efficient designs and throughput., Comment: Under review for possible publication in the IEEE

Extremely large-scale antenna arrays are poised to play a pivotal role in sixth-generation (6G) networks. Utilizing such arrays often results in a near-field spherical wave transmission environment, enabling the generation of focused beams, which introduces new degrees of freedom for wireless localization. In this paper, we consider a beam-focusing design for localizing multiple sources in the radiating near-field. Our formulation accommodates various expected types of implementations of large antenna arrays, including hybrid analog/digital architectures and dynamic metasurface antennas (DMAs). We consider a direct localization estimation method exploiting curvature-of-arrival of impinging spherical wavefront to obtain user positions. In this regard, we adopt a two-stage approach configuring the array to optimize near-field positioning. In the first step, we focus only on adjusting the array coefficients to minimize the estimation error. We obtain a closed-form approximate solution based on projection and the better one based on the Riemann gradient algorithm. We then extend this approach to simultaneously localize and focus the beams via a sub-optimal iterative approach that does not rely on such knowledge. The simulation results show that near-field localization accuracy based on a hybrid array or DMA can achieve performance close to that of fully digital arrays at a lower cost, and DMAs can attain better performance than hybrid solutions with the same aperture., Comment: 13 pages, 11 figures

Deep learning is envisioned to facilitate the operation of wireless receivers, with emerging architectures integrating deep neural networks (DNNs) with traditional modular receiver processing. While deep receivers were shown to operate reliably in complex settings for which they were trained, the dynamic nature of wireless communications gives rise to the need to repeatedly adapt deep receivers to channel variations. However, frequent re-training is costly and ineffective, while in practice, not every channel variation necessitates adaptation of the entire DNN. In this paper, we study concept drift detection for identifying when does a deep receiver no longer match the channel, enabling asynchronous adaptation, i.e., re-training only when necessary. We identify existing drift detection schemes from the machine learning literature that can be adapted for deep receivers in dynamic channels, and propose a novel soft-output detection mechanism tailored to the communication domain. Moreover, for deep receivers that preserve conventional modular receiver processing, we design modular drift detection mechanisms, that simultaneously identify when and which sub-module to re-train. The provided numerical studies show that even in a rapidly time-varying scenarios, asynchronous adaptation via modular drift detection dramatically reduces the number of trained parameters and re-training times, with little compromise on performance.

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field., Comment: arXiv admin note: text overlap with arXiv:2002.04290

Various communication technologies are expected to utilize mobile ad hoc networks (MANETs). By combining MANETs with non-orthogonal multiple access (NOMA) communications, one can support scalable, spectrally efficient, and flexible network topologies. To achieve these benefits of NOMA MANETs, one should determine the transmission protocol, particularly the superposition code. However, the latter involves lengthy optimization that has to be repeated when the topology changes. In this work, we propose an algorithm for rapidly optimizing superposition codes in multi-hop NOMA MANETs. To achieve reliable tunning with few iterations, we adopt the emerging deep unfolding methodology, leveraging data to boost reliable settings. Our superposition coding optimization algorithm utilizes a small number of projected gradient steps while learning its per-user hyperparameters to maximize the minimal rate over past channels in an unsupervised manner. The learned optimizer is designed for both settings with full channel state information, as well as when the channel coefficients are to be estimated from pilots. We show that the combination of principled optimization and machine learning yields a scalable optimizer, that once trained, can be applied to different topologies. We cope with the non-convex nature of the optimization problem by applying parallel-learned optimization with different starting points as a form of ensemble learning. Our numerical results demonstrate that the proposed method enables the rapid setting of high-rate superposition codes for various channels., Comment: Under review for publication in the IEEE

In this paper, we consider a scenario with one UAV equipped with a ULA, which sends combined information and sensing signals to communicate with multiple GBS and, at the same time, senses potential targets placed within an interested area on the ground. We aim to jointly design the transmit beamforming with the GBS association to optimize communication performance while ensuring high sensing accuracy. We propose a predictive beamforming framework based on a dual DNN solution to solve the formulated nonconvex optimization problem. A first DNN is trained to produce the required beamforming matrix for any point of the UAV flying area in a reduced time compared to state-of-the-art beamforming optimizers. A second DNN is trained to learn the optimal mapping from the input features, power, and EIRP constraints to the GBS association decision. Finally, we provide an extensive simulation analysis to corroborate the proposed approach and show the benefits of EIRP, SINR performance and computational speed.

Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. It typically involves a set of heterogeneous devices locally training neural network (NN) models in parallel with periodic centralized aggregations. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. Conventional approaches discard incomplete intra-model updates done by stragglers, alter the amount of local workload and architecture, or resort to asynchronous settings; which all affect the trained model performance under tight training latency constraints. In this work, we propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion. SALF allows stragglers to synchronously convey partial gradients, having each layer of the global model be updated independently with a different contributing set of users. We provide a theoretical analysis, establishing convergence guarantees for the global model under mild assumptions on the distribution of the participating devices, revealing that SALF converges at the same asymptotic rate as FL with no timing limitations. This insight is matched with empirical observations, demonstrating the performance gains of SALF compared to alternative mechanisms mitigating the device heterogeneity gap in FL.

We investigate the application of the factor graph framework for blind joint channel estimation and symbol detection on time-variant linear inter-symbol interference channels. In particular, we consider the expectation maximization (EM) algorithm for maximum likelihood estimation, which typically suffers from high complexity as it requires the computation of the symbol-wise posterior distributions in every iteration. We address this issue by efficiently approximating the posteriors using the belief propagation (BP) algorithm on a suitable factor graph. By interweaving the iterations of BP and EM, the detection complexity can be further reduced to a single BP iteration per EM step. In addition, we propose a data-driven version of our algorithm that introduces momentum in the BP updates and learns a suitable EM parameter update schedule, thereby significantly improving the performance-complexity tradeoff with a few offline training samples. Our numerical experiments demonstrate the excellent performance of the proposed blind detector and show that it even outperforms coherent BP detection in high signal-to-noise scenarios., Comment: Accepted for publication in the IEEE Transactions on Communications

Future wireless systems are envisioned to utilize the large spectra available at THz bands for wireless communications. Extremely massive multiple-input multiple-output (MIMO) antennas can be costly and power inefficient for wideband THz communications. An alternative antenna technology, which can achieve low cost and power efficient THz signaling, is based on leaky wave antennas (LWAs). In this paper, we explore the usage of the LWAs for wideband downlink multi-user THz communications. We propose a model for LWA-aided communication systems that faithfully captures the antenna operations. We that LWAs yield frequency-dependent beams, where the equivalent wideband channel induces dependence between angle, frequency, and spectral lobe width. We identify the LWAs inherent frequency-selective beamsteering capabilities as motivating multi-band THz communications that deviate from conventional orthogonal frequency-division, and employ non-identical subbands. Then, we propose an alternating optimization algorithm for jointly optimizing the LWA configuration along with the spectral division and power allocation to maximize the achievable sum-rate. Our numerical results show that a single LWA can generate diverse beampatterns, exhibiting performance comparable to costly fully digital MIMO. Interestingly, we demonstrate that allowing transmission with non-identical subbands leverages the characteristics of LWA-based channels compared to uniform division, yielding improved beamsteering that translate to higher rates., Comment: Under review in the IEEE

Distributed optimization is a fundamental framework for collaborative inference and decision making in decentralized multi-agent systems. The operation is modeled as the joint minimization of a shared objective which typically depends on observations gathered locally by each agent. Distributed optimization algorithms, such as the common D-ADMM, tackle this task by iteratively combining local computations and message exchanges. One of the main challenges associated with distributed optimization, and particularly with D-ADMM, is that it requires a large number of communications, i.e., messages exchanged between the agents, to reach consensus. This can make D-ADMM costly in power, latency, and channel resources. In this work we propose unfolded D-ADMM, which follows the emerging deep unfolding methodology to enable D-ADMM to operate reliably with a predefined and small number of messages exchanged by each agent. Unfolded D-ADMM fully preserves the operation of D-ADMM, while leveraging data to tune the hyperparameters of each iteration of the algorithm. These hyperparameters can either be agent-specific, aiming at achieving the best performance within a fixed number of iterations over a given network, or shared among the agents, allowing to learn to distributedly optimize over different networks. For both settings, our unfolded D-ADMM operates with limited communications, while preserving the interpretability and flexibility of the original D-ADMM algorithm. We specialize unfolded D-ADMM for two representative settings: a distributed estimation task, considering a sparse recovery setup, and a distributed learning scenario, where multiple agents collaborate in learning a machine learning model. Our numerical results demonstrate that the proposed approach dramatically reduces the number of communications utilized by D-ADMM, without compromising on its performance.

State estimation of dynamical systems from noisy observations is a fundamental task in many applications. It is commonly addressed using the linear Kalman filter (KF), whose performance can significantly degrade in the presence of outliers in the observations, due to the sensitivity of its convex quadratic objective function. To mitigate such behavior, outlier detection algorithms can be applied. In this work, we propose a parameter-free algorithm which mitigates the harmful effect of outliers while requiring only a short iterative process of the standard update step of the KF. To that end, we model each potential outlier as a normal process with unknown variance and apply online estimation through either expectation maximization or alternating maximization algorithms. Simulations and field experiment evaluations demonstrate competitive performance of our method, showcasing its robustness to outliers in filtering scenarios compared to alternative algorithms.

Combining the classical Kalman filter (KF) with a deep neural network (DNN) enables tracking in partially known state space (SS) models. A major limitation of current DNN-aided designs stems from the need to train them to filter data originating from a specific distribution and underlying SS model. Consequently, changes in the model parameters may require lengthy retraining. While the KF adapts through parameter tuning, the black-box nature of DNNs makes identifying tunable components difficult. Hence, we propose Adaptive KalmanNet (AKNet), a DNN-aided KF that can adapt to changes in the SS model without retraining. Inspired by recent advances in large language model fine-tuning paradigms, AKNet uses a compact hypernetwork to generate context-dependent modulation weights. Numerical evaluation shows that AKNet provides consistent state estimation performance across a continuous range of noise distributions, even when trained using data from limited noise settings., Comment: Accepted by ICASSP 2024

Sparse arrays enable resolving more direction of arrivals (DoAs) than antenna elements using non-uniform arrays. This is typically achieved by reconstructing the covariance of a virtual large uniform linear array (ULA), which is then processed by subspace DoA estimators. However, these method assume that the signals are non-coherent and the array is calibrated; the latter often challenging to achieve in sparse arrays, where one cannot access the virtual array elements. In this work, we propose Sparse-SubspaceNet, which leverages deep learning to enable subspace-based DoA recovery from sparse miscallibrated arrays with coherent sources. Sparse- SubspaceNet utilizes a dedicated deep network to learn from data how to compute a surrogate virtual array covariance that is divisible into distinguishable subspaces. By doing so, we learn to cope with coherent sources and miscalibrated sparse arrays, while preserving the interpretability and the suitability of model-based subspace DoA estimators., Comment: Project is still under work

Achieving high-resolution Direction of Arrival (DoA) recovery typically requires high Signal to Noise Ratio (SNR) and a sufficiently large number of snapshots. This paper presents NUV-DoA algorithm, that augments Bayesian sparse reconstruction with spatial filtering for super-resolution DoA estimation. By modeling each direction on the azimuth's grid with the sparsity-promoting normal with unknown variance (NUV) prior, the non-convex optimization problem is reduced to iteratively reweighted least-squares under Gaussian distribution, where the mean of the snapshots is a sufficient statistic. This approach not only simplifies our solution but also accurately detects the DoAs. We utilize a hierarchical approach for interference cancellation in multi-source scenarios. Empirical evaluations show the superiority of NUV-DoA, especially in low SNRs, compared to alternative DoA estimators., Comment: This paper has 5 pages including reference, 11 figures. This paper has been accepted to ICASSP 2024 - 2024 International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

Recent years have witnessed a growing interest in tracking algorithms that augment Kalman Filters (KFs) with Deep Neural Networks (DNNs). By transforming KFs into trainable deep learning models, one can learn from data to reliably track a latent state in complex and partially known dynamics. However, unlike classic KFs, conventional DNN-based systems do not naturally provide an uncertainty measure, such as error covariance, alongside their estimates, which is crucial in various applications that rely on KF-type tracking. This work bridges this gap by studying error covariance extraction in DNN-aided KFs. We begin by characterizing how uncertainty can be extracted from existing DNN-aided algorithms and distinguishing between approaches by their ability to associate internal features with meaningful KF quantities, such as the Kalman Gain (KG) and prior covariance. We then identify that uncertainty extraction from existing architectures necessitates additional domain knowledge not required for state estimation. Based on this insight, we propose Bayesian KalmanNet, a novel DNN-aided KF that integrates Bayesian deep learning techniques with the recently proposed KalmanNet and transforms the KF into a stochastic machine learning architecture. This architecture employs sampling techniques to predict error covariance reliably without requiring additional domain knowledge, while retaining KalmanNet's ability to accurately track in partially known dynamics. Our numerical study demonstrates that Bayesian KalmanNet provides accurate and reliable tracking in various scenarios representing partially known dynamic systems., Comment: Submitted for publication in the IEEE

Federated learning (FL) is an emerging paradigm that allows a central server to train machine learning models using remote users' data. Despite its growing popularity, FL faces challenges in preserving the privacy of local datasets, its sensitivity to poisoning attacks by malicious users, and its communication overhead. The latter is additionally considerably dominant in large-scale networks. These limitations are often individually mitigated by local differential privacy (LDP) mechanisms, robust aggregation, compression, and user selection techniques, which typically come at the cost of accuracy. In this work, we present compressed private aggregation (CPA), that allows massive deployments to simultaneously communicate at extremely low bit rates while achieving privacy, anonymity, and resilience to malicious users. CPA randomizes a codebook for compressing the data into a few bits using nested lattice quantizers, while ensuring anonymity and robustness, with a subsequent perturbation to hold LDP. The proposed CPA is proven to result in FL convergence in the same asymptotic rate as FL without privacy, compression, and robustness considerations, while satisfying both anonymity and LDP requirements. These analytical properties are empirically confirmed in a numerical study, where we demonstrate the performance gains of CPA compared with separate mechanisms for compression and privacy for training different image classification models, as well as its robustness in mitigating the harmful effects of malicious users., Comment: arXiv admin note: text overlap with arXiv:2208.10888

Joint communications and sensing (JCAS) is envisioned as a key feature in future wireless communications networks. In massive MIMO-JCAS systems, hybrid beamforming (HBF) is typically employed to achieve satisfactory beamforming gains with reasonable hardware cost and power consumption. Due to the coupling of the analog and digital precoders in HBF and the dual objective in JCAS, JCAS-HBF design problems are very challenging and usually require highly complex algorithms. In this paper, we propose a fast HBF design for JCAS based on deep unfolding to optimize a tradeoff between the communications rate and sensing accuracy. We first derive closed-form expressions for the gradients of the communications and sensing objectives with respect to the precoders and demonstrate that the magnitudes of the gradients pertaining to the analog precoder are typically smaller than those associated with the digital precoder. Based on this observation, we propose a modified projected gradient ascent (PGA) method with significantly improved convergence. We then develop a deep unfolded PGA scheme that efficiently optimizes the communications-sensing performance tradeoff with fast convergence thanks to the well-trained hyperparameters. In doing so, we preserve the interpretability and flexibility of the optimizer while leveraging data to improve performance. Finally, our simulations demonstrate the potential of the proposed deep unfolded method, which achieves up to 33.5% higher communications sum rate and 2.5 dB lower beampattern error compared with the conventional design based on successive convex approximation and Riemannian manifold optimization. Furthermore, it attains up to a 65% reduction in run time and computational complexity with respect to the PGA procedure without unfolding., Comment: This paper has been submitted to Journal of Selected Topics in Signal Processing

In conventional joint communications and sensing (JCAS) designs for multi-carrier multiple-input multiple-output (MIMO) systems, the dual-functional waveforms are often optimized for the whole frequency band, resulting in limited communications--sensing performance tradeoff. To overcome the limitation, we propose employing a subset of subcarriers for JCAS, while the communications function is performed over all the subcarriers. This offers more degrees of freedom to enhance the communications performance under a given sensing accuracy. We first formulate the rate maximization under the sensing accuracy constraint to optimize the beamformers and JCAS subcarriers. The problem is solved via Riemannian manifold optimization and closed-form solutions. Numerical results for an 8x4 MIMO system with 64 subcarriers show that compared to the conventional subcarrier sharing scheme, the proposed scheme employing 16 JCAS subcarriers offers 60% improvement in the achievable communications rate at the signal-to-noise ratio of 10 dB. Meanwhile, this scheme generates the sensing beampattern with the same quality as the conventional JCAS design., Comment: This paper was accepted to presented at the 22nd IEEE Statistical Signal Processing (SSP) workshop (Hanoi, Vietnam)

Signal processing traditionally relies on classical statistical modeling techniques. Such model-based methods utilize mathematical formulations that represent the underlying physics, prior information and additional domain knowledge. Simple classical models are useful but sensitive to inaccuracies and may lead to poor performance when real systems display complex or dynamic behavior. More recently, deep learning approaches that use deep neural networks are becoming increasingly popular. Deep learning systems do not rely on mathematical modeling, and learn their mapping from data, which allows them to operate in complex environments. However, they lack the interpretability and reliability of model-based methods, typically require large training sets to obtain good performance, and tend to be computationally complex. Model-based signal processing methods and data-centric deep learning each have their pros and cons. These paradigms can be characterized as edges of a continuous spectrum varying in specificity and parameterization. The methodologies that lie in the middle ground of this spectrum, thus integrating model-based signal processing with deep learning, are referred to as model-based deep learning, and are the focus here. This monograph provides a tutorial style presentation of model-based deep learning methodologies. These are families of algorithms that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches. Such model-based deep learning methods exploit both partial domain knowledge, via mathematical structures designed for specific problems, as well as learning from limited data. We accompany our presentation with running examples, in super-resolution, dynamic systems, and array processing. We show how they are expressed using the provided characterization and specialized in each of the detailed methodologies.

Direction of arrival (DoA) estimation is a fundamental task in array processing. A popular family of DoA estimation algorithms are subspace methods, which operate by dividing the measurements into distinct signal and noise subspaces. Subspace methods, such as Multiple Signal Classification (MUSIC) and Root-MUSIC, rely on several restrictive assumptions, including narrowband non-coherent sources and fully calibrated arrays, and their performance is considerably degraded when these do not hold. In this work we propose SubspaceNet; a data-driven DoA estimator which learns how to divide the observations into distinguishable subspaces. This is achieved by utilizing a dedicated deep neural network to learn the empirical autocorrelation of the input, by training it as part of the Root-MUSIC method, leveraging the inherent differentiability of this specific DoA estimator, while removing the need to provide a ground-truth decomposable autocorrelation matrix. Once trained, the resulting SubspaceNet serves as a universal surrogate covariance estimator that can be applied in combination with any subspace-based DoA estimation method, allowing its successful application in challenging setups. SubspaceNet is shown to enable various DoA estimation algorithms to cope with coherent sources, wideband signals, low SNR, array mismatches, and limited snapshots, while preserving the interpretability and the suitability of classic subspace methods., Comment: Under review for publication in the IEEE

Artificial intelligence (AI) is envisioned to play a key role in future wireless technologies, with deep neural networks (DNNs) enabling digital receivers to learn to operate in challenging communication scenarios. However, wireless receiver design poses unique challenges that fundamentally differ from those encountered in traditional deep learning domains. The main challenges arise from the limited power and computational resources of wireless devices, as well as from the dynamic nature of wireless communications, which causes continual changes to the data distribution. These challenges impair conventional AI based on highly-parameterized DNNs, motivating the development of adaptive, flexible, and light-weight AI for wireless communications, which is the focus of this article. Here, we propose that AI-based design of wireless receivers requires rethinking of the three main pillars of AI: architecture, data, and training algorithms. In terms of architecture, we review how to design compact DNNs via model-based deep learning. Then, we discuss how to acquire training data for deep receivers without compromising spectral efficiency. Finally, we review efficient, reliable, and robust training algorithms via meta-learning and generalized Bayesian learning. Numerical results are presented to demonstrate the complementary effectiveness of each of the surveyed methods. We conclude by presenting opportunities for future research on the development of practical deep receivers

The Kalman filter (KF) is a widely-used algorithm for tracking dynamic systems that are captured by state space (SS) models. The need to fully describe a SS model limits its applicability under complex settings, e.g., when tracking based on visual data, and the processing of high-dimensional signals often induces notable latency. These challenges can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work, we study tracking from high-dimensional measurements under complex settings using a hybrid model-based/data-driven approach. By gradually tackling the challenges in handling the observations model and the task, we develop Latent-KalmanNet, which implements tracking from high-dimensional measurements by leveraging data to jointly learn the KF along with the latent space mapping. Latent-KalmanNet combines a learned encoder with data-driven tracking in the latent space using the recently proposed-KalmanNet, while identifying the ability of each of these trainable modules to assist its counterpart via providing a suitable prior (by KalmanNet) and by learning a latent representation that facilitates data-aided tracking (by the encoder). Our empirical results demonstrate that the proposed Latent-KalmanNet achieves improved accuracy and run-time performance over both model-based and data-driven techniques by learning a surrogate latent representation that most facilitates tracking, while operating with limited complexity and latency., Comment: Under review for publication in the IEEE

Hybrid multiple-input multiple-output (MIMO) is an attractive technology for realizing extreme massive MIMO systems envisioned for future wireless communications in a scalable and power-efficient manner. However, the fact that hybrid MIMO systems implement part of their beamforming in analog and part in digital makes the optimization of their beampattern notably more challenging compared with conventional fully digital MIMO. Consequently, recent years have witnessed a growing interest in using data-aided artificial intelligence (AI) tools for hybrid beamforming design. This article reviews candidate strategies to leverage data to improve real-time hybrid beamforming design. We discuss the architectural constraints and characterize the core challenges associated with hybrid beamforming optimization. We then present how these challenges are treated via conventional optimization, and identify different AI-aided design approaches. These can be roughly divided into purely data-driven deep learning models and different forms of deep unfolding techniques for combining AI with classical optimization.We provide a systematic comparative study between existing approaches including both numerical evaluations and qualitative measures. We conclude by presenting future research opportunities associated with the incorporation of AI in hybrid MIMO systems., Comment: This work has been submitted to the IEEE for possible publication

Hybrid beamforming (HBF) is a key enabler for wideband terahertz (THz) massive multiple-input multiple-output (mMIMO) communications systems. A core challenge with designing HBF systems stems from the fact their application often involves a non-convex, highly complex optimization of large dimensions. In this paper, we propose HBF schemes that leverage data to enable efficient designs for both the fully-connected HBF (FC-HBF) and dynamic sub-connected HBF (SC-HBF) architectures. We develop a deep unfolding framework based on factorizing the optimal fully digital beamformer into analog and digital terms and formulating two corresponding equivalent least squares (LS) problems. Then, the digital beamformer is obtained via a closed-form LS solution, while the analog beamformer is obtained via ManNet, a lightweight sparsely-connected deep neural network based on unfolding projected gradient descent. Incorporating ManNet into the developed deep unfolding framework leads to the ManNet-based FC-HBF scheme. We show that the proposed ManNet can also be applied to SC-HBF designs after determining the connections between the radio frequency chain and antennas. We further develop a simplified version of ManNet, referred to as subManNet, that directly produces the sparse analog precoder for SC-HBF architectures. Both networks are trained with an unsupervised training procedure. Numerical results verify that the proposed ManNet/subManNet-based HBF approaches outperform the conventional model-based and deep unfolded counterparts with very low complexity and a fast run time. For example, in a simulation with 128 transmit antennas, it attains a slightly higher spectral efficiency than the Riemannian manifold scheme, but over 1000 times faster and with a complexity reduction of more than by a factor of six (6)., Comment: This paper has been submitted to IEEE Transaction on Signal Processing

We analytically derive from first physical principles the functional dependence of wireless channels on the RIS configuration for generic (i.e., potentially complex-scattering) RIS-parametrized radio environments. The wireless channel is a linear input-output relation that depends non-linearly on the RIS configuration because of two independent mechanisms: i) proximity-induced mutual coupling between close-by RIS elements; ii) reverberation-induced long-range coupling between all RIS elements. Mathematically, this "structural" non-linearity originates from the inversion of an "interaction" matrix that can be cast as the sum of an infinite Born series [for i)] or Born-like series [for ii)] whose $K$th term physically represents paths involving $K$ bounces between the RIS elements [for i)] or wireless entities [for ii)]. We identify the key physical parameters that determine whether these series can be truncated after the first and second term, respectively, as tacitly done in common cascaded models of RIS-parametrized wireless channels. Numerical results obtained with the physics-compliant PhysFad model and experimental results obtained with a RIS prototype in an anechoic (echo-free) chamber and rich-scattering reverberation chambers corroborate our analysis. Our findings raise doubts about the reliability of existing performance analysis and channel-estimation protocols for cases in which cascaded models poorly describe the physical reality., Comment: 30 pages, 5 figures, submitted to an IEEE Journal

Deep learning is envisioned to play a key role in the design of future wireless receivers. A popular approach to design learning-aided receivers combines deep neural networks (DNNs) with traditional model-based receiver algorithms, realizing hybrid model-based data-driven architectures. Such architectures typically include multiple modules, each carrying out a different functionality dictated by the model-based receiver workflow. Conventionally trained DNN-based modules are known to produce poorly calibrated, typically overconfident, decisions. Consequently, incorrect decisions may propagate through the architecture without any indication of their insufficient accuracy. To address this problem, we present a novel combination of Bayesian deep learning with hybrid model-based data-driven architectures for wireless receiver design. The proposed methodology, referred to as modular Bayesian deep learning, is designed to yield calibrated modules, which in turn improves both accuracy and calibration of the overall receiver. We specialize this approach for two fundamental tasks in multiple-input multiple-output (MIMO) receivers - equalization and decoding. In the presence of scarce data, the ability of modular Bayesian deep learning to produce reliable uncertainty measures is consistently shown to directly translate into improved performance of the overall MIMO receiver chain.

Polar codes have promising error-correction capabilities. Yet, decoding polar codes is often challenging, particularly with large blocks, with recently proposed decoders based on list-decoding or neural-decoding. The former applies multiple decoders or the same decoder multiple times with some redundancy, while the latter family utilizes emerging deep learning schemes to learn to decode from data. In this work we introduce a novel polar decoder that combines the list-decoding with neural-decoding, by forming an ensemble of multiple weighted belief-propagation (WBP) decoders, each trained to decode different data. We employ the cyclic-redundancy check (CRC) code as a proxy for combining the ensemble decoders and selecting the most-likely decoded word after inference, while facilitating real-time decoding. We evaluate our scheme over a wide range of polar codes lengths, empirically showing that gains of around 0.25dB in frame-error rate could be achieved. Moreover, we provide complexity and latency analysis, showing that the number of operations required approaches that of a single BP decoder at high signal-to-noise ratios.

Hybrid precoding plays a key role in realizing massive multiple-input multiple-output (MIMO) transmitters with controllable cost. MIMO precoders are required to frequently adapt based on the variations in the channel conditions. In hybrid MIMO, here precoding is comprised of digital and analog beamforming, such an adaptation involves lengthy optimization and depends on accurate channel state information (CSI). This affects the spectral efficiency when the channel varies rapidly and when operating with noisy CSI. In this work we employ deep learning techniques to learn how to rapidly and robustly optimize hybrid precoders, while being fully interpretable. We leverage data to learn iteration-dependent hyperparameter settings of projected gradient sum-rate optimization with a predefined number of iterations. The algorithm maps channel realizations into hybrid precoding settings while preserving the interpretable flow of the optimizer and improving its convergence speed. To cope with noisy CSI, we learn to optimize the minimal achievable sum-rate among all tolerable errors, proposing a robust hybrid precoding based on the projected conceptual mirror prox minimax optimizer. Numerical results demonstrate that our approach allows using over ten times less iterations compared to that required by conventional optimization with shared hyperparameters, while achieving similar and even improved sum-rate performance., Comment: 30 Pages, 9 figures

Multiple-input multiple-output (MIMO) systems exploit spatial diversity to facilitate multi-user communications with high spectral efficiency by beamforming. As MIMO systems utilize multiple antennas and radio frequency (RF) chains, they are typically costly to implement and consume high power. A common method to reduce the cost of MIMO receivers is utilizing less RF chains than antennas by employing hybrid analog/digital beamforming (HBF). However, the added analog circuitry involves active components whose consumed power may surpass that saved in RF chain reduction. An additional method to realize power-efficient MIMO systems is to use low-resolution analog-to-digital converters (ADCs), which typically compromises signal recovery accuracy. In this work, we propose a power-efficient hybrid MIMO receiver with low-quantization rate ADCs, by jointly optimizing the analog and digital processing in a hardware-oriented manner using task-specific quantization techniques. To mitigate power consumption on the analog front-end, we utilize efficient analog hardware architecture comprised of sparse low-resolution vector modulators, while accounting for their properties in design to maintain recovery accuracy and mitigate interferers in congested environments. To account for common mismatches induced by non-ideal hardware and inaccurate channel state information, we propose a robust mismatch aware design. Supported by numerical simulations and power analysis, our power-efficient MIMO receiver achieves comparable signal recovery performance to power-hungry fully-digital MIMO receivers using high-resolution ADCs. Furthermore, our receiver outperforms the task-agnostic HBF receivers with low-rate ADCs in recovery accuracy at lower power and successfully copes with hardware mismatches., Comment: 13 pages, 14 figures, Submitted to IEEE Transactions on Signal Processing

Integrated sensing and communications (ISAC) are envisioned to be an integral part of future wireless networks, especially when operating at the millimeter-wave (mmWave) and terahertz (THz) frequency bands. However, establishing wireless connections at these high frequencies is quite challenging, mainly due to the penetrating pathloss that prevents reliable communication and sensing. Another emerging technology for next-generation wireless systems is reconfigurable intelligent surfaces (RISs), which are capable of modifying harsh propagation environments. RISs are the focus of growing research and industrial attention, bringing forth the vision of smart and programmable signal propagation environments. In this article, we provide a tutorial-style overview of the applications and benefits of RISs for sensing functionalities in general, and for ISAC systems in particular. We highlight the potential advantages when fusing these two emerging technologies, and identify for the first time that: i) joint sensing and communications designs are most beneficial when the channels referring to these operations are coupled, and that ii) RISs offer means for controlling this beneficial coupling. The usefulness of RIS-aided ISAC goes beyond the individual obvious gains of each of these technologies in both performance and power efficiency. We also discuss the main signal processing challenges and future research directions which arise from the fusion of these two emerging technologies., Comment: 37 pages, 9 figures

Sixth generation (6G) cellular communications are expected to support enhanced wireless localization capabilities. The widespread deployment of large arrays and high-frequency bandwidths give rise to new considerations for localization applications. First, emerging antenna architectures, such as dynamic metasurface antennas (DMAs), are expected to be frequently utilized thanks to the achievable high angular resolution and low hardware complexity. Further, wireless localization is likely to take place in the radiating near-field (Fresnel) region, which provides new degrees of freedom, because of the adoption of arrays with large apertures. While current studies mostly focus on the use of costly fully-digital antenna arrays, in this paper we investigate how DMAs can be applied for near-field localization of a single user. We use a direct positioning estimation method based on curvature-of-arrival of the impinging wavefront to obtain the user location, and characterize the effects of DMA tuning on the estimation accuracy. Next, we propose an algorithm for configuring the DMA to optimize near-field localization, by first tuning the adjustable DMA coefficients to minimize the estimation error using postulated knowledge of the actual user position. Finally, we propose a sub-optimal iterative algorithm that does not rely on such knowledge. Simulation results show that the DMA-based near-field localization accuracy could approach that of fully-digital arrays at lower cost., Comment: 5 pages, 4 figures

Electrocardiography (ECG) signals play a pivotal role in many healthcare applications, especially in at-home monitoring of vital signs. Wearable technologies, which these applications often depend upon, frequently produce low-quality ECG signals. While several methods exist for ECG denoising to enhance signal quality and aid clinical interpretation, they often underperform with ECG data from wearable technology due to limited noise tolerance or inadequate flexibility in capturing ECG dynamics. This paper introduces HKF, a hierarchical and adaptive Kalman filter, which uses a proprietary state space model to effectively capture both intra- and inter-heartbeat dynamics for ECG signal denoising. HKF learns a patient-specific structured prior for the ECG signal's intra-heartbeat dynamics in an online manner, resulting in a filter that adapts to the specific ECG signal characteristics of each patient. In an empirical study, HKF demonstrated superior denoising performance (reduced mean-squared error) while preserving the unique properties of the waveform. In a comparative analysis, HKF outperformed previously proposed methods for ECG denoising, such as the model-based Kalman filter and data-driven autoencoders. This makes it a suitable candidate for applications in extramural healthcare settings., Comment: Submitted to Transactions on Signal Processing

Stochastic control deals with finding an optimal control signal for a dynamical system in a setting with uncertainty, playing a key role in numerous applications. The linear quadratic Gaussian (LQG) is a widely-used setting, where the system dynamics is represented as a linear Gaussian statespace (SS) model, and the objective function is quadratic. For this setting, the optimal controller is obtained in closed form by the separation principle. However, in practice, the underlying system dynamics often cannot be faithfully captured by a fully known linear Gaussian SS model, limiting its performance. Here, we present LQGNet, a stochastic controller that leverages data to operate under partially known dynamics. LQGNet augments the state tracking module of separation-based control with a dedicated trainable algorithm. The resulting system preserves the operation of classic LQG control while learning to cope with partially known SS models without having to fully identify the dynamics. We empirically show that LQGNet outperforms classic stochastic control by overcoming mismatched SS models., Comment: Submitted to ICASSP23

Dynamic metasurface antennas (DMAs) provide a new paradigm to realize large-scale antenna arrays for future wireless systems. In this paper, we study the downlink millimeter wave (mmWave) DMA systems with limited number of radio frequency (RF) chains. By using a specific DMA structure, an equivalent mmWave channel model is first explicitly characterized. Based on that, we propose an effective joint microstrip selection and beamforming scheme to accommodate for the limited number of RF chains. A low-complexity digital beamforming solution with channel gain-based microstrip selection is developed, while the analog beamformer is obtained via a coordinate ascent method. The proposed scheme is numerically shown to approach the performance of DMAs without RF chain reduction, verifying the effectiveness of the proposed schemes., Comment: 5page,4figure,conference

Pairs trading is a family of trading techniques that determine their policies based on monitoring the relationships between pairs of assets. A common pairs trading approach relies on describing the pair-wise relationship as a linear Space State (SS) model with Gaussian noise. This representation facilitates extracting financial indicators with low complexity and latency using a Kalman Filter (KF), that are then processed using classic policies such as Bollinger Bands (BB). However, such SS models are inherently approximated and mismatched, often degrading the revenue. In this work, we propose KalmenNet-aided Bollinger bands Pairs Trading (KBPT), a deep learning aided policy that augments the operation of KF-aided BB trading. KBPT is designed by formulating an extended SS model for pairs trading that approximates their relationship as holding partial co-integration. This SS model is utilized by a trading policy that augments KF-BB trading with a dedicated neural network based on the KalmanNet architecture. The resulting KBPT is trained in a two-stage manner which first tunes the tracking algorithm in an unsupervised manner independently of the trading task, followed by its adaptation to track the financial indicators to maximize revenue while approximating BB with a differentiable mapping. KBPT thus leverages data to overcome the approximated nature of the SS model, converting the KF-BB policy into a trainable model. We empirically demonstrate that our proposed KBPT systematically yields improved revenue compared with model-based and data-driven benchmarks over various different assets., Comment: Submitted to Transactions on Signal Processing

Simultaneous localization and mapping (SLAM) is a method that constructs a map of an unknown environment and localizes the position of a moving agent on the map simultaneously. Extended Kalman filter (EKF) has been widely adopted as a low complexity solution for online SLAM, which relies on a motion and measurement model of the moving agent. In practice, however, acquiring precise information about these models is very challenging, and the model mismatch effect causes severe performance loss in SLAM. In this paper, inspired by the recently proposed KalmanNet, we present a robust EKF algorithm using the power of deep learning for online SLAM, referred to as Split-KalmanNet. The key idea of Split-KalmanNet is to compute the Kalman gain using the Jacobian matrix of a measurement function and two recurrent neural networks (RNNs). The two RNNs independently learn the covariance matrices for a prior state estimate and the innovation from data. The proposed split structure in the computation of the Kalman gain allows to compensate for state and measurement model mismatch effects independently. Numerical simulation results verify that Split-KalmanNet outperforms the traditional EKF and the state-of-the-art KalmanNet algorithm in various model mismatch scenarios., Comment: 6 pages, 6 figures

The Kalman filter (KF) is a widely-used algorithm for tracking the latent state of a dynamical system from noisy observations. For systems that are well-described by linear Gaussian state space models, the KF minimizes the mean-squared error (MSE). However, in practice, observations are corrupted by outliers, severely impairing the KFs performance. In this work, an outlier-insensitive KF is proposed, where robustness is achieved by modeling each potential outlier as a normally distributed random variable with unknown variance (NUV). The NUVs variances are estimated online, using both expectation-maximization (EM) and alternating maximization (AM). The former was previously proposed for the task of smoothing with outliers and was adapted here to filtering, while both EM and AM obtained the same performance and outperformed the other algorithms, the AM approach is less complex and thus requires 40 percentage less run-time. Our empirical study demonstrates that the MSE of our proposed outlier-insensitive KF outperforms previously proposed algorithms, and that for data clean of outliers, it reverts to the classic KF, i.e., MSE optimality is preserved

Deep neural networks (DNNs) allow digital receivers to learn to operate in complex environments. To do so, DNNs should preferably be trained using large labeled data sets with a similar statistical relationship as the one under which they are to infer. For DNN-aided receivers, obtaining labeled data conventionally involves pilot signalling at the cost of reduced spectral efficiency, typically resulting in access to limited data sets. In this paper, we study how one can enrich a small set of labeled pilots data into a larger data set for training deep receivers. Motivated by the widespread use of data augmentation techniques for enriching visual and text data, we propose dedicated augmentation schemes that exploits the characteristics of digital communication data. We identify the key considerations in data augmentations for deep receivers as the need for domain orientation, class (constellation) diversity, and low complexity. Following these guidelines, we devise three complementing augmentations that exploit the geometric properties of digital constellations. Our combined augmentation approach builds on the merits of these different augmentations to synthesize reliable data from a momentary channel distribution, to be used for training deep receivers. Furthermore, we exploit previous channel realizations to increase the reliability of the augmented samples., Comment: The source code is given in https://github.com/tomerraviv95/data-augmentations-for-receivers, and a YouTube tutorial in https://www.youtube.com/watch?v=N5QfLlH-Lqw

Federated learning (FL) is an emerging paradigm for training machine learning models using possibly private data available at edge devices. The distributed operation of FL gives rise to challenges that are not encountered in centralized machine learning, including the need to preserve the privacy of the local datasets, and the communication load due to the repeated exchange of updated models. These challenges are often tackled individually via techniques that induce some distortion on the updated models, e.g., local differential privacy (LDP) mechanisms and lossy compression. In this work we propose a method coined joint privacy enhancement and quantization (JoPEQ), which jointly implements lossy compression and privacy enhancement in FL settings. In particular, JoPEQ utilizes vector quantization based on random lattice, a universal compression technique whose byproduct distortion is statistically equivalent to additive noise. This distortion is leveraged to enhance privacy by augmenting the model updates with dedicated multivariate privacy preserving noise. We show that JoPEQ simultaneously quantizes data according to a required bit-rate while holding a desired privacy level, without notably affecting the utility of the learned model. This is shown via analytical LDP guarantees, distortion and convergence bounds derivation, and numerical studies. Finally, we empirically assert that JoPEQ demolishes common attacks known to exploit privacy leakage.

Non-orthogonal communications are expected to play a key role in future wireless systems. In downlink transmissions, the data symbols are broadcast from a base station to different users, which are superimposed with different power to facilitate high-integrity detection using successive interference cancellation (SIC). However, SIC requires accurate knowledge of both the channel model and channel state information (CSI), which may be difficult to acquire. We propose a deep learningaided SIC detector termed SICNet, which replaces the interference cancellation blocks of SIC by deep neural networks (DNNs). Explicitly, SICNet jointly trains its internal DNN-aided blocks for inferring the soft information representing the interfering symbols in a data-driven fashion, rather than using hard-decision decoders as in classical SIC. As a result, SICNet reliably detects the superimposed symbols in the downlink of non-orthogonal systems without requiring any prior knowledge of the channel model, while being less sensitive to CSI uncertainty than its model-based counterpart. SICNet is also robust to changes in the number of users and to their power allocation. Furthermore, SICNet learns to produce accurate soft outputs, which facilitates improved soft-input error correction decoding compared to model-based SIC. Finally, we propose an online training method for SICNet under block fading, which exploits the channel decoding for accurately recovering online data labels for retraining, hence, allowing it to smoothly track the fading envelope without requiring dedicated pilots. Our numerical results show that SICNet approaches the performance of classical SIC under perfect CSI, while outperforming it under realistic CSI uncertainty.

Artificial intelligence (AI) technologies, and particularly deep learning systems, are traditionally the domain of large-scale cloud servers, which have access to high computational and energy resources. Nonetheless, in Internet-of-Things (IoT) networks, the interface with the real-world is carried out using edge devices that are limited in hardware and can communicate. The conventional approach to provide AI processing to data collected by edge devices involves sending samples to the cloud, at the cost of latency, communication, connectivity, and privacy concerns. Consequently, recent years have witnessed a growing interest in enabling AI-aided inference on edge devices by leveraging their communication capabilities to establish collaborative inference. This article reviews candidate strategies for facilitating the transition of AI to IoT devices via collaboration. We identify the need to operate in different mobility and connectivity constraints as a motivating factor to consider multiple schemes, which can be roughly divided into methods where inference is done remotely, i.e., on the cloud, and those that infer on the edge. We identify the key characteristics of each strategy in terms of inference accuracy, communication latency, privacy, and connectivity requirements, providing a systematic comparison between existing approaches. We conclude by presenting future research challenges and opportunities arising from the concept of collaborative inference., Comment: This work has been submitted to the IEEE for possible publication