Search

Recent years have seen renewed attention to arithmetic coding (AC). This is thanks to the use of AC for distribution matching (DM) to control the channel input distribution in probabilistic amplitude shaping. There are two main problems inherent to AC: (1) its required arithmetic precision grows linearly with the input length, and (2) high-precision multiplications and divisions are required. Here, we introduce a multiplication-free AC-based DM technique via three lookup tables (LUTs) which solves both problems above. These LUTs are used to approximate the high-precision multiplications and divisions by additions and subtractions. The required precision of our approach is shown to grow logarithmically with the input length. We prove that this approximate technique maintains the invertibility of DM. At an input length of 1024 symbols, the proposed technique achieves negligible rate loss ($, 6 pages, 4 figures, presented at the ISIT 2022

The article reviews significant advances in networked signal and information processing, which have enabled in the last 25 years extending decision making and inference, optimization, control, and learning to the increasingly ubiquitous environments of distributed agents. As these interacting agents cooperate, new collective behaviors emerge from local decisions and actions. Moreover, and significantly, theory and applications show that networked agents, through cooperation and sharing, are able to match the performance of cloud or federated solutions, while offering the potential for improved privacy, increasing resilience, and saving resources.

A new method for multimodal sensor fusion is introduced. The technique relies on a two-stage process. In the first stage, a multimodal generative model is constructed from unlabelled training data. In the second stage, the generative model serves as a reconstruction prior and the search manifold for the sensor fusion tasks. The method also handles cases where observations are accessed only via subsampling i.e. compressed sensing. We demonstrate the effectiveness and excellent performance on a range of multimodal fusion experiments such as multisensory classification, denoising, and recovery from subsampled observations., Comment: Under review for Nature Scientific Reports

The pervasiveness of Wi-Fi signals provides significant opportunities for human sensing and activity recognition in fields such as healthcare. The sensors most commonly used for passive Wi-Fi sensing are based on passive Wi-Fi radar (PWR) and channel state information (CSI) data, however current systems do not effectively exploit the information acquired through multiple sensors to recognise the different activities. In this paper, we explore new properties of the Transformer architecture for multimodal sensor fusion. We study different signal processing techniques to extract multiple image-based features from PWR and CSI data such as spectrograms, scalograms and Markov transition field (MTF). We first propose the Fusion Transformer, an attention-based model for multimodal and multi-sensor fusion. Experimental results show that our Fusion Transformer approach can achieve competitive results compared to a ResNet architecture but with much fewer resources. To further improve our model, we propose a simple and effective framework for multimodal and multi-sensor self-supervised learning (SSL). The self-supervised Fusion Transformer outperforms the baselines, achieving a F1-score of 95.9%. Finally, we show how this approach significantly outperforms the others when trained with as little as 1% (2 minutes) of labelled training data to 20% (40 minutes) of labelled training data., Comment: 9 pages, 7 figures, submitted to IET Wireless Sensor Systems

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and often even discrete optimization task. While the potential of gradient-based learning via the unrolling of iterative recovery algorithms has been demonstrated, it has remained unclear how to leverage this technique when the set of admissible measurement operators is structured and discrete. We tackle this problem by combining unrolled optimization with Gumbel reparametrizations, which enable the computation of low-variance gradient estimates of categorical random variables. Our approach is formalized by GLODISMO (Gradient-based Learning of DIscrete Structured Measurement Operators). This novel method is easy-to-implement, computationally efficient, and extendable due to its compatibility with automatic differentiation. We empirically demonstrate the performance and flexibility of GLODISMO in several prototypical signal recovery applications, verifying that the learned measurement matrices outperform conventional designs based on randomization as well as discrete optimization baselines., Comment: The current version of the manuscript has been accepted to IEEE Journal on Selected Areas in Information Theory

Environmental scene reconstruction is of great interest for autonomous robotic applications, since an accurate representation of the environment is necessary to ensure safe interaction with robots. Equally important, it is also vital to ensure reliable communication between the robot and its controller. Large Intelligent Surface (LIS) is a technology that has been extensively studied due to its communication capabilities. Moreover, due to the number of antenna elements, these surfaces arise as a powerful solution to radio sensing. This paper presents a novel method to translate radio environmental maps obtained at the LIS to floor plans of the indoor environment built of scatterers spread along its area. The usage of a Least Squares (LS) based method, U-Net (UN) and conditional Generative Adversarial Networks (cGANs) were leveraged to perform this task. We show that the floor plan can be correctly reconstructed using both local and global measurements.

We show that a 0.9 dB SNR improvement can be obtained via short-blocklength enumerative sphere shaping for single-span transmission at 56 GBd. This gain vanishes for higher symbol rates and a larger number of spans., Comment: 3 pages, 3 figures, presented at the OECC/PSC 2022

We propose a variational Bayesian inference procedure for online nonlinear system identification. For each output observation, a set of parameter posterior distributions is updated, which is then used to form a posterior predictive distribution for future outputs. We focus on the class of polynomial NARMAX models, which we cast into probabilistic form and represent in terms of a Forney-style factor graph. Inference in this graph is efficiently performed by a variational message passing algorithm. We show empirically that our variational Bayesian estimator outperforms an online recursive least-squares estimator, most notably in small sample size settings and low noise regimes, and performs on par with an iterative least-squares estimator trained offline., 6 pages, 4 figures. Accepted to the American Control Conference 2022

Sparse modeling for signal processing and machine learning has been at the focus of scientific research for over two decades. Among others, supervised sparsity-aware learning comprises two major paths paved by: a) discriminative methods and b) generative methods. The latter, more widely known as Bayesian methods, enable uncertainty evaluation w.r.t. the performed predictions. Furthermore, they can better exploit related prior information and naturally introduce robustness into the model, due to their unique capacity to marginalize out uncertainties related to the parameter estimates. Moreover, hyper-parameters associated with the adopted priors can be learnt via the training data. To implement sparsity-aware learning, the crucial point lies in the choice of the function regularizer for discriminative methods and the choice of the prior distribution for Bayesian learning. Over the last decade or so, due to the intense research on deep learning, emphasis has been put on discriminative techniques. However, a come back of Bayesian methods is taking place that sheds new light on the design of deep neural networks, which also establish firm links with Bayesian models and inspire new paths for unsupervised learning, such as Bayesian tensor decomposition. The goal of this article is two-fold. First, to review, in a unified way, some recent advances in incorporating sparsity-promoting priors into three highly popular data modeling tools, namely deep neural networks, Gaussian processes, and tensor decomposition. Second, to review their associated inference techniques from different aspects, including: evidence maximization via optimization and variational inference methods. Challenges such as small data dilemma, automatic model structure search, and natural prediction uncertainty evaluation are also discussed. Typical signal processing and machine learning tasks are demonstrated., 64 pages, 16 figures, 6 tables, 98 references, submitted to IEEE Signal Processing Magazine

This paper is motivated by a simple question: Can we design and build battery-free devices capable of machine learning and inference in underwater environments? An affirmative answer to this question would have significant implications for a new generation of underwater sensing and monitoring applications for environmental monitoring, scientific exploration, and climate/weather prediction. To answer this question, we explore the feasibility of bridging advances from the past decade in two fields: battery-free networking and low-power machine learning. Our exploration demonstrates that it is indeed possible to enable battery-free inference in underwater environments. We designed a device that can harvest energy from underwater sound, power up an ultra-low-power microcontroller and on-board sensor, perform local inference on sensed measurements using a lightweight Deep Neural Network, and communicate the inference result via backscatter to a receiver. We tested our prototype in an emulated marine bioacoustics application, demonstrating the potential to recognize underwater animal sounds without batteries. Through this exploration, we highlight the challenges and opportunities for making underwater battery-free inference and machine learning ubiquitous., Comment: 6 pages, HotMobile '22, March 9-10, 2022, Tempe, AZ, USA

Significance The ability to detect sparse signals from noisy, high-dimensional data is a top priority in modern science and engineering. For optimal results, current approaches need to tune parameters that depend on the level of noise, which is often difficult to estimate. We develop a parameter-free, computationally efficient, ℓ1-norm minimization approach that has a zero false discovery rate (no false positives) with high probability for any level of noise while it detects the exact location of sparse signals when the noise is not too large., The ability to detect sparse signals from noisy, high-dimensional data is a top priority in modern science and engineering. It is well known that a sparse solution of the linear system Aρ=b0 can be found efficiently with an ℓ1-norm minimization approach if the data are noiseless. However, detection of the signal from data corrupted by noise is still a challenging problem as the solution depends, in general, on a regularization parameter with optimal value that is not easy to choose. We propose an efficient approach that does not require any parameter estimation. We introduce a no-phantom weight τ and the Noise Collector matrix C and solve an augmented system Aρ+Cη=b0+e, where e is the noise. We show that the ℓ1-norm minimal solution of this system has zero false discovery rate for any level of noise, with probability that tends to one as the dimension of b0 increases to infinity. We obtain exact support recovery if the noise is not too large and develop a fast Noise Collector algorithm, which makes the computational cost of solving the augmented system comparable with that of the original one. We demonstrate the effectiveness of the method in applications to passive array imaging.

A central problem in analog wireless sensor networks is to design the gain or phase-shifts of the sensor nodes (i.e. the relaying configuration) in order to achieve an accurate estimation of some parameter of interest at a fusion center, or more generally, at each node by employing a distributed parameter estimation scheme. In this paper, by using an over-parametrization of the original design problem, we devise a cyclic optimization approach that can handle tuning both gains and phase-shifts of the sensor nodes, even in intricate scenarios involving sensor selection or discrete phase-shifts. Each iteration of the proposed design framework consists of a combination of the Gram-Schmidt process and power method-like iterations, and as a result, enjoys a low computational cost. Along with formulating the design problem for a fusion center, we further present a consensus-based framework for decentralized estimation of deterministic parameters in a distributed network, which results in a similar sensor gain design problem. The numerical results confirm the computational advantage of the suggested approach in comparison with the state-of-the-art methods---an advantage that becomes more pronounced when the sensor network grows large., Accepted for publication in IEEE Transactions on Signal and Information Processing over Networks

Network slicing provides introduces customized and agile network deployment for managing different service types for various verticals under the same infrastructure. To cater to the dynamic service requirements of these verticals and meet the required quality-of-service (QoS) mentioned in the service-level agreement (SLA), network slices need to be isolated through dedicated elements and resources. Additionally, allocated resources to these slices need to be continuously monitored and intelligently managed. This enables immediate detection and correction of any SLA violation to support automated service assurance in a closed-loop fashion. By reducing human intervention, intelligent and closed-loop resource management reduces the cost of offering flexible services. Resource management in a network shared among verticals (potentially administered by different providers), would be further facilitated through open and standardized interfaces. Open radio access network (O-RAN) is perhaps the most promising RAN architecture that inherits all the aforementioned features, namely intelligence, open and standard interfaces, and closed control loop. Inspired by this, in this article we provide a closed-loop and intelligent resource provisioning scheme for O-RAN slicing to prevent SLA violations. In order to maintain realism, a real-world dataset of a large operator is used to train a learning solution for optimizing resource utilization in the proposed closed-loop service automation process. Moreover, the deployment architecture and the corresponding flow that are cognizant of the O-RAN requirements are also discussed., 7 pages, 3 figures, 1 table

Aerial Base Stations (ABSs) have gained significant importance in the next generation of wireless networks for accommodating mobile ground users and flash crowds with high convenience and quality. However, to achieve an efficient ABS network, many factors pertaining to ABS flight, governing laws and information transmissions must be studied. In this article, multi-drone communications are studied in three major aspects, survivability, coverage, and mobility laws, which optimize the multi-tier ABS network to avoid issues related to inter-cell interference, deficient energy, frequent handovers, and lifetime. The article includes simulation results of hierarchical ABS allocations for handling a set of users over a defined geographical area. Several open issues and challenges are presented to provide deep insights into the ABS network management and its utility framework., Comment: 7 pages, 6 figures

The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors. While the empirical imaging performance and the theoretical convergence properties of these algorithms have been widely investigated, their recovery properties have not previously been theoretically analyzed. We address this gap by showing how to establish theoretical recovery guarantees for PnP/RED by assuming that the solution of these methods lies near the fixed-points of a deep neural network. We also present numerical results comparing the recovery performance of PnP/RED in compressive sensing against that of recent compressive sensing algorithms based on generative models. Our numerical results suggest that PnP with a pre-trained artifact removal network provides significantly better results compared to the existing state-of-the-art methods., 27 pages, 13 figures

Decentralized algorithms for stochastic optimization and learning rely on the diffusion of information as a result of repeated local exchanges of intermediate estimates. Such structures are particularly appealing in situations where agents may be hesitant to share raw data due to privacy concerns. Nevertheless, in the absence of additional privacy-preserving mechanisms, the exchange of local estimates, which are generated based on private data can allow for the inference of the data itself. The most common mechanism for guaranteeing privacy is the addition of perturbations to local estimates before broadcasting. These perturbations are generally chosen independently at every agent, resulting in a significant performance loss. We propose an alternative scheme, which constructs perturbations according to a particular nullspace condition, allowing them to be invisible (to first order in the step-size) to the network centroid, while preserving privacy guarantees. The analysis allows for general nonconvex loss functions, and is hence applicable to a large number of machine learning and signal processing problems, including deep learning.

The electroencephalogram (EEG) provides a non-invasive, minimally restrictive, and relatively low cost measure of mesoscale brain dynamics with high temporal resolution. Although signals recorded in parallel by multiple, near-adjacent EEG scalp electrode channels are highly-correlated and combine signals from many different sources, biological and non-biological, independent component analysis (ICA) has been shown to isolate the various source generator processes underlying those recordings. Independent components (IC) found by ICA decomposition can be manually inspected, selected, and interpreted, but doing so requires both time and practice as ICs have no particular order or intrinsic interpretations and therefore require further study of their properties. Alternatively, sufficiently-accurate automated IC classifiers can be used to classify ICs into broad source categories, speeding the analysis of EEG studies with many subjects and enabling the use of ICA decomposition in near-real-time applications. While many such classifiers have been proposed recently, this work presents the ICLabel project comprised of (1) an IC dataset containing spatiotemporal measures for over 200,000 ICs from more than 6,000 EEG recordings, (2) a website for collecting crowdsourced IC labels and educating EEG researchers and practitioners about IC interpretation, and (3) the automated ICLabel classifier. The classifier improves upon existing methods in two ways: by improving the accuracy of the computed label estimates and by enhancing its computational efficiency. The ICLabel classifier outperforms or performs comparably to the previous best publicly available method for all measured IC categories while computing those labels ten times faster than that classifier as shown in a rigorous comparison against all other publicly available EEG IC classifiers., Comment: Intended for NeuroImage. Updated from version one with minor editorial and figure changes

There is often latent network structure in spatial and temporal data and the tools of network analysis can yield fascinating insights into such data. In this paper, we develop a nonparametric method for network reconstruction from spatiotemporal data sets using multivariate Hawkes processes. In contrast to prior work on network reconstruction with point-process models, which has often focused on exclusively temporal information, our approach uses both temporal and spatial information and does not assume a specific parametric form of network dynamics. This leads to an effective way of recovering an underlying network. We illustrate our approach using both synthetic networks and networks constructed from real-world data sets (a location-based social media network, a narrative of crime events, and violent gang crimes). Our results demonstrate that, in comparison to using only temporal data, our spatiotemporal approach yields improved network reconstruction, providing a basis for meaningful subsequent analysis --- such as community structure and motif analysis --- of the reconstructed networks.

The purpose of this article is to develop and study a decentralized strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution, while the regularizers are assumed deterministic, but are not required to be differentiable or even continuous. The individual, regularized, cost functions are distributed across a strongly connected network of agents, and the Pareto optimal solution is sought by appealing to a multiagent diffusion strategy. To this end, the regularizers are smoothed by means of infimal convolution, and it is shown that the Pareto solution of the approximate smooth problem can be made arbitrarily close to the solution of the original nonsmooth problem. Performance bounds are established under conditions that are weaker than assumed before in the literature and, hence, applicable to a broader class of adaptation and learning problems.

Network utilisation efficiency can, at least in principle, often be improved by dynamically re-configuring routing policies to better distribute on-going large data transfers. Unfortunately, the information necessary to decide on an appropriate reconfiguration - details of on-going and upcoming data transfers such as their source and destination and, most importantly, their volume and duration - is usually lacking. Fortunately, the increased use of scheduled transfer services, such as FTS, makes it possible to collect the necessary information. However, the mere detection and characterisation of larger transfers is not sufficient to predict with confidence the likelihood a network link will become overloaded. In this paper we present the use of LSTM-based models (CNN-LSTM and Conv-LSTM) to effectively estimate future network traffic and so provide a solid basis for formulating a sensible network configuration plan., Comment: vCHEP2021 conference proceedings

Drifters deployed in close proximity collectively provide a unique observational data set with which to separate mesoscale and submesoscale flows. In this paper we provide a principled approach for doing so by fitting observed velocities to a local Taylor expansion of the velocity flow field. We demonstrate how to estimate mesoscale and submesoscale quantities that evolve slowly over time, as well as their associated statistical uncertainty. We show that in practice the mesoscale component of our model can explain much first and second-moment variability in drifter velocities, especially at low frequencies. This results in much lower and more meaningful measures of submesoscale diffusivity, which would otherwise be contaminated by unresolved mesoscale flow. We quantify these effects theoretically via computing Lagrangian frequency spectra, and demonstrate the usefulness of our methodology through simulations as well as with real observations from the LatMix deployment of drifters. The outcome of this method is a full Lagrangian decomposition of each drifter trajectory into three components that represent the background, mesoscale, and submesoscale flow., Accepted in Fluids

Performance trade-offs between linear shaping and nonlinear tolerance of the recently introduced 4D orthant-symmetric 128-ary modulation format are investigated. Numerical simulations show 9.3% reach increase with respect to the 7b4D-2A8PSK format and probabilistically-shaped 16QAM with short blocklength., Comment: 4 pages, 5 figures, accepted by ECOC 2020 SC6

Wide area networking infrastructures (WANs), particularly science and research WANs, are the backbone for moving large volumes of scientific data between experimental facilities and data centers. With demands growing at exponential rates, these networks are struggling to cope with large data volumes, real-time responses, and overall network performance. Network operators are increasingly looking for innovative ways to manage the limited underlying network resources. Forecasting network traffic is a critical capability for proactive resource management, congestion mitigation, and dedicated transfer provisioning. To this end, we propose a nonautoregressive graph-based neural network for multistep network traffic forecasting. Specifically, we develop a dynamic variant of diffusion convolutional recurrent neural networks to forecast traffic in research WANs. We evaluate the efficacy of our approach on real traffic from ESnet, the U.S. Department of Energy's dedicated science network. Our results show that compared to classical forecasting methods, our approach explicitly learns the dynamic nature of spatiotemporal traffic patterns, showing significant improvements in forecasting accuracy. Our technique can surpass existing statistical and deep learning approaches by achieving approximately 20% mean absolute percentage error for multiple hours of forecasts despite dynamic network traffic settings., Comment: 10 Pages, 11 Figures

Electronic health records (EHRs) contain structured and unstructured data of significant clinical and research value. Various machine learning approaches have been developed to employ information in EHRs for risk prediction. The majority of these attempts, however, focus on structured EHR fields and lose the vast amount of information in the unstructured texts. To exploit the potential information captured in EHRs, in this study we propose a multimodal recurrent neural network model for cardiovascular risk prediction that integrates both medical texts and structured clinical information. The proposed multimodal bidirectional long short-term memory (BiLSTM) model concatenates word embeddings to classical clinical predictors before applying them to a final fully connected neural network. In the experiments, we compare performance of different deep neural network (DNN) architectures including convolutional neural network and long short-term memory in scenarios of using clinical variables and chest X-ray radiology reports. Evaluated on a data set of real world patients with manifest vascular disease or at high-risk for cardiovascular disease, the proposed BiLSTM model demonstrates state-of-the-art performance and outperforms other DNN baseline architectures., 8 pages, 2 figures

Graph signal processing (GSP) is an important methodology for studying data residing on irregular structures. As acquired data is increasingly taking the form of multi-way tensors, new signal processing tools are needed to maximally utilize the multi-way structure within the data. In this paper, we review modern signal processing frameworks generalizing GSP to multi-way data, starting from graph signals coupled to familiar regular axes such as time in sensor networks, and then extending to general graphs across all tensor modes. This widely applicable paradigm motivates reformulating and improving upon classical problems and approaches to creatively address the challenges in tensor-based data. We synthesize common themes arising from current efforts to combine GSP with tensor analysis and highlight future directions in extending GSP to the multi-way paradigm., In review for IEEE Signal Processing Magazine

In this paper, we present a novel statistical model, $\textit{the generalized-Gaussian-Rician}$ (GG-Rician) distribution, for the characterization of synthetic aperture radar (SAR) images. Since accurate statistical models lead to better results in applications such as target tracking, classification, or despeckling, characterizing SAR images of various scenes including urban, sea surface, or agricultural, is essential. The proposed statistical model is based on the Rician distribution to model the amplitude of a complex SAR signal, the in-phase and quadrature components of which are assumed to be generalized-Gaussian distributed. The proposed amplitude GG-Rician model is further extended to cover the intensity SAR signals. In the experimental analysis, the GG-Rician model is investigated for amplitude and intensity SAR images of various frequency bands and scenes in comparison to state-of-the-art statistical models that include $\mathcal{K}$, Weibull, Gamma, and Lognormal. In order to decide on the most suitable model, statistical significance analysis via Kullback-Leibler divergence and Kolmogorov-Smirnov statistics are performed. The results demonstrate the superior performance and flexibility of the proposed model for all frequency bands and scenes and its applicability on both amplitude and intensity SAR images. The Matlab package is available at https://github.com/oktaykarakus/GG-Rician-SAR-Image-Modelling., 20 Pages, 11 figures, 5 tables

The one-bit spatial Sigma-Delta concept has recently been proposed as an approach for achieving low distortion and low power consumption for massive multi-input multi-output systems. The approach exploits users located in known angular sectors or spatial oversampling to shape the quantization noise away from desired directions of arrival. While reducing the antenna spacing alleviates the adverse impact of quantization noise, it can potentially deteriorate the performance of the massive array due to excessive mutual coupling. In this paper, we analyze the impact of mutual coupling on the uplink spectral efficiency of a spatial one-bit Sigma-Delta massive MIMO architecture, and compare the resulting performance degradation to standard one-bit quantization as well as the ideal case with infinite precision. Our simulations show that the one-bit Sigma-Delta array is particularly advantageous in space-constrained scenarios, can still provide significant gains even in the presence of mutual coupling when the antennas are closely spaced., Presented in SPAWC 2020

We consider wireless transmission of images in the presence of channel output feedback. From a Shannon theoretic perspective feedback does not improve the asymptotic end-to-end performance, and separate source coding followed by capacity-achieving channel coding, which ignores the feedback signal, achieves the optimal performance. It is well known that separation is not optimal in the practical finite blocklength regime; however, there are no known practical joint source-channel coding (JSCC) schemes that can exploit the feedback signal and surpass the performance of separation-based schemes. Inspired by the recent success of deep learning methods for JSCC, we investigate how noiseless or noisy channel output feedback can be incorporated into the transmission system to improve the reconstruction quality at the receiver. We introduce an autoencoder-based JSCC scheme, which we call DeepJSCC-f, that exploits the channel output feedback, and provides considerable improvements in terms of the end-to-end reconstruction quality for fixed-length transmission, or in terms of the average delay for variable-length transmission. To the best of our knowledge, this is the first practical JSCC scheme that can fully exploit channel output feedback, demonstrating yet another setting in which modern machine learning techniques can enable the design of new and efficient communication methods that surpass the performance of traditional structured coding-based designs., IEEE Journal on Selected Areas in Information Theory (JSAIT), to appear

We present the implementation of binary and ternary neural networks in the hls4ml library, designed to automatically convert deep neural network models to digital circuits with FPGA firmware. Starting from benchmark models trained with floating point precision, we investigate different strategies to reduce the network's resource consumption by reducing the numerical precision of the network parameters to binary or ternary. We discuss the trade-off between model accuracy and resource consumption. In addition, we show how to balance between latency and accuracy by retaining full precision on a selected subset of network components. As an example, we consider two multiclass classification tasks: handwritten digit recognition with the MNIST data set and jet identification with simulated proton-proton collisions at the CERN Large Hadron Collider. The binary and ternary implementation has similar performance to the higher precision implementation while using drastically fewer FPGA resources., Update to MLST journal version

Research has shown that deep neural networks contain significant redundancy, and thus that high classification accuracy can be achieved even when weights and activations are quantized down to binary values. Network binarization on FPGAs greatly increases area efficiency by replacing resource-hungry multipliers with lightweight XNOR gates. However, an FPGA's fundamental building block, the K-LUT, is capable of implementing far more than an XNOR: it can perform any K-input Boolean operation. Inspired by this observation, we propose LUTNet, an end-to-end hardware-software framework for the construction of area-efficient FPGA-based neural network accelerators using the native LUTs as inference operators. We describe the realization of both unrolled and tiled LUTNet architectures, with the latter facilitating smaller, less power-hungry deployment over the former while sacrificing area and energy efficiency along with throughput. For both varieties, we demonstrate that the exploitation of LUT flexibility allows for far heavier pruning than possible in prior works, resulting in significant area savings while achieving comparable accuracy. Against the state-of-the-art binarized neural network implementation, we achieve up to twice the area efficiency for several standard network models when inferencing popular datasets. We also demonstrate that even greater energy efficiency improvements are obtainable., Comment: arXiv admin note: substantial text overlap with arXiv:1904.00938. Accepted manuscript uploaded 02/03/20. DOA 01/03/20

We consider the design of distributed algorithms that govern the manner in which agents contribute to a social sensing platform. Specifically, we are interested in situations where fairness among the agents contributing to the platform is needed. A notable example are platforms operated by public bodies, where fairness is a legal requirement. The design of such distributed systems is challenging due to the fact that we wish to simultaneously realise an efficient social sensing platform, but also deliver a predefined quality of service to the agents (for example, a fair opportunity to contribute to the platform). In this paper, we introduce iterated function systems (IFS) as a tool for the design and analysis of systems of this kind. We show how the IFS framework can be used to realise systems that deliver a predictable quality of service to agents, can be used to underpin contracts governing the interaction of agents with the social sensing platform, and which are efficient. To illustrate our design via a use case, we consider a large, high-density network of participating parked vehicles. When awoken by an administrative centre, this network proceeds to search for moving missing entities of interest using RFID-based techniques. We regulate which vehicles are actively searching for the moving entity of interest at any point in time. In doing so, we seek to equalise vehicular energy consumption across the network. This is illustrated through simulations of a search for a missing Alzheimer's patient in Melbourne, Australia. Experimental results are presented to illustrate the efficacy of our system and the predictability of access of agents to the platform independent of initial conditions., Comment: 18 pages, 6 figures

This paper considers the secrecy performance of several schemes for multi-antenna transmission to single-antenna users with full-duplex (FD) capability against randomly distributed single-antenna eavesdroppers (EDs). These schemes and related scenarios include transmit antenna selection (TAS), transmit antenna beamforming (TAB), artificial noise (AN) from the transmitter, user selection based their distances to the transmitter, and colluding and non-colluding EDs. The locations of randomly distributed EDs and users are assumed to be distributed as Poisson Point Process (PPP). We derive closed form expressions for the secrecy outage probabilities (SOP) of all these schemes and scenarios. The derived expressions are useful to reveal the impacts of various environmental parameters and user's choices on the SOP, and hence useful for network design purposes. Examples of such numerical results are discussed., Comment: Paper accepted for publication in IEEE Transactions on Information Forensics and Security

We propose deep learning based communication methods for adaptive-bandwidth transmission of images over wireless channels. We consider the scenario in which images are transmitted progressively in layers over time or frequency, and such layers can be aggregated by receivers in order to increase the quality of their reconstructions. We investigate two scenarios, one in which the layers are sent sequentially, and incrementally contribute to the refinement of a reconstruction, and another in which the layers are independent and can be retrieved in any order. Those scenarios correspond to the well known problems of \textit{successive refinement} and \textit{multiple descriptions}, respectively, in the context of joint source-channel coding (JSCC). We propose DeepJSCC-$l$, an innovative solution that uses convolutional autoencoders, and present three architectures with different complexity trade-offs. To the best of our knowledge, this is the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. Numerical results show that DeepJSCC-$l$ can learn to transmit the source progressively with negligible losses in the end-to-end performance compared with a single transmission. Moreover, DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes, with the additional advantage of graceful degradation with channel SNR.

Convolutional Neural Networks (CNNs) are now a well-established tool for solving computational imaging problems. Modern CNN-based algorithms obtain state-of-the-art performance in diverse image restoration problems. Furthermore, it has been recently shown that, despite being highly overparameterized, networks trained with a single corrupted image can still perform as well as fully trained networks. We introduce a formal link between such networks through their neural tangent kernel (NTK), and well-known non-local filtering techniques, such as non-local means or BM3D. The filtering function associated with a given network architecture can be obtained in closed form without need to train the network, being fully characterized by the random initialization of the network weights. While the NTK theory accurately predicts the filter associated with networks trained using standard gradient descent, our analysis shows that it falls short to explain the behaviour of networks trained using the popular Adam optimizer. The latter achieves a larger change of weights in hidden layers, adapting the non-local filtering function during training. We evaluate our findings via extensive image denoising experiments 1.

Recently, analog compute-in-memory (CIM) architectures based on emerging analog non-volatile memory (NVM) technologies have been explored for deep neural networks (DNNs) to improve scalability, speed, and energy efficiency. Such architectures, however, leverage charge conservation, an operation with infinite resolution, and thus are susceptible to errors. Thus, the inherent stochasticity in any analog NVM used to execute DNNs, will compromise performance. Several reports have demonstrated the use of analog NVM for CIM in a limited scale. It is unclear whether the uncertainties in computations will prohibit large-scale DNNs. To explore this critical issue of scalability, this article first presents a simulation framework to evaluate the feasibility of large-scale DNNs based on CIM architecture and analog NVM. Simulation results show that DNNs trained for high-precision digital computing engines are not resilient against the uncertainty of the analog NVM devices. To avoid such catastrophic failures, this article introduces the analog bi-scale representation for the DNN, and the Hessian-aware Stochastic Gradient Descent training algorithm to enhance the inference accuracy of trained DNNs. As a result of such enhancements, DNNs such as Wide ResNets for CIFAR-100 image recognition problem are demonstrated to have significant performance improvements in accuracy without adding cost to the inference hardware .

Glioblastoma is profoundly heterogeneous in microstructure and vasculature, which may lead to tumor regional diversity and distinct treatment response. Although successful in tumor sub-region segmentation and survival prediction, radiomics based on machine learning algorithms, is challenged by its robustness, due to the vague intermediate process and track changes. Also, the weak interpretability of the model poses challenges to clinical application. Here we proposed a machine learning framework to semi-automatically fine-tune the clustering algorithms and quantitatively identify stable sub-regions for reliable clinical survival prediction. Hyper-parameters are automatically determined by the global minimum of the trained Gaussian Process (GP) surrogate model through Bayesian optimization(BO) to alleviate the difficulty of tuning parameters for clinical researchers. To enhance the interpretability of the survival prediction model, we incorporated the prior knowledge of intra-tumoral heterogeneity, by segmenting tumor sub-regions and extracting sub-regional features. The results demonstrated that the global minimum of the trained GP surrogate can be used as sub-optimal hyper-parameter solutions for efficient. The sub-regions segmented based on physiological MRI can be applied to predict patient survival, which could enhance the clinical interpretability for the machine learning model., Comment: This is an older version of a paper titled "Adaptive unsupervised learning with better feature representation for intra-tumor partitioning and survival prediction in glioblastoma," and the two should be combined into one

This paper investigates unmanned aerial vehicle (UAV)-aided backscatter communication (BackCom) networks, where the UAV is leveraged to help the backscatter device (BD) forward signals to the receiver. Based on the presence or absence of a direct link between BD and receiver, two protocols, namely transmit-backscatter (TB) protocol and transmit-backscatter-relay (TBR) protocol, are proposed to utilize the UAV to assist the BD. In particular, we formulate the system throughput maximization problems for the two protocols by jointly optimizing the time allocation, reflection coefficient and UAV trajectory. Different static/dynamic circuit power consumption models for the two protocols are analyzed. The resulting optimization problems are shown to be non-convex, which are challenging to solve. We first consider the dynamic circuit power consumption model, and decompose the original problems into three sub-problems, namely time allocation optimization with fixed UAV trajectory and reflection coefficient, reflection coefficient optimization with fixed UAV trajectory and time allocation, and UAV trajectory optimization with fixed reflection coefficient and time allocation. Then, an efficient iterative algorithm is proposed for both protocols by leveraging the block coordinate descent method and successive convex approximation (SCA) techniques. In addition, for the static circuit power consumption model, we obtain the optimal time allocation with a given reflection coefficient and UAV trajectory and the optimal reflection coefficient with low computational complexity by using the Lagrangian dual method. Simulation results show that the proposed protocols are able to achieve significant throughput gains over the compared benchmarks.

In this paper, we investigate the resource allocation for uplink non-orthogonal multiple access (NOMA) enabled low-power wide-area (LPWA) networks to support the massive connectivity of users/nodes. Here, LPWA nodes communicate with a central gateway through resource blocks like channels, transmission times, bandwidths, etc. The nodes sharing the same resource blocks suffer from intra-cluster interference and possibly inter-cluster interference, which makes current LPWA networks unable to support the massive connectivity. Using the minimum transmission rate metric to highlight the interference reduction that results from the addition of NOMA, and while assuring user throughput fairness, we decompose the minimum rate maximization optimization problem into three sub-problems. First, a low-complexity sub-optimal nodes clustering scheme is proposed assigning nodes to channels based on their normalized channel gains. Then, two types of transmission time allocation algorithms are proposed that either assure fair or unfair transmission time allocation between LPWA nodes sharing the same channel. For a given channel and transmission time allocation, we further propose an optimal power allocation scheme. Simulation evaluations demonstrate approximately 100dB improvement of the selected metric for a single network with 4000 active nodes., Comment: IEEE Global Communications Conference (Globecom'2019)

This paper presents secrecy analyses of a full-duplex MIMOME network which consists of two full-duplex multi-antenna users (Alice and Bob) and an arbitrarily located multi-antenna eavesdropper (Eve). The paper assumes that Eve's channel state information (CSI) is completely unknown to Alice and Bob except for a small radius of secured zone. The first part of this paper aims to optimize the powers of jamming noises from both users. To handle Eve's CSI being unknown to users, the focus is placed on Eve at the most harmful location, and the large matrix theory is applied to yield a hardened secrecy rate to work on. The performance gain of the power optimization in terms of maximum tolerable number of antennas on Eve is shown to be significant. The second part of this paper shows two analyses of anti-eavesdropping channel estimation (ANECE) that can better handle Eve with any number of antennas. One analysis assumes that Eve has a prior statistical knowledge of its CSI, which yields lower and upper bounds on secure degrees of freedom of the system as functions of the number (N) of antennas on Eve and the size (K) of information packet. The second analysis assumes that Eve does not have any prior knowledge of its CSI but performs blind detection of information, which yields an approximate secrecy rate for the case of K being larger than N.

Sensitive inferences and user re-identification are major threats to privacy when raw sensor data from wearable or portable devices are shared with cloud-assisted applications. To mitigate these threats, we propose mechanisms to transform sensor data before sharing them with applications running on users' devices. These transformations aim at eliminating patterns that can be used for user re-identification or for inferring potentially sensitive activities, while introducing a minor utility loss for the target application (or task). We show that, on gesture and activity recognition tasks, we can prevent inference of potentially sensitive activities while keeping the reduction in recognition accuracy of non-sensitive activities to less than 5 percentage points. We also show that we can reduce the accuracy of user re-identification and of the potential inference of gender to the level of a random guess, while keeping the accuracy of activity recognition comparable to that obtained on the original data., Accepted to appear in Pervasive and Mobile computing (PMC) Journal, Elsevier

The time average expected age of information (AoI) is studied for status updates sent from an energy-harvesting transmitter with a finite-capacity battery. The optimal scheduling policy is first studied under different feedback mechanisms when the channel and energy harvesting statistics are known. For the case of unknown environments, an average-cost reinforcement learning algorithm is proposed that learns the system parameters and the status update policy in real time. The effectiveness of the proposed methods is verified through numerical results.

In this paper, we consider the amplify-and-forward relay networks in mmWave systems and propose a hybrid precoder/combiner design approach. The phase-only RF precoding/combining matrices are first designed to support multi-stream transmission, where we compensate the phase for the eigenmodes of the channel. Then, the baseband precoders/combiners are performed to achieve the maximum mutual information. Based on the data processing inequality for the mutual information, we first jointly design the baseband source and relay nodes to maximize the mutual information before the destination baseband receiver. The proposed low-complexity iterative algorithm for the source and relay nodes is based on the equivalence between the mutual information maximization and the weighted MMSE. After we obtain the optimal precoder and combiner for the source and relay nodes, we implement the MMSE-SIC filter at the baseband receiver to keep the mutual information unchanged, thus obtaining the optimal mutual information for the whole relay system. Simulation results show that our algorithm achieves better performance with lower complexity compared with other algorithms in the literature. In addition, we also propose a robust joint transceiver design for imperfect channel state information.

We propose a joint source and channel coding (JSCC) technique for wireless image transmission that does not rely on explicit codes for either compression or error correction; instead, it directly maps the image pixel values to the complex-valued channel input symbols. We parameterize the encoder and decoder functions by two convolutional neural networks (CNNs), which are trained jointly, and can be considered as an autoencoder with a non-trainable layer in the middle that represents the noisy communication channel. Our results show that the proposed deep JSCC scheme outperforms digital transmission concatenating JPEG or JPEG2000 compression with a capacity achieving channel code at low signal-to-noise ratio (SNR) and channel bandwidth values in the presence of additive white Gaussian noise (AWGN). More strikingly, deep JSCC does not suffer from the ``cliff effect'', and it provides a graceful performance degradation as the channel SNR varies with respect to the SNR value assumed during training. In the case of a slow Rayleigh fading channel, deep JSCC learns noise resilient coded representations and significantly outperforms separation-based digital communication at all SNR and channel bandwidth values., Comment: To appear in IEEE Transactions on Cognitive Communications and Networking

This letter analyzes the performances of a simple reconstruction method, namely the Projected Back-Projection (PBP), for estimating the direction of a sparse signal from its phase-only (or amplitude-less) complex Gaussian random measurements, i.e., an extension of one-bit compressive sensing to the complex field. To study the performances of this algorithm, we show that complex Gaussian random matrices respect, with high probability, a variant of the Restricted Isometry Property (RIP) relating to the l1 -norm of the sparse signal measurements to their l2 -norm. This property allows us to upper-bound the reconstruction error of PBP in the presence of phase noise. Monte Carlo simulations are performed to highlight the performance of our approach in this phase-only acquisition model when compared to error achieved by PBP in classical compressive sensing., Comment: 4 pages, 2 figures

Achievable information rates are used as a metric to design novel modulation formats via geometric shaping. The proposed geometrically shaped 256-ary constellation achieves SNR gains of up to 1.18 dB., Comment: Additional references have been added

The key obstacle to achieving the full potential of the millimeter wave (mmWave) band has been the poor propagation characteristics of wireless signals in this band. One approach to overcome this issue is to use antennas that can support higher gains while providing beam adaptability and diversity, i.e., reconfigurable antennas. In this article, we present a new architecture for mmWave multiple-input multiple-output (MIMO) communications that uses a new class of reconfigurable antennas. More specifically, the proposed lens-based antennas can support multiple radiation patterns while using a single radio frequency chain. Moreover, by using a beam selection network, each antenna beam can be steered in the desired direction. Further, using the proposed reconfigurable antenna in a MIMO architecture, we propose a new signal processing algorithm that uses the additional degrees of freedom provided by the antennas to overcome propagation issues at mmWave frequencies. Our simulation results show that the proposed reconfigurable antenna MIMO architecture significantly enhances the performance of mmWave communication systems.

New communication standards need to deal with machine-to-machine communications, in which users may start or stop transmitting at any time in an asynchronous manner. Thus, the number of users is an unknown and time-varying parameter that needs to be accurately estimated in order to properly recover the symbols transmitted by all users in the system. In this paper, we address the problem of joint channel parameter and data estimation in a multiuser communication channel in which the number of transmitters is not known. For that purpose, we develop the infinite factorial finite state machine model, a Bayesian nonparametric model based on the Markov Indian buffet that allows for an unbounded number of transmitters with arbitrary channel length. We propose an inference algorithm that makes use of slice sampling and particle Gibbs with ancestor sampling. Our approach is fully blind as it does not require a prior channel estimation step, prior knowledge of the number of transmitters, or any signaling information. Our experimental results, loosely based on the LTE random access channel, show that the proposed approach can effectively recover the data-generating process for a wide range of scenarios, with varying number of transmitters, number of receivers, constellation order, channel length, and signal-to-noise ratio., 15 pages, 15 figures

Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption. One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases. Such an approach usually results in a decline in performance. Here, we introduce a method for designing optimally heterogeneously quantized versions of deep neural network models for minimum-energy, high-accuracy, nanosecond inference and fully automated deployment on chip. With a per-layer, per-parameter type automatic quantization procedure, sampling from a wide range of quantizers, model energy consumption and size are minimized while high accuracy is maintained. This is crucial for the event selection procedure in proton-proton collisions at the CERN Large Hadron Collider, where resources are strictly limited and a latency of ${\mathcal O}(1)~\mu$s is required. Nanosecond inference and a resource consumption reduced by a factor of 50 when implemented on field-programmable gate array hardware are achieved.

Current optical coherent transponders technology is driving data rates towards 1 Tb/s/{\lambda}and beyond. This trend requires both high-performance coded modulation schemes and efficient implementation of the forward-error-correction (FEC) decoder. A possible solution to this problem is combining advanced multidimensional modulation formats with low-complexity hybrid HD/SD FEC decoders. Following this rationale, in this paper we combine two recently introduced coded modulation techniques:the geometrically-shaped 4D-64 polarization ring-switched and the soft-aided bit-marking-scaled reliability decoder. This joint scheme enabled us to experimentally demonstrate the transmission of 11x218 Gbit/s channels over transatlantic distances at 5.2bit/4D-sym. Furthermore, a 30% reach increase is demonstrated over PM-8QAM and conventional HD-FEC decoding for product codes., Comment: This work has been submitted to IEEE Photonics Technology Letters

We introduce the elliptical Ornstein-Uhlenbeck (OU) process, which is a generalisation of the well-known univariate OU process to bivariate time series. This process maps out elliptical stochastic oscillations over time in the complex plane, which are observed in many applications of coupled bivariate time series. The appeal of the model is that elliptical oscillations are generated using one simple first order stochastic differential equation (SDE), whereas alternative models require more complicated vectorised or higher order SDE representations. The second useful feature is that parameter estimation can be performed semi-parametrically in the frequency domain using the Whittle Likelihood. We determine properties of the model including the conditions for stationarity, and the geometrical structure of the elliptical oscillations. We demonstrate the utility of the model by measuring periodic and elliptical properties of Earth's polar motion., Comment: To appear in Statistics and Its Interface