Search

The ongoing decarbonisation of modern electricity systems has led to a substantial increase of operational uncertainty, particularly due to the large-scale integration of renewable energy generation. However, the expanding space of possible operating points renders necessary the development of novel security assessment approaches. In this paper we focus on the use of security rules where classifiers are trained offline to characterize previously unseen points as safe or unsafe. This paper proposes a novel deep learning-based feature extraction framework for building security rules. We show how deep autoencoders can be used to transform the space of conventional state variables (e.g., power flows) to a small number of dimensions where we can optimally distinguish between safe and unsafe operation. The proposed framework is data-driven and can be useful in multiple applications within the context of security assessment. To achieve high accuracy, a novel objective-based loss function is proposed to address the issue of imbalanced safe/unsafe classes that characterize electricity system operation. Furthermore, an R-vine copula-based model is proposed to sample historical data and generate large populations of anticipated system states for training. The superior performance of the proposed framework is demonstrated through a series of case studies and comparisons using the load and wind generation data from the French transmission system, which have been mapped to the IEEE 118-bus system. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning approach to the adaptive and sequential beamforming design problem for the initial access phase in a mmWave environment with a single-path channel. For a single-user scenario where the problem is equivalent to designing the sequence of sensing beamformers to learn the angle of arrival (AoA) of the dominant path, we propose a novel deep neural network (DNN) that designs the adaptive sensing vectors sequentially based on the available information so far at the base station (BS). By recognizing that the AoA posterior distribution is a sufficient statistic for solving the initial access problem, we use the posterior distribution as the input to the proposed DNN for designing the adaptive sensing strategy. However, computing the posterior distribution can be computationally challenging when the channel fading coefficient is unknown. To address this issue, this paper proposes to use an estimate of the fading coefficient to compute an approximation of the posterior distribution. Further, this paper shows that the proposed DNN can deal with practical beamforming constraints such as the constant modulus constraint. Numerical results demonstrate that compared to the existing adaptive and non-adaptive beamforming schemes, the proposed DNN-based adaptive sensing strategy achieves a significantly better AoA acquisition performance. [ABSTRACT FROM AUTHOR]

This paper investigates the impact of uncoordinated, coordinated, and smart charging of plug-in hybrid electric vehicles (PHEVs) on the optimal operation of microgrids (MGs) incorporating the dynamic line rating (DLR) security constraint. The DLR constraint, particularly in the islanding mode, influences the ampacity of MG feeders, when distribution lines reach their maximum capacity. To overcome any line outage or contingency situation, smart PHEVs are utilized to help improve the grid security. However, using PHEVs can cause higher power losses and feeder overloading issues. To address these concerns, a reconfiguration technique is employed in this paper. A heuristic algorithm, known as the collective decision-based optimization algorithm, is utilized to overcome the non-convexity and nonlinearity of the problem. The unscented transform technique is employed to model DLR uncertainties caused by solar radiation, load demand, and weather temperature, as well as PHEVs’ uncertainties caused by varying charging strategies, numbers of PHEVs being charged, charging start time, and charging duration. Moreover, a deep learning gated recurrent unit technique is designed to forecast renewable power output for mitigating the uncertainties in renewable energy components. A modified IEEE 33-bus test network is deployed to evaluate the efficiency and performance of the proposed model. [ABSTRACT FROM AUTHOR]

This paper shows that deep neural network (DNN) can be used for efficient and distributed channel estimation, quantization, feedback, and downlink multiuser precoding for a frequency-division duplex massive multiple-input multiple-output system in which a base station (BS) serves multiple mobile users, but with rate-limited feedback from the users to the BS. A key observation is that the multiuser channel estimation and feedback problem can be thought of as a distributed source coding problem. In contrast to the traditional approach where the channel state information (CSI) is estimated and quantized at each user independently, this paper shows that a joint design of pilots and a new DNN architecture, which maps the received pilots directly into feedback bits at the user side then maps the feedback bits from all the users directly into the precoding matrix at the BS, can significantly improve the overall performance. This paper further proposes robust design strategies with respect to channel parameters and also a generalizable DNN architecture for varying number of users and number of feedback bits. Numerical results show that the DNN-based approach with short pilot sequences and very limited feedback overhead can already approach the performance of conventional linear precoding schemes with full CSI. [ABSTRACT FROM AUTHOR]

To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design. [ABSTRACT FROM AUTHOR]

The papers in this special section focus on increasing interests in the development and implementation of advanced artificial intelligence (AI) and machine learning (ML) methods for tackling the reliability and system health prognostics challenges in various industrial applications. [ABSTRACT FROM AUTHOR]

Today’s and emerging computing tasks are extremely demanding in terms of storage, energy efficiency, and computing efficiency; data-intensive/big-data applications and Internet-of-Things are couple of examples. In addition, today’s computer architectures and device technologies are facing major challenges making them incapable to deliver the required functionalities and features. Computers are facing the three well-known walls: the memory wall, the instruction level parallelism wall, and the power wall. Similarly, nanoscale CMOS technology is facing three walls: the reliability wall, the leakage wall, and the cost wall. In order for computing systems to continue to deliver sustainable benefits for the foreseeable future society, alternative computing architectures have to be explored in the light of emerging new device technologies. Using memristive device technologyto enable new computing paradigms such as computation-in-memory architecture–is one of the emerging alternatives that could provide a huge potential in terms of energy and computing efficiency. [ABSTRACT FROM AUTHOR]

Reconfigurable intelligent surface (RIS) is capable of intelligently manipulating the phases of the incident electromagnetic wave to improve the wireless propagation environment between the base-station (BS) and the users. This paper addresses the joint user scheduling, RIS configuration, and BS beamforming problem in an RIS-assisted downlink network with limited pilot overhead. We show that graph neural networks (GNN) with permutation invariant and equivariant properties can be used to appropriately schedule users and to design RIS configurations to achieve high overall throughput while accounting for fairness among the users. As compared to the conventional methodology of first estimating the channels then optimizing the user schedule, RIS configuration and the beamformers, this paper shows that an optimized user schedule can be obtained directly from a very short set of pilots using a GNN, then the RIS configuration can be optimized using a second GNN, and finally the BS beamformers can be designed based on the overall effective channel. Numerical results show that the proposed approach can utilize the received pilots more efficiently than the conventional channel estimation based approach, and can generalize to systems with an arbitrary number of users. [ABSTRACT FROM AUTHOR]

Link prediction aims at inferring missing links or predicting future ones based on the currently observed network. This topic is important for many applications such as social media, bioinformatics and recommendation systems. Most existing methods focus on homogeneous settings and consider only low-order pairwise relations while ignoring either the heterogeneity or high-order complex relations among different types of nodes, which tends to lead to a sub-optimal embedding result. This paper presents a method named Heterogeneous Hypergraph Variational Autoencoder (HeteHG-VAE) for link prediction in heterogeneous information networks (HINs). It first maps a conventional HIN to a heterogeneous hypergraph with a certain kind of semantics to capture both the high-order semantics and complex relations among nodes, while preserving the low-order pairwise topology information of the original HIN. Then, deep latent representations of nodes and hyperedges are learned by a Bayesian deep generative framework from the heterogeneous hypergraph in an unsupervised manner. Moreover, a hyperedge attention module is designed to learn the importance of different types of nodes in each hyperedge. The major merit of HeteHG-VAE lies in its ability of modeling multi-level relations in heterogeneous settings. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Traffic state estimation (TSE) bifurcates into two main categories, model-driven and data-driven (e.g., machine learning, ML) approaches, while each suffers from either deficient physics or small data. To mitigate these limitations, recent studies introduced hybrid methods, such as physics-informed deep learning (PIDL), which contains both model-driven and data-driven components. This paper contributes an improved paradigm, called physics-informed deep learning with a fundamental diagram learner (PIDL + FDL), which integrates ML terms into the model-driven component to learn a functional form of a fundamental diagram (FD), i.e., a mapping from traffic density to flow or velocity. The proposed PIDL + FDL has the advantages of performing the TSE learning, model parameter identification, and FD estimation simultaneously. This paper focuses on highway TSE with observed data from loop detectors, using traffic density or velocity as traffic variables. We demonstrate the use of PIDL + FDL to solve popular first-order and second-order traffic flow models and reconstruct the FD relation as well as model parameters that are outside the FD term. We then evaluate the PIDL + FDL-based TSE using the Next Generation SIMulation (NGSIM) dataset. The experimental results show the superiority of the PIDL + FDL in terms of improved estimation accuracy and data efficiency over advanced baseline TSE methods, and additionally, the capacity to properly learn the unknown underlying FD relation. [ABSTRACT FROM AUTHOR]

Flight delay prediction has recently gained growing popularity due to the significant role it plays in efficient airline and airport operation. Most of the previous prediction works consider the single-airport scenario, which overlooks the time-varying spatial interactions hidden in airport networks. In this paper, the flight delay prediction problem is investigated from a network perspective (i.e., multi-airport scenario). To model the time-evolving and periodic graph-structured information in the airport network, a flight delay prediction approach based on the graph convolutional neural network (GCN) is developed in this paper. More specifically, regarding that GCN cannot take both delay time-series and time-evolving graph structures as inputs, a temporal convolutional block based on the Markov property is employed to mine the time-varying patterns of flight delays through a sequence of graph snapshots. Moreover, considering that unknown occasional air routes under emergency may result in incomplete graph-structured inputs for GCN, an adaptive graph convolutional block is embedded into the proposed method to expose spatial interactions hidden in airport networks. Through extensive experiments, it has been shown that the proposed approach outperforms benchmark methods with a satisfying accuracy improvement at the cost of acceptable execution time. The obtained results reveal that deep learning approach based on graph-structured inputs have great potentials in the flight delay prediction problem. [ABSTRACT FROM AUTHOR]

Monitoring of complex industrial processes can be achieved by obtaining process data by utilising various sensing modalities. The recent emergence of deep learning provides a new routine for processing multi-sensor information. However, the learning ability of shallow neural networks is insufficient, and the data amount required by deep networks is often too large for industrial scenarios. This paper provides a novel deep transfer learning method as a possible solution that offers an advantage of better learning ability of the deep network without the requirement for a large amount of training data. This paper presents how Transformer with self-attention trained from natural language can be transferred to the sensor fusion task. Our proposed method is tested on 3 datasets: condition monitoring of a hydraulic system, bearing, and gearbox dataset. The results show that the Transformer trained from natural language can effectively reduce the required data amount for using deep learning in industrial sensor fusion with high prediction accuracy. The difficult and uncertain artificial feature engineering which requires a large workload can also be eliminated, as the deep networks are able to extract features automatically. In addition, the self-attention mechanism of Transformer aids in the identification of critical sensors, hence the interpretability of deep learning in industrial sensor fusion can be improved. [ABSTRACT FROM AUTHOR]

Millimeter-wave radar and machine vision are both important means for intelligent vehicles to perceive the surrounding environment. Aiming at the problem of multi-sensor fusion, this paper proposes the object detection method of millimeter-wave radar and vision fusion. Radar and camera complement each other, and radar data fusion in machine vision network can effectively reduce the rate of missed detection under insufficient light conditions, and improve the accuracy of remote small object detection. The radar information is processed by mapping transformation neural network to obtain the mask map, so that radar information and visual information in the same scale. A multi-data source deep learning object detection network (MS-YOLO) based on millimeter-wave radar and vision fusion was proposed. Homemade datasets were used for training and testing. This maximized the use of sensor information and improved the detection accuracy under the premise of ensuring the detection speed. Compared with the original YOLOv5 (the fifth version of the You Only Look Once) network, the results show that the MS-YOLO network meets the accuracy requirements better. Among the models, the large model of MS-YOLO has the highest accuracy with an mAP reaching 0.888. The small model of MS-YOLO has good accuracy and speed, and the mAP reaches 0.841 while maintaining a high frame rate of 65 fps. [ABSTRACT FROM AUTHOR]

For deep learning model training, most existing supervised learning-based person re-identification (Re-ID) models require considerable data with annotations as samples. However, it is labor-intensive to generate labeled data in many real-world situations. Meanwhile, the large scale of the existing models increases the load of model learning. To this end, this paper proposes a person Re-ID method by deep batch active learning and knowledge distillation. With the goal of minimizing the human labeling cost and maximizing the performance of the person Re-ID model, a batch active learning algorithm is applied to person Re-ID, which selects samples with both uncertainty and diversity, and the model needs only a small amount of labeled data to achieve a high performance. In addition, a knowledge distillation approach is used to compress the original backbone model to reduce the model scale while maintaining the model performance. Furthermore, experiments on two public datasets demonstrate the effectiveness and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

The vanilla Few-shot Learning (FSL) learns to build a classifier for a new concept from one or very few target examples, with the general assumption that source and target classes are sampled from the same domain. Recently, the task of Cross-Domain Few-Shot Learning (CD-FSL) aims at tackling the FSL where there is a huge domain shift between the source and target datasets. Extensive efforts on CD-FSL have been made via either directly extending the meta-learning paradigm of vanilla FSL methods, or employing massive unlabeled target data to help learn models. In this paper, we notice that in the CD-FSL task, the few labeled target images have never been explicitly leveraged to inform the model in the training stage. However, such a labeled target example set is very important to bridge the huge domain gap. Critically, this paper advocates a more practical training scenario for CD-FSL. And our key insight is to utilize a few labeled target data to guide the learning of the CD-FSL model. Technically, we propose a novel Generalized Meta-learning based Feature-Disentangled Mixup network, namely GMeta-FDMixup. We make three key contributions of utilizing GMeta-FDMixup to address CD-FSL. Firstly, we present two mixup modules – mixup-P and mixup-M that help facilitate utilizing the unbalanced and disjoint source and target datasets. These two novel modules enable diverse image generation for training the model on the source domain. Secondly, to narrow the domain gap explicitly, we contribute a novel feature disentanglement module that learns to decouple the domain-irrelevant and domain-specific features. By stripping the domain-specific features, we alleviate the negative effects caused by the domain inductive bias. Finally, we repurpose a new contrastive learning module, dubbed ConL. ConL prevents the model from only capturing category-related features via introducing contrastive loss. Thus, the generalization ability on novel categories is improved. Extensive experimental results on two benchmarks show the superiority of our setting and the effectiveness of our method. Code and models will be released. [ABSTRACT FROM AUTHOR]

Multiview video coding (MVC) aims to compress the multiview video through the elimination of video redundancies, where the quality of the reference frame directly affects the compression efficiency. In this paper, we propose a deep virtual reference frame generation method based on a disparity-aware reference frame generation network (DAG-Net) to transform the disparity relationship between different viewpoints and generate a more reliable reference frame. The proposed DAG-Net consists of a multi-level receptive field module, a disparity-aware alignment module, and a fusion reconstruction module. First, a multi-level receptive field module is designed to enlarge the receptive field, and extract the multi-scale deep features of the temporal and inter-view reference frames. Then, a disparity-aware alignment module is proposed to learn the disparity relationship, and perform disparity shift on the inter-view reference frame to align it with the temporal reference frame. Finally, a fusion reconstruction module is utilized to fuse the complementary information and generate a more reliable virtual reference frame. Experiments demonstrate that the proposed reference frame generation method achieves superior performance for multiview video coding. [ABSTRACT FROM AUTHOR]

This paper introduces a deep learning modular network for side-channel analysis. Our approach features a deep learning architecture with the capability to exchange parts (modules) with other neural networks. We aim to introduce reusable trained modules into side-channel analysis instead of building architectures from scratch for each evaluation, reducing the body of work. Our experiments demonstrate that our architecture feasibly assesses a side-channel evaluation, suggesting that learning transferability is possible using the architecture we propose in this paper. [ABSTRACT FROM AUTHOR]

In near coast navigation, buoys and beacons convey essential information about dangers. At night-time, selected buoys send out individual blink-sequences that are marked in sea charts. International regulations require that navigation officer on watch makes visual confirmation of objects and their type in order to navigate safely. With rapid developments of highly automated vessels, this duty needs be carried out by algorithms that detect and locate objects without human intervention. At night-time, this requires algorithms that decode blink sequences and are able to classify from this information. The paper investigates this problem and suggests an algorithm that solves the problem. Convolutional Neural Networks (CNN) with Gated Recurrent Units (GRU) are developed for classification. A dedicated architecture is suggested that includes both temporal and color decoding to obtain unique precision. We demonstrate how networks are trained on synthetically generated data, and the paper shows, on real-world data, how the suggested approach yields 100.0% accurate results on 44 real-world recordings while being robust to inaccuracy in actual blink sequences. Comparison with baseline signal processing and with a recent state-of-the-art 3D CNN model shows that the new blink-sequence classifier outperforms alternative algorithms. A showcase of the results of this work is available in this video: https://youtu.be/KEi8qNnKV2w. [ABSTRACT FROM AUTHOR]

As a fundamental vision task, facial expression recognition has made substantial progress recently. However, the recognition performance often degrades significantly in real-world scenarios due to the lack of robust facial features. In this paper, we propose an effective facial feature learning method that takes the advantage of facial chirality to discover the discriminative features for facial expression recognition. Most previous studies implicitly assume that human faces are symmetric. However, our work reveals that the facial asymmetric effect can be a crucial clue. Given a face image and its reflection without additional labels, we decouple the emotion-invariant facial features from the input image pair to better capture the emotion-related facial features. Moreover, as our model aligns emotion-related features of the image pair to enhance the recognition performance, the value of precise facial landmark alignment as a pre-processing step is reconsidered in this paper. Experiments demonstrate that the learned emotion-related features outperform the state of the art methods on several facial expression recognition benchmarks as well as real-world occlusion datasets, which manifests the effectiveness and robustness of the proposed model. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning approach to a class of active sensing problems in wireless communications in which an agent sequentially interacts with an environment over a predetermined number of time frames to gather information in order to perform a sensing or actuation task for maximizing some utility function. In such an active learning setting, the agent needs to design an adaptive sensing strategy sequentially based on the observations made so far. To tackle such a challenging problem in which the dimension of historical observations increases over time, we propose to use a long short-term memory (LSTM) network to exploit the temporal correlations in the sequence of observations and to map each observation to a fixed-size state information vector. We then use a deep neural network (DNN) to map the LSTM state at each time frame to the design of the next measurement step. Finally, we employ another DNN to map the final LSTM state to the desired solution. We investigate the performance of the proposed framework for adaptive channel sensing problems in wireless communications. In particular, we consider the adaptive beamforming problem for mmWave beam alignment and the adaptive reconfigurable intelligent surface sensing problem for reflection alignment. Numerical results demonstrate that the proposed deep active sensing strategy outperforms the existing adaptive or nonadaptive sensing schemes. [ABSTRACT FROM AUTHOR]

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation. [ABSTRACT FROM AUTHOR]

Efficient point cloud compression is essential for applications like virtual and mixed reality, autonomous driving, and cultural heritage. This paper proposes a deep learning-based inter-frame encoding scheme for dynamic point cloud geometry compression. We propose a lossy geometry compression scheme that predicts the latent representation of the current frame using the previous frame by employing a novel feature space inter-prediction network. The proposed network utilizes sparse convolutions with hierarchical multiscale 3D feature learning to encode the current frame using the previous frame. The proposed method introduces a novel predictor network for motion compensation in the feature domain to map the latent representation of the previous frame to the coordinates of the current frame to predict the current frame’s feature embedding. The framework transmits the residual of the predicted features and the actual features by compressing them using a learned probabilistic factorized entropy model. At the receiver, the decoder hierarchically reconstructs the current frame by progressively rescaling the feature embedding. The proposed framework is compared to the state-of-the-art Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC) schemes standardized by the Moving Picture Experts Group (MPEG). The proposed method achieves more than 88% BD-Rate (Bjøntegaard Delta Rate) reduction against G-PCCv20 Octree, more than 56% BD-Rate savings against G-PCCv20 Trisoup, more than 62% BD-Rate reduction against V-PCC intra-frame encoding mode, and more than 52% BD-Rate savings against V-PCC P-frame-based inter-frame encoding mode using HEVC. These significant performance gains are cross-checked and verified in the MPEG working group. [ABSTRACT FROM AUTHOR]

Thanks to the many submissions we have received, we can present this second part of our special issue on "AI-eXplained." In here, we continue our mission to demystify the intricate world of artificial intelligence and make it accessible to a broader audience. As AI continues to evolve and integrate into various aspects of our lives, it becomes increasingly important to bridge the gap between experts and those eager to understand the inner workings of AI systems. Our goal remains unchanged—to present AI concepts in a comprehensible, engaging, and interactive manner, empowering our readers to explore and grasp the enchanting world of AI. [ABSTRACT FROM AUTHOR]

In this paper, we study how to learn a harmonious deep reinforcement learning (DRL) based lane-changing strategy for autonomous vehicles without Vehicle-to-Everything (V2X) communication support. The basic framework of this paper can be viewed as a multi-agent reinforcement learning in which different agents will exchange their strategies after each round of learning to reach a zero-sum game state. Unlike cooperation driving, harmonious driving only relies on individual vehicles’ limited sensing results to balance overall and individual efficiency. Specifically, we propose a well-designed reward that combines individual efficiency with overall efficiency for harmony, instead of only emphasizing individual interests like competitive strategy. Testing results show that competitive strategy often leads to selfish lane change behaviors, anarchy of crowd, and thus the degeneration of traffic efficiency. In contrast, the proposed harmonious strategy can promote traffic efficiency in both free flow and traffic jam than the competitive strategy. This interesting finding indicates that we should take care of the reward setting for reinforcement learning-based AI robots (e.g., automated vehicles) design, when the utilities of these robots are not strictly in alignment. [ABSTRACT FROM AUTHOR]

This paper proposes an end-to-end trainable one-stage parking slot detection method for around view monitor (AVM) images. The proposed method simultaneously acquires global information (entrance, type, and occupancy of parking slot) and local information (location and orientation of junction) by using a convolutional neural network (CNN), and integrates them to detect parking slots with their properties. This method divides an AVM image into a grid and performs a CNN-based feature extraction. For each cell of the grid, the global and local information of the parking slot is obtained by applying convolution filters to the extracted feature map. Final detection results are produced by integrating the global and local information of the parking slot through non-maximum suppression (NMS). Since the proposed method obtains most of the information of the parking slot using a fully convolutional network without a region proposal stage, it is an end-to-end trainable one-stage detector. In experiments, this method was quantitatively evaluated using the public dataset (PS2.0) and outperforms previous methods by showing both recall and precision of 99.77%, type classification accuracy of 100%, and occupancy classification accuracy of 99.31% while processing 60 frames per second. [ABSTRACT FROM AUTHOR]

Ride-sourcing services are now reshaping the way people travel by effectively connecting drivers and passengers through mobile internets. Online matching between idle drivers and waiting passengers is one of the most key components in a ride-sourcing system. The average pickup distance or time is an important measurement of system efficiency since it affects both passengers’ waiting time and drivers’ utilization rate. It is naturally expected that a more effective bipartite matching (with smaller average pickup time) can be implemented if the platform accumulates more idle drivers and waiting passengers in the matching pool. A specific passenger request can also benefit from a delayed matching since he/she may be matched with closer idle drivers after waiting for a few seconds. Motivated by the potential benefits of delayed matching, this paper establishes a two-stage framework which incorporates a combinatorial optimization and multi-agent deep reinforcement learning methods. The multi-agent reinforcement learning methods are used to dynamically determine the delayed time for each passenger request (or the time at which each request enters the matching pool), while the combinatorial optimization conducts an optimal bipartite matching between idle drivers and waiting passengers in the matching pool. Four tailored reinforcement learning methods, delayed multi-agent deep Q learning (Delayed-M-DQN), delayed multi-agent actor-critic (Delayed-M-A2C), delayed multi-agent Proximal Policy Optimization (Delayed-M-PPO), and delayed multi-agent actor-critic with experience replay (Delayed-M-ACER), are developed. Through extensive empirical experiments with a well-designed simulator, we show that the proposed framework is able to remarkably improve system performances, by well balancing the trade-off among pick-up time, matching time, successful matching rate. [ABSTRACT FROM AUTHOR]

With the increasing demand for location-based services in indoor environments, research on indoor localization technology has become an urgent need. In complex indoor environments, strong multipath effects can greatly interfere with the propagation of wireless signals, which may seriously degrade localization accuracy. To solve the problem, this paper proposes an indoor localization system DuLoc based on fine-grained subcarriers and a dual-channel convolutional neural network (CNN). In this system, abnormal sampling points are removed by the Isolation Forest method to deal with environmental noise. In order to select more stable and effective signals, the number of clusters of each subcarrier is analyzed by the Mean Shift clustering method, and the fingerprint database is divided into two sub-databases according to the threshold. The dual-channel CNN model is constructed to optimize the classification results with the data characteristics. Those two types of subcarrier data construct the input feature images to train the dual-channel CNN model. Then the final location coordinate is calculated by the probability weighted centroid method. Experimental results are presented to demonstrate that DuLoc has higher localization accuracy in complex indoor environments compared with the other existing methods. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have drawn considerable attention for the analysis of synthetic aperture radar (SAR) data. In this study, major subareas of SAR data analysis that have been tackled by CNNs are systematically reviewed, such as automatic target recognition, land use and land cover classification, segmentation, change detection, object detection, and image denoising. Special emphasis has been given to practical techniques such as data augmentation and transfer learning. Complex-valued CNNs, which have been introduced to exploit phase information embedded in SAR complex images, have also been extensively reviewed. To conclude this review paper, open challenges and future research directions are highlighted. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal signal detection problem with a particular interest in large-scale multiple-input multiple-output (MIMO) systems. The problem is NP-hard and can be solved optimally by searching the shortest path on the decision tree. Unfortunately, the existing optimal search algorithms often involve prohibitively high complexities, which indicates that they are infeasible in large-scale MIMO systems. To address this issue, we propose a general heuristic search algorithm, namely, hyper-accelerated tree search (HATS) algorithm. The proposed algorithm employs a deep neural network (DNN) to estimate the optimal heuristic, and then use the estimated heuristic to speed up the underlying memory-bounded search algorithm. This idea is inspired by the fact that the underlying heuristic search algorithm reaches the optimal efficiency with the optimal heuristic function. Simulation results show that the proposed algorithm reaches almost the optimal bit error rate (BER) performance in large-scale systems, while the memory size can be bounded. In the meanwhile, it visits nearly the fewest tree nodes. This indicates that the proposed algorithm reaches almost the optimal efficiency in practical scenarios, and thereby it is applicable for large-scale systems. Besides, the code for this paper is available at https://github.com/skypitcher/hats. [ABSTRACT FROM AUTHOR]

Identifying vehicle maneuvers in the context of Connected Vehicles (CV) system brings huge potentials to enhance traffic safety. However, this process requires various advanced sensors, which are either available for luxury vehicles or expensive to install. Differently, smartphone is a more feasible choice with high penetration rate and various built-in sensors. Among the existing studies of applying the smartphone to detect vehicle maneuvers, most treated the detection as a classification problem without considering the real-world application. For example, the smartphone was fixed. Too many descriptive features were generated from the sensor data. To alleviate these problems, this paper developed a vehicle maneuvers detection system using a common smartphone with GPS, gyroscope, accelerometer, and magnetometer sensors. We first released the constraints on the smartphone’s position through a coordination system reorientation method. Then, simply filtered sensor data were directly used. A stacked-LSTM model was built to detect the vehicle maneuvers considering the time-dependency of the sensor data. This paper compared the performance of the proposed system with previous studies and various machine learning methods, including LightGBM, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forest. Extensive experimental results indicated that the proposed system accurately detected different vehicle maneuvers with an average F1-score of 0.98, precision of 0.97, and recall of 0.98, which outperformed the counterparts. Moreover, the model can be easily transferred to different drivers and locations. The system is robust and suitable for the real-time application as it requires simple processing of smartphone sensor data. [ABSTRACT FROM AUTHOR]

Sea state estimation is beneficial for marine systems to enhance on-board decision-making and improve work efficiency. In the era of ship intelligence, artificial intelligence has greatly promoted the technology of sensing environment, such as by using the deep learning. However, it is difficult to collect enough motion data from a marine system to train a deep learning model. In addition, the model for sea state estimation is trained using the data from a specific marine system; applying the model directly to another marine system may result in performance degradation. In this paper, a supervised transfer learning based framework for sea state estimation (STLSSE) is proposed. The STLSSE focuses on knowledge transfer when the collected data for the source marine system is sufficient but the collected data of the target marine system is scarce. In STLSSE, a data pairing algorithm is proposed to determine the relationship of the source and the target marine system. Based on these paired data, a Siamese convolutional neural network, including a new proposed residual fully convolutional network and two novel attention modules, is designed for the semantic alignment. Moreover, the conventional contrastive loss is improved to characterize the distributions when there are only few samples in the target marine system. The extensive comparisons between STLSSE and state-of-the-art transfer learning approaches show its superior performance. The comparisons with state-of-the-art attention modules has verified the competitiveness of the proposed attention modules. The key parameters and each component of STLSSE are emphasized in the ablation and sensitivity studies. [ABSTRACT FROM AUTHOR]

With the rapid growth of the number of flights, the traditional radar system has been unable to meet the needs of flight supervision. At the same time, it also puts forward higher requirements for air traffic management (ATM). Automatic dependent surveillance-broadcast (ADS-B) is a promising technology in the next generation of air traffic control (ATC). However, the openness of ADS-B system brings the opportunities for terrorists to tamper with data. In this paper, we propose a novel aircraft coordinate prediction hybrid model based on deep learning. The proposed model combines inception modules and long short-term memory (LSTM) modules. Inception modules are used to extract the spatial features of dataset, and LSTM modules are used to extract the temporal features of dataset. In addition, we use the ADS-B signal strength instead of its specific information to obtain aircraft coordinates. Signal strength is not easily tampered with, but it carries limited information. Therefore, this scheme sacrifices a certain precision for reliability. Inception modules and LSTM modules are combined in different ways to perform experiments on the real-world ADS-B datasets from OpenSky network. The experimental results show that the proposed 2-Inception-LSTM is the local optimal model. The prediction error is within 10 km. It can be suitable for situations where the positioning accuracy of aircraft coordinates is not pursued, but the positioning reliability must be guaranteed. [ABSTRACT FROM AUTHOR]

Recent years have seen an explosion in the demand for the Internet of Things (IoT). IoT makes any physical object smart by providing sensing capability. The physical objects are embedded with sensors that form a network of sensors. However, sensors are vulnerable to faults due to energy depletion, software failures, and hardware failures. The existing fault detection schemes put huge computational overhead on the battery-limited and low computational capacity sensor nodes that are subject to the premature death of the sensor nodes. Also, they suffer from poor fault detection accuracy and huge false alarm rate that significantly reduce the overall performance of the networks. In this paper, an energy-aware intelligent fault detection scheme is proposed for IoT-enabled wireless sensor networks that significantly improve fault detection accuracy and reduce false alarm rate. A novel 3-Tier hard fault detection mechanism is used for detecting hardware unit faults of the sensor nodes. Furthermore, an optimized deep learning mechanism is used for various soft fault detection that prevents premature death of sensor nodes. The paper mathematically analyses the proposed scheme in terms of energy consumption, time complexity, and message complexity. Extensive simulations show the enhanced performance of the proposed scheme compared with the state-of-the-art algorithms in terms of fault detection accuracy, false alarm rate, false-positive rate, energy consumption, and network lifetime. [ABSTRACT FROM AUTHOR]

This paper presents a solution to the video prediction problem based on the deep learning paradigm. Predicted frames generated by existing video prediction models are often blurry and have difficulty maintaining accuracy in multi-step prediction. To overcome these limitations, this paper presents a deep learning model, named the deep pixel restoration loop coding network (DPR-LC-Net), which employs the concept of predictive coding and adopts pixels in the real frames. While making a long-term prediction, it can generate clear prediction frames with few errors. The calculation process of DPR-LC-Net is multi-sequential: it completes calculations in the form of an approximate loop from top to bottom and then from left to right. After predicting subsequent steps, DPR-LC-Net both calculates errors and removes them from sequential prediction. Finally, the model includes a unique pixel restoration module that works efficiently on pixels in the preceding real frames to generate predicted frames, thereby improving the clarity of the predicted frames. Extensive experiments using four video datasets illustrate that the prediction performance of DPR-LC-Net is superior to that of the state-of-the-art models. [ABSTRACT FROM AUTHOR]

The papers in this special issue focus on deep learning for image/video restoration and compression. The huge success of deep-learning–based approaches in computer vision has inspired research in learned solutions to classic image/video processing problems, such as denoising, deblurring, dehazing, deraining, super-resolution (SR), and compression. Hence, learning-based methods have emerged as a promising nonlinear signal-processing framework for image/ video restoration and compression. Recent works have shown that learned models can achieve significant performance gains, especially in terms of perceptual quality measures, over traditional methods. Hence, the state of the art in image restoration and compression is getting redefined. This special issue covers the state of the art in learned image/video restoration and compression to promote further progress in innovative architectures and training methods for effective and efficient networks for image/video restoration and compression. [ABSTRACT FROM AUTHOR]

January-March2021 PublishedbytheIEEEComputerSociety IEEEPervasiveComputing 45 ACTIVITYRECOGNITION The field of activity recognition using wearable sensing and computing platforms and intelligent sensor data analysis methods has a long standing within the wearable computing research community. Hoelzemann et al. 7 analyzed how transfer learning, another methodforeconomicuseofexistingdatasetsandpretrained classifiers,canor cannot beused for wearable computing settings. EDITORS:OliverAmft,oliver.amft@fau.de KristofVanLaerhoven,kvl@eti.uni-siegen.de DEPARTMENT:WEARABLECOMPUTING ISWC2020 AGathertownofCyborgs ThomasPl€ otz,GeorgiaInstituteofTechnology,Atlanta,GA30332,USA JenniferHealey,AdobeResearch,SanJose,CA95110-2704,USA KristofVanLaerhoven,UbiquitousComputing,UniversityofSiegen,57076Siegen,Germany T he International Symposium on Wearable Computers (ISWC) is the flagship conference on wearable computing focusing on design, algorithmic foundations, and deployments. Given the prevalence of supervised machine learning techniques for deriving activity recognition systems, researchers have become creative in how to bootstrap even large deep neural network based recognition systems when only smallamountsoflabeledsampledataareavailable. [Extracted from the article]

Accurate breast cancer detection using automated algorithms remains a problem within the literature. Although a plethora of work has tried to address this issue, an exact solution is yet to be found. This problem is further exacerbated by the fact that most of the existing datasets are imbalanced, i.e., the number of instances of a particular class far exceeds that of the others. In this paper, we propose a framework based on the notion of transfer learning to address this issue and focus our efforts on histopathological and imbalanced image classification. We use the popular VGG-19 as the base model and complement it with several state-of-the-art techniques to improve the overall performance of the system. With the ImageNet dataset taken as the source domain, we apply the learned knowledge in the target domain consisting of histopathological images. With experimentation performed on a large-scale dataset consisting of 277,524 images, we show that the framework proposed in this paper gives superior performance than those available in the existing literature. Through numerical simulations conducted on a supercomputer, we also present guidelines for work in transfer learning and imbalanced image classification. [ABSTRACT FROM AUTHOR]

This paper investigates the feasibility of novel data-driven deep learning (DL) models to predict the solution of Maxwell’s equations for low-frequency electromagnetic (EM) devices. With ground truth (empirical evidence) data being generated from a finite-element analysis solver, a deep convolutional neural network is trained in a supervised manner to learn a mapping for magnetic field distribution for topologies of different complexities of geometry, material, and excitation, including a simple coil, a transformer, and a permanent magnet motor. Preliminary experiments show DL model predictions in close agreement with the ground truth. A probabilistic model is introduced to improve the accuracy and to quantify the uncertainty in the prediction, based on Monte Carlo dropout. This paper establishes a basis for a fast and generalizable data-driven model used in the analysis, design, and optimization of EM devices. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

Vehicle-type classification is considered a core module for many intelligent transportation applications, such as speed monitoring, smart parking systems, and traffic analysis. In this paper, many vision-based classification techniques were presented relying only on a digital camera without the need for any extra hardware components. Dimension and color are two important characteristics of any digital image that affect the cost of the digital camera used in the image acquisition. In this paper, we present a comprehensive study of the effect of these two characteristics on the vehicle classification process in terms of accuracy and performance. We apply a set of different state-of-the-art image classifiers to the BIT-Vehicle and LabelMe data sets. Each data set is downscaled into different scales to generate a variety of spatial resolutions of each data set. Besides, we examine the effect of color by converting each color version to a gray-scale one. At last, we draw a valid conclusion in regards to the impact of these two characteristics (i.e., dimension and color) on the classification accuracy and performance of the image classification methods using more than 46 000 individual experiments. Experimental results show that there is no significant influence of both color and spatial resolutions of the vehicle images on the classification results obtained by most state-of-the-art image classification methods. However, there is a correlation between the spatial resolution and the processing time required by most image classification methods. Our findings can play an important role in saving not only money, but also time for vehicle-type classification systems. [ABSTRACT FROM AUTHOR]

This paper proposes a smartphone-based Driving Data Recorder (DDR). The proposed DDR has the functions of accurate speed estimation and intelligent traffic scene understanding. DDRs are used to store the relevant driving data to provide feedback on driver behavior for accident analysis, insurance issue, and so on. The conventional DDRs are standalone devices with multiple sensors. The current DDR products record many useless data or lose important information. On the other hand, the widely used smartphones already have the hardware conditions to replace the conventional DDR products. This paper proposes to develop the intelligent DDR in the smartphones. Considering the requirements of the DDRs, two functions are developed in this paper: motion sensor-based speedometer and vision sensor-based scene understanding. The proposed speedometer function adopts double-layered Long Short-Term Memory (LSTM) network as the model, which can estimate the vehicle speed directly from gyroscope and accelerometer of a smartphone. The scene understanding function can detect road facilities such as traffic lights, crosswalks, and stop lines. The driving data recorded in those areas are very important for analyzing driver behaviors. In the development of the scene understanding function, maintaining high detection accuracy with reduced computation cost is significant due to the limitation of smartphones’ processing resources. This paper uses a lightweight architecture deep learning network to achieve the goal. The proposed system has been evaluated using the real traffic data. Speed estimation function only has 1.8 km/h of speed mean error. In addition, there is no accumulated error even for a long time driving. The evaluation of the scene understanding function indicates that the proposed method can provide a high-accuracy detection at 2 FPS, which is faster than the state-of-the-art method. [ABSTRACT FROM AUTHOR]

Electrical capacitance tomography exhibits great potentials in the visualization measurement of industrial processes, and high-precision images are of great significance for the reliability and usefulness of measurement results. In this paper, we propose a deep learning-based inversion method to ameliorate the reconstruction accuracy. With the aid of the deep learning methodology, the prior from the images reconstructed by a certain imaging technique to the true images is abstracted and stored in the deep extreme learning machine. A new cost function is constructed to encapsulate the prior from the proposed deep learning model and the domain expertise about imaging targets, and the split Bregman algorithm and the fast iterative shrinkage thresholding technique are combined into a new numerical method to effectively solve it to get the final reconstruction. The numerical and experimental results validate that the inversion method proposed in this paper reduces the reconstruction artifacts and deformations and leads to the much improvement in the imaging quality. [ABSTRACT FROM AUTHOR]

A comprehensive and comprehensible summary of existing deep neural networks (DNNs) helps practitioners understand the behaviour and evolution of DNNs, offers insights for architecture optimization, and sheds light on the working mechanisms of DNNs. However, this summary is hard to obtain because of the complexity and diversity of DNN architectures. To address this issue, we develop DNN Genealogy, an interactive visualization tool, to offer a visual summary of representative DNNs and their evolutionary relationships. DNN Genealogy enables users to learn DNNs from multiple aspects, including architecture, performance, and evolutionary relationships. Central to this tool is a systematic analysis and visualization of 66 representative DNNs based on our analysis of 140 papers. A directed acyclic graph is used to illustrate the evolutionary relationships among these DNNs and highlight the representative DNNs. A focus + context visualization is developed to orient users during their exploration. A set of network glyphs is used in the graph to facilitate the understanding and comparing of DNNs in the context of the evolution. Case studies demonstrate that DNN Genealogy provides helpful guidance in understanding, applying, and optimizing DNNs. DNN Genealogy is extensible and will continue to be updated to reflect future advances in DNNs. [ABSTRACT FROM AUTHOR]

An editorial is presented on addressing the need for bringing together leading efforts in non-Euclidean deep learning across all communities. Topics include Convolutional Neural Networks (CNNs) relying on classical signal processing models limiting the applicability to data with underlying Euclidean grid-like structure; and multiple levels of abstraction being introduced similarly to multi-layer networks.

This paper proposes a deep reinforcement learning-based video streaming scheme for mobility-aware vehicular networks, e.g., vehicles on the highway. We consider infrastructure-assisted and mmWave-based scenarios in which the macro base station (MBS) cannot directly provide the streaming service to vehicles due to the short range of mmWave beams so that small mmWave base stations (mBSs) along the road deliver the desired videos to users. Motivated by the fact that a video stream consists of sequential chunks and smooth streaming has to be provided while users pass through multiple mBSs at high speed, the MBS proactively pushes some video chunks of the desired contents to mBSs. This is done to support vehicles that are currently covered and/or will be by each mBS. We formulate the dynamic video delivery scheme that adaptively determines 1) which content, 2) what quality and 3) how many chunks to be proactively delivered from the MBS to mBSs using Markov decision process (MDP). Since it is difficult for the MBS to track all the channel conditions and the network states have extensive dimensions, we adopt the deep deterministic policy gradient (DDPG) algorithm for the DRL-based video delivery scheme. This paper finally shows that the DRL agentlearns a streaming policy that pursues high average quality while limiting packet drops, avoiding playback stalls, reducing quality fluctuations and saving backhaul usage. [ABSTRACT FROM AUTHOR]

Automatic feature extraction is one of the most advantageous merits of deep neural network (DNN), meanwhile, it is an important part for intelligent bearing fault diagnosis. However, most of fault diagnosis methods based on DNN usually excavate the complex relations from original time sequence signals which only present the fault information in time domain. Convolutional Neural Network (CNN) has demonstrated powerful feature learning capabilities in bearing fault diagnosis and the deeper the diagnosis model is, the better the recognition performance is, which resulted in some problems. In order to enrich the fault information from different views and enhance the discrimination for features learned from diagnosis network, this paper proposed a bearing fault diagnosis method based on multi-domain information fusion and improved residual dense network. The original signal and its transformed signals composed the multi-channel input, which contained more comprehensive information and will benefit the deep learning. Then it designed a residual dense network and introduced the convolution attention mechanism which can discriminate the importance of features further improve the feature extraction capability and efficiency of diagnosis network. Finally, it achieved the fault classification, analyzed the effects of key parameters and compared with other diagnosis to verify the effectiveness by lots of experimental results. [ABSTRACT FROM AUTHOR]

This paper investigates an air-ground integrated multi-access edge computing system, which is deployed by an infrastructure provider (InP). Under a business agreement with the InP, a third-party service provider provides computing services to the subscribed mobile users (MUs). MUs compete for the shared spectrum and computing resources over time to achieve their distinctive goals. From the perspective of an MU, we deliberately define the age of update to capture the staleness of information from refreshing computation outcomes. Given the system dynamics, we model the interactions among MUs as a stochastic game. In the Nash equilibrium without cooperation, each MU behaves in accordance with the local system states and conjectures. We can hence transform the stochastic game into a single-agent Markov decision process. As another major contribution, we develop an online deep reinforcement learning (RL) scheme that adopts two separate double deep Q-networks to approximate the Q-factor and the post-decision Q-factor, respectively. The deep RL scheme allows each MU to optimize the behaviours with unknown dynamic statistics. Numerical experiments show that our proposed scheme outperforms the baselines in terms of the average utility under various system conditions. [ABSTRACT FROM AUTHOR]