Search

• The paper fills the gap in the literature by reviewing and analyzing the current studies that apply deep learning for Covid-19 forecasting. • The initial search identified 152 studies of which 53 passed the quality control. • The existing literature is categorized using a model-based taxonomy. • The description of the models along with their performance evaluation is presented. • Recommendations for future improvements are provided. The Covid-19 pandemic has galvanized scientists to apply machine learning methods to help combat the crisis. Despite the significant amount of research there exists no comprehensive survey devoted specifically to examining deep learning methods for Covid-19 forecasting. In this paper, we fill the gap in the literature by reviewing and analyzing the current studies that use deep learning for Covid-19 forecasting. In our review, all published papers and preprints, discoverable through Google Scholar, for the period from Apr 1, 2020 to Feb 20, 2022 which describe deep learning approaches to forecasting Covid-19 were considered. Our search identified 152 studies, of which 53 passed the initial quality screening and were included in our survey. We propose a model-based taxonomy to categorize the literature. We describe each model and highlight its performance. Finally, the deficiencies of the existing approaches are identified and the necessary improvements for future research are elucidated. The study provides a gateway for researchers who are interested in forecasting Covid-19 using deep learning. [ABSTRACT FROM AUTHOR]

Foreign species can deteriorate the environment and the economy of a country. To automatically monitor biosecurity threats at country borders, this paper investigates compact deep networks for accurate and real-time object segmentation for hyperspectral images. To this end, knowledge distillation (KD) approaches compress the model by distilling the knowledge of a large teacher network to a compact student network. However, when the student is over-compressed, the performance of standard KD methods degrades significantly due to the large capacity gap between the teacher and the student. This gap can be addressed by adding medium-sized teacher assistants, but training them incurs significant computation and hence is impractical. To address this problem, this paper proposes a new framework called Knowledge Distillation from Multi-head Teacher (KDM), which distills the knowledge of a multi-head teacher to the student. By encapsulating multiple teachers in a single network, our proposed KDM assists the learning of a very compact student and significantly reduces the training time. We also introduce Bio-HSI, a new large benchmark hyperspectral image dataset of 3,125 high-resolution images with dense segmentation ground truth. This new, large dataset can be expected to advance research on deep models for hyperspectral image segmentation. Evaluated on this dataset, the student trained via our KDM has 762 times fewer parameters than the state-of-the-art segmentation model (i.e., HRNet), while achieving competitive accuracy. [ABSTRACT FROM AUTHOR]

This paper presents an effective weakly supervised learning algorithm LLP-AAE to leverage the adversarial autoencoder (AAE) for learning from label proportions (LLP), in which only the bag-level proportional information is available. Our LLP-AAE utilizes an autoencoder backbone and performs adversarial training in latent space to match the aggregated posterior distribution of hidden coding with the prior distributions. In this way, apart from the reconstruction task, the encoder is also dedicated to producing fake samples, in order to deceive discriminators as far as possible. Ultimately, the encoder is employed as a competent label predictor for unseen data. In addition to the LLP classifier, our model can also achieve controllable samples generation by feeding the decoder with gradually changing latent code, which is proven to be useful for a better LLP performance. We also provide a panoramic explanation for LLP-AAE by regarding the LLP problem as an alternative learning procedure between proportion-based pseudo label generation and discriminative reconstruction. Experiments on six benchmark image datasets demonstrate the advantage of our method both in style manipulation with the latent feature representation and comparable multi-class LLP performance with the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Recently, many efforts have been concentrated on land use land cover (LULC) classification due to rapid urbanization, environmental pollution, agriculture drought, frequent floods, and climate change. However, various aspects have attracted hyperspectral imaging due to there being informative discriminative features, such as spectral-spatial features. To this end, this paper is a comprehensive and systematic review of LULC classification using hyperspectral images by reviewing four significant research investigations. Moreover, the four investigations have addressed the following points: (1) the main components of the hyperspectral imaging, the modes of hyperspectral imaging with data acquisition methods, and the intrinsic differences between hyperspectral image and multispectral image, (2) the role of machine learning in LULC classification, and the standard deep learning methods: Convolution Neural Network (CNN), Stacked Autoencoder (SAE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), and Generative Adversarial Network (GAN), (3) the standard benchmark hyperspectral datasets and the evaluation criteria, (4) the main challenges of LULC classification with the possible solutions for the limited training samples issue, the promising future directions, and finally the recent applications for LULC classification. [ABSTRACT FROM AUTHOR]

The performance of deep learning-based methods depends mainly on the availability of large-scale labeled learning data. However, obtaining precisely annotated examples is challenging in the medical domain. Although some semi-supervised deep learning methods have been proposed to train models with fewer labels, only a few studies have focused on the uncertainty caused by the low quality of the images and the lack of annotations. This paper addresses the above issues using Dempster-Shafer theory and deep learning: 1) a semi-supervised learning algorithm is proposed based on an image transformation strategy; 2) a probabilistic deep neural network and an evidential neural network are used in parallel to provide two sources of segmentation evidence; 3) Dempster's rule is used to combine the two pieces of evidence and reach a final segmentation result. Results from a series of experiments on the BraTS2019 brain tumor dataset show that our framework achieves promising results when only some training data are labeled. [ABSTRACT FROM AUTHOR]

Coronary artery disease is among one of the diseases human suffer most. Intravascular coronary arterial image analysis consists of denoising, segmentation, detection, and three-dimensional reconstruction, having a significant meaning for auxiliary diagnosis and treatment of coronary artery disease. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are the two most commonly applied intravascular coronary arterial imaging techniques. Based on these fundamental imaging techniques, in recent years, many advanced technologies from traditional machine learning algorithms to deep learning methods were employed in the analysis of intravascular coronary arterial images and made huge progress in this field. In this survey, we reviewed more than one hundred papers published in top journals or conferences such as Neural Networks and MICCAI. These papers proposed approaches or schemes for the intravascular coronary arterial image analysis, including lumen border segmentation, atherosclerotic plaque characterization, media-adventitia segmentation, stent strut detection, and three-dimensional reconstruction. Our survey began with introducing coronary artery intravascular imaging techniques, essential neural networks, and deep learning and then presented an across-the-board review of methods, applications, and trends of intravascular image analysis. This survey is more comprehensive than other articles not only for its scope and reference number but also for discussing the future direction in this field. Compared to other review papers in this field, this article could assist beginners in constructing a basic knowledge frame of coronary artery intravascular image analysis methods and brought state-of-the-art progress in this field to fellow researchers. We hope this paper could benefit either the beginners for coronary arterial image analysis or experienced researchers. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially in the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction–Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on large-scale classification and segmentation datasets, and our ARB-Loss can achieve state-of-the-art performance via only one-stage training instead of 2-stage learning like nowadays SOTA works. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has been shown to be very effective for many application domains of machine learning (ML), including image classification, voice recognition, natural language processing, and bioinformatics. The success of DL techniques is directly related to the availability of large amounts of training data. However, in many cases, the data are sensitive to the users and should be protected to preserve the privacy. Privacy-preserving deep learning (PPDL) has thus become a very active research field to ensure the training process and use of DL models are productive without exposing or leaking information about the data. This paper aims to provide a comprehensive survey of PPDL. We concentrate on the risks that affect data privacy in DL and conduct a detailed investigation into the models that ensure privacy. Finally, we propose a set of evaluation criteria, detailing the advantages and disadvantages of the solutions. Based on the analyzed strengths and weaknesses, the paper has highlighted some important research problems and application cases that have not been studied and these point to certain open research directions. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) has emerged as a powerful paradigm for training Machine Learning (ML), particularly Deep Learning (DL) models on multiple devices or servers while maintaining data localized at owners' sites. Without centralizing data, FL holds promise for scenarios where data integrity, privacy and security and are critical. However, this decentralized training process also opens up new avenues for opponents to launch unique attacks, where it has been becoming an urgent need to understand the vulnerabilities and corresponding defense mechanisms from a learning algorithm perspective. This review paper takes a comprehensive look at malicious attacks against FL, categorizing them from new perspectives on attack origins and targets, and providing insights into their methodology and impact. In this survey, we focus on threat models targeting the learning process of FL systems. Based on the source and target of the attack, we categorize existing threat models into four types, Data to Model (D2M), Model to Data (M2D), Model to Model (M2M) and composite attacks. For each attack type, we discuss the defense strategies proposed, highlighting their effectiveness, assumptions and potential areas for improvement. Defense strategies have evolved from using a singular metric to excluding malicious clients, to employing a multifaceted approach examining client models at various phases. In this survey paper, our research indicates that the to-learn data, the learning gradients, and the learned model at different stages all can be manipulated to initiate malicious attacks that range from undermining model performance, reconstructing private local data, and to inserting backdoors. We have also seen these threat are becoming more insidious. While earlier studies typically amplified malicious gradients, recent endeavors subtly alter the least significant weights in local models to bypass defense measures. This literature review provides a holistic understanding of the current FL threat landscape and highlights the importance of developing robust, efficient, and privacy-preserving defenses to ensure the safe and trusted adoption of FL in real-world applications. The categorized bibliography can be found at: https://github.com/Rand2AI/Awesome-Vulnerability-of-Federated-Learning. [ABSTRACT FROM AUTHOR]

Every day, enormous amounts of biomedical texts discussing various biomedical topics are produced. Revealing strong semantic connections hidden in those unstructured data is essential for many interesting applications such as knowledge base development for the biomedical domain as well as drug repurposing and drug–disease associations. Literature based discovery (LBD) is a well-known paradigm that refers to the issues of finding new hidden knowledge in scientific literature by connecting pieces of semantically-related information belonging to independent documents. This challenging research area has been extensively investigated by the research community and different proposals adopting natural language processing, text mining, machine learning and recently deep learning have been developed. This paper exploits a very focused task, it surveys a collection of research papers published in the recent years that have adopted Deep Learning for literature based discovery as an effective technique to discover new relationships between existing knowledge in biomedical domain. The study provides an analysis of the key characteristics of each work surveyed, including the Literature based discovery application area, the deep learning method used, the type of analyzed data, and the results obtained. Recognizing the significance of Pre-trained Language Models (PLMs), another primary aim of this paper is to offer an extensive overview of the latest developments in pre-trained language models within the field of biomedicine. This focus will primarily be on how they are applied to downstream tasks associated with Literature-Based Discovery in the biomedical domain. Additionally, the survey highlights the key drawbacks of the current state-of-the-art proposals, as well as the challenges that require further study by the research community. [ABSTRACT FROM AUTHOR]

Precision Agriculture (PA) promises to boost crop productivity while reducing agricultural costs and environmental footprints, and therefore is attracting ever-increasing interests in both academia and industry. This management strategy is underpinned by various advanced technologies including Unmanned Aerial Vehicle (UAV) sensing systems and Artificial Intelligence (AI) perception algorithms. In particular, due to their unique advantages such as a low cost, high spatio-temporal resolutions, flexibility, automation functions and minimized risk of operation, UAV sensing systems have been extensively applied in many civilian applications including PA since 2010. In parallel, AI algorithms (deep learning since 2012 in particular) are also drawing ever-increasing attention in different fields, since they are able to analyse an unprecedented volume/velocity/variety of data (semi-) automatically, which are also becoming computationally practical with the advancements of cloud computing, Graphics Processing Units and parallel computing. In this survey paper, therefore, a thorough review is performed on recent use of UAV sensing systems (e.g., UAV platforms, external sensing units) and AI algorithms (mainly supervised learning algorithms) in PA applications throughout the crop life-cycle, as well as the challenges and prospects for future development of UAVs and AI in agriculture sector. It is envisioned that this review is able to provide a timely technical reference, demystifying and promoting research, deployment and successful exploitation of AI empowered UAV perception systems for PA, and therefore contributing to addressing future agricultural and human nutrition challenges. [ABSTRACT FROM AUTHOR]

Low-light image enhancement studies how to improve the quality of images captured under poor lighting conditions, which is of real-world importance. Currently, convolutional neural network (CNN)-based methods with state-of-the-art performance have become the mainstream of research. However, most CNN-based methods improve the performance of the algorithm by increasing the width and depth of the neural network, which requires large computing device resources. In this paper, we propose a knowledge distillation method for low light image enhancement. The proposed method uses a teacher-student framework in which the teacher network tries to transfer the rich knowledge to the student network. The student network learns the knowledge of image enhancement under the supervision of ground truth images and under the guidance of the teacher network simultaneously. Knowledge transfer between the teacher-student network is accomplished by distillation loss based on attention maps. We designed a gradient-guided low-light image enhancement network that can be divided into an enhancement branch and a gradient branch, where the enhancement branch is learned under the guidance of the gradient branch to better preserve structural information. The teacher and student networks use a similar structure, but they have different model sizes. The teacher network has more parameters and more powerful learning capabilities than the student network. With the help of knowledge distillation, our approach can improve the performance of the student network without increasing the computational burden during the testing phase. The qualitative and quantitative experimental results demonstrate the superiority of our method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Stuttering is a speech disorder during which the flow of speech is interrupted by involuntary pauses and repetition of sounds. Stuttering identification is an interesting interdisciplinary domain research problem involving pathology, psychology, acoustics, and signal processing, making it hard and complicated to detect. Recent developments in the machine and deep learning have dramatically revolutionized the speech domain, however minimal attention has been given to stuttering identification. This work fills the gap by trying to bring researchers together from interdisciplinary fields. In this paper, we comprehensively review acoustic features, and statistical and deep learning-based stuttering/disfluency classification methods. We also present several challenges and possible future directions. [ABSTRACT FROM AUTHOR]

• We propose an automated layer-wise pipeline for randomized feed-forward neural networks. • We propose a Sample-level Attention Strategy to solve the capabilities problem. • Several ablation experiments are conducted to verify the effect of hyper-parameters. The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework's capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures. [ABSTRACT FROM AUTHOR]

The complex and varied urban road conditions have always been a difficult and a pivotal component in the study of driverless technology especially at intersections. In China, it is necessary for driverless cars to understand the gestures of traffic police. To identify the traffic police gesture at the intersection, the key step is to detect the traffic police at the intersection. At present, the research on traffic police detection is still in its infancy, there exists common problems such as slow detection speed and other real time problems in this method, and there is not a standardized traffic police data set ether. For the real time problems, this paper introduces the attention mechanism, and proposes a new real-time detection method of traffic police based on attention mechanism. The method proposed in this paper has strong robustness and can quickly complete the target detection task. For the data set problem, this paper analyzes and discloses similar data sets published through research. In the meanwhile, this paper collects 24,530 video data on the actual road, and it extracts and saves 12,000 pictures containing traffic police at frame rate of 30FPS as traffic police detection training and validated data set. [ABSTRACT FROM AUTHOR]

For NP-hard combinatorial optimization problems, it is usually challenging to find high-quality solutions in polynomial time. Designing either an exact algorithm or an approximate algorithm for these problems often requires significantly specialized knowledge. Recently, deep learning methods have provided new directions to solve such problems. In this paper, an end-to-end deep reinforcement learning framework is proposed to solve this type of combinatorial optimization problems. This framework can be applied to different problems with only slight changes of input, masks, and decoder context vectors. The proposed framework aims to improve the models in literacy in terms of the neural network model and the training algorithm. The solution quality of TSP and the CVRP up to 100 nodes are significantly improved via our framework. Compared with the best results of the state-of-the-art methods, the average optimality gap is reduced from 4.53% to 3.67% for TSP with 100 nodes and from 7.34% to 6.68% for CVRP with 100 nodes when using the greedy decoding strategy. Besides, the proposed framework can be used to solve a multi-depot CVRP case without any structural modification. Furthermore, our framework uses about 1/3 ∼ 3/4 training samples compared with other existing learning methods while achieving better results. The results performed on randomly generated instances, and the benchmark instances from TSPLIB and CVRPLIB confirm that our framework has a linear running time on the problem size (number of nodes) during training and testing phases and has a good generalization performance from random instance training to real-world instance testing. [ABSTRACT FROM AUTHOR]

Retinal optical coherence tomography (OCT) images provide fundamental information regarding the health of the posterior eye (e.g., the retina and choroid). Thus, the development of automatic image analysis methods (e.g., segmentation) is fundamental to provide clinicians and researchers with quantitative data that facilitates decision making. In recent years, various machine learning (ML) methods have been developed to perform these automated image analyses, improving performance over traditional methods, increasing repeatability, and reducing the use of time-consuming manual analysis. Deep learning (DL), a sub-field of ML, represents a new approach that can improve image processing outcomes. Results published to date demonstrate that DL architectures generally achieve superior performance to previously proposed methods based on traditional image analysis or early ML techniques. Thus, DL methods for OCT image analysis have provided an important advance in the area of retinal layer segmentation in images from healthy eyes as well as those with pathologies. This paper provides a comprehensive narrative literature review of current DL layer segmentation methods applied to OCT images of the posterior segment of the eye. The manuscript provides an overview on the state-of-the-art in deep learning as well as highlighting some important areas for future developments to extend the analysis methods in this field. [ABSTRACT FROM AUTHOR]

Recent research has shown that models induced by machine learning, and in particular by deep learning, can be easily fooled by an adversary who carefully crafts imperceptible, at least from the human perspective, or physically plausible modifications of the input data. This discovery gave birth to a new field of research, the adversarial machine learning, where new methods of attacks and defense are developed continuously, mimicking what is happening from a long time in cybersecurity. In this paper we will show that the drawbacks of inducing models from data less prone to be misled can actually provide some benefits when it comes to assessing their generalization abilities. We will show these benefits both from a theoretical perspective, using state-of-the-art statistical learning theory, and both with practical examples. [ABSTRACT FROM AUTHOR]

Recently, deep learning technology has achieved remarkable progress in compressed video quality enhancement. However, the existing methods fail to consider the fact that the regions with different compression losses contain varied effective information. To address this issue, this paper proposes a Compression Loss-based Spatial-Temporal Attention (CLSTA) module that predicts the compression loss-based attention of each pixel when restoring target frames. Based on this, the pixels with lower compression losses contribute more to restoring target frames. Meanwhile, the compression-domain information in the compressed video stream is exploited to overcome the difficulty in inferring compression loss-based attention directly from the pixel-domain. With a slight modification, the CLSTA module can be easily integrated into the existing methods for end-to-end training. A series of experiments have been conducted in this study to validate the effectiveness and generality of the CLSTA module for compressed video quality enhancement. By adding the CLSTA module to the state-of-the-art method STDF, the proposed method achieves an average PSNR improvement of 0.96 dB and a BD-rate reduction of 26.85% under the Low-Delay P configuration, which is respectively 0.06 dB and 1.83% better than the baseline STDF. In the best case, the proposed method obtains 0.19 dB higher PSNR than STDF for a single frame. [ABSTRACT FROM AUTHOR]

In continual learning over deep neural networks (DNNs), the rehearsal strategy , in which the previous exemplars are jointly trained with new samples, is commonly employed for the purpose of addressing catastrophic forgetting. Unfortunately, due to the memory limit, rehearsal-based techniques inevitably cause the class imbalance issue leading to a DNN biased toward new tasks having more samples. Existing works mostly focus on correcting such a bias in the fully connected layer or classifier. In this paper, we newly discover that class imbalance tends to make old classes even more highly correlated with their similar new classes in the feature space, which turns out to be the major reason behind catastrophic forgetting, called inter-task forgetting. To alleviate inter-task forgetting, we propose a novel class incremental learning method, called attractive & repulsive training (ART) , which effectively captures the previous feature space into a set of class-wise flags , and thereby makes old and new similar classes less correlated in the new feature space. In our empirical study, our ART method is observed to be quite effective to improve the performance of the state-of-the-art methods by substantially mitigating inter-task forgetting. Our implementation is available at: https://github.com/bigdata-inha/ART/. [ABSTRACT FROM AUTHOR]

Algorithmic bias indicates the discrimination caused by algorithms, which occurs with protected features such as gender and race. Many researchers have tried to define the fairness and devise methods to mitigate bias, but it is still premature to reach the unanimous definition and evaluation metrics of fairness according to society, times and cultures. In this paper, we introduce three evaluation metrics such as parity difference, equalized opportunity and equalized odds that can deal with various definitions of algorithmic bias, and concretely divide the three general approaches further into seven methods with some challenges, resulting in relabeling, generation, fair representation (for pre-processing), constraint optimization, regularization (for in-processing), calibration and thresholding (for post-processing). Among them, the pre-processing method is widely used due to its versatility, but it has limitation to deal with the information on data and features related with bias appropriately. In order to preserve the characteristics of the original data while excluding the information about the features causing bias, we propose a preprocessing approach based on information theory that avoids collision in the dual optimization, where the latent space is divided into two subspaces. Experiments are conducted with the well-known benchmark datasets of Census and COMPAS, and two real-world tasks: facial emotion recognition and text sentiment analysis. The information theoretic approach is promising to achieve fair machine learning by reducing the bias caused by several features such as age, race and gender. [ABSTRACT FROM AUTHOR]

[Display omitted] • Tackle the under-studied semi-supervised image regression problem to relief the heavy workload of manually annotation for regression tasks. • Applying manifold smoothness to image regression tasks that transfers the SSL problem to kernel learning tasks. • A novel and efficient algorithm, ManiDKL, learns the regression parameters in a Bayesian manner. • Outperforms all existing semi-supervised methods for image regression tasks. Semi-supervised learning (SSL) aims at utilizing the vast unlabeled data to help the supervised training. While existing SSL methods have shown promising results on image classification tasks, most of them rely on the cluster assumption that does not apply to image regression tasks. In this paper, we address the under-studied semi-supervised image regression problem, of which the outputs are continuous values instead of categorical distributions. To tackle this challenging task, we propose an algorithm, called ManiDKL, with the idea that the prediction function should be smooth with respect to the intrinsic manifold of data distribution and behave similarly on both labeled and unlabeled data. In particular, we propose a framework that implements the Tikhonov regularization with generative manifold learning to ensure manifold smoothness of regression function and also reduces the problem to kernel learning. Then a semi-supervised non-parametric Bayesian based deep kernel learning algorithm is proposed, in which unlabeled data are incorporated through posterior regularization. We show the effectiveness of ManiDKL with extensive experiments. It shows that ManiDKL performs comparatively with state-of-the-art SSL image classification methods. Most importantly, we show the superiority of ManiDKL over all existing SSL regression methods on public image datasets. [ABSTRACT FROM AUTHOR]

Effective animal intelligent monitoring is of great value in terms of ecological protection and endangered specie conservation. At present, computer vision technologies have shed light on animal intelligent monitoring. Especially, numerous deep learning-based models and methods have been developed to address various challenges, and have made substantial strides in this field. However, there are still several problems to be solved and related areas to be mined, such as exploring new strategies to enhance the robustness and generalization ability of models, designing novel models for complex environments, and establishing large-scale publicly available animal datasets for performance verification of models. Therefore, we comprehensively elaborated and analyzed existing works on animal intelligent monitoring based on advanced information science technologies, so as to provide useful information assistance for relevant researchers. In this paper, we focus on three primary task fields: precise animal localization, tracking and individual identification. Specifically, we elucidate the definition and significance of each monitoring task, and summarize the baseline models for addressing different problems. We provide a specific analysis of strategies and prototypes of the models and methods employed in each tasks following by the technical progression from traditional machine learning to deep learning. In addition, we make a comparison and analysis of the relevant methods, summarize their similarities and differences between them, and point out the advantages and disadvantages of these methods. Finally, we present several unresolved challenges and problems in animal intelligent monitoring and provide potential research directions in the future. We expect that our review can serve as reference and guidance for related research fields. [ABSTRACT FROM AUTHOR]

Neural networks and deep learning algorithms are designed to function similarly to biological synaptic structures. However, classical deep learning algorithms fail to fully capture the need for continuous learning; this has led to the advent of incremental learning. Incremental learning adds new challenges that are handled differently by modern state-of-the-art approaches. Some of these include: utilization of network memory as additional knowledge increases the size of the network, open-set recognition to be able to identify unrecognized information, and efficient knowledge distillation as most incremental learning algorithms are prone to catastrophic forgetting of previously learned knowledge. Recent advancements achieve incremental learning through a multitude of methods. Most methods are characterized by augmenting the normal algorithm of neural network training by both directly modifying the neural network structure and by adding additional learning steps. This paper analyzes and provides a comprehensive survey of existing methods and various techniques used for incremental learning. A novel categorization of the methods is also introduced based on recent trends of the state-of-the-art solutions. The study focuses on methods that provide incremental learning success as well as discusses emerging patterns in new research. [ABSTRACT FROM AUTHOR]

Image classification, which is a fundamental element of Computer Vision (CV) and Artificial Intelligence (AI), has been researched intensively in numerous domains and embedded in many products. However, with the exponential increase in the number of images and the complexity of the required tasks, deep-learning classification algorithms demand more intensive resources and computational power to train the models and update the massive number of parameters. Quantum computing is a new research technology with a promising capability of exponential speed up and operational parallelization with its unique phenomena including superposition and entanglement. Researchers have already started utilizing Quantum Deep Learning (QDL) and Quantum Machine Learning (QML) in image classification. Yet, to our knowledge, there exists no comprehensive published literature review on quantum image classification. Therefore, this paper analyzes the advances in this field by dividing the studies based on a unique taxonomy, discussing the limitations, summarizing essential aspects of each research, and finally, emphasizing the gaps, challenges, and recommendations. One of the key challenges presented in the paper is that quantum computers are in the Noisy Intermediate-Scale Quantum (NISQ) era, where they contain a limited number of noisy qubits, therefore challenging complex quantum classifiers and complex images from advanced datasets. This research recommends constructing a novel quantum image encoding method that adapts to the available number of qubits and enables RGB images as a critical contribution to the existing research. [ABSTRACT FROM AUTHOR]

Developing a wireless indoor positioning system with high accuracy, reliability, and reasonable cost has been the focus of many researchers. Recent studies have shown that visible-light-based positioning (VLP) systems have better positioning accuracy than radio-frequency-based systems. A notable highlight of those research articles is their combination of VLP and machine learning (ML) to improve the positioning performance in both two-dimensional and three-dimensional spaces. In this paper, in addition to describing VLP systems and well-known positioning algorithms, we analyze, evaluate, and summarize the ML techniques that have been applied recently. We break these into four categories: supervised learning, unsupervised learning, reinforcement, and deep learning. We also provide deep discussion of articles published during the past five years in terms of their proposed algorithm, space (2D/3D), experimental method (simulation/experiment), positioning accuracy, type of collected data, type of optical receiver, and number of transmitters. [ABSTRACT FROM AUTHOR]

Smart home applications are pervasive and have gained popularity due to the overwhelming use of Internet of Things (IoT). The revolution in IoT technologies made homes more convenient, efficient and perhaps more secure. The need to advance smart home technology is necessary at this stage as IoT is abundantly used in automation industry. However, most of the proposed solutions are lacking in certain key areas of the system i.e., high interoperability, data independence, privacy, and optimization in general. The use of machine learning algorithms requires high-end hardware and are usually deployed on servers, where computation is convenient, but at the cost of bandwidth. However, more recently edge AI enabled systems are being proposed to shift the computation burden from the server side to the client side enabling smart devices. In this paper, we take advantage of the edge AI enabled technology to propose a fully featured cohesive system for smart home based on IoT and edge computing. The proposed system makes use of industry standards adopted for fog computing as well as providing robust responses from connected IoT sensors in a typical smart home. The proposed system employs edge devices as a computational platform in terms of reducing energy costs and provides security, while remotely controlling all appliances behind a secure gateway. A case study of human fall detection is evaluated by a custom lightweight deep neural network architecture implemented over the edge device of the proposed framework. The case study was validated using the Le2i dataset. During the training, the early stopping threshold was achieved with 98% accuracy for training set and 94% for validation set. The model size of the network was 6.4 MB which is significantly lower than other networks with similar performance. [ABSTRACT FROM AUTHOR]

With the rapid development of Internet technology and the explosive growth of digital text, opinion mining has become one of the important research hotspots in the field of natural language processing (NLP). In recent years, neural network based deep learning algorithms have been applied in the field of opinion mining. Considering the relation between temporal and spatial dimensions of text data and the characteristics of natural language itself, traditional deep learning algorithms cannot be comprehensive in the processing of fully feature extraction. In this paper, we propose a new deep learning framework for opinion mining, which includes a temporal feature extraction layer that consists of two layers of bidirectional simple recurrent unit (Bi-SRU) networks extracting features at the word and grammar levels; a semantic feature extraction layer that mainly contains a multi-head attention module; a spatial feature extraction layer with dilated convolution that is used to extract opinion preference features. The Internet movie database (IMDb) is used to verify the performance of the proposed framework. The experiment results show that the proposed framework can effectively improve the classification accuracy, whose performance is better than that of the compared algorithms. [ABSTRACT FROM AUTHOR]

[Display omitted] • A steering angle prediction method is proposed for autonomous ego-vehicles, based on an end-to-end deep learning algorithm. • A temporal and spatial attention block is designed to better capture long- and short-term memory and visual information. • The experimental results performed on the comma2k19 dataset are better than the baselines. • An attention mechanism is visualized to explain the trigger mechanism of steering behavior prediction. In recent years, autonomous vehicles have attracted many researchers from academia and industry. Their efforts include object detection and tracking in the fields of computer vision, probabilistic-fusion and decision-making algorithms, etc. But most current methods that directly map from image pixels to steering behavior are not able to generate accurate results, especially under scenarios such as light and weather change drastically; this is largely due to the fact that these methods ignore the temporal relationship between frames. In this paper, we propose a novel end-to-end deep learning framework based on temporal and spatial attention mechanism, which aims to solve the problem of inaccurate vehicle steering angle prediction in complex environments and the difficulty of model interpretation. First, we use video sequence and historical steering angle sequence as inputs to the model, instead of just using a single frame as input. Second, we design a temporal attention mechanism to capture the long- and short-term memory in input visual information, and a spatial attention mechanism to capture key objects in the image and obtain their position information. This is achieved by inserting carefully-designed SE-Net, ConvLSTM and CNN layers into the appropriate layers of the network framework. Finally, we demonstrate the feasibility of the proposed model with public Comma2k19 , with comparison to current advanced methods. Experimental results show that compared with state-of-the-art methods, the average absolute error (MAE) values of our model on the training set and testing set are reduced by 10.2% and 6.3%, respectively, and has more accurate steering prediction performance. In addition, we explain the trigger mechanism of steering behavior prediction by visualizing the spatial attention map and temporal attention score on Comma2k19 and Udacity datasets, which further demonstrates that the proposed model can learn human-like driving behavior. [ABSTRACT FROM AUTHOR]

As a powerful machine learning method, deep learning has attracted the attention of numerous researchers. While exploring a high-performance neural network model, the floating-point operations of a neural network model are also increasing. In recent years, many researchers have noticed that efficiency is also one of important indicators to measure the property of neural network models. Obviously, the efficient neural network model is more helpful to deploy on mobile and embedded devices. Therefore, the efficient neural network model becomes a hot research spot. In this paper, we review the methods related to the structural design of efficient convolution neural networks in recent years. According to the characteristics of these methods, we divide them into three kinds of methods: model pruning, efficient architecture, and neural architecture search. Detailed analyses of each method are presented to demonstrate their advantages and disadvantages. Then, we comprehensively compare them in detail and propose many suggestions about the design of the efficient convolution neural network model structure. Inspired by these suggestions, we built a new efficient neural network model, SharedNet. And the SharedNet obtains the best accuracy of manually-designed efficient CNN models on the ImageNet dataset. [ABSTRACT FROM AUTHOR]

Online transfer learning (OTL) has attracted much attention in recent years. It is designed to handle the transfer learning tasks, where the data of the target domain isn't available in advance but may arrive in an online manner, which may be a more realistic scenario in practice. However, there typically are two limitations of existing OTL algorithms. 1) Existing OTL algorithms are based on shallow online learning models (SOLMs), e.g., linear or kernel models. Due to this limitation of SOLMs they cannot effectively learn complex nonlinear functions in complicated application and the OTL algorithms based on SOLMs cannot either. 2) Existing algorithms only utilize the latest arrived instance to adjust the model. In this way, the previously arrived instances are not utilized. It may be better to utilize the previously arrived instances as well. In this paper, to overcome the abovementioned two limitations, a buffered online transfer learning (BOTL) algorithm is proposed. In the proposed BOTL algorithm, the learner is designed as a deep learning model, referred to as Online Hedge Neural Network (OHNN). In order to enable the OHNN to be effectively learned in an online manner, we propose a buffered online learning framework that utilizes several previously arrived instances to assist learning. Further, to enhance the performance of the OHNN, a model learned in the source domain is transferred to the target domain. The regret bound of the proposed BOTL algorithm is analyzed theoretically. Experimental results on realistic datasets illustrate that the proposed BOTL algorithm can achieve lower mistake rate than the algorithms compared. [ABSTRACT FROM AUTHOR]

The smartphone-based human activity recognition (HAR) provides abundant healthcare guidance for telemedicine and clinic treatment. The continually increasing daily activities cause many difficulties for recognition and labeling. Although multimodal data fusion and artificial intelligence (AI) techniques can solve these problems, big data collection and labeling are still heavy. This paper proposes a remarkable depth data-guided framework based on smartphones for complex HAR and automatic labeling. The hardware platform is utilized to collect information of depth vision from the Microsoft Kinect camera and Inertial Measurement Unit (IMU) signals from the smartphone simultaneously. This framework consists of five clustering layers and deep learning (DL) based classification model to identify 12 complex daily activities. The results show that the hierarchical k-medoids (Hk-medoids) algorithm obtains the labels with high accuracy (93.89 %). Furthermore, the performance evaluation of the DCNN model is evaluated better by comparing it with other machine learning (ML) and DL methods. [ABSTRACT FROM AUTHOR]

Edge computing and artificial intelligence (AI), especially deep learning algorithms, are gradually intersecting to build the novel system, namely edge intelligence. However, the development of edge intelligence systems encounters several challenges, and one of these challenges is the computational gap between computation-intensive deep learning algorithms and less-capable edge systems. Due to the computational gap, many edge intelligence systems cannot meet the expected performance requirements. To bridge the gap, a plethora of new techniques and optimization methods were proposed in the past years: lightweight deep learning models, network compression, and efficient neural architecture search. Although some reviews or surveys have partially covered this large body of literature, we lack a systematic and comprehensive review to discuss all aspects of these deep learning techniques which are critical for edge intelligence implementation. As various and diverse methods, applicable to edge systems, are proposed, a holistic review would enable edge computing engineers and the community to understand the state-of-the-art deep learning techniques that are instrumental for edge intelligence and to facilitate the development of edge intelligence systems. This paper surveys the representative and latest deep learning techniques that are useful for edge intelligence systems, including hand-crafted models, model compression, hardware-aware neural architecture search, and adaptive deep learning models. Finally, based on observations and simple experiments we conducted, we discuss some future directions. [ABSTRACT FROM AUTHOR]

The linear active disturbance rejection control (LADRC) has been applied in many control practices and achieved satisfactory results. However, the controller with fixed parameters cannot achieve the optimal control performance for the controlled plant. In order to optimize the control effort of LADRC with online parameter adjustment, a deep reinforcement learning algorithm, the deep deterministic policy gradient (DDPG), is applied to enhance the LADRC in this paper for the ship course control and to obtain the optimized parameters of LADRC in different conditions. Specifically, the proposed strategy adopts the deep neural network to adjust the control parameters according to the measured states. This makes the proposed method applicable to the nonlinear control systems with high dimensional continuous states and actions. In addition, the stability of the closed-loop system is analyzed based on the Lyapunov method. In simulations, comparisons with Q-learning based LADRC and conventional LADRC controllers are also presented. Simulation results are provided to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In the past few decades, numerous studies have demonstrated the effectiveness of diet modification in preventing and controlling different chronic diseases, bringing increased attention to the creation of nutritional recipes. Currently, the development of recipes is primarily based on personal experience rather than nutritional specifications. To this end, this paper first uses kernel canonical correlation analysis to demonstrate the relationship between nutrients in food and recipes. Based on this relationship, a new recipe expression method is put forward, which objectively reflects differing importance of an ingredient in differing recipes. Then, recipes are composed using an auto-encoder in deep learning, and a fusion model of two auto-encoders is proposed for better concocting recipes. This paper uses two machine learning methods, namely, non-negative matrix factorization and two-step regularized least squares, to form recipes. To tackle overfitting and instability in non-negative matrix factorization during the training of recipe model, we introduce the Frobenius norm to redefine the objective function and add non-smooth sparse matrices. Similar food has similar taste, but their nutrients might differ. This paper also considers nutrients as a kernel matrix of the two-step regularized least squares, which can effectively avoid the occurrence of different food combinations with similar taste. Experimental results show that developing recipes based on nutrients in food is feasible and effective in the context of machine learning. [ABSTRACT FROM AUTHOR]

Deep learning technologies have been rapidly developing recently. They have shown excellent performances in many fields. However, deep learning networks have weak adaptability to small sample sizes.Usually, deep learning networks require tens of thousands of samples to avoid overfitting. In contrast, the kernel method can deal with small sample sizes. In this paper, we use only dozens of samples to train a successful classifier. Deep multiple kernel learning (DMKL) has emerged in recent years. It combines the ideas of deep learning and multiple kernel learning. Unfortunately, the DMKL network with a fixed structure cannot adapt to data of different dimensions and sizes. Therefore, in this paper, we propose a novel depth-width-scaling multiple kernel learning (DWS-MKL) algorithm. It has the ability to adjust the architecture according to the input data. We optimize the estimation of leave-one-out error by using the span bound instead of the dual objective function. Finally, we choose a large number of data sets from the UCI benchmark data sets for classification tasks. We also evaluate DWS-MKL on the MNIST data set. The experimental results show that different frameworks have different performances on the same dataset. Our DWS-MKL algorithm obtains better classification results than the state-of-the-art kernel learning algorithms, as determined by a thorough comparison. The encouraging results demonstrate that our method has the potential to improve generalization performance of the model. [ABSTRACT FROM AUTHOR]

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general. [ABSTRACT FROM AUTHOR]

[Display omitted] • A context helps the agent to learn knowledge from previous tasks. • A simple multi-task objective meta-training procedure is proposed to run faster. • A quadratic penalty technique is used to preserve the historical experience. • A predictive model is applied to further accelerate the adaptation phase. In this paper, we present a context-based meta-reinforcement learning approach to tackle the challenging data-inefficiency problem of Hyperparameter Optimization (HPO). Specifically, we design an agent which sequentially selects hyperparameters to maximize the expected accuracy of the machine learning algorithm on the validation set. First, we design a context variable that learns the latent embedding of prior experience, and the agent can solve the new tasks efficiently conditioned on it. Second, we employ a multi-task objective method that aims to maximize the average reward across all the meta-training tasks to meta-train the agent. Third, in the adaptation phase, we introduce a quadratic penalty technique to achieve better performance of the agent. Finally, to further improve the efficiency in the adaptation phase, we use a predictive model to evaluate the accuracy of machine learning algorithm instead of training it. We evaluate our approach on 18 real-world datasets and the results demonstrate that our approach outperforms other state-of-the-art optimization methods in terms of test set accuracy and runtime performance. [ABSTRACT FROM AUTHOR]

• We merge reweighing and adversarial approaches for mitigating bias in machine learning models, while keeping the best from both. • The proposed method can provide interpretable information about fairness of individual instances. • We provide theoretical analysis of properties of adversarial re-weighting. • We explore several variants of instance weight estimation including probabilistic ones. • We evaluate the method on four different real-world datasets, compared to the state-of-the-art techniques, and provide qualitative analysis. With growing awareness of societal impact of artificial intelligence, fairness has become an important aspect of machine learning algorithms. The issue is that human biases towards certain groups of population, defined by sensitive features like race and gender, are introduced to the training data through data collection and labeling. Two important directions of fairness ensuring research have focused on (i) instance weighting in order to decrease the impact of more biased instances and (ii) adversarial training in order to construct data representations informative of the target variable, but uninformative of the sensitive attributes. In this paper we propose a Fair Adversarial Instance Re-weighting (FAIR) method, which uses adversarial training to learn instance weighting function that ensures fair predictions. Merging the two paradigms, it inherits desirable properties from both interpretability of reweighting and end-to-end trainability of adversarial training. We propose four different variants of the method and, among other things, demonstrate how the method can be cast in a fully probabilistic framework. Additionally, theoretical analysis of FAIR models' properties is provided. We compare FAIR models to ten other related and state-of-the-art models and demonstrate that FAIR is able to achieve a better trade-off between accuracy and unfairness. To the best of our knowledge, this is the first model that merges reweighting and adversarial approaches by means of a weighting function that can provide interpretable information about fairness of individual instances. [ABSTRACT FROM AUTHOR]

• A novel context-aware deep kernel network that learns context is proposed. • The learning algorithms for layerwise, stationary and classwise contexts are proposed. • The results on the ImageCLEF, COREL5k and NUS-WIDE benchmarks validate their effectiveness. Context plays a crucial role in visual recognition as it provides complementary clues for different learning tasks including image classification and annotation. As the performances of these tasks are currently reaching a plateau, any extra knowledge, including context, should be leveraged in ordficant leaps in these performances. In the particular scenario of kernel machines, context-aware kernel design aims at learning positive semi-definite similarity functions which return high values not only when data share similar contents, but also similar structures (a.k.a. contexts). However, the use of context in kernel design has not been fully explored; indeed, context in these solutions is handcrafted instead of being learned. In this paper, we introduce a novel deep network architecture that learns context in kernel design. This architecture is fully determined by the solution of an objective function mixing a content term that captures the intrinsic similarity between data, a context criterion which models their structure and a regularization term that helps designing smooth kernel network representations. The solution of this objective function defines a particular deep network architecture whose parameters correspond to different variants of learned contexts including layerwise, stationary and classwise; larger values of these parameters correspond to the most influencing contextual relationships between data. Extensive experiments conducted on the challenging ImageCLEF Photo Annotation, Corel5k and NUS-WIDE benchmarks show that our deep context networks are highly effective for image classification and the learned contexts further enhance the performance of image annotation. [ABSTRACT FROM AUTHOR]

The Cognitive Packet Network (CPN) bases its routing decisions and flow control on the Random Neural Network (RNN) Reinforcement Learning algorithm; this paper proposes the addition of a Deep Learning (DL) Cluster management structure to the CPN for Quality of Service metrics (Delay Loss and Bandwidth), Cyber Security keys (User, Packet and Node) and Management decisions (QoS, Cyber and CEO). The RNN already models how neurons transmit information using positive and negative impulsive signals whereas the proposed additional Deep Learning structure emulates the way the brain learns and takes decisions; this paper presents a brain model as the combination of both learning algorithms, RNN and DL. The proposed model has been simulated under different network sizes and scenarios and it has been validated against the CPN itself without DL clusters. The simulation results are promising; the presented CPN with DL clusters as a mechanism to transmit, learn and make packet routing decisions is a step closer to emulate the way the brain transmits information, learns the environment and takes decisions. [ABSTRACT FROM AUTHOR]

Combinatorial optimization problems have been widely used in various fields. And many types of combinatorial optimization problems can be generalized into the model of unconstrained binary quadratic programming (UBQP). Therefore, designing an effective and efficient algorithm for UBQP problems will also contribute to solving other combinatorial optimization problems. Pointer network is an end-to-end sequential decision structure and combines with deep learning technology. With the utilization of the structural characteristics of combinatorial optimization problems and the ability to extract the rule behind the data by deep learning, pointer network has been successfully applied to solve several classical combinatorial optimization problems. In this paper, a pointer network based algorithm is designed to solve UBQP problems. The network model is trained by supervised learning (SL) and deep reinforcement learning (DRL) respectively. Trained pointer network models are evaluated by self-generated benchmark dataset and ORLIB dataset respectively. Experimental results show that pointer network model trained by SL has strong learning ability to specific distributed dataset. Pointer network model trained by DRL can learn more general distribution data characteristics. In other words, it can quickly solve problems with great generalization ability. As a result, the framework proposed in this paper for UBQP has great potential to solve large scale combinatorial optimization problems. [ABSTRACT FROM AUTHOR]

In this paper, we address a challenging task in real-word applications, i.e., automatic reading recognition for pointer meters, called PRM. 1 1 PRM: Pointer merters Recognition based on Mask-RCNN. This application is valuable in the fields of military, industry, and aerospace. However, the accuracy of recognizing the readings of pointer meters by machine vision is, oftentimes, affected by several factors, such as uneven illumination in each image, large range variation of illumination in different images, complex backgrounds, tilting of pointer meters, image blur, and scale change, resulting in the recognized readings with unacceptable accuracy. In this paper, a new robust approach to reading recognition of pointer meters is proposed. The proposed method consists of three main contributions: (1) constructing a novel deep learning algorithm in which the PrRoIPooling is used in lieu of the RoiAlign in the existing Mask-RCNN, (2) classifying the type of pointer meters while fitting the pointer binary mask, and (3) calculating the readings of pointer meters by the proposed angle method. In addition, we also report and release a new dataset for the community. Experiments show that the new algorithm can significantly improve the accuracy of the recognized readings of pointer meters, meanwhile, the proposed approach is also robust to the natural environments and computationally efficient. [ABSTRACT FROM AUTHOR]

• Reviews more than 45 recent solutions papers, and more than 40 different privacy-preserving deep learning techniques. • Proposes a multi-level taxonomy that classifies the privacy-preserving deep learning techniques. • Summarizes evaluation results of the reviewed solutions with respect to performance metrics. • Discusses and outline a number of learned lessons of each privacy-preserving task. • Presents solutions comparison, highlights open research challenges and provides some recommendations for future research. Deep learning is one of the advanced approaches of machine learning, and has attracted a growing attention in the recent years. It is used nowadays in different domains and applications such as pattern recognition, medical prediction, and speech recognition. Differently from traditional learning algorithms, deep learning can overcome the dependency on hand-designed features. Deep learning experience is particularly improved by leveraging powerful infrastructures such as clouds and adopting collaborative learning for model training. However, this comes at the expense of privacy, especially when sensitive data are processed during the training and the prediction phases, as well as when training model is shared. In this paper, we provide a review of the existing privacy-preserving deep learning techniques, and propose a novel multi-level taxonomy, which categorizes the current state-of-the-art privacy-preserving deep learning techniques on the basis of privacy-preserving tasks at the top level, and key technological concepts at the base level. This survey further summarizes evaluation results of the reviewed solutions with respect to defined performance metrics. In addition, it derives a set of learned lessons from each privacy-preserving task. Finally, it highlights open research challenges and provides some recommendations as future research directions. [ABSTRACT FROM AUTHOR]

This paper presents an empirical exploration of the use of capsule networks for text classification. While it has been shown that capsule networks are effective for image classification, the research regarding their validity in the domain of text has been initiated recently. In this paper, we show that capsule networks indeed have the potential for text classification and that they have several advantages over convolutional neural networks. As well, we compare our proposed model to the initial studies regarding capsule network-based text classification. We further suggest a simple routing method that effectively reduces the computational complexity of dynamic routing. We utilized seven benchmark datasets to demonstrate that capsule networks, along with the proposed routing method provide comparable results. [ABSTRACT FROM AUTHOR]

Super-resolution aims to enhance the quality of a low-resolution image to create a high-resolution one. Remarkable advances are witnessed in this field using machine learning techniques. This paper presents a super-resolution method based on generative adversarial networks (GAN) with quantum feature enhancement. The proposed framework uses a feature enhancement layer inspired by the quantum superposition principle. The layer was added to the state-of-the art super-resolution GAN (SRGAN) original model to enhance its performance. The model was trained and evaluated using two publicly available high-resolution aerial images datasets taken by an unmanned aerial vehicle. A set of statistically significant experiments are reported to show its performance. The structural similarity index metric (SSIM), t-distributed stochastic neighbor embedding (t-SNE) and peak signal-to-noise ratio (PSNR) are adopted to evaluate the performance of this proposal against SRGAN model. Results show that this proposal outperforms SRGAN in term of image reconstruction quality by 8% in similarity. [ABSTRACT FROM AUTHOR]

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the Q -values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon (n), and our empirical experiments show that a poorly chosen static value for n can in many cases lead to worse performance than single-step DQN. Inspired by the success of n -step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call 'Elastic Step DQN' (ES-DQN) to alleviate overestimation bias in DQNs. ES-DQN dynamically varies the step size horizon in multi-step updates based on the similarity between states visited. Our empirical evaluation shows that ES-DQN out-performs n -step with fixed n updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias. [ABSTRACT FROM AUTHOR]

In recent years, a constant and fast information growing has characterized digital applications in the majority of real-life scenarios. Thus, a new information asset, namely Big Data, has been defined and lead to different challenges, mainly related to data storage, management and analysis. Focusing on the last challenge, several Big Data analytics techniques have been developed, based on Machine Learning and Deep Learning paradigms. When dealing with Big Data, traditional approaches often take a lot of time to produce even a single predictive model, due to the extremely high demand of computational resources. The design of approaches specifically oriented to Big Data is required to overcome these computational issues. Most solutions rely on the deployment of Big Data analytics infrastructures on a cluster of machines and/or on parallelization techniques. When deployment and parallelization apply to Machine Learning and Deep Learning, we can refer to the terms Distributed Machine Learning and Distributed Deep Learning, respectively. We here discuss the main principles and features of Distributed Machine Learning and Distributed Deep Learning frameworks. The main contribution of this work is a survey of solutions proposed in the literature, through the investigation of selected features and capabilities. In particular, the survey provides a comparative analysis according to the following classification criteria: implemented parallelization technique, supporting device, supported architecture, implemented communication mode, working mode, and class of algorithms. The paper also gives an overview of the most commonly used criteria and metrics for the performance evaluation of analyzed frameworks; finally, some emerging but promising optimization techniques are reviewed apart from our classification. • Big Data analytics leads to challenges related to data management and analysis. • An analysis of the related solutions proposed in the literature is required. • The analysis is focused on Distributed Machine Learning and Deep Learning. • Overview of framework performance evaluation criteria with optimization techniques. [ABSTRACT FROM AUTHOR]

Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks. [ABSTRACT FROM AUTHOR]