Search

Data augmentation strategies have always been important in machine learning techniques and play a unique role in model performance optimization processes. Therefore, in recent years, these techniques have become popular in the artificial intelligence field. In this paper, a new data augmentation strategy is proposed based on the interpretation algorithm of deep convolutional neural networks, i.e., constructing new training samples by deeply exploiting key features extracted from interpretable networks to achieve sample augmentation. Thus, a novel supervised data augmentation approach known as Supervised Data Augmentation–Key Feature Extraction (SDA-KFE) was proposed. By introducing the Neural Network Interpreter-Segmentation Recognition and Interpretation (NNI-SRI) algorithm, an augmentation strategy is proposed that can balance the high accuracy and high robustness of the final model while ensuring a large amount of data augmentation. The advantages of the SDA-KFE algorithm are mainly reflected in the following aspects. First, it is easy to implement. This algorithm is implemented based on the lightweight NNI-SRI algorithm, which lays the foundation for the implementation of SDA-KFE so that it can be easily implemented on convolutional neural networks. Second, this model, which is widely applicable, can be applied to almost any deep convolutional network. Through research and experiments on this proposed algorithm, SDA-KFE can be applied in graphical image binary classification and multiclassification models. Third, SDA-KFE can rapidly construct data samples with diverse variations. Under the premise of determining the classification labels of the generated samples, the distribution of the feature unit composition of the samples can be controlled. Compared with traditional data augmentation methods, SDA-KFE can control the direction of the model performance, i.e., the balance between the pursuit of high accuracy and robust performance of the model. Therefore, the novel supervised augmentation approach proposed in this paper is relevant for optimizing deep convolutional neural networks, solving model overfitting, augmenting data types, etc. The data augmentation algorithm proposed in this paper can be regarded as a useful supplement to traditional data augmentation methods, such as horizontal or vertical image flipping, cropping, color transformation, extension and rotation. [ABSTRACT FROM AUTHOR]

Moisture diffusion is a common phenomenon in geotechnical engineering, and its induced deformation seriously affects the stability of the engineering structure, such as embankment slope instability and tunnel surrounding rock deformation. Numerical simulation is an effective method for moisture diffusion-deformation coupling computation, but it has a large computational cost and a high learning threshold for ordinary engineers. In this paper, a surrogate model based on deep convolutional neural networks is presented for moisture diffusion-deformation coupling computation. First, designing the neural network structure includes the dense blocks and transition layers, and hyperparameters. Then, the moisture diffusion-deformation coupling model in the finite discrete element method (FDEM) software package MultiFracS is used to obtain the high-fidelity simulation data for the moisture diffusion-deformation examples. The simulation data and the key parameter (elastic modulus and Poisson's ratio) are processed into the image data structure (matrix) for training the surrogate model. Finally, the root means square error and the correlation coefficient are used to evaluate the effectiveness of the surrogate model. The results reveal that, rather than taking several hours to run a numerical model, the surrogate model only takes a few seconds to obtain the deformation and stress under a given moisture field and material parameters, which significantly improves prediction efficiency. Using this surrogate model, the engineers can obtain the deformation law just only modifying key parameters. Moreover, the surrogate model can be packaged into a mobile app to provide support for rapid decision-making on the project site. [ABSTRACT FROM AUTHOR]

• Local and global self-attention mechanisms are used for character embedding. • CNN with multi-size filters are used to extract character information for NER. • A cross-attention method that fuses character and word embedding for NER is proposed • A modified Mogrifier LSTM is presented to improve the performance of NER. • Proposed methods integrated with a transformer-based model achieve good performance. Clinical and biomedical concept extraction is critical in medical analysis using clinical and biomedical documents from professional literature, EHRs and PHRs. Named entity recognition (NER) accurately marks essential information in the literature based on the characteristics of the target entity, providing a method for extracting clinical and biomedical concepts. The performance of NER is heavily embedding-dependent, so recent studies have proposed the method of generating word embedding from character-level information, which can strengthen the representation ability for word embedding. In this paper, we present a novel neural network model including an attention mechanism network and a convolutional neural network (CNN) to further improve character-level embedding. First, an attention mechanism is applied simultaneously to the local and global character embedding. Then, a CNN with multi-size filters is used to extract more information from the character level, which can capture more meaningful features from words with various lengths. In addition, a cross-attention method is used to leverage the interaction between word embedding and character embedding to generate the final word representation. Finally, we modified Mogrifier LSTM to make it suitable for NER tasks and integrated it into our model. Experimental results show that our method is effective and that the model performs better than the baseline models. We also apply our methods proposed in this paper to the transformer-based model and obtain a 90.36 F1-score on NCBI-Disease. [ABSTRACT FROM AUTHOR]

Designing a Convolutional Neural Networks (CNN) is a complex task and requires expert knowledge to optimize the performance and network architecture. In this paper, a novel data-driven approach is proposed to determine the architecture of CNN models. The proposed Autonomous Convolutional Neural Networks (AutoCNNThe executable code and original numerical results can be downloaded from (https://tinyurl.com/AutoCNN)) algorithm introduces data driven strategies for addition of new convolutional layers, pruning of redundant filters and training cycle optimization. AutoCNN is evaluated using MNIST, MNIST-rot-back-image, Fashion MNIST and the ADHD200 datasets to measure the performance on small datasets with varied feature distributions. The results indicate that AutoCNN optimizes the CNN network architecture and helps maximise the classification performance. The data-driven network determination approach introduced in this paper was found to not only provides competitive performance similar to existing evolutionary computation based network determination algorithms in literature, but was found to be an effective optimization tool to improve the performance of existing CNN architectures. Further, the AutoCNN was found to highly immune to noise in the dataset and has proven to be effective method to transfer knowledge between related datasets. Therefore, the AutoCNN is a highly versatile CNN architecture determination tool that has a wide range of applications in the field of autonomous driving, medical image analysis, image enhancement, camera based security monitoring and image based fault detection. [ABSTRACT FROM AUTHOR]

The early and accurate detection of COVID-19 is vital nowadays to avoid the vast and rapid spread of this virus and ease lockdown restrictions. As a result, researchers developed methods to diagnose COVID-19. However, these methods have several limitations. Therefore, presenting new methods is essential to improve the diagnosis of COVID-19. Recently, investigation of the electrocardiogram (ECG) signals becoming an easy way to detect COVID-19 since the ECG process is non-invasive and easy to use. Therefore, we proposed in this paper a novel end-to-end deep learning model (ECG-COVID) based on ECG for COVID-19 detection. We employed several deep Convolutional Neural Networks (CNNs) on a dataset of 1109 ECG images, which is built for screening the perception of COVID-19 and cardiac patients. After that, we selected the most efficient model as our model for evaluation. The proposed model is end-to-end where the input ECG images are fed directly to the model for the final decision without using any additional stages. The proposed method achieved an average accuracy of 98.81%, Precision of 98.8%, Sensitivity of 98.8% and, F1-score of 98.81% for COVID-19 detection. As cases of corona continue to rise and hospitalizations continue again, hospitals may find our study helpful when dealing with these patients who did not get significantly worse. [ABSTRACT FROM AUTHOR]

Accurate and timely crop yield prediction is difficult to achieve due to the nonlinear and dynamic spatial–temporal correlations included during the crop growth process. The latest approaches utilize deep learning techniques to implicitly capture sequential dependencies but fail to consider prior knowledge. In this paper, a novel Knowledge-guided Spatial–Temporal Attention Graph Network (KSTAGE), which incorporates prior knowledge to model dynamic temporal relationships and geospatial topology structures, is proposed. The proposed KSTAGE first utilizes a 3D convolutional neural network (CNN) to embed the initial spectral feature. Following its development, a new Knowledge-guided Temporal Multi-head Attention Algorithm (KTMA), the core of KSTAGE, is applied to automatically generate temporal attention weights from a self-attention mechanism under the guidance of prior knowledge. Furthermore, a new strategy for aligning self-attention scores with the prior distribution is developed. Finally, a location-aware Spatial Attention Graph Network utilizing geospatial knowledge is proposed to aggregate the spatial neighborhood features for the final yield prediction. The experimental results obtained on county-level yield prediction tasks in China and the contiguous United States (CONUS) demonstrate that the proposed KSTAGE achieves obvious improvements over the baselines. [ABSTRACT FROM AUTHOR]

• A deep metric learning method is presented for facial expression recognition. • Chi-squared distance is modified for deep metric learning in histogram space. • A new fully histogram based Convolutional Neural Network is proposed. • A triplet loss based on modified chi-squared distance is proposed to learn the CNN. Facial expression recognition is a challenging problem in machine learning. There is much research has been conducted in this field; however, the accuracy of facial expression recognition, especially in uncontrolled conditions, needs much improvement. In this paper, a deep histogram metric learning in a Convolutional Neural Network (CNN) is presented for facial expression recognition. The proposed CNN utilizes a histogram calculation layer to provide statistical description of feature maps at the output of the convolutional layers. To train the proposed CNN in histogram space, a learnable matrix (equivalent to the fully connected layer) is introduced in chi-squared distance equation. Then, the modified equation is used in the loss function. The recognition rates of the proposed CNN for seven-class facial expression recognition on four well-known databases including CK+, MMI, SFEW, and RAF-DB are 98.47%, 83.41%, 61.01%, and 89.28%, respectively. The results show superiority of the proposed CNN compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. Moreover, very few research works investigated the limitations of robust Convolutional Neural Networks beyond the well-known accuracy drop on natural images. In this paper, we describe surprising properties of these models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. We also highlight limitations and failure modes that were not discussed in prior works. Through extensive analyses on a wide range of architectures and datasets, we empirically demonstrate that adversarially-trained Convolutional Neural Networks do not exploit efficiently the model capacity and that the simplicity biases induced by Adversarial Training may lead to undesired behaviors. [ABSTRACT FROM AUTHOR]

• Equity value and debt maturity make this portfolio more complete by using KMV model. • The objectives of the proposed portfolio include order moments and default distance. • Improved NSGA-III with chaos map is developed to solve five-objective optimization. • Static penalty is transformed the constrained optimization to unconstrained problem. • CNN-BiLSTM is firstly used to predict trend when selecting stocks. Portfolio management is a critical aspect of investment strategies, with the goal to balance the low-risk and high-return investments. Despite this, existing portfolios frequently overlook the integration of stock selection outcomes and underutilize data from listed companies, leading to suboptimal portfolio performance. Addressing these shortcomings, this paper introduces a hybrid system involving stock selection and portfolio optimization. In stock selection, the system employs a combination of convolutional neural network and bi-directional recurrent neural network to predict stock trends. This approach enables the identification of stocks likely to appreciate in value, setting the stage for their inclusion in the subsequent optimization process. For portfolio optimization, the study formulates a five-objective optimization problem that incorporates mean, variance, skewness, kurtosis, and distance-to-default as key considerations. To solve the many-objective constrained optimization problem, an advanced strategy employing a static penalty function and an improved Non-dominated Sorting Genetic Algorithm III (NSGA-III) based on tent chaotic mapping is utilized. The efficacy of the proposed hybrid system is rigorously tested through three sets of ablation experiments alongside two discussions focused on its robustness and computational efficiency. The findings from these investigations reveal that the hybrid system outperforms traditional approaches, reducing risks and improving returns for investors. [ABSTRACT FROM AUTHOR]

• A new imputation approach based on convolutional neural networks is proposed. • A loss function for imputation is designed in the setting of end-to-end learning. • Fuzzy clustering driven data organization is proposed for creating spatial property. The presence of missing data is a challenging issue in processing real-world datasets. It is necessary to improve the data quality by imputing the missing values so that effective learning from data can be achieved. Recently, deep learning has become the most powerful type of machine learning techniques, which can be used for discovering the hidden knowledge that exists in a large dataset to make accurate predictions. In this paper, we propose an imputation method that involves using a convolutional neural network to impute the missing values. The missing value of each instance is imputed essentially by using a trained kernel. The weights of the kernel are determined by learning from the given data that are arranged spatially in the data matrix. The kernel carries out a weighted sum of neighboring elements in an array for imputing the missing values. In addition, in the absence of the true values with which the missing values are expected to be replaced, a loss function is designed without the need to know the true value. Our method is evaluated on UCI datasets in comparison with state-of-the-art methods. The experimental results show that the proposed approach performs closely to or better than other methods. [ABSTRACT FROM AUTHOR]

• We developed a new recommender system using CNN with cross convolutional filters. • The cross filters capture the higher-order interactions between users and items. • The proposed method uses global pooling to alleviate the overfitting problem. [Display omitted] With the recent development of online transactions, recommender systems have increasingly attracted attention in various domains. The recommender system supports the users' decision making by recommending items that are more likely to be preferred. Many studies in the field of deep learning-based recommender systems have attempted to capture the complex interactions between users' and items' features for accurate recommendation. In this paper, we propose a recommender system based on the convolutional neural network using the outer product matrix of features and cross convolutional filters. The proposed method can deal with the various types of features and capture the meaningful higher-order interactions between users and items, giving greater weight to important features. Moreover, it can alleviate the overfitting problem since the proposed method includes the global average or max pooling instead of the fully connected layers in the structure. Experiments showed that the proposed method performs better than the existing methods, by capturing important interactions and alleviating the overfitting issue. [ABSTRACT FROM AUTHOR]

Electrocardiography (ECG) is a sophisticated tool for the diagnosis of myocardial infarction (MI). Deep learning approaches can support MI diagnosis based on ECG data. However, due to privacy issues and the sensitive nature of ECG data, the use of deep learning methods to access large amounts of ECG data for training remains challenging. Insufficient data will lead to a decrease in diagnosis performance. In addition, most deep learning approaches are trained with multiple-lead ECG data, thereby limiting the extension to portable single-lead ECG monitoring devices. Investigating automated single-lead ECG interpretation is a continuing concern in the development of mobile and wearable ECG monitoring devices. This paper proposes an automated myocardial infarction detection model — SLC-GAN — that synthesizes single-lead ECG data with high morphological similarity through generative adversarial networks (GANs). It automatically detects MI using convolutional neural networks (CNNs) with real ECG data and synthetic ECGs from the GAN. The empirical results indicate that our new SLC-GAN method performs impressively compared with other MI classification methodologies on single-lead ECG from the PTB Diagnostic ECG Database. The MI classification accuracy of SLC-GAN reaches 99.06% with 5-fold cross-validation. [ABSTRACT FROM AUTHOR]

This paper proposes an integrated optimization algorithm (IOA) designed for solving complicated optimization problems that are non-convex, non-differentiable, non-continuous, or computationally intensive. IOA is synthesized from 5 sub-algorithms: follower search, leader search, wanderer search, crossover search, and role learning. The follower search finds better solutions by tracing the leaders. The leader search refines current optimal solutions by approaching or deviating from the central point of the population and then executes a single-round coordinate descent. The wanderer search carries out comprehensive search space expansion. The crossover search generates offspring using solutions from superior parents. Role learning automates the process in which a search agent decides whether to become a follower or a wanderer. A global optima estimation framework (GOEF) is proposed to offer guidelines for designing an efficient optimization algorithm, and IOA is proved to attain global optima. A differentiable integrated optimization algorithm (DIOA) that extends gradient descent is put forward to train deep learning models. Empirical case studies conclude that IOA shows a much faster convergence speed and finds better solutions than the other 8 comparative algorithms based on 27 benchmark functions. IOA has also been applied to solve unit commitment problems in the power system and shows satisfactory results. A power line sub-image classification model based on a convolutional neural network (CNN) is optimized by DIOA. Compared with the pure gradient descent approach, DIOA converges significantly faster and obtains a high test set accuracy with much fewer training epochs. [ABSTRACT FROM AUTHOR]

The brain neurons' electrical activities represented by Electroencephalogram (EEG) signals are the most common data for diagnosing Epilepsy seizure, which is considered a chronic nervous disorder that cannot be controlled medically using surgical operation or medications with more than 40 % of Epilepsy seizure case. With the progress and development of artificial intelligence and deep learning techniques, it becomes possible to detect these seizures over the observation of the non-stationary-dynamic EEG signals, which contain important information about the mental state of patients. This paper provides a concerted deep machine learning model consisting of two simultaneous techniques detecting the activity of epileptic seizures using EEG signals. The time-frequency image of EEG waves and EEG raw waves are used as input components for the convolution neural network (CNN) and recurrent neural network (RNN) with long- and short-term memory (LSTM). Two processing signal methods have been used, Short-Time Fourier Transform (STFT) and Continuous Wavelet Transformation (CWT), have been used for generating spectrogram and scalogram images with sizes of 77 × 75 and 32 × 32, respectively. The experimental results showed a detection accuracy of 99.57 %, 99.57 % using CWT Scalograms, and 99.26 %, 97.12 % using STFT spectrograms as CNN input for the Bonn University dataset and the CHB-MIT dataset, respectively. Thus, the proposed models provide the ability to detect epileptic seizures with high success compared to previous studies. [ABSTRACT FROM AUTHOR]

Turing dynamics mainly focuses on non-homogeneous self-adaptive spatial patterns of reaction diffusion systems in a continuous space. If defining the diffusion environment as network structure, then network patterns are available under Turing instability conditions. In allusion to parameter estimation for network Turing-instable systems, this paper proposes a new recognition method using artificial neural networks. When diffusion occurs on quadrilateral lattice networks, a deep convolutional neural network (CNN) is built to invert the unknown parameters. The results on training set and validation set indicate that the CNN model exhibits great robustness without overfitting and gradient explosion. Meanwhile, the prediction performance on two test sets is excellent, with average relative errors of the estimators ending up at 0.68% and 1.04%, respectively. When diffusion occurs on non-regular networks, a spatial domain graph convolutional network (GCN) is established, which is slightly less robust than the CNN. The average relative error of the estimators on both test sets ends up between 1.1% and 2.8%, which is still in the ideal range. • The parameter estimation method for network Turing systems is investigated. • A 14-layer deep neural network is designed for systems on lattice networks. • A 6-layer graph neural network is designed for systems on irregular networks. [ABSTRACT FROM AUTHOR]

Network pruning has been a hot topic in recent years, and many popular pruning methods rely on network design expertise. However, the pruning process usually involves manual intervention and can be difficult for users who lack prior knowledge. Automatic pruning using evolutionary algorithms shows great promise, but it must address the challenge of performing time-consuming model evaluations and searching through a large solution space. Dataset distillation is a technique that compresses the original dataset to decrease the cost of fine-tuning models. In this paper, we explore the potential of using the distilled dataset to exhibit a similar role as the real dataset in network pruning, and proposed the evolutionary pruning framework using distilled dataset. Specifically, the network pruning pipeline is carried out on the distilled dataset to significantly reduce the model evaluation cost, and the number of filters in the convolutional layer is directly coded to narrow the search space. In addition, a tailored evolutionary algorithm is proposed that takes the form of constrained optimization to search the most suitable pruned network. The experiments conducted on VGG16, VGG19, ResNet56, and ResNet110 demonstrate that the proposed method reduces at least 41.56% of the flops and achieves competitive results with little compromising accuracy. • Distillation dataset is first applied to network pruning. • A constrained evolutionary algorithm with double-balanced multi-branch is proposed. • Deep convolutional neural network pruning is performed using the constrained evolutionary algorithm. [ABSTRACT FROM AUTHOR]

Recommender system has recently received a lot of attention in the information service community. In many application scenarios, such as Internet of Things (IoTs) environments, item multimodal auxiliary information (such as text and image) can be obtained to expand their feature representation and to increase user satisfaction with recommendations. Motivated by this fact, this paper introduces a novel two-stage embedding model (TSEM), which adequately leverage item multimodal auxiliary information to substantially improve recommendation performance. Specifically, it encompasses two sequential stages: graph convolutional embedding (GCE) and multimodal joint fuzzy embedding (MJFE). In the former, we first generate a bipartite graph for user-item interactions, and then utilize it to construct user and item backbone features via a spatial-based graph convolutional network (SGCN). While in the latter, by employing item multimodal auxiliary information, we integrate multi-task deep learning, deterministic Softmax, and fuzzy Softmax into a convolutional neural network (CNN)-based learning framework, which is optimized to obtain user backbone features and item semantic-enhanced fuzzy (SEF) features accurately. After TSEM converges, user backbone features and item SEF features can be utilized to calculate user preferences on items via Euclidean distance. Extensive experiments over two real-world datasets show that the proposed TSEM model significantly outperforms the state-of-the-art baselines in terms of various evaluation metrics. [ABSTRACT FROM AUTHOR]

Technology has grown rapidly in recent years, and new solutions that rely on Machine Learning (ML) and Artificial Intelligence (AI) are introduced every day. With such fast-paced advancement, inspecting and fully comprehending how given models make decisions is becoming problematic. The complex decision-making process of these models has become a black box, making it challenging to unravel how they work; therefore, eXplainable Artificial Intelligence (XAI) methods are crucial for further development. This paper discusses how state-of-the-art techniques determine classifications and why they need to be revised to understand the prediction-generating process fully. It compares those existing solutions with the new method called Principal Image Sections Mapping - PRISM, which relies on Principal Component Analysis and allows visualising the most significant features recognised by a given Convolutional Neural Network. PRISM is implemented in a piece of software called TorchPRISM that can generate and present the clustering based on the method's output. The result can indicate ambiguous classes discrimination; thus, the possibility of automating the output analysis process is also discussed. The paper's main objective is to examine how PRISM enhances the current understanding of the decision-making process and introduce a tool that can facilitate analysing the output. PRISM implementation (TorchPRISM) can be found in the public GitHub repository: https://github.com/szandala/TorchPRISM • PRISM method can explain CNN's decisions in an understandable way for various users. • PRISM uses PCA to point features and highlights them using an RGB-coloured image mask. • PRISM's effectiveness is assessed based on qualitative and quantitative measures. • TorchPRISM software allows the identification of ambiguous classes in large datasets. • PRISM overcomes the deficiencies of existing eXplainable AI and saliency map methods. [ABSTRACT FROM AUTHOR]

• A novel framework called CJC-net is proposed for learning from noisy labels. • The cyclical-training strategy are combined with Co-teaching method. • We propose an improved joint loss function, which is useful for cyclical training. • CJC-net is easy to implement, and is not sensitive to hyper-parameters and network structures. • Extensive experimental results demonstrate the advantages of our method. The recent success of deep convolutional neural networks (CNNs) is mostly due to the availability of large-scale datasets with accurate annotations. However, the collection of such large datasets with clean annotations is time-consuming and not always feasible. In this paper, we propose a novel framework for learning with noisy labels, called the Cyclical training method with Joint loss and Co-teaching strategy net (CJC-net), where the net means our method is insensitive to the structures of the CNN. CJC-net pretrains two networks simultaneously and then performs the cyclical training strategy under an improved co-teaching method based on the two pretrained networks. During the training process, we adjust the learning rates of the two networks to make the network states periodically transfer from overfitting to underfitting. The cumulative loss of each sample under two networks is recorded; and the higher the cumulative loss of a sample is, the higher the probability of identifying it as a noisy label or a hard label. Then, we remove those samples with a high loss and fine-tune the two networks using the remaining data. The experimental results on several datasets demonstrate that CJC-net is superior to many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A spatial kernel generation module is proposed. • A spectral kernel generation module is proposed. • A multi-scale spectral channel attention mechanism is proposed. • A strategy for feature fusion through convolution is proposed. • Our network structure has better classification results than the state-of-the-art. With the widespread use of deep learning methods, more and more classification models based on hyperspectral images (HSI) have been continuously proposed. However, due to the characteristics of high dimensionality and low resolution of HSI, the traditional convolutional neural network (CNN) model cannot effectively process it. In this paper, we propose a classification network, called spatial and spectral kernels generation network (SSKNet), to generate spatial and spectral convolution kernels based on image characteristics, and use them to replace the traditional initialization convolution kernels. We can obtain representative kernels with stronger spatial correlation with the region segmentation, clustering, and mapping operations of the spatial kernels generation module (SaKG Module). Simultaneously, we also propose a spectral kernels generation module (SeKG module), which integrates the multi-scale correlation characteristics of different bands into the spectral attention mechanism, making the generated spectral kernels more accurate. Combining the spatial and spectral kernels allows the network to extract salient features and achieve feature fusion efficiently. Experimental results based on multiple HSI data sets show that the proposed method has better classification accuracy and generalization performance than existing methods. [ABSTRACT FROM AUTHOR]

• A novel deep subspace clustering method is proposed. • Global graph structure is modeled into the method. • A novel residual auto-encoder is designed as the backbone network. • A joint loss function is formulated to train the network. • State-of-the-art performance is achieved for hyperspectral image clustering. Hyperspectral image (HSI) clustering is a challenging task due to the high complexity of HSI data. Subspace clustering methods have achieved promising clustering performance in HSI clustering. However, the existing subspace clustering methods calculate linear affinity rather than nonlinear relationships between data points and often ignore high-level feature extraction, resulting in low clustering accuracy. To address these issues, this paper proposes a Graph Regularized Residual Subspace Clustering Network (GR-RSCNet) that jointly learns deep spectral-spatial representation and robust nonlinear affinity via a deep neural network. Technically, we recast a graph regularized subspace clustering model as a special self-expressive (SE) layer that is integrated into a deep residual convolutional autoencoder. Benefitting from the residual structure, GR-RSCNet can be easily trained from scratch by optimizing a joint loss function consisting of reconstruction error, global manifold regularization term, and self-representation loss. We extensively evaluate the proposed GR-RSCNet for four popular HSI datasets, demonstrating that the GR-RSCNet can achieve state-of-the-art clustering results that significantly outperform many prior arts. [ABSTRACT FROM AUTHOR]

In this paper, we design a framework to predict the value of time series for nonlinear systems. In order to achieve this goal, many studies of applications and plans for machine learning and even deep learning become currently popular. First, we select four nonlinear systems: including a proposed four-dimensional chaotic system, Lorenz system, Duffing oscillator, and Rössler attractor. The framework has three learning parts as Long Short-Term Memory (LSTM) based on Generate Performance Model (GPM), ensemble learning based on Restrict and Control Model (RCM), and one-dimensional convolution neural network (1-DCNN) of dirichlet distribution based on Overall Verification Model (OVM). Before learning steps, we exploit K-means method as pre-processing and hypothesis verification to improve the prediction accuracy. After learning steps, we construct four forecasting progresses as Point by Point Generated Method (PPGM), Sequence Full Generated Method (SFGM), Sequence Multiple Generated Method (SMGM), and Improvement with RCM and OVM (IPRO) to predict the value of the time steps. Finally, we use Mean Average Error (MAE) as the criterion of the prediction, and estimate the accuracy by comparing the error region of the average standard deviation. [ABSTRACT FROM AUTHOR]

Deep learning technology is widely used in many modern fields and a number of models and software frameworks have been proposed. However, it is still very difficult to process deep learning tasks efficiently on traditional high performance computing (HPC) systems. In this paper, we propose swFLOW: a large-scale distributed framework for deep learning on Sunway TaihuLight. Based on the performance analysis results of convolutional neural network (CNN), we optimize the convolutional layer, and get 10.42× speedup compared to the original version. As for distributed training, we use elastic averaging stochastic gradient descent (EASGD) algorithm to reduce communication. On 512 processes, we get a parallel efficiency of 81.01% with communication period τ = 8. Particularly, a decentralized implementation of distributed swFLOW system is presented to alleviate bottleneck of the central server. By using distributed swFLOW system, we can scale the batch size up to 4096 among 1024 concurrent processes for cancerous region detection algorithm. The successful application on swFLOW reveals the great opportunity for joint combination of deep learning and HPC system. [ABSTRACT FROM AUTHOR]

With the extensive use of structural health monitoring technologies, vibration-based structural damage detection becomes a crucial task in both academic and industrial communities. Following the noteworthy trends of data-driven paradigms in recent years, some solutions have been released to identify, localize, and classify damages via deep neural networks. However, some deficiencies still exist for effective damage-intensive feature extraction and representation. To overcome such a problem, this paper proposes a novel end-to-end structural damage detection neural model by taking the advantages of the Convolutional Neural Network and Bidirectional Gated Recurrent Unit in parallel. The well-known IASC-ASCE benchmark and TCRF dataset are used for evaluation. The experimental results show that the proposed approach can achieve a better detecting effect than other existing manners. [ABSTRACT FROM AUTHOR]

• The multi-hop connection rate improve the transmission efficiency of the signal features learned by the multi-hop network. • The receptive field extension scope determines how many extracted features in each layer are used for the final results. • In the experiment, the diverse signal lengths are contrasted and analyzed by the public and real signal dataset. Automatic modulation classification (AMC) is one of the core technologies in non-cooperative communication. In the complex wireless environment, it is not easy to quickly and accurately recognize the modulation styles of signals by conventional methods. The deep learning method (DLM) can deal with the problem and achieve good effects. In conventional DLMs, the length of input data is fixed. However, the signal length in communication is changing, which may not make full use of the DLMs' input signal information to improve the recognition accuracy. In this paper, the deep multi-hop convolutional neural network (CNN) is employed to learn the time-domain signal features with different lengths. The proposed network includes the multi-hop connection rate and the receptive field extension scope to dispose of the limitation. The experiment shows that the proposed network can achieve better recognition results under the sparse multi-hop network structure. The reception field extension scope is also conducive to further improve the recognition effects. Finally, the proposed network has shorter training time and smaller parameters, which is more convenient for training the network and deploying in the existing communication system. [ABSTRACT FROM AUTHOR]

Scholarly venue recommendation is an emerging field due to a rapid surge in the number of scholarly venues concomitant with exponential growth in interdisciplinary research and cross collaboration among researchers. Finding appropriate publication venues is confronted as one of the most challenging aspects in paper publication as a larger proportion of manuscripts face rejection due to a disjunction between the scope of the venue and the field of research pursued by the research article. We present CLAVER--an integrated framework of Convolutional Layer, bi-directional LSTM with an Attention mechanism-based scholarly VEnue Recommender system. The system is the first of its kind to integrate multiple deep learning-based concepts, that requires only the abstract and the title of a manuscript to identify academic venues. An extensive and exhaustive set of experiments conducted on the DBLP dataset certify that the postulated model CLAVER performs better than most of the modern techniques as entrenched by standard metrics such as stability, accuracy, MRR, average venue quality, precision@k, nDCG@k and diversity. [ABSTRACT FROM AUTHOR]

Discrimination of mirco-earthquake on source depth plays an important role in the field of micro-seismic monitoring. Conventional machine learning methods for data classification rely on carefully hand-engineered features that are vulnerable to low signal-to-noise ratio. Convolutional neural networks (CNNs) demonstrate some merits in dealing with structured data modelling where a set of meaningful features can be automatically extracted from sample learning. This paper explores the use of machine learning techniques for discrimination between deep and shallow mirco-seismic events. A benchmarked dataset including 444 micro-earthquakes from an underground cavern collapse in South Louisiana is employed for performance evaluation in this study, where several feature-based classifiers are compared against the CNN classifier. Empirical results show that the deep learning method outperforms the conventional classification techniques in discriminating the source depth. [ABSTRACT FROM AUTHOR]

Recommender system has recently attracted a lot of attention in the information service community. Currently, most recommendation models use deep neural networks to learn user preferences for items and make the final recommendations. However, these current models have not effectively captured the deep semantic features of users and items and have not fully used the auxiliary information of the items. This may result in unsatisfactory recommendations for users' projects. In order to solve the above problem, in this paper, a novel recommendation model called RM-DRL (Recommendation Model based on Deep Representation Learning) was proposed. It mainly consists of two modules: Information Preprocessing and Feature Representation. The former generates the user's primitive feature vectors and the items used in the latter. The latter consists of two phases: Representation Learning for Item Features (RL-IF) and Representation Learning for User Features (RL-UF). The RL-IF takes the primitive feature vectors of the item as input and uses a multi-layer Convolutional Neural Network (CNN) to learn to accurately produce the semantic feature vector of the item through multi-task learning. In RL-UF, the user primitive feature vectors and semantic feature vectors of the user preference history, and the positive and negative items were taken as input, and a novel Attention-Integrated Gated Recurrent Unit (AIGRU) neural network was proposed to learn to accurately produce user semantic feature vector. After the Feature Representation module converges, the semantic feature vectors of the users and the items can be used to calculate the users' preferences on the items via vector dot product. Extensive experiments on five real-world datasets show that RM-DRL remarkably outperforms state-of-the-art baselines in solving the recommendation problem. [ABSTRACT FROM AUTHOR]

Failure detection is employed in the industry to improve system performance and reduce costs due to unexpected malfunction events. So, a good dataset of the system is desirable for designing an automated failure detection system. However, industrial process datasets are unbalanced and contain little information about failure behavior due to the uniqueness of these events and the high cost for running the system just to get information about the undesired behaviors. For this reason, performing correct training and validation of automated failure detection methods is challenging. This paper proposes a methodology called FaultFace for failure detection on Ball-Bearing joints for rotational shafts using deep learning techniques to create balanced datasets. The FaultFace methodology uses 2D representations of vibration signals denominated faceportraits obtained by time–frequency transformation techniques. From the obtained faceportraits, a Deep Convolutional Generative Adversarial Network is employed to produce new faceportraits of the nominal and failure behaviors to get a balanced dataset. A Convolutional Neural Network is trained for fault detection employing the balanced dataset. The FaultFace methodology is compared with other deep learning techniques to evaluate its performance in for fault detection with unbalanced datasets. Obtained results show that FaultFace methodology has a good performance for failure detection for unbalanced datasets. [ABSTRACT FROM AUTHOR]

• We propose an algorithm that extracts opcode execution sequences from a binary file without execution. • We formulate a CRNN, which detects malware using N extracted opcode sequences. • We provide an interpretation of the proposed model by analyzing the hidden layer activation of the CRNN. This paper presents a novel malware-detection model with a convolutional recurrent neural network using opcode sequences. Statistically, an executable file is considered as a set of consecutive machine codes. First, the theoretical foundation on which opcode sequences can be used to detect malware has been discussed. Next, an algorithm for extracting opcode sequences from executables and a deep learning-based malware-detection method that uses the opcode sequences as input have been presented. The proposed model comprises an opcode-level convolutional autoencoder that transforms a long opcode sequence to a relatively short compressed sequence at the front end and a dynamic recurrent neural network classifier that performs a prediction task using the codes generated by the opcode-level convolutional autoencoder at the rear end. Experimentally, the proposed model provided a malware-detection accuracy of 96 % , receiver operating characteristic-area under the curve of 0.99 , and true positive rate (TPR) of 95 %. The highest accuracy and TPR achieved by existing malware-detection methods using opcode sequences were 97 % and 82 % , respectively. Compared with this method, the proposed model delivered a slightly lower accuracy of 96 % but a considerably larger TPR of 95 %. Therefore, the proposed model is capable of more reliable malware detection. [ABSTRACT FROM AUTHOR]

In the past decade, deep learning techniques have powered many aspects of our daily life, and drawn ever-increasing research interests. However, conventional deep learning approaches, such as deep belief network (DBN), restricted Boltzmann machine (RBM), and convolutional neural network (CNN), suffer from time-consuming training process due to fine-tuning of a large number of parameters and the complicated hierarchical structure. Furthermore, the above complication makes it difficult to theoretically analyze and prove the universal approximation of those conventional deep learning approaches. In order to tackle the issues, multilayer extreme learning machines (ML-ELM) were proposed, which accelerate the development of deep learning. Compared with conventional deep learning, ML-ELMs are non-iterative and fast due to the random feature mapping mechanism. In this paper, we perform a thorough review on the development of ML-ELMs, including stacked ELM autoencoder (ELM-AE), residual ELM, and local receptive field based ELM (ELM-LRF), as well as address their applications. In addition, we also discuss the connection between random neural networks and conventional deep learning. [ABSTRACT FROM AUTHOR]

With the proliferation of deep learning techniques, a significant number of applications related to home care systems have emerged recently. In particular, detecting abnormal events in a smart home environment has been extensively studied. In this paper, we adopt deep learning techniques, including convolutional neural networks (CNNs) and long short-term memory (LSTM) networks, to construct deep networks to learn the long-term dependencies from videos for human behavior recognition in a multiview framework. We adopt two cameras as our sensors to efficiently overcome the problem of occlusions and contour ambiguity for improving the accuracy performance of the multiview framework. After performing a series of image preprocessing on the raw data, we obtain human silhouette images as the input to our training model. In addition, because real-world datasets are complicated for analysis, labeling data is time consuming. Therefore, we present an image clustering method based on a stacked convolutional autoencoder (SCAE), which generates clustering labels for autolabeling. Finally, we set up our experimental environment as a normal residence to collect a large dataset, and the experimental results demonstrate the novelty of our proposed models. [ABSTRACT FROM AUTHOR]

• An efficient 3D skeleton-based action recognition using Deep CNNs. • A novel encoding technique to transform pose-transition feature to image. • Fine-tuning an action recognition model with the generated data of action images. • Outstanding recognition accuracy on different challenging 3D action dataset. • Outperformance of accuracy with other deep learning-based approaches. Recently, skeleton-based human action recognition has received more interest from industrial and research communities for many practical applications thanks to the popularity of depth sensors. A large number of conventional approaches, which have exploited handcrafted features with traditional classifiers, cannot learn high-level spatiotemporal features to precisely recognize complex human actions. In this paper, we introduce a novel encoding technique, namely Pose-Transition Feature to Image (PoT2I), to transform skeleton information to image-based representation for deep convolutional neural networks (CNNs). The spatial joint correlations and temporal pose dynamics of an action are exhaustively depicted by an encoded color image. For learning action models, we fine-tune end-to-end a pre-trained network to thoroughly capture multiple high-level features at multi-scale action representation. The proposed method is benchmarked on several challenging 3D action recognition datasets (e.g., UTKinect-Action3D, SBU-Kinect Interaction, and NTU RGB+D) with different parameter configurations for performance analysis. Outstanding experimental results with the highest accuracy of 90.33% on the most challenging NTU RGB+D dataset demonstrate that our action recognition method with PoT2I outperforms state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• EMAU-Net accurately determines the acceleration/deceleration areas of FHR signal. • We propose a novel baseline estimation method based on deep learning. • Our method achieves a new record on public and private databases. Automatic baseline determination is crucial for reducing the occurrence of fetal acidosis in clinical practice. However, there is a nonnegligible gap between the results of automatic baseline determination and the consensus of experts. In this paper, we propose a novel deep learning approach for baseline determination. First, potential accelerations/decelerations are recognized from the fetal heart rate and excluded by an ensemble multiattention U-Net. Then, the reference baseline and reliable interval are calculated via long- and short-term filters. Based on the filters, unreliable points for estimating the baseline are removed, and the final baseline is determined. We evaluate the performance of the proposed method on a public and a private database. Compared with state-of-the-art methods, our method yields better performance (the root mean square difference between baselines (BL. RMSD), F-measures for acceleration and deceleration (Acc/Dec. F-measures), the synthetic inconsistency coefficient (SI), and the morphological analysis discordance index (MADI) are 2.84 bpm, 0.80, 0.77, 48.9% and 3.94%, respectively) on the public database. The proposed method performs optimally in all metrics on the private database (BL. RMSD, Acc/Dec. F-measures, SI, and MADI are 1.75 bpm, 0.88, 0.80, 43.5%, and 3.11%, respectively). The experimental results indicate the effectiveness and generalizability of the proposed method. [ABSTRACT FROM AUTHOR]

Symbolic Time Intervals (STIs) describe events having a non-zero time duration, which occur in a wide range of application domains. In this paper, we target the challenge of STIs series classification (STIC), which refers to the categorization of series of STIs. Over the recent years several advancements have been made in STIC, all of which are based on either distance-metrics or feature-based traditional classifiers, mostly relying on hand-engineering of features. Due to the high computational cost of either distance calculation or feature extraction, most methods also have quite little potential to scale. We introduce INSTINCT – a novel deep learning-based framework for STIC, which 1) proposes an almost fully information-preserving transformation of raw STIs series into real matrices, and 2) presents a novel ensemble of deep inception-based convolutional neural networks for their classification. The evaluation is applied to the six real-world STIC benchmark datasets and demonstrates that INSTINCT significantly improves accuracy over seven state-of-the-art methods, as well as over three deep learning-based baselines. In addition, a comprehensive architecture study of INSTINCT is conducted as well as a scalability analysis, reporting an overall time complexity which is linear in each of the main properties of the input STIs series. • Novel deep learning-based framework for Symbolic Time Intervals series classification. • Representation of raw Symbolic Time Intervals series as real matrices. • Inception-based networks ensemble for Symbolic Time Intervals series classification. • Improved classification accuracy over state-of-the-art. • Linear time complexity. [ABSTRACT FROM AUTHOR]

• A shock detection method based on Convolutional Neural Networks is proposed. • A novel loss function is designed for shock waves. • Compared with existing methods, ours is more efficient or yields better results. • The good generalization of our method is demonstrated by extensive experiments. As one of the main technologies of flow visualization, shock detection plays a key role in feature identification and has been intensively studied. However, existing methods take too much execution time to meet the requirement of post-processing on large-scale Computational Fluid Dynamics (CFD) flow field data. To address this problem, in this paper, we propose a detection method for shock waves based on Convolutional Neural Networks (CNN) and design a novel loss function to optimize the detection results. In specific, the proposed method samples small patches from flow field data, and trains a detection network which includes multiple convolutional layers. This network is responsible for generating shock values and finding the location of shock waves. Compared with the existing shock detection methods which are not based on deep learning, our method has great advantages in detection time. Compared with the ones based on deep learning, our method gives a better detection result of shock waves. Extensive experimental results demonstrate the good generalization of the proposed method on many datasets. [ABSTRACT FROM AUTHOR]

Semantic web image retrieval is useful to end-users for semantic image searches over the Internet. This paper aims to develop image retrieval techniques for large-scale web image databases. An advanced retrieval system, termed Multi-concept Retrieval using Bimodal Deep Learning (MRBDL), is proposed and implemented using Convolutional Neural Networks (CNNs) which can effectively capture semantic correlations between a visual image and its free contextual tags. Different from existing approaches using multiple and independent concepts in a query, MRBDL considers multiple concepts as a holistic scene for retrieval model learning. In particular, we first use a bimodal CNN to train a holistic scene classifier in two modalities, and then semantic correlations of the sub-concepts included in the images are leveraged to boost holistic scene recognition. The predicted semantic scores obtained from holistic scene classifier are combined with complementary information on web images to improve the retrieval performance. Experiments have been carried out over two publicly available web image databases. The results show that our proposed approach performs favorably compared with several other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Dengue is a mosquito-borne viral disease widely spread in tropical and subtropical regions. Its adverse impact on the human health and global economies cannot be overstated. In order to implement more effective vector control measures, mechanisms that can more accurately forecast dengue cases are needed more urgently than before. In this paper, a novel hybrid architecture which has the advantages of both convolutional neural networks and recurrent neural networks is being proposed to forecast weekly dengue incidence. The forecasting performance of this architecture reveals that the deep hybrid architecture outperforms other frequently used deep learning models in dengue forecasting tasks. We have also evaluated the proposed models against state-of-the-art studies in the literature, demonstrating that our proposed hybrid models utilizing recurrent networks with convolutional layers can provide a significant boost in dengue forecasting. • Hybrid deep learning models are developed for dengue incidence forecasting. • The models give excellent forecasting accuracy up to 87.72% for weekly cases. • Up to 4 weeks in advance forecasting is provided. • The effects of different number of deep layers and used features are discussed. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has attracted an increasing amount of attention in machine learning and artificial intelligence areas. Currently, many deep learning network-related architectures such as deep neural networks (DNNs), convolutional neural network (CNN), wavelet scattering network (ScatNet) and principal component analysis network (PCANet) have been proposed. The most effective network is PCANet, which has achieved promising performance in image classification, such as for face, object and handwritten digit recognition. PCANet can only handle data that are represented by single-view features. In this paper, we present a canonical correlation analysis network (CCANet) to address image classification, in which images are represented by two-view features. The CCANet learns two-view multistage filter banks by a canonical correlation analysis (CCA) method and constructs a cascaded convolutional deep network. Then, we incorporate filters with binaryzation and block-wise histogram processes to form the final depth structure. In addition, we introduce a variation of CCANet—dubbed RandNet-2—in which the filter banks are randomly generated. Extensive experiments are conducted using the ETH-80, Yale-B, and USPS databases for object classification, face classification and handwritten digits classification, respectively. The experimental results demonstrate that the CCANet algorithm is more effective than PCANet, RandNet-1 and RandNet-2. [ABSTRACT FROM AUTHOR]

The inverse identification of nonhomogeneous material properties from measured displacement/strain fields, especially when noise exists, is crucial for both engineering and material science. The conventional physics-based solutions either require time-consuming iterative calculations, or are sensitive to noise. While the new machine learning methods either need excess data for high-dimensional matchups, or mainly apply to case-by-case analyses with informed physics. In this paper, to solve the complex matchup between the measured displacement/strain fields and the randomly distributed modulus field rapidly and robustly, a novel method of deep learning in frequency domain is proposed, with discrete cosine transform (DCT) to achieve frequency domain transformation as well as dimensionality reduction and convolutional neural network (CNN) to implement learning in frequency domain. Results show that our method not only has high prediction accuracy on zero-noise samples (with L 1 -error of 4.249%) but also presents great robustness to noise (with L 1 -error of 5.085% on large-noise samples). Besides, by our method, only one-time training on a dataset with mixed noise is basically enough to deal with arbitrary levels of noise (with L 1 -errors below 5.202%), improving the efficiency significantly in practical applications. Moreover, our method can be directly transferred to neighbor sampling spaces with different sampling points, showing a great generalization. The study provides a powerful approach for inverse identification of material properties and promises for wide applications such as real-time elastography and high-throughput non-destructive evaluation techniques. [ABSTRACT FROM AUTHOR]

• A review on the applications of artificial intelligence on battling against covid-19 is performed. Colloquially known as coronavirus, the Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2), that causes CoronaVirus Disease 2019 (COVID-19), has become a matter of grave concern for every country around the world. The rapid growth of the pandemic has wreaked havoc and prompted the need for immediate reactions to curb the effects. To manage the problems, many research in a variety of area of science have started studying the issue. Artificial Intelligence is among the area of science that has found great applications in tackling the problem in many aspects. Here, we perform an overview on the applications of AI in a variety of fields including diagnosis of the disease via different types of tests and symptoms, monitoring patients, identifying severity of a patient, processing covid-19 related imaging tests, epidemiology, pharmaceutical studies, etc. The aim of this paper is to perform a comprehensive survey on the applications of AI in battling against the difficulties the outbreak has caused. Thus we cover every way that AI approaches have been employed and to cover all the research until the writing of this paper. We try organize the works in a way that overall picture is comprehensible. Such a picture, although full of details, is very helpful in understand where AI sits in current pandemonium. We also tried to conclude the paper with ideas on how the problems can be tackled in a better way and provide some suggestions for future works. [ABSTRACT FROM AUTHOR]

• The frequency matrix chaos game representation images of gene sequences are input to convolutional spiking neural networks to predict essential genes. • Convolutional spiking neural networks can well extract features that distinguish essential and nonessential genes. • Intra-organism and cross-organism essential gene prediction experiments and comparative studies have been carried out. [Display omitted] Chaos game representation (CGR) is a useful one-to-one visualization tool to represent nucleotide sequences, in which both local and global patterns of nucleotides can be graphically described. Deep learning networks have been proved to achieve outstanding performance on feature extraction and image recognition. In this paper, we use convolutional spiking neural networks (SNNs) with reward-modulated spike-timing-dependent plasticity (R-STDP) learning rule to learn from the frequency matrix chaos game representation (FCGR) images of essential and non-essential genes of 32 bacteria in the DEG database and make intra-organism and cross-organism essential gene predictions. For intra-organism predictions, our highest accuracy(ACC) score is 0.90 and the average ACC is 0.78, and for cross-organism predictions, our highest ACC is 0.79 and the average ACC is 0.68. Compared with the results of traditional machine learning classifiers training with FCGR images or numerical fractal features pre-calculated from CGR representations, our intra-organism prediction results are much better for all the bacteria or most bacteria, respectively, indicating that our spiking neural networks can make better essential gene prediction by extracting the gene features directly from the FCGR images of essential and nonessential genes. Compared with essential gene prediction methods using gene sequence features and topological features, our cross-organism prediction results can achieve performance close to or even better than such methods, while requiring much fewer input features. [ABSTRACT FROM AUTHOR]

• Classification of normal, viral pneumonia and Covid-19 chest x-ray images. • Proposed a deep convolutional neural network model for early detection of COVID19 cases using chest x-ray images. • Trained the model on a dataset prepared by collecting x-ray images from publically available databases. • Achieved an average accuracy of 97.20% for detecting COVID-19 and an average accuracy of 96.69% for three-class classification (COVID-19 vs. normal vs. viral pneumonia). The results obtained by our proposed model are superior compared to other studies in the literature. • Promising results indicate that this model can be an interesting tool to help radiologists in the diagnosis and early detection of COVID-19 cases. The COVID-19 pandemic is an emerging respiratory infectious disease, also known as coronavirus 2019. It appears in November 2019 in Hubei province (in China), and more specifically in the city of Wuhan, then spreads in the whole world. As the number of cases increases with unprecedented speed, many parts of the world are facing a shortage of resources and testing. Faced with this problem, physicians, scientists and engineers, including specialists in Artificial Intelligence (AI), have encouraged the development of a Deep Learning model to help healthcare professionals to detect COVID-19 from chest X-ray images and to determine the severity of the infection in a very short time, with low cost. In this paper, we propose CVDNet, a Deep Convolutional Neural Network (CNN) model to classify COVID-19 infection from normal and other pneumonia cases using chest X-ray images. The proposed architecture is based on the residual neural network and it is constructed by using two parallel levels with different kernel sizes to capture local and global features of the inputs. This model is trained on a dataset publically available containing a combination of 219 COVID-19, 1341 normal and 1345 viral pneumonia chest x-ray images. The experimental results reveal that our CVDNet. These results represent a promising classification performance on a small dataset which can be further achieve better results with more training data. Overall, our CVDNet model can be an interesting tool to help radiologists in the diagnosis and early detection of COVID-19 cases. [ABSTRACT FROM AUTHOR]

• Developed deep learning-based models to forecast emergency department overcrowding. • Eight deep learning models have been compared for ED visits forecasting. • Real Datasets from the ED at CHRU Lille, France, investigated to serve the case study. • Two types of forecasting are conducted: one- and multi-step-ahead forecasting. • Results show the superior performance of the VAE compared to the other algorithms. As the demand for medical cares has considerably expanded, the issue of managing patient flow in hospitals and especially in emergency departments (EDs) is certainly a key issue to be carefully mitigated. This can lead to overcrowding and the degradation of the quality of the provided medical services. Thus, the accurate modeling and forecasting of ED visits are critical for efficiently managing the overcrowding problems and enable appropriate optimization of the available resources. This paper proposed an effective method to forecast daily and hourly visits at an ED using Variational AutoEncoder (VAE) algorithm. Indeed, the VAE model as a deep learning-based model has gained special attention in features extraction and modeling due to its distribution-free assumptions and superior nonlinear approximation. Two types of forecasting were conducted: one- and multi-step-ahead forecasting. To the best of our knowledge, this is the first time that the VAE is investigated to improve forecasting of patient arrivals time-series data. Data sets from the pediatric emergency department at Lille regional hospital center, France, are employed to evaluate the forecasting performance of the introduced method. The VAE model was evaluated and compared with seven methods namely Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Convolutional LSTM Network (ConvLSTM), restricted Boltzmann machine (RBM), Gated recurrent units (GRUs), and convolutional neural network (CNN). The results clearly show the promising performance of these deep learning models in forecasting ED visits and emphasize the better performance of the VAE in comparison to the other models. [ABSTRACT FROM AUTHOR]