Search

Deep neural networks are extensively used in text mining and Natural Language Processing is to enable computers to understand, analyze, and generate natural language data, such as text or speech, but semantic resources, such as taxonomies and ontologies, are not fully included in deep learning. In this paper, we use Deep Convolutional Neural Network (Deep CNN) to classify research papers using the Computer Science Ontology, an ontology of research areas in the field of computer science. It takes as input the abstract and keywords of a particular research paper and returns the relevant research topic. To evaluate our ontology, we used a gold standard dataset that includes research articles. To further improve text classification results, we propose to design a Deep CNN model. We then used ontology matching to reduce the classes and get better results. Experimental results show that the proposed approach outperforms the one with the highest precision, recall, and F1-score. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is a condition that degenerates over time and impairs speech and pronunciation because brain cells have died. This research work aims to predict parkinson disease using the voice features extracted from speech signals recorded from PD individuals with dysphonic speech disorders by employing deep learning algorithms. PD is challenging to diagnose early on in the clinical presentation. To address the issue in machine learning methods, this paper proposes a neural network model by processing speech signals to classify PD using the University of California Irvine (UCI) machine learning repository dataset. Initially, a pre-loss reduction module is created by using pre-sampling to make the dataset balanced by reducing the dimensionality and maintaining the size of the space without influencing the learning process for data preparation. The relevant features are derived using a novel multi-agent salp swarm (MASS) algorithm, and a novel Parkinson classification neural network (PCNN) is proposed to classify Parkinson's patients with high accuracy employing these derived features. The result shows that the models that use MASS-PCNN produce higher classification accuracy of 99.1%, precision of 97.8%, recall of 94.7% and F1-score of 0.995 when paralleled to the existing models. As an outcome, the suggested model will perform superior to common convolutional neural networks. [ABSTRACT FROM AUTHOR]

This paper delves into the specialized domain of human action recognition, focusing on the Identification of Indian classical dance poses, specifically Bharatanatyam. Within the dance context, a "Karana" embodies a synchronized and harmonious movement encompassing body, hands, and feet, as defined by the Natyashastra. The essence of Karana lies in the amalgamation of nritta hasta (hand movements), sthaana (body postures), and chaari (leg movements). Although numerous, Natyashastra codifies 108 karanas, showcased in the intricate stone carvings adorning the Nataraj temples of Chidambaram, where Lord Shiva's association with these movements is depicted. Automating pose identification in Bharatanatyam poses challenges due to the vast array of variations, encompassing hand and body postures, mudras (hand gestures), facial expressions, and head gestures. To simplify this intricate task, this research employs image processing and automation techniques. The proposed methodology comprises four stages: acquisition and pre-processing of images involving skeletonization and Data Augmentation techniques, feature extraction from images, classification of dance poses using a deep learning network-based convolution neural network model (InceptionResNetV2), and visualization of 3D models through mesh creation from point clouds. The use of advanced technologies, such as the MediaPipe library for body key point detection and deep learning networks, streamlines the identification process. Data augmentation, a pivotal step, expands small datasets, enhancing the model's accuracy. The convolution neural network model showcased its effectiveness in accurately recognizing intricate dance movements, paving the way for streamlined analysis and interpretation. This innovative approach not only simplifies the identification of Bharatanatyam poses but also sets a precedent for enhancing accessibility and efficiency for practitioners and researchers in the Indian classical dance. [ABSTRACT FROM AUTHOR]

This paper introduces a deep learning approach for accurately predicting the position and size of cracks in steam turbine blades using frequency response function data obtained from finite element analysis. Training deep neural networks for crack prediction necessitates an extensive dataset comprising various crack conditions, along with corresponding position and size information. However, obtaining sufficient high-quality and well-balanced experimental data for diverse crack conditions poses challenges. To address this issue, we leverage finite element analysis techniques to effectively amass a large volume of training data. Our methodology involves frequency response analysis, incorporating a crack modeling approach based on node sharing elimination and model order reduction (MOR). Automating this process enables us to efficiently generate a diverse dataset of crack blade frequency response data in a short timeframe, focusing on cracks in a single blade. The collected data is then transformed into two-dimensional image data, including magnitude similarity maps and magnitude frequency response maps, which serve as training data for our convolutional neural network (CNN) model. Evaluation of the CNN model's performance and accuracy for crack position and size prediction highlights its remarkable precision in determining the initiation position and size of blade cracks. Utilizing the proposed prediction model, there was a 95 % probability of accurately predicting the crack position within 3.1 % of the entire airfoil area. Furthermore, successful crack size predictions were achieved for approximately 96.4 % of the entire test dataset within an error range of ±1 mm. This underscores the potential capability of neural network training and crack prediction by leveraging training data derived from the finite element analysis and MOR. Our approach demonstrates promising implications for enhancing steam turbine blade maintenance and structural integrity assessment, as it streamlines data acquisition and provides an efficient solution for predicting critical crack characteristics. [ABSTRACT FROM AUTHOR]

This paper introduces the Hybrid Quantum Deep Neural Network (HQDNN), a pioneering model that amalgamates classical Convolutional Neural Network (CNN) architecture with Parameterized Quantum Circuits (PmQC) for image classification tasks. The background elucidates the intricacies of CNN and PmQC operations, emphasizing their respective components and functions. HQDNN comprises quantum convolutional and pooling layers alongside a classical fully connected layer, demonstrating its effectiveness through simulations on the MNIST dataset. Results reveal HQDNN's superior learning ability and classification performance compared to traditional CNN, particularly with the inclusion of a pooling layer. This work contributes to the evolution of quantum machine learning, opening avenues for the practical integration of quantum computing in image classification workflows. Future research may explore scalability and diverse applications of HQDNN, marking a promising convergence of quantum computing and deep learning methodologies. [ABSTRACT FROM AUTHOR]

With the increasing proportion of solar energy in the energy system, accurate solar irradiance forecast is of great significance for low-cost energy scheduling. This paper proposes a new forecasting idea for day-ahead solar irradiance forecast on the day-ahead scale: Firstly, based on formula derivation and big data correlation analysis, this paper finds out multiple parameters related to GHI, and jointly uses these parameters to forecast GHI. Error revision during morning period (ERDMP) is innovatively proposed on this basis, towards a more accurate daytime forecast. In order to prove the reliability and universality of the method, relevant data from five different-climatic regions are respectively used in the experiment. The multi-variable historical data-based day-ahead solar irradiance forecast uses deep neural networks, including Recurrent Neural Network (RNN), Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) network. ERDMP uses a linear AutoRegressive (AR) model to predict the daytime error coefficients based on the morning error coefficients. According to the results, through the proposed ERDMP, Mean Absolute Error (MAE) decreases by about 25% to 30%, Root Mean Squared Error (RMSE) decreases by about 20%, and R2 increases by about 5% to 10% when compared with the initial error of multi-parameter prediction models and other advanced models. [ABSTRACT FROM AUTHOR]

A traditional approach for panoramic image generation consists of a random sample consensus (RANSAC) algorithm on a set of scale-invariant feature transform (SIFT) correspondences to generate a homography matrix between two images. Although producing adequate results for some type of images, hand-crafted SIFT features are not robust enough for highly varying natural images and the iterative RANSAC algorithm with its randomness does not always find the desired homography matrix. Recently, deep neural networks have been producing significant results in many challenging computer vision problems by learning features from large amounts of data. However, only very few recent works have been applied deep learning to panoramic image generation with the objective of finding feature correspondences and estimating homography matrix. Moreover, the absence of a proper dataset for the image stitching task hinders the standardization of models and comparison of their results. This paper attempts to generate panoramic images by extensively experimenting with various approaches using deep neural networks. The best proposed deep learning model achieved 7.31 and 1.07 pixels of the average absolute value loss for corner difference in X and Y directions, respectively. At the same time, qualitative results demonstrate superiority in comparison with the state-of-the-art SIFT+RANSAC algorithm. Specifically, in 72% of time the proposed framework either produced better results than SIFT+RANSAC or results of the proposed approach and SIFT+RANSAC were indistinguishable. Although SIFT + RANSAC produces better results in 28% of the time with respect to the loss function, our results are still visually comparable in many of these cases. Finally, a novel panoramic image generation dataset is introduced in this paper. [ABSTRACT FROM AUTHOR]

Combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs) produces a powerful architecture for video classification problems as spatial–temporal information can be processed simultaneously and effectively. Using transfer learning, this paper presents a comparative study to investigate how temporal information can be utilized to improve the performance of video classification when CNNs and RNNs are combined in various architectures. To enhance the performance of the identified architecture for effective combination of CNN and RNN, a novel action template-based keyframe extraction method is proposed by identifying the informative region of each frame and selecting keyframes based on the similarity between those regions. Extensive experiments on KTH and UCF-101 datasets with ConvLSTM-based video classifiers have been conducted. Experimental results are evaluated using one-way analysis of variance, which reveals the effectiveness of the proposed keyframe extraction method in the sense that it can significantly improve video classification accuracy. [ABSTRACT FROM AUTHOR]

Fostered by technological and theoretical developments, deep neural networks (DNNs) have achieved great success in many applications, but their training via mini-batch stochastic gradient descent (SGD) can be very costly due to the possibly tens of millions of parameters to be optimized and the large amounts of training examples that must be processed. The computational cost is exacerbated by the inefficiency of the uniform sampling typically used by SGD to form the training mini-batches: since not all training examples are equally relevant for training, sampling these under a uniform distribution is far from optimal, making the case for the study of improved methods to train DNNs. A better strategy is to sample the training instances under a distribution where the probability of being selected is proportional to the relevance of each individual instance; one way to achieve this is through importance sampling (IS), which minimizes the gradients' variance w.r.t. the network parameters, consequently improving convergence. In this paper, an IS-based adaptive sampling method to improve the training of DNNs is introduced. This method exploits side information to construct the optimal sampling distribution and is dubbed regularized adaptive sampling (RAS). Experimental comparison using deep convolutional networks for classification of the MNIST and CIFAR-10 datasets shows that when compared against SGD and against another sampling method in the state of the art, RAS produces improvements in the speed and variance of the training process without incurring significant overhead or affecting the classification. [ABSTRACT FROM AUTHOR]

Diabetic Retinopathy (DR) is the major cause of blindness, which seriously threatens the world's vision health. Limited medical resources make early diagnosis and a large-scale screening of DR difficult. Most of the current automatic diagnostic methods are mostly based on deep learning and large-scale labeled data. However, the insufficiency of manual annotations for medical images still is a great challenge of training deep neural networks. Self-supervised learning methods are proposed to learn general features from dataset without manual annotations. Inspired by this, we proposed a deep learning based DR classification model (SimCLR-DR). In this paper, we first use contrastive self-learning algorithm to pre-train the encoder based on convolution network with unlabeled retinal images, then retrain the encoder with classifier on a small annotated training data to detect referable DR. The experimental results on Kaggle dataset show that this proposed method can overcome the training data insufficiency problem and performs better than transfer learning. SimCLR-DR is a good beginning for other deep learning based medical image detection approaches facing the challenge of insufficient annotated data. Figure presents an overview of the proposed framework, which contains three main steps: (i) Data preprocessing; (ii) Pretext task of SimCLR-DR based on contrastive learning; (iii) Downstream Task of SimCLRDR based on CNN. [ABSTRACT FROM AUTHOR]

Hurricanes are tropical storms that cause immense damage to human life and property. Rapid assessment of damage caused by hurricanes is extremely important for the first responders. But this process is usually slow, expensive, labor intensive and prone to errors. The advancements in remote sensing and computer vision help in observing Earth at a different scale. In this paper, a new Convolutional Neural Network model has been designed with the help of satellite images captured from the areas affected by hurricanes. The model will be able to assess the damage by detecting damaged and undamaged buildings based upon which the relief aid can be provided to the affected people on an immediate basis. The model is composed of five convolutional layers, five pooling layers, one flattening layer, one dropout layer and two dense layers. Hurricane Harvey dataset consisting of 23,000 images of size 128 × 128 pixels has been used in this paper. The proposed model is simulated on 5750 test images at a learning rate of 0.00001 and 30 epochs with the Adam optimizer obtaining an accuracy of 0.95 and precision of 0.97. The proposed model will help the emergency responders to determine whether there has been damage or not due to the hurricane and also help those to provide relief aid to the affected people. [ABSTRACT FROM AUTHOR]

The ever increasing demand for electricity leads to the advancement of distributed generation (DG). Almost all DG sources are renewable nature. One of the major complications with the high penetration of DG sources is islanding. The islanding may damage the clients and their equipment. As per the IEEE 1547 DG interconnection standards, the islanding will be identified in two seconds and the DG must be turned off. In this paper, an advanced islanding detection process stands on a deep learning technique with continuous wavelet transforms and convolution neural networks implemented. This approach transforms the time series information into scalogram images, and later, the images are used to train and test the islanding and non-islanding events. The outcomes are correlated with the artificial neural networks and fuzzy logic methods. The comparison shows that the proposed deep learning approach efficiently detects the islanding and non-islanding events. [ABSTRACT FROM AUTHOR]

In this paper, we present a lightweight and effective change detection model, called TinyCD. This model has been designed to be faster and smaller than current state-of-the-art change detection models due to industrial needs. Despite being from 13 to 140 times smaller than the compared change detection models, and exposing at least a third of the computational complexity, our model outperforms the current state-of-the-art models by at least 1 % on both F1-score and IoU on the LEVIR-CD dataset, and more than 8 % on the WHU-CD dataset. To reach these results, TinyCD uses a Siamese U-Net architecture exploiting low-level features in a globally temporal and locally spatial way. In addition, it adopts a new strategy to mix features in the space-time domain both to merge the embeddings obtained from the Siamese backbones, and, coupled with an MLP block, it forms a novel space-semantic attention mechanism, the Mix and Attention Mask Block (MAMB). [ABSTRACT FROM AUTHOR]

Plant diseases introduce significant yield and quality losses to the food production industry, worldwide. Early identification of an epidemic could lead to more effective management of the disease and potentially reduce yield loss and limit excessive input costs. Image processing and deep learning techniques have shown promising results in distinguishing healthy and infected plants at early stages. In this paper, the potential of four convolutional neural network models, including Xception, Residual Networks (ResNet)50, EfficientNetB4, and MobileNet, in the detection of rust disease on three commercially important field crops was evaluated. A dataset of 857 positive and 907 negative samples captured in the field and greenhouse environments were used. Training and testing of the algorithms were conducted using 70% and 30% of the data, respectively where the performance of different optimizers and learning rates were tested. Results indicated that EfficientNetB4 model was the most accurate model (average accuracy = 94.29%) in the disease detection followed by ResNet50 (average accuracy = 93.52%). Adaptive moment estimation (Adam) optimizer and learning rate of 0.001 outperformed all other corresponding hyperparameters. The findings from this study provide insights into the development of tools and gadgets useful in the automated detection of rust disease required for precision spraying. [ABSTRACT FROM AUTHOR]

The explosive growth of geospatial data is increasing requirements for automatic and efficient data learning abilities. Many deep learning methods have been widely applied for geospatial data understanding tasks, such as road networks and geospatial object detection. However, the demands for more accurate learning of high-level features require the use of deeper neural networks. To further improve the learning efficiency of deep neural networks, in this paper, we propose an improved convolutional neural network named EDense. First, we use its dense connectivity to integrate a CNN with an extreme learning machine. Then, we expand the kernels in the convolutional layers to increase the width of the network model. Furthermore, we propose one-feature EDense (OF-EDense), which is a simplified version of EDense, to fit conditions in which the number of parameters is strictly limited. Finally, the experimental results fully demonstrate the strong learning ability and high learning efficiency of EDense. [ABSTRACT FROM AUTHOR]

Breast cancer (BC) is spreading more and more every day. Therefore, a patient's life can be saved by its early discovery. Mammography is frequently used to diagnose BC. The classification of mammography region of interest (ROI) patches (i.e., normal, malignant, or benign) is the most crucial phase in this process since it helps medical professionals to identify BC. In this paper, a hybrid technique that carries out a quick and precise classification that is appropriate for the BC diagnosis system is proposed and tested. Three different Deep Learning (DL) Convolution Neural Network (CNN) models—namely, Inception-V3, ResNet50, and AlexNet—are used in the current study as feature extractors. To extract useful features from each CNN model, our suggested method uses the Term Variance (TV) feature selection algorithm. The TV-selected features from each CNN model are combined and a further selection is performed to obtain the most useful features which are sent later to the multiclass support vector machine (MSVM) classifier. The Mammographic Image Analysis Society (MIAS) image database was used to test the effectiveness of the suggested method for classification. The mammogram's ROI is retrieved, and image patches are assigned to it. Based on the results of testing several TV feature subsets, the 600-feature subset with the highest classification performance was discovered. Higher classification accuracy (CA) is attained when compared to previously published work. The average CA for 70% of training is 97.81%, for 80% of training, it is 98%, and for 90% of training, it reaches its optimal value. Finally, the ablation analysis is performed to emphasize the role of the proposed network's key parameters. [ABSTRACT FROM AUTHOR]

The number of deep learning (DL) layers increases, and following the performance of computing nodes improvement, the output accuracy of deep neural networks (DNN) faces a bottleneck problem. The resident network (RN) based DNN model was applied to address these issues recently. This paper improved the RN and developed a rectified linear unit (ReLU) based conditional generative adversarial nets (cGAN) to classify plantar pressure images. A foot scan system collected the plantar pressure images, in which normal (N), planus (PL), and talipes equinovarus feet (TE) data-sets were acquired subsequently. The 9-foot types named N, PL, TE, N-PL, N-TE, PL-N, PL-TE, TE-N, and TE-PL were classified using the proposed DNN models, named resident network-based conditional generative adversarial nets (RNcGAN). It improved the RN structure firstly and the cGAN system hereafter. In the classification of plantar pressure images, the pixel-level state matrix can be direct as an input, different from the previous image classification task with image reduction and feature extraction. cGAN can directly output the pixels of the image without any simplification. Finally, the model achieved better results in the evaluation indicators of accuracy (AC), sensitivity (SE), and F1-measurement (F1) by comparing to artificial neural networks (ANN), k-nearest neighbor (kNN), Fast Region-based Convolution Neural Network (Fast R-CNN), visual geometry group (VGG16), scaled-conjugate-gradient convolution neural networks (SCG-CNN), GoogleNet, AlexNet, ResNet-50–177, and Inception-v3. The final prediction of class accuracy is 95.17%. Foot type classification is vital for producing comfortable shoes in the industry. [ABSTRACT FROM AUTHOR]

Careful monitoring of plant conditions and their diagnosis are necessary, but a human cannot control a large area of land where the crop grows. This paper proposes the solution of this problem. Early diagnosis and accurate detection of plant leaf diseases can prevent the spread of the disease. In the last decade, machine learning methods and image classification tools have been used to identify and diagnose plant diseases. This paper proposes an accurate approach to identify plant leaf diseases based on the deep convolutional neural network and gated recurrent units. The PlantVillage dataset of damaged and healthy plant leaves images is used. The proposed model is trained to identify common plant leaf diseases of 14 species. The experimental results are compared to other well-known models. This study shows that the proposed model based on deep learning provides the best solution in the diagnosis of plant diseases with high accuracy, and that the gated recurrent unit neural network considered as a classifier can improve the accuracy of the convolutional neural network model. The comparison results demonstrated that the proposed approach achieved higher performance than other models. [ABSTRACT FROM AUTHOR]

Deep learning techniques, in particular generative models, have taken on great importance in medical image analysis. This paper surveys fundamental deep learning concepts related to medical image generation. It provides concise overviews of studies which use some of the latest state-of-the-art models from last years applied to medical images of different injured body areas or organs that have a disease associated with (e.g., brain tumor and COVID-19 lungs pneumonia). The motivation for this study is to offer a comprehensive overview of artificial neural networks (NNs) and deep generative models in medical imaging, so more groups and authors that are not familiar with deep learning take into consideration its use in medicine works. We review the use of generative models, such as generative adversarial networks and variational autoencoders, as techniques to achieve semantic segmentation, data augmentation, and better classification algorithms, among other purposes. In addition, a collection of widely used public medical datasets containing magnetic resonance (MR) images, computed tomography (CT) scans, and common pictures is presented. Finally, we feature a summary of the current state of generative models in medical image including key features, current challenges, and future research paths. [ABSTRACT FROM AUTHOR]

Machine Learning is an important field of research in current trends. The extended field of machine learning is Deep Learning and is used for various research areas such as neural networks, image and signal processing, pattern recognition, etc. The handwritten digit recognition is an important task or process included in various applications such as car number plate recognition, staff identity number detection, etc. This paper proposed the design and analysis of various deep learning algorithms such as deep neural networks, convolutional neural networks, LeNet-5, AlexNet and MiniVGGNet for handwritten digit recognition using MNIST dataset. [ABSTRACT FROM AUTHOR]

Despite the enormous performance of deep neural networks (DNNs), recent studies have shown their vulnerability to adversarial examples (AEs), i.e., carefully perturbed inputs designed to fool the targeted DNN. Currently, the literature is rich with many effective attacks to craft such AEs. Meanwhile, many defense strategies have been developed to mitigate this vulnerability. However, these latter showed their effectiveness against specific attacks and does not generalize well to different attacks. In this paper, we propose a framework for defending DNN classifier against adversarial samples. The proposed method is based on a two-stage framework involving a separate detector and a denoising block. The detector aims to detect AEs by characterizing them through the use of natural scene statistic (NSS), where we demonstrate that these statistical features are altered by the presence of adversarial perturbations. The denoiser is based on block matching 3D (BM3D) filter fed by an optimum threshold value estimated by a convolutional neural network (CNN) to project back the samples detected as AEs into their data manifold. We conducted a complete evaluation on three standard datasets, namely MNIST, CIFAR-10 and Tiny-ImageNet. The experimental results show that the proposed defense method outperforms the state-of-the-art defense techniques by improving the robustness against a set of attacks under black-box, gray-box and white-box settings. The source code is available at: https://github.com/kherchouche-anouar/2DAE. [ABSTRACT FROM AUTHOR]

Age estimation from face images can be profitably employed in several applications, ranging from digital signage to social robotics, from business intelligence to access control. Only in recent years, the advent of deep learning allowed for the design of extremely accurate methods based on convolutional neural networks (CNNs) that achieve a remarkable performance in various face analysis tasks. However, these networks are not always applicable in real scenarios, due to both time and resource constraints that the most accurate approaches often do not meet. Moreover, in case of age estimation, there is the lack of a large and reliably annotated dataset for training deep neural networks. Within this context, we propose in this paper an effective training procedure of CNNs for age estimation based on knowledge distillation, able to allow smaller and simpler "student" models to be trained to match the predictions of a larger "teacher" model. We experimentally show that such student models are able to almost reach the performance of the teacher, obtaining high accuracy over the LFW+, LAP 2016 and Adience datasets, but being up to 15 times faster. Furthermore, we evaluate the performance of the student models in the presence of image corruptions, and we demonstrate that some of them are even more resilient to these corruptions than the teacher model. [ABSTRACT FROM AUTHOR]

Plant diseases are the major factor behind production loss in agriculture. The traditional manual methods for disease detection in plants involve expert knowledge that may be biased. The modern computing techniques and image processing can assist non-experts in plant disease management. Recently, deep learning techniques have observed remarkable success in image-based health assessment of plants. In this paper, the state-of-the-art pre-trained convolutional neural network (CNN) models are fine-tuned to detect and diagnose the diseases in apple crop using digital images. The experiments are performed on a publicly available dataset PlantVillage. The dataset consists of three classes of apple diseases including Scab, Black Rot, and Cedar Rust, and one class of Healthy leaves. The experimental results on ten well-known CNN models DenseNet201, DenseNet169, InceptionV3, InceptionResNetV2, MobileNet, MobileNetV2, ResNet50, VGG16, VGG19, and Xception showed that deep learning techniques can accurately discriminate the apple diseases. DenseNet201 outperformed the other models with an accuracy of 98.75%. The high accuracy shows that CNN-based methods could be a useful alternative to the conventional methods. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) has become a significant area of research in human behavior analysis, human–computer interaction, and pervasive computing. Recently, deep learning (DL)-based methods have been applied successfully to time-series data generated from smartphones and wearable sensors to predict various activities of humans. Even though DL-based approaches performed very well in activity recognition, they are still facing challenges in handling time series data. Several issues persist with time-series data, such as difficulties in feature extraction, heavily biased data, etc. Moreover, most of the HAR approaches rely on manual feature engineering. In this paper, to design a robust classification model for HAR using wearable sensor data, a hybrid of convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) is used. The proposed multibranch CNN-BiLSTM network does automatic feature extraction from the raw sensor data with minimal data pre-processing. The use of CNN and BiLSTM makes the model capable of learning local features as well as long-term dependencies in sequential data. The different filter sizes used in the proposed model can capture various temporal local dependencies and thus helps to improve the feature extraction process. To evaluate the model performance, three benchmark datasets, i.e., WISDM, UCI-HAR, and PAMAP2, are utilized. The proposed model has achieved 96.05%, 96.37%, and 94.29% accuracies on WISDM, UCI-HAR, and PAMAP2 datasets, respectively. The obtained experimental results demonstrate that the proposed model outperforms the other compared approaches. [ABSTRACT FROM AUTHOR]

In the previous works on heat stress detection, various state-of-the-art machine learning techniques had been used to detect stress patterns from electroencephalographic (EEG) signals. Since the handcrafted feature engineering-based approaches pose certain limitations and are sensitive to transform the nonlinearities of EEG, deep learning techniques have drawn attention in the domain of EEG-based applications. Moreover, existing approaches for stress detection consider the whole frequency band (delta to gamma) which conceals the redundant and lossy information that increased the false detection rate. Also, these approaches were implemented only to identify the stress in binary classes and confined their application to identifying the level of stress. Therefore, a multi-scale channel-wise convolution-based multi-level heat stress assessment has been proposed in this paper. The novel contributions of this work are (1) designing a multi-scale convolutional neural network (MS-CNN) for extracting precise information from individual frequency bands of EEG, (2) use of sparse connectivity to reduce the redundant information and increase the performance with less trainable parameters, and (3) multi-level heat stress assessment for precise identification of the severity of stress. The proposed approaches were evaluated on pre-recorded data of 10 rats in a simulated laboratory environment. The high accuracy of approximately 96–97% and 90–92% has been achieved for binary and multi-level stress detection. There is approximately a 6 to 50% reduction in the trainable parameters with a 2% improvement of accuracy achieved on adopting channel-wise convolution over MS-CNN and deep convolutional neural network (DCNN). This demonstrates the effectiveness of the adopted multiscale feature extraction and sparse connectivity to improve the performance with a less complex model. [ABSTRACT FROM AUTHOR]

Diabetic Retinopathy (DR), one of the most progressive sight-threatening diseases caused by the long-term diabetic condition, can lead to vision impairment and blindness later. Early diagnosis and timely treatment help control and avert DR from its progression. However, manual grading is exceptionally challenging and arduous due to the complex anatomical features in the retina. Therefore, developing an automated diagnostic method for screening DR is obligatory. This paper proposes a deep learning-enabled optimized feature selection approach to classify the stage of DR severity in the fundus image. At first, a pre-processing phase eradicates the noise and improves the contrast in the retinal fundus image. Then, blood vessel segmentation is performed using the Coherence Enhancing Energy Based Regularized Level Set Evolution method in the green channel fundus image. Subsequently, the optic disk is segmented with Canny Anisotropic Diffusion filter and morphological transformations. Next, the candidate lesion region is detected using an Attention-based Fusion Network (AFU-Net). Then, shape and texture features are extracted, and then, the optimal subset of features is selected using the Improved Harris Hawk Optimization algorithm. Finally, a deep Convolutional Neural Network classifies the DR stages, and the model weight is updated using the same algorithm. The proposed method achieved superior performance in two benchmark public datasets compared with the existing state-of-the-art methods using F1-score, accuracy, sensitivity, and specificity measures. [ABSTRACT FROM AUTHOR]

As a low-frequency and smooth signal, the bias field has a certain destructive effect on magnetic resonance (MR) images and is the main obstacle for doctors' diagnosis and image processing (such as segmentation, texture analysis, and registration). Before analyzing a damaged MR image, a preprocessing step is required to correct the bias field in the image. Unlike traditional bias field removal algorithms based on signal models and a priori assumptions, deep learning methods do not require precisely modeling signals and bias fields and do not need to adjust parameters. An MR image with the bias field is input and the corrected MR image is output after the deep neural network being trained on a large training set. In this paper, we propose taking the original image and the local feature images of the bias field in multiple frequency bands obtained by a Log-Gabor filter bank as input, correcting the bias field of a brain MR image through a deep separable convolutional neural network. Meanwhile, to speed up the training process and improve bias correction performance, we apply residual learning and batch normalization. We conducted the same test on BrainWeb simulation database and Human Connectome Project real data set, the consistency of qualitative and quantitative evaluation shows that our training model demonstrates better performance than the traditional state-of-the-art N4 and non-iterative multi-scale (NIMS) methods. Especially for the images with high-intensity non-uniformity level, the bias field has been well corrected. [ABSTRACT FROM AUTHOR]

Handwriting recognition is an important area in pattern recognition. For many years, Tai Le has been widely used in Southwest China and Southeast Asia, which makes it of great interest for recognition research. The characteristics of the highly similar characters in Tai Le, such as its large proportion of similar characters and the randomness of its writing, bring great challenges to the task of recognition. In this paper, a method based on ensemble deep learning for offline handwritten Tai Le characters is proposed. First, the handwritten Tai Le character dataset SDH2019.2 was constructed and preprocessed. Then, an ensemble deep convolutional neural network (EDCNN) model was constructed by using a stacking strategy. Thirty deep neural network (DNN) and logistic regression algorithms were integrated into a strong Tai Le classifier by stacking. Experiments showed that the proposed model is competitive with the base DNN model and other ensemble models. The results indicate that the performance of Tai Le recognition by the stacking ensemble-based deep neural network model is high, with an accuracy of 98.85%. Additionally, its precision, recall and F1-score of 98.87%, 98.85% and 98.85%, respectively, are superior to those of other classic neural network models. To verify the general applicability of EDCNN, its effectiveness was also verified by recognizing MNIST handwritten digits and Devanagari handwritten characters. [ABSTRACT FROM AUTHOR]

The monitoring of physical activities and recognition of motion disorders belong to important diagnostical tools in neurology and rehabilitation. The goal of the present paper is in the contribution to this topic by (1) analysis of accelerometric signals recorded by wearable sensors located at specific body positions and by (2) implementation of deep learning methods to classify signal features. This paper uses the general methodology to analysis of accelerometric signals acquired during cycling at different routes followed by the global positioning system. The experimental dataset includes 850 observations that were recorded by a mobile device in the spine area (L3 verterbra) for cycling routes with the different slope. The proposed methodology includes the use of deep learning convolutional neural networks with five layers applied to signal values transformed into the frequency domain without specification of any signal features. The accuracy of discrimination between different motion patterns for the uphill and downhill cycling and recognition of 4 classes associated with different route slopes was 96.6% with the loss criterion of 0.275 for sigmoidal activation functions. These results were compared with those evaluated for selected sets of features estimated for each observation and classified by the support vector machine, Bayesian methods, and the two-layer neural network. The best cross-validation error of 0.361 was achieved for the two-layer neural network model with the sigmoidal and softmax transfer functions. Our methodology suggests that deep learning neural networks are efficient in the assessment of motion activities for automated data processing and have a wide range of applications, including rehabilitation, early diagnosis of neurological problems, and possible use in engineering as well. [ABSTRACT FROM AUTHOR]

Molecular property prediction (MPP) is vital in drug discovery and drug reposition. Deep learning-based MPP models capture molecular property-related features from various molecule representations. In this paper, we propose a molecule sequence embedding and prediction model facing with MPP task. We pre-trained a bi-directional encoder representations from Transformers (BERT) encoder to obtain the semantic representation of compound fingerprints, called Fingerprints-BERT (FP-BERT), in a self-supervised learning manner. Then, the encoded molecular representation by the FP-BERT is input to the convolutional neural network (CNN) to extract higher-level abstract features, and the predicted properties of the molecule are finally obtained through fully connected layer for distinct classification or regression MPP tasks. Comparison with the baselines shows that the proposed model achieves high prediction performance on all of the classification tasks and regression tasks. [ABSTRACT FROM AUTHOR]

Artificial neural network (ANN) is one of the technologies used for emerging real-world problems. Activation functions (AF) are used in deep learning architectures to make decisions in the hidden and output layers. An AF influences the dynamics of training and performance of an ANN. This paper proposes a novel AF called E-Tanh by extending the Tanh AF for ANN models. When used in three shallow ANN models with 2, 4, and 6 hidden layers with the Modified National Institute of Standards and Technology database (MNIST), it gave a maximum accuracy of 99.5% for handwritten digits classification than the results produced by the existing AFs. The proposed AF E-Tanh outperformed other existing and well-known AFs ReLU, Sigmoid, Tanh, Swish, and E-Swish. In deep ANNs with 5–50 hidden layers, the proposed AF's prediction accuracy of up to 25 hidden layers was higher than the existing AFs on the MNIST database. When used in shallow CNN for the CIFAR10 database, it outperformed other AFs. When used in wide residual network, it performed poorly in classifying images in CIFAR10 and CIFAR100 database but performed competitively with all other AFs in CNN models on the MNIST dataset. [ABSTRACT FROM AUTHOR]

Breast cancer is a common malignancy and a leading cause of cancer-related deaths in women worldwide. Its early diagnosis can significantly reduce the morbidity and mortality rates in women. To this end, histopathological diagnosis is usually followed as the gold standard approach. However, this process is tedious, labor-intensive, and may be subject to inter-reader variability. Accordingly, an automatic diagnostic system can assist to improve the quality of diagnosis. This paper presents a deep learning approach to automatically classify hematoxylin-eosin-stained breast cancer microscopy images into normal tissue, benign lesion, in situ carcinoma, and invasive carcinoma using our collected dataset. Our proposed model exploited six intermediate layers of the Xception (Extreme Inception) network to retrieve robust and abstract features from input images. First, we optimized the proposed model on the original (unnormalized) dataset using 5-fold cross-validation. Then, we investigated its performance on four normalized datasets resulting from Reinhard, Ruifrok, Macenko, and Vahadane stain normalization. For original images, our proposed framework yielded an accuracy of 98% along with a kappa score of 0.969. Also, it achieved an average AUC-ROC score of 0.998 as well as a mean AUC-PR value of 0.995. Specifically, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. For normalized images, the proposed architecture performed better for Makenko normalization compared to the other three techniques. In this case, the proposed model achieved an accuracy of 97.79% together with a kappa score of 0.965. Also, it attained an average AUC-ROC score of 0.997 and a mean AUC-PR value of 0.991. Especially, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. These results demonstrate that our proposed model outperformed the baseline AlexNet as well as state-of-the-art VGG16, VGG19, Inception-v3, and Xception models with their default settings. Furthermore, it can be inferred that although stain normalization techniques offered competitive performance, they could not surpass the results of the original dataset. [ABSTRACT FROM AUTHOR]

In recent years, the Internet of Things (IoT) has received a lot of attention. It has been used in many applications such as the control industry, industrial plants, and medicine. In this regard, a fundamental necessity is to implement security in IoT. To this end, Network intrusion detection systems (NIDSs) have been recently in the detection of network attacks and threats. Currently, these systems use a variety of deep learning (DL) models such as the convolutional neural networks to improve the detection of attacks. However, almost all current DL-based NIDSs are made up of many layers, and therefore, they need a lot of processing resources because of their high number of parameters. On the other hand, due to the lack of processing resources, such inefficient DL models are unusable in IoT devices. This paper presents a very accurate NIDS that is named DFE, and it uses a very lightweight and efficient neural network based on the idea of deep feature extraction. In this model, the input vector of the network is permuted in a 3D space, and its individual values are brought close together. This allows the model to extract highly discriminative features using a small number of layers without the need to use large 2D or 3D convolution filters. As a result, the network can achieve an accurate classification using a significantly small number of required calculations. This makes the DFE ideal for real-time intrusion detection by IoT devices with limited processing capabilities. The efficacy of the DFE has been evaluated using three popular public datasets named UNSW-NB15, CICIDS2017, and KDDCup99, and the results show the superiority of the proposed model over the state-of-the-art algorithms [ABSTRACT FROM AUTHOR]

Multimedia IoT (M-IoT) is an emerging type of Internet of things (IoT) relaying multimedia data (images, videos, audio and speech, etc.). The rapid growth of M-IoT devices enables the creation of a massive volume of multimedia data with different characteristics and requirements. With the development of artificial intelligence (AI), AI-based multimedia IoT systems have been recently designed and deployed for various video-based services for contemporary daily life, like video surveillance with high definition (HD) and ultra-high definition (UHD) and mobile multimedia streaming. These new services need higher video quality in order to meet the quality of experience (QoE) required by the users. Versatile video coding (VVC) is the new video coding standard that achieves significant coding efficiency over its predecessor high-efficiency video coding (HEVC). Moreover, VVC can achieve up to 30% BD rate savings compared to HEVC. Inspired by the rapid advancements in deep learning, we propose in this paper a wide-activated squeeze-and-excitation deep convolutional neural network (WSE-DCNN) technique-based video quality enhancement for VVC. Therefore, we replace the conventional in-loop filtering in VVC by the proposed WSE-DCNN model that eliminates the compression artifacts in order to improve visual quality and hence increase the end user QoE. The obtained results prove that the proposed in-loop filtering technique achieves - 2.85 %, - 8.89 %, and - 10.05 % BD rate reduction for luma and both chroma components under random access configuration. Compared to the traditional CNN-based filtering approaches, the proposed WSE-DCNN-based in-loop filtering framework achieves efficient performance in terms of RD cost. [ABSTRACT FROM AUTHOR]

Facial emotion recognition extracts the human emotions from the images and videos. As such, it requires an algorithm to understand and model the relationships between faces and facial expressions and to recognize human emotions. Recently, deep learning models are utilized to improve the performance of facial emotion recognition. However, the deep learning models suffer from the overfitting issue. Moreover, deep learning models perform poorly for images which have poor visibility and noise. Therefore, in this paper, an efficient deep learning-based facial emotion recognition model is proposed. Initially, contrast-limited adaptive histogram equalization (CLAHE) is applied to improve the visibility of input images. Thereafter, a modified joint trilateral filter is applied to the obtained enhanced images to remove the impact of impulsive noise. Finally, an efficient deep convolutional neural network is designed. Adam optimizer is also utilized to optimize the cost function of deep convolutional neural networks. Experiments are conducted by using the benchmark dataset and competitive human emotion recognition models. Comparative analysis demonstrates that the proposed facial emotion recognition model performs considerably better compared to the competitive models [ABSTRACT FROM AUTHOR]

The banking sector is on the eve of a serious transformation and the thrust behind it is artificial intelligence (AI). Novel AI applications have been already proposed to deal with challenges in the areas of credit scoring, risk assessment, client experience and portfolio management. One of the most critical challenges in the aforementioned sector is fraud detection upon streams of transactions. Recently, deep learning models have been introduced to deal with the specific problem in terms of detecting and forecasting possible fraudulent events. The aim is to estimate the unknown distribution of normal/fraudulent transactions and then to identify deviations that may indicate a potential fraud. In this paper, we elaborate on a novel multistage deep learning model that targets to efficiently manage the incoming streams of transactions and detect the fraudulent ones. We propose the use of two autoencoders to perform feature selection and learn the latent data space representation based on a nonlinear optimization model. On the delivered significant features, we subsequently apply a deep convolutional neural network to detect frauds, thus combining two different processing blocks. The adopted combination has the goal of detecting frauds over the exposed latent data representation and not over the initial data. [ABSTRACT FROM AUTHOR]

Malignant melanoma is one of the most dreadful skin cancer types caused by the abnormal growth of melanocyte cells. Deep convolutional neural network (CNN) models are becoming prominent for the automated diagnosis of melanoma from dermoscopic images. Although being incredibly accurate, the "black-box" nature of deep CNN models due to the lack of proper interpretability still prevents their wide-spread use in clinical settings. This paper proposes an explainable CNN-based stacked ensemble framework to detect melanoma skin cancer at earlier stages. In the stacking ensemble framework, the transfer learning concept is used where multiple CNN sub-models that perform the same classification task are assembled. A new model called a meta-learner uses all the sub-models' predictions and generates the final prediction results. The model is evaluated using an open-access dataset containing both benign and malignant melanoma images. An explainability method is developed by shapely adaptive explanations to produce heatmaps that visualize the areas of melanoma images that are most indicative of the disease. This provides interpretability of our model's decision in a manner understandable to dermatologists. Evaluation results show the effectiveness of our ensemble model with a high degree of accuracy (95.76%), sensitivity (96.67%), and AUC (0.957). [ABSTRACT FROM AUTHOR]

Feature selection, reducing number of input variables to develop classification model, is an important process to reduce computational and modeling complexity and affects the performance of image process. In this paper, we have proposed new statistical approaches for feature selection based on sample selection. We have applied our new approaches to grapevine leaves data that possess properties of shape, thickness, featheriness, and slickness that are investigated in images. To analyze such kind of data by using image process, thousands of features are created and selection of features plays important role to predict the outcome properly. In our numerical study, convolutional neural networks have been used as feature extractors and then obtained features from the last average pooling layer to detect the type of grapevine leaves from images. These features have been reduced by using our suggested four statistical methods: simple random sampling, ranked set sampling, extreme ranked set sampling, moving extreme ranked set sampling. Then, selected features have been classified with artificial neural network and we obtained the best accuracy of 97.33% with our proposed approaches. Based on our empirical analysis, it has been determined that the proposed approach exhibits efficacy in the classification of grapevine leaf types. Furthermore, it possesses the potential for integration into various computational devices. [ABSTRACT FROM AUTHOR]

Based on physics-informed deep learning method, the deep learning model is proposed for thermal fluid fields reconstruction. This method applied fully-connected layers to establish the mapping function from design variables and space coordinates to physical fields of interest, and then the performance characteristics Nusselt number Nu and Fanning friction factor f can be calculated from the reconstructed fields. Compared with reconstruction model based on convolutional neural network, the improved model shows no constrains on mesh generation and it improves the physical interpretability by introducing conservation laws in loss functions. To validate this method, the forced convection of the water-Al2O3 nanofluids is utilized to construct training dataset. As shown in this paper, this deep neural network can reconstruct the physical fields and consequently the performance characteristics accurately. In the comparisons with other classical machine learning methods, our reconstruction model is superior for predicting performance characteristics. In addition to the effect of training size on prediction power, the extrapolation performance (an important but rarely investigated issue) for important design parameters are also explored on unseen testing datasets. [ABSTRACT FROM AUTHOR]

A Data Center Network (DCN) is composed of a large number of computing and storage nodes that are interconnected by well-organized switches. The Software-Defined Networking (SDN) based DCN (SD-DCN) improves resource utilization and provides virtual network access by separating the data plane and control plane of DCN. However, the routing strategies in current SD-DCN systems are based on traditional mechanisms that lack in real-time modification and are less efficient in resource utilization. To overcome these limitations, Convolutional Neural Network (CNN) deep learning model is proposed in this paper to improve the routing computation in SD-DCN, i.e., FAT-tree topology. The CNN deep learning model gives intelligent paths according to online training of traffic patterns. Moreover, the achieved network performance is compared with specific existing routing algorithms for SD-DCN. It is observed that the average network throughput is almost doubled for hot-spot traffic as compared with existing routing algorithms OSPF and FlowDCN. The experimental results show that, compared to ANN, the proposed model has increased the average network throughput by approximately 40%. Also, the proposed CNN model has outperformed the Artificial Neural Network (ANN) model in terms of average network delay and packet loss rate. Similarly, the overall bandwidth utilization is achieved by approximately 70% as compared to existing mechanisms. [ABSTRACT FROM AUTHOR]

Accurate sensing and understanding of gestures can improve the quality of human–computer interaction and show great theoretical significance and application potentials in the fields of smart home, assisted medical care and virtual reality. WiFi channel state information (CSI)-based device-free wireless gesture recognition requires no sensors and has a series of advantages such as permission for non-line-of-sight scenario, low cost, preserving for personal privacy and working in the dark night. Although most of the current WiFi CSI-based gesture recognition approaches can achieve good performance, they are difficult to adapt to the new domains. Therefore, this paper proposes ML-WiGR, a novel approach for device-free gesture recognition in cross-domain applications. ML-WiGR applies convolutional neural networks (CNN) and long short-term memory (LSTM) neural networks as the basic model for gesture recognition to extract spatial and temporal features. Combined with the meta-learning training mechanism, ML-WiGR can dynamically adjust the learning rate and meta-learning rate in training process adaptively and optimize the initial parameters of a basic model for gesture recognition, only using a few samples and several iterations to adapt to the new domain. In the experiments, the approach is tested under a variety of scenarios. The results show that ML-WiGR can achieve comparable performance against existing approaches with only a small number of samples for training in cross-domains. [ABSTRACT FROM AUTHOR]

This paper presents a novel method for improving the invariance of convolutional neural networks (CNNs) to selected geometric transformations in order to obtain more efficient image classifiers. A common strategy employed to achieve this aim is to train the network using data augmentation. Such a method alone, however, increases the complexity of the neural network model, as any change in the rotation or size of the input image results in the activation of different CNN feature maps. This problem can be resolved by the proposed novel convolutional neural network models with geometric transformations embedded into the network architecture. The evaluation of the proposed CNN model is performed on the image classification task with the use of diverse representative data sets. The CNN models with embedded geometric transformations are compared to those without the transformations, using different data augmentation setups. As the compared approaches use the same amount of memory to store the parameters, the improved classification score means that the proposed architecture is more optimal. [ABSTRACT FROM AUTHOR]

Gating transformation demonstrates great potential in recent deep convolutional neural networks design, enriching the feature representation and alleviating noisy signals by modeling the inter-channel dependencies using learnable parameters. However, the utilization of scaling approaches to reduce the redundancy of the hand-crafted attention mechanism has rarely been investigated. This paper proposes a novel scaled gated convolution that enables attention-enhanced CNNs to overcome the paradox between performance and redundancy. Our scaled gated convolution is a simple and effective alternative compared with both vanilla convolution and attention-enhanced convolutions, which can be easily applied to modern CNNs in a plug-and-play manner. Exhaustive experiments demonstrate that stacking scaled gated convolutions in baselines can significantly improve the performance in a broad range of visual recognition tasks, including image recognition, object detection, instance segmentation, keypoint detection, and panoptic segmentation, while obtaining a better trade-off between performance and attentive redundancy. [ABSTRACT FROM AUTHOR]

Different gestures have different action speeds, directions, and trajectories that can cause distinctive effects on the propagation of WiFi signals. In this paper, we present a new approach that uses deep transfer learning techniques to recognize gestures based on the channel state information (CSI) extracted from WiFi signals. Firstly, the CSI streams of gestures are captured and the gesture segments are extracted based on the CSI amplitude changes, and then the WiFi-based gesture recognition problem is innovatively converted to an image classification problem by expressing CSI streams as an image matrix. After that, two deep transfer learning methods are applied to recognize gestures using high-level features extracted by deep convolutional neural network (CNN) and fine-tuned CNN models. We evaluated our method using a collected dataset with 12 gestures in two environments, and the experimental results demonstrated that the proposed method outperformed other state-of-the-art WiFi-based gesture recognition methods. [ABSTRACT FROM AUTHOR]

The deep learning techniques have achieved great success in the application of human activity recognition (HAR). In this paper, we propose a technique for HAR that utilizes the RGB and skeleton information with the help of a convolutional neural network (Convnet) and long short-term memory (LSTM) as a recurrent neural network (RNN). The proposed method has two parts: first, motion representation images like motion history image (MHI) and motion energy image (MEI) have been created from the RGB videos. The convnet has been trained, using these images with feature-level fusion. Second, the skeleton data have been utilized with a proposed algorithm that develops skeleton intensity images, for three views (top, front and side). Each view is first analyzed by a convnet, that generates the set of feature maps, which are fused for further analysis. On top of convnet sub-networks, LSTM has been used to exploit the temporal dependency. The softmax scores from these two independent parts are later combined at the decision level. Apart from the given approach for HAR, this paper also presents a strategy that utilizes the concept of cyclic learning rate to develop a multi-modal neural network by training the model only once to make the system more efficient. The suggested approach privileges for the perfect utilization of RGB and skeleton data available from an RGB-D sensor. The proposed approach has been tested on three famous and challenging multimodal datasets which are UTD-MHAD, CAD-60 and NTU-RGB + D120. Results have shown that the stated method gives a satisfactory result as compared to the other state-of-the-art systems. [ABSTRACT FROM AUTHOR]

Smart cities are shifting the presence of people from physical world to cyber world (cyberspace). Along with the facilities for societies, the troubles of physical world, such as bullying, aggression and hate speech, are also taking their presence emphatically in cyberspace. This paper aims to dig the posts of social media to identify the bullying comments containing text as well as image. In this paper, we have proposed a unified representation of text and image together to eliminate the need for separate learning modules for image and text. A single-layer Convolutional Neural Network model is used with a unified representation. The major findings of this research are that the text represented as image is a better model to encode the information. We also found that single-layer Convolutional Neural Network is giving better results with two-dimensional representation. In the current scenario, we have used three layers of text and three layers of a colour image to represent the input that gives a recall of 74% of the bullying class with one layer of Convolutional Neural Network. [ABSTRACT FROM AUTHOR]

In this paper, we investigate Hebbian learning strategies applied to Convolutional Neural Network (CNN) training. We consider two unsupervised learning approaches, Hebbian Winner-Takes-All (HWTA), and Hebbian Principal Component Analysis (HPCA). The Hebbian learning rules are used to train the layers of a CNN in order to extract features that are then used for classification, without requiring backpropagation (backprop). Experimental comparisons are made with state-of-the-art unsupervised (but backprop-based) Variational Auto-Encoder (VAE) training. For completeness,we consider two supervised Hebbian learning variants (Supervised Hebbian Classifiers—SHC, and Contrastive Hebbian Learning—CHL), for training the final classification layer, which are compared to Stochastic Gradient Descent training. We also investigate hybrid learning methodologies, where some network layers are trained following the Hebbian approach, and others are trained by backprop. We tested our approaches on MNIST, CIFAR10, and CIFAR100 datasets. Our results suggest that Hebbian learning is generally suitable for training early feature extraction layers, or to retrain higher network layers in fewer training epochs than backprop. Moreover, our experiments show that Hebbian learning outperforms VAE training, with HPCA performing generally better than HWTA. [ABSTRACT FROM AUTHOR]

Forecasting has always been the backbone of planning studies in the power system. With the advent of the restructured power system and also the integration of renewable energy sources, it has become more challenging. Accurate load forecasting leads to a reduction in supply–demand mismatch, whereas solar insolation/wind speed forecasting improves generation side planning. With the rapid growth in computer-based intelligence, self-learning mechanisms together with continuously growing volume of the dataset and affordable cost-effective data storage options, deep learning performance keeps getting better and has made a huge impact to solve the complex problems associated with the power system network. In this paper, we propose a classical long short-term memory neural network model for a combined day-ahead load and solar insolation forecasting of residential consumers equipped with rooftop PV based on weather classification. The effectiveness of the proposed model based on mean absolute error, mean bias error and coefficient of correlation (R-Squared) is compared with that of the standard artificial neural network and convolutional neural network models. [ABSTRACT FROM AUTHOR]

Online banking and financial services using mobile applications are seeing a persistent growth among customers, who are using these for their financial transactions. This rise in the use of such applications in smart devices has increased security concerns. There is need for secure mechanisms to prevent fraud and protect personal information. This paper investigates the use of biometric identification in banking and financial services, which leverage the use of smartphones and tablets. While customer engagement and brand loyalty are important concerns, these services are making use of biometric authentication to make customer interactions more secure. However, as technology is growing rapidly, spoofing attacks are becoming common. In this paper, authors have proposed a robust framework to detect spoofing attacks in fingerprint recognition. The process of spoofing detection involves contrast enhancement using histogram equalization and a deep convolutional neural network architecture. Authors have validated the results on various biometric spoofing benchmarks, each one containing real and spoofed samples of user fingerprints. The results indicate that our proposed framework performs better as evaluated against other existing pre-trained CNN models and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Modeling the semivariogram to characterize spatial continuity requires expert geostatistical knowledge and domain expertise about the spatial phenomenon of interest. Moreover, although practitioners may have experience in semivariograms, their interpretations may vary due to experimental semivariogram noise and ambiguity. In general, modeling semivariograms remains highly subjective. This paper presents a data-driven, deep learning-based automated semivariogram modeling method known as automatic semivariogram modeling with convolution-based deep learning (ASMC) that improves the utilization of available spatial information to reduce the subjectivity of semivariogram modeling. Training models are generated by sequential Gaussian simulation (SGS) and labeled with their associated semivariogram parameters (i.e., maximum correlation length, aspect ratio of major and minor direction ranges, and azimuth of major direction). ASMC consists of two convolutional neural networks (CNNs). The first CNN model maps the sparse spatial samples to the exhaustive SGS-derived spatial models, and the second CNN maps the SGS spatial model to the semivariogram parameters. Both CNNs are trained with realistic spatial training data, and their validity is also checked with validation data withheld from training. Two-dimensional synthetic, but realistic, case studies demonstrate that the first CNN successfully learns the spatial characteristics among spatial data and generates realistic subsurface model estimates. The second CNN learns the spatial context of the estimated subsurface model and successfully predicts the semivariogram parameters with greater than 96% accuracy. The proposed machine, deep learning-based workflow improves the utilization and objectivity of spatial data in semivariogram-based spatial continuity modeling. With the optimal design of the experiment for training and tuning of model hyperparameters, this method may be generalized for application for a wide range of spatial modeling projects. [ABSTRACT FROM AUTHOR]