Search

Recent technological developments have enabled computers to identify and categorize facial expressions to determine a person's emotional state in an image or a video. This process, called "Facial Expression Recognition (FER)", has become one of the most popular research areas in computer vision. In recent times, deep FER systems have primarily concentrated on addressing two significant challenges: the problem of overfitting due to limited training data availability, and the presence of expression-unrelated variations, including illumination, head pose, image resolution, and identity bias. In this paper, a comprehensive survey is provided on deep FER, encompassing algorithms and datasets that offer insights into these intrinsic problems. Initially, this paper presents a detailed timeline showcasing the evolution of methods and datasets in deep facial expression recognition (FER). This timeline illustrates the progression and development of the techniques and data resources used in FER. Then, a comprehensive review of FER methods is introduced, including the basic principles of FER (components such as preprocessing, feature extraction and classification, and methods, etc.) from the pro-deep learning era (traditional methods using handcrafted features, i.e., SVM and HOG, etc.) to the deep learning era. Moreover, a brief introduction is provided related to the benchmark datasets (there are two categories: controlled environments (lab) and uncontrolled environments (in the wild)) used to evaluate different FER methods and a comparison of different FER models. Existing deep neural networks and related training strategies designed for FER, based on static images and dynamic image sequences, are discussed. The remaining challenges and corresponding opportunities in FER and the future directions for designing robust deep FER systems are also pinpointed. [ABSTRACT FROM AUTHOR]

In this paper, a method based on transfer learning is proposed to recover the three-dimensional shape of cultural relics faces from a single old photo. It can simultaneously reconstruct the three-dimensional facial structure and align the texture of the cultural relics with fewer training samples. The UV position map is used to represent the three-dimensional shape in space and act as the output of the network. A convolutional neural network is used to reconstruct the UV position map from a single 2D image. In the training process, the human face data is used for pre-training, and then a small amount of artifact data is used for fine-tuning. A deep learning model with strong generalization ability is trained with fewer artifact data, and a three-dimensional model of the cultural relic face can be reconstructed from a single old photograph. The methods can train more complex deep networks without a large amount of cultural relic data, and no over-fitting phenomenon occurs, which effectively solves the problem of fewer cultural relic samples. The method is verified by restoring a Chinese Terracotta Warrior with facial cracks and a Buddha with a broken nose. Other applications can be used in the fields such as texture recovery, facial feature extraction, and three-dimensional model estimation of the damaged cultural relics or sculptures in the photos. [ABSTRACT FROM AUTHOR]

Background: Glaucoma, the second leading cause of global blindness, demands timely detection due to its asymptomatic progression. This paper introduces an advanced computerized system, integrates Machine Learning (ML), convolutional neural networks (CNNs), and image processing for accurate glaucoma detection using medical imaging data, surpassing prior research efforts. Method: Developing a hybrid glaucoma detection framework using CNNs (ResNet50, VGG-16) and Random Forest. Models analyze pre-processed retinal images independently, and post-processing rules combine predictions for an overall glaucoma impact assessment. Result: The hybrid framework achieves a significant 95.41% accuracy, with precision and recall at 99.37% and 88.37%, respectively. The F1 score, balancing precision and recall, reaches a commendable 93.52%. These results highlight the robustness and effectiveness of the hybrid framework in accurate glaucoma diagnosis. Conclusion: In summary, our research presents an innovative hybrid framework combining CNNs and traditional ML models for glaucoma detection. Using ResNet50, VGG-16, and Random Forest in an ensemble approach yields remarkable accuracy, precision, recall, and F1 score. These results showcase the methodology's potential to enhance glaucoma diagnosis, emphasizing its promising role in early detection and preventing irreversible vision loss. The integration of ML and DNNs in medical imaging analysis suggests a valuable path for future advancements in ophthalmic healthcare. [ABSTRACT FROM AUTHOR]

Ship detection aims to automatically identify whether there are ships in the images, precisely classifies and localizes them. Regardless of whether utilizing early manually designed methods or deep learning technology, ship detection is dedicated to exploring the inherent characteristics of ships to enhance recall. Nowadays, high-precision ship detection plays a crucial role in civilian and military applications. In order to provide a comprehensive review of ship detection in optical remote-sensing images (SDORSIs), this paper summarizes the challenges as a guide. These challenges include complex marine environments, insufficient discriminative features, large scale variations, dense and rotated distributions, large aspect ratios, and imbalances between positive and negative samples. We meticulously review the improvement methods and conduct a detailed analysis of the strengths and weaknesses of these methods. We compile ship information from common optical remote sensing image datasets and compare algorithm performance. Simultaneously, we compare and analyze the feature extraction capabilities of backbones based on CNNs and Transformer, seeking new directions for the development in SDORSIs. Promising prospects are provided to facilitate further research in the future. [ABSTRACT FROM AUTHOR]

Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a common and high-risk sleep-related breathing disorder. Snoring detection is a simple and non-invasive method. In many studies, the feature maps are obtained by applying a short-time Fourier transform (STFT) and feeding the model with single-channel input tensors. However, this approach may limit the potential of convolutional networks to learn diverse representations of snore signals. This paper proposes a snoring sound detection algorithm using a multi-channel spectrogram and convolutional neural network (CNN). The sleep recordings from 30 subjects at the hospital were collected, and four different feature maps were extracted from them as model input, including spectrogram, Mel-spectrogram, continuous wavelet transform (CWT), and multi-channel spectrogram composed of the three single-channel maps. Three methods of data set partitioning are used to evaluate the performance of feature maps. The proposed feature maps were compared through the training set and test set of independent subjects by using a CNN model. The results show that the accuracy of the multi-channel spectrogram reaches 94.18%, surpassing that of the Mel-spectrogram that exhibits the best performance among the single-channel spectrograms. This study optimizes the system in the feature extraction stage to adapt to the superior feature learning capability of the deep learning model, providing a more effective feature map for snoring detection. [ABSTRACT FROM AUTHOR]

Diabetes is a widely spread disease that significantly affects people's lives. The leading cause is uncontrolled levels of blood glucose, which develop eye defects over time, including Diabetic Retinopathy (DR), which results in severe visual loss. The primary factor causing blindness is considered to be DR in diabetic patients. DR treatment tries to control the disease's severity, as it is irreversible. The primary goal of this effort is to create a reliable method for automatically detecting the severity of DR. This paper proposes a new automated system (DR-NASNet) to detect and classify DR severity using an improved pretrained NASNet Model. To develop the DR-NASNet system, we first utilized a preprocessing technique that takes advantage of Ben Graham and CLAHE to lessen noise, emphasize lesions, and ultimately improve DR classification performance. Taking into account the imbalance between classes in the dataset, data augmentation procedures were conducted to control overfitting. Next, we have integrated dense blocks into the NASNet architecture to improve the effectiveness of classification results for five severity levels of DR. In practice, the DR-NASNet model achieves state-of-the-art results with a smaller model size and lower complexity. To test the performance of the DR-NASNet system, a combination of various datasets is used in this paper. To learn effective features from DR images, we used a pretrained model on the dataset. The last step is to put the image into one of five categories: No DR, Mild, Moderate, Proliferate, or Severe. To carry this out, the classifier layer of a linear SVM with a linear activation function must be added. The DR-NASNet system was tested using six different experiments. The system achieves 96.05% accuracy with the challenging DR dataset. The results and comparisons demonstrate that the DR-NASNet system improves a model's performance and learning ability. As a result, the DR-NASNet system provides assistance to ophthalmologists by describing an effective system for classifying early-stage levels of DR. [ABSTRACT FROM AUTHOR]

A significant majority of the population in India makes their living through agriculture. Different illnesses that develop due to changing weather patterns and are caused by pathogenic organisms impact the yields of diverse plant species. The present article analyzed some of the existing techniques in terms of data sources, pre-processing techniques, feature extraction techniques, data augmentation techniques, models utilized for detecting and classifying diseases that affect the plant, how the quality of images was enhanced, how overfitting of the model was reduced, and accuracy. The research papers for this study were selected using various keywords from peer-reviewed publications from various databases published between 2010 and 2022. A total of 182 papers were identified and reviewed for their direct relevance to plant disease detection and classification, of which 75 papers were selected for this review after exclusion based on the title, abstract, conclusion, and full text. Researchers will find this work to be a useful resource in recognizing the potential of various existing techniques through data-driven approaches while identifying plant diseases by enhancing system performance and accuracy. [ABSTRACT FROM AUTHOR]

Automatic extraction of building contours from high-resolution images is of great significance in the fields of urban planning, demographics, and disaster assessment. Network models based on convolutional neural network (CNN) and transformer technology have been widely used for semantic segmentation of buildings from high resolution remote sensing images (HRSI). However, the fixed geometric structure and the local receptive field of the convolutional kernel are not good at global feature extraction, and the transformer technique with self-attention mechanism introduces computational redundancies and extracts local feature details poorly in the process of modeling the global contextual information. In this paper, a dual-branch fused reconstructive transformer network, DFRTNet, is proposed for efficient and accurate building extraction. In the encoder, the traditional transformer is reconfigured by designing the local and global feature extraction module (LGFE); the branch of global feature extraction (GFE) performs dynamic range attention (DRA) based on the idea of top-k attention for extracting global features; furthermore, the branch of local feature extraction (LFE) is used to obtain fine-grained features. The multilayer perceptron (MLP) is employed to efficiently fuse the local and global features. In the decoder, a simple channel attention module (CAM) is used in the up-sampling part to enhance channel dimension features. Our network achieved the best segmentation accuracy on both the WHU and Massachusetts building datasets when compared to other mainstream and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Biometric technology is fast gaining pace as a veritable developmental tool. So far, biometric procedures have been predominantly used to ensure identity and ear recognition techniques continue to provide very robust research prospects. This paper proposes to identify and review present techniques for ear biometrics using certain parameters: machine learning methods, and procedures and provide directions for future research. Ten databases were accessed, including ACM, Wiley, IEEE, Springer, Emerald, Elsevier, Sage, MIT, Taylor & Francis, and Science Direct, and 1121 publications were retrieved. In order to obtain relevant materials, some articles were excused using certain criteria such as abstract eligibility, duplicity, and uncertainty (indeterminate method). As a result, 73 papers were selected for in-depth assessment and significance. A quantitative analysis was carried out on the identified works using search strategies: source, technique, datasets, status, and architecture. A Quantitative Analysis (QA) of feature extraction methods was carried out on the selected studies with a geometric approach indicating the highest value at 36%, followed by the local method at 27%. Several architectures, such as Convolutional Neural Network, restricted Boltzmann machine, auto-encoder, deep belief network, and other unspecified architectures, showed 38%, 28%, 21%, 5%, and 4%, respectively. Essentially, this survey also provides the various status of existing methods used in classifying related studies. A taxonomy of the current methodologies of ear recognition system was presented along with a publicly available occlussion and pose sensitive black ear image dataset of 970 images. The study concludes with the need for researchers to consider improvements in the speed and security of available feature extraction algorithms. [ABSTRACT FROM AUTHOR]

Alzheimer's disorder (AD) causes permanent impairment in the brain's memory of the cellular system, leading to the initiation of dementia. Earlier detection of Alzheimer's disease in the initial stages is challenging for researchers. Deep learning and machine learning‐based techniques can help resolve many issues associated with brain imaging exploration. Brain MR Images (Brain‐MRI) are used to detect Alzheimer's in computable research work. To correctly categorize the stages of Alzheimer's disease, discriminative features need to be extracted from the MR images. Recently, many studies have used deep learning methods for the early detection of this disorder. However, overfitting degrades the deep learning method's performance because the dataset's selection images are smaller and imbalanced. Some studies could not reach more discriminative and effectual attention‐aware features for Alzheimer's stage classification to increase the model performance. In this paper, we develop a novel hierarchical residual attention learning‐inspired multistage conjoined twin network (HRAL‐CTNN) to classify the stages of Alzheimer's. We used augmentation approaches to scale insufficient and imbalanced data. The HRAL‐CTNN is efficiently overcoming the issues of not obtaining efficient attention‐aware and generative features for Alzheimer's stage classification. The proposed model solved the problem of redundant features by extracting attentive discriminant features, and scaling imbalance data by data augmentation, after that training and validation using HRAL‐CTNN. The execution of this proposed work has been performed on the ADNI MRI dataset. This work achieved outstanding accuracy of 99.97 ±$$ \pm $$ 0.01% and F1 score of 99.30 ±$$ \pm $$ 0.02% for Alzheimer's stage classification. This model proposed by our group outperformed the existing related studies in terms of the model's performance score. [ABSTRACT FROM AUTHOR]

Prediction of conformational B-cell epitopes (CBCE) is an essential phase for vaccine design, drug invention, and accurate disease diagnosis. Many laboratorial and computational approaches have been developed to predict CBCE. However, laboratorial experiments are costly and time consuming, leading to the popularity of Machine Learning (ML)-based computational methods. Although ML methods have succeeded in many domains, achieving higher accuracy in CBCE prediction remains a challenge. To overcome this drawback and consider the limitations of ML methods, this paper proposes a novel DL-based framework for CBCE prediction, leveraging the capabilities of deep learning in the medical domain. The proposed model is named Deep Learning-based Temporal Convolutional Neural Network (DL-TCNN), which hybridizes empirical hyper-tuned 1D-CNN and TCN. TCN is an architecture that employs causal convolutions and dilations, adapting well to sequential input with extensive receptive fields. To train the proposed model, physicochemical features are firstly extracted from antigen sequences. Next, the Synthetic Minority Oversampling Technique (SMOTE) is applied to address the class imbalance problem. Finally, the proposed DL-TCNN is employed for the prediction of CBCE. The model's performance is evaluated and validated on a benchmark antigen–antibody dataset. The DL-TCNN achieves 94.44% accuracy, and 0.989 AUC score for the training dataset, 78.53% accuracy, and 0.661 AUC score for the validation dataset; and 85.10% accuracy, 0.855 AUC score for the testing dataset. The proposed model outperforms all the existing CBCE methods. [ABSTRACT FROM AUTHOR]

With the advancement of technology, the demand for high quality and sustainable electrical energy has been increased due to the widespread use of electrical devices in our daily lives. The issue of power quality in the power system is of great importance for the smooth and long-lasting operation of the electrical devices. Besides, large penetration of the hybrid power system (HPS) into the existing power grid injects the inevitable issues related to the power quality. Therefore, it is very important to detect and eliminate the power quality disturbances (PQDs) in order to obtain quality power. This paper presents a new approach deep learning-based system that can detect PQDs in the HPS. A new feature extraction approach is used to obtain the optimum Stockwell Transform (ST) contour image by applying the ST to a PQD signal. The resulting image files are given to the convolutional neural network (CNN) algorithm. Besides, optimum hyperparameters of CNN are determined by using Bayesian optimization algorithm (BOA). Thus, a recognition approach that both effectively extracts the features of PQDs and has high classification performance is proposed in this paper. The proposed recognition system is named as ST and Bayesian optimization-based CNN (STBOACNN). In order to test the performance of the proposed STBOACNN approach, PQD data obtained from the HPS with converter-based distributed generations are used. The experimental results showed that the STBOACNN is a new and effective approach that can classify PQDs occurring in the HPS with high recognition performance. [ABSTRACT FROM AUTHOR]

This main intention of this paper is to adopt a new disease detection model for plant leaves. The proposed model involves several steps such as pre‐processing, leaf segmentation, abnormality segmentation, feature extraction and detection. Image scaling and contrast enhancement are performed during the pre‐processing phase. Once the pre‐processing is done, the segmentation phase starts with leaf segmentation by binary thresholding method and abnormality segmentation by K‐means clustering. Further, the local binary pattern and grey‐level co‐occurrence matrix features are extracted and a dimensionality reduced technique called principle component analysis is determined. As the main novelty, the weighted feature extraction is performed, in which the weight functions are optimized by the self‐adaptive deer hunting optimization (SA‐DHOA). Another contribution of this paper is to implement a hybrid classifier for disease detection. Here, the extracted weighted features are subjected to a support vector machine, and the abnormality segmented image is subjected to convolutional neural network (CNN), which is a deep learning algorithm that can learn the features automatically. Here, the same SA‐DHOA is used to improve the performance of CNN, which is termed as SA‐DHOA‐SCNN. Finally, the performance analysis confirms the maximum success rate of the proposed model over other conventional methods. [ABSTRACT FROM AUTHOR]

Simple Summary: The use of computer vision technology has improved the effectiveness of individual sheep identification, but existing methods face challenges such as large parameter sizes, slow recognition speeds, and difficult deployment. To address this issue, we have made improvements and optimizations based on the Retinaface face recognition model to create a balanced speed-accuracy sheep face recognition model. The optimized model has fewer parameters, simpler computations, faster inference speeds, and higher recognition accuracy, making it well-suited for deployment on resource-constrained edge devices. This research is expected to promote the application of deep learning-based sheep face recognition methods in production. The recognition of sheep faces based on computer vision has improved the efficiency and effectiveness of individual sheep identification, providing technical support for the development of smart farming. However, current recognition models have problems such as large parameter sizes, slow recognition speed, and difficult deployment. Therefore, this paper proposes an efficient and fast basic module called Eblock and uses it to build a lightweight sheep face recognition model called SheepFaceNet, which achieves the best balance between speed and accuracy. SheepFaceNet includes two modules: SheepFaceNetDet for detection and SheepFaceNetRec for recognition. SheepFaceNetDet uses Eblock to construct the backbone network to enhance feature extraction capability and efficiency, designs a bidirectional FPN layer (BiFPN) to enhance geometric location ability, and optimizes the network structure, which affects inference speed, to achieve fast and accurate sheep face detection. SheepFaceNetRec uses Eblock to construct the feature extraction network, uses ECA channel attention to improve the effectiveness of feature extraction, and uses multi-scale feature fusion to achieve fast and accurate sheep face recognition. On our self-built sheep face dataset, SheepFaceNet recognized 387 sheep face images per second with an accuracy rate of 97.75%, achieving an advanced balance between speed and accuracy. This research is expected to further promote the application of deep-learning-based sheep face recognition methods in production. [ABSTRACT FROM AUTHOR]

The deep-learning-based image super-resolution opens a new direction for the remote sensing field to reconstruct further information and details from captured images. However, most current SR works try to improve the performance by increasing the complexity of the model, which results in significant computational costs and memory consumption. In this paper, we propose a lightweight model named pixel-wise attention residual network for optical remote sensor images, which can effectively solve the super-resolution task of multi-satellite images. The proposed method consists of three modules: the feature extraction module, feature fusion module, and feature mapping module. First, the feature extraction module is responsible for extracting the deep features from the input spatial bands with different spatial resolutions. Second, the feature fusion module with the pixel-wise attention mechanism generates weight coefficients for each pixel on the feature map and fully fuses the deep feature information. Third, the feature mapping module is aimed to maintain the fidelity of the spectrum by adding the fused residual feature map directly to the up-sampled low-resolution images. Compared with existing deep-learning-based methods, the major advantage of our method is that for the first time, the pixel-wise attention mechanism is incorporated in the task of super-resolution fusion of remote sensing images, which effectively improved the performance of the fusion network. The accuracy assessment results show that our method achieved superior performance of the root mean square error, signal-to–reconstruction ratio error, universal image quality index, and peak signal noise ratio compared to competing approaches. The improvements in the signal-to-reconstruction ratio error and peak signal noise ratio are significant, with a respective increase of 0.15 and 0.629 dB for Sentinel-2 data, and 0.196 and 1 dB for Landsat data. [ABSTRACT FROM AUTHOR]

The classification problem of Arabic texts is an important topic recently, which requires to solve it by designing classification systems. For this purpose, it is needed to collect, clean, and represent the data by pre-processing techniques based on NLP field, as well as to train and test the proposed models by machine learning and deep learning algorithms. In this paper, we present an approach for Arabic text classification based on a combination of NLP techniques and deep learning algorithms, using TF-IDF, BOW and Word embedding for feature extraction and proposing an architecture for convolutional neural networks, in addition we trained and tested our model on three popular datasets. At the end, a comparison is performed with other approaches to evaluate the performance of our model and its efficiency. The experimental results demonstrate its effectiveness, with accuracy rates of 94.47% for One Hot Encoding, 93.98% for BOW, and 93.58% for TF-IDF on the CNN dataset. Similarly, on the BBC dataset, One Hot Encoding achieved an accuracy of 90.97%, outperforming TF-IDF (86.35%) and BOW (87.19%). Furthermore, the OSAC dataset yielded high accuracy for all approaches, with TF-IDF at 99.19%, BOW at 99.42%, and One Hot Encoding at 98.86%. These results confirm the superiority of our proposed approach in Arabic text classification, demonstrating its potential for real-world applications in various domains. [ABSTRACT FROM AUTHOR]

The growth of biomedical engineering has made depression diagnosis via electroencephalography (EEG) a trendy issue. The two significant challenges to this application are EEG signals' complexity and non-stationarity. Additionally, the effects caused by individual variances may hamper the generalization of detection systems. Given the association between EEG signals and particular demographics, such as gender and age, and the influences of these demographic characteristics on the incidence of depression, it would be preferable to include demographic factors during EEG modeling and depression detection. The main objective of this work is to develop an algorithm that can recognize depression patterns by studying EEG data. Following a multiband analysis of such signals, machine learning and deep learning techniques were used to detect depression patients automatically. EEG signal data are collected from the multi-modal open dataset MODMA and employed in studying mental diseases. The EEG dataset contains information from a traditional 128-electrode elastic cap and a cutting-edge wearable 3-electrode EEG collector for widespread applications. In this project, resting EEG readings of 128 channels are considered. According to CNN, training with 25 epoch iterations had a 97% accuracy rate. The patient's status has to be divided into two basic categories: major depressive disorder (MDD) and healthy control. Additional MDD include the following six classes: obsessive-compulsive disorders, addiction disorders, conditions brought on by trauma and stress, mood disorders, schizophrenia, and the anxiety disorders discussed in this paper are a few examples of mental illnesses. According to the study, a natural combination of EEG signals and demographic data is promising for the diagnosis of depression. [ABSTRACT FROM AUTHOR]

Cloud and snow recognition technology is of great significance in the field of meteorology, and is also widely used in remote sensing mapping, aerospace, and other fields. Based on the traditional method of manually labeling cloud-snow areas, a method of labeling cloud and snow areas using deep learning technology has been gradually developed to improve the accuracy and efficiency of recognition. In this paper, from the perspective of designing an efficient and lightweight network model, a cloud snow recognition model based on a lightweight feature map attention network (Lw-fmaNet) is proposed to ensure the performance and accuracy of the cloud snow recognition model. The model is improved based on the ResNet18 network with the premise of reducing the network parameters and improving the training efficiency. The main structure of the model includes a shallow feature extraction module, an intrinsic feature mapping module, and a lightweight adaptive attention mechanism. Overall, in the experiments conducted in this paper, the accuracy of the proposed cloud and snow recognition model reaches 95.02%, with a Kappa index of 93.34%. The proposed method achieves an average precision rate of 94.87%, an average recall rate of 94.79%, and an average F1-Score of 94.82% for four sample recognition classification tasks: no snow and no clouds, thin cloud, thick cloud, and snow cover. Meanwhile, our proposed network has only 5.617M parameters and takes only 2.276 s. Compared with multiple convolutional neural networks and lightweight networks commonly used for cloud and snow recognition, our proposed lightweight feature map attention network has a better performance when it comes to performing cloud and snow recognition tasks. [ABSTRACT FROM AUTHOR]

With the growing diversity of cyberattacks in recent years, anomaly-based intrusion detection systems that can detect unknown attacks have attracted significant attention. Furthermore, a wide range of studies on anomaly detection using machine learning and deep learning methods have been conducted. However, many machine learning and deep learning-based methods require significant effort to design the detection feature values, extract the feature values from network packets, and acquire the labeled data used for model training. To solve the aforementioned problems, this paper proposes a new model called DOC-IDS, which is an intrusion detection system based on Perera's deep one-class classification. The DOC-IDS, which comprises a pair of one-dimensional convolutional neural networks and an autoencoder, uses three different loss functions for training. Although, in general, only regular traffic from the computer network subject to detection is used for anomaly detection training, the DOC-IDS also uses multi-class labeled traffic from open datasets for feature extraction. Therefore, by streamlining the classification task on multi-class labeled traffic, we can obtain a feature representation with highly enhanced data discrimination abilities. Simultaneously, we perform variance minimization in the feature space, even on regular traffic, to further improve the model's ability to discriminate between normal and abnormal traffic. The DOC-IDS is a single deep learning model that can automatically perform feature extraction and anomaly detection. This paper also reports experiments for evaluating the anomaly detection performance of the DOC-IDS. The results suggest that the DOC-IDS offers higher anomaly detection performance while reducing the load resulting from the design and extraction of feature values. [ABSTRACT FROM AUTHOR]

The precise classification of crop types using hyperspectral remote sensing imaging is an essential application in the field of agriculture, and is of significance for crop yield estimation and growth monitoring. Among the deep learning methods, Convolutional Neural Networks (CNNs) are the premier model for hyperspectral image (HSI) classification for their outstanding locally contextual modeling capability, which facilitates spatial and spectral feature extraction. Nevertheless, the existing CNNs have a fixed shape and are limited to observing restricted receptive fields, constituting a simulation difficulty for modeling long-range dependencies. To tackle this challenge, this paper proposed two novel classification frameworks which are both built from multilayer perceptrons (MLPs). Firstly, we put forward a dilation-based MLP (DMLP) model, in which the dilated convolutional layer replaced the ordinary convolution of MLP, enlarging the receptive field without losing resolution and keeping the relative spatial position of pixels unchanged. Secondly, the paper proposes multi-branch residual blocks and DMLP concerning performance feature fusion after principal component analysis (PCA), called DMLPFFN, which makes full use of the multi-level feature information of the HSI. The proposed approaches are carried out on two widely used hyperspectral datasets: Salinas and KSC; and two practical crop hyperspectral datasets: WHU-Hi-LongKou and WHU-Hi-HanChuan. Experimental results show that the proposed methods outshine several state-of-the-art methods, outperforming CNN by 6.81%, 12.45%, 4.38% and 8.84%, and outperforming ResNet by 4.48%, 7.74%, 3.53% and 6.39% on the Salinas, KSC, WHU-Hi-LongKou and WHU-Hi-HanChuan datasets, respectively. As a result of this study, it was confirmed that the proposed methods offer remarkable performances for hyperspectral precise crop classification. [ABSTRACT FROM AUTHOR]

Large-scale labeled data are generally required to train deep neural networks in order to obtain better performance in visual feature learning from images or videos for computer vision applications. To avoid extensive cost of collecting and annotating large-scale datasets, as a subset of unsupervised learning methods, self-supervised learning methods are proposed to learn general image and video features from large-scale unlabeled data without using any human-annotated labels. This paper provides an extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos. First, the motivation, general pipeline, and terminologies of this field are described. Then the common deep neural network architectures that used for self-supervised learning are summarized. Next, the schema and evaluation metrics of self-supervised learning methods are reviewed followed by the commonly used datasets for images, videos, audios, and 3D data, as well as the existing self-supervised visual feature learning methods. Finally, quantitative performance comparisons of the reviewed methods on benchmark datasets are summarized and discussed for both image and video feature learning. At last, this paper is concluded and lists a set of promising future directions for self-supervised visual feature learning. [ABSTRACT FROM AUTHOR]

This paper proposes a data valuation algorithm for inertial measurement unit-based human activity recognition (IMU-based HAR) data based on meta reinforcement learning. Unlike previous studies that received feature-level input, the algorithm in this study added a feature extraction structure to the data valuation algorithm, and it can receive raw-level inputs and achieve excellent performance. As IMU-based HAR data are multivariate time-series data, the proposed algorithm incorporates an architecture capable of extracting both local and global features by inserting a transformer encoder after the one-dimensional convolutional neural network (1D-CNN) backbone in the data value estimator. In addition, the 1D-CNN-based stacking ensemble structure, which exhibits excellent efficiency and performance on IMU-based HAR data, is used as a predictor to supervise model training. The Berg balance scale (BBS) IMU-based HAR dataset and the public datasets, UCI-HAR, WISDM, and PAMAP2, are used for performance evaluation in this study. The valuation performance of the proposed algorithm is observed to be excellent on IMU-based HAR data. The rate of discovering corrupted data is higher than 96% on all datasets. In addition, classification performance is confirmed to be improved by the suppression of discovery of low-value data. [ABSTRACT FROM AUTHOR]

The polysomnogram (PSG) is the gold standard for evaluating sleep quality and disorders. Attempts to automate this process have been hampered by the complexity of the PSG signals and heterogeneity among subjects and recording hardwares. Most of the existing methods for automatic sleep stage scoring rely on hand-engineered features that require prior knowledge of sleep analysis. This paper presents an end-to-end deep transfer learning framework for automatic feature extraction and sleep stage scoring based on a single-channel EEG. The proposed framework was evaluated over the three primary signals recommended by the American Academy of Sleep Medicine (C4-M1, F4-M1, O2-M1) from two data sets that have different properties and are recorded with different hardware. Different Time–Frequency (TF) imaging approaches were evaluated to generate TF representations for the 30 s EEG sleep epochs, eliminating the need for complex EEG signal pre-processing or manual feature extraction. Several training and detection scenarios were investigated using transfer learning of convolutional neural networks (CNN) and combined with recurrent neural networks. Generating TF images from continuous wavelet transform along with a deep transfer architecture composed of a pre-trained GoogLeNet CNN followed by a bidirectional long short-term memory (BiLSTM) network showed the best scoring performance among all tested scenarios. Using 20-fold cross-validation applied on the C4-M1 channel, the proposed framework achieved an average per-class accuracy of 91.2%, sensitivity of 77%, specificity of 94.1%, and precision of 75.9%. Our results demonstrate that without changing the model architecture and the training algorithm, our model could be applied to different single-channel EEGs from different data sets. Most importantly, the proposed system receives a single EEG epoch as an input at a time and produces a single corresponding output label, making it suitable for real time monitoring outside sleep labs as well as to help sleep lab specialists arrive at a more accurate diagnoses. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

Channel attention mechanisms have been commonly applied in many visual tasks for effective performance improvement. It is able to reinforce the informative channels as well as to suppress the useless channels. Recently, different channel attention modules have been proposed and implemented in various ways. Generally speaking, they are mainly based on convolution and pooling operations. In this paper, we propose Gaussian process embedded channel attention (GPCA) module and further interpret the channel attention schemes in a probabilistic way. The GPCA module intends to model the correlations among the channels, which are assumed to be captured by beta distributed variables. As the beta distribution cannot be integrated into the end-to-end training of convolutional neural networks (CNNs) with a mathematically tractable solution, we utilize an approximation of the beta distribution to solve this problem. To specify, we adapt a Sigmoid-Gaussian approximation, in which the Gaussian distributed variables are transferred into the interval [0,1]. The Gaussian process is then utilized to model the correlations among different channels. In this case, a mathematically tractable solution is derived. The GPCA module can be efficiently implemented and integrated into the end-to-end training of the CNNs. Experimental results demonstrate the promising performance of the proposed GPCA module. Codes are available at https://github.com/PRIS-CV/GPCA. [ABSTRACT FROM AUTHOR]

Intelligent detection of marine organism plays an important part in the marine economy, and it is significant to detect marine organisms quickly and accurately in a complex marine environment for the intelligence of marine equipment. The existing object detection models do not work well underwater. This paper improves the structure of EfficientDet detector and proposes the EfficientDet-Revised (EDR), which is a new marine organism object detection model. Specifically, the MBConvBlock is reconstructed by adding the Channel Shuffle module to enable the exchange of information between the channels of the feature layer. The fully connected layer of the attention module is removed and convolution is used to cut down the amount of network parameters. The Enhanced Feature Extraction module is constructed for multi-scale feature fusion to enhance the feature extraction ability of the network to different objects. The results of experiments demonstrate that the mean average precision (mAP) of the proposed method reaches 91.67% and 92.81% on the URPC dataset and the Kaggle dataset, respectively, which is better than other object detection models. At the same time, the processing speed reaches 37.5 frame per second (FPS) on the URPC dataset, which can meet the real-time requirements. It can provide a useful reference for underwater robots to perform tasks such as intelligent grasping. [ABSTRACT FROM AUTHOR]

This paper is concerned with individual identification by late fusion of two-stream deep networks from Electromyogram (EMG) signals. EMG signal has more advantages on security compared to other biosignals exposed visually, such as the face, iris, and fingerprints, when used for biometrics, at least in the aspect of visual exposure, because it is measured through contact without any visual exposure. Thus, we propose an ensemble deep learning model by late information fusion of convolutional neural networks (CNN) and long short-term memory (LSTM) from EMG signals for robust and discriminative biometrics. For this purpose, in the ensemble model's first stream, one-dimensional EMG signals were converted into time–frequency representation to train a two-dimensional convolutional neural network (EmgCNN). In the second stream, statistical features were extracted from one-dimensional EMG signals to train a long short-term memory (EmgLSTM) that uses sequence input. Here, the EMG signals were divided into fixed lengths, and feature values were calculated for each interval. A late information fusion is performed by the output scores of two deep learning models to obtain a final classification result. To confirm the superiority of the proposed method, we use an EMG database constructed at Chosun University and a public EMG database. The experimental results revealed that the proposed method showed performance improvement by 10.76% on average compared to a single stream and the previous methods. [ABSTRACT FROM AUTHOR]

This paper generalizes the Attention in Attention (AiA) mechanism, in P. Fang et al., 2019 by employing explicit mapping in reproducing kernel Hilbert spaces to generate attention values of the input feature map. The AiA mechanism models the capacity of building inter-dependencies among the local and global features by the interaction of inner and outer attention modules. Besides a vanilla AiA module, termed linear attention with AiA, two non-linear counterparts, namely, second-order polynomial attention and Gaussian attention, are also proposed to utilize the non-linear properties of the input features explicitly, via the second-order polynomial kernel and Gaussian kernel approximation. The deep convolutional neural network, equipped with the proposed AiA blocks, is referred to as Attention in Attention Network (AiA-Net). The AiA-Net learns to extract a discriminative pedestrian representation, which combines complementary person appearance and corresponding part features. Extensive ablation studies verify the effectiveness of the AiA mechanism and the use of non-linear features hidden in the feature map for attention design. Furthermore, our approach outperforms current state-of-the-art by a considerable margin across a number of benchmarks. In addition, state-of-the-art performance is also achieved in the video person retrieval task with the assistance of the proposed AiA blocks. [ABSTRACT FROM AUTHOR]

Learning how to capture long-range dependencies and restore spatial information of down-sampled feature maps are the basis of the encoder-decoder structure networks in medical image segmentation. U-Net based methods use feature fusion to alleviate these two problems, but the global feature extraction ability and spatial information recovery ability of U-Net are still insufficient. In this paper, we propose a Global Feature Reconstruction (GFR) module to efficiently capture global context features and a Local Feature Reconstruction (LFR) module to dynamically up-sample features, respectively. For the GFR module, we first extract the global features with category representation from the feature map, then use the different level global features to reconstruct features at each location. The GFR module establishes a connection for each pair of feature elements in the entire space from a global perspective and transfers semantic information from the deep layers to the shallow layers. For the LFR module, we use low-level feature maps to guide the up-sampling process of high-level feature maps. Specifically, we use local neighborhoods to reconstruct features to achieve the transfer of spatial information. Based on the encoder-decoder architecture, we propose a Global and Local Feature Reconstruction Network (GLFRNet), in which the GFR modules are applied as skip connections and the LFR modules constitute the decoder path. The proposed GLFRNet is applied to four different medical image segmentation tasks and achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Deep learning is the subject of increasing research for fruit tree detection. Previously developed deep-learning-based models are either too large to perform real-time tasks or too small to extract good enough features. Moreover, there has been scarce research on the detection of pomelo trees. This paper proposes a pomelo tree-detection method that introduces the attention mechanism and a Ghost module into the lightweight model network, as well as a feature-fusion module to improve the feature-extraction ability and reduce computation. The proposed method was experimentally validated and showed better detection performance and fewer parameters than some state-of-the-art target-detection algorithms. The results indicate that our method is more suitable for pomelo tree detection. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel methodology based on machine learning for identifying the most appropriate from a set of available state-of-the-art object detectors for a given application. Our particular interest is to develop a road map for identifying verifiably optimal selections, especially for challenging applications such as detecting small objects in a mixed-size object dataset. State-of-the-art object detection systems often find the localisation of small-size objects challenging since most are usually trained on large-size objects. These contain abundant information as they occupy a large number of pixels relative to the total image size. This fact is normally exploited by the model during training and inference processes. To dissect and understand this process, our approach systematically examines detectors' performances using two very distinct deep convolutional networks. The first is the single-stage YOLO V3 and the second is the double-stage Faster R-CNN. Specifically, our proposed method explores and visually illustrates the impact of feature extraction layers, number of anchor boxes, data augmentation, etc., utilising ideas from the field of explainable Artificial Intelligence (XAI). Our results, for example, show that multi-head YOLO V3 detectors trained using augmented data produce better performance even with a fewer number of anchor boxes. Moreover, robustness regarding the detector's ability to explain how a specific decision was reached is investigated using different explanation techniques. Finally, two new visualisation techniques are proposed, WS-Grad and Concat-Grad, for identifying explanation cues of different detectors. These are applied to specific object detection tasks to illustrate their reliability and transparency with respect to the decision process. It is shown that the proposed techniques can result in high resolution and comprehensive heatmaps of the image areas, significantly affecting detector decisions as compared to the state-of-the-art techniques tested. [ABSTRACT FROM AUTHOR]

Implementing key engineering solutions to optimise the operation of energy industries requires daily electricity demand forecasting and including uncertainty, to promote markets insight analysis as part of their strategic planning, regulating and supplying electricity to consumers. This paper proposes hybrid artificial intelligence models combining convolutional neural networks (CNN) as a feature extraction algorithm with extreme learning machines (ELM) as a framework to predict electricity demand with confidence intervals generated by Kernel Density Estimation (KDE) approaches. In order to develop CELM-KDE model, time-lagged series of daily electricity demand with local climate variables based on the air temperature, atmospheric vapour pressure, evaporation, solar radiation, humidity and sea level pressures are used to train the proposed CELM-KDE hybrid model. In order to fully evaluate the newly developed model from a point-based, as well as a probabilistic prediction strategy, the observed and predicted electricity demand as well as the probability distribution of errors are analysed using KDE method that operates without prior data distribution assumptions. Based on observed and predicted electricity demand and the relevant probabilistic confidence intervals generated by the CELM-KDE model, the final results show that the proposed method attains significantly better probability interval predictions than traditionally-used point-based models. The proposed CELM-KDE model is demonstrated to be highly effective in providing a comprehensive coverage of predicted errors, as well as providing greater insights into the average bandwidth and detailed predicted electricity demand in the testing stage. The results also indicate that the proposed hybrid model is a reliable decision support tool to develop engineering solutions in area of energy modelling, monitoring and forecasting, which could potentially be useful to the industry policymakers. We show that the point-and probabilistic-based electricity demand predictive models can be employed as an effective tool to improve accuracy of forecasting and provision of insights for national electricity markets and key energy industry stakeholder application tools. [Display omitted] • A hybrid deep learning model for daily electricity demand prediction is proposed. • A Kernel Density Estimation is applied to derive probabilistic prediction intervals. • The algorithm quantifies uncertainties in point-based electricity demand predictions. • The proposed model outperforms deep learning models using statistical uncertainty indicators. [ABSTRACT FROM AUTHOR]

Retinal detachment (RD) is a severe condition that causes decreased visual acuity and blindness if left untreated timely. The early screening and identification of retinal detachment can ameliorate the successful rate of visible results and RD. The manual screening of retinal detachment is a labor-intensive and time-consuming task. This paper is concerned with pre-trained deep learning networks for feature extraction and classification. Deep learning models need large amounts of training data since they involve many parameters. This is a severe problem in medical informatics where the amount of data available is very low, and ground truth data is a small fraction of the same. The domain of Retinal Detachment (RD) is no different. Typical public domain databases related to RD typically have a few hundred images, leading to fitting issues for deep learning models. This work investigates the role of transfer learning for feature extraction and classification of RD and Non-RD color fundus images. We have also analyzed the performance of different deep neural networks through fundus imaging to detect (RD) eyes and Non-RD eyes. The deep convolutional networks such as AlexNet, InceptionV3, GoogleNet, VGG19, DenseNet, and ResNet50 were trained and tested on publically available datasets of RD and Non-RD fundus images. A ResNet50 framework through transfer learning shows the best classification performance in terms of Accuracy, Sensitivity, Specificity, Precision, and F1 score values of 99.50%, 99.00%, 99.99%, 99.99%, and 99.49%, respectively, and best for detecting RD and Non-RD fundus images compared to other learning models. This study inferred the promising results for a diagnostic system for retinal detachment with relatively high sensitivity and specificity. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) point cloud classification methods based on deep learning have good classification performance; however, they adapt poorly to diverse datasets and their classification accuracy must be improved. Therefore, FSDCNet, a neural network model based on the fusion of static and dynamic convolution, is proposed and applied for multiview 3D point cloud classification in this paper. FSDCNet devises a view selection method with fixed and random viewpoints, which effectively avoids the overfitting caused by the traditional fixed viewpoint. A local feature extraction operator of dynamic and static convolution adaptive weight fusion was designed to improve the model's adaptability to different types of datasets. To address the problems of large parameters and high computational complexity associated with the current methods of dynamic convolution, a lightweight and adaptive dynamic convolution operator was developed. In addition, FSDCNet builds a global attention pooling, integrating the most crucial information on different view features to the greatest extent. Due to these characteristics, FSDCNet is more adaptable, can extract more fine-grained detailed information, and can improve the classification accuracy of point cloud data. The proposed method was applied to the ModelNet40 and Sydney Urban Objects datasets. In these experiments, FSDCNet outperformed its counterparts, achieving state-of-the-art point cloud classification accuracy. For the ModelNet40 dataset, the overall accuracy (OA) and average accuracy (AA) of FSDCNet in a single view reached 93.8% and 91.2%, respectively, which were superior to those values for many other methods using 6 and 12 views. FSDCNet obtained the best results for 6 and 12 views, achieving 94.6%, 93.3%, 95.3%, and 93.6% in OA and AA metrics, respectively. For the Sydney Urban Objects dataset, FSDCNet achieved an OA and F1 score of 81.2% and 80.1% in a single view, respectively, which were higher than most of the compared methods. In 6 and 12 views, FSDCNet reached an OA of 85.3% and 83.6% and an F1 score of 85.5% and 83.7%, respectively. [ABSTRACT FROM AUTHOR]

Currently, deep-learning-based methods have been widely used in fault diagnosis to improve the diagnosis efficiency and intelligence. However, most schemes require a great deal of labeled data and many iterations for training parameters. They suffer from low accuracy and over fitting under the few-shot scenario. In addition, a large number of parameters in the model consumes high computing resources, which is far from practical. In this paper, a multi-scale and lightweight Siamese network architecture is proposed for the fault diagnosis with few samples. The architecture proposed contains two main modules. The first part implements the feature vector extraction of sample pairs. It is composed of two lightweight convolutional networks with shared weights symmetrically. Multi-scale convolutional kernels and dimensionality reduction are used in these two symmetric networks to improve feature extraction and reduce the total number of model parameters. The second part takes charge of calculating the similarity of two feature vectors to achieve fault classification. The proposed network is validated by multiple datasets with different loads and speeds. The results show that the model has better accuracy, fewer model parameters and a scale compared to the baseline approach through our experiments. Furthermore, the model is also proven to have good generalization capability. [ABSTRACT FROM AUTHOR]

To ensure the safe operation of highway traffic lines, given the imperfect feature extraction of existing road pit defect detection models and the practicability of detection equipment, this paper proposes a lightweight target detection algorithm with enhanced feature extraction based on the YOLO (You Only Look Once) algorithm. The BIFPN (Bidirectional Feature Pyramid Network) network structure is used for multi-scale feature fusion to enhance the feature extraction ability, and Varifocal Loss is used to optimize the sample imbalance problem, which improves the accuracy of road defect target detection. In the evaluation test of the model in the constructed PCD1 (Pavement Check Dataset) dataset, the mAP@.5 (mean Average Precision when IoU = 0.5) of the BV-YOLOv5S (BiFPN Varifocal Loss-YOLOv5S) model increased by 4.1%, 3%, and 0.9%, respectively, compared with the YOLOv3-tiny, YOLOv5S, and B-YOLOv5S (BiFPN-YOLOv5S; BV-YOLOv5S does not use the Improved Focal Loss function) models. Through the analysis and comparison of experimental results, it is proved that the proposed BV-YOLOv5S network model performs better and is more reliable in the detection of pavement defects and can meet the needs of road safety detection projects with high real-time and flexibility requirements. [ABSTRACT FROM AUTHOR]

In aerospace, marine, and other heavy industries, bearing fault diagnosis has been an essential part of improving machine life, reducing economic losses, and avoiding safety problems caused by machine bearing failures. Most existing bearing fault diagnosis methods face challenges in extracting the fault features from raw bearing fault data. Compared with traditional methods for bearing fault characteristics extraction, deep neural networks can automatically extract intrinsic features without expert knowledge. The convolutional neural network (CNN) was utilized most widely in extracting representative features of bearing faults. Fundamental to this, the hybrid models based on the CNN and individual classifiers were proposed to diagnose bearing faults. However, CNN may not be suitable for all bearing fault classifiers. It is crucial to identify the classifiers which can maximize the CNN feature extraction ability. In this paper, four hybrid models based on CNN were built, and their fault detection accuracy and efficiency were compared. The comparative analysis showed that the random forest (RF) and support vector machine (SVM) could make full use of the CNN feature extraction ability. [ABSTRACT FROM AUTHOR]

In this paper, we propose a symmetric series convolutional neural network (SS-CNN), which is a novel deep convolutional neural network (DCNN)-based super-resolution (SR) technique for ultrasound medical imaging. The proposed model comprises two parts: a feature extraction network (FEN) and an up-sampling layer. In the FEN, the low-resolution (LR) counterpart of the ultrasound image passes through a symmetric series of two different DCNNs. The low-level feature maps obtained from the subsequent layers of both DCNNs are concatenated in a feed forward manner, aiding in robust feature extraction to ensure high reconstruction quality. Subsequently, the final concatenated features serve as an input map to the latter 2D convolutional layers, where the textural information of the input image is connected via skip connections. The second part of the proposed model is a sub-pixel convolutional (SPC) layer, which up-samples the output of the FEN by multiplying it with a multi-dimensional kernel followed by a periodic shuffling operation to reconstruct a high-quality SR ultrasound image. We validate the performance of the SS-CNN with publicly available ultrasound image datasets. Experimental results show that the proposed model achieves a high-quality reconstruction of the ultrasound image over the conventional methods in terms of peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), while providing compelling SR reconstruction time. [ABSTRACT FROM AUTHOR]

(1) Background and objective: Cardiovascular disease is one of the most common causes of death in today's world. ECG is crucial in the early detection and prevention of cardiovascular disease. In this study, an improved deep learning method is proposed to diagnose abnormal and normal ECG accurately. (2) Methods: This paper proposes a CNN-FWS that combines three convolutional neural networks (CNN) and recursive feature elimination based on feature weights (FW-RFE), which diagnoses abnormal and normal ECG. F1 score and Recall are used to evaluate the performance. (3) Results: A total of 17,259 records were used in this study, which validated the diagnostic performance of CNN-FWS for normal and abnormal ECG signals in 12 leads. The experimental results show that the F1 score of CNN-FWS is 0.902, and the Recall of CNN-FWS is 0.889. (4) Conclusion: CNN-FWS absorbs the advantages of convolutional neural networks (CNN) to obtain three parts of different spatial information and enrich the learned features. CNN-FWS can select the most relevant features while eliminating unrelated and redundant features by FW-RFE, making the residual features more representative and effective. The method is an end-to-end modeling approach that enables an adaptive feature selection process without human intervention. [ABSTRACT FROM AUTHOR]

To better extract feature maps from low-resolution (LR) images and recover high-frequency information in the high-resolution (HR) images in image super-resolution (SR), we propose in this paper a new SR algorithm based on a deep convolutional neural network (CNN). The network structure is composed of the feature extraction part and the reconstruction part. The extraction network extracts the feature maps of LR images and uses the sub-pixel convolutional neural network as the up-sampling operator. Skip connection, densely connected neural networks and feature map fusion are used to extract information from hierarchical feature maps at the end of the network, which can effectively reduce the dimension of the feature maps. In the reconstruction network, we add a 3 X 3 convolution layer based on the original sub-pixel convolution layer, which can allow the reconstruction network to have better nonlinear mapping ability. The experiments show that the algorithm results in a significant improvement in PSNR, SSIM, and human visual effects as compared with some state-of-the-art algorithms based on deep learning. [ABSTRACT FROM AUTHOR]

Remotely sensed images and their allied areas of application have been the charm for a long time among researchers. Remote imagery has a vast area in which it is serving and achieving milestones. From the past, after the advent of AL, ML, and DL-based computing, remote imagery is related techniques for processing and analyzing are continuously growing and offering countless services like traffic surveillance, earth observation, land surveying, and other agricultural areas. As Artificial intelligence has become the charm of researchers, machine learning and deep learning have been proven as the most commonly used and highly effective techniques for object detection. AI & ML-based object segmentation & detection makes this area hot and fond to the researchers again with the opportunities of enhanced accuracy in the same. Several researchers have been proposed their works in the form of research papers to highlight the effectiveness of using remotely sensed imagery for commercial purposes. In this article, we have discussed the concept of remote imagery with some preprocessing techniques to extract hidden and fruitful information from them. Deep learning techniques applied by various researchers along with object detection, object recognition are also discussed here. This literature survey is also included a chronological review of work done related to detection and recognition using deep learning techniques. [ABSTRACT FROM AUTHOR]

Recently, deep learning has become a promising technique for constructing gesture recognition classifiers from surface electromyography (sEMG) signals in human–computer interaction. In this paper, we propose a gesture recognition method with sEMG signals based on a deep multi-parallel convolutional neural network (CNN), which solves the problem that traditional machine learning methods may lose too much useful information during feature extraction. CNNs provide an efficient way to constrain the complexity of feedforward neural networks by weight sharing and restriction to local connections. Sophisticated feature extraction is to be avoided and hand gestures are to be classified directly. A multi-parallel and multi-convolution layer convolution structure is proposed to classify hand gestures. Experiment results show that in comparison with five traditional machine learning methods, the proposed method could achieve higher accuracy. [ABSTRACT FROM AUTHOR]

Micro-expression spotting is a fundamental step in the micro-expression analysis. This paper proposes a novel network based convolutional neural network (CNN) for spotting multi-scale spontaneous micro-expression intervals in long videos. We named the network as Micro-Expression Spotting Network (MESNet). It is composed of three modules. The first module is a 2+1D Spatiotemporal Convolutional Network, which uses 2D convolution to extract spatial features and 1D convolution to extract temporal features. The second module is a Clip Proposal Network, which gives some proposed micro-expression clips. The last module is a Classification Regression Network, which classifies the proposed clips to micro-expression or not, and further regresses their temporal boundaries. We also propose a novel evaluation metric for spotting micro-expression. Extensive experiments have been conducted on the two long video datasets: CAS(ME)2 and SAMM, and the leave-one-subject-out cross-validation is used to evaluate the spotting performance. Results show that the proposed MESNet effectively enhances the F1-score metric. And comparative results show the proposed MESNet has achieved a good performance, which outperforms other state-of-the-art methods, especially in the SAMM dataset. [ABSTRACT FROM AUTHOR]

Time series classification (TSC) has been around for recent decades as a significant research problem for industry practitioners as well as academic researchers. Due to the rapid increase in temporal data in a wide range of disciplines, an incredible amount of algorithms have been proposed. This paper proposes robust approaches based on state-of-the-art techniques, bidirectional long short-term memory (BiLSTM), fully convolutional network (FCN), and attention mechanism. A BiLSTM considers both forward and backward dependencies, and FCN is proven to be good at feature extraction as a TSC baseline. Therefore, we augment BiLSTM and FCN in a hybrid deep learning architecture, BiLSTM-FCN. Moreover, we similarly explore the use of the attention mechanism to check its efficiency on BiLSTM-FCN and propose another model ABiLSTM-FCN. We validate the performance on 85 datasets from the University of California Riverside (UCR) univariate time series archive. The proposed models are evaluated in terms of classification testing error and f1-score and also provide performance comparison with various existing state-of-the-art techniques. The experimental results show that our proposed models perform comprehensively better than the existing state-of-the-art methods and baselines. [ABSTRACT FROM AUTHOR]

Recently, automatic diagnostic approaches have been widely used to classify ocular diseases. Most of these approaches are based on a single imaging modality (e.g., fundus photography or optical coherence tomography (OCT)), which usually only reflect the oculopathy to a certain extent, and neglect the modality-specific information among different imaging modalities. This paper proposes a novel modality-specific attention network (MSAN) for multi-modal retinal image classification, which can effectively utilize the modality-specific diagnostic features from fundus and OCT images. The MSAN comprises two attention modules to extract the modality-specific features from fundus and OCT images, respectively. Specifically, for the fundus image, ophthalmologists need to observe local and global pathologies at multiple scales (e.g., from microaneurysms at the micrometer level, optic disc at millimeter level to blood vessels through the whole eye). Therefore, we propose a multi-scale attention module to extract both the local and global features from fundus images. Moreover, large background regions exist in the OCT image, which is meaningless for diagnosis. Thus, a region-guided attention module is proposed to encode the retinal layer-related features and ignore the background in OCT images. Finally, we fuse the modality-specific features to form a multi-modal feature and train the multi-modal retinal image classification network. The fusion of modality-specific features allows the model to combine the advantages of fundus and OCT modality for a more accurate diagnosis. Experimental results on a clinically acquired multi-modal retinal image (fundus and OCT) dataset demonstrate that our MSAN outperforms other well-known single-modal and multi-modal retinal image classification methods. [ABSTRACT FROM AUTHOR]

Change/motion detection is a challenging problem in video analysis and surveillance system. Recently, the state-of-the-art methods using the sample-based background model have demonstrated astonishing results with this problem. However, they are ineffective in the dynamic scenes that contain complex motion patterns. In this paper, we introduce a novel data-driven approach that combines the sample-based background model with a feature extractor obtained by training a triplet network. We construct the network by three identical convolutional neural networks, each of which is called a motion feature network. Our network can automatically learn motion patterns from small image patches and transform input images of any size into feature embeddings for high-level representations. The sample-based background model of each pixel is then employed by using the color information and the extracted feature embeddings. We also propose an approach to generate triplet examples from CDNet 2014 for training our network model from scratch. The offline trained network can be used on the fly without re-training on any video sequence before each execution. Therefore, it is feasible for real-time surveillance systems. In this paper, we show that our method outperforms the other state-of-the-art methods on CDNet 2014 and other benchmarks (BMC and Wallflower). [ABSTRACT FROM AUTHOR]

This paper proposes a novel method for voltage dip classification using deep convolutional neural networks. The main contributions of this paper include: 1) to propose a new effective deep convolutional neural network architecture for automatically learning voltage dip features, rather than extracting hand-crafted features; 2) to employ the deep learning in an effective two-dimensional (2-D) transform domain, under space-phasor model (SPM), for efficient learning of dip features; 3) to characterize voltage dips by 2-D SPM-based deep learning, which leads to voltage dip features independent of the duration and sampling frequency of dip recordings; and 4) to develop robust automatically-extracted features that are insensitive to training and test datasets measured from different countries/regions. Experiments were conducted on datasets containing about 6000 measured voltage dips spread over seven classes measured from several different countries. Results have shown good performance of the proposed method: average classification rate is about 97% and false alarm rate is about 0.50%. The test results from the proposed method are compared with the results from two existing dip classification methods. The proposed method is shown to outperform these existing methods. [ABSTRACT FROM AUTHOR]

The visual cues from multiple support regions of different sizes and resolutions are complementary in classifying a candidate box in object detection. Effective integration of local and contextual visual cues from these regions has become a fundamental problem in object detection. In this paper, we propose a gated bi-directional CNN (GBD-Net) to pass messages among features from different support regions during both feature learning and feature extraction. Such message passing can be implemented through convolution between neighboring support regions in two directions and can be conducted in various layers. Therefore, local and contextual visual patterns can validate the existence of each other by learning their nonlinear relationships and their close interactions are modeled in a more complex way. It is also shown that message passing is not always helpful but dependent on individual samples. Gated functions are therefore needed to control message transmission, whose on-or-offs are controlled by extra visual evidence from the input sample. The effectiveness of GBD-Net is shown through experiments on three object detection datasets, ImageNet, Pascal VOC2007 and Microsoft COCO. Besides the GBD-Net, this paper also shows the details of our approach in winning the ImageNet object detection challenge of 2016, with source code provided on https://github.com/craftGBD/craftGBD. In this winning system, the modified GBD-Net, new pretraining scheme and better region proposal designs are provided. We also show the effectiveness of different network structures and existing techniques for object detection, such as multi-scale testing, left-right flip, bounding box voting, NMS, and context. [ABSTRACT FROM AUTHOR]

Winding condition assessment is an essential task for operating transformers, and the vibration method provides a low-cost and non-intrusive approach. In this paper, a novel feature extraction method based on vibration analysis is proposed, which converts the vibration monitoring data with load information into a vibration image. Then, a deep learning approach based on convolutional neural network (CNN) is used to classify the images belong to different classes. In the laboratory experiment, free vibration tests are performed on an on-load winding model, which are used to verify the relationship between the natural frequency and the electromagnetic force under different clamping forces. During the field experiment, transformers are divided into three categories, including normal, degraded and anomalous, and the proposed scheme is trained and tested by using the vibration samples acquired from more than 100 operating transformers. The performance of the CNN classifier under different input sizes is investigated, which achieves 98.3% overall accuracy. Besides, the confusion matrices obtained by other methods are compared, such as artificial neural network (ANN), support vector machine (SVM) and naive Bayes classifier (NBC). The results show that the proposed scheme including the vibration image extraction method and the CNN classifier offers superior performance in winding fault diagnosis. [ABSTRACT FROM AUTHOR]

For a long time, learning spatiotemporal features with deep neural networks has been a difficult task in the field of computer vision. In this paper, we present a novel deep architecture, termed as Bifurcated Convolutional Neural Network (BifurcatedNet) to learn the discriminative video representation in an end-to-end manner. In our work, the BifurcatedNet is built upon the stacking bifurcated blocks that aim at simultaneously capturing the static appearance information and the temporal dynamic from input data. Specifically, the bifurcated block is composed of two separated branch, i.e., an appearance branch and a dynamic branch. The appearance branch employs 2D convolutional operation to obtain the spatial responses of image pixels or filters of each input frame, while the design of the dynamic branch is based on the spatio-temporal convolutional operation to exploit the temporal dynamic between pixels and filter response across multiple frames. Multiple experiments are conducted on two popular action recognition benchmarks: UCF101 and HMDB51. With only RGB input, the BifurcatedNet obtains the superior performance over the existing state-of-the-art models under the same experimental setting. The proposed BifurcatedNet is also implemented in a two-stream fashion by using both RGB and optical flow input, and still achieves the state-of-the-art performance, demonstrating the effectiveness of the network design. Furthermore, in order to evaluate the generalization ability, we conduct experiments on the Chalearn LAP IsoGD dataset and find that our model works well in gesture recognition tasks. [ABSTRACT FROM AUTHOR]