Search

The aim of image inpainting is to fill in damaged areas according to certain rules based on information about the adjacent positions of missing areas and the overall structure of the image, a technique that plays a key role in various tasks in computer vision. With the rapid development of deep learning, researchers have combined it with image inpainting and achieved excellent performance. To gain insight into the techniques involved, this paper summarizes the latest research advances in the field of image inpainting. Firstly, existing classical image inpainting methods are reviewed, and traditional image inpainting methods and their advantages and disadvantages are introduced. Secondly, three classical network models are outlined, and the image inpainting methods are classified into single-stage, multi-stage and a priori condition-guided approaches according to different network types and model structures. Representative algorithms among them are selected and their important technical improvement ideas are analyzed and summarized. Then, the common datasets commonly used in image inpainting tasks and the evaluation metrics used to evaluate inpainting results are introduced. The paper presents a comprehensive summary of the various algorithms in terms of network models and inpainting methods, and selects representative algorithms for quantitative and qualitative comparative analysis. Finally, the future development trends and research directions have prospected, and the current problems of image inpainting are summarized. [ABSTRACT FROM AUTHOR]

Speech emotion recognition (SER) has recently increased because of vast innovations in human–computer interaction and affective computing. In recent years, numerous deep learning-based schemes presented for SER have shown significant improvement over the traditional machine learning approaches. Most deep learning-based faced SER systems face challenges due to data imbalance problem that occurs due to unequal samples in the database. The input to two-dimensional CNN uses traditional MFCC for SER. It degrades the quality of deep attributes because of the higher variance, frequency resolution problem and spectral leakage problem of traditional MFCC. This paper proposed the novel Multi-taper Mel Frequency Logarithmic Spectrogram to enrich the Deep Convolutional Neural Network effectiveness for SER. Further, Generative Adversarial Network is used for speech emotion data augmentation during training to deal with data scarcity problems in SER. The performance of the proposed SER scheme is validated using the Berlin EmoDB and RAVDESS datasets. The proposed method provides SER accuracy of 96.65% and 97.12% for the EmoDB and RAVDESS dataset, respectively, and significantly improves over the recent techniques. [ABSTRACT FROM AUTHOR]

Knee ligament tear injury is frequent in many volleyball, football, basketball, and cricket players. In the past, various deep learning-based ACL tear detection schemes using knee magnetic resonance imaging (MRI) have been presented. It has shown challenges in ACL tear detection performance due to class imbalance issue arising due to uneven training samples and poor feature representation. This paper presents a simple and effective ACL knee ligament tear detection method with a convolutional neural network (ATD-CNN) to lessen the intricacy of the network. Further, the self-attention mechanism is used to improve the feature representation of MRI image information, neglect the irrelevant information in deep features, and enhance the classification accuracy. To diminish the class imbalance issue, generative adversarial network (GAN) is used to construct the synthetic database. The performance of the ATD-CNN with self-attention is assessed on the MRNet database using precision, accuracy, F1-score, and recall. ATD-CNN provides an accuracy of 90.10% for the original and 93.93% and augmented datasets. However, the ATD-CNN without attention mechanism resulted in 89.60% and 92.30% accuracy for original and augmented dataset. The proposed ATD-CNN model indicates that it can be utilized to detect ACL tears automatically and outperforms the existing schemes for tear detection. [ABSTRACT FROM AUTHOR]

As vital equipment in modern industry, the health state of rotating machinery influences the production process and equipment safety. However, rotating machinery generally operates in a normal state most of the time, which results in limited fault data, thus greatly constraining the performance of intelligent fault diagnosis methods. To solve this problem, this paper proposes a novel fault diagnosis method for rotating machinery with limited multisensor fusion samples based on the fused attention-guided Wasserstein generative adversarial network (WGAN). Firstly, the dimensionality of collected multisensor data is reduced to three channels by principal component analysis, and then the one-dimensional data of each channel are converted into a two-dimensional pixel matrix, of which the RGB images are obtained by fusing the three-channel two-dimensional images. Subsequently, the limited RGB samples are augmented to obtain sufficient samples utilizing the fused attention-guided WGAN combined with the gradient penalty (FAWGAN-GP) method. Lastly, the augmented samples are applied to train a residual convolutional neural network for fault diagnosis. The effectiveness of the proposed method is demonstrated by two case studies. When training samples per class are 50, 35, 25, and 15 on the KAT-bearing dataset, the average classification accuracy is 99.9%, 99.65%, 99.6%, and 98.7%, respectively. Meanwhile, the methods of multisensor fusion and the fused attention mechanism have an average improvement of 1.51% and 1.09%, respectively, by ablation experiments on the WT gearbox dataset. [ABSTRACT FROM AUTHOR]

Rice which is a staple food crop in most Asian countries mainly suffers from higher yield loss due to different factors, and one of the common factors that affect rice yield is rice leaf disease. Many countries do not allow the plantation of new rice varieties unless they are disease-resistant. Hence, disease control serves as an important factor for rice production and this can be effectively accomplished by identifying the rice disease at an early stage which is required for timely pesticide application and disease control. The rice leaf diseases are mainly inspected by the farmers with their naked eye which is error-prone, time-consuming, and often leads to huge yield loss if predicted wrong. This process is effectively handled by different researchers via the use of computer vision and machine learning techniques. To overcome the drawbacks associated with manual processing and low recognition accuracy of different statistical and machine learning techniques, this paper presents a novel concept that uses 3D 2D CNN for feature extraction and an improved backtracking search (IBS) algorithm-optimized deep generative adversarial network (DGAN) for classification. The 3D2D deep convolutional neural network (DCNN) is formed by integrating the 3D fast-learning block with a 2DCNN for extracting the rice disease features such as lesions and shape. The IBS algorithm-optimized GAN architecture is used for rice disease classification and the IBS algorithm is mainly adopted to overcome the instability and overfitting issues that arise in the DGAN. The proposed IBS-optimized GAN architecture offers improved accuracy of 98.7% which is relatively higher than the existing techniques such as extreme gradient boosting (XGBoost), transfer learning techniques, and support vector machine (SVM). [ABSTRACT FROM AUTHOR]

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F 1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach. • Collection of renal pathological images with annotations of glomerulus classes. • A flattened Xception with feature pyramid network (FX-FPN) for glomerulus detection. • Generative glomerulus morphology augmentation to improve classification performance. • A two-stage detection and classification framework with performance evaluation. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

Accurate channel estimation at the base station (BS) is crucial for reliable, high-throughput communication in massive multiple input multiple output (MIMO) systems. However, the process of estimating channel matrices in massive MIMO systems is prone to pilot contamination attacks. These attacks not only degrade the efficiency of channel estimation but also increase the probability of information leakage. In this paper, a method is being proposed for detecting pilot contamination signals using a deep learning model, namely a generative adversarial network (GAN), at the BS of a massive MIMO system. GAN comprises two interconnected models, the first model is the generator, which creates synthetic samples, and the second model is the discriminator, which distinguishes between real and generator-generated samples. Residual block-based convolutional neural networks (CNNs) like ResNet and reverse ResNet are implemented as discriminator model of GAN for improving the detection of pilot-contaminated signals. The generator, structured as an autoencoder, generates artificial data resembling real data from noisy signals. The simulation results demonstrate that the proposed model effectively identifies the pilot-contaminated attacks; this effectiveness is evaluated through performance matrices called accuracy and loss function. Furthermore, the performance of the model is observed by varying the number of BS antennas and pilot lengths. [ABSTRACT FROM AUTHOR]