Search

Medical image diagnosis using deep learning has shown significant promise in clinical medicine. However, it often encounters two major difficulties in real-world applications: (1) domain shift, which invalidates the trained model on new datasets, and (2) class imbalance problems leading to model biases towards majority classes. To address these challenges, this paper proposes a transfer learning solution, named Dynamic Weighting Translation Transfer Learning (DTTL), for imbalanced medical image classification. The approach is grounded in information and entropy theory and comprises three modules: Cross-domain Discriminability Adaptation (CDA), Dynamic Domain Translation (DDT), and Balanced Target Learning (BTL). CDA connects discriminative feature learning between source and target domains using a synthetic discriminability loss and a domain-invariant feature learning loss. The DDT unit develops a dynamic translation process for imbalanced classes between two domains, utilizing a confidence-based selection approach to select the most useful synthesized images to create a pseudo-labeled balanced target domain. Finally, the BTL unit performs supervised learning on the reassembled target set to obtain the final diagnostic model. This paper delves into maximizing the entropy of class distributions, while simultaneously minimizing the cross-entropy between the source and target domains to reduce domain discrepancies. By incorporating entropy concepts into our framework, our method not only significantly enhances medical image classification in practical settings but also innovates the application of entropy and information theory within deep learning and medical image processing realms. Extensive experiments demonstrate that DTTL achieves the best performance compared to existing state-of-the-art methods for imbalanced medical image classification tasks. [ABSTRACT FROM AUTHOR]

The histopathological image classification method, based on deep learning, can be used to assist pathologists in cancer recognition in colon histopathology. The popularization of automatic and accurate histopathological image classification methods in this way is of great significance. However, smaller medical institutions with limited medical resources may lack colon histopathology image training sets with reliable labeled information; thus they may be unable to meet the needs of deep learning for many labeled training samples. Therefore, in this paper, the colon histopathological image set with rich label information from a certain medical institution is taken as the source domain; the colon histopathological image set from a smaller medical institution with limited medical resources is taken as the target domain. Considering the potential differences between histopathological images obtained by different institutions, this paper proposes a classification learning framework, namely unsupervised domain adaptation with local structure preservation for colon histopathological image classification, which can learn an adaptive classifier by performing distribution alignment and preserving intra-domain local structure to predict the labels of the colon histopathological images from institutions with lower medical resources. Extensive experiments demonstrate that the proposed framework shows significant improvement in accuracy and specificity of colon histopathological images without reliable labeled information compared to models without unsupervised domain adaptation. Specifically, in an affiliated hospital in Fuyang City, Anhui Province, the classification accuracy of benign and malignant colon histopathological images reaches 96.21%. The results of comparative experiments also show promising classification performance of our method in comparison with other unsupervised domain adaptation methods. [ABSTRACT FROM AUTHOR]

The recognition of steel microstructure images plays a crucial role in the metallographic analysis process. Although some progress has been made through the application of artificial intelligence algorithms, several challenges remain. First, existing algorithms exhibit weak nonlinear feature extraction capabilities and noticeable limitations. Second, they overlook the intrinsic noise and redundant interference present in microscopic images. To address these issues, this paper investigates the automatic recognition of metallographic tissues by leveraging residual structures in deep neural networks. An enhanced residual network model based on transfer learning is proposed, which utilizes the pre-trained weights from the ImageNet dataset to facilitate learning with small sample data. This network offers higher classification accuracy and higher F1 scores. In addition, a deep residual shrinkage network model based on an attention mechanism is proposed. This model incorporates an attention sub-network into the original residual module and employs a soft threshold function to eliminate redundant features, including noise. The proposed algorithms are evaluated against various convolutional neural networks using 20 types of metallographic test sets. The experimental results showed that both methods have high accuracy rates of 95% and 94.44%, respectively, and F1 scores of 0.9464 and 0.9419. While maintaining the complexity of the model, there has been a significant improvement in accuracy, and the models exhibit strong generalization capabilities. Our research contributes to enhancing production efficiency, strengthening quality control, and improving material performance through computer vision technology. [ABSTRACT FROM AUTHOR]

The recognition of steel microstructure images plays a crucial role in the metallographic analysis process. Although some progress has been made through the application of artificial intelligence algorithms, several challenges remain. First, existing algorithms exhibit weak nonlinear feature extraction capabilities and noticeable limitations. Second, they overlook the intrinsic noise and redundant interference present in microscopic images. To address these issues, this paper investigates the automatic recognition of metallographic tissues by leveraging residual structures in deep neural networks. An enhanced residual network model based on transfer learning is proposed, which utilizes the pre-trained weights from the ImageNet dataset to facilitate learning with small sample data. This network offers higher classification accuracy and higher F1 scores. In addition, a deep residual shrinkage network model based on an attention mechanism is proposed. This model incorporates an attention sub-network into the original residual module and employs a soft threshold function to eliminate redundant features, including noise. The proposed algorithms are evaluated against various convolutional neural networks using 20 types of metallographic test sets. The experimental results showed that both methods have high accuracy rates of 95% and 94.44%, respectively, and F1 scores of 0.9464 and 0.9419. While maintaining the complexity of the model, there has been a significant improvement in accuracy, and the models exhibit strong generalization capabilities. Our research contributes to enhancing production efficiency, strengthening quality control, and improving material performance through computer vision technology. [ABSTRACT FROM AUTHOR]

Dry eye disease (DED) is a chronic eye disease and a common complication among the world's population. Evaporation of moisture from tear film or a decrease in tear production leads to an unstable tear film which causes DED. The tear film breakup time (TBUT) test is a common clinical test used to diagnose DED. In this test, DED is diagnosed by measuring the time at which the first breakup pattern appears on the tear film. TBUT test is subjective, labour-intensive and time-consuming. These weaknesses make a computer-aided diagnosis of DED highly desirable. The existing computer-aided DED detection techniques use expensive instruments for image acquisition which may not be available in all eye clinics. Moreover, among these techniques, TBUT-based DED detection techniques are limited to finding only tear film breakup area/time and do not identify the severity of DED, which can essentially be helpful to ophthalmologists in prescribing the right treatment. Additionally, a few challenges in developing a DED detection approach are less illuminated video, constant blinking of eyes in the videos, blurred video, and lack of public datasets. This paper presents a novel TBUT-based DED detection approach that detects the presence/absence of DED from TBUT video. In addition, the proposed approach accurately identifies the severity level of DED and further categorizes it as normal, moderate or severe based on the TBUT. The proposed approach exhibits high performance in classifying TBUT frames, detecting DED, and severity grading of TBUT video with an accuracy of 83%. Also, the correlation computed between the proposed approach and the Ophthalmologist's opinion is 90%, which reflects the noteworthy contribution of our proposed approach. [ABSTRACT FROM AUTHOR]

Understanding the attitudes of users about different topics through their huge amount of comments and opinions on social networks is an emerging hot issue. Analyzing the sentiments of these posts and comments provides useful information utilized for many applications. Multimodal sentiments often contain more information than single-modal sentiments, including text, image, audio and video, which leads to better performance of multimodal sentiment analysis (MSA) compared to single-modal sentiment analysis (SSA). In this paper, a capsule network-based deep ensemble transfer learning approach, called DSY-ETL-MSA, is proposed for MSA on images and texts, the results of which are fused using Yager theory. Capsule networks and ensemble learning methods improve classification performance, and the deep transfer learning approach reduces the training time. A hybrid deep architecture is used for automatic feature extraction in the proposed method. For analyzing the sentiment of image modality, a pre-trained VGG16 model, fine-tuned on the datasets, is used to extract high-level features for image classification. A capsule convolutional neural network (CNN) is also separately used for extracting image features and classifying them. In the text modality, the pre-trained GloVe model is exploited to embed words and 2 separate classifiers are employed for text classifications. The Yager fusion rules are finally used for early and late fusions of the results of the classifiers. The results of the text and image classifiers are combined separately as the early fusion and the final results of them are fused at the decision level as the late fusion. The proposed model is empirically evaluated on the MVSA and T4SA datasets. The significant performance improvement compared to a variety of former methods is shown in the experimental results. The final accuracies obtained by the proposed method on the MVSA and T4SA datasets are 0.9866 and 0.9996, respectively. [ABSTRACT FROM AUTHOR]

Traditional machine learning methods require the assumption that training and test data are drawn from the same distribution, which proves challenging in real-world applications. Moreover, deep learning models require a substantial amount of labeled data for training in classification tasks and limited samples may lead to overfitting. In many real-world scenarios, there is an insufficient supply of labeled samples within the target domain for learning. Transfer learning offers an effective solution, allowing knowledge from a source domain to be transferred to a target domain. Additionally, data augmentation enhances model generalization by increasing data samples, particularly beneficial when dealing with limited target domain data. In this paper, we synergistically enhance the model's performance on classification tasks by integrating transfer learning techniques with a data augmentation strategy. By conducting numerous experiments across various datasets, we verified the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

Planktonic organisms play a pivotal role within aquatic ecosystems, serving as the foundation of the aquatic food chain while also playing a critical role in climate regulation and the production of oxygen. In recent years, the advent of automated systems for capturing in-situ images has led to a huge influx of plankton images, making manual classification impractical. This, at the same time, has opened up opportunities for the application of machine learning and deep learning solutions. This paper undertakes an extensive analysis of the broad range of computer vision techniques and methodologies that have emerged to facilitate the automatic analysis of small- to large-scale datasets containing plankton images. By focusing on different computer vision tasks, we present findings and limitations in order to offer a comprehensive overview of the current state-of-the-art, while also pinpointing the open challenges that demand further research and attention. • Provides a common level of knowledge in computer vision for plankton image analysis. • Describes related state-of-the-art methods and open-source benchmark datasets. • Provides a comprehensive overview of milestones, limitations, and open challenges. • Analyzes works in plankton image analysis from a modern computer vision viewpoint. [ABSTRACT FROM AUTHOR]

• Double random phase encoding (DRPE) was evaluated on complex datasets obtained from road surveillance cameras in the real world. • Previous works of DRPE cryptanalysis based on deep learning by using only simple datasets were additionally investigated on complex datasets. • The feasibility of DRPE to classification tasks was evaluated on complex datasets by using adaptive transfer learning models. • The proposed DRPE image classification scheme showed the highest classification accuracy compared to state-of-the-art classification schemes, even in a data-limited case. • We identified that recovering DRPE images were challenging, especially in complex datasets, but the imperceptible features representing the class of the images were contained in the DRPE images. Encrypting data can convert original data into a form that cannot be recognized and conceal personal information. To ensure data security, analyzing cryptographic algorithms used in data encryption is critical. The previous works evaluated the robustness of the optical cryptographic algorithm, Double Random Phase Encoding (DRPE), and verified the vulnerability of DRPE to plaintext recovery attacks based on deep learning. Although they could prove that DRPE is not secure by recovering the DRPE images into the original images, they evaluated DRPE only on simple datasets, which were binary or grayscale images with a single object. Thus, in this paper, we investigated the previous works of plaintext recovery on DRPE by using complex datasets. To our knowledge, it is the first study to evaluate DRPE on complex datasets. In addition to the plaintext recovery, we classified DRPE images of complex datasets and verified the feasible points in DRPE for classification tasks based on deep learning. Adaptive transfer learning, the modified version of the classical transfer learning for DRPE image classification, was proposed and utilized to train the DRPE image classification models. Experimental results showed that recovering the DRPE images of the complex datasets was challenging. In contrast, the imperceptible features representing the class of the images were in DRPE images regardless of whether the dataset was simple or complex. The classification performance of the proposed DRPE image classification scheme was better than those of state-of-the-art classification schemes, even with a small dataset. Furthermore, data augmentation with the proposed scheme improved the classification accuracy in DRPE image classification on a small dataset. In addition, our DRPE image classification schemes were applied to encrypted images by other optical cryptographic algorithms and the classification results were compared with that of the encrypted images by classical DRPE. [ABSTRACT FROM AUTHOR]

• A pretrained model on augmented data and fine-tuned by using AI and 3D contour features for assessing facial attractiveness is proposed. • The proposed model overcomes the subjective inconsistency and unreliability common to all traditional rating methods. • This model provides an accurate 3D information of full facial that is unavailable in previous studies using either 2D or 3D measurement. • 3D facial images and deep transfer learning have been firstly combined for evaluating the facial attractiveness in patients undergoing OGS. • The developed web browser–based user interface contributes to effective doctor–patient communication and decision-making. In this paper, we investigate a new approach based on a combination of three-dimensional (3D) facial images and deep transfer learning (TL) with fine-grained image classification (FGIC) for quantitative evaluation of facial attractiveness. The 3D facial surface images of patients with and without filtering and the publicly available SCUT-FBP5500 dataset was used for transfer training and model pre-training, respectively. Experimental results show that a bilinear CNN model with a Gaussian filter freezing 80 % of the weights exhibit the strongest performance and lowest average error as a deep learning prediction model; the model was subsequently adopted for automatic assessment of facial attractiveness in clinical application. This is the first TL model with FGIC using 3D facial images for automatic quantitative evaluation of facial attractiveness in patients undergoing Orthognathic surgery (OGS). The developed web browser–based user interface enables effective and rapid assessment, thus contributing to effective patient–clinician communication and decision-making. [ABSTRACT FROM AUTHOR]

Deep learning architectures have demonstrated outstanding results in image classification in the last few years. However, applying sophisticated neural network architectures in small datasets remains challenging. In this context, transfer learning is a promising approach for dealing with this scenario. Generally, the available pre-trained architectures adopt a standard fixed input, which usually implies resizing and cropping the input images in the preprocessing phase, causing information loss. Besides, images present visual features in different scales in real-world scenarios, and most common approaches do not consider this fact. In this paper, we propose an approach that applies transfer learning for dealing with small datasets and leverages visual features extracted by pre-trained models from different scales. We based our approach on graph convolutional networks (GCN) that take graphs representing the images in different scales as input and whose nodes are characterized by features extracted by pre-trained models from regular image patches of different scales. Since GCN can deal with graphs with different numbers of nodes, our approach can deal naturally with images of heterogeneous sizes without discarding relevant information. We evaluated our approach in two datasets: a set of geological images and a publicly available dataset, both presenting characteristics that challenge traditional approaches. We tested our approach by adopting three different pre-trained models as feature extractors: two efficient pre-trained CNN models (DenseNet and ResNeXt) and one Vision Transformer (CLIP). We compared our approach with two conventional approaches for dealing with image classification. The experiments show that our approach achieves better results than the conventional approaches for this task. • We propose multiscale patch-based graphs to represent image information. • Transfer learning is adopted for dealing with small datasets. • The approach can handle images with heterogeneous sizes. • The approach represents visual information at multiple scales. • The approach is base on convolutional graph neural networks. [ABSTRACT FROM AUTHOR]