Search

Deep convolutional neural network (DCNN) models have been widely used to diagnose skin lesions, and some of them have achieved diagnostic results comparable to or even better than dermatologists. Most publicly available skin lesion datasets used to train DCNN were dermoscopic images. Expensive dermoscopic equipment is rarely available in rural clinics or small hospitals in remote areas. Therefore, it is of great significance to rely on clinical images for computer-aided diagnosis of skin lesions. This paper proposes an improved dual-branch fusion network called CR-Conformer. It integrates a DCNN branch that can effectively extract local features and a Transformer branch that can extract global features to capture more valuable features in clinical skin lesion images. In addition, we improved the DCNN branch to extract enhanced features in four directions through the convolutional rotation operation, further improving the classification performance of clinical skin lesion images. To verify the effectiveness of our proposed method, we conducted comprehensive tests on a private dataset named XJUSL, which contains ten types of clinical skin lesions. The test results indicate that our proposed method reduced the number of parameters by 11.17 M and improved the accuracy of clinical skin lesion image classification by 1.08%. It has the potential to realize automatic diagnosis of skin lesions in mobile devices. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer. [ABSTRACT FROM AUTHOR]

Peripheral vision is a vital component of human visual processing that allows for efficient and accurate recognition of visual features across diverse regions of the visual field. Analogously, endoscopic images often exhibit peripheral regions of blur, due to their inherent imaging properties. Previous strategies employing either coarse-grained global attention or fine-grained local attention to enhance performance have often inadvertently compromised the intrinsic self-attention mechanism of multilayer transformers, leading to less optimal solutions. This research introduces Self-Peripheral-Attention (SPA), an innovative mechanism that incorporates peripheral vision modeling into self-attention, so as to enhance the accuracy and efficiency of classification and segmentation tasks in endoscopic imaging. SPA synthesizes fine-grained central and coarse-grained peripheral interactions and possesses three primary characteristics: (i) peripheral contextualization aggregation; (ii) interaction between coarse-grained peripheral and fine-grained central features facilitated by depthwise dilated convolution; (iii) element-wise affine transformation to integrate attention into the value. The effectiveness and generalizability of the proposed SPA-Net were assessed on XJUEE, XJUEE-SEG, Kvasir and Kvasir-SEG endoscopy datasets. The results underscore the potential of peripheral vision modeling in self-attention for augmenting machine perception models. The associated code can be accessed at https://github.com/huoxiangzuo/SPA. • Self-Peripheral-Attention blends peripheral vision with self-attention. • Central detail features and multi-level peripheral features adaptively fused. • SPA-Net's performance validated with four endoscopic datasets. • Peripheral vision in self-attention improves machine perception potential. [ABSTRACT FROM AUTHOR]