Search

Skin cancer is a significant public health issue, making accurate and early diagnosis crucial. This study proposes a novel and efficient hybrid deep‐learning model for accurate skin cancer diagnosis. The model first employs DeepLabV3+ for precise segmentation of skin lesions in dermoscopic images. Feature extraction is then carried out using three pretrained models: MobileNetV2, EfficientNetB0, and DenseNet201 to ensure balanced performance and robust feature learning. These extracted features are then concatenated, and the ReliefF algorithm is employed to select the most relevant features. Finally, obtained features are classified into eight categories: actinic keratosis, basal cell carcinoma, benign keratosis, dermatofibroma, melanoma, melanocytic nevus, squamous cell carcinoma, and vascular lesion using the kNN algorithm. The proposed model achieves an F1 score of 93.49% and an accuracy of 94.42% on the ISIC‐2019 dataset, surpassing the best individual model, EfficientNetB0, by 1.20%. Furthermore, the evaluation of the PH2 dataset yielded an F1 score of 94.43% and an accuracy of 94.44%, confirming its generalizability. These findings signify the potential of the proposed model as an expedient, accurate, and valuable tool for early skin cancer detection. They also indicate combining different CNN models achieves superior results over the results obtained from individual models. [ABSTRACT FROM AUTHOR]

Streamflow time series data typically exhibit nonlinear and nonstationary characteristics that complicate precise estimation. Recently, multifactorial machine learning (ML) models have been developed to enhance the performance of streamflow predictions. However, the lack of interpretability within these ML models raises concerns about their inner workings and reliability. This paper introduces an innovative hybrid architecture, the TCN-LSTM-Multihead-Attention model, which combines two layers of temporal convolutional networks (TCN) followed by one layer of long short-term memory (LSTM) units, integrated with a Multihead-Attention mechanism for predicting streamflow with streamflow causation–driven prediction samples (RCDP), employing local and global interpretability studies through Shapley values and partial dependency analysis. The find_peaks method was used to identify peak flow events in the test dataset, validating the model's generality and uncovering the physical causative patterns of streamflow. The results show that (1) compared to the LSTM model with the same hyperparameter settings, the proposed TCN-LSTM-Multihead-Attention hybrid model increased the R2 by 52.9%, 2.5%, 43.1%, and 10.7% respectively at four stations in the test set predictions using RCDP samples. Moreover, comparing the prediction results of the hybrid model under different samples in Hengshan station, the R2 for RCDP increased by 5.06% and 1.22% compared to streamflow autoregressive prediction samples (RAP) and meteorological-soil volumetric water content coupled autoregressive prediction samples (MCSAP) respectively. (2) Historical streamflow data from the preceding 3 days predominantly influences predictions due to strong autocorrelation, with flow quantity (Q) typically emerging as the most significant feature alongside precipitation (P), surface soil moisture (SSM), and adjacent station flow data. (3) During periods of low and normal flow, historical data remains the most crucial factor; however, during flood periods, the roles of upstream inflow and precipitation become significantly more pronounced. This model facilitates the identification and quantification of various hydrodynamic impacts on flow predictions, including upstream flood propagation, precipitation, and soil moisture conditions. It also elucidates the model's nonlinear relationships and threshold responses, thereby enhancing the interpretability and reliability of streamflow predictions. [ABSTRACT FROM AUTHOR]

Vision transformers (ViTs) are increasingly utilized for HSI classification due to their outstanding performance. However, ViTs encounter challenges in capturing global dependencies among objects of varying sizes, and fail to effectively exploit the spatial–spectral information inherent in HSI. In response to this limitation, we propose a novel solution: the multi-scale spatial–spectral transformer (MSST). Within the MSST framework, we introduce a spatial–spectral token generator (SSTG) and a token fusion self-attention (TFSA) module. Serving as the feature extractor for the MSST, the SSTG incorporates a dual-branch multi-dimensional convolutional structure, enabling the extraction of semantic characteristics that encompass spatial–spectral information from HSI and subsequently tokenizing them. TFSA is a multi-head attention module with the ability to encode attention to features across various scales. We integrated TFSA with cross-covariance attention (CCA) to construct the transformer encoder (TE) for the MSST. Utilizing this TE to perform attention modeling on tokens derived from the SSTG, the network effectively simulates global dependencies among multi-scale features in the data, concurrently making optimal use of spatial–spectral information in HSI. Finally, the output of the TE is fed into a linear mapping layer to obtain the classification results. Experiments conducted on three popular public datasets demonstrate that the MSST method achieved higher classification accuracy compared to state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]

In recent years, the development of super-resolution (SR) algorithms based on convolutional neural networks has become an important topic in enhancing the resolution of multi-channel remote sensing images. However, most of the existing SR models suffer from the insufficient utilization of spectral information, limiting their SR performance. Here, we derive a novel hybrid SR network (HSRN) which facilitates the acquisition of joint spatial–spectral information to enhance the spatial resolution of multi-channel remote sensing images. The main contributions of this paper are three-fold: (1) in order to sufficiently extract the spatial–spectral information of multi-channel remote sensing images, we designed a hybrid three-dimensional (3D) and two-dimensional (2D) convolution module which can distill the nonlinear spectral and spatial information simultaneously; (2) to enhance the discriminative learning ability, we designed the attention structure, including channel attention, before the upsampling block and spatial attention after the upsampling block, to weigh and rescale the spectral and spatial features; and (3) to acquire fine quality and clear texture for reconstructed SR images, we introduced a multi-scale structural similarity index into our loss function to constrain the HSRN model. The qualitative and quantitative comparisons were carried out in comparison with other SR methods on public remote sensing datasets. It is demonstrated that our HSRN outperforms state-of-the-art methods on multi-channel remote sensing images. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been widely used with its powerful discriminative ability in change detection (CD), but most CNN-based methods are still exploring ways to capture relatively long-range context in spatial-temporal domain. The recent vision transformer (ViT), which is based on the self-attention mechanism, has been applied in CD to model long-range dependencies. However, such transformer-based architectures do not fully consider the potential of interdependencies among the high-level semantic feature maps and easily overlook local detail features, resulting in a noncompact interior of the large-scale change area and missing small changes. Therefore, we propose a new transformer-based hybrid network called interact-feature transformer network with spatial detail enhancement module (IFTSDNet), which takes advantage of transformers to capture long-range context, and of CNNs to extract local information. We design an interact-feature transformer (IFT), which can not only obtain the global contextual information, but also achieve the interactions of high-level semantic feature maps. The spatial detail enhancement module (SDEM) with a group of various receptive fields is built to refine spatial features, which incorporates more discriminative feature representations. Comparative experiments prove the effectiveness of the proposed method, which shows better performance than four recent transformer-based methods. The code will be available at https://github.com/wanglinlin0219/IFTSDNet. [ABSTRACT FROM AUTHOR]

Vision transformers (ViTs) are increasingly utilized for HSI classification due to their outstanding performance. However, ViTs encounter challenges in capturing global dependencies among objects of varying sizes, and fail to effectively exploit the spatial–spectral information inherent in HSI. In response to this limitation, we propose a novel solution: the multi-scale spatial–spectral transformer (MSST). Within the MSST framework, we introduce a spatial–spectral token generator (SSTG) and a token fusion self-attention (TFSA) module. Serving as the feature extractor for the MSST, the SSTG incorporates a dual-branch multi-dimensional convolutional structure, enabling the extraction of semantic characteristics that encompass spatial–spectral information from HSI and subsequently tokenizing them. TFSA is a multi-head attention module with the ability to encode attention to features across various scales. We integrated TFSA with cross-covariance attention (CCA) to construct the transformer encoder (TE) for the MSST. Utilizing this TE to perform attention modeling on tokens derived from the SSTG, the network effectively simulates global dependencies among multi-scale features in the data, concurrently making optimal use of spatial–spectral information in HSI. Finally, the output of the TE is fed into a linear mapping layer to obtain the classification results. Experiments conducted on three popular public datasets demonstrate that the MSST method achieved higher classification accuracy compared to state-of-the-art (SOTA) methods.

In recent years, the development of super-resolution (SR) algorithms based on convolutional neural networks has become an important topic in enhancing the resolution of multi-channel remote sensing images. However, most of the existing SR models suffer from the insufficient utilization of spectral information, limiting their SR performance. Here, we derive a novel hybrid SR network (HSRN) which facilitates the acquisition of joint spatial–spectral information to enhance the spatial resolution of multi-channel remote sensing images. The main contributions of this paper are three-fold: (1) in order to sufficiently extract the spatial–spectral information of multi-channel remote sensing images, we designed a hybrid three-dimensional (3D) and two-dimensional (2D) convolution module which can distill the nonlinear spectral and spatial information simultaneously; (2) to enhance the discriminative learning ability, we designed the attention structure, including channel attention, before the upsampling block and spatial attention after the upsampling block, to weigh and rescale the spectral and spatial features; and (3) to acquire fine quality and clear texture for reconstructed SR images, we introduced a multi-scale structural similarity index into our loss function to constrain the HSRN model. The qualitative and quantitative comparisons were carried out in comparison with other SR methods on public remote sensing datasets. It is demonstrated that our HSRN outperforms state-of-the-art methods on multi-channel remote sensing images.

This paper introduces a hybrid filter bank-based convolutional network to develop a 3D face recognition system in different orientations. The filter banks approach has been mainly used for feature representation. The hybridization in filter banks is primarily generated by a fusion of principal component analysis (PCA) and independent component analysis (ICA) filters. Currently, the deep convolutional neural network (DCNN) has taken a significant step for improving the classification compared to other learning, though the feature learning mechanism of DCNN is not definite. We have used the cascaded linear convolutional network for 3D face classification using a composite filter-based network named PICANet. The networks consist of different layers: convolutional layer, nonlinear processing layer, pooling layer, and classification layer. The main advantage of these networks over DCNN is that the network structure is simple and computationally efficient. We have tested the proposed system on three accessible 3D face databases: Frav3D, GavabDB, and Casia3D. Considering different faces in Frav3D, GavabDB, and Casia3D, the system acquired 96.93%, 87.7%, and 89.21% recognition rates using the proposed hybrid network. [ABSTRACT FROM AUTHOR]

Smoke has visually elusive appearances, especially in low-light conditions, so it is quite difficult to quickly and accurately detect smoke from images. To address these challenges, we design a dual-encoder structure of Transformer and Convolutional Neural Network (CNN) to propose an effective Multi-scale Interactive Fusion Network (MIFNet) for smoke image segmentation. To improve the presentation of features, we propose a Local Feature Enhancement Propagation (LFEP) module to enhance spatial details. To optimize global and local features for efficient fusion, we integrate LFEP into the original Transformer to replace the traditional multi-head self-attention mechanism. Then, we propose a Multi-level Attention Coupled Module (MACM) to fuse Transformer and CNN features of the dual-encoder. MACM can flexibly focus on information interaction between different levels of two encoding paths. Finally, we design a Prior-guided Multi-scale Fusion Decoder (PMFD), which combines prior knowledge with a multi-scale feature fusion strategy to improve the performance of segmentation. Experimental results demonstrate that MIFNet substantially outperforms the state-of-the-art methods. MIFNet achieves a mean Intersection over Union (mIoU) of 81.6 % on the synthetic smoke (SYN70 K) dataset, and a remarkable accuracy of 98.3 % on the forest smoke dataset. [ABSTRACT FROM AUTHOR]

Aiming to bridge the gaps identified in existing research, which lacks insights from the fusion of deep learning methods and multi-source factors for drought prediction, and the interpretability of factors influencing drought levels, this research develops an innovative, interpretable, and explainable deep HybridNet model, Squeeze-and-Excitation Convolutional Neural Network Bidirectional Long Short-Term Memory with Attention (SE-CNN-BiLSTM-Attention), for standardized precipitation index (SPI) prediction utilizing large-scale climate indices and local meteorological conditions. It not only achieves high accuracy but also elucidates the influencing factors of drought levels across various events and regions, capturing both long-term trends and short-term, real-time dynamics. Key findings include: (1) The hybrid model, which blends historical SPI sequences with climatic features and atmospheric circulation factors, notably improves SPI prediction accuracy over multiple time scales compared to other models discussed in this paper and demonstrates greater sensitivity to moderate and mild droughts. (2) Shorter time scales yield less accurate predictions due to their nonlinearity and volatility. In contrast, longer time scales capture long-term patterns and trends more effectively, enhancing predictive accuracy and serving as the optimal time scale for calculating the SPI and modeling drought conditions. (3) The prediction accuracy of SPI in arid and humid regions surpasses that in semi-arid and semi-humid central areas. With longer time scales, regions with high R2 values expand from northern and eastern areas to a broader range. (4) Historical SPI and precipitation patterns are significant predictors in drought prediction models across different climatic zones, with large-scale atmospheric circulation also playing a critical role. At Mohe Station, with minimal atmospheric circulation impact, local climate primarily determines the SPI. At Turpan Station, despite having high temperatures, temperature’s influence on SPI is limited. (5) During the 1967–1969 drought in Turpan, historical SPI’s negative impact on predictions peaked at an extreme drought level (SPI24 =  − 2.21). The shift from extreme drought in June 1968 to light drought in 1969 was mainly driven by increased daily precipitation and mean temperature. This study provides a framework for multi-scale, cross-regional drought risk assessment and addresses the previously unmet demand for more detailed and accurate forecasting methods under different climate conditions. [ABSTRACT FROM AUTHOR]

Accurate estimation of lithium-ion battery state-of-health (SOH) is important for the safe operation of electric vehicles; however, in practical applications, the accuracy of SOH estimation is affected by uncertainty factors, including human operation, working conditions, etc. To accurately estimate the battery SOH, a hybrid neural network based on the dilated convolutional neural network and the bidirectional gated recurrent unit, namely dilated CNN-BiGRU, is proposed in this paper. The proposed data-driven method uses the voltage distribution and capacity changes in the extracted battery discharge curve to learn the serial data time dependence and correlation. This method can obtain more accurate temporal and spatial features of the original battery data, resulting higher accuracy and robustness. The effectiveness of dilated CNN-BiGRU for SOH estimation is verified on two publicly lithium-ion battery datasets, the NASA Battery Aging Dataset and Oxford Battery Degradation Dataset. The experimental results reveal that the proposed model outperforms the compared data-driven methods, e.g., CNN-series and RNN-series. Furthermore, the mean absolute error (MAE) and root mean square error (RMSE) are limited to within 1.9% and 3.3%, respectively, on the NASA Battery Aging Dataset. [ABSTRACT FROM AUTHOR]

We present a processing-in-memory (PIM)-based hardware platform, referred to as MONETA, for on-chip acceleration of inference and learning in hybrid convolutional spiking neural network. MONETAuses 8T static random-access memory (SRAM)-based PIM cores for vector matrix multiplication (VMM) augmented with spike-time-dependent-plasticity (STDP) based weight update. The spiking neural network (SNN)-focused data flow is presented to minimize data movement in MONETAwhile ensuring learning accuracy. MONETAsupports on-line and on-chip training on PIM architecture. The STDP-trained convolutional neural network within SNN (ConvSNN) with the proposed data flow, 4-bit input precision, and 8-bit weight precision shows only 1.63% lower accuracy in CIFAR-10 compared to the STDP accuracy implemented by the software. Further, the proposed architecture is used to accelerate a hybrid SNN architecture that couples off-chip supervised (back propagation through time) and on-chip unsupervised (STDP) training. We also evaluate the hybrid network architecture with the proposed data flow. The accuracy of this hybrid network is 10.84% higher than STDP trained accuracy result and 1.4% higher compared to the backpropagated training-based ConvSNN result with the CIFAR-10 dataset. Physical design of MONETAin 65 nm complementary metal-oxide-semiconductor (CMOS) shows 18.69 tera operation per second (TOPS)/W, 7.25 TOPS/W and 10.41 TOPS/W power efficiencies for the inference mode, learning mode, and hybrid learning mode, respectively. [ABSTRACT FROM AUTHOR]

The pre-trained word embedding models become widely used in Natural Language Processing (NLP), but they disregard the context and sense of the text. We study in this paper, the capacity of pre-trained BERT model (Bidirectional Encoder Representations from Transformers) for the Arabic language to classify Arabic tweets using a hybrid network of two famous models; Bidirectional Long Short Term Memory (BiLSTM) and Gated Recurrent Unit (GRU) inspired by the great achievement of deep learning algorithms. In this context, we finetuned the Arabic BERT (AraBERT) parameters and we used it on three merged datasets to impart its knowledge for the Arabic sentiment analysis. For that, we lead the experiments by comparing the AraBERT model in one hand in the word embedding phase, with a statics pre-trained word embeddings method namely AraVec and FastText, and on another hand in the classification phase, we compared the hybrid model with convolutional neural network (CNN), long short-term memory (LSTM), BiLSTM, and GRU, which are prevalently preferred in sentiment analysis. The results demonstrate that the fine-tuned AraBERT model, combined with the hybrid network, achieved peak performance with up to 94% accuracy. [ABSTRACT FROM AUTHOR]

Purpose: Spatial resolution is an important parameter for magnetic resonance imaging (MRI). High‐resolution MR images provide detailed information and benefit subsequent image analysis. However, higher resolution MR images come at the expense of longer scanning time and lower signal‐to‐noise ratios (SNRs). Using algorithms to improve image resolution can mitigate these limitations. Recently, some convolutional neural network (CNN)‐based super‐resolution (SR) algorithms have flourished on MR image reconstruction. However, most algorithms usually adopt deeper network structures to improve the performance. Methods: In this study, we propose a novel hybrid network (named HybridNet) to improve the quality of SR images by increasing the width of the network. Specifically, the proposed hybrid block combines a multipath structure and variant dense blocks to extract abundant features from low‐resolution images. Furthermore, we fully exploit the hierarchical features from different hybrid blocks to reconstruct high‐quality images. Results: All SR algorithms are evaluated using three MR image datasets and the proposed HybridNet outperformed the comparative methods with peak a signal‐to‐noise ratio (PSNR) of 42.12 ± 0.92 dB, 38.60 ± 2.46 dB, 35.17 ± 2.96 dB and a structural similarity index (SSIM) of 0.9949 ± 0.0015, 0.9892 ± 0.0034, 0.9740 ± 0.0064, respectively. Besides, our proposed network can reconstruct high‐quality images on an unseen MR dataset with PSNR of 33.27 ± 1.56 and SSIM of 0.9581 ± 0.0068. Conclusions: The results demonstrate that HybridNet can reconstruct high‐quality SR images from degraded MR images and has good generalization ability. It also can be leveraged to assist the task of image analysis or processing. [ABSTRACT FROM AUTHOR]

Multilabel image classification aims to assign images to multiple possible labels. In this task, each image may be associated with multiple labels, making it more challenging than the single-label classification problems. For instance, convolutional neural networks (CNNs) have not met the performance requirement in utilizing statistical dependencies between labels in this study. Additionally, data imbalance is a common problem in machine learning that needs to be considered for multilabel medical image classification. Furthermore, the concatenation of a CNN and a transformer suffers from the disadvantage of lacking direct interaction and information exchange between the two models. To address these issues, we propose a novel hybrid deep learning model called CTransCNN. This model comprises three main components in both the CNN and transformer branches: a multilabel multihead attention enhanced feature module (MMAEF), a multibranch residual module (MBR), and an information interaction module (IIM). The MMAEF enables the exploration of implicit correlations between labels, the MBR facilitates model optimization, and the IIM enhances feature transmission and increases nonlinearity between the two branches to help accomplish the multilabel medical image classification task. We evaluated our approach using publicly available datasets, namely the ChestX-ray11 and NIH ChestX-ray14, along with our self-constructed traditional Chinese medicine tongue dataset (TCMTD). Extensive multilabel image classification experiments were conducted comparing our approach with excellent methods. The experimental results demonstrate that the framework we have developed exhibits strong competitiveness compared to previous research. Its robust generalization ability makes it applicable to other medical multilabel image classification tasks. [ABSTRACT FROM AUTHOR]

The revolutionary progress of deep learning and data mining technology has greatly contributed to the rapid development of maritime Internet of Things. In order to advance intelligent maritime traffic services, it is necessary to accurately predict the future trajectories of vessels, which is conducive to collision avoidance, maritime surveillance and abnormal behavior detection. In this work, a new method of vessel trajectory prediction based on sparse multi-graph convolutional hybrid network with spatio-temporal awareness is proposed, namely SMCHN model, which uses sparse spatial graph to capture the adaptive interactions between vessels and remove redundant interactions, and sparse temporal graph can be used to simulate the movement trends of vessels, bringing the movement behavior of vessels closer and closer to reality. In addition, the hybrid network is designed to fuse information from different sources by adjusting weights. Finally, the temporal convolutional network with gating mechanism is used to predict the future trajectories of vessels. Experimental results show that the SMCHN model improves 31%, 66% and 54% in short-term, medium-term and long-term vessel trajectory prediction compared with the optimal baseline model. • We propose a new vessel trajectory prediction model based on sparse multi-graph convolutional hybrid network, which simulates effective interactions and movement trends between vessels in the spatial and temporal dimensions, and embeds the vessel trajectory features with spatio-temporal awareness into the prediction framework. • We use the sparse spatial graph to model effective social interactions between vessels, and the sparse temporal graph to model the movement trends of vessels to improve the predictive ability of vessel trajectories. They use the self-attention mechanism to learn the asymmetric dense interaction scores between vessel trajectory points, and then these dense interaction scores are fused and fed back into the asymmetric convolutional neural network to obtain sparse social interaction features and movement trend features between vessels. • We use the graph convolutional network for cascade learning, and the hybrid network can dynamically adjust the weight according to the contribution degree of features to fuse the sparse social interaction features and movement trend features of vessels. Then, we use the temporal convolutional network with gating mechanism to learn the deep feature representation and predict the parameters of bivariate Gaussian distribution of vessel trajectories, and generate the future trajectories of vessels. • We demonstrate the advantages of the proposed model over state-of-the-art methods on a natural and publicly available AIS dataset. The experimental results show that our proposed model can more accurately predict short-term, medium-term and long-term trajectories of vessels, and provide technical support for maritime vessel monitoring and early warning. [ABSTRACT FROM AUTHOR]

Traffic Signs Recognition (TSR) is a key technology to implement Automatic Driving System (ADS) and Advanced Driver Assistant System (ADAS). Numerous efforts have been endeavored to improve the TSR accuracy and speed, and Convolution Neural Networks (CNN) are usually employed. However, most of the existing works require large network scale and thus suffer from the drawback of high computation and power consumption. To alleviate this issue, in this paper, we propose an SNN–CNN hybrid network with RRAM-implemented weights to improve the storage and computing efficiency of TSR. Specifically, we utilize SNN to quickly determine the superclass that a traffic sign belongs to. Then utilize CNNs to determine the subclass accurately, thus the complex TSR task is decomposed into multiple simple tasks. Finally, the network weights are binarized to alleviate the implementation of the hybrid network on RRAM-based accelerators, which have higher power efficiency than CMOS logic based ones. Experimental results indicate that, when compared with state of the art CNN method, the SNN–CNN hybrid network can achieve similar accuracy with less weight scale (−69.21%) and lower power consumption (−81.55%). Binarizing the hybrid network can further reduce the weights storage requirement by another 96.875% with only 1.518% accuracy loss. When the network weights are implemented with state of the art RRAM array (resistance variation is 5%), the proposed network can achieve a mean accuracy of 96.47% with 95% ∼ 97.5% confidence interval. [ABSTRACT FROM AUTHOR]