Search

• For fishing net detection, range gated laser imaging technology is used to obtain high quality underwater fishing net images with less water backscattering and background noise. • For fishing net recognition, a deep-learning based dual-phase training strategy is proposed to avoid overfitting. • The highest overall accuracy of the proposed method reaches 95.49% in fishing net classification task. In this paper, a mask-guided deep learning fishing net detection and recognition method based on underwater range gated laser imaging is proposed. Range gated laser imaging technology is used to obtain high quality underwater fishing net images with less water backscattering effect and background noise. A dual-phase training strategy including mask-guided feature extraction phase and classification finetune phase is proposed to avoid overfitting of training the neural network. The mask-guided feature extraction phase takes advantages of image segmentation training from synthetic dataset to get a better feature extraction performance. The highest overall accuracy of the proposed method reaches 95.49% in fishing net classification task under finetuned weight configuration. The proposed method can effectively help unmanned underwater vehicles and robots from entangling by fishing nets as well as retrieving derelict fishing nets for marine environment protection. [ABSTRACT FROM AUTHOR]

• We provide a UAV-based multispectral imagery approach objectifying FHM strategies. • A CNN was trained using 17,643 images of trees with health status and species labels. • Analysis was done at 235 plots distributed across Bavaria over three years. • The highly heterogenous dataset benefits model generalization and transferability. • We provide a pipeline to generate reference data and for DL image classification. Uncrewed Aerial Vehicles (UAVs) have emerged as a promising tool for complementing terrestrial surveys, offering unique advantages for forest health monitoring (FHM). UAVs have the potential to improve or even replace core tasks such as crown condition assessment, bridging the gap between ground-based surveys and traditional remote sensing platforms. However, present approaches have not yet fully exploited the very high temporal resolution and flexible and convenient utilization that UAVs offer even under cloudy skies. In this paper, we provide a standardized data pipeline to semi-automatically generate reference data and for monitoring forest health by merging ground-based and UAV-based data related to species-specific forest health. Furthermore, we investigated the potential of Convolutional Neural Networks (CNNs) to classify the main tree species and their crown conditions based on the reference data. Therefore, we acquired high resolution multispectral drone imagery of 235 different ICP large scale forest monitoring plots (Level-I plots) distributed across Bavaria for three consecutive years (2020–2022). Using this highly heterogeneous time-series dataset, encompassing diverse weather and lighting conditions, forest stand characteristics, and spatial distribution of study areas, we successfully classified five tree species, three genus level classes and dead trees, including the health status of the main tree species occurring in Germany. This way we managed to classify 14 distinct classes with an average macro F1-score of 0.61 using the EfficientNet CNN architecture. The highest class-specific F1-score apart from the class of dead trees (0.97) was achieved by the class of Picea abies healthy (0.80). If participating countries of the ICP Forests program adopt our approach to harmonize terrestrial and UAV-based monitoring, many ground-based tasks could be reduced or replaced, leading to significant time and cost savings. We provide standardized and open-source monitoring and analysis strategies that can be potentially extended throughout Europe. Our findings demonstrate that UAV monitoring and deep learning can modernize forest management for efficiency and sustainability. We recommend integrating drones with ground surveys in forest monitoring systems to take advantage of their benefits. [ABSTRACT FROM AUTHOR]

Gait recognition is an important biometric technology that allows for the remote collection of stakeholders' characteristics, without requiring their explicit cooperation. It has gained considerable attention in the fields of criminal investigation and intelligent security. Previous studies have shown that local gait features can enhance gait recognition performance by improving robustness to disturbances. However, global gait features also play a crucial role in gait recognition. Many researchers have utilized convolutional operations to extract global features, but these operations tend to focus on features within the receptive field, neglecting those outside of it. Therefore, the potential of global gait features has not been fully explored. In this paper, we propose a gait recognition framework based on vision transformers, aiming to enhance the extraction of global gait features. We introduce an adaptive multi-frame global feature mapping (AMGM) method to address the challenge of inconsistent feature dimensions caused by variations in the number of gait frames when fusing global and local features. We evaluate our model on the latest datasets, and the experimental results demonstrate a significant breakthrough. Notably, our model achieves state-of-the-art recognition accuracy, particularly in scenarios where subjects are wearing coats. Additionally, our model achieves remarkable improvements in recognition accuracy through training with small sample sets. [ABSTRACT FROM AUTHOR]

Vision transformers (ViTs) have become one of the dominant frameworks for vision tasks in recent years because of their ability to efficiently capture long-range dependencies in image recognition tasks using self-attention. In fact, both CNNs and ViTs have advantages and disadvantages in vision tasks, and some studies suggest that the use of both may be an effective way to balance performance and computational cost. In this paper, we propose a new hybrid network based on CNN and transformer, using CNN to extract local features and transformer to capture long-distance dependencies. We also proposed a new feature map resolution reduction based on Discrete Cosine Transform and self-attention, named DCT-Attention Down-sample (DAD). Our DctViT-L achieves 84.8% top-1 accuracy on ImageNet 1K, far outperforming CMT, Next-ViT, SpectFormer and other state-of-the-art models, with lower computational costs. Using DctViT-B as the backbone, RetinaNet can achieve 46.8% mAP on COCO val2017, which improves mAP by 2.5% and 1.1% with less calculation cost compared with CMT-S and SpectFormer as the backbone. • We propose a DAD block for feature map compression and feature extraction using DCT. • We design new convolutional blocks and transformer blocks. • We propose a CNN and Transformer hybrid neural network family DctViT. • We experimentally demonstrate the superior performance of the DctViT network. [ABSTRACT FROM AUTHOR]

Understanding the attitudes of users about different topics through their huge amount of comments and opinions on social networks is an emerging hot issue. Analyzing the sentiments of these posts and comments provides useful information utilized for many applications. Multimodal sentiments often contain more information than single-modal sentiments, including text, image, audio and video, which leads to better performance of multimodal sentiment analysis (MSA) compared to single-modal sentiment analysis (SSA). In this paper, a capsule network-based deep ensemble transfer learning approach, called DSY-ETL-MSA, is proposed for MSA on images and texts, the results of which are fused using Yager theory. Capsule networks and ensemble learning methods improve classification performance, and the deep transfer learning approach reduces the training time. A hybrid deep architecture is used for automatic feature extraction in the proposed method. For analyzing the sentiment of image modality, a pre-trained VGG16 model, fine-tuned on the datasets, is used to extract high-level features for image classification. A capsule convolutional neural network (CNN) is also separately used for extracting image features and classifying them. In the text modality, the pre-trained GloVe model is exploited to embed words and 2 separate classifiers are employed for text classifications. The Yager fusion rules are finally used for early and late fusions of the results of the classifiers. The results of the text and image classifiers are combined separately as the early fusion and the final results of them are fused at the decision level as the late fusion. The proposed model is empirically evaluated on the MVSA and T4SA datasets. The significant performance improvement compared to a variety of former methods is shown in the experimental results. The final accuracies obtained by the proposed method on the MVSA and T4SA datasets are 0.9866 and 0.9996, respectively. [ABSTRACT FROM AUTHOR]

• As the steep switching device has double-gate, the threshold voltage can be adaptively controlled. • The adjustment of the threshold can be employed to mitigate the accuracy deterioration. • Off current of proposed integrate and fire neuron circuit decreases significantly. • Adjustable threshold of the neuron circuit can achieve a high inference accuracy. Deep learning has shown impressive capabilities in tasks like speech recognition and image classification. However, modern deep neural networks often demand a significant number of weights and extensive computational resources, creating efficiency challenges for applications on edge devices. To address these issues, researchers have introduced deep spiking neural networks (DSNNs) that leverage specialized hardware for synapses and neurons. DSNNs offer a potential solution by improving efficiency in edge-device applications. In this paper, the hardware based DSNN with integrate and fire neuron using steep switching device was investigated. We propose integrate and fire neuron using steep switching device to implement rate coding as input encoding method. Because the steep switching device has double-gate, the threshold voltage of the neuron circuits can be adaptively controlled, which changes the rates of input pulse. Hence, the adjustment of the threshold of neuron can be employed to mitigate the accuracy deterioration resulting from the transformation from deep neural networks (DNNs) to DSNNs. In addition, the off-current of proposed integrate and fire neuron circuit decreases significantly as the steep switching device has steep subthreshold swing. A system simulation of a hardware based DSNN shows that the adjustable threshold of the neuron circuit can achieve a high inference accuracy of 98.36 % which is comparable to that obtained with software based DNN. [ABSTRACT FROM AUTHOR]

Deep learning-based no-reference image quality assessment (NRIQA) methods have demonstrated advanced performance. In this paper, a deep learning-based NRIQA method with strong error-aware and content-aware capabilities is proposed, which consists of Error-Aware Reconstruction Network (EARNet) module, Content Feature Extraction Network (CFENet) module, and Subjective Quality Regression Network (SQRNet) module. We first build a database to pre-train EARNet to obtain the ability to extract error features. For content features, CFENet pre-trained on large-scale image classification tasks is adopted to extract. The pre-trained EARNet and CFENet are serially connected with SQRNet so that the features received by SQRNet are both error-aware and content-aware. Extensive experimental results show that the proposed method achieves state-of-the-art performance on many well-known IQA databases. The robustness of the proposed method is verified on the large-scale Waterloo Exploration Database (WED), and its superiority is demonstrated by the group maximum differentiated (gMAD) competition game. Furthermore, we also verify that the proposed EARNet is highly extensible, which can further improve the performance of the existing deep learning-based NRIQA method. • A blind image quality assessment method based on deep learning is proposed. • The proposed model has both error and content awareness. • The proposed method has advanced performance and high scalability. • Creatively designed a joint loss to guide the model in generating error maps. [ABSTRACT FROM AUTHOR]

The performance of diagnostic Computer-Aided Design (CAD) systems for retinal diseases depends on the quality of the retinal images being screened. Thus, many studies have been developed to evaluate and assess the quality of such retinal images. However, most of them did not investigate the relationship between the accuracy of the developed models and the quality of the visualization of interpretability methods for distinguishing between gradable and non-gradable retinal images. Consequently, this paper presents a novel framework called "FGR-Net" to automatically assess and interpret underlying fundus image quality by merging an autoencoder network with a classifier network. The FGR-Net model also provides an interpretable quality assessment through visualizations. In particular, FGR-Net uses a deep autoencoder to reconstruct the input image in order to extract the visual characteristics of the input fundus images based on self-supervised learning. The extracted features by the autoencoder are then fed into a deep classifier network to distinguish between gradable and ungradable fundus images. FGR-Net is evaluated with different interpretability methods, which indicates that the autoencoder is a key factor in forcing the classifier to focus on the relevant structures of the fundus images, such as the fovea, optic disk, and prominent blood vessels. Additionally, the interpretability methods can provide visual feedback for ophthalmologists to understand how our model evaluates the quality of fundus images. The experimental results showed the superiority of FGR-Net over the state-of-the-art quality assessment methods, with an accuracy of > 89 % and an F1-score of > 87 %. The code is publicly available at https://github.com/saifalkh/FGR-Net. • FGR-NET is a deep learning model to assess fundus image gradeability. • FGR-NET consists of two parallel deep networks: autoencoder and classifier. • Using self-supervised learning, autoencoder is to extract the images' key features. • A classifier is to distinguish between gradable and ungradable fundus images. • Interpretability is performed to understand how FGR-NET evaluates the gradeability. [ABSTRACT FROM AUTHOR]

Food image classification is an important research direction in the field of computer vision and machine learning. However food image classification faces great challenges when dealing with foods with similar shapes but different nutritional values. In order to improve this problem, this paper proposes a high-accuracy food image classification with data augmentation and feature enhancement through vision transformer (AlsmViT), which can accurately handle foods with similar shapes but different nutritional values, which is expected to help people better manage their diet and improve their health. Our approach incorporates Augmentplus, LayerScale, and multi-layer perception mechanisms for feature local enhancement. Our models are trained and validated on the public datasets Food-101 and Vireo Food-172, respectively, where the accuracy of the AlsmViT-L model validation set is 95.17% and 94.29%, respectively. Compared with other state-of-the-art self-supervised methods, our proposed method exhibits higher accuracy in food image classification tasks. • A food image classification method based on Vision Transformer was developed. • Data enhancement solves the over-fitting problem of Vision Transformer in small sample data sets. • This method combines a multi-layer perception mechanism with data enhancement and feature local enhancement. • The classification results have higher accuracy on two challenging food data sets. [ABSTRACT FROM AUTHOR]

Traditional machine learning methods require the assumption that training and test data are drawn from the same distribution, which proves challenging in real-world applications. Moreover, deep learning models require a substantial amount of labeled data for training in classification tasks and limited samples may lead to overfitting. In many real-world scenarios, there is an insufficient supply of labeled samples within the target domain for learning. Transfer learning offers an effective solution, allowing knowledge from a source domain to be transferred to a target domain. Additionally, data augmentation enhances model generalization by increasing data samples, particularly beneficial when dealing with limited target domain data. In this paper, we synergistically enhance the model's performance on classification tasks by integrating transfer learning techniques with a data augmentation strategy. By conducting numerous experiments across various datasets, we verified the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

The acquisition of rock property information is at the core of regional geological survey and mineral exploration, but hand-crafted feature-based methods are heavily influenced by human prior knowledge and have limited transferability. End-to-end deep learning techniques, exemplified by convolutional neural networks (CNNs), have attained significant accomplishments in the domain of image classification. However, previous end-to-end CNN-based methods are hard to focus on image critical areas, and they also cannot make full use of global alignment dependency of the rocks. In this paper, rock image classification via a RockS2Net is proposed. The RockS2Net framework adopts a Siamese architecture, characterized by two branches that share parameters, enabling the efficient extraction of both global and local features. This approach facilitates the extraction of global features from entire images and focuses on critical areas to extract local features. The architecture of spatial transformer network (STN) is introduced to transform microscopic images of rock sections to their critical areas. By fusing local features and global features, the properties obtained from microscopic images of rock sections can be more accurately predicted. To test the robust generalizability of the proposed method, the constructed CHN-Rock images dataset is used for experiments and evaluation. Experimental results show that the accuracy of the proposed RockS2Net on the CHN-Rock image dataset is 2–3% higher than that of other rock image classification networks. • The proposed RockS2Net can effectively extract local features of the critical areas and global features from microscopic images of rock sections. • The proposed SGD block, which can strengthen the transfer of features and reduces the loss of features, fuses the two-stream global-local features, and discriminative features of the rock sections can be obtained. • The proposed STN blocks capture key regions by transforming microscopic images of rock sections to reduce the influence of irrelevant features on rock image classification. • The introduced idea of extended convolution is applied to the framework for solving the information loss caused by rock image resolution reduction and down sampling in rock image classification problems. [ABSTRACT FROM AUTHOR]

In this paper, we present H-CapsNet, a capsule network for hierarchical image classification. Our network makes use of the natural capacity of CapsNets (capsule networks) to capture hierarchical relationships. Thus, our network is such that each multi-layer capsule network accounts for each of the class hierarchies using dedicated capsules. Further, we make use of a modified hinge loss that enforces consistency amongst the hierarchies involved. We also present a strategy to dynamically adjust the training parameters to achieve a better balance between the class hierarchies under consideration. We have performed experiments using several widely available datasets and compared them against several alternatives. In our experiments, H-CapsNet delivers a margin of improvement over competing hierarchical classification networks elsewhere in the literature. • Propose H-CapsNet for hierarchical classification with dedicated capsules per level. • Enforce hierarchical consistency with label-tree guided modified hinge-loss. • Adjust training parameters dynamically to balance hierarchical level contributions. • Deliver a margin of improvement over alternatives on benchmark datasets. [ABSTRACT FROM AUTHOR]

Pine Wilt Disease (PWD) is a devastating disease that affects forest ecosystems and has no known cure. Early detection is essential for suppressing infections of PWD. Recent efforts have focused on leveraging deep learning-based algorithms for the early detection of PWD. However, data collection, labeling, and quality assurance for such a method are costly and challenging. Particularly for PWD, the data collection period is limited, restricting the diversity of the dataset. To overcome these challenges, this paper introduces a virtual forest considering PWD created using 3D rendering tools, from which we built a synthetic dataset. Furthermore, to ensure the resemblance of the synthetic data to the real data, we employed Image-to-Image (I2I) translation techniques. We used the EfficientNetv2-S model to compare the results from each dataset. We were able to confirm the potential of our model trained solely on the PWD synthetic dataset for real PWD detection. Moreover, the model trained on an ensemble comprising both real and synthetic data exhibited improved performance, achieving an F1 Score of 92.88%. Using the I2I translation technique for an ensemble of real and synthetic data also demonstrated enhanced performance. This result confirms the validity and practicality of the synthetic data proposed in this study. The findings of this study are expected to contribute to the broader forest ecosystem preservation and agricultural management by extending to other forest disease detection and agricultural fields. Details of our dataset and code are available at https://github.com/dydgns2017/PWD-Synthetic-Dataset. • Pine Wilt Disease requires early detection due to its severity and lack of cure. • Our synthetic dataset creation outperforms traditional PWD data collection process. • Real and synthetic data combination improves PWD F1 Score to 92.88%. • Synthetic data method aids forest preservation, applies to other agriculture. [ABSTRACT FROM AUTHOR]

Planktonic organisms play a pivotal role within aquatic ecosystems, serving as the foundation of the aquatic food chain while also playing a critical role in climate regulation and the production of oxygen. In recent years, the advent of automated systems for capturing in-situ images has led to a huge influx of plankton images, making manual classification impractical. This, at the same time, has opened up opportunities for the application of machine learning and deep learning solutions. This paper undertakes an extensive analysis of the broad range of computer vision techniques and methodologies that have emerged to facilitate the automatic analysis of small- to large-scale datasets containing plankton images. By focusing on different computer vision tasks, we present findings and limitations in order to offer a comprehensive overview of the current state-of-the-art, while also pinpointing the open challenges that demand further research and attention. • Provides a common level of knowledge in computer vision for plankton image analysis. • Describes related state-of-the-art methods and open-source benchmark datasets. • Provides a comprehensive overview of milestones, limitations, and open challenges. • Analyzes works in plankton image analysis from a modern computer vision viewpoint. [ABSTRACT FROM AUTHOR]

• Double random phase encoding (DRPE) was evaluated on complex datasets obtained from road surveillance cameras in the real world. • Previous works of DRPE cryptanalysis based on deep learning by using only simple datasets were additionally investigated on complex datasets. • The feasibility of DRPE to classification tasks was evaluated on complex datasets by using adaptive transfer learning models. • The proposed DRPE image classification scheme showed the highest classification accuracy compared to state-of-the-art classification schemes, even in a data-limited case. • We identified that recovering DRPE images were challenging, especially in complex datasets, but the imperceptible features representing the class of the images were contained in the DRPE images. Encrypting data can convert original data into a form that cannot be recognized and conceal personal information. To ensure data security, analyzing cryptographic algorithms used in data encryption is critical. The previous works evaluated the robustness of the optical cryptographic algorithm, Double Random Phase Encoding (DRPE), and verified the vulnerability of DRPE to plaintext recovery attacks based on deep learning. Although they could prove that DRPE is not secure by recovering the DRPE images into the original images, they evaluated DRPE only on simple datasets, which were binary or grayscale images with a single object. Thus, in this paper, we investigated the previous works of plaintext recovery on DRPE by using complex datasets. To our knowledge, it is the first study to evaluate DRPE on complex datasets. In addition to the plaintext recovery, we classified DRPE images of complex datasets and verified the feasible points in DRPE for classification tasks based on deep learning. Adaptive transfer learning, the modified version of the classical transfer learning for DRPE image classification, was proposed and utilized to train the DRPE image classification models. Experimental results showed that recovering the DRPE images of the complex datasets was challenging. In contrast, the imperceptible features representing the class of the images were in DRPE images regardless of whether the dataset was simple or complex. The classification performance of the proposed DRPE image classification scheme was better than those of state-of-the-art classification schemes, even with a small dataset. Furthermore, data augmentation with the proposed scheme improved the classification accuracy in DRPE image classification on a small dataset. In addition, our DRPE image classification schemes were applied to encrypted images by other optical cryptographic algorithms and the classification results were compared with that of the encrypted images by classical DRPE. [ABSTRACT FROM AUTHOR]

Activation functions play an important role in deep learning models by introducing non-linearity to the output of a neuron, enabling the network to learn complex patterns and non-linear relationships in data and make predictions on more complex tasks. Deep learning models' most commonly used activation functions are Purelin, Sigmoid, Tansig, Rectified Linear Unit (ReLU), and Exponential Linear Unit (ELU), which exhibit limitations such as non-differentiability, vanishing gradients, and neuron inactivity with negative values. These functions are typically defined over a finite range, and their outputs are integers or real numbers. Using fractional calculus in designing activation functions for neural networks has shown promise in improving the performance of deep learning models in specific applications. These activation functions can capture more complex non-linearities than traditional integer-order activation functions, improving performance on tasks such as image classification and time series prediction. This paper focuses on deriving and testing linear and non-linear fractional-order forms of activation functions and their variants. The linear activation function includes Purelin. In contrast, the non-linear activation functions are Binary Step, Sigmoid, Tansig, ReLU, ELU, Gaussian Error Linear Unit (GELU), Hexpo, and their variants. Besides, the standard formula has been implemented and used in developing the fractional-order linear activation function. Furthermore, various expansion series, such as Euler and Maclaurin, have been used to design non-linear fractional-order activation functions and their variants. The single- and multi-layer fractional-order neural network models have been developed using the designed fractional-order activation functions. The simulation study uses developed fractional-order neural network models for predicting the Texas wind turbine systems' generated power. The performance of single and multi-layer fractional-order neural network models has been evaluated by changing the activation functions in the hidden layer while keeping the Purelin function constant at the output layer. Experiments on neural network models demonstrate that the designed fractional-order activation functions outperform traditional functions like Sigmoid, Tansig, ReLU, ELU, and their variants, effectively addressing limitations. • Activation functions are derived into fractional functions using fractional calculus. • The fractional neural network models have been made by adopting derived functions. • A Case study has been done on the Texas wind turbine with the models to predict the power. • The models' performance has been evaluated using the fractional functions. [ABSTRACT FROM AUTHOR]

• A pretrained model on augmented data and fine-tuned by using AI and 3D contour features for assessing facial attractiveness is proposed. • The proposed model overcomes the subjective inconsistency and unreliability common to all traditional rating methods. • This model provides an accurate 3D information of full facial that is unavailable in previous studies using either 2D or 3D measurement. • 3D facial images and deep transfer learning have been firstly combined for evaluating the facial attractiveness in patients undergoing OGS. • The developed web browser–based user interface contributes to effective doctor–patient communication and decision-making. In this paper, we investigate a new approach based on a combination of three-dimensional (3D) facial images and deep transfer learning (TL) with fine-grained image classification (FGIC) for quantitative evaluation of facial attractiveness. The 3D facial surface images of patients with and without filtering and the publicly available SCUT-FBP5500 dataset was used for transfer training and model pre-training, respectively. Experimental results show that a bilinear CNN model with a Gaussian filter freezing 80 % of the weights exhibit the strongest performance and lowest average error as a deep learning prediction model; the model was subsequently adopted for automatic assessment of facial attractiveness in clinical application. This is the first TL model with FGIC using 3D facial images for automatic quantitative evaluation of facial attractiveness in patients undergoing Orthognathic surgery (OGS). The developed web browser–based user interface enables effective and rapid assessment, thus contributing to effective patient–clinician communication and decision-making. [ABSTRACT FROM AUTHOR]

This paper discusses how automatic speech recognition systems are and could be designed, in order to best exploit the discriminative information encoded in human speech. This contrasts with many recent machine learning approaches that apply general recognition architectures to signals to identify, with little concern for the nature of the input. The implicit assumption has often been that training can automatically discover the useful properties that exist in signals, with minimal manual intervention. These approaches may be suitable for some tasks such as image recognition, where the diversity of visual input is vast; e.g., an image may be any (natural or synthetic) scene that a camera views. We first examine what makes speech special, i.e., a natural signal from a complex tube, driven by a source that is quasi-periodic and/or noisy, aiming to communicate a wide variety of information, using the different vocal systems of human speakers. Then, we view how pertinent features are extracted from speech via efficient means, related to the objectives of communication. We see how to reliably and efficiently identify the different units of oral language. We learn from the history of attempts to do ASR, e.g., why they succeeded and how improved methods exploited the increasing availability of data and computer power (in particular, deep neural networks). Finally, we suggest ways to render ASR both more accurate and efficient. This work is aimed at both newcomers to ASR and experts, in terms of presenting issues broadly, but without mathematical or algorithmic details, which are readily found in the references. [ABSTRACT FROM AUTHOR]

Monitoring and maintenance of photovoltaic (PV) systems are critical in order to ensure continuous power generation and prevent operation drops. Manual inspection of high-resolution Electroluminescence (EL) images of PV modules requires human effort and time. Some research rely on manually created features, which cannot ensure the classification stage's effectiveness. Contrarily, Deep learning models have been widely used for fast and accurate image classification. However, these standard deep learning models could produce errors, especially in the presence of noisy or inter-class small variation data which is the case with PV images. In this paper, we introduce an end-to-end deep learning model that combines handcrafted and automatic feature extraction to produce better PV image classification accuracy. Using a deep neural network and histogram of oriented gradient (HoG) of PV images, this work makes a significant contribution by directly learning a hybrid model that refines the leveraged feature vector. Our experimental results show better performance compared with six state-of-the-art methods that use the same or different baseline deep learning model. • A hybrid model with end-to-end training that leverages hybrid feature extraction. • The hybrid model outperforms existing cutting-edge methods on PV images. • The classification model helps to identify defects in EL images. [ABSTRACT FROM AUTHOR]

Deep learning architectures have demonstrated outstanding results in image classification in the last few years. However, applying sophisticated neural network architectures in small datasets remains challenging. In this context, transfer learning is a promising approach for dealing with this scenario. Generally, the available pre-trained architectures adopt a standard fixed input, which usually implies resizing and cropping the input images in the preprocessing phase, causing information loss. Besides, images present visual features in different scales in real-world scenarios, and most common approaches do not consider this fact. In this paper, we propose an approach that applies transfer learning for dealing with small datasets and leverages visual features extracted by pre-trained models from different scales. We based our approach on graph convolutional networks (GCN) that take graphs representing the images in different scales as input and whose nodes are characterized by features extracted by pre-trained models from regular image patches of different scales. Since GCN can deal with graphs with different numbers of nodes, our approach can deal naturally with images of heterogeneous sizes without discarding relevant information. We evaluated our approach in two datasets: a set of geological images and a publicly available dataset, both presenting characteristics that challenge traditional approaches. We tested our approach by adopting three different pre-trained models as feature extractors: two efficient pre-trained CNN models (DenseNet and ResNeXt) and one Vision Transformer (CLIP). We compared our approach with two conventional approaches for dealing with image classification. The experiments show that our approach achieves better results than the conventional approaches for this task. • We propose multiscale patch-based graphs to represent image information. • Transfer learning is adopted for dealing with small datasets. • The approach can handle images with heterogeneous sizes. • The approach represents visual information at multiple scales. • The approach is base on convolutional graph neural networks. [ABSTRACT FROM AUTHOR]

Ultrasound has become an indispensable clinical scanning and diagnostic tool because of its convenience, rapidity, and radiation-free properties. In ultrasound imaging, the resolution and detection depth of ultrasound images are closely related to the frequency of the transmitted ultrasound. Ultrasound with low transmission frequency has a wide detection range, but its images have problems such as low resolution, blurred edge contours and obvious background noise. In contrast, the higher transmission frequency, the smaller the detection range, but the resolution has improved. The rapid development of deep learning provides new approaches and enormous potential for improving resolution of ultrasound images and acquiring high-resolution images with an extensive detection range. In this paper, the attention mechanism is combined into a convolutional neural network based on the encoder-decoder, and the feature maps extracted at different scales are merged by skip connections to preserve more structure and contrast details, thus realizing the improvement of lower-frequency ultrasound image resolution. This work utilizes the capacitive micromechanical ultrasound transducer (CMUT) probe for dataset acquisition. The network's performance is validated by simulated datasets for point targets and experimental datasets for breast body models. The experimental results show that for ultrasound images, the network output map can improve by 0.2 in structural similarity and at least 11 dB in peak signal-to-noise ratio. The proposed network can effectively preserve ultrasound images' structural and textural information, improve edge contour definition, and significantly suppress noise and imaging artifacts, which has significant development potential in ultrasound imaging. [Display omitted] • Different ultrasound emission frequencies have significant impact on imaging quality. • Datasets of different frequencies using capacitive micromechanical ultrasound transducer (CMUT) probes. • Using deep learning to improve the quality of low-frequency images by extracting features from high-frequency images. • With incorporation of the attention mechanism, the network highlights the prominent features through skip connections while maintaining computational efficiency. [ABSTRACT FROM AUTHOR]

Hypercomplex convolutional neural networks (HCNNs) have recently been used to improve deep learning architectures due to their ability to share weights across input channels and thus improve the cohesiveness of learned representations within the layers. Much work has been done to study the effect of hypercomplex representation in CNNs. However, to date, none of these models have used fully hypercomplex architectures, meaning that some of the layers in these earlier networks used conventional real-valued calculations that did not engage cross-channel weight sharing. Here, we introduce and study full hypercomplex CNNs by ensuring all layers perform hypercomplex calculations. For the earlier HCNNs, the real-valued layers are found in: (I) in the front end of the networks, (II) in the back end of the networks, and (III) in the residual blocks of the networks. The present research examines the performance of HCNNs when the modules mentioned above are replaced with hypercomplex equivalents. These representational networks have outperformed and shown state-of-the-art results compared to previous hypercomplex models (on some specific datasets). A disadvantage of these representational networks is that they consume high computational costs. To reduce this cost, novel separable hypercomplex networks (SHNNs) are proposed. They are created by factoring a quaternion convolutional module into two consecutive separable vectormap convolutional modules. As the successive layers of deep HCNNs perform local hierarchical grouping on increasingly abstract features, these groupings are responsible for the long-distance interaction problems found in the dense layers. To handle these problems, researchers have applied attention mechanisms in hypercomplex space. This paper offers a perspective on the basic concepts of representational networks and attention-based representational networks with the focus on (1) reviewing deep full HCNNs, including hypercomplex-based fully connected dense layer; (2) analyzing hypercomplex-based separable concept; (3) describing how the hypercomplex spatial layer of residual bottleneck block is replaced with an attention layer; and (4) providing a comprehensive summary comparing the performance of recent deep HCNNs, deep convolutional neural networks, and original attention based networks. We compare the performance of the networks in terms of accuracy and the number of trainable parameters on several image classification datasets. • Hypercomplex CNNs with and without attention introduce cross-channel weight sharing. • Cross-channel weight sharing enables discovery of a new class of input correlations. • Hypercomplex networks can be further optimized without loss of accuracy. • Toward this goal, separable hypercomplex operations are introduced. [ABSTRACT FROM AUTHOR]