Search

Acquiring incomplete k-space matrices is an effective way to accelerate Magnetic Resonance Imaging (MRI). It is an important and challenging task to accurately reconstruct images from such under-sampled k-space matrices. On the one hand, neither image-domain oriented nor frequency-domain oriented deep Convolutional Neural Networks can simultaneously employ both frequency features and spatial features for cooperatively improving reconstruction accuracy. On the other hand, existing dual-domain reconstruction methods adopt heavy encoder-decoder frameworks, resulting in low efficiency and information loss in the process of pooling. To deal with these problems, in this paper, we propose a full-resolution dual-domain reconstruction network, called DIIK-Net. The DIIK-Net consists of a full-resolution frequency-domain branch, a full-resolution image-domain branch, and cross-domain interaction modules between the two branches. The first novelty of the proposed method is that the features of each block of frequency-domain branch are extracted by 1 × 1 filters, which reduces computational cost and captures rich contextual information. Due to the fact that an element in frequency domain conveys information of the whole image, 1 × 1 convolutional blocks are able to extract large contextual information with the interaction of image domain. The second novelty is that the image-domain branch consists of a very small number of 3 × 3 convolutional blocks and each block has very large field of perception due to integration of frequency domain. The third novelty lies in the simple and effective cross-domain interaction module. Experimental results on the challenging fastMRI dataset demonstrate that the proposed method is capable of achieving higher reconstruction accuracy with a few number of parameters. [ABSTRACT FROM AUTHOR]

An accurate, frequently updated, automatic and reproducible mapping procedure to identify seasonal cultivated crops is a prerequisite for many crop monitoring activities. Deep learning was demonstrated to be an effective mapping approach already successfully applied to decametric resolution satellite images (like Sentinel-2 data) to produce yearly crop maps. In this framework, algorithm training is performed with ground truth typically consisting of spatially explicit information available after the end of the season (e.g. yearly crop maps and/or farmer declaration for subsidies at parcel level); however, such data (i) does not allow performing in-season prediction, and (ii) does not provide temporal details fundamental to describe a dynamic crop succession and/or to understand crop management (i.e. planting and harvesting). In this paper we present a Deep Neural Network-based approach capable of generating (i) a crop map of the current season at a specific point in time ("In season mapping" conventionally at the end of the current year), along with (ii) all intermediate maps during the season able to describe in near real-time the evolution of crop presence ("Dynamic-mapping" at the temporal granularity of satellite imagery revisiting, e.g., 5 days for Sentinel-2 data). This approach adopts a smart training procedure of a Deep Neural model by exploiting historical satellite data and ground truth. We introduce a method to automatically generate "short-term" ground truth maps (i.e. 5 days reference) starting from the "long-term" ones (i.e. available yearly static reference) and characterizing temporally the different crop presence by performing a phenological analysis of historical time series. The model was trained and validated in Lombardy (North of Italy) exploiting multi-annual authoritative crop maps from 2016 to 2019. Validation was performed both in time (same areas used for training in a different year) and space (different location) for the year 2019. The quantitative error metrics calculation and Spatio-temporal analysis clearly demonstrate that the model can predict in-season crop presence with a generalization capacity over the long-term (yearly maps: OA > 70% and Kappa > 0.64%) and that the short-term predictions (5 days maps) are coherent with the reference information from expert knowledge (local crop calendars). The model can produce dynamically along the season short-term maps with a medium-high crop-specific User Accuracy at the maximum green-up phase (UA > 53% up to 95%). These products are of extreme interest for final users providing information at the peak of plant development that dynamically changes according to the considered crop, the specific location and the investigated season. These results demonstrate that it is possible to produce a crop map early in the season and extract useful additional information such as crop intensity (e.g. double crops presence) and crop dynamics related to different sowing dates. [ABSTRACT FROM AUTHOR]

• Thispaperfirstlyproposed a supervised-unsupervised combinednetwork for high-fidelitypansharpening. • Thispaperproposed a supervised networkby multiscale mechanisms, dilated convolution and skip connection. • This paperproposeda universalunsupervisedspatial-spectral joint compensationnetwork. Deep learning for pansharpening method has become a hot research topic in recent years due to the impressive performance, and the convolutional neural networks (CNN)-based pansharpening methods on Wald's protocol (i.e., the general adoption of the network learned at a coarser reduced resolution scale to the finer full resolution) have been dominating for a long time in this research area. However, the scale-invariant assumption may not be accurate enough to make full use of the spatial and spectral information of original panchromatic (PAN) and multispectral (MS) images at full resolution. In this paper, a Supervised-Unsupervised combined Fusion Network (SUFNet) for high-fidelity pansharpening is proposed to alleviate this problem. First, by comprehensively considering the robustness of the network with reference label images, a novel supervised network based on Wald's protocol is proposed by integrating the multiscale mechanisms, dilated convolution, and skip connection, termed SMDSNet. Then, an interesting Unsupervised Spatial-Spectral Compensation Network (USSCNet) without real high-spatial-resolution (HR) MS label image is proposed to enhance the spatial and spectral fidelity of the SMDSNet. The qualitative and quantitative results in reduced resolution and full resolution experiments on different satellite datasets demonstrate the competitive performance of the proposed method. Furthermore, the proposed USSCNet can be employed as a universal spatial-spectral compensation framework for other pansharpening methods. [ABSTRACT FROM AUTHOR]

Distracted driving is one of the key factors that cause drivers to ignore potential road hazards and then lead to accidents. Existing efforts in distracted behavior recognition are mainly based on deep learning (DL) methods, which identifies distracted behaviors by analyzing static characteristics of images. However, the convolutional neural network (CNN) — based DL methods lack the causal reasoning ability for behavior patterns. The uncertainty of driving behaviors, noise of the collected data, and occlusion between body agents, bring additional challenges to existing DL methods to recognize distracted behaviors continuously and accurately. Therefore, in this paper, we propose a distracted behavior recognition method based on the Temporal–Spatial double-line DL network (TSD-DLN) and causal And-or graph (C-AOG). TSD-DLN fuses the attention feature extracted from the dynamic optical flow information and the spatial feature of the single video frame to recognize the distracted driving posture. Furthermore, a causal knowledge fence based on C-AOG is fused with TSD-DLN to improve the recognition robustness. The C-AOG represents the causality of behavior state fluent change and adopts counterfactual reasoning to suppress behavior recognition failures caused by frame features distortion or occlusion between body agents. We compared the performance of the proposed method with other state-of-the-art (SOTA) DL methods on two public datasets and self-collected dataset. Experimental results demonstrate that proposed method significantly outperforms other SOTA methods when acquiring distracted driving behavior by processing consecutive frames. In addition, the proposed method exhibits accurate continuous recognition and robustness under incomplete observation scenarios. • A novel behavior recognition model based on multi-features fusion is proposed. • A hierarchical causality graph is designed for behavior fluent inferring. • Counterfactual reasoning mechanism for misrecognition suppression is proposed. • A new behavior dataset representing the distracted driving process is constructed. [ABSTRACT FROM AUTHOR]

Accurate and automatic segmentation of pancreatic tumors and organs from medical images is important for clinical diagnoses and making treatment plans for patients with pancreatic cancer. Although deep learning methods have been widely adopted for this task, the segmentation accuracy, especially for pancreatic tumors, still needs to be further improved because (1) phenotypic differences, such as volumes, tend to make the models focus on pancreatic learning, resulting in insufficient tumor feature selection; (2) deep learning models may fall into local optima, leading to unsatisfactory segmentation results for tumors and pancreas. To alleviate the above issues, in this paper, we propose a 3D fully convolutional neural network with three temperature guided modules, namely, balance temperature loss, rigid temperature optimizer and soft temperature indictor, to realize joint segmentation of the pancreas and tumors. Specifically, balance temperature loss is designed to dynamically adjust the learning points between tumors and the pancreas to balance the selected features, and it is aimed at improving the accuracy of tumor segmentation without losing pancreas information. Rigid temperature optimizer is proposed to accept nonimproving moves probabilistically to adaptively avoid local optima. To further refine the segmentation results, we propose the soft temperature indictor to guide the network into a fine-tuning state automatically when the model tends to stability. Our experimental results are more accurate than the fourteen top-ranking methods in pancreas and tumors segmentation on the MSD pancreas dataset and six top-ranking methods in brain tumors segmentation. Ablation studies verify the effectiveness of the three temperature guided modules. [ABSTRACT FROM AUTHOR]

Residual Network (ResNet) achieves deeper and wider networks with high-performance gains, representing a powerful convolutional neural network architecture. In this paper, we propose architectural refinements to ResNet that address the information flow through several layers of the network, including the input stem, downsampling block, projection shortcut, and identity blocks. We will show that our collective refinements facilitate stable backpropagation by preserving the norm of the error gradient within the residual blocks, which can reduce the optimization difficulties of training very deep networks. Our proposed modifications enhance the learning dynamics, resulting in high accuracy and inference performance by enforcing norm-preservation throughout the network training. The effectiveness of our method is verified by extensive experimental results on five computer vision tasks, including image classification (ImageNet and CIFAR-100), video classification (Kinetics-400), multi-label image recognition (MS-COCO), object detection and semantic segmentation (PASCAL VOC). We also empirically show consistent improvements in generalization performance when applying our modifications over different networks to provide new insights and inspire new architectures. The source code is publicly available at: https://github.com/bharatmahaur/LeNo. [ABSTRACT FROM AUTHOR]

Inspired by the successful use of deep learning in computer vision, in this paper we introduce ForCNN , a novel deep learning method for univariate time series forecasting that mixes convolutional and dense layers in a single neural network. Instead of using conventional, numeric representations of time series data as input to the network, the proposed method considers visual representations of it in the form of images to directly produce point forecasts. Three variants of deep convolutional neural networks are examined to process the images, the first based on VGG-19, the second on ResNet-50, while the third on a self-designed architecture. The performance of the proposed approach is evaluated using time series of the M3 and M4 forecasting competitions. Our results suggest that image-based time series forecasting methods can outperform both standard and state-of-the-art forecasting models. • An image-based deep learning method for time series forecasting is proposed. • Numeric time series data are converted to images using simple line plots. • Deep convolutional neural networks are employed for producing forecasts from each image. • The proposed, image-based, method outperforms state-of-the-art forecasting methods. [ABSTRACT FROM AUTHOR]

Gene Expression Data is the biological data to extract meaningful hidden information from the gene dataset. This gene information is used for disease diagnosis especially in cancer treatment based on the variations in gene expression levels. DNA microarray is an efficient method for gene expression classification and prediction of cancer disease for specific types of cancer. Due to the abundance of computing power, deep learning (DL) has become a widespread technique in the healthcare sector. The gene expression dataset has a limited number of samples but a large number of features. Data augmentation is needed for gene expression datasets to overcome the dimensionality problem in gene data. It is a technique to generating the synthetic samples to increase the diversity of data. Deep learning methods are designed to learn and extract the features that come from the raw input data in the form of multidimensional arrays. This paper reviews the existing research in deep learning techniques like Feed Forward Neural Network (FFN), Convolutional Neural Network (CNN), Autoencoder (AE) and Recurrent Neural Network (RNN) for the classification and prediction of cancer disease and its types through gene expression data analysis. [ABSTRACT FROM AUTHOR]

• Deep learning architecture for neuroimaging reconstruction. • Efficient convolutional neural networks based on separable convolutions. • Multimodality learning and interpretability of latent embeddings. • Clustering individuals phenotypes with structural multimodal brain features. Recently, it has become progressively more evident that classic diagnostic labels are unable to accurately and reliably describe the complexity and variability of several clinical phenotypes. This is particularly true for a broad range of neuropsychiatric illnesses such as depression and anxiety disorders or behavioural phenotypes such as aggression and antisocial personality. Patient heterogeneity can be better described and conceptualized by grouping individuals into novel categories, which are based on empirically-derived sections of intersecting continua that span both across and beyond traditional categorical borders. In this context, neuroimaging data (i.e. the set of images which result from functional/metabolic (e.g. functional magnetic resonance imaging, functional near-infrared spectroscopy, or positron emission tomography) and structural (e.g. computed tomography, T1-, T2- PD- or diffusion weighted magnetic resonance imaging) carry a wealth of spatiotemporally resolved information about each patient's brain. However, they are usually heavily collapsed a priori through procedures which are not learned as part of model training, and consequently not optimized for the downstream prediction task. This is due to the fact that every individual participant usually comes with multiple whole-brain 3D imaging modalities often accompanied by a deep genotypic and phenotypic characterization, hence posing formidable computational challenges. In this paper we design and validate a deep learning architecture based on generative models rooted in a modular approach and separable convolutional blocks (which result in a 20-fold decrease in parameter utilization) in order to a) fuse multiple 3D neuroimaging modalities on a voxel-wise level, b) efficiently convert them into informative latent embeddings through heavy dimensionality reduction, c) maintain good generalizability and minimal information loss. As proof of concept, we test our architecture on the well characterized Human Connectome Project database (n = 974 healthy subjects), demonstrating that our latent embeddings can be clustered into easily separable subject strata which, in turn, map to different phenotypical information (including organic, neuropsychological, personality variables) which was not included in the embedding creation process. The ability to extract meaningful and separable phenotypic information from brain images alone can aid in creating multi-dimensional biomarkers able to chart spatio-temporal trajectories which may correspond to different pathophysiological mechanisms unidentifiable to traditional data analysis approaches. In turn, this may be of aid in predicting disease evolution as well as drug response, hence supporting mechanistic disease understanding and also empowering clinical trials. [Display omitted] [ABSTRACT FROM AUTHOR]

Accurate relative pose estimation of a spacecraft during space landing operation is critical to ensure a safe and successful landing. This paper presents a 3D Light Detection and Ranging (LiDAR) based AI relative navigation architecture solution for autonomous space landing. The proposed architecture is based on a hybrid Deep Recurrent Convolutional Neural Network (DRCNN) combining a Convolutional Neural Network (CNN) with an Recurrent Neural Network (RNN) based on a Long–Short Term Memory (LSTM) network. The acquired 3D LiDAR data is converted into a multi-projected images and feed the DRCNN with depth and other multi-projected imagery. The CNN module of the architecture allows an efficient representation of features, and the RNN module, as an LSTM, provides robust navigation motion estimates. A variety of landing scenarios are considered, simulated and experimented to evaluate the efficiency of the proposed architecture. A LiDAR based imagery data (Range, Slope, and Elevation) is initially created using PANGU (Planet and Asteroid Natural Scene Generation Utility) software and an evaluation of the proposed solution using this data is conducted. Tests using an instrumented Aerial Robot in Gazebo software to simulate landing scenarios on a synthetic but representative lunar terrain (3D digital elevation model) is proposed. Finally, real experiments using a real flying drone equipped with a Velodyne VLP16 3D LiDAR sensor to generate real 3D scene point clouds while landing on a designed down scaled lunar moon landing surface are conducted. All the test results achieved show that the suggested architecture is capable of delivering good 6 Degree of Freedom (DoF) pose precision with a good and reasonable computation. • Effective feature representation due to the adoption of the CNN module. • The RNN (LSTM) module provides a robust and efficient modelling of the navigation kinematics. • The hybrid network is flexible to train and test on different data modalities. [ABSTRACT FROM AUTHOR]

To develop an efficient brain–computer interface (BCI) system, electroencephalography (EEG) measures neuronal activities in different brain regions through electrodes. Many EEG-based motor imagery (MI) studies do not make full use of brain network topology. In this paper, a deep learning framework based on a modified graph convolution neural network (M-GCN) is proposed, in which temporal-frequency processing is performed on the data through modified S-transform (MST) to improve the decoding performance of original EEG signals in different types of MI recognition. MST can be matched with the spatial position relationship of the electrodes. This method fusions multiple features in the temporal-frequency-spatial domain to further improve the recognition performance. By detecting the brain function characteristics of each specific rhythm, EEG generated by imaginary movement can be effectively analyzed to obtain the subjects' intention. Finally, the EEG signals of patients with spinal cord injury (SCI) are used to establish a correlation matrix containing EEG channel information, the M-GCN is employed to decode relation features. The proposed M-GCN framework has better performance than other existing methods. The accuracy of classifying and identifying MI tasks through the M-GCN method can reach 87.456%. After 10-fold cross-validation, the average accuracy rate is 87.442%, which verifies the reliability and stability of the proposed algorithm. Furthermore, the method provides effective rehabilitation training for patients with SCI to partially restore motor function. [ABSTRACT FROM AUTHOR]

Convolutional Neural Network is one of the famous members of the deep learning family of neural network architectures, which is used for many purposes, including image classification. In spite of the wide adoption, such networks are known to be highly tuned to the training data (samples representing a particular problem), and they are poorly reusable to address new problems. One way to change this would be, in addition to trainable weights, to apply trainable parameters of the mathematical functions, which simulate various neural computations within such networks. In this way, we may distinguish between the narrowly focused task-specific parameters (weights) and more generic capability-specific parameters. In this paper, we suggest a couple of flexible mathematical functions (Generalized Lehmer Mean and Generalized Power Mean) with trainable parameters to replace some fixed operations (such as ordinary arithmetic mean or simple weighted aggregation), which are traditionally used within various components of a convolutional neural network architecture. We named the overall architecture with such an update as a hyper-flexible convolutional neural network. We provide mathematical justification of various components of such architecture and experimentally show that it performs better than the traditional one, including better robustness regarding the adversarial perturbations of testing data. [ABSTRACT FROM AUTHOR]

Deep learning is a powerful tool for feature representation, and many methods based on convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been applied on fault diagnoses for chemical processes. However, unlike attention mechanisms, these networks are inefficient when extracting features of long-term dependencies. The transformer method employs a self-attention mechanism and sequence-to-sequence model originally designed for natural language processing (NLP). This approach has attracted significant attention in recent years due to its great success in NLP fields. The fault diagnosis of a chemical process is a task based on multi-variable time series, which are similar to text sequences with a greater focus on long-term dependencies. This paper proposes a modified transformer model called Target Transformer, which includes not only a self-attention mechanism, but also a target-attention mechanism for chemical process fault diagnoses. The Tennessee Eastman (TE) process was used to evaluate our method's performance. [ABSTRACT FROM AUTHOR]

Deep Learning (DL) has been recently employed to build smart systems that perform incredibly well in a wide range of tasks, such as image recognition, machine translation, and self-driving cars. In several fields the considerable improvement in the computing hardware and the increasing need for big data analytics has boosted DL work. In recent years physiological signal processing has strongly benefited from deep learning. In general, there is an exponential increase in the number of studies concerning the processing of electromyographic (EMG) signals using DL methods. This phenomenon is mostly explained by the current limitation of myoelectric controlled prostheses as well as the recent release of large EMG recording datasets, e.g. Ninapro. Such a growing trend has inspired us to seek and review recent papers focusing on processing EMG signals using DL methods. Referring to the Scopus database, a systematic literature search of papers published between January 2014 and March 2019 was carried out, and sixty-five papers were chosen for review after a full text analysis. The bibliometric research revealed that the reviewed papers can be grouped in four main categories according to the final application of the EMG signal analysis: Hand Gesture Classification, Speech and Emotion Classification, Sleep Stage Classification and Other Applications. The review process also confirmed the increasing trend in terms of published papers, the number of papers published in 2018 is indeed four times the amount of papers published the year before. As expected, most of the analyzed papers (≈ 60 %) concern the identification of hand gestures, thus supporting our hypothesis. Finally, it is worth reporting that the convolutional neural network (CNN) is the most used topology among the several involved DL architectures, in fact, the sixty percent approximately of the reviewed articles consider a CNN. [ABSTRACT FROM AUTHOR]

• In this paper we propose a new Convolutional Neural Network operating on graph space. • We define new convolution and pooling operators applied directly to graphs. • We prove that our operators can be used with backpropagation techniques. • We prove experimentally that the architecture reaches the state of the art in a classification context. [Display omitted] Convolutional neural networks (CNNs), in a few decades, have outperformed the existing state of the art methods in classification context. However, in the way they were formalised, CNNs are bound to operate on euclidean spaces. Indeed, convolution is a signal operation that are defined on euclidean spaces. This has restricted deep learning main use to euclidean-defined data such as sound or image. And yet, numerous computer application fields (among which network analysis, computational social science, chemo-informatics or computer graphics) induce non-euclideanly defined data such as graphs, networks or manifolds. In this paper we propose a new convolution neural network architecture, defined directly into graph space. Convolution and pooling operators are defined in graph domain thanks to a graph matching procedure between the input signal and a filter. We show its usability in a back-propagation context. Experimental results show that our model performance is at state of the art level on simple tasks. It shows robustness with respect to graph domain changes and improvement with respect to other euclidean and non-euclidean convolutional architectures. [ABSTRACT FROM AUTHOR]

• This paper proposed a novel two-stage spatiotemporal features learning method based on a Siamese 3D CNN for MEs recognition. • The focal loss was introduced to address the problem of inefficient model training caused by the class imbalance in the MEs datasets. • An adaptive construction method based on adaptive CNN was proposed to construct the key-frames sequence to summarize the original MEs. Micro-expressions (MEs) are spontaneous and involuntary facial subtle reactions which often reveal the genuine emotions within human beings. Recognizing MEs automatically is becoming increasingly crucial for many areas such as diagnosis and security. However, the short time duration and low spatial intensity of MEs pose great challenges for accurately recognizing them. Additionally, the lack of sufficient and balanced spontaneous MEs data also makes this problem even harder to solve, and some adaptive modeling strategies have been quite urgent recently. To this end, this paper draws inspirations from few-shot learning to propose a novel two-stage learning (i.e., prior learning and target learning) method based on a siamese 3D convolutional neural network for MEs recognition (MERSiamC3D). Specifically, in the prior learning stage, the proposed MERSiamC3D is used to extract the generic features of MEs. In the target learning stage, the structure and parameters of the MERSiamC3D will be carefully adjusted and the Focal Loss is adopted for high-level features learning. Afterwards, in order to effectively retain the spatiotemporal information of the original MEs video, an adaptive construction method based on adaptive convolutional neural network is proposed to construct the key-frames sequence to summarize the original MEs video, which is able to help drop the redundant frames and relatively highlight the movement of the apex frame. Then, the new key-frames are taken as the input of the two-stage learning method. Finally, through extensive evaluations and experiments on three publically available MEs datasets, the proposed method in this work could outperform traditional methods and other deep learning baselines, which provides a novel insight on how to leverage scarce data for MEs recognition. [ABSTRACT FROM AUTHOR]

[Display omitted] • Proposal of a large-scale dataset for active fire detection in Landsat-8 imagery. • Deep learning can approximate and improve the results from handcrafted algorithms. • Deep learning networks can approximate combinations of handcrafted algorithms. Active fire detection in satellite imagery is of critical importance to the management of environmental conservation policies, supporting decision-making and law enforcement. This is a well established field, with many techniques being proposed over the years, usually based on pixel or region-level comparisons involving sensor-specific thresholds and neighborhood statistics. In this paper, we address the problem of active fire detection using deep learning techniques. In recent years, deep learning techniques have been enjoying an enormous success in many fields, but their use for active fire detection is relatively new, with open questions and demand for datasets and architectures for evaluation. This paper addresses these issues by introducing a new large-scale dataset for active fire detection, with over 150,000 image patches (more than 200 GB of data) extracted from Landsat-8 images captured around the world in August and September 2020, containing wildfires in several locations. The dataset was split in two parts, and contains 10-band spectral images with associated outputs, produced by three well known handcrafted algorithms for active fire detection in the first part, and manually annotated masks in the second part. We also present a study on how different convolutional neural network architectures can be used to approximate these handcrafted algorithms, and how models trained on automatically segmented patches can be combined to achieve better performance than the original algorithms – with the best combination having 87.2% precision and 92.4% recall on our manually annotated dataset. The proposed dataset, source codes and trained models are available on Github (https://github.com/pereira-gha/activefire), creating opportunities for further advances in the field. [ABSTRACT FROM AUTHOR]

• A crack data set with various forms is constructed by manual collection. • Proposes a network model suitable for multi shape object segmentation. • Proposes a kind of boundary weighted loss function which is suitable for the segmentation network of small objects. • Our model has better detection effect than the existing crack segmentation network in this scene. The existence of cracks will affect the stability of the building. It is very important to identify and deal with the cracks in time to ensure the safety and stability of the building. Based on the above background, the purpose of this paper is to study the method of building crack recognition and total quality management based on deep learning. This paper focuses on the computer vision technology in artificial intelligence, studies the image classification algorithm and semantic segmentation algorithm based on the deep learning method, and applies it to the field of building crack image analysis. In this paper, we use the deep convolution neural network to design the building image crack classification model and segmentation model, realize the identification and analysis of building cracks, and build a building crack analysis system, which can significantly improve the efficiency of building crack detection. Then, based on the image processing technology, the quantitative analysis of the fracture segmentation results is carried out. Through the basic morphological methods such as corrosion, expansion, opening and closing operations, the segmentation mark map, skeleton map and geometric parameter information of the fracture are obtained, which further provides the maintenance and judgment basis for professional engineers. The experimental results show that compared with FCN, the accuracy of rfcn-a is improved by 5.98%, the precision is improved by 6.07%, and the real and f'score are improved by 3.11% and 6.01%, respectively. [ABSTRACT FROM AUTHOR]

Urban ride-hailing demand prediction is a crucial but challenging task for intelligent transportation system construction. Predictable ride-hailing demand can facilitate more reasonable vehicle scheduling and online car-hailing platform dispatch. Conventional deep learning methods with no external structured data can be accomplished via hybrid models of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) by meshing plentiful pixel-level labeled data, but data sparsity of high grid granularity in spatial perspective and limited learning capabilities of long-term dependencies in temporal perspective are still two striking bottlenecks. To address these problems, we propose a novel virtual graph modeling approach to focus on significant demand regions and a novel Deep Multi-View Spatio-temporal Virtual Graph Neural Network (DMVST-VGNN) to strengthen the learning capabilities of spatial dynamics and long-term temporal dependencies. Specifically, DMVST-VGNN integrates structures of 1D CNN, Multi-Graph Attention Neural Network and Transformer Network, which correspond to short-term temporal dynamics view, spatial dynamics view and long-term temporal dynamics view respectively. In this paper, multiple experiments are conducted on two large-scale New York City datasets in higher granularity prediction scenes. And the experimental results demonstrate the effectiveness of DMVST-VGNN framework in ride-hailing demand prediction, no matter in spatial scale or the temporal scale. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting is an indispensable task for traffic routing and disaster avoidance. Due to its strenuous movement, even the most recent deep learning techniques in computer vision deliver unsatisfactory performance. The main reason can be attributed to the following two aspects: 1) The traditional convolution in the input has a limited field to extract spatial representation. 2) The convolution in state-to-state connection might lead to mismatch problems and inaccurate strength predictions. To address the two drawbacks, we propose an innovative algorithm the Reconstitute Spatiotemporal LSTM (RST-LSTM) based on Convolutional Recurrent Neural Network (ConvRNN). In the proposed model, we present the local and global reconstitution scheme into the current input and the hidden state respectively. The local reconstitution (LR) can adaptively adjust the perceptual field of convolution so as to extract more useful spatial information and exclude invalid representation. Moreover, in the hidden state, the global reconstitution (GR) is embedded to alleviate the problem of mismatching between the current input and hidden state. Experimental results in MovingMNIST++ show that our approach can achieve the best predictions for those data with more drastic changes in the adjacent time. For radar data in CIKM AnalytiCup 2017 (we name it RadarCIKM in this paper), our method outperforms the state-of-the-art competitors. Furthermore, we notice that GR can promote the nowcasting in the high radar echo region and LR can reduce the error. [ABSTRACT FROM AUTHOR]

Deep learning methods are typically data-hungry and require many labelled samples. Unfortunately, the amount of effort required to label the data has significantly hindered the application of deep learning methods, especially in 3D modelling tasks requiring heterogeneous samples. This paper proposes a semi-supervised adversarial recognition strategy embedded in the inverse procedural modelling engine to reduce data annotation costs for learning to model 3D façades. Beginning with textured level-of-details models, we use convolutional neural networks to recognise the types and estimate the parameters of windows from image patches. The window types and parameters are then assembled into the procedural grammar. A simple procedural engine is built inside off-the-shelf 3D modelling software, producing fine-grained window geometries. To obtain a useful model from a few labelled samples, we leverage a generative adversarial network to train the feature extractor in a semi-supervised manner. The adversarial training strategy exploits the unlabelled data to stabilise the training phase. Experiments using publicly available façade image datasets reveal that the proposed methods can improve classification accuracy and parameter estimation by approximately 10% and 50%, respectively, under the same network structure. In addition, performance gains are more pronounced when testing against unseen data featuring different façade styles. [ABSTRACT FROM AUTHOR]

Multi-focus image fusion methods can be mainly divided into two categories: transform domain methods and spatial domain methods. Recent emerged deep learning (DL)-based methods actually satisfy this taxonomy as well. In this paper, we propose a novel DL-based multi-focus image fusion method that can combine the complementary advantages of transform domain methods and spatial domain methods. Specifically, a residual architecture that includes a multi-scale feature extraction module and a dual-attention module is designed as the basic unit of a deep convolutional network, which is firstly used to obtain an initial fused image from the source images. Then, the trained network is further employed to extract features from the initial fused image and the source images for a similarity comparison, aiming to detect the focus property of each source pixel. The final fused image is obtained by selecting corresponding pixels from the source images and the initial fused image according to the focus property map. Experimental results show that the proposed method can effectively preserve the original focus information from the source images and prevent visual artifacts around the boundary regions, leading to more competitive qualitative and quantitative performance when compared with the state-of-the-art fusion methods. • We propose a DL-based multi-focus image fusion framework that combines the advantages of TD and SD methods. • A CNN is designed to achieve two targets: initial fusion and focus property detection. • A residual block with multi-scale feature extraction and dual-attention is presented. • Experimental results show that our method obtains the state-of-the-art performance. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks have achieved state-of-the-art performance in image classification. Their lack of ability to recognise the spatial relationship between features, however, leads to misclassification of the variants of the same image. Capsule Networks were introduced to address this issue by incorporating the spatial information of image features into neural networks. In this paper, we are interested in showcasing the digit recognition task on CAPTCHA images, widely considered a challenge for computers in relation to human capabilities. Our intention is to provide a rigorous empirical regime in which we can compare the competitive performance of Capsule Networks against the Convolutional Neural Networks. Indeed since CAPTCHA distorts images, by adjusting the spatial positioning of features, we aim to demonstrate the advantages and limitations of Capsule Networks architecture. We train the Capsule Networks with Dynamic Routing version and the convolutional-neural-network-based deep-CAPTCHA baseline model to predict the digit sequences on numerical CAPTCHAs, investigate the performance results and propose two improvements to the Capsule Networks model. [ABSTRACT FROM AUTHOR]

Recently, the successful applications of convolutional neural network (CNN) in computer vision have attracted considerable attentions. Particularly, deep learning models with attention mechanisms have shown impressive performance in hyperspectral (HS) pansharpening. However, most of these existing models follow an early/late-fusion strategy and do not take full advantage of the hierarchical features. In this paper, we specifically design a novel end-to-end multi-level feature fusion network with cross-layer guided attention for HS pansharpening, termed as HP-MFFN, which allows the network to extract the hierarchical features level by level. Specifically, as the different levels of the network have different receptive fields and contain different details, the hierarchical features extracted from the HS image and panchromatic (PAN) image are refined by the cross-layer guided attention fusion module (called CLGAF) to yield more effective spatial-spectral features with fine details and rich semantics. The experimental results conducted on widely-used datasets demonstrate that HP-MFFN provides high-quality pansharpened HS images in terms of perceptually and quantitatively. [ABSTRACT FROM AUTHOR]

Neural networks have shown tremendous growth in recent years to solve numerous problems. Various types of neural networks have been introduced to deal with different types of problems. However, the main goal of any neural network is to transform the non-linearly separable input data into more linearly separable abstract features using a hierarchy of layers. These layers are combinations of linear and nonlinear functions. The most popular and common non-linearity layers are activation functions (AFs), such as Logistic Sigmoid, Tanh, ReLU, ELU, Swish and Mish. In this paper, a comprehensive overview and survey is presented for AFs in neural networks for deep learning. Different classes of AFs such as Logistic Sigmoid and Tanh based, ReLU based, ELU based, and Learning based are covered. Several characteristics of AFs such as output range, monotonicity, and smoothness are also pointed out. A performance comparison is also performed among 18 state-of-the-art AFs with different networks on different types of data. The insights of AFs are presented to benefit the researchers for doing further research and practitioners to select among different choices. The code used for experimental comparison is released at: https://github.com/shivram1987/ActivationFunctions. [ABSTRACT FROM AUTHOR]

Improving existing neural network architectures can involve several design choices such as manipulating the loss functions, employing a diverse learning strategy, exploiting gradient evolution at training time, optimizing the network hyper-parameters, or increasing the architecture depth. The latter approach is a straightforward solution, since it directly enhances the representation capabilities of a network; however, the increased depth generally incurs in the well-known vanishing gradient problem. In this paper, borrowing from different methods addressing this issue, we introduce an interlaced multi-task learning strategy, defined SIRe, to reduce the vanishing gradient in relation to the object classification task. The presented methodology directly improves a convolutional neural network (CNN) by preserving information from the input image through interlaced auto-encoders (AEs), and further refines the base network architecture by means of skip and residual connections. To validate the presented methodology, a simple CNN and various implementations of famous networks are extended via the SIRe strategy and extensively tested on five collections, i.e., MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, and Caltech-256; where the SIRe-extended architectures achieve significantly increased performances across all models and datasets, thus confirming the presented approach effectiveness. • Designing an interlaced multi-task learning approach to affect the gradient. • Further improving the proposed interlaced approach via skip/residual connections. • Applying the proposed SIRe methodology to other well-known architectures. [ABSTRACT FROM AUTHOR]

In this paper, a clustering method for KMedoids based on dynamic time warping (DTW-KMedoids) is designed to analyze multi-channel signals, and a lightweight network, clustered blueprint separable convolutional neural network (CBS-CNN), is established to perform fault diagnosis of high-speed train (HST) bogie. The motivation for proposing the novel method is to address the problems of large network size and high training cost in deep learning. First, DTW-KMedoids is adopted to cluster the channels of multi-channel signals. Second, based on the principles of blueprint separable convolution (BSConv) and mixed depthwise convolution (MixConv), a lightweight convolution model construction strategy called clustered blueprint separable convolution (CBS-Conv) is proposed, which uses the same blueprint to convolute the data of the channels in a cluster. Third, CBS-CNN is established, with multiple branches to process data from different clusters, and the computational result of each branch is connected by the proposed Connect layer. Finally, by virtue of the learned features from training, the model completes the end-to-end HST bogie fault diagnosis task, where the usefulness of CBS-CNN in detecting bogie failures including component performance degradation, component failures, and composite failures is validated. Further experiments show that CBS-CNN has a remarkable ability to adapt itself to different task environments and objectives. [ABSTRACT FROM AUTHOR]

Reliable prediction of solar thermal power is essential for optimal operation and control of renewable energy driven distributed power systems. This paper presents a Convolutional Neural Networks (CNNs) based multivariate approach for forecasting power generation from solar thermal collectors over multiple horizons simultaneously. It also demonstrates an application of solar thermal power generation forecasting in a building cooling system as part of a predictive central controller. Historical data from an evacuated collector field and a single axis tracking collector field have been used to develop the prediction models and assess the performance of the proposed approach. Experimental results show that the proposed approach provides accurate prediction for multiple forecast horizons: MAPE is 2.99%–4.18% for 30 min to 24 h ahead prediction. The proposed approach utilising both historical and predicted future weather data achieves 25%–37% improvements of accuracy compared to its univariate counterpart that uses only lagged power data as input. It also outperforms existing data driven approaches based on NNs, LSTM, and RF, and achieves 5.46%–21.28% statistically significant improvements compared to them. ● Application of Convolutional Neural Networks to forecast solar thermal power output. ● Data cleaning based on an unsupervised ML method (called local outlier factor). ● Input variables selection by applying mutual information. ● Evaluation of prediction model using data for two types of solar thermal collectors. ● 5–21% improvement of prediction accuracy over existing data-driven approaches. [ABSTRACT FROM AUTHOR]

The evaluation of node importance is a critical research topic in network science, widely applied in social networks, transport systems, and computer networks. Prior works addressing this topic either consider a single metric or assign weights for multiple metrics or select features by handcraft, which exist one-sidedness and subjectivity issues. In this paper, to tackle these problems, we propose a new approach named CGNN to identify influential nodes based on deep learning methods, including Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs). CGNN obtains the feature matrices by the contraction algorithm and gets the labels by the Susceptible-Infected-Recovered (SIR) model, which will be leveraged for learning the hidden representations of nodes without utilizing any network metrics as features. We adopt three evaluation criteria to verify CGNN concerning effectiveness and distinguishability, including Kendall's τ correlation coefficient, monotonicity index (MI), and ranking distribution function (RDF). Nine baselines are employed to compare with CGNN on thirty synthetic networks and twelve real-world networks from different domains. Simulation results demonstrate that CGNN manifests better performance than the baselines, in which the values of τ are large and significantly increase, the values of MI approach to 1, and the points in the RDF curves distribute more uniformly. These results may provide reference significance for controlling epidemic spreading and enhancing network robustness. [ABSTRACT FROM AUTHOR]

Multimodal registration is a challenging problem in visual computing, commonly faced during medical image-guided interventions, data fusion and 3D object retrieval. The main challenge of multimodal registration is finding accurate correspondence between modalities, since different modalities do not exhibit the same characteristics. This paper explores how the coherence of different modalities can be utilized for the challenging task of 3D multimodal registration. A novel deep learning multimodal registration framework is proposed by introducing a siamese deep learning architecture, especially designed for aligning and fusing modalities of different structural and physical principles. The cross-modal attention blocks lead the network to establish correspondences between features of different modalities. The proposed framework focuses on the alignment of 3D point clouds and the micro-CT 3D volumes of the same object. A multimodal dataset consisting of real micro-CT scans and their synthetically generated 3D models (point clouds) is presented and utilized for evaluating our methodology. [Display omitted] • The problem of multimodal 3D registration of CT volumes and 3D point clouds is formally defined. • A framework for multimodal 3D registration of CT volumes and 3D point clouds is proposed. • A DLN that combines regular CNNs suited for data with standard grid structure and GDL suited for unstructured data. • The proposed network employs siamese architecture for effective multimodality fusion. • A multimodal dataset for evaluating algorithms for aligning CT volumes and point clouds is created. [ABSTRACT FROM AUTHOR]

Point clouds are gaining prominence as a method for representing 3D shapes, but their irregular structure poses a challenge for deep learning methods. In this paper we propose CloudWalker, a novel method for learning 3D shapes using random walks. Previous works attempt to adapt Convolutional Neural Networks (CNNs) or impose a grid or mesh structure to 3D point clouds. This work presents a different approach for representing and learning the shape from a given point set. The key idea is to impose structure on the point set by multiple random walks through the cloud for exploring different regions of the 3D object. Then we learn a per-point and per-walk representation and aggregate multiple walk predictions at inference. Our approach achieves state-of-the-art results for two 3D shape analysis tasks: classification and retrieval. • We propose CloudWalker a novel representation for point clouds using random walks. • This work presents a different approach for representing and learning 3D shape. • Our walks impose order and overcome the lack of connectivity and structure. • We learn per-point and per-walk representation and aggregate multiple walks per shape. • Our approach achieves SOTA results for 3D shape analysis tasks. [ABSTRACT FROM AUTHOR]

Integrating the special electromagnetic characteristics of Synthetic Aperture Radar (SAR) in deep neural networks is essential in order to enhance the explainability and physics awareness of deep learning. In this paper, we first propose a novel physically explainable convolutional neural network for SAR image classification, namely physics guided and injected learning (PGIL). It comprises three parts: (1) explainable models (XM) to provide prior physics knowledge, (2) physics guided network (PGN) to encode the knowledge into physics-aware features, and (3) physics injected network (PIN) to adaptively introduce the physics-aware features into classification pipeline for label prediction. A hybrid Image-Physics SAR dataset format is proposed for evaluation, with both Sentinel-1 and Gaofen-3 SAR data being experimented. The results show that the proposed PGIL substantially improve the classification performance in case of limited labeled data compared with the counterpart data-driven CNN and other pre-training methods. Additionally, the physics explanations are discussed to indicate the interpretability and the physical consistency preserved in the predictions. We deem the proposed method would promote the development of physically explainable deep learning in SAR image interpretation field. [ABSTRACT FROM AUTHOR]

• We propose a new architecture that outperforms the linear convolution. • It expands convolution to a higher degree kernel function without additional weights. • We opt for Taylor series kernel which maps data in both implicit and explicit ways. • This architecture is able to learn more complex patterns thus be more discriminative. • The experiments demonstrate that our architecture outperforms the convolution layer. The ever-growing depth and width of Convolutional Neural Networks (CNNs) drastically increases the number of their parameters and requires more powerful devices to train and deploy. In this paper, we propose a new architecture that outperforms the classical linear convolution function by expanding the latter to a higher degree kernel function without additional weights. We opt for Taylor Series Kernel which maps input data to a higher-dimensional Reproducing Kernel Hilbert Space (RKHS). Mapping features to a higher-order RKHS is performed in both implicit and explicit ways. For the former way, we compute several polynomial kernels of different degrees leveraging the kernel trick. Whereas, the latter way is achieved by concatenating the result of these polynomial kernels. The proposed Taylor Series Kernelized Convolution (TSKC) is able to learn more complex patterns than the linear convolution kernel and thus be more discriminative. The experiments conducted on Facial Expression Recognition (FER) datasets demonstrate that TSKC outperforms the ordinary convolution layer without additional parameters. [ABSTRACT FROM AUTHOR]

• Proposed a novel scheme for texture image retrieval. • Proposed scheme exploits the strength of both hand-crafted and deep features. • Modified distance function is used for image retrieval in wavelet feature space. • Distance function is weighted using output class memberships of the query image. • Efficiency of the proposed method found to be promising. [Display omitted] In the modern digital era, with the availability of low-cost hardware like sensors and cameras, a huge amount of image databases are being created for diverse applications. These databases give rise to the need of developing efficient content-based image retrieval (CBIR) systems. Major efforts have been put over the past two decades to develop different global and low-level texture features to build efficient CBIR systems. However, designing texture features that are suitable for distance-based retrieval is always a challenging task. Recently, Convolution Neural Networks have shown promising results for object detection and classification. CNNs are also applied to build classifier-based retrieval systems. However, the classifier-based retrieval methods can retrieve images only from the predicted class. Therefore, the performance of such system greatly depends on classification performance of the classifier. This paper proposes a method that exploits the strength of the Convolutional Neural Networks for predicting the class membership of the query image for all output classes and retrieve images using a modified distance function in the wavelet feature space. The performance of the proposed method is evaluated using three popular texture datasets of varying complexity and found to be superior to all competing methods considered. [ABSTRACT FROM AUTHOR]

This paper presents a novel deep learning approach for reconstruction of local through-the-thickness fiber orientation distribution (FOD) in discontinuous long-fiber, prepreg-platelet molded composites (PPMC). The thermal-residual strains on composite surfaces are used as inputs to train a fully convolutional neural network, named U-Net, to predict spatially varying, local FOD. High fidelity synthetic data was generated via computational simulation of PPMC with stochastic material orientation state and was used for training the deep learning model. The proposed U-Net model allowed for rapid recognition of PPMC morphology by solving the inverse structural mechanics problem of determining the average fiber orientation through the composite thickness based on the provided surface strain measurements. Upon training and validation, the U-Net deep learning model was deployed to rapidly predict complex distributions of the local through-the-thickness FOD in PPMC. [ABSTRACT FROM AUTHOR]

The digitization of engineering drawings is crucial for efficient reuse, distribution, and archiving. Existing computer vision approaches for digitizing engineering drawings typically assume the input drawings have high quality. However, in reality, engineering drawings are often blurred and distorted due to improper scanning, storage, and transmission, which may jeopardize the effectiveness of existing approaches. This paper focuses on restoring and recognizing low-quality engineering drawings, where an end-to-end framework is proposed to improve the quality of the drawings and identify the graphical symbols on them. The framework uses K-means clustering to classify different engineering drawing patches into simple and complex texture patches based on their gray level co-occurrence matrix statistics. Computer vision operations and a modified Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) model are then used to improve the quality of the two types of patches, respectively. A modified Faster Region-based Convolutional Neural Network (Faster R-CNN) model is used to recognize the quality-enhanced graphical symbols. Additionally, a multi-stage task-driven collaborative learning strategy is proposed to train the modified ESRGAN and Faster R-CNN models to improve the resolution of engineering drawings in the direction that facilitates graphical symbol recognition, rather than human visual perception. A synthetic data generation method is also proposed to construct quality-degraded samples for training the framework. Experiments on real-world electrical diagrams show that the proposed framework achieves an accuracy of 98.98% and a recall of 99.33%, demonstrating its superiority over previous approaches. Moreover, the framework is integrated into a widely-used power system software application to showcase its practicality. The reference codes and data can be found at https://github.com/Lattle-y/AI-recognition-for-lq-ed.git Future work will focus on improving the generalizability of the proposed framework to different quality degradation scenarios and extrapolating the application to different engineering domains. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

The harmful utilization of DeepFake technology poses a significant threat to public welfare, precipitating a crisis in public opinion. Existing detection methodologies, predominantly relying on convolutional neural networks and deep learning paradigms, focus on achieving high in-domain recognition accuracy amidst many forgery techniques. However, overseeing the intricate interplay between textures and artifacts results in compromised performance across diverse forgery scenarios. This paper introduces a groundbreaking framework, denoted as Texture and Artifact Detector (TAD), to mitigate the challenge posed by the limited generalization ability stemming from the mutual neglect of textures and artifacts. Specifically, our approach delves into the similarities among disparate forged datasets, discerning synthetic content based on the consistency of textures and the presence of artifacts. Furthermore, we use a model ensemble learning strategy to judiciously aggregate texture disparities and artifact patterns inherent in various forgery types, thereby enabling the model's generalization ability. Our comprehensive experimental analysis, encompassing extensive intra-dataset and cross-dataset validations along with evaluations on both video sequences and individual frames, confirms the effectiveness of TAD. The results from four benchmark datasets highlight the significant impact of the synergistic consideration of texture and artifact information, leading to a marked improvement in detection capabilities. • We build an artifact and texture inconsistency detector for deepfake detection. • Texture coherence is analyzed via a novel mask extraction method and an autoencoder. • The use of an ensemble training strategy improved the ability to generalize. • Extensive experiments verified the effectiveness of the model we built. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

The image noise generated by the welding process, such as arc light, splash, and smoke, brings significant challenges for the laser vision sensor-based welding robot to locate the weld seam and accurately conduct automatic welding. Currently, deep learning-based approaches surpass traditional methods in flexibility and robustness. However, their significant computational cost leads to a mismatch with the real-time requirement of automated welding. In this paper, we propose an efficient hybrid architecture of Convolutional Neural Network (CNN) and transformer, referred to as Dynamic Squeeze Network (DSNet), for real-time weld seam segmentation. More precisely, a lightweight segmentation framework is developed to fully leverage the advantages of the transformer structure without significantly increasing computational overhead. In this respect, an efficient encoder, which aims to increase its features diversity, has been designed and resulted in substantial improvement of encoding performance. Moreover, we propose a plug-and-play lightweight attention module that generates more effective attention weights by exploiting statistical information of weld seam data and introducing linear priors. Extensive experiments on weld seam images using NVIDIA GTX 1050Ti show that our approach reduces the number of parameters by 54x, decreases computational complexity by 34x, and improves inference speed by 33x compared to the baseline method TransUNet. DSNet achieves superior accuracy (78.01% IoU, 87.64% Dice) and speed performance (100 FPS) with lower model complexity and computational burden than most state-of-the-art methods. The code is available at https://github.com/hackerschen/DSNet. [ABSTRACT FROM AUTHOR]

This paper demonstrates L-band quantum-dash laser (QDL) based orbital angular momentum (OAM) structured light in a free space optics communication (FSO) system. A 4-ary OAM-shift-keying pattern coding communication system, based on Laguerre Gaussian (LG) and superposition LG (MuxLG) mode families, has been investigated under a foggy FSO channel. In addition, joint mode identification and channel condition estimation have been developed at the receiver side using advanced deep learning (DL) methods. We utilize and compare the performance of the convolutional neural networks (CNN) and UNET algorithms. An experimental setup has been conducted using an in-house controlled foggy chamber which allows an FSO transmission of 3-m length. Furthermore, we propose a data balancing approach to the experimental dataset by data augmentation. Visibility prediction results have shown a measured root mean square error of 17(18) m and 10(10) m for 4-ary LG(MuXLG) using CNN and UNET models, respectively. Moreover, the DL models provide an average mode classification accuracy of 94% under various channel visibility conditions. • OAM mode patterns was generated using QDL source and identified under foggy channel conditions. • Eight modes have been formed from the LG mode family. • A controlled chamber environment to emulate the outdoor fog condition was considered. • CNN and UNET DL methods were used as classifiers and regressors for mode identification and channel condition prediction. • Visibility parameter can be estimated with 17 and 10 m root mean square error (RMSE) using CNN and UNET techniques. [ABSTRACT FROM AUTHOR]

Skin diseases are reported to contribute 1.79% of the global burden of disease. The accurate diagnosis of specific skin diseases is known to be a challenging task due, in part, to variations in skin tone, texture, body hair, etc. Classification of skin lesions using machine learning is a demanding task, due to the varying shapes, sizes, colors, and vague boundaries of some lesions. The use of deep learning for the classification of skin lesion images has been shown to help diagnose the disease at its early stages. Recent studies have demonstrated that these models perform well in skin detection tasks, with high accuracy and efficiency. Our paper proposes an end-to-end framework for skin lesion classification, and our contributions are two-fold. Firstly, two fundamentally different algorithms are proposed for segmenting and extracting features from images during image preprocessing. Secondly, we present a deep convolutional neural network model, S-MobileNet that aims to classify 7 different types of skin lesions. We used the HAM10000 dataset, which consists of 10000 dermatoscopic images from different populations and is publicly available through the International Skin Imaging Collaboration (ISIC) Archive. The image data was preprocessed to make it suitable for modeling. Exploratory data analysis (EDA) was performed to understand various attributes and their relationships within the dataset. A modified version of a Gaussian filtering algorithm and SFTA was applied for image segmentation and feature extraction. The processed dataset was then fed into the S-MobileNet model. This model was designed to be lightweight and was analyzed in three dimensions: using the Relu Activation function, the Mish activation function, and applying compression at intermediary layers. In addition, an alternative approach for compressing layers in the S-MobileNet architecture was applied to ensure a lightweight model that does not compromise on performance. The model was trained using several experiments and assessed using various performance measures, including, loss, accuracy, precision, and the F1-score. Our results demonstrate an improvement in model performance when applying a preprocessing technique. The Mish activation function was shown to outperform Relu. Further, the classification accuracy of the compressed S-MobileNet was shown to outperform S-MobileNet. To conclude, our findings have shown that our proposed deep learning-based S-MobileNet model is the optimal approach for classifying skin lesion images in the HAM10000 dataset. In the future, our approach could be adapted and applied to other datasets, and validated to develop a skin lesion framework that can be utilized in real-time. [ABSTRACT FROM AUTHOR]

Weed and crop segmentation is becoming an increasingly integral part of precision farming that leverages the current computer vision and deep learning technologies. Research has been extensively carried out based on images captured with a camera from various platforms. Unmanned aerial vehicles (UAVs) and ground-based vehicles including agricultural robots are the two popular platforms for data collection in fields. They all contribute to site-specific weed management (SSWM) to maintain crop yield. Currently, the data from these two platforms is processed separately, though sharing the same semantic objects (weed and crop). In our paper, we have proposed a novel method with a new deep learning-based model and the enhanced data augmentation pipeline to train field images alone and subsequently predict both field images and UAV images for weed segmentation and mapping. The network learning process is visualized by feature maps at shallow and deep layers. The results show that the mean intersection of union (IOU) values of the segmentation for the crop (maize), weeds, and soil background in the developed model for the field dataset are 0.744, 0.577, 0.979, respectively, and the performance of aerial images from an UAV with the same model, the IOU values of the segmentation for the crop (maize), weeds and soil background are 0.596, 0.407, and 0.875, respectively. To estimate the effect on the use of plant protection agents, we quantify the relationship between herbicide spraying saving rate and grid size (spraying resolution) based on the predicted weed map. The spraying saving rate is up to 90 % when the spraying resolution is at 1.78 × 1.78 cm2. The study shows that the developed deep convolutional neural network could be used to classify weeds from both field and aerial images and delivers satisfactory results. To achieve this performance, it is crucial to perform preprocessing techniques that reduce dataset differences between two distinct domains. [ABSTRACT FROM AUTHOR]

• To design an automatic multi-disease detection in CXR images by ensemble model. • To adopt ODeepLabv3 to perform segmentation process for multi-disease detection. • To adopt a new ensemble model named Optimized Ensemble Transfer Learning (OETL). • To design a Mutation Rate-based Lion Algorithm (MR-LA) to obtain high accuracy. • To optimize the epochs in ensemble classifier with the help of MR-LA algorithm. Chest radiography is a comparatively low-cost, most probably applied in a medical process that brings crucial information to discover diagnostic conclusions. Chest X-rays (CXR) are used in the prognosis of chest diseases that includes asthma, lung cancer, pneumonia, and COVID-19. Artificial intelligence detects the multi-disease automatically to enhance its proficiency and performance by solving image detection complications with various machine learning and deep learning approaches. Convolutional Neural Network (CNN) has been designed for the advancement of computerized recognition systems. For the classification of medical images, texture, shape, size, and tissue constitution are essential features for detecting diseases. Hence, huge input features are merged with deep CNN validated to increase the effectiveness of CXR analysis. In addition, multi-scale features are visualized in CNN for the detection of variable sizes of thoracic diseases. This paper has implemented a novel multi-disease diagnosis model using chest X-ray images through deep learning approaches, and thus, it helps to minimize the computational cost. The standard CXR images are gathered from the standard datasets. After, the pre-processing is performed for cleaning and contrast the images. Further, the image segmentation is carried out using Optimized DeepLabv3 (ODeepLabv3), where the optimization of parameters in DeepLabv3 is done by a new Mutation Rate-based Lion Algorithm (MR-LA). Then, multi-disease classification is carried out through Optimized Ensemble Transfer Learning (OETL), where the OETL is carried out through VGG16, ResNet, ImageNet, MobileNet, GoogleNet, Inception, and Xception. Here, the parameter optimization of all models is done by the same MR-LA. The proposed model improved the effectiveness and performance in expressions of sensitivity, precision, and specificity factors and get higher classification and detection accuracy. The outcome of the recommended method over other models is identified by the relative analysis conducted. [ABSTRACT FROM AUTHOR]

Within the contemporary dairy industry, the effective monitoring of cheese ripeness constitutes a critical yet challenging task. This paper proposes the first public dataset encompassing images of cheese wheels that depict various products at distinct stages of ripening and introduces an innovative hybrid approach, integrating machine learning and computer vision techniques to automate the detection of cheese ripeness. By leveraging deep learning and shallow learning techniques, the proposed method endeavors to overcome the limitations associated with conventional assessment methodologies. It aims to provide automation, precision, and consistency in the evaluation of cheese ripeness, delving into a hierarchical classification for the simultaneous classification of distinct cheese types and ripeness levels and presenting a comprehensive solution to enhance the efficiency of the cheese production process. By employing a lightweight hierarchical feature aggregation methodology, this investigation navigates the intricate landscape of preprocessing steps, feature selection, and diverse classifiers. We report a noteworthy achievement, attaining a best F-measure score of 0.991 through the merging of features extracted from EfficientNet and DarkNet-53, opening the field to concretely address the complexity inherent in cheese quality assessment. [ABSTRACT FROM AUTHOR]

In the face of uncontrolled offensive content on social media, automated detection emerges as a critical need. This paper tackles this challenge by proposing a novel approach for identifying offensive language in multilingual, code-mixed, and script-mixed settings. The study presents a novel multilingual hybrid dataset constructed by merging diverse monolingual and bilingual resources. Further, we systematically evaluate the impact of input representations (Word2Vec, Global Vectors for Word Representation (or GloVe), Bidirectional Encoder Representations from Transformers (or BERT), and uniform initialization) and deep learning models (Convolutional Neural Network (or CNN), Bidirectional Long Short Term Memory (or Bi-LSTM), Bi-LSTM-Attention, and fine-tuned BERT) on detection accuracy. Our comprehensive experiments on a dataset of 42,560 social media comments from five languages (English, Hindi, German, Tamil, and Malayalam) reveal the superiority of fine-tuned BERT. Notably, it achieves a macro average F 1 -score of 0.79 for monolingual tasks and an impressive 0.86 for code-mixed and script-mixed tasks. These findings significantly advance offensive language detection methodologies and shed light on the complex dynamics of multilingual social media, paving the way for more inclusive and safer online communities. [ABSTRACT FROM AUTHOR]

• a combination of 3DCNN-LSTM is proposed. • an attention mechanism is proposed. • a data stacking method to perceive the correlation between features. Although architectures such as recurrent neural networks (RNN) perform well in time series prediction, there is still challenges for existing deep learning methods in extracting comprehensive spatio-temporal features. This paper proposes a fault diagnosis method (3D-ALCNN) using a three-dimensional convolutional neural network (3DCNN), a long-short-term memory network (LSTM) and an attentional mechanism to enhance the correlation perception among important features. The 3DCNN and LSTM are firstly used to extract temporal features and a data stacking method to adapt the one-dimensional data to the input requirements of 3DCNN is adopted. Then, the correlation between different features is identified through the attention mechanism and the neural network is guided to focus on more important features. Eventually, the model output is fed into a classifier for fault classification. Simulation experiments on the industrial coke furnace show that the 3D-ALCNN model outperforms existing methods in terms of fault diagnosis accuracy and efficiency, especially when dealing with chemical data with complex time dependencies. [ABSTRACT FROM AUTHOR]

• We propose deep learning models for classification of cancer cells belonging to different organs on holographic images. • We present self-supervised pretraining and hyperparameter tuning for efficient training on biomedical holographic datasets. • Perform extensive experiments with self-supervised frameworks and compared self-supervised learning with supervised learning and proved the effectiveness of the former for holographic images classification. • Proposed work can be an effective tool for the automatic classification of cancer cells for medical diagnosis using self-supervised learning. Image-based stain-free elliptical cancer cell classification is challenging, due to the inter-class morphological similarity. In this paper, we address the classification of different types of cancer cell lines (lung, breast, bladder, and skin) by utilizing self-supervised learning, and compare it with supervised learning based on convolutional neural network. Digital holography in a microscopic configuration was used to obtain stain-free quantitative phase images representing the intracellular content and morphology of cells. The performance of self-supervised learning in natural images shows promising results, and consistently closes the gap between self-supervised and supervised learning. The ability of self-supervised learning to effectively utilize unlabeled data for training is instrumental in the biomedical domain, where labeled data is scarce. Our goal is to study different self-supervised frameworks of biomedical holographic data, and determine how they can be utilized to advance liquid biopsy for the detection of cancer cells. After extensive experimentation, we conclude that self-supervised learning improves the classification performance on cancer cell datasets, and outperforms supervised learning when training data is limited, which is mostly the case in biomedical imaging. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]