Search

Municipal papers and polyethylene terephthalate (PET) wastes are primary constituents of municipal solid waste (MSW) which necessitates effective disposal. Co-pyrolysis is an efficient transformation technology for the production of valuable gas and liquid fuels. Therefore, co-pyrolysis characteristics, kinetics, gas and liquid products of cellulose and PET mixtures were investigated in a two-stage reactor by multiple characterizations coupling with deep learning approach. The results indicated that there exhibited positive interaction for cellulose and PET co-pyrolysis in terms of mass difference (δ m) and mass loss rate difference (δ r) between experimental and calculated values. The thermal decomposition process and synergistic effects (δ m and δ r) were accurately predicted by deep learning approaches (R 2 =0.9823–0.9993) using temperature, catalyst, mixing ratio and heating rate variables. Feature importance revealed that the temperature, catalyst, heating rate and mixing ratio were in the descending order to affect the remaining mass from TG curves. And the temperature played a predominant role in synergistic effects. The Flynn-Wall-Ozawa (FWO) method was more suitable for fitting the pyrolysis process to obtain activation energy (E a) with higher R 2 (0.94714–0.99640) compared with Kissinger-Akahira-Sunose (KAS) method. When the mixing ratio of cellulose and PET was 1:1, the E a was significantly reduced to 80.73 KJ/mol, and further reduced to 73.04 KJ/mol by introducing the catalyst, as compared to E a of individual cellulose (178.95 KJ/mol) and PET (198.34 KJ/mol). Co-pyrolysis of cellulose and PET led to a reduction of carbohydrates, furans, alcohols, ketones, and phenols, while the content of acids was remarkably increased which was consistent with the observation in FTIR that the absorption peak of C＝O became sharp and strong. The introduction of ZSM-11 resulted in an increase in aromatics and a decrease in acids content owing to aromatization and deoxygenation. This work is expected to offer beneficial guidance for the efficient disposal of MSW and the application of deep learning methods. [Display omitted] • The interactive pathway for catalytic co-pyrolysis of cellulose and PET was proposed. • The activation energy reached the lowest 73.04 KJ/mol when the mixing ratio was 1:1. • Optimized convolutional neural network predicted synergistic effects with R 2 > 0.99. • Feature importance for TG was temperature > catalyst > heating rate > mixing ratio. [ABSTRACT FROM AUTHOR]

In order to accurately estimate the period of X-ray pulsars to improve navigation performance, this paper proposes a pulsar period estimation method based on photon energy using reverse Hilbert coding and double Convolutional Neural Networks (CNNs) discrimination. This method uses photon energy information to perform epoch folding. It converts one-dimensional contour information into two-dimensional image signals through reverse Hilbert curve and recognizes contour image features through CNN models to determine the optimal period. According to the simulation results, the period estimation accuracy of this method is 0.5350 ns, which is 65.71% higher than the χ 2 epoch folding method. The contour phase accuracy is 0.0004767, which is 41.93% higher than the χ 2 epoch folding method. At the same time, the observation time and noise interference that may affect the accuracy of period estimation are also quantitatively analyzed. In addition, in order to simulate real navigation scenarios, this paper also conducts dynamic period estimation experiments on PSR B0531+21 pulsar in a specific period of time. The method proposed in this paper has the advantages of high precision, fast calculation speed, strong anti-interference ability and accurate dynamic period estimation, which can significantly improve the navigation performance of X-ray pulsars. • The proposed period estimation method uses photon energy information for contour weighted folding. • The proposed method uses reverse Hilbert curve to convert one-dimensional contour information into two-dimensional image information. • Convolutional Neural Network models are used to optimize pulsar period estimation. [Display omitted] [ABSTRACT FROM AUTHOR]

Autonomous spacecraft relative navigation via monocular images became a hot topic in the past few years and, recently, received a further push thanks to the constantly growing field of artificial neural networks and the publication of several spaceborne image datasets. Despite the proliferation of spacecraft relative-state initialization algorithms developed, most architectures adopt computationally expensive solutions relying on convolutional neural networks (CNNs) that provide accurate output at the cost of a high computational burden that seems unfeasible for current spaceborne hardware. The paper addresses this issue by proposing a novel pose initialization algorithm based on lightweight CNNs. Inspired by previous state-of-the-art algorithms, the developed architecture leverages a fast and accurate target detection CNN followed by a line segment detection CNN capable of running with low inference time on mobile devices. The line segments and their junctions are grouped into complex geometrical groups, reducing the solution search space, and subsequently, they are adopted to extract the final pose estimate. As a main outcome, the analyses demonstrate that the lightweight architecture developed scores high accuracy in the pose estimation task, with a mean estimation error of less than 10 cm in translation and 2.5°in rotation. The baseline algorithm scores a mean SLAB error of 0.04552 with a standard deviation of 0.22972 in the test dataset. Detailed analyses demonstrate that the uncertainties on the overall pose score are driven mainly by errors in the relative attitude, which gives the highest contribution to the pose error metric adopted. The analyses on the error distributions point out that the uncertainties on the estimated relative position are higher in the camera boresight axis direction. Concerning the relative attitude, the algorithm proposed has higher uncertainties in estimating directions of the target x and y axes due to ambiguities related to the target geometry. Notably, the target detection CNN trained in this work outperforms the previous top scores in the benchmark dataset. The performances of the proposed algorithm have been investigated further by analyzing the effects on the accuracy due to the relative distance and the presence of background in the images. Lastly, the paper delves into the possibility of adopting a sub-portion of the 2D-to-3D match matrix made by the most complex perceptual groups identified that positively affects the overall run-time, pointing out the performances in terms of accuracy of the estimates and providing a comparison of both the baseline and the reduced match matrix versions against state-of-the-art algorithms concerning relative position and attitude errors and solution availability, highlighting the high accuracy and solution availability of the proposed architectures. • New relative pose initialization algorithm based on the detection of line segments. • Detailed literature review and description of available spaceborne image datasets. • Improvement of state of the art in spacecraft detection using lightweight CNN. • Assessment of accuracy of the proposed architecture and error sources analyses. • Proven feasibility of the approach via comparison with other architectures. [ABSTRACT FROM AUTHOR]

Impact crater detection is one of the most important and critical studies of the geological history of planetary surfaces. Recent literature on crater identification demonstrates the powerful potential of deep-learning algorithms. However, these methods potentially assumed that the detection is conducted in devices with sufficient computing resources, and used networks with massive learnable parameters to obtain superb performance. In this paper, we proposed a lightweight tensorial convolutional neural network (CNN) for lunar impact crater detection. Specifically, we develop a tensorial you only look once (TYOLO) network by representing the convolutional kernels in tensor train (TT) decomposition. Thus, the parameter redundancy of TYOLO can be greatly reduced. To the best of our knowledge, this is the first work that considers the lightweight CNN model for impact crater detection. We demonstrate the lunar impact crater detection results over the ChangE-2 lunar digital orthophotos model (DOM) and digital evaluated model (DEM). Experimental results show that the proposed TYOLO network can obtain 1.44 × speed up and 39 × compression with 2.4 % improvement in crater detection. Overall, the proposed network demonstrates great potential in spacecraft with emerging computing resource constraints. [ABSTRACT FROM AUTHOR]

Cameras are rapidly becoming the choice for on-board sensors towards space rendezvous due to their small form factor and inexpensive power, mass, and volume costs. When it comes to docking, however, they typically serve a secondary role, whereas the main work is done by active sensors such as lidar. This paper documents the development of a proposed AI-based (artificial intelligence) navigation algorithm intending to mature the use of on-board visible wavelength cameras as a main sensor for docking and on-orbit servicing (OOS), reducing the dependency on lidar and greatly reducing costs. Specifically, the use of AI enables the expansion of the relative navigation solution towards multiple classes of scenarios, e.g., in terms of targets or illumination conditions, which would otherwise have to be crafted on a case-by-case manner using classical image processing methods. Multiple convolutional neural network (CNN) backbone architectures are benchmarked on synthetically generated data of docking manoeuvres with the International Space Station (ISS), achieving position and attitude estimates close to 1 % range-normalised and 1 deg, respectively, an established rule of thumb for the navigation measurement accuracy during final approach. The integration of the solution with a physical prototype of the refuelling mechanism is validated in laboratory using a robotic arm to simulate a berthing procedure. • A novel AI-based solution for spacecraft docking and refuelling is presented. • First demonstration of deep learning for vision-based pose estimation in docking. • Various CNN backbones are benchmarked on photorealistic ISS refuelling datasets. • ResNet-101 reports average 1.19% error in position, 0.29 ∘ in attitude per trajectory. • Prototypes of the developed mechanism are validated in lab with 7-DOF manipulator. [ABSTRACT FROM AUTHOR]

The large number of near-Earth asteroids (NEAs) has greatly impacted human space activities and Earth security. However, detecting NEAs in astronomical images with complex, varying backgrounds is still extremely challenging. In this paper, we propose a deep segmentation assisted asteroid tracking algorithm, termed DSAT, to construct a possible pipeline for faint NEA tracking in astronomical images. First, the single-frame object detection problem is converted to a segmentation problem, enabling robust extraction of faint potential moving objects. Then, a multiframe motion prior-based moving object tracking algorithm is proposed to find real NEAs. We further propose a distance tolerance criterion to help DSAT achieve effective tracking in practical situations when detection has partially failed. Finally, the pipeline is tested with both simulated and real astronomical images at different SNRs and in crowded fields. The results showed that our pipeline has the potential to detect and track faint NEAs in complex backgrounds. Our code is publicly available at https://github.com/zhenhongdu/DeepSegAsteroidTracker. [ABSTRACT FROM AUTHOR]

• Proposing physics-informed DCNN for HFI-SCB temperature field reconstruction. • Proposing Deep MC-QR method to quantify data uncertainty caused by noise. • Proposing HFI-SCB reliability analysis method based on interval multilevel BN. Temperature field reconstruction is essential for analyzing the reliability of a high-density functionally integrated satellite circuit board (HFI-SCB). As a representative deep learning model, the deep convolutional neural network (DCNN) is a powerful tool for reconstructing the HFI-SCB temperature field. However, DCNN needs a lot of labeled data to learn its parameters, which is contrary to the fact that actual satellite engineering can only acquire noisy unlabeled data. Thus, this paper proposes an unsupervised method, i.e., the physics-informed deep Monte Carlo quantile regression method, for reconstructing the HFI-SCB temperature field and quantifying the data uncertainty caused by sensor noise. The proposed method combines a DCNN with known physics knowledge to reconstruct an accurate HFI-SCB temperature field using only monitoring point temperatures. Besides, the proposed method can quantify the data uncertainty by the Monte Carlo quantile regression. Based on the reconstructed temperature field and the quantified data uncertainty, this paper builds an interval multilevel Bayesian Network to analyze the HFI-SCB reliability. Two case studies are used to validate the proposed method. [ABSTRACT FROM AUTHOR]

The complex flow of a supersonic jet (Mach number 2.4–2.5) around the blunt aerodynamic body (consisting of a cylinder with a diameter of 16 mm and a truncated cone) was studied using the high-speed shadowgraph technique and computer vision-based visual data processing. The problem considered is directly related to the rocket and space systems stages safe separation problem in a sufficiently dense atmosphere. For systems with a sequential arrangement of stages, their separation often occurs under the rocket engine jet action. The occurrence of lateral forces in violation of axial symmetry can lead to overturning of the discarded stage and undesirable interaction with the outgoing stage. The goals of the paper were to study the pulsation characteristics of the incoming supersonic jet streamlining the body, as well as the pulsations of the bow shock wave in both axisymmetric and asymmetric flows. The recording frame rate in the experiments was up to 775 000 fps. Computer vision and various machine learning approaches were used to process a large number of shadowgraph images. Canny edge detection and Hough transform, as well as a convolutional neural network (CNN) based on YOLOv2 architecture, were used to automatically track the position of the bow shock relative to the body. Jet sound generation was studied - the amplitude dependence on the distance from the nozzle and the evolution in time. The position of the bow shock relative to the streamlined body and flow pulsations over time were analyzed. Bow shock pulsation spectra at zero angle of attack were compared to those at a small angle of attack (α = 2.5°–3°). The power spectral density was also estimated. The slope of the spectra for the zero angle of attack case was found to be horizontal and the spectra for the small angle of attack case had a slope close to −5/3. The strongest of the measured bow shock frequencies were found to be close to those of the jet mixing layer pulsations. The hardware signal spectrum was also measured and this signal can be considered as noise. The described approach makes it possible to obtain the physical characteristics of the flow at any point of interest. It was also shown that the automation of flow visualization experiments using computer vision techniques allows to increase the speed of data processing and obtaining new physical information, which may be important for engineers developing aircraft and spacecraft technologies. • Supersonic jet flow around a blunt body was studied experimentally and also numerically. • Computer vision tools were applied to supersonic flow investigation. • Spectra of jet generated acoustic waves were determined. • Time dependent bow shock positions were automatically measured by the neural network. • Amplitudes of bow shock pulsations and flow fluctuations depend on the angle of the body relative to the jet direction. [ABSTRACT FROM AUTHOR]

Thanks to its richness in extractable features, Hyperspectral images (HSI) find an accelerated use in medical, industrial, agricultural, and environmental fields. In this paper, we present a wavelet-based reduction technique that creates a Hypercube containing the most significant features extracted from the original HSI and representing a multi-dimensional array that is utilized for training a Convolutional Neural Network (CNN), which is designed here to classify different types of surfaces or materials. The performance of this approach is tested and proved using two distinct datasets. Then, we compare the same approach with the PCA, a widely used reduction method. The most important contribution of this paper is the implementation of an FPGA-based parallel accelerator to train the same suggested CNN in only 10% of the computational time compared to the classical CPU-based techniques. The Microblaze will be explained and exploited here to play the role of an embedded microprocessor. • A Wavelet-based technique gives a reduced Hypercube rich in pixels spectrum features. • A CNN is designed to classify different types of surfaces or materials. • An FPGA-based parallel design is created to reduce 90% of the CNN training time. • The Microblaze is used to play the role of an embedded microprocessor. [ABSTRACT FROM AUTHOR]

In this paper, the non-uniform illumination enhancement problem of underwater images under the artificial light sources conditions is investigated based on Convolution Neural Network (CNN). First, we propose a trainable end-to-end enhancer called NUIENet, for enhancing the non-uniform illumination of underwater images. The proposed model consists of correction network and fusion layers. The correction network adopts the encoder–decoder structure with skip connections to enhance the features of different channels in the HSV domain, and then these enhanced features are fused by the fusion layers to obtain the desired high-quality images. Second, we built an underwater images dataset using Generative Adversarial Network (GAN) and Gaussian Function. Finally, both qualitative and quantitative experimental results show that the proposed method can produce better performance compared to other state-of-the-art enhancement methods on both real-word and synthetic underwater dataset. • This paper proposes a non-uniform illumination enhancer CNN-based which uses the encoder–decoder structure with skip connections to enhance the underwater images with NUI to the desired high-quality images. • To boost underwater imaging processing, we construct a dataset of the underwater image with NUI based on the GAN and Gaussian function which contains NUI images and their corresponding high-quality reference image. • Compared with other state-of-the-art NUIE methods, the proposed network achieves a nature color correction and superior or equivalent visibility improvement. [Display omitted] [ABSTRACT FROM AUTHOR]

In recent years, many image denoising methods have been proposed based on convolutional neural networks (CNNs). While these methods have shown continuous performance improvement by introducing various mechanisms and structures, their computational cost tends to become increasingly expensive, owing to the resulting complex network architectures. This paper aims at winning the trade-off between computational efficiency and denoising performance for CNN-based image denoisers. Towards this end, we draw inspirations from traditional variational models with wavelet analysis operators for CNN architecture design. A model-inspired CNN is proposed with four key modules: iterative encoding-decoding units inspired by the iterative denoising process, directional convolutions inspired by the separable wavelet filters, inception modules inspired by the multi-scale analysis of wavelets, and stage-wise connections inspired by the adding noise back operation. In experiments, our CNN shows high computational efficiency in both training and test, with competitive results to state-of-the-art approaches. • A neural network inspired by the key ingredients of wavelet analysis model. • A sequential encoding-decoding network for image denoising. • A deep image denoiser well balancing effectiveness and computational efficiency. [ABSTRACT FROM AUTHOR]

• The deep learning model can better extract the deeper features of the multi-point deformation sequence. • MVMD algorithm reduces the complexity of multi-point displacement sequence. • The model considers the spatiotemporal characteristics of dam deformation, the prediction accuracy is obviously improved. It is difficult for single measuring point model to synchronously characterize the spatial correlation of deformation sequence of concrete dam and the similarity of displacements of different measuring points. Considering the limitation of conventional model in dealing with spatial-temporal data structure, In this study, a deep learning technique for multi-point and multi-output deformation prediction of concrete dams is proposed, which is based on Multivariate Variational Mode Decomposition (MVMD) and three-dimensional convolutional neural network (3D-CNN) combined model. Firstly, the multi-point sequence is decomposed into several relatively stable sub-sequences with different frequency scales by MVMD, which reduces the complexity of the multi-point displacement sequence. Then, In order to enhance the model 's ability to fit the time and spatial features of the sequence, the 3D-CNN model is used to predict the decomposed components and reconstruct the displacement data of the measuring points. This study focuses on the principle, process and calculation steps of the proposed model, and verifies the proposed model through the deformation data of three typical measuring points. The test results show that the prediction model proposed in this paper is an effective method for dam deformation prediction. [ABSTRACT FROM AUTHOR]

The concept of video motion magnification has become increasingly relevant due to its ability to detect small and invisible motions that can be of great value in a variety of applications. A variety of approaches have been developed to magnify these motions and variations. While both Eulerian and Lagrangian processing methods are widely used for motion magnification, Eulerian approaches are more commonly employed due to their lower computational cost. This paper provides an overview of the powerful Eulerian motion magnification techniques. We begin with a brief introduction to technical concepts associated with Eulerian motion techniques such as pyramids and filters in image processing. Additionally, we provide a comparison between the Lagrangian and Eulerian perspectives, followed by a comprehensive overview of the various Eulerian motion magnification (EVM) techniques available. Finally, we present implementation results and a comparative analysis of some of the Eulerian motion techniques. [Display omitted] • Detecting Imperceptible Motions: Explore motion magnification's role in revealing subtle movements. • Demystifying Eulerian Processing: Simplify complex concepts, mathematical foundations. • Real-World Applications: Illustrate motion magnification's utility in healthcare, construction, etc. • Eulerian Advantages: Compare and emphasize the strength of Eulerian over Lagrangian methods. • Comprehensive Survey: Cover a range of Eulerian motion magnification techniques. [ABSTRACT FROM AUTHOR]

The existing surface roughness theoretical predictive models exhibit high parameter sensitivity and complexity, resulting in low reliability. Moreover, machine learning-based predictive models are constrained by the training parameter range, limiting their precision and generalization. In response to these limitations and from the frequency domain perspective, this paper proposes an innovative surface roughness (Ra) predictive model for silicon nitride ceramics grinding based on the dynamic grinding force model. First, a dynamic grinding force model was established using single grain scratch test results from both ductile and brittle removal stages, with model parameter estimation assisted by Genetic Algorithm. The results demonstrated a Mean Relative Error (MRE) of 19.51% between experimental and theoretical dynamic grinding force values, and a Relative Error (RE) of 9.37% for average grinding force. Subsequently, the frequency domain characteristics of both experimental grinding force signals and surface topography texture curves were extracted, which exhibited a significant correlation based on cosine similarity and Kendall's tau. Finally, leveraging the frequency domain characteristics of grinding force, the surface roughness predictive model was developed through the Convolutional Neural Network. The predictive model achieved an MRE of 12.79%, with random grinding experiments further confirming the model's accuracy with an MRE of 13.65%. These findings indicate that the proposed surface roughness predictive model offers high generalization capability and accurate prediction across a broader parameter range. [ABSTRACT FROM AUTHOR]

Depth estimation is crucial for scene understanding and downstream tasks, especially the self-supervised training methods showing great potential. The overall structure and local details of the scene are essential for improving the quality of depth estimation. The proposal of Monodepth2 has led to significant progress in self-supervised monocular depth estimation. However, Monodepth2 uses the most basic encoder–decoder architecture. The limited data flow information of the network leads to a large semantic gap between the encoder and the decoder, which reduces the accuracy of the network for fine-grained feature recognition. Monodepth2 adopts Resnet18 pre-trained on the Imagenet dataset as the encoder. This traditional convolutional pooling structure results in a loss of pixel information in the network at every scale. In order to solve this problem, this paper proposes an improved DepthNet. The network adopts Hrnet in semantic segmentation as the base encoder, which adopts an advanced multi-scale fusion method in the whole process, thus avoiding the loss of pixel information. An additional densely connected U-Net is employed at the decoder side to provide more information flow. Furthermore, the semantic gap between the encoder and decoder is reduced by adding different numbers of residual connections and channel attention on each layer. The network structure can be regarded as a collection of fully convolutional networks. Since the deep features of the network have a higher correlation with the vertical position, we add a spatial location attention module to the deep-level network to reduce this semantic gap. The approach performs significantly well on the KITTI dataset benchmark, with several performance criteria comparable to supervised monocular depth inference methods. • This work is a deep estimation network for scene reconstruction and scene understanding. This network redesigns the self-supervised monocular depth framework from an entirely new perspective. The network uses HrNet from the field of semantic segmentation as the base encoder, which employs a progressive multi-scale fusion approach throughout, thus avoiding the loss of pixel information. • An additional densely connected U-Net is used at the decoder side to provide further information flow. To reduce the semantic gap between codecs, we add a different number of residual connections and channel attention on each layer. The network is not trained with the help of other auxiliary networks, and the performance of the depth estimation is improved only by stimulating the network's potential. • This work achieves best-in-class accuracy in monocular depth estimation. When the model in this paper is used for 3D scene reconstruction, it can perform a complete recovery of the scene structure. [ABSTRACT FROM AUTHOR]

• Summarizing various models of GAN. • Discussing the advantages and applications of GAN in medical image fusion field. • Analyzing the challenges faced by GAN and envisioning the future directions of GAN. Generative Adversarial Network (GAN) is a research hotspot in deep generative models, which has been widely used in the field of medical image fusion. This paper summarizes GAN models from the following four aspects: firstly, the basic principles of GAN are expounded from two aspects: basic model and training process; secondly, variant GAN models are summarized into three directions (Probability Distribution Distance, Overall Network Architecture, Neural Network Structure), from the methods based on f-divergence, the methods based on IPM, Single-Generator and Dual-Discriminators GAN, Multi-Generators and Single-Discriminator GAN, Multi-Generators and Multi-Discriminators GAN, Conditional Constraint GAN, Convolutional Neural Network structure GAN and Auto-Encoder Neural Network structure GAN are eight dimensions to summarize the typical models in recent years; thirdly, the advantages and application of GAN models in the field of medical image fusion are explored from three aspects; fourthly, the main challenges faced by GAN and the challenges faced by GAN models in medical image fusion field are discussed and the future prospects are given. This paper systematically summarizes various models of GAN, advantages and challenges of GAN models in medical image fusion field, which is very important for the future research of GAN. [Display omitted] [ABSTRACT FROM AUTHOR]

During the last few years, many deep learning (DL) methods have been proposed for the classification of hyperspectral images (HSI), and the final results show that these models have great potential. However, most existing deep models only consider the original spectral HSI datacube, excluding other sources of information such as textural and geometrical features. In addition to their computational complexity, most DL approaches cannot effectively consider local nonlinear relationships between features. In order to address these issues, this paper develops a new DL approach called MLKD-CNN, which uses multifeature local kernel descriptors (MLKDs) and convolutional neural networks (CNN) to classify HSIs. This method produces local kernel descriptors (LKDs) of spectral, extended morphological profiles, and Gabor features in the feature generation step. A simple CNN is then used to classify MLKD generated from spatial and spectral features. The proposed MLKD considers local nonlinear relationships of both spectral and spectral features, making them capable of achieving high accuracy levels while reducing the computational burden of deep methods. The experiments on four benchmark HSIs show that the proposed MLKD-CNN outperforms some recent HSI classification methods in terms of accuracy and computation time. [ABSTRACT FROM AUTHOR]

To capture the real-world luminance values, High Dynamic Range (HDR) image processing has been developed. HDR images have a richer content than the widely-used Standard Dynamic Range (SDR) images, and are used in a number of situations, e.g. in film industry. As HDR displays are more and more commercially available, we need to be able to process HDR images as well as SDR ones (for example, devising denoising algorithms, inpainting or anti-aliasing). The most powerful methods to process images are now deep neural networks. However, the training of such networks calls for a lot of images, and HDR images datasets are relatively small. One way to generate HDR images is inverse tone mapping operators (iTMOs). They are algorithms that expand the dynamic range of SDR images. In this paper, we propose HDR-LFNet, a novel iTMO, and its HDR training dataset. Our method merges several existing handcrafted iTMOs, combined in a supervised neural network to produce an HDR output. Our lightweight network requires less training images than state-of-the-art methods, and has faster training phase. Besides, the quality of the generated images is similar to the state-of-the-art. We present the architecture as well as the subjective and experimental evaluations of our method. [Display omitted] • We propose a new Inverse Tone Mapping Operator to create High Dynamic Range images. • Convolutional Neural Network are used to fuse together existing methods. • We performed a subjective study to evaluate our method. • The network's weights and training set are available online. [ABSTRACT FROM AUTHOR]

Fusing hyperspectral images (HSIs) and multispectral images (MSIs) is an economic and feasible way to obtain images with both high spectral resolution and spatial resolution. Due to the limited receptive field of convolution kernels, fusion methods based on convolutional neural networks (CNNs) fail to take advantage of the global relationship in a feature map. In this paper, to exploit the powerful capability of Transformer to extract global information from the whole feature map for fusion, we propose a novel Multiscale Spatial–spectral Transformer Network (MSST-Net). The proposed network is a two-branch network that integrates the self-attention mechanism of the Transformer to extract spectral features from HSI and spatial features from MSI, respectively. Before feature extraction, cross-modality concatenations are performed to achieve cross-modality information interaction between the two branches. Then, we propose a spectral Transformer (SpeT) to extract spectral features and introduce multiscale band/patch embeddings to obtain multiscale features through SpeTs and spatial Transformers (SpaTs). To further improve the network's performance and generalization, we proposed a self-supervised pre-training strategy, in which a masked bands autoencoder (MBAE) and a masked patches autoencoder (MPAE) are specially designed for self-supervised pre-training of the SpeTs and SpaTs. Extensive experiments on simulated and real datasets illustrate that the proposed network can achieve better performance when compared to other state-of-the-art fusion methods. The code of MSST-Net will be available at http://www.jiasen.tech/papers/ for the sake of reproducibility. [Display omitted] • A multiscale spatial–spectral Transformer network is proposed. • Spectral multi-head self-attention is designed to extract spectral features. • Multiscale band/patch embeddings are introduced to extract Multiscale features. • A self-supervised pre-training strategy is developed. [ABSTRACT FROM AUTHOR]

Identification of human actions from video has gathered much attention in past few years. Most of the computer vision tasks such as Health Care Activity Detection, Suspicious Activity detection, Human Computer Interactions etc. are based on the principle of activity detection. Automatic labelling of activity from videos frames is known as activity detection. With the introduction of deep networks, the process of activity detection is clustered into two groups known as hand-crafted feature based approach and automatic feature extraction approach. This paper focuses on various approaches used in recent literature based on traditional and automatic approach. Moreover, hierarchy for different approaches under them such as space based, motion based, genetic based, fuzzy based, dictionary based are discussed. With introduction of Convolutional Neural Networks and Recurrent Neural Networks, automatic learning capability from input modality makes them first choice to be implemented for activity recognition. In this paper various approaches have been analyzed according to methodology, accuracy, classifier and datasets. [ABSTRACT FROM AUTHOR]

• Manually creating BIM models is a high-cost task • Semantic segmentation can be used to automatically create BIM models from images • Convolutional Neural Networks are effective for segmentation but are not explainable • An explainable AI model allows a human user to interact with it • Fuzzy Rule-based systems can be used as explainable AI models Building Information Modelling (BIM) is a standard digital process that fuses buildings information from different sources into a 3D model during their lifecycle. For new construction sites using BIM, it is possible to monitor the cost, schedule, and changes throughout the lifecycle; however, existing buildings do not have a BIM model. Manually creating the BIM models for existing buildings is a high-cost task, both in time and money, hence there is a need for extracting information from available paper-based documentation and fuse it into a BIM model. The struggle of facility management and utility companies to fully adopt a BIM process (due to their high volumes of paper-based documentation of existing buildings) has led to the research on creating these 3D BIM models from 2D floor plan images. This paper presents a novel processing pipeline to extract 2D digital information from floorplans, fusing it into a 3D BIM model. The work focuses on fusing the available information to create the structure of the building in BIM format, which is considered the essential step before looking on working with other sources of data. In this process, we introduce a type-2 fuzzy logic based Explainable Artificial Intelligence (XAI) approach for the semantic segmentation step. The approach consists of using the outputs of type-2 fuzzy logic systems to classify a pixel as wall or background, by using information around and from the pixel of interest as the inputs to the system. After the semantic segmentation step, the output of the type-2 fuzzy logic goes through a noise removal process and finally a transformation from 2D to 3D by assigning the corresponding BIM tag to each identified element. The proposed type-2 fuzzy logic semantic segmentation approach produced comparable results (97.3% mean Intersection over Union (IoU) performance metric value) to the opaque box model approach based on Convolutional Neural Network (CNN) (99.3% mean IoU performance metric value). However, the type-2 fuzzy XAI system benefits from being an augmentable and interpretable model, which means that human users can understand the decision process and modify the model using their expert knowledge. [ABSTRACT FROM AUTHOR]

This paper delivers an exhaustive analysis of the fusion of multi-sensor technologies, including traditional sensors such as cameras, Light Detection and Ranging(LiDAR), Radio Detection and Ranging(RADAR), and ultrasonic sensors, with Artificial Intelligence(AI) powered methodologies in obstacle detection for Autonomous Vehicles(AVs). With the growing momentum in AVs adoption, a heightened need exists for versatile and resilient obstacle detection systems. Our research delves into study of literatures, where proposed approaches assimilate data from this diverse sensor suite, integrated through Deep Learning(DL) techniques, to refine AV performance. Recent advancements and prevailing challenges within the domain are thoroughly examined, with particular focus on the integration of sensor fusion techniques, the facilitation of real-time processing via edge and fog computing, and the implementation of advanced artificial intelligence architectures, including Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), and Generative Adversarial Networks(GANs), to enhance data interpretation efficacy. In conclusion, the paper underscores the critical contribution of multi-sensor arrays and deep learning in enhancing the safety and reliability of autonomous vehicles, offering significant perspectives for future research and technological progress. [ABSTRACT FROM AUTHOR]

In recent years, decomposition-based combination models have been widely used in wind power prediction. This type of method decomposes the highly volatile wind power into some relatively smooth subsequences, which reduces the difficulty of modeling. However, this might use information from future data in advance, creating the illusion of high prediction accuracy. Therefore, this paper proposes a wind power ultra-short-term prediction model based on fixed scale dual mode decomposition (FSDMD) and deep learning networks. First, the wind power series after fixed scale blocking is decomposed using ensemble empirical mode decomposition (EEMD), and use the improved variational mode decomposition (VMD) based on Spearman rank order correlation coefficient (SROCC) to decompose the obtained high-frequency components twice. Then, the appropriate mode components were selected by calculating the SROCC and experimental analysis, and combined with the convolutional neural network (CNN) and the bidirectional long short-term memory (BiLSTM) network to train the model. Finally, the historical data of wind turbines in a wind farm in Northwest China is used for example verification, and the comparison with other models in the two scenarios of sufficient and insufficient features. The results show that the proposed FSDMD–CNN–BiLSTM model has high prediction accuracy in both scenarios. Especially in the scenario of insufficient features, compared with CNN-BiLSTM model, RMSE, MAE and MAPE are reduced by 8.20,14.24 and 0.15, respectively. In addition, this paper verifies that mode decomposition can improve the performance of prediction model without using future features, which provides ideas for solving similar problems. [ABSTRACT FROM AUTHOR]

Accurate detection of pavement markings at the pixel level is crucial for enhancing traffic safety. The majority of current advanced deep-learning networks predominantly focus on localized features, neglecting the global context of pavement image. Such networks often result in discontinuous segmentation outcomes and suboptimal recovery of local details. In this paper, a robust model named C-Transformer is proposed to provide an effective solution to this challenge. The contributions of this paper primarily involve two aspects. Firstly, the proposed C-Transformer is designed to succinctly integrate convolution operations and self-attention, facilitating a comprehensive understanding of essential features. Secondly, an efficient Feed-Forward Network called Inverse Residual Feed-Forward Network is also proposed in this paper and deployed in C-Transformer to improve latent representations. Experimental results demonstrate that, compared to other state-of-the-art networks, the proposed C-Transformer achieves a performance enhancement of 0.93% in F-measure and a 1.64% improvement in Intersection-Over-Union. In particular, the robustness and effectiveness of the C-Transformer in accurate pavement marking detection are proved through field test results. This paper illustrates the feasibility of employing a hybrid Convolutional neural network-Transformer-based network for automatic robust pavement marking detection under noisy conditions. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

The paper investigates the application of machine learning (ML) for voltage sag source localization (VSSL) in electrical power systems. To overcome feature-selection challenges for traditional ML methods and provide more meaningful sequential features for deep learning methods, the paper proposes three time-sample-based feature forms, and evaluates an existing feature form. The effectiveness of these feature forms is assessed using k-means clustering with k = 2 referred to as downstream and upstream classes, according to the direction of voltage sag origins. Through extensive voltage sag simulations, including noises in a regional electrical power network, k-means identifies a sequence involving the multiplication of positive-sequence current magnitude with the sine of its angle as the most prominent feature form. The study develops further traditional ML methods such as decision trees (DT), support vector machine (SVM), random forest (RF), k-nearest neighbor (KNN), an ensemble learning (EL), and a designed one-dimensional convolutional neural network (1D-CNN). The results found that the combination of 1D-CNN or SVM with the most prominent feature achieved the highest accuracies of 99.37% and 99.13%, respectively, with acceptable/fast prediction times, enhancing VSSL. The exceptional performance of the CNN was also approved by field measurements in a real power network. However, selecting the best ML methods for deployment requires a trade-off between accuracy and real-time implementation requirements. The research findings benefit network operators, large factory owners, and renewable energy park producers. They enable preventive maintenance, reduce equipment downtime/damage in industry and electrical power systems, mitigate financial losses, and facilitate the assignment of power-quality penalties to responsible parties. • Comprehensive study on enhanced voltage sag source localization assisted by ML. • Proposing three new time sample-based feature forms, effective for ML methods. • Identifying the most influential feature form (form 4) utilizing k-means clustering. • Developing diverse supervised models, including a designed one-dimensional CNN. • CNN-Feature form 4 achieved 99.37% accuracy with an acceptable prediction speed. [ABSTRACT FROM AUTHOR]

Pipe wall-thinning is a phenomenon whereby the thickness of pipes in a nuclear power plant decreases over time owing to extended operational years. This thickness reduction is caused by various long-term thermal aging mechanisms such as Flow-Accelerated Corrosion, Liquid Droplet Impingement Erosion, cavitation, and flashing. Reducing the thickness of the secondary system pipes to the point of rupture can lead to severe human casualties and significant economic losses. Consequently, domestic power plant operators regularly manage the power plant pipes. Ongoing research focuses on the identification and management of pipe wall-thinning. However, previous studies have encountered problems in accurately judging the decrease in pipe wall-thinning in the presence of noise. To overcome this, Convolutional Neural Network (CNN) models based on image feature analysis have been used. This approach allows for the differentiation of pipe wall-thinning feature from small-sized noise in a single image. However, there were difficulties in making accurate judgments for large-sized noise that resembled the feature of pipe wall-thinning. This paper aims to analyze the limitations of the current pipe wall-thinning evaluation methods and to achieve accurate pipe wall-thinning discrimination through time-series analysis of continuous pipe wall-thinning data. The proposed method employs a Convolutional Recurrent Neural Network (CRNN) model, integrating the Recurrent Neural Network(RNN) model with the CNN model. The image feature of the pipes, extracted using CNN, are utilized as inputs for the RNN. This enables the observation of how the image features of the pipes change over time. This feature differentiates from the time-series feature of noise that occurs suddenly. Through this method, the paper proposes a new approach for effectively identifying the gradual decrease in pipe wall-thinning, enabling precise assessment of pipe wall-thinning progression. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

Roundwood sorting is still a manual process in many Swiss sawmills, requiring employees to visually inspect and categorize thousands of logs per day. The heavy workload can be both physically and mentally taxing and can lead to increased rates of human error. State-of-the-art automation systems like X-ray log scanners are expensive and difficult to integrate into existing process lines. This paper proposes a novel recommendation system that leverages recent advances in image classification to automate roundwood classification by quality and species. The system integrates a camera to capture cross-sectional images of logs and record numerical data, such as length, taper, and diameter. The analysis of the resulting dataset highlights the challenges of data imbalance and noise, which makes classification difficult and, in some cases, impossible. However, by using selected datasets with reduced noise, state-of-the-art Convolutional Neural Networks (CNNs) can extract quality and species features. Quality models learn from a manually selected and simplified dataset, featuring samples that experts can clearly classify based on the image's information. Species models are trained on a label-noise-reduced dataset, reflecting real-world complexity. The accuracy on the selected dataset for three quality classes is 80%. The species determination is less challenging and reaches 91% accuracy on a synchronized dataset for the main species spruce and fir. Overall, this paper highlights the potential of Machine Learning in augmenting the roundwood sorting processes and presents a novel system that can improve the efficiency and accuracy of the process. [Display omitted] • Automation of roundwood sorting: Replaces manual sorting with image-based AI. • Integrated camera system in roundwood sorting to collect labeled dataset. • Species prediction: 91% accuracy in spruce–fir distinction. • Quality prediction on complexity reduced dataset: 80% accuracy between three main quality levels. • Efficient, adaptable & scalable system which is easy to integrate into existing process lines. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

The paper investigates the application of machine learning (ML) for voltage sag source localization (VSSL) in electrical power systems. To overcome feature-selection challenges for traditional ML methods and provide more meaningful sequential features for deep learning methods, the paper proposes three time-sample-based feature forms, and evaluates an existing feature form. The effectiveness of these feature forms is assessed using k-means clustering with k = 2 referred to as downstream and upstream classes, according to the direction of voltage sag origins. Through extensive voltage sag simulations, including noises in a regional electrical power network, k-means identifies a sequence involving the multiplication of positive-sequence current magnitude with the sine of its angle as the most prominent feature form. The study develops further traditional ML methods such as decision trees (DT), support vector machine (SVM), random forest (RF), k-nearest neighbor (KNN), an ensemble learning (EL), and a designed one-dimensional convolutional neural network (1D-CNN). The results found that the combination of 1D-CNN or SVM with the most prominent feature achieved the highest accuracies of 99.37% and 99.13%, respectively, with acceptable/fast prediction times, enhancing VSSL. The exceptional performance of the CNN was also approved by field measurements in a real power network. However, selecting the best ML methods for deployment requires a trade-off between accuracy and real-time implementation requirements. The research findings benefit network operators, large factory owners, and renewable energy park producers. They enable preventive maintenance, reduce equipment downtime/damage in industry and electrical power systems, mitigate financial losses, and facilitate the assignment of power-quality penalties to responsible parties. • Comprehensive study on enhanced voltage sag source localization assisted by ML. • Proposing three new time sample-based feature forms, effective for ML methods. • Identifying the most influential feature form (form 4) utilizing k-means clustering. • Developing diverse supervised models, including a designed one-dimensional CNN. • CNN-Feature form 4 achieved 99.37% accuracy with an acceptable prediction speed. [ABSTRACT FROM AUTHOR]

Pipe wall-thinning is a phenomenon whereby the thickness of pipes in a nuclear power plant decreases over time owing to extended operational years. This thickness reduction is caused by various long-term thermal aging mechanisms such as Flow-Accelerated Corrosion, Liquid Droplet Impingement Erosion, cavitation, and flashing. Reducing the thickness of the secondary system pipes to the point of rupture can lead to severe human casualties and significant economic losses. Consequently, domestic power plant operators regularly manage the power plant pipes. Ongoing research focuses on the identification and management of pipe wall-thinning. However, previous studies have encountered problems in accurately judging the decrease in pipe wall-thinning in the presence of noise. To overcome this, Convolutional Neural Network (CNN) models based on image feature analysis have been used. This approach allows for the differentiation of pipe wall-thinning feature from small-sized noise in a single image. However, there were difficulties in making accurate judgments for large-sized noise that resembled the feature of pipe wall-thinning. This paper aims to analyze the limitations of the current pipe wall-thinning evaluation methods and to achieve accurate pipe wall-thinning discrimination through time-series analysis of continuous pipe wall-thinning data. The proposed method employs a Convolutional Recurrent Neural Network (CRNN) model, integrating the Recurrent Neural Network(RNN) model with the CNN model. The image feature of the pipes, extracted using CNN, are utilized as inputs for the RNN. This enables the observation of how the image features of the pipes change over time. This feature differentiates from the time-series feature of noise that occurs suddenly. Through this method, the paper proposes a new approach for effectively identifying the gradual decrease in pipe wall-thinning, enabling precise assessment of pipe wall-thinning progression. [ABSTRACT FROM AUTHOR]

• This paper chooses consecutive dissolved gases data (CDGD) as indexer for transformer diagnosis, which encapsulates more plentiful characteristics reflecting the real condition of transformer. • A novel structure called multichannel consecutive data cross-extraction (MCDC) is proposed, which aims to comprehensively harness the intrinsic characteristics of CDGD by successionally exploit the hidden token-wised emphasis and coherence in various dimensions of CDGD clued by information carried on other dimensions. • A 1-dimensional convolution neural network attention (1DCNN-attention) mechanism is introduced and integrated into MCDC to enhance the applicability in transformer diagnosis scenarios. • This paper verifies the superior generalization ability of deep learning methods. This sort of ability is of great significance when there are needs for transfer diagnostics across voltage levels. MCDC achieves the best generalization performance. Power transformer plays a critical role in grid infrastructure, and its diagnosis is paramount for maintaining stable operation. However, the current methods for transformer diagnosis focus on discrete dissolved gas analysis, neglecting deep feature extraction of multichannel consecutive data. The unutilized sequential data contains the significant temporal information reflecting the transformer condition. In light of this, the structure of multichannel consecutive data cross-extraction (MCDC) is proposed in this article in order to comprehensively exploit the intrinsic characteristic and evaluate the states of transformer. Moreover, for the better accommodation in scenario of transformer diagnosis, one dimensional convolution neural network attention (1DCNN-attention) mechanism is introduced and offers a more efficient solution given the simplified spatial complexity. Finally, the effectiveness of MCDC and the superior generalization ability, compared with other algorithms, are validated in experiments conducted on a dataset collected from real operation cases of power transformer. Additionally, the better stability of 1DCNN-attention has also been certified. [ABSTRACT FROM AUTHOR]

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F 1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach. • Collection of renal pathological images with annotations of glomerulus classes. • A flattened Xception with feature pyramid network (FX-FPN) for glomerulus detection. • Generative glomerulus morphology augmentation to improve classification performance. • A two-stage detection and classification framework with performance evaluation. [ABSTRACT FROM AUTHOR]

Intrusion Detection System (IDS) can ensure the network security by identifying network intrusions according to the abnormal traffic data. However, the intrusion detection data has the problem of high dimensionality and changes with network and attack environments, which leads to the poor performance and poor portability of intrusion detection algorithms. Therefore, this paper proposes an intrusion detection algorithm based on joint symmetric uncertainty and hyperparameter optimized fusion neural network. Firstly, a feature selection method based on symmetric uncertainty and approximate Markov blanket is proposed, which fully considers the correlation and redundancy of features, and also the correlation between combined features and the class label, so as to reduce the data dimensionality. Secondly, the CNN-LSTM classifier fused with Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) is used to extract the spatial features and temporal features to improve the classification performance. Finally, the Particle Swarm Optimization (PSO) algorithm is improved and used to automatically optimize the hyperparameters of the classifier, so that the classifier can be applied to different intrusion detection datasets with better generalization ability and portability. Experiments have verified the effectiveness and superiority of the proposed algorithm on multiple evaluation indicators. • An effective algorithm for intrusion detection is proposed in this paper. • Feature selection is based on symmetric uncertainty and approximate Markov blanket. • A fusion neural network is constructed to extract the spatial and temporal features. • The PSO algorithm is improved to automatically optimize the hyperparameters. [ABSTRACT FROM AUTHOR]

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders' representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

In recent studies, deep learning methodologies have shown significant promise in crack detection. However, their practical implementation faces challenges due to the intricate diversity of structural surfaces and the inherent narrowness of cracks. To mitigate these problems, this paper introduces SegFormer, an efficient semantic segmentation model with hierarchical Transformer, for crack detection on concrete and asphalt surfaces in multiple scenarios. The combination of Cross-Entropy (CE) and Dice loss functions is employed to enhance the detection of fine cracks. Additionally, the paper presents an evaluation framework and discusses metrics for assessing crack segmentation results to provide a more precise and comprehensive analysis of model performance. Experimental results indicate that SegFormer outperforms Convolutional Neural Networks (CNNs) such as FCN, U-Net, and DeepLabV3 utilizing different backbones. Notably, the integration of multiple loss functions contributes to a more stable training process, expedites convergence, and yields enhanced results compared to models utilizing individual loss functions. • SegFormer is introduced for fine crack detection in various scenarios. • The combination of CE and Dice loss functions enhances crack detection. • A comprehensive model evaluation framework is presented. • Metrics for fine crack segmentation assessment are discussed. [ABSTRACT FROM AUTHOR]

Sensor techniques and emerging CNN models have greatly facilitated the development of collaborative fault diagnosis. Existing CNN models apply different fusion schemes to achieve reliable fault identification based on multisensor data. Few CNN models, however, take into account both the intrinsic correlations and the distribution gap between different signals, which may result in a limited exploration of multisource data. To address this issue, a novel convolutional fusion framework called a collaborative fusion convolutional neural network (CFCNN) is developed in this paper. More specifically, a multiscale shrinkage denoising module (MSDM) is developed first to extract multilevel modality-specific features from different mechanical signals. Then, drawing inspiration from the intermediate fusion scheme, a central fusion module (CFM) is introduced to explore the intrinsic correlations and integrate cross-modal features. Moreover, an online label smoothing training (OLST) strategy is applied to reduce overfitting and promote better classification performance of CFCNN. The developed CFCNN is expected to shed new light on collaborative fault diagnosis using the intermediate fusion scheme. The efficacy of the developed CFCNN is verified through the cylindrical rolling bearing dataset and the planetary gearbox dataset. • A novel convolutional fusion framework named CFCNN is developed for collaborative fault recognition. • A central fusion module (CFM) is put forward for effective feature fusion. • An online label smoothing (OLS) algorithm is introduced to reduce overfitting and facilitate model convergence. • Two case studies are conducted to verify the effectiveness of the proposed diagnostic framework. [ABSTRACT FROM AUTHOR]

This paper introduces an adaptive Convolutional Neural Network (CNN)-based Unscented Kalman Filter for the pose estimation of uncooperative spacecraft. The validation is carried out at Stanford's robotic Testbed for Rendezvous and Optical Navigation on the Satellite Hardware-In-the-loop Rendezvous Trajectories (SHIRT) dataset, which simulates vision-based rendezvous trajectories of a servicer spacecraft to PRISMA's Tango spacecraft. The proposed navigation system is stress-tested on synthetic as well as realistic lab imagery by simulating space-like illumination conditions on-ground. The validation is performed at different levels of the navigation system by first training and testing the adopted CNN on SPEED+, Stanford's spacecraft pose estimation dataset with specific emphasis on domain shift between a synthetic domain and an Hardware-In-the-Loop domain. A novel data augmentation scheme based on light randomization is proposed to improve the CNN robustness under adverse viewing conditions, reaching centimeter-level and 10 degree-level pose errors in 80% of the SPEED+ lab images. Next, the entire navigation system is tested on the SHIRT dataset. Results indicate that the inclusion of a new scheme to adaptively scale the heatmaps-based measurement error covariance based on filter innovations improves filter robustness by returning centimeter-level position errors and moderate attitude accuracies, suggesting that a proper representation of the measurements uncertainty combined with an adaptive measurement error covariance is key in improving the navigation robustness. [ABSTRACT FROM AUTHOR]

Recently, super-resolution (SR) techniques based on deep learning have taken more and more attention, aiming to improve the images and videos resolutions. Most of the SR methods are related to other fields of computer vision such as image classification, image segmentation, and object detection. Based on the success of the image SR task, many image SR surveys are introduced to summarize the recent work in the image SR domains. However, there is no survey to summarize the SR models for the lightweight image SR domain. In this paper, we present a comprehensive survey of the state-of-the-art lightweight SR models based on deep learning. The SR techniques are grouped into six major categories: include convolution, residual, dense, distillation, attention, and extremely lightweight based models. Also, we cover some other issues related to the SR task, such as benchmark datasets and metrics for performance evaluation. Finally, we discuss some future directions and open problems, that may help other community researchers in the future. [Display omitted] • Lightweight image super-resolution based on deep learning models are reviewed. • The network design, loss function, framework, and training datasets are considered. • The network design structure of the reviewed models are categorized into six types. • The future research directions of lightweight image super-resolution are indicated. [ABSTRACT FROM AUTHOR]

The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN and DRINet, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to 36% , and achieves higher accuracy than the voxel-based model with up to 34 × speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to 8 × speedups. Also, our MPVConv only needs 65% of the GPU memory required by the latest point-voxel-based model (DRINet). The source code of our method is attached in https://github.com/NWUzhouwei/MPVConv. [Display omitted] • In this paper, we propose Multi Point-Voxel Convolution (MPVConv) which takes the advantages of both the voxel- and point-based methods, and can work with different backbones for a wide range of 3D tasks. • Our MPVConv increases not only the neighboring collection for point-based features, but also the independence among voxel-based features. • 1×1×1 3D CNN is adopted in fusion module to strengthen the information association and nonlinearity between point-based and voxel-based features. • We replace the tri-linear interpolation with nearest-neighbor interpolation to reduce the computation cost of voxel feature interpolation. [ABSTRACT FROM AUTHOR]

Forming a UAV cluster with multiple UAVs to collaborate in accomplishing tasks is the future development of battlefield. The UAV cluster can collaborate by communicating through inter-aircraft links and can quickly and accurately perform complex tasks such as path planning, collaborative reconnaissance, collaborative sensing and attack. In the process of Unmanned Aerial Vehicle (UAV) cooperative operation, obtaining the real-time attitude information of each UAV is preliminary to implement collaborative cooperation. However, in practice, due to the internal malfunction of sensors and mutual interference among UAVs, the acquired attitudes often suffer from data missing. This paper proposed a novel missing data reconstruction method for UAV's attitude in the case of sensor failures. The attitude data of UAV is obtained through an advanced UAV simulation platform AirSim. Through fusing the temporal convolutional network (TCN) and Bi-directional long short-term memory (Bi-LSTM), a deep regression framework is established for missing data construction. In addition, we evaluate the reliability of the proposed method by comparing with different baseline models and different combinations of data missing. The performance of our method is quantified using metrics of root-mean-square-error (RMSE), mean absolute error (MAE), determination coefficient (DC), mean absolute percentage error (MAPE), and symmetric mean absolute percentage error (SMAPE) score. The averaged performance values of our proposed method outperform other baseline algorithms. The results of the experiment show that the study could well-address the issue of missing data reconstruction and provide reliable attitude data. • A deep learning framework for UAV's missing data reconstruction is proposed. • TCN and Bi-LSTM is fused to reconstruct the missing data of UAV's attitude. • The performance for missing data reconstruction under different sensor failures is evaluated. [ABSTRACT FROM AUTHOR]

In recent years, deep learning (DL) methods have proven their efficiency for various computer vision (CV) tasks such as image classification, natural language processing, and object detection. However, training a DL model is expensive in terms of both complexities of the network structure and the amount of labeled data needed. In addition, the imbalance among available labeled data for different classes of interest may also adversely affect the model accuracy. This paper addresses these issues using a new convolutional neural network (CNN) based architecture. The proposed network incorporates both spatial and spectral information that combines two sub-networks: spatial-CNN and spectral-CNN. The spectral-CNN extracts spectral information, while spatial-CNN captures spatial information. Moreover, to make the features more robust, a multiscale spatial CNN architecture is introduced using different kernels. The final feature vector is formed by concatenating the outputs obtained from both spatial-CNN and spectral-CNN. To address the data imbalance problem, a generative adversarial network (GAN) was used to generate data for the underrepresented class. Finally, relatively a shallower network architecture was used to reduce the number of parameters in the network and improve the processing speed. The proposed model was trained and tested on Senitel-2 images for the classification of the debris-covered glacier. The results showed that the proposed method is well-suited for mapping and monitoring debris-covered glaciers at a large scale with high classification accuracy. In addition, we compared the proposed method with conventional machine learning approaches, support vector machine (SVM), random forest (RF) and multilayer perceptron (MLP). [ABSTRACT FROM AUTHOR]

When imaging at a long distance by ordinary visible cameras, the wavelength of visible light is easily interfered by fog or atmospheric effects, resulting in blurry or lost details in RGB images. However, NIR cameras are robust to the long distance imaging without color. In this paper, we propose long-range imaging using multispectral fusion of RGB and NIR images, called LRINet. We adopt unsupervised learning for the fusion to solve the absence of ground truth. LRINet is an end-to-end network based on convolutional neural networks (CNNs) that consists of three steps for the multispectral fusion: feature extraction, fusion and reconstruction. To align unpaired data and treat the discrepancy between RGB and NIR images, we construct WarpingNet to warp NIR features, and add it into the feature extraction. Since dark channel prior (DCP) provides distance from the camera by light transmission degree, we combine it with structure loss as the weight for fusion. To ensure the color fidelity, LRINet operates the fusion on the input of the RGB luma channel and NIR image. Experimental results show that LRINet produces natural-looking color images with clear details by successfully treating discrepancy between RGB and NIR images and outperforms state-of-art fusion models in term of both visual quality and quantitative measurements. • We propose long range imaging using multispectral fusion of RGB and NIR images, called LRINet. • LRINet consists of three steps for image fusion: feature extraction, fusion, and reconstruction. • We introduce WarpingNet into the feature extraction step to align the unpaired RGB and NIR images. • We combine DCP with structure loss to achieve long range imaging while keeping the original tone. • LRINet successfully treats discrepancy between two images and generates natural looking fusion images. [ABSTRACT FROM AUTHOR]

As the need for sensor fusion systems has grown, developing methods to find correspondences between images with different spectral ranges has become increasingly important. Since most images do not share low-level information, such as textures and edges, existing matching approaches fail even with convolutional neural networks (CNNs). In this paper, we propose an end-to-end metric learning method, called SPIMNet (SPectral-Invariant Matching Network) for robust cross- and multi-spectral image patch matching. While existing methods based on CNNs learn matching features directly from cross- and multi-spectral image patches, SPIMNet transforms across spectral bands and discriminates for similarity in three steps. First, (1) SPIMNet is adjusted for a feature domain by introducing a domain translation network; then (2) two Siamese networks learn to match the adjusted features with the same spectral domain; and (3) the matching features are fed to fully-connected layers to determine the identity of the patches as a classification task. By effectively incorporating each step, SPIMNet achieved competitive results on a variety of challenging datasets, including both VIS–NIR and VIS–Thermal image pairs. Our code is available at https://github.com/koyeongmin/SPIMNet. [Display omitted] • The method is based on domain conversion for matching between VIS and NIR images. • The method uses metric learning to integrate information from the two domains. • This work studies the effect of the domain conversion method and loss function. • This work achieves noteworthy performance in three public datasets. • The result can be adapted to various applications such as depth estimation. [ABSTRACT FROM AUTHOR]

Brain tumor segmentation in multimodal MRI has great significance in clinical diagnosis and treatment. The utilization of multimodal information plays a crucial role in brain tumor segmentation. However, most existing methods focus on the extraction and selection of deep semantic features, while ignoring some features with specific meaning and importance to the segmentation problem. In this paper, we propose a brain tumor segmentation method based on the fusion of deep semantics and edge information in multimodal MRI, aiming to achieve a more sufficient utilization of multimodal information for accurate segmentation. The proposed method mainly consists of a semantic segmentation module, an edge detection module and a feature fusion module. In the semantic segmentation module, the Swin Transformer is adopted to extract semantic features and a shifted patch tokenization strategy is introduced for better training. The edge detection module is designed based on convolutional neural networks (CNNs) and an edge spatial attention block (ESAB) is presented for feature enhancement. The feature fusion module aims to fuse the extracted semantic and edge features, and we design a multi-feature inference block (MFIB) based on graph convolution to perform feature reasoning and information dissemination for effective feature fusion. The proposed method is validated on the popular BraTS benchmarks. The experimental results verify that the proposed method outperforms a number of state-of-the-art brain tumor segmentation methods. The source code of the proposed method is available at https://github.com/HXY-99/brats. • Proposes a brain tumor segmentation method by fusing semantic and edge features. • Presents a Swin Transformer-based semantic segmentation module with an SPD strategy. • Presents a CNN-based edge detection module and an edge spatial attention block. • Presents a graph convolution-based multi-feature inference block for feature fusion. • Achieves very promising results on the popular BraTS benchmarks. [ABSTRACT FROM AUTHOR]

• Proposing a method to calculate the adjacency matrix of the relational graph of aspect words and a method to calculate the adjacency matrix of a syntactic dependency graph. • Proposesing a multiple ASGCN model architectures, and designing an experiment to compare the sentiment classification performance of these architectures. • Experimental results show that the proposed model can percept the semantic relationships. Aspect-level sentiment analysis aims to identify the sentiment polarity of aspect words in sentences. The existing research methods only focus on the grammatical dependencies between words, ignoring the semantic connections between aspect words and the dependency types between words, which limits the performance of the aspect-level sentiment analysis model. Therefore, this paper proposes an aspect-sentence Graph Convolutional Networks model (ASGCN) to perceive more comprehensive semantic information. Specifically, the model consists of sentence-focused GCN (SentenceGCN) and aspects-focused GCN (AspectsGCN) sub models. In the SentenceGCN model, this paper proposes a method to calculate the adjacency matrix (A s) of syntactic dependency graph, which uses the position encoding mechanism and pays attention to the influence of different dependency types on semantics, so that SentenceGCN can capture the semantic information of the whole sentence more comprehensively. In the AspectsGCN model, this paper also proposes a method to calculate the adjacency matrix (A a) of aspect words, which models the relational graph as fully connected and gives weight to the edges between aspect words according to the position, so that the AspectsGCN can pay attention to the semantic relation between different aspect words in the sentence. The proposed model outperforms all baseline models with 86.34 % accuracy and 79.96 F1 score, which indicates that there are more advantages in perceiving semantic information in ASGCN. [ABSTRACT FROM AUTHOR]

Groundwater level (GWL) prediction is important for ecological protection and resource utilization; it helps in formulating policies for artificial groundwater recharge, modifying the number of extraction wells, etc., and can support sustainable human development as well as inform water resource management decisions. However, climate change, anthropogenic impacts, and the complex coupling between surface water and groundwater increase the difficulty of predicting groundwater levels. The model proposed in this paper combines external data as well as multiple models. The method leverages long and short-term memory (LSTM) and convolutional neural network (CNN) models, combined with secondary modal decomposition and slime mould algorithm (SMA), together with an adaptive weight module (AWM). The study applies this method to predict GWL for three different hydrological conditions in China, specifically for the Jinan Baotu Spring, Heihu Spring, and Zhongtianshe watershed of Taihu Lake. A comparison of metrics such as mean absolute error and Nash efficiency coefficient for single and hybrid models shows that the model in this paper is more advantageous than the single model and other hybrid models. The interpretability of the model is enhanced by SHAP values that demonstrate the degree of contribution of the input variables. This paper uses SHAP analyses to identify the key drivers affecting groundwater levels. These factors must be detected in order to develop groundwater resource protection measures. [Display omitted] • Multivariate fusion data including hydrology and meteorology are used as model input. • A secondary modal decomposition module for historical groundwater level data was utilized. • The neural network hyperparameters are optimized using the slime mould algorithm. • Aggregate subsets of prediction with adaptive modules rather than linear summation. • The interpretable SHAP model measures the degree of influence of external variables. [ABSTRACT FROM AUTHOR]

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new deformable convolutional network with attention mechanism to deal with the Chinese character of various dynasties. These ancient Chinese characters are hieroglyph with special spatial structure and gradually evolving into modern Chinese characters. However, because of the intrinsic limitations of traditional convolutional networks in the model, the geometric structure of the convolutional module is fixed. Therefore, we build a new architecture called attention-based deformable networks for this task. Besides, we construct a new dataset called the Chinese characters from Various Dynasties Dataset (CCDD), which includes the evolution of Chinese characters between major dynasties. Deformable convolution based on attention mechanism can obtain those more important offsets, resulting in better performance than the original deformable convolution. In the experimental stage, extensive experiments validate the performance of our approach. Firstly, the effectiveness of this network was verified on two text datasets, Mnist(99.38%) and notMnist(98.22%) both comparing with several modules. Finally, the best performance was achieved on the CCDD dataset(91.50%) compared to other classical deep learning networks. This is the first paper to apply deep learning methods to recognize and classify the characters of major dynasties in China. And archaeological workers need to be able to recognize Chinese characters from various periods in China, and our algorithm can assist them in completing this task to a certain extent. It can also help cultural relic enthusiasts roughly determine the age of cultural relics. [ABSTRACT FROM AUTHOR]