Search

Aiming at the high false negative rate and low accuracy in crowded scenes, a pedestrian detection network based on multi-scale features and mutual supervision is proposed. To effectively extract pedestrian feature information in complex scenes, a network combining PANet pyramid network and mixed dilated convolutions is used to extract feature information. Then, a mutual supervision detection network for head-body detection is designed, which utilizes the mutual supervision of head bounding boxes and full-body bounding boxes to obtain more accurate pedestrian detection results. The proposed network achieves 13.5% MR-2 performance on CrowdHuman dataset, with an improvement of 3.6% compared to the YOLOv5 network, and a simultaneous improvement of 3.5% in average precision (AP). On CityPersons dataset, it achieves 48.2% MR-2 performance, with 2.3% improvement compared to the YOLOv5 network, and a simultaneous improvement of 2.8% in AP. The results indicate that the proposed network demonstrates good detection performance in densely crowded scenes. [ABSTRACT FROM AUTHOR]

The depth image obtained by consumer-level depth cameras generally has low resolution and missing regions due to the limitations of the depth camera hardware and the method of depth image generation. Despite the fact that many studies have been done on RGB image completion and super-resolution, a key issue with depth images is that there will be evident jagged boundaries and a significant loss of geometric information. To address these issues, we introduce a multi-scale progressive fusion network for depth image completion and super-resolution in this paper, which has an asymptotic structure for integrating hierarchical features in different domains. We employ two separate branches to learn the features of a multi-scale image given a depth image and its corresponding RGB image. The extracted features are then fused into different level features of these two branches using a step-by-step strategy to recreate the final depth image. To confine distinct borders and geometric features, a multi-dimension loss is also designed. Extensive depth completion and super-resolution studies reveal that our proposed method outperforms state-of-the-art methods both qualitatively and quantitatively. The proposed methods are also applied to two human–robot interaction applications, including a remote-controlled robot based on an unmanned ground vehicle (UGV), AR-based toolpath planning, and automatic toolpath extraction. All these experimental results indicate the effectiveness and potential benefits of the proposed methods. [ABSTRACT FROM AUTHOR]

Automatic imaging monitoring technology has been developed as a mature solution to address oil spill accidents. However, when airborne equipment is used to monitor the area of leakage, there is a chance of relative motion between the equipment and the target, leading to blurred images. This can hinder subsequent monitoring tasks, such as the identification and segmentation of hazardous chemical targets. Therefore, it is crucial to deblur these images, as sharper images can substantially improve the precision and accuracy of these tasks. In light of this, this paper presents a novel multi-scale recurrent deblurring network, leveraging the Convolutional Block Attention Module (CBAM) attention mechanism, to restore clear images in an end-to-end manner. The CBAM attention mechanism has been incorporated, which learns features in both the spatial and channel domains, thereby enhancing the deblurring ability of the network by combining the attention mechanisms of multiple domains. To evaluate the effectiveness of the proposed method, we applied the deblurring network to our self-built dataset of blurred ultraviolet (UV) images of colorless chemicals floating on water surfaces. Compared to the SRN-Deblur deblurring network, the proposed approach yields improved PSNR and SSIM results. Moreover, when compared with the state-of-the-art DeblurGANv2 method, it performs better in the SSIM. These results demonstrate that the proposed multi-scale recurrent deblurring network, based on the CBAM attention mechanism, exhibits a superior deblurring performance. [ABSTRACT FROM AUTHOR]

Abstract Bearings are key components of mechanical equipment, and fault diagnosis is a necessary and important measure to ensure bearing safety. Driven by industrial big data and deep learning (DL), intelligent fault diagnosis (IFD) has made great progress in recent years. However, most of the existing methods mainly focus on the fault diagnosis of individual bearings, and the feature extraction and fault classification rely on traditional networks and expert experience, which cannot meet the diagnostic requirements of cross-bearing conditions. To fill this research gap, this paper proposes a multi-scale attention-based transfer model (MSATM). First, the collected vibration signals are converted into time–frequency maps as samples, and the proposed MSATM employs multi-scale residual learning and attention mechanism to adaptively extract sensitive fault features, and recognizes faults of new bearings by deep transfer learning using the trained MSATM. A large number of experimental results based on a bearing benchmark validate the effectiveness and superiority of the proposed method and provide a promising tool for cross-bearing fault diagnosis.

Despite significant progress has been made in image and video deblurring, much less attention has been paid to process ultra high-definition (UHD) videos (e.g., 4K resolution). In this work, we propose a novel deep model for fast and accurate UHD video deblurring (UHDVD). The proposed UHDVD is achieved by a depth-wise separable-patch architecture, which operates with a multi-scale integration scheme to achieve a large receptive field without adding the number of generic convolutional layers and kernels. Additionally, we adopt the temporal feature attention module to effectively exploit the temporal correlation between video frames to obtain clearer recovered images. We design an asymmetrical encoder–decoder architecture with residual channel-spatial attention blocks to improve accuracy and reduce the depth of the network appropriately. Consequently, the proposed UHDVD achieves real-time performance on 4K videos at 30 fps. To train the proposed model, we build a new dataset comprised of 4K blurry videos and corresponding sharp frames using three different smartphones. Extensive experimental results show that our network performs favorably against the state-of-the-art methods on the proposed 4K dataset and existing 720p and 2K benchmarks in terms of accuracy, speed, and model size. [ABSTRACT FROM AUTHOR]

Individual recognition of fish within aquaculture fish groups is crucial for monitoring the growth status of each fish and is a key technology for precision and green aquaculture. However, accurately recognizing fish faces challenges due to the significant variations in fish postures and the influence of complex environments. This article aims to present a highly accurate method for underwater fish individual recognition, offering technical support for intelligent fish farming. To achieve this goal, a novel lightweight underwater fish individual recognition method based on local feature guidance training was proposed for precision aquaculture in this study. The method designs a multi-scale network based on deep learning and utilizes SE_ResNext for feature extraction. This network consists of a branch representing the global feature and a branch representing the local feature. During training, the full image of the fish individual is inputted as a coarse scale to the global branch (G-branch) to learn the global feature and calculate the ID loss. The image of each part formed after segmenting the fish individual is inputted as a fine-scale input to the partial branch (P-branch) to learn the local feature. Subsequently, the global feature and local feature are combined to form the fusion feature, and the Triplet loss and ID loss are calculated on the fusion feature. Finally, the loss calculated in the global feature and the loss calculated in the fusion feature are weighted and summed to obtain the total loss. Through the design of the loss function, the global feature is consistently influenced by the local feature, and the local feature serves to guide the global feature. During prediction, the global feature, having learned the information from the local feature, allows for the sole utilization of the G-branch to recognize fish individuals. This approach significantly reduces the number of parameters and computational requirements, resulting in a lightweight network. The method achieves high accuracy, with both Rank-1 and Rank-5 accuracy rates surpassing 95%. Remarkably, even with low image resolution and slight obscuration of the fish body, the Rank-1 accuracy remains above 90%. [ABSTRACT FROM AUTHOR]

The widespread availability of digitized fine art collections in museums and galleries has generated a demand for efficient software tools. These tools enable rapid retrieval and semantic classification of art images. Traditional image classification methods often rooted in shallow structure learning algorithms offer the capacity to extract various image features. However, there is need for deep understanding of fundamental painting knowledge during this process due to the reason that certain features may be lost. In this paper, we present an innovating approach for classifying digital paintings based on artist attribution. Our approach centers on the creations of multi-scale pyramid representation derived from a given painting image which facilitate the incorporation of both global and local information within a single image. The training of Convolutional Neural Network (CNN) algorithm is performed for the assignment of class labels on each level of the pyramid. To establish connections among localized image patches, we use Markov Random Fields through the optimization of the Gibbs energy function. We validate the proposed approach which referred to as the multi-scale CNN framework using fusion-based Markov random field using two challenging painting image datasets: WikiArt and Web Gallery of Art (WGA). The proposed model archives better results using WikiArt dataset for painting image classification, i.e., precision (74.12%), recall (70.23%), F1-score (72.65%), and accuracy (80.00%). Using Web Gallery of Art (WGA) dataset for painting image classification, the model achieves the following results: precision (75.88%), recall (77.25%), F1-score (76.44%), and accuracy (78.00%). Our proposed method outperforms state-of-the-art methods excelling not only in image classification but also in terms of computational efficiency. [ABSTRACT FROM AUTHOR]

In this paper, the recognition of fidgety speech emotion is studied, and real-world speech emotions are collected to enhance emotion recognition in practical scenarios, especially for cognitive tasks. We first focused on eliciting fidgety emotions and data acquisition for general math learning. Students practice mathematics by performing operations, solving problems, and orally responding to questions, all of which are recorded as audio data. Subsequently, the teacher evaluates the accuracy of these mathematical exercises by scoring, which reflects the cognitive outcomes of the students. Secondly, we propose an end-to-end speech emotion model based on a multi-scale one-dimensional (1-D) residual convolutional neural network. Finally, we conducted an experiment to recognize fidgety speech emotions by testing various classifiers, including SVM, LSTM, 1-D CNN, and the proposed multi-scale 1-D CNN. The experimental results show that the classifier we constructed can identify fidgety emotion well. After conducting a thorough analysis of fidgety emotions and their influence on the learning process, a clear relationship between the two was apparent. The automatic recognition of fidgety emotions is valuable for assisting on-line math teaching. [ABSTRACT FROM AUTHOR]

Cross-modal audio–visual correlation learning has been an interesting research topic, which aims to capture and understand semantic correspondences between audio and video. We face two challenges during audio–visual correlation learning: (i) audio and visual feature sequences, respectively, belong to different feature spaces, and (ii) semantic mismatch between audio and visual sequences inevitably happens. To solve these challenges, existing works mainly focus on how to efficiently extract discriminative features, while ignoring the abundant granular features of audio and visual modalities. In this work, we introduce the multi-scale network with shared cross-attention (MSNSCA) module for audio–visual correlation learning, a supervised representation learning framework for capturing semantic audio–visual correspondences by integrating a multi-scale feature extraction module with strong cross-attention into an end-to-end trainable deep network. MSNSCA can extract more effective audio–visual particle features with excellent audio–visual semantic matching capability. Experiments on various audio–visual learning tasks, including audio–visual matching and retrieval on benchmark datasets, demonstrate the effectiveness of the proposed MSNSCA model. [ABSTRACT FROM AUTHOR]

Automatic imaging monitoring technology has been developed as a mature solution to address oil spill accidents. However, when airborne equipment is used to monitor the area of leakage, there is a chance of relative motion between the equipment and the target, leading to blurred images. This can hinder subsequent monitoring tasks, such as the identification and segmentation of hazardous chemical targets. Therefore, it is crucial to deblur these images, as sharper images can substantially improve the precision and accuracy of these tasks. In light of this, this paper presents a novel multi-scale recurrent deblurring network, leveraging the Convolutional Block Attention Module (CBAM) attention mechanism, to restore clear images in an end-to-end manner. The CBAM attention mechanism has been incorporated, which learns features in both the spatial and channel domains, thereby enhancing the deblurring ability of the network by combining the attention mechanisms of multiple domains. To evaluate the effectiveness of the proposed method, we applied the deblurring network to our self-built dataset of blurred ultraviolet (UV) images of colorless chemicals floating on water surfaces. Compared to the SRN-Deblur deblurring network, the proposed approach yields improved PSNR and SSIM results. Moreover, when compared with the state-of-the-art DeblurGANv2 method, it performs better in the SSIM. These results demonstrate that the proposed multi-scale recurrent deblurring network, based on the CBAM attention mechanism, exhibits a superior deblurring performance.

Rib segmentation based on chest X-ray images is essential in the computer-aided diagnosis systems of lung cancer, which serves as an important step in the quantitative analysis of various types of lung diseases. However, the traditional methods are unable to segment ribs effectively due to the unclear edges and overlapping regions in X-ray images. A novel rib segmentation framework based on Unpaired Sample Augmentation and Multi-Scale Network is presented in this paper, aiming to improve the accuracy of ribs segmentation with limited labeled samples. First, the algorithm learns pneumonia-related texture changes via unpaired chest x-ray images and generates various augmented samples. Then, a multi-scale network attempts to learn hierarchical features using global supervision. Finally, the refined segmentation result of each organ is achieved by using a deep separation module and a comprehensive loss function. Specifically, the hierarchical features can greatly improve the robustness of multi-organ segmentation networks. The complex multi-organ segmentation task with limited labeled data is simplified with the designed deep separation module. We justify the proposed framework through extensive experiments. It achieves good performance with DSC, Precision, Recall, and Jaccard of 88.03, 88.25, 88.36, and 79.02%, respectively. The DSC value increases nearly by 3% compared to other popular methods. The experimental results show that our algorithm presents better segmentation performance for the overlapping region and fuzzy region of multiple organs, which holds research value and prospects for application. [ABSTRACT FROM AUTHOR]

Super-resolution is a widely used technology in many applications, such as video repair. Aiming at the insufficiency of the Fast Super-Resolution Convolutional Neural Networks (FSRCNN) method, an image super-resolution reconstruction method based on multi-scale joint network is proposed. Firstly, based on multi-scale structures, a feature sampling model is proposed to extract the features of Low-Resolution (LR) image. Secondly, the features are enhanced by feature fusion and sub-pixel convolutional layer. Finally, a joint loss function involving Mean Square Error (MSE) loss and Peak Signal to Noise Ratio (PSNR) loss is proposed to improve the optimization of the networks training. Comparison experiments were carried out on the sets of Set5, Set14, and BSD100. The experimental results show that the method has superiority against the state-of-the-art methods. Finally, the proposed method is applied to increase the resolutions of the television dramas "Journey to the West" and "The Dream of Red Mansion", which achieves good visual effect. [ABSTRACT FROM AUTHOR]

The segmentation of pepper leaves from pepper images is of great significance for the accurate control of pepper leaf diseases. To address the issue, we propose a bidirectional attention fusion network combing the convolution neural network (CNN) and Swin Transformer, called BAF-Net, to segment the pepper leaf image. Specially, BAF-Net first uses a multi-scale fusion feature (MSFF) branch to extract the long-range dependencies by constructing the cascaded Swin Transformer-based and CNN-based block, which is based on the U-shape architecture. Then, it uses a full-scale feature fusion (FSFF) branch to enhance the boundary information and attain the detailed information. Finally, an adaptive bidirectional attention module is designed to bridge the relation of the MSFF and FSFF features. The results on four pepper leaf datasets demonstrated that our model obtains F1 scores of 96.75%, 91.10%, 97.34% and 94.42%, and IoU of 95.68%, 86.76%, 96.12% and 91.44%, respectively. Compared to the state-of-the-art models, the proposed model achieves better segmentation performance. [ABSTRACT FROM AUTHOR]

Predicting recurrence in patients with non-small cell lung cancer (NSCLC) before treatment is vital for guiding personalized medicine. Deep learning techniques have revolutionized the application of cancer informatics, including lung cancer time-to-event prediction. Most existing convolutional neural network (CNN) models are based on a single two-dimensional (2D) computational tomography (CT) image or three-dimensional (3D) CT volume. However, studies have shown that using multi-scale input and fusing multiple networks provide promising performance. This study proposes a deep learning-based ensemble network for recurrence prediction using a dataset of 530 patients with NSCLC. This network assembles 2D CNN models of various input slices, scales, and convolutional kernels, using a deep learning-based feature fusion model as an ensemble strategy. The proposed framework is uniquely designed to benefit from (i) multiple 2D in-plane slices to provide more information than a single central slice, (ii) multi-scale networks and multi-kernel networks to capture the local and peritumoral features, (iii) ensemble design to integrate features from various inputs and model architectures for final prediction. The ensemble of five 2D-CNN models, three slices, and two multi-kernel networks, using 5 × 5 and 6 × 6 convolutional kernels, achieved the best performance with an accuracy of 69.62%, area under the curve (AUC) of 72.5%, F1 score of 70.12%, and recall of 70.81%. Furthermore, the proposed method achieved competitive results compared with the 2D and 3D-CNN models for cancer outcome prediction in the benchmark studies. Our model is also a potential adjuvant treatment tool for identifying NSCLC patients with a high risk of recurrence.

This paper investigates the issue of crowd counting for crowd images. A novel method named two-task convolutional neural network (T 2 CNN) is proposed to simultaneously learn two tasks for crowd counting: the tasks of dense degree classification and density map estimation. Basically, T 2 CNN contains two main modules: dense degree classification (DDC) and density map estimation. Generally, it is sufficient for crowd counting to estimate density maps of images. It is known that images may have different dense degrees and the local regions in the same image have different dense degrees. Thus, dense degree may provide the high-level prior of images and is excepted to help the estimation of density maps. To better estimate density maps of images, T 2 CNN incorporates the DDC module into the task of density map estimation. T 2 CNN takes the scale-adaptive convolutional neural network as the estimator of density maps, which adopts multi-scale layers to tackle the scale and perspective variations that exist in crowd images commonly. To verify the effectiveness and robustness of our model, we conduct extensive experiments on the Shanghaitech, UCF_CC_50 and WorldExpo'10 datasets. Experimental results indicate that T 2 CNN achieves good performance. [ABSTRACT FROM AUTHOR]

• An efficient algorithm, MSAC-SOA is proposed for theft detecting for a real-word scenario. • The proposed approach integrates with the ESSD facilitates precise identification and localization of objects within each frame. • An automatic and immediate message system integration to the control room for detection of theft. In the current age of technological progress, surveillance cameras are extensively placed in densely populated areas to encourage public safety. However, detecting stealing activities within surveillance footage remains a challenge. The global issue of stealing leads to significant casualties and financial losses annually. There has been limited research on efficient techniques to identify this anomaly. In this manuscript, an efficient multi-scale attention-based convolutional neural network with skill optimization algorithm (MSAC-SOA) is proposed for detecting theft during day and night in surveillance videos. This approach integrates with the enhanced single-shot multibook detector (ESSD) for object detection in video footage, facilitating the precise identification and localization of objects within each frame. Subsequently, the proposed MSAC-SOA effectively detects and classifies instances of stealing. Finally, a theft detection alert message is sent to the control room. This approach is assessed with two datasets: University of Central Florida (UCF) crime dataset for daytime theft detection and real-time video data for nighttime theft detection. The proposed method demonstrates faster computation times compared to previous techniques, highlighting its efficiency and suitability in terms of various merits for real-life videos of diverse scenarios. It exhibits robust performance across various metrics, including 99.88% accuracy, 99.66% precision, 100% recall, 99.89% F1-score, 99.83% area under the ROC curve (AUC), and 99.83% specificity. [ABSTRACT FROM AUTHOR]

Distributed acoustic sensing (DAS) is regarded as a novel acquisition technology for seismic data. Compared with conventional electrical geophones, DAS has a series of obvious advantages including low-cost, high spatial resolution, good coverage, and strong resistance to the harsh environment. Noise attenuation is an essential step in seismic data processing. However, there are two main difficulties faced by the denoising task of DAS seismic data. On the one hand, some background noise in DAS seismic data, such as optical low-frequency noise, horizontal noise, and fading noise, is unique and not presented in the conventional seismic data; on the other hand, the signal-to-noise ratio (SNR) of DAS seismic data is relatively low. Recently, a convolutional neural network (CNN) has shown superior denoising performance compared to the traditional method. To follow this promising trend, we propose a multi-scale interactive convolutional neural network (MSI-Net) and apply it to denoise the challenging DAS seismic data. Different from most of the existing CNN architecture used in seismic data denoising, the MSI-Net considers both coarse-scale and fine-scale features by improving the inherent serial convolution to multi-scale parallel convolution, which is beneficial to recover detailed information. Moreover, we utilize some connections to achieve the information interaction between different scales, which promotes the flow of information and enables the network to extract more informative multi-scale features from the DAS seismic data. Moreover, both synthetic and real examples demonstrate that the proposed MSI-Net can effectively attenuate a variety of unique DAS background noise and also completely recover the weak signals. Compared with conventional CNN architecture, MSI-Net exhibits better performance in global SNR and local details.

In this paper, we propose an efficient algorithm to directly restore a clear image from a hazy input, which can be adapted for nighttime image dehazing. The proposed algorithm hinges on a trainable neural network realized in an encoder–decoder architecture. The encoder is exploited to capture the context of the derived input images, while the decoder is employed to estimate the contribution of each input to the final dehazed result using the learned representations attributed to the encoder. The constructed network adopts a novel fusion-based strategy which derives three inputs from an original input by applying white balance (WB), contrast enhancing (CE), and gamma correction (GC). We compute pixel-wise confidence maps based on the appearance differences between these different inputs to blend the information of the derived inputs and preserve the regions with pleasant visibility. The final clear image is generated by gating the important features of the derived inputs. To train the network, we introduce a multi-scale approach to avoid the halo artifacts. Extensive experimental results on both synthetic and real-world images demonstrate that the proposed algorithm performs favorably against the state-of-the-art dehazing for nighttime images. [ABSTRACT FROM AUTHOR]

The detection technology of pulmonary nodules based on the deep learning method is constantly developing, improving the detection rate and diagnostic accuracy of pulmonary nodules, assisting doctors in the task of pulmonary nodules examination. A deep learning method based on region proposal network(RPN) structure was proposed to detect pulmonary nodules, as well as an improved faster region-based convolutional neural network(Faster R-CNN) was investigated. The performance of multiple classification networks was compared for the false-positive phase of pulmonary nodules. The squeeze-and-excitation (SE) structure and ResNeXt residual block were used to form the feature extraction module. Combined with the UNet + + network structure, the multi-scale results were output. Finally, the multi-scale results were applied to the 3D RPN candidate detection network and R-CNN network, then the candidate nodule detection network with higher sensitivity and lower false positive rate was obtained. In the stage of the false-positive nodules task, a 3D deep convolutional neural network(3D DCNN) was used to remove false positive pulmonary nodules for candidate pulmonary nodules, effectively removing some false positive pulmonary nodules and improving the sensitivity of multiple FPs/ scan checkpoints. The results show that the sensitivity of the network was 98.8% (8 FP/ scan), and the competition performance metric (CPM) was 0.879 In terms of the removal of false-positive nodules, the 3D DCNN network could achieve the best effect in several networks, reaching a false-positive removal rate of 15.6%. It is concluded that the candidate nodule network further improved the detection sensitivity, and the network model achieved an outstanding detection effect. In false-positive network removal, as a false positive network, 3D DCNN had a better effect than some other network models. [ABSTRACT FROM AUTHOR]

This letter aims to combine spatial and frequency information for single image deblurring. Although some methods have tried to use frequency information to perform deblurring, they only simply process the different frequencies information separately or concatenate the real part and imaginary part of frequency features but ignore the strong correlation between them. To address this problem, we propose a simple but effective frequency interaction pipeline to realize the mutual conversion of the real part and the imaginary part. Then, we construct a spatial-frequency conversion module (SFCM) to promote the mutual conversion between the frequency information and the spatial information. Based on the proposed components, we build a multi-scale deblurring network, dubbed SFDNet, which can fully exploit coarse and middle-level information in spatial and frequency domains for finer scale image deblurring. Extensive experiments on the GoPro and HIDE datasets demonstrate that the proposed network outperforms the state-of-the-art methods both quantitatively and visually. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Jilin University (Science Edition) / Jilin Daxue Xuebao (Lixue Ban) is the property of Zhongguo Xue shu qi Kan (Guang Pan Ban) Dian zi Za zhi She and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The existing pedestrian detection suffers the low accuracy when the environment changes dramatically. In order to solve the problem, a pedestrian detection model combining information fusion and multi-scale detection is proposed. The model is composed of a retinex algorithm and an improved YOLOv3 algorithm. Retinex algorithm is selected as the preprocessing algorithm to improve the brightness and contrast of pedestrians. The model improves the YOLOv3 algorithm by adding multiple scale detections. The K-means is used to determine the number of optimal anchors and the aspect ratio. By testing on the standard data set, the mean average precision (mAP) of the joint detection model increases from the original 80.69–91.07%, and the recall increases from 65.22 to 87.48%. The comparative experiments show that the improved model performs good robustness and generalization ability on the problem of low pedestrian detection accuracy in complex environments. [ABSTRACT FROM AUTHOR]

Raindrops adhered to a glass window or camera lens appear in various blurring degrees and resolutions due to the difference in the degrees of raindrops aggregation. The removal of raindrops from a rainy image remains a challenging task because of the density and diversity of raindrops. The abundant location and blur level information are strong prior guide to the task of raindrop removal. However, existing methods use a binary mask to locate and estimate the raindrop with the value 1 (adhesion of raindrops) and 0 (no adhesion), which ignores the diversity of raindrops. Meanwhile, it is noticed that different scale versions of a rainy image have similar raindrop patterns, which makes it possible to employ such complementary information to represent raindrops. In this work, we first propose a soft mask with the value in [−1,1] indicating the blurring level of the raindrops on the background, and explore the positive effect of the blur degree attribute of raindrops on the task of raindrop removal. Secondly, we explore the multi-scale fusion representation for raindrops based on the deep features of the input multi-scale images. The framework is termed uncertainty guided multi-scale attention network (UMAN). Specifically, we construct a multi-scale pyramid structure and introduce an iterative mechanism to extract blur-level information about raindrops to guide the removal of raindrops at different scales. We further introduce the attention mechanism to fuse the input image with the blur-level information, which will highlight raindrop information and reduce the effects of redundant noise. Our proposed method is extensively evaluated on several benchmark datasets and obtains convincing results. [ABSTRACT FROM AUTHOR]

Forest fires have been a major threat to forest ecosystems and its biodiversity, as well as the environment in general, particularly in the Mediterranean regions. To mitigate fire spreading, this study aims at finding a fire-break solution for territories prone to fire occurrence. To the effect, here follows a model to map and predict phase transitions in fire regimes (spanning fires vs. penetrating fires) based on terrain morphology. The structure consists of a 2-scale network using site percolation and SIR epidemiology rules in a cellular automata to model local fire Dynamics. The target area for the application is the region of Serra de Ossa in Portugal, due to its wildfire incidence. The study considers the cases for a Moore neighbourhood of warm cells of radius 1 and 2 and also considers a heterogeneous terrain with 3 classes of vegetation. Phase transitions are found for different combinations of fire risk for each of these classes and use these values to parametrize the resulting landscape network.

Given a video sequence, video frame interpolation aims to synthesize an in-between frame of two consecutive frames. In this paper, we propose a multi-scale position feature transform (MS-PFT) network for video frame interpolation where two parallel prediction networks and one optimization network are designed to predict the features of target frame and generate the final interpolation result, respectively. To increase the fidelity of the synthesised frames, we propose to apply a position feature transform (PFT) layer in the residual blocks of the prediction networks to estimate scaling factors which help evaluate different degrees of the importance of deep features around a target pixel. A PFT layer utilizes optical flow to extract and generate position features and then adjusts the learning process of our model. We further extend our model into a multi-scale structure in which each scale of the network shares the same parameters to maximise the efficiency of our network with model size unchanged. The experiments show that our method can handle the challenging scenarios like occlusion and large motion effectively and the proposed method outperforms those state-of-the-art approaches on different datasets. [ABSTRACT FROM AUTHOR]

Recent deep learning-based methods for saliency detection have proved the effectiveness of integrating features with different scales. They usually design various complex architectures of network, e.g., multiple networks, to explore the multi-scale information of images, which is expensive in computation and memory. Feature maps produced with different subsampling convolutional layers have different spatial resolutions; therefore, they can be used as the multi-scale features to reduce the costs. In this paper, by exploiting the in-network feature hierarchy of convolutional networks, we propose a novel multi-scale network for saliency detection (MSNSD) consisting of three modules, i.e., bottom-up feature extraction, top-down feature connection and multi-scale saliency prediction. Moreover, to further boost the performance of MSNSD, an input image-aware saliency aggregation method is proposed based on the ridge regression, which combines MSNSD with some well-performed handcrafted shallow models. Extensive experiments on several benchmarks show that the proposed MSNSD outperforms the state-of-the-art saliency methods with less computational and memory complexity. Meanwhile, our aggregation method for saliency detection is effective and efficient to combine deep and shallow models and make them complementary to each other. [ABSTRACT FROM AUTHOR]

As a new type of remote sensing sensor, unmanned aerial vehicle (UAV) has been used in various fields such as medical treatment, transportation, environmental monitoring, disaster warning, animal protection and military increasingly. Since UAV images are acquired from multiple flying altitudes, perspectives with high speed, objects in UAV images have various scales and perspectives with different distributions, which brings a series of problems to object detection in UAV images.To address these problems, we propose an object detection method based on multi‐scale dilated convolutional neural network. It improves existing detection methods by a creative multi‐scale dilated convolutional module which facilitates the whole network to learn deep features with increased field of view perception and further improves the performance of object detection in UAV images.We adopt three comparative experiments on base network and our proposed method. And experimental results show that our proposed network has a high precision and recall for object detection in UAV images. Moreover, objects are detected with high performance in multiple perspectives, various scales and complex backgrounds, which indicates the effectiveness and robustness of our method. Object detection in UAV image is significant in both civil and military fields. However, existing methods are limited with objects in multiple perspectives, scales and backgrounds.Our proposed method improves the performance of existing networks by dilated convolutional operator. Experimental results demonstrate the effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

In order to acquire a high resolution multispectral (HRMS) image with the same spectral resolution as multispectral (MS) image and the same spatial resolution as panchromatic (PAN) image, pansharpening, a typical and hot image fusion topic, has been well researched. Various pansharpening methods that are based on convolutional neural networks (CNN) with different architectures have been introduced by prior works. However, different scale information of the source images is not considered by these methods, which may lead to the loss of high-frequency details in the fused image. This paper proposes a pansharpening method of MS images via multi-scale deep residual network (MSDRN). The proposed method constructs a multi-level network to make better use of the scale information of the source images. Moreover, residual learning is introduced into the network to further improve the ability of feature extraction and simplify the learning process. A series of experiments are conducted on the QuickBird and GeoEye-1 datasets. Experimental results demonstrate that the MSDRN achieves a superior or competitive fusion performance to the state-of-the-art methods in both visual evaluation and quantitative evaluation.

Abstract Based on the multi-fractal properties of financial time series, Value-at-Risk (VaR) and price fluctuation correlation, we construct China's stock market networks and analyze empirically three networks' topological features, their stability and their relationship with international stock market indices. The results show that: (1) In the three types of networks, the stock price network does not have scale-free features, multi-scale network and risk network both have small-world and scale-free characteristics; (2) The stock market volatility and the network stability coefficient are Granger causality, the early changes of the stock market volatility can effectively explain the changes of the network stability coefficient, the network connectivity and clustering coefficient are negatively correlated with the stock market volatility, the volatility of the international stock index in Hong Kong, Japan, and the United States has a positive effect on the network stability coefficient, and the international gold or crude oil markets has a negative effect on it; (3) By applying the probit binary selection model test, we found that the network structure is more capable of explaining abnormal fluctuations than international market factors and that the stock network with higher network stability coefficient and higher eigenvector centrality of financial institutions indicates that downward fluctuation of stock price will intensify in the future. From the empirical results, we see that the risk network is more robust and provides a reference for the analysis of the short-term risks and stability of the Chinese stock market. In order to maintain the stability of the stock market, the regulatory authorities should pay a close attention to internal and external factors, increase network connectivity and integration, and actively guard against the risk spillover effects of financial institutions. Highlights • The price network, multi-scale network and risk network are built. • Risk network is shown to be more robust. • Granger causality exists between stock market volatility and network stability. • Relationship between international stock market volatility and network stability is investigated. • Probit model is applied to forecast abnormal fluctuations of China's stock market network. [ABSTRACT FROM AUTHOR]

Predicting recurrence in patients with non-small cell lung cancer (NSCLC) before treatment is vital for guiding personalized medicine. Deep learning techniques have revolutionized the application of cancer informatics, including lung cancer time-to-event prediction. Most existing convolutional neural network (CNN) models are based on a single two-dimensional (2D) computational tomography (CT) image or three-dimensional (3D) CT volume. However, studies have shown that using multi-scale input and fusing multiple networks provide promising performance. This study proposes a deep learning-based ensemble network for recurrence prediction using a dataset of 530 patients with NSCLC. This network assembles 2D CNN models of various input slices, scales, and convolutional kernels, using a deep learning-based feature fusion model as an ensemble strategy. The proposed framework is uniquely designed to benefit from (i) multiple 2D in-plane slices to provide more information than a single central slice, (ii) multi-scale networks and multi-kernel networks to capture the local and peritumoral features, (iii) ensemble design to integrate features from various inputs and model architectures for final prediction. The ensemble of five 2D-CNN models, three slices, and two multi-kernel networks, using 5 × 5 and 6 × 6 convolutional kernels, achieved the best performance with an accuracy of 69.62%, area under the curve (AUC) of 72.5%, F1 score of 70.12%, and recall of 70.81%. Furthermore, the proposed method achieved competitive results compared with the 2D and 3D-CNN models for cancer outcome prediction in the benchmark studies. Our model is also a potential adjuvant treatment tool for identifying NSCLC patients with a high risk of recurrence., Competing Interests: The authors declare there are no competing interests., (©2023 Kim et al.)

The segmentation of pepper leaves from pepper images is of great significance for the accurate control of pepper leaf diseases. To address the issue, we propose a bidirectional attention fusion network combing the convolution neural network (CNN) and Swin Transformer, called BAF-Net, to segment the pepper leaf image. Specially, BAF-Net first uses a multi-scale fusion feature (MSFF) branch to extract the long-range dependencies by constructing the cascaded Swin Transformer-based and CNN-based block, which is based on the U-shape architecture. Then, it uses a full-scale feature fusion (FSFF) branch to enhance the boundary information and attain the detailed information. Finally, an adaptive bidirectional attention module is designed to bridge the relation of the MSFF and FSFF features. The results on four pepper leaf datasets demonstrated that our model obtains F1 scores of 96.75%, 91.10%, 97.34% and 94.42%, and IoU of 95.68%, 86.76%, 96.12% and 91.44%, respectively. Compared to the state-of-the-art models, the proposed model achieves better segmentation performance. The code will be available at the website: https://github.com/fangchj2002/BAF-Net., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Fang, Jiang, Zhang, Sun, Hu, Liu, Gong, Liu and Fu.)

Vessel segmentation in fundus images is a key procedure in the diagnosis of ophthalmic diseases, which can play a role in assisting doctors in diagnosis. Although current deep learning-based methods can achieve high accuracy in segmenting fundus vessel images, the results are not satisfactory in segmenting microscopic vessels that are close to the background region. The reason for this problem is that thin blood vessels contain very little information, with the convolution operation of each layer in the deep network, this part of the information will be randomly lost. To improve the segmentation ability of the small blood vessel region, a multi-input network (MINet) was proposed to segment vascular regions more accurately. We designed a multi-input fusion module (MIF) in the encoder, which is proposed to acquire multi-scale features in the encoder stage while preserving the microvessel feature information. In addition, to further aggregate multi-scale information from adjacent regions, a multi-scale atrous spatial pyramid (MASP) module is proposed. This module can enhance the extraction of vascular information without reducing the resolution loss. In order to better recover segmentation results with details, we designed a refinement module, which acts on the last layer of the network output to refine the results of the last layer of the network to get more accurate segmentation results. We use the HRF, CHASE_DB1 public dataset to validate the fundus vessel segmentation performance of the MINet model. Also, we merged these two public datasets with our collected Ultra-widefield fundus image (UWF) data as one dataset to test the generalization ability of the model. Experimental results show that MINet achieves an F 1 score of 0.8324 on the microvessel segmentation task, achieving a high accuracy compared to the current mainstream models., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

• We propose a conceptually novel yet simple framework termed Multi-label Learning guided self-paced Clustering, to address the unsupervised person Re-ID problem. • Our framework provides better unsupervised discriminative features with three crucial modules, namely a multi-scale network which obtains global and local person representations, a multi-label learning module which trains the network with memory bank and multi-label classification loss, and a self-paced clustering module which removes noisy samples and assigns pseudo labels for training. • Extensive experiments on three challenging large-scale datasets demonstrated the effectiveness of all the modules. Our framework finally achieves state-of-the-art performance on these datasets. Although unsupervised person re-identification (Re-ID) has drawn increasing research attention recently, it remains challenging to learn discriminative features without annotations across disjoint camera views. In this paper, we address the unsupervised person Re-ID with a conceptually novel yet simple framework, termed as Multi-label Learning guided self-paced Clustering (MLC). MLC mainly learns discriminative features with three crucial modules, namely a multi-scale network, a multi-label learning module, and a self-paced clustering module. Specifically, the multi-scale network generates multi-granularity person features in both global and local views. The multi-label learning module leverages a memory feature bank and assigns each image with a multi-label vector based on the similarities between the image and feature bank. After multi-label training for several epochs, the self-paced clustering joins in training and assigns a pseudo label for each image. The benefits of our MLC come from three aspects: i) the multi-scale person features for better similarity measurement, ii) the multi-label assignment based on the whole dataset ensures that every image can be trained, and iii) the self-paced clustering removes some noisy samples for better feature learning. Extensive experiments on three popular large-scale Re-ID benchmarks demonstrate that our MLC outperforms previous state-of-the-art methods and significantly improves the performance of unsupervised person Re-ID. [ABSTRACT FROM AUTHOR]

Forest fires have been a major threat to forest ecosystems and its biodiversity, as well as the environment in general, particularly in the Mediterranean regions. To mitigate fire spreading, this study aims at finding a fire-break solution for territories prone to fire occurrence. To the effect, here follows a model to map and predict phase transitions in fire regimes (spanning fires vs. penetrating fires) based on terrain morphology. The structure consists of a 2-scale network using site percolation and SIR epidemiology rules in a cellular automata to model local fire Dynamics. The target area for the application is the region of Serra de Ossa in Portugal, due to its wildfire incidence. The study considers the cases for a Moore neighbourhood of warm cells of radius 1 and 2 and also considers a heterogeneous terrain with 3 classes of vegetation. Phase transitions are found for different combinations of fire risk for each of these classes and use these values to parametrize the resulting landscape network. [ABSTRACT FROM AUTHOR]

• Dynamic Topological Analysis on spatial patterns of multifocal lesions. • A new K-simplex Filtration for the construction of dynamic community network. • A novel Decomposed Persistence for tracking evolution of dynamic communities. • Spatial lesion pattern demonstrates predictive value towards diagnosis and prognosis. • Our framework provides visual interpretability for trustworthy clinical assistance. [Display omitted] Quantitatively analysing the spatial patterns of multifocal lesions on clinical MRI is an important step towards a better understanding of the disease and for precision medicine, which is yet to be properly explored by feature engineering and deep learning methods. Network science addresses this issue by explicitly modeling the inter-lesion topology. However, the construction of the informative graph with optimal edge sparsity and quantification of community graph structures are the current challenges in network science. In this paper, we address these challenges with a novel Dynamic Topology Analysis framework on the basis of persistent homology, aiming to investigate the predictive values of global geometry and local clusters of multifocal lesions. Firstly, Dynamic Hierarchical Network is proposed to construct informative global and community-level topology over multi-scale networks from sparse to dense. Multi-scale global topology is constructed with a nested sequence of Rips complexes, from which a new K-simplex Filtration is designed to generate a higher-level topological abstraction for community identification based on the connectivity of k-simplices in the Rips Complex. Secondly, to quantify multi-scale community structures, we design a new Decomposed Community Persistence algorithm to track the dynamic evolution of communities, and then summarise the evolutionary communities incorporated with a customisable descriptor. The quantified community features are encapsulated with global geometric invariants for topological pattern analysis. The proposed framework was evaluated on both diagnostic differentiation and prognostic prediction for multiple sclerosis that is a typical multifocal disease, and achieved ROC_AUC 0.875 and 0.767, respectively, outperforming seven state-of-the-art persistent homology methods and the reported performance of six feature engineering and deep learning methods. [ABSTRACT FROM AUTHOR]

In order to acquire a high resolution multispectral (HRMS) image with the same spectral resolution as multispectral (MS) image and the same spatial resolution as panchromatic (PAN) image, pansharpening, a typical and hot image fusion topic, has been well researched. Various pansharpening methods that are based on convolutional neural networks (CNN) with different architectures have been introduced by prior works. However, different scale information of the source images is not considered by these methods, which may lead to the loss of high-frequency details in the fused image. This paper proposes a pansharpening method of MS images via multi-scale deep residual network (MSDRN). The proposed method constructs a multi-level network to make better use of the scale information of the source images. Moreover, residual learning is introduced into the network to further improve the ability of feature extraction and simplify the learning process. A series of experiments are conducted on the QuickBird and GeoEye-1 datasets. Experimental results demonstrate that the MSDRN achieves a superior or competitive fusion performance to the state-of-the-art methods in both visual evaluation and quantitative evaluation. [ABSTRACT FROM AUTHOR]