Search

Due to the large intra-class differences between the same categories and the scale imbalance between different categories in the remote sensing image dataset, the semantic segmentation task presents the problem of small-scale object information loss, the imbalance between foreground and background, and simultaneously the background dominates, which seriously affects the performance of the network model. To solve the above problems, this paper proposes an efficient bilateral branch depth neural network model based on the U-Net depth neural network, named BBU-Net. Firstly, one branch of the network learns the distribution characteristics of the original data, and the other focuses on difficult samples. Then the two branches improve the representation and classification ability of the neural network by accumulating learning strategies. Finally, considering the geometric diversity of remote sensing images, this paper adopts test time augmentation and reflection padding strategies and proposes a balanced weighted loss function named CombineLoss to alleviate the imbalance in the training process. The depth neural network proposed in this paper was first tested on the Inria Aerial Image Labeling Dataset, and 87.53% of mean intersection over union and 97.4% of mean pixel accuracy were obtained, respectively. At the same time, to verify the model's complexity, the model proposed in this paper is compared with the neural network based on integrated learning. The comparison results show that the spatial complexity of the network proposed in this paper is much lower than the neural network obtained by integrated learning, and the parameters are also much smaller than the neural network based on integrated learning. Then use the satellite building dataset I in the WHU Building Dataset and mainstream semantic segmentation methods for multiple groups of comparative experiments. The experimental results show that the method proposed in this paper can effectively extract the semantic information of remote sensing images, significantly improve the imbalance of remote sensing image data, improve the performance of the network model, and achieve a good semantic segmentation effect, which fully proves the effectiveness of this method. [ABSTRACT FROM AUTHOR]

Logistics parcel detection technology is critical for unmanned sorting. YOLOv5, the state-of-the-art (SOTA) object detection model, is a classic network widely used in engineering. However, on the premise of fast and accurate detection of logistics parcels, it confronts certain challenges of high computing load and parameter quantity. To address these issues, this paper proposes a two-scale lightweight deep learning model named SFYOLOv5. In this article, a lightweight feature extraction module called Pruned-Shuffle-Block (PSB) is proposed. Meanwhile, a double-layer pyramid structure for the medium and large target detection is created in accordance with the image size distribution of logistics parcels. With this structure, the Floating Point of Operations (FLOPs) and parameters in feature extraction were significantly reduced. In addition, a down sampling module named Focus for Downsampling (FFD) is designed and attention modules are introduced to extract high-level semantic information in logistics parcels. These modules not only compensate for the loss caused by down sampling but also improve the mean Average Precision (mAP). Finally, the comparison experiment is performed by using the self-built logistics parcel dataset. The results show that the mAP of the model reaches 99.1%, the number of model parameters decreased by 92.14%, and the FLOPs decreased by 89.57% compared with the existing SOTA model. This model can be used in logistics parcel intelligent sorting. [ABSTRACT FROM AUTHOR]

Scene classification and recognition have always been one of the most challenging tasks of scene understanding due to the inherent ambiguity in visual scenes. The core of scene classification and recognition tasks is scene representation. Deep learning advances in computer vision, especially deep CNNs, have significantly improved scene representation in the last decade. Deep convolutional features extracted from deep CNNs provide discriminative representations of the images and are widely used in various computer vision tasks, such as scene classification. Deep convolutional features capture the appearance characteristics of the image and the spatial information about different image regions. Meanwhile, the semantic and context information obtained from high-level concepts about scene images, such as objects and their relationships, can significantly contribute to identifying scene images. Therefore, in this paper, we divide visual scenes into two categories, object-based and layout-based. Object-based scenes are scenes that have scene-specific objects and, based on those objects, can be described and identified. In contrast, the layout-based scenes do not have scene-specific objects and are described and identified based on the appearance and layout of the image. This paper proposes a new neural network model for representing and classifying visual scenes, which we call G-CNN (GNN-CNN). The proposed model includes two modules, feature extraction and feature fusion, and the feature extraction module composes of visual and semantic branches. The visual branch is responsible for extracting deep CNN features from the image, and the semantic branch is responsible for extracting semantic GNN features from the scene graph corresponding to the image. The feature fusion module is a novel two-stream neural network that fuses the CNN and GNN feature vectors to produce a comprehensive representation of the scene image. Finally, a fully-connected classifier classified the obtained comprehensive feature vector into one of the pre-defined categories. The proposed model has been evaluated on three benchmark scene datasets, UIUC Sports, MIT67, and SUN397, and obtained classification accuracy of 99.91%, 96.01%, and 85.32%, respectively. In addition, a new dataset named Scene40, which has been introduced in our previous paper, is also used for further evaluation of the proposed method. The comparison results based on classification accuracy criteria show that the proposed model can outperform the best previous methods on three benchmark scene datasets. [ABSTRACT FROM AUTHOR]

For the limitations of global feature-based deep learning point cloud registration algorithms in partial point cloud registration, this paper proposes a partial point cloud registration algorithm NcPE-PNLK combining global features and correspondence. The NcPE-PNLK algorithm introduces a feature interaction module to complete the information interaction between two point clouds in the feature extraction stage, which can improve the credibility of the feature. Moreover, the algorithm predicts the correspondence through the non-corresponding point estimation module, which reduces the influence of non-overlapping regions on the global feature and effectively solves the problem of performance degradation of the global feature registration algorithms in partial point cloud registration. We test the registration performance of NcPE-PNLK on synthetic scene dataset and real dataset in this paper. The experimental results show that NcPE-PNLK can effectively reduce the impact of non-overlapping regions in the registration process and achieve better performance compared with the global feature-based registration algorithms. In addition, compared with the correspondence-based registration algorithms, the NcPE-PNLK algorithm does not need to calculate correspondence precisely, which can achieve high-precision partial point cloud registration with guaranteed efficiency. [ABSTRACT FROM AUTHOR]

The traditional local binary pattern (LBP) is susceptible to the influence of the centre pixel and noise and cannot accurately identify wire rope surface defects. To solve this problem, an image segmentation-based central multiscale local binary pattern (ISCM-LBP) and grey level cooccurrence matrix (GLCM) feature fusion method is proposed in this paper for defect recognition. Image segmentation and multiple scales are introduced into the local binary pattern algorithm to improve the image detail description and suppress noise sensitivity. Second, the centre pixel is connected with the neighbourhood pixel to enhance the robustness of the centre pixel. To further improve the image integrity description, PCA dimensionality reduction and GLCM feature fusion are performed on the features extracted by the ISCM-LBP algorithm, and the steel wire rope surface defects are identified by a support vector machine classifier. Experimental results show that the overall recognition rate reaches 97.5%, which is at least 5% higher than that of other algorithms and can effectively identify various defects on the surface of wire rope. [ABSTRACT FROM AUTHOR]

In recent years, single-image super-resolution (SISR) has acquired tremendous progress with the development of deep learning. However, the majority of SISR methods based on deep learning focus on building more complex networks, which inevitably lead to the problems of computational and memory costs. Thus, these methods may fail to be applied in real-world scenarios. To solve this problem, this paper proposes a lightweight convolution network combined with transformer for SISR named as MMSR. Specifically, an efficient convolutional neural network (CNN) based on multi-scale feature fusion is designed for local feature extraction, which is called MFF-CNN. In addition, we propose a simple and efficient multiple attention block (MAB) to further utilize the context information in features. MAB incorporates channel attention and transformer to help network obtain similar features at a long-term dependence, making full use of global information to further refine texture details. Finally, this paper provides comprehensive results for different settings of the entire network. Experimental results on common used datasets demonstrate that the proposed method can achieve better performances at the 2 × , 3 × and 4 × scales than other state-of-the-art lightweight methods. [ABSTRACT FROM AUTHOR]

Being a significant repository of Buddhist imagery, Thangka images are valuable historical materials of Tibetan studies, which covers many domains such as Tibetan history, politics, culture, social life and even traditional medicine and astronomy. Thangka cultural element images are the essence of Thangka images. Hence, Thangka cultural element images classification is one of the most important works of knowledge representation and mining in the field of Thangka and is the foundation of digital protection of Thangka images. However, due to the limited quantity, high complexity and the intricate textures of Thangka images, the classification of Thangka images is limited to a small number of categories and coarse granularity. Thus, a novel fusion texture feature dual-branch Thangka cultural elements classification model based on the attention mechanism and self-supervised contrastive learning has been proposed in this paper. Specifically, to address the issue of insufficient labeled samples and improve the classification performance, this method utilizes a large amount of unlabeled irrelevant data to pre-train the feature extractor through self-supervised learning. During the fine-tuning stage of the downstream task, a dual-branch feature extraction structure incorporating texture features has been designed, and MS Triplet Attention proposed by us is used for the integration of important features. Additionally, to address the problem of sample imbalance and the existence of a large number of difficult samples in the Thangka cultural element dataset, the Gradient Harmonizing Mechanism Loss has been adopted, and it has been improved by introducing a self-designed adaptive mechanism. The experimental results on Thangka cultural elements dataset prove the superiority of the proposed method over the state-of-the-art methods. The source code of our proposed algorithm and the related datasets is available at https://github.com/WiniTang/MS-BiCLR. [ABSTRACT FROM AUTHOR]

Shape recognition is an active research topic in the field of computer vision and graphic computing. Nevertheless, existing methods are still poor in accuracy and efficiency in some extent, which greatly limits their application in computer vision system. This paper investigates the restraint of feature structure that intrinsically deteriorates recognition performance. Furthermore, we propose a fast shape recognition method based on a bi-level restraint reduction of contour coding (CC2RR), which provides more effective theoretical support for the practical application of the visual algorithm. CC2RR reduces restraint performed from contour feature extraction and expression, respectively. First, for shape contour, the restraint of contour feature extraction is reduced by transforming the direction of contour points to contour segments; second, for the encoded contour segment, the restraint of the contour feature expression is reduced; in other words, the current direction is reduced to the previous and the next direction. Guided by these insights, Hamming code distance is used to match the coding features after the twofold restraint reduction, and the results are obtained. Experimental results verify that the method significantly improves the performance, which runs up to 500 times faster than the existing description methods based on shape contours while increasing robustness. This makes the method useful in practical software system. [ABSTRACT FROM AUTHOR]

Deep learning shows great potential in low-light image enhancement, which can improve image brightness and contrast while keeping image natural. However, due to the lack of prior knowledge extracted manually or excessive amplification of noise, these methods result in poor quality of enhanced images. To solve these challenging problems, this paper proposes a dual-branch grayscale image enhancement network based on dense residual and attention mechanism. Low-light grayscale image is used as input. Firstly, features that fuse deep and shallow information are extracted through dense residual convolution branch network; secondly, texture features are extracted through U-Net branch network combined with attention mechanism; then the extracted features are integrated, and finally luminance is adjusted by a brightness adjustment module to output an enhanced grayscale image. In addition, a joint loss function is designed to measure the network training loss from brightness, texture, contrast and noise. A large number of quantitative and qualitative experiments on LOL and VE-LOL datasets show that the proposed method improves Peak Signal to Noise Ratio Index and Structural Similarity Index by 19.65 - 59.76% and 5.61 - 85.53%, respectively, compared with EnlightenGAN, KinD++, RUAS, LLFlow, etc. The proposed method is superior to the most famous methods, thanks to the deep feature extraction and fusion capability of dense residual convolutional network and texture extraction capability of U-Net network combined with attention mechanism. [ABSTRACT FROM AUTHOR]

This paper proposes an unconstrained human gaze estimation approach for medium-distance scene based on monocular vision. A recurrent convolution neural network for gaze estimation is designed to construct the mapping relationship from the face image to the 3D gaze vector in the camera space, so as to complete the estimation of the human gaze. Firstly, based on the VICON system and a monocular camera, the face image sequence and gaze vector sequence of human gaze behavior are collected in different situations of medium-distance scenes to construct the time-synchronized gaze estimation dataset. Secondly, the recurrent convolutional gaze estimation neural network is designed, and the head pose feature and eye appearance feature are extracted from single frame image with monocular vision image sequences as input. Thirdly, the extracted gaze features are fused in the spatial dimension and the temporal dimension to estimate the direction of the human gaze. Finally, the cross-validation and experimental evaluation of the estimation errors of the constructed gaze estimation model are carried out. The results show that the Cross-Person experimental average error is 7.65°, and the Person-Specific experimental error is 3.88°. Compared with the Gaze360 method, the proposed approach reduces the average evaluation error of the gaze angle by 5.0% in the Cross-Person experiments and 16.6% with the Person-Specific experiments. These experiments verify the effectiveness of the proposed approach, and it has better generalization ability and robustness. [ABSTRACT FROM AUTHOR]

Multispectral images, with their unique three-dimensional characteristics, require specialized spatial-spectral feature extraction modules to achieve superior compression results. Current end-to-end compression frameworks underperform compared to advanced coding algorithms, primarily due to insufficient spectral feature extraction at high bit rates and challenges in guiding entropy coding. To address these issues, this paper proposes the Mixture Autoregressive Spectral Attention Network (MARSA-Net), featuring two attention mechanisms: Coor-Spec and LD-CAM, and an autoregressive component. Our evaluation on real datasets from satellites demonstrates MARSA-Net's superiority over traditional algorithms, including H.266/VVC, underlining its potential in multispectral image compression. This research contributes to improved compression methods and extends our understanding of spectral feature extraction in multispectral imagery. [ABSTRACT FROM AUTHOR]

Raw point cloud (PC) data acquired by 3D sensors or reconstruction algorithms inevitably contain noise and outliers, which can seriously impact downstream tasks, such as surface reconstruction and target detection. To address this problem, this paper proposes an innovative denoising method including the adaptive feature extraction (AFE) module, the gradient field estimation (GFE) module, and the further accelerated gradient ascent (FAGA) module. The method is based on considering the noisy PC as a convolutional distribution of clean PC and noise, and denoising is achieved by updating the positions of the points through gradient ascent iterations. Specifically, for a given noisy PC as input, we first extract global features in the AFE module, which are used as conditions to dynamically guide the extraction of local and non-local features to achieve adaptive feature acquisition. Next, these adaptive features are used as input to the GFE module to estimate the gradient field of the PCs and combined with our proposed FAGA module for denoising operations. Extensive qualitative and quantitative experiments are conducted on synthetic and natural PC datasets, and the results show that the proposed method exhibits superior performance relative to previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the growth in utilizing desktop sharing and remote control applications in recent years for many purposes like online education and remote working, quality assessment (QA) of screen images has become a hot topic. It could be used to enhance the user's quality experience. Currently, most screen image QA methods require a reference image, and the existing blind/no-reference methods do not consider both the image's content and chrominance degradations. This paper proposes a novel blind quality assessment method for screen content images (SCIs) through block-based content representation, which extracts content- and chromatic-based features on local, semi-global, and global scales. Our proposed edge histogram descriptor- and statistical moment-based (EHDSM) method divides the image into 16 blocks and then describes each block using its local edge and semi-global chrominance features. It also takes the global chrominance features into account to investigate how the image's color information is changed in the presence of chrominance distortions. Local features are extracted using edge histogram descriptor, while the semi-global and global features are measured by computing the statistical moments. Next, the quality assessment is achieved by training a support vector regression (SVR) model. Extensive experiments on three commonly used SCI datasets have verified the superiority of our proposed EHDSM method compared with the state-of-the-art blind screen content image quality assessment methods. [ABSTRACT FROM AUTHOR]

Over the last decade, Online Social Media platforms have witnessed a dramatic expansion due to the substantial reliance of individuals on these communication channels. These platforms are widely utilized to convey emotions, share opinions, and express preferences through various means such as artworks, multimedia contents, and blogs. Researchers are exploring these individual-specific traits for biometric identification. Aesthetic biometric systems utilize users' unique preferences across various subjective forms such as images, music, and textual contents. This study introduces a novel multi-modal aesthetic system, with a primary contribution to the development of an attention-based fusion method for person identification. The proposed identification system leverages a deep pre-trained model for high-level feature extraction from visual and auditory modalities. The paper introduces a novel fusion architecture named attention-based residual fusion network (ARF-Net) to incorporate two heterogeneous aesthetic feature vectors. The proposed model yielded a 99.38% identification accuracy on the Aesthetic Image Audio 32 (AIA32) dataset and 98.02% identification accuracy on Aesthetic Image Audio 52 (AIA52) dataset, outperforming other aesthetic biometric systems. The proposed architecture stands out for its efficiency, showcasing a lightweight architecture with minimal parameters, ensuring optimal performance in different modalities. [ABSTRACT FROM AUTHOR]

3D point cloud registration is a basic task in computer vision. In recent years, various of learning-based methods have been proposed to solve this problem. These methods effectively overcome the original problem of over-reliance on initial conditions and enhance the ability of obtaining the corresponding relationship. However, few methods pay enough attention to local features and tend to cause some mismatches. Therefore, this paper proposes two networks to extract local features sufficiently. To obtain a more accurate correspondence relationship between the point clouds, we propose a feature weight allocation network (FWANet), in which the expression ability of feature is enhanced using the proposed significant feature extraction module. Besides that, we utilize an interference elimination module to remove the interference points and enhance the internal correlation of point clouds. We also propose a spatial structural generation network (SSGNet), which fully utilizes the spatial location information to determine the spatial correspondence and generate a reliable connection after concatenating multi-dimensional features. At last, a complete feature space can be effectively captured after combining our FWANet with SSGNet together. We conducted multiple experiments on ModelNet40 datasets and achieved excellent results. Experimental results on four types of data demonstrate the superiority of our algorithm against the state-of-the-art ones. Our code will be available at https://github.com/liu-zikang/registration as soon as the paper is accepted. [ABSTRACT FROM AUTHOR]

In the past decades, a variety of learning-based algorithms have been emerged to try to explore a better solution for stereo matching by leveraging various machine learning algorithms. For enriching learning-based stereo matching algorithm's methodologies, we cast the disparity estimation as a regression problem by leveraging Structured Support Vector Machine (SSVM) in this paper. There are three categories of features have been extracted on account of disparity cues for training the SSVM. Particularly, one of the three feature is named as 'Coarse-to-Fine PatchMatch Filtering', which effectively exploits region and pixel disparity cues. For attaining region disparity cues, we adopt MeshStereo and MeshStereo with Cross-Scale algorithms; for attaining pixel disparity cues, PatchMatch and Cross-Scale PatchMatch stereo matching algorithms are utilized. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed algorithm reveals comparable accuracy with other challenging learning-based ones. It is worth pointing out that our proposal performs over several orders of magnitude faster than others on training time. [ABSTRACT FROM AUTHOR]

Fabric defect detection is an important part of the textile industry to ensure product quality. To solve the problems such as the difficulty of detecting small defects and the coexistence of multi-scale defects in fabric defect detection, a fabric defect detection method based on the feature purification fusion structure is proposed in this paper. Specifically, we improve the feature extraction network to enhance the network's ability to focus on small defective features and effectively reduce the model parameters. The existing methods use direct fusion between multi-level feature maps, which will lead to feature confusion. Therefore, we propose the feature purification fusion structure (FPF), which includes the semantic information supplementation strategy (SIS) and the detail information supplementation strategy (DIS). SIS extracts valid information from the deep feature map and supplements it to the shallow feature map, weakening feature information irrelevant to the shallow feature map detection task. DIS adaptively supplements the feature information required by the deep feature map detection task from the shallow feature map. FPF improves the ability of the network to detect small defects and effectively mitigates the aliasing effect generated during feature fusion. The experimental results show that compared to the baseline model YOLOv5s algorithm, our model achieved a 6.8% improvement in detection accuracy, with an average inference frame rate of 37.6 FPS, demonstrating better detection performance in fabric defect detection. Furthermore, extending this model to aluminum profile defect datasets also demonstrates strong performance. [ABSTRACT FROM AUTHOR]

The strip steel is a common metallic material with a wide range of applications in various industries. However, the issue of surface defects that possess high concealment and low discrimination, which arises during the process of inspecting the quality of strip steel, imposes limitations on the overall quality of strip steel products. The challenging task of industrial quality inspection stems from the difficulty and inefficiency of detecting these defects. This paper addresses the aforementioned challenges by introducing a fast segmentation network model called the feature reused network (FR-Net), which aims to improve the detection of small defects and enhance real-time detection performance in the strip steel surface quality inspection process. In FR-Net, a feature fusion module is used to construct a feature reused fusion bypass to improve the segmentation accuracy of small defects. In addition, a lightweight feature refinement module is proposed to enhance the expression capability of the feature extraction network without increasing the computational effort. Finally, an atrous spatial pyramid pooling module with residual connectivity is proposed to fuse deep features of different scales and enhance the perception of objects of different scales. Experiments on the publicly available datasets showed that the proposed FR-Net achieved the mean intersection over union (mIoU) for NEU-Seg and SD-Saliency-900 datasets were 84.53% and 87.24%, respectively. Meanwhile, the detection speed on a single GPU was 58 FPS. Finally, the size of the proposed FR-Net model is only 45 MB, achieving a good trade-off between accuracy, speed, and model size, thus, providing a new solution for network model deployment on industrial equipment. [ABSTRACT FROM AUTHOR]

Image fusion refers to the process of synergistic combination of useful sensory information from multiple images to synthesize a composite image with greater information content and increased practical value. It aims to maximize pertinent information specific to a sensor while minimizing uncertainty and redundancy in the fused output. In this paper, the authors have proposed a simple yet cohesive framework for 2D greyscale image fusion using morphological differential features. The features are extracted using morphological open–close filters applied at multiple scales using an isotropic structuring element which brings out categorical bright and dark features from the source images. At each scale, the bright (and dark) differential features are mutually compared using higher-valued AC coefficients obtained in the DCT domain within a block. The scale-specific fused features are recursively added to form an image containing high-frequency information from all conceivable scales. The fused image is achieved by superimposing the cumulative feature image onto a suitable base image. The base image is obtained by using a morphological weighted version of pseudomedian filter over the source images using the largest homothetic of the structuring element. The superiority of the framework is empirically verified in different domains of fusion, i.e. multi-focus, multi-sensor, multi-exposure, and multi-spectral image fusion. The proposed approach has surpassed the state-of-the-art unified fusion algorithms in terms of qualitative and quantitative evaluation with a perfect resource-time trade-off. Furthermore, the proposed method has been extended to greyscale–colour and colour–colour image pairs qualifying it for anatomical–functional image fusion. [ABSTRACT FROM AUTHOR]

Image generation has always been one of the important research directions in the field of computer vision. It has rich applications in virtual reality, image design, and video synthesis. Our experiments proved that the proposed multi-style image generative network can efficiently generate high-quality images with different artistic styles based on the semantic images. Compared with the current state-of-the-art methods, the result generation speed of our proposed method is the fastest. In this paper, we focus on implementing arbitrary style transfer based on semantic images with high resolution ( 512 × 1024 ). We propose a new multi-channel generative adversarial network which uses fewer parameters to generate multi-style images. The network framework consists of a content feature extraction network, a style feature extraction network, and a content-stylistic feature fusion network. Our qualitative experiments show that the proposed multi-style image generation network can efficiently generate semantic-based, high-quality images with multiple artistic styles and with greater clarity and richer details. We adopt a user preference study, and the results show that the results generated by our method are more popular. Our speed study shows that our proposed method has the fastest result generation speed compared to the current state-of-the-art methods. We publicly release the source code of our project, which can be accessed at https://github.com/JuanMaoHSQ/Multi-style-image-generation-based-on-semantic-image. [ABSTRACT FROM AUTHOR]

The detection of surface defects is crucial to industrial manufacturing. In recent years, numerous detection methods based on computer vision have been successfully applied in the industry. However, industrial defect detection is still full of challenges. In one aspect, most of the industrial defects are extremely small. In another aspect, even though the intra-class defects have numerous similar elements, their outward appearances differ significantly. In this paper, we propose a dual-structure attention-based multi-level feature fusion network (DaMFFN) to address these two issues. In the first attention-based multi-level feature extraction structure, we introduce novel attention pooling to preserve more detailed information about the defective features of the tiny defect by giving certain regions varying weights. In the second attention-based multi-level feature fusion structure, we propose channel attention to capture the defect feature with the greatest potential for discrimination rather than all possible defect features, which is employed to prevent the incorrect detection of intra-class defects. The experiments demonstrate that the detection performance of the DaMFFN is better than other methods in five surface defect datasets. [ABSTRACT FROM AUTHOR]

The challenge of image reconstruction from very-low-resolution images is made exceedingly difficult by multiple degradation factors in practical applications. Traditional methods do not consider the interactions between these degradation factors, so the results are often insufficient. To reconstruct low-resolution blurry images, both super-resolution and deblurring processes must be applied. In this paper, we propose a joint super-resolution and a deblurring model with integrated processing of the degradation factors to obtain better image quality. The joint model includes two branches, a super-resolution module and deblurring module, and both of them share the same feature extraction module. The super-resolution module consists of multiple layers of residual blocks. The deblurring module supports the robustness of the super-resolution module through feature feed-back in the learning process, by introducing an image blurring feature description into the feature representation. To create modules with high magnification, the base two-branch model is also used in two stages with scale recursion. A second-stage deblurring module receives the output of the first-stage super-resolution module and improves the deblurring capability when the image is further magnified. The modules enhance each other, significantly improve the quality of very-low-resolution text images, and maintain a low model complexity. A step-by-step training strategy is applied to reduce second-stage training difficulty. Experiments show that our approach significantly outperforms state-of-the-art methods in terms of image quality and optical character recognition accuracy, and with a lower computational cost. [ABSTRACT FROM AUTHOR]

In the real-world 24/7 surveillance systems, the images collected during the day and night are visible light images and infrared images, respectively. Infrared images lack color and texture information. In this case, it is more practical to use cross-modality person re-identification (re-ID) to process visible-infrared images. In fact, the cross-modality semantic alignment and specific discriminative feature extraction of different modalities are important for the improvement of modal performance. Therefore, a Semantic Consistent Feature Construction and Multi-granularity Feature learning (SCC–MGL) method is proposed for visible-infrared person re-ID in this paper. The SCC–MGL consists of a Semantic Consistent Feature Construction (SCC) module and a Multi-Granularity Information Enhancement (MGIE) module. In SCC, the features of different modalities are guided by analyzing the relation between feature maps channels and pedestrian's body parts to form consistent semantic information on the corresponding channels, which reduces the impact caused by the misalignment of semantic information. In MGIE, a local modality difference elimination strategy is proposed to remove the modality difference. Meanwhile, the local feature discrimination is improved by reasonably constraining multi-granularity features. The effectiveness and superiority of proposed method are validated by experimental results from SYSU-MM01 and RegDB datasets. [ABSTRACT FROM AUTHOR]

Multiple object tracking is challenging due to the complex spatiotemporal relationship and the occlusion of different targets. Most existing methods use separate neural networks to generate robust features for data association inside the targets' bounding boxes. Unlike existing methods that just consider each target and the trajectory formed independently while ignoring the context information between tracklet and intra-frame detection, this paper proposes a tracking algorithm that combines multi-channel features with learnable graph matching. In brief, we use global and local salient features to model the appearance of intra-frame targets based on parallel graphs and mining high-order intra-context relationships using a completely undirected graph relationship between trajectories and detections. On this basis, we establish a multi-channel message communication mechanism through the maximum pooling mechanism to transmit the maximum dissimilation registration results to the graph matching layer, thereby forming a more robust allocation result in the data association process. Lastly, the evaluations on MOTChallenge benchmarks verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Few-shot text style transfer faces two main challenges: The first challenge is the limited availability of reference style text, while the second challenge is the varying degrees of differences between the style reference image and the source image. Existing methods mainly focus on the influence of local style and global style feature extraction on text style, but they ignore the crucial role played by the difference between the style reference image and the source image on style characteristics, especially in Chinese, which has its own unique ideograph structure. To address this issue, this paper proposes Layer Similarity Guiding Few-shot Chinese Style Transfer (LSG-FCST). LSG-FCST can not only build a transfer network by encoding the content and style characteristics from low-level to high-level semantics, but it can also discover the similarity characteristics between the style reference image and the source image through the attention mechanism. Furthermore, LSG-FCST can integrate the style features generated by the similarity features of different layers and generate the target image through asymmetric decoding. In the self-built text image dataset, we consider three types of visibility situations for the test images in the training set: seen fonts unseen characters, unseen fonts seen characters, and unseen fonts unseen characters. The experiments show that LSG-FCST outperforms the state-of-the-art methods. The code and dataset can be accessed at https://github.com/LYM1111/LSG-FCST. [ABSTRACT FROM AUTHOR]

Edge detection is a key step in various image processing tasks. Edge detection based on deep learning is usually composed of encoding and decoding networks. Encoding networks are usually built based on classifiers (e.g., VGG16) while focusing on the construction of decoding networks. In this paper, an encoding–decoding network is proposed by simulating the visual pathway of the retina-lateral geniculate nucleus (LGN)- the primary visual cortex (V1)-V2-V4- the inferior temporal cortex. Bio-inspired Feature Cascade Network (BFCN) was designed to simulate the transmission modes of feedforward propagation, horizontal connection, and feedback propagation among neurons in the IT, which is conducive to enhancing the characteristic analysis ability of the decoding network. Firstly, to simulate the information processing model of feedforward propagation, a Feedforward Propagation Network is designed to fully fuse the underlying information. Secondly, to simulate the information processing model of the horizontal connection between neurons, the Inter-Layer Information module (ILI) is designed to process the interlayer information of FPNet, which is beneficial to enhancing the feature extraction ability. Finally, to simulate the feedback propagation, the Proximity Combination Network (PCNet) is designed to integrate the feature prediction of each stage and strengthen the generalization ability of the network. Experimental results show that the proposed contour detection model outperforms current similar models. [ABSTRACT FROM AUTHOR]

Chinese calligraphy is one of the excellent expressions of Chinese traditional art. But people without domain knowledge of calligraphy can hardly read, appreciate, or learn this art form, due to it contains many brush strokes with unique shapes and complicate structural topological relationship. In this paper, we explore the solution of text sequence recognition of calligraphy, which is a challenging task because traditional algorithms of text recognition can rarely obtain the satisfied results for the varied styles of calligraphy. Therefore, based on a trainable neural network, this paper proposes an easy recognition method, which combines feature sequence extraction based on DenseNet (Dense Convolutional Network) model, sequence modeling and transcription into a consolidated architecture. Compared with previous algorithms for text recognition, it has two distinctive properties: One is to acquire the artistic features on shapes and structures of different styles of calligraphic characters and the other is to handel sequences in arbitrary lengths without character segmentation. Our experimental results prove that in contrast with several common recognition methods, our method of Chinese calligraphic character in diverse styles demonstrates greater robustness, and the recognition accuracy rate reaches 84.70%. [ABSTRACT FROM AUTHOR]

Strong demand for autonomous vehicles and the wide availability of 3D sensors are continuously fueling the proposal of novel methods for 3D object detection. In this paper, we provide a comprehensive survey of recent developments from 2012–2021 in 3D object detection covering the full pipeline from input data, over data representation and feature extraction to the actual detection modules. We introduce fundamental concepts, focus on a broad range of different approaches that have emerged over the past decade, and propose a systematization that provides a practical framework for comparing these approaches with the goal of guiding future development, evaluation, and application activities. Specifically, our survey and systematization of 3D object detection models and methods can help researchers and practitioners to get a quick overview of the field by decomposing 3DOD solutions into more manageable pieces. [ABSTRACT FROM AUTHOR]

Object detection from UAV (unmanned aerial vehicle) images is a crucial and challenging task in the field of computer vision. The task suffers from the difficulties of small dense objects, low pixel occupation of objects, and features that are not easily extracted in images. In this paper, we proposed a multilayer feature fusion algorithm named SCA-YOLO (spatial and coordinate attention enhancement YOLO) for small object detection with hybrid attention mechanisms. It uses the single-stage detection algorithm YOLOv5 as the base framework. Firstly, a hybrid attention module with associated coordinate attention is designed to enhance the feature extraction of small objects. Secondly, to address the problem that small objects are vulnerable to being disturbed by the complex background information on UAV images, an improved SEB (simple and efficient bottleneck) module is designed to further distinguish foreground and background features. Thirdly, a multilayer feature fusion structure is built to perform channel stitching of shallow and deep feature maps, as well as to enrich the semantic information of shallow features by adding horizontal jump connections. Finally, experiments are conducted on the VisDrone2020 dataset, which involves a large number of small objects photographed by drones. In addition, we also conduct extended experiments on the DOTA dataset and PASCAL VOC dataset. Comparative experimental results indicate that the proposed method considerably improves the accuracy of small object detection on multiple benchmark datasets. [ABSTRACT FROM AUTHOR]

3D object detection, as an essential part of perception, plays a principal role in the autonomous driving system. The cost-competitive monocular 3D object detection has drawn increasing attention recently. However, it still suffers an inferior accuracy especially for occluded objects due to the limited camera view. Inspired by compositional models, in which an object is represented as a combination of multiple components, this paper proposes a new monocular 3D object detection method that decreases the impact of occlusion by utilizing an object's front and side cues. To do this, the features are extracted from a decoupled front and side representation and then fused by an attention-based module to obtain a more consistent feature distribution. An uncertainty-guided depth ensemble based on geometry is further applied to refine the depth prediction. Experiment results demonstrate that as compared to the conventional methods, the proposed method significantly improves the detection performance for occluded objects while still satisfying real-time efficiency, with the Average Precision on 40 recall positions (AP40), respectively, increasing by 10.23% for partly occluded objects and 12.22% for mostly occluded objects in the KITTI benchmark. The codes are released at https://github.com/kagurua/Front-Side-Det [ABSTRACT FROM AUTHOR]

In recent years, due to the powerful feature extraction capabilities of convolutional neural networks (CNNs), many single-image super-resolution (SISR) methods based on CNN have achieved remarkable results. However, more model parameters and higher computational cost make these methods unsuitable for devices with limited computing power. In this paper, a lightweight multi-dimension feature fusion network (LMDFFN) is proposed to reduce model parameters. In nonlinear mapping module of the LMDFFN, the lightweight feature extraction block (LFEB) and the multi-dimensional feature extraction block (MDFEB) embedded in each LFEB are designed to significantly improve the reconstruction performance. The channel split operation can group the features. Combined with square-kernel convolution kernel and asymmetric convolution kernel, different types of convolution kernels can focus on different types of features, and the grouping mechanism can effectively reduce the complexity of the model. In addition, a channel and spatial attention module (CSAM) dedicated to SISR is proposed to focus on image details and different semantic level in feature maps. Moreover, a lightweight reconstruction module is designed with depth-wise separable convolution, and the designed spatial attention module is integrated into the reconstruction module to focus on more important features in the reconstruction stage. Compared with recent lightweight SISR methods, LMDFFN has better reconstruction accuracy with lower model parameters and computational complexity, and the reconstructed images have higher visual quality. [ABSTRACT FROM AUTHOR]

This paper presents a novel and efficient multitask learning framework for image translation and saliency detection from remote sensing images, which mainly contains the image translation network-weight sharing attention GAN (WSA-GAN) and the salient object detection network-boundary guidance network (BGNet). WSA-GAN can be used to generate a large number of synthetic infrared remote sensing images (IRIs) or optical remote sensing images (ORIs) from the corresponding complementary modality images. Then, a new multimodal context-aware learning is proposed for feature extraction and to coordinate the entanglement of latent features in the multimodal context of ORIs and IRIs. Since convolutional neural networks do not perform well when the object has directional variance, our framework introduces the attention-aware CapsNet (AACNet) to alleviate the problem and enhance the feature expressiveness. In addition, knowledge distillation strategy is introduced in AACNet to reduce the model complexity. Finally, the multiscale feature learning network and the boundary-aware block are designed to generate more accurate saliency detection results with clear boundaries. Experimental results demonstrate that the presented image translation and salient object detection networks outperform other approaches. [ABSTRACT FROM AUTHOR]

Underwater object detection is a fascinating but challengeable subject in computer vision. Features are difficult to extract due to the color cast and blur of underwater images. Moreover, given the small scale of the underwater object, some details will be lost after several layers of convolution. Therefore, a multi-scale aggregation feature pyramid network is proposed to integrate multi-scale features and improve underwater object detection performance. Specifically, a lightweight and efficient network is used to extract the basic features. A special subnet is designed to improve the feature extraction capability of the backbone network to enrich the detailed features of small underwater objects. In addition, a multi-scale feature pyramid is proposed to enrich feature map. Each feature map enhances contextual information through a combination of up-sampling and down-sampling. The centerness strategy of the fully convolutional one-stage object detection head is improved by adding corner point regression to enhance the recall rate of small objects. Generalized intersection over union (GIoU) instead of IoU can better reflect the degree of coincidence between the actual box and the predicted box. Therefore, the regression loss is changed to GIoU loss. This paper evaluates the network on the underwater image dataset and obtains 78.90% mAP. Meanwhile, the experiment on the PASCAL VOC datasets is conducted and gets 84.3% mAP. [ABSTRACT FROM AUTHOR]

In order to address the problem of low efficiency and accuracy of artificial visual inspection for the quality of toy sets, this paper investigates the defect detection of toy sets based on machine vision. Firstly, an improved U-Net (IU-Net) is developed by introducing the Squeeze-and-Excitation (SE) block into the U-Net and modifying its objective function. Then, a convolutional neural network (CNN) is constructed for feature extraction and defect matching. Finally, a defect detection model for toy sets is created by integrating these two models. In addition, when it comes to a new-variety toy set, the IU-Net model is trained through transfer learning with a small number of new-variety data samples. This can reduce the dependence on new-variety data samples and improve the efficiency of model training process. The experimental results show that the defect detection accuracy of the proposed method is 100% and 99.43% for the whole toy set and single component, respectively, which meets the requirements of industrial automation quality inspection of toy sets. [ABSTRACT FROM AUTHOR]

The geometric distortion in panoramic images significantly mediates the performance of saliency detection method based on traditional CNN. The strategy of dynamically expanding convolution kernel can achieve good results, but it also produces a lot of computational overhead in the process of reading the adjacency list, which decreases the computational efficiency. The appearance of graph convolution provides a new way to solve such problems. Although using graph convolution can effectively extract the structural features of the graph, it reduces the accuracy of the model resulting from ignoring the spatial features of the image signal. To this end, this paper proposes a construction method of the multi-scale graph structure of the panoramic image and a panoramic image saliency detection model composed of an image saliency feature extraction network and multi-scale saliency feature fusion network combining the image structure information and spatial information in the panoramic image. First, we establish a graph structure consisting of root and leaf nodes obtained by super-pixel segmentation at different scales and spherical Fibonacci sampling, respectively. Then, a feature extraction network composed of two graph convolution layers and two one-dimensional auto-encoders with the same parameterization is used to extract the salient features of the multi-scale graph structure. Finally, the U-Net network fuses the multi-scale saliency features to get the final saliency map. The results show that the proposed model performs better than the state-of-the-art models in terms of calculation speed and accuracy. [ABSTRACT FROM AUTHOR]

Micro-expression is an expression that reveals one's true feelings and can be potentially applied in various domains such as healthcare, safety interrogation, and business negotiation. The micro-expression recognition is thus far being judged manually by psychologists and trained experts, which consumes a lot of human effort and time. Recently, the development of the deep learning network has proven promising performance in many computer vision-related tasks. Amongst, micro-expression recognition adopts the deep learning methodology to improve the feature learning capability and model generalization. This paper introduces a lightweight apex-based enhanced network that improves by extending one of the state-of-the-art, shallow triple stream three-dimensional CNN. Concretely, the network is first pre-trained with a macro-expression dataset to encounter the small data problem. The features were extracted from CASME II, SMIC, and SAMM datasets. Moreover, thorough recognition results comparison of the datasets are the optical flow-guided features. Besides, an eye masking technique is introduced to reduce noise interference such as eye blinking and glasses reflection issues. The results obtained have an accuracy of 79.19 % and an F1-score of 75.9 % . Comprehensive experimentation had been conducted on the composite dataset that consists of is provided by comparing it with recent methods. Detailed qualitative and quantitative results are reported and discussed. [ABSTRACT FROM AUTHOR]

Head pose estimation methods without facial key points have emerged as a promising research field. However, there remain several unsolved challenges. For example, the current methods incur a computational cost, require large memory, and are difficult to deploy in practical applications. We propose a lightweight high-precision head pose estimation method based on a dual-stream convolutional neural network for overcoming these issues. The network comprises a dual-stream lightweight backbone network, external attention module, and soft stagewise regression (SSR) module. Dual-stream lightweight backbone network can extract original image features more effectively while keeping low computational overhead. External attention module can enhance the feature map extraction from the backbone network and improve the feature attention. SSR module calculates the probability of the head in each direction and predicts the head pose by regression. Extensive experiments on Annotated Facial Landmarks in the Wild (AFLW2000) and Biwi Kinect Head Pose Database (BIWI) datasets demonstrate that the model proposed in this paper has fewer parameters and lower estimation errors than the state-of-the-art methods in the field of head pose estimation in recent years. [ABSTRACT FROM AUTHOR]

In view of the high dimensionality, nonrigidity, multiscale variation and the influence of illumination and angle on facial expressions, it is quite difficult to obtain facial expression images or videos using computers and analyze facial morphology and changes to accurately obtain the emotional changes of the subjects. Existing facial expression recognition algorithms have the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. The facial expression recognition method based on deep learning has problems such as overfitting, gradient explosion and parameter initialization. Therefore, this paper develops a facial expression recognition algorithm based on the deep learning method. An adaptive model parameter initialization based on the multilayer maxout network linear activation function is proposed to initialize the convolutional neural network (CNN) and the long–short-term memory network (LSTM) method. It can effectively overcome the gradient disappearance and gradient explosion problems in the deep learning model training process. At the same time, the convolutional neural network with an LSTM memory unit is used to extract the related information from the image sequence, and the facial expression judgment is based on a single-frame image and historical-related information. However, the top-level structure of the CNN model is a fully connected feedforward neural network, which undertakes the task of expression classification. Therefore, the SVM classification method replaces the top-level classifier to further improve the expression classification accuracy. Experiments show that the facial expression recognition method proposed in this paper not only accurately identifies various expressions but also has good adaptive ability. This is because the method achieves the adaptive initialization of the parameters of the deep learning model construction process and also analyzes the relevance of the expression database expression, thereby improving the accuracy of expression recognition. [ABSTRACT FROM AUTHOR]

Video-based facial expression recognition is a very challenging task. The expression features portrayed by traditional ResNet18 are not rich enough, while the classical method LSTM to process expression videos may not extract effective temporal features for cases with weaker emotion intensity. This paper proposes a spatiotemporal attention network to extract more diverse spatial features and more effective temporal relationships. Firstly, a spatial attention module is used to enhance the expression features extracted by the ResNet18 and remove redundancy. Then, multiple levels of information are combined to extract richer expression features. Meanwhile, the video stream is divided into a series of video clips using a sliding window, and the temporal features are extracted from each small clip by the LSTM, which is a simple but effective way to divide the video. Third, considering the different importance of expression features extracted from each window for the results of expression recognition of the whole video, an attention-based score fusion module is proposed to fuse expression information from multiple windows. We perform comprehensive experiments on in-the-wild FER benchmarks (AFEW8.0 and HUST-MM). Quantitative and qualitative analyses demonstrate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

As one of fundamental texture classification methods, LBP-based descriptors have attracted considerable attention due to the efficiency, simplicity, and high performance. However, most of binary pattern methods cannot effectively capture the texture information with scale changes. Inspired by this, this paper proposes a multi-scale threshold integration encoding strategy for texture classification. The essence of this strategy is to introduce the multi-scale local texture information in the view of thresholding. Based on this, we propose the local multi-scale center pattern, local multi-scale sign pattern, and local multi-scale magnitude pattern to extract and describe the multi-scale local texture information. Then, the three sub-patterns are jointly combined to generate the final descriptor for texture classification tasks. The experimental results on three popular texture databases significantly demonstrate that the proposed texture descriptor is very discriminative and powerful for visual texture classification tasks. [ABSTRACT FROM AUTHOR]

Tea-cake content-based image retrieval (CBIR) is an essential issue in tea-cake traceability. Popular CBIR methods work on medical and social networking, whereas tea-cake images have intensive and parallel representations among classes. While facing tea-cake CBIR, low inter-class and high intra-class distances increase retrieval difficulties using traditional CBIR methods. Thus, this paper proposes a tea-cake CBIR approach based on deep neural networks to retrieve tea-cakes. In the model, we establish a feature extraction model with designed dense blocks, where a cross-entropy loss function is explored to train the model and catch detailed features. Furthermore, to decrease intra-class and expand inter-class interval, a masked autoregressive discriminative normalization flow (MADNF) is presented to map the gained features in high dimensions to corresponding representations in Gaussian spaces. Particularly, a maximum likelihood function is developed to train MADNF for avoiding non-convergence. Extensive experiments on the tea-cake dataset show our method has significant performance compared with current competitors. Furthermore, experiments on the bird species dataset further demonstrate the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

Compared with the traditional hashing methods, deep hashing methods generate hash codes with rich semantic information and greatly improve the performances in the image retrieval field. However, it is unsatisfied for current deep hashing methods to predict the similarity of hard example pairs. There exist two main factors affecting the ability of learning these pairs, which are weak key features extraction and the shortage of hard example pairs. In this paper, we give a novel end-to-end model to extract the key feature and obtain hash code with the accurate semantic information. In addition, we redesign an indicator to assess the hard degree of pairs and update penalty weights of them in the proposed hard pair-wise loss. It effectively alleviates the shortage problem. Experimental results on CIFAR-10 and NUS–WIDE demonstrate that our model outperformances the mainstream hashing-based image retrieval methods. [ABSTRACT FROM AUTHOR]

Automatic facial expression recognition plays a crucial role in computer vision, and pattern recognition. However, most existing deep learning-based facial expression classifiers usually obtain high average accuracy but have poor recognition accuracy for difficult expressions, like fear and disgust. In this paper, we propose a novel end-to-end architecture termed two-stream inter-class variation enhancement network, which learns the high-level semantic features and subtle inter-class variations in a joint fashion. More precisely, the global feature extraction network is used to extract spatial-channel semantic features, and the variations between different expressions are modeled by a distinction-reinforced network. The outputs of these two streams are weighted integrated in the expression classification network. In addition, a class balanced-weighted cross-entropy loss is designed to further improve feature discrimination. Experiment results indicate that the proposed network can significantly improve the recognition of difficult expressions and achieve a satisfactory average recognition accuracy of 73.67% on FER2013, 86.17% on RAFDB, 98.19% on CK+, and 98.85% on Oulu-CASIA, which outperforms the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Effective segmentation of skin scars is an important part of judicial scar identification. The current evaluation procedure mainly focuses on visual examination, and the accuracy needs to be improved. This paper proposes an unsupervised scar segmentation method based on saliency detection, which can correctly extract scars in complex scar conditions. The image Gaussian pyramid feature map is extracted for feature extraction, and the clustering algorithm is used to fuse the contrast features and spatial features to establish the skin scar saliency map. Finally, the skin scar region is segmented through post-processing. The experimental results show that the precision of the proposed method is improved by 18.4 % over the suboptional method compared with seven unsupervised methods on private datasets provided by a local forensic department, which proves that the proposed method supports the optimization and improvement of skin scar segmentation. [ABSTRACT FROM AUTHOR]

As an advanced image editing technology, image inpainting leaves very weak traces in the tampered image, causing serious security issues, particularly those based on deep learning. In this paper, we propose the global–local feature fusion network (GLFFNet) to locate the image regions tampered by inpainting based on deep learning. GLFFNet consists of a two-stream encoder and a decoder. In the two-stream encoder, a spatial self-attention stream (SSAS) and a noise feature extraction stream (NFES) are designed. By a transformer network, the SSAS extracts global features regarding deep inpainting manipulations. The NFES is constructed by the residual blocks, which are used to learn manipulation features from noise maps produced by filtering the input image. Through a feature fusion layer, the features output by the encoder is fused and then fed into the decoder, where the up-sampling and convolutional operations are employed to derive the confidential map for inpainting manipulation. The proposed network is trained by the designed two-stage loss function. Experimental results show that GLFFNet achieves a high location accuracy for deep inpainting manipulations and effectively resists JPEG compression and additive noise attacks. [ABSTRACT FROM AUTHOR]

Objective assessment of image quality seeks to predict image quality without human perception. Given that the ultimate goal of a blind/no-reference image quality assessment (BIQA) algorithm is to provide a score consistent with the subject's prediction, it makes sense to design an algorithm that resembles human behavior. Recently, a large number of image features have been introduced to image quality assessment. However, only a few of these features are generated by using the computational mechanisms of the visual cortex. In this paper, we propose bioinspired algorithms to extract image features for BIQA by simulating the visual cortex. We extract spatial features like texture and energy from images by mimicking the retinal circuit. We extract spatial-frequency features from images by imitating the simple cell of the primary visual cortex. And we extract color features from images by employing the color opponent mechanism of the biological vision system. Then, by using the statistical features derived from these physiologically plausible features, we train a support vector regression model to predict image quality. The experimental results show that the proposed algorithm is more consistent with subjective evaluations than the comparison algorithms in predicting image quality. [ABSTRACT FROM AUTHOR]

Surface defect detection of industrial products has become a promising area of research. Among the existing defect detection algorithms, most of the CNN-based methods can achieve the task of defect detection under ideal experimental conditions. However, the accuracy of defect detection is easily affected by the different lighting conditions of the environment and the inconsistency of the defect scale. Therefore, general deep learning methods have difficulties in solving the problem of defect detection in complex scenes. In this paper, a defect detection network based on low-light enhancement and multi-scale feature extraction (LE–MSFE–DDNet) is proposed. There are two blocks in the proposed network, including a low-light enhancement block and a SE-FP block. In the low-light enhancement block, the deep network is applied to enhance light adaptation of the deconstructed low-light feature map in our network. The influence of illumination inconsistency is weakened by the introduction of this block. In the SE-FP block, the dependencies between different channels are combined with the multi-scale feature extraction. The defects with different scales are accurately located through the combination of this block and our network. In addition, a Fine Cans Defect dataset based on the surface of fine cans is collected by this paper to verify the feasibility of the proposed network. The proposed model is compared with the state-of-the-art object detection network and the proposed method achieves 94.3% average accuracy on the Fine Cans Defect dataset. The experimental results show that the proposed method outperforms the state-of-the-art method for surface defect detection. [ABSTRACT FROM AUTHOR]

Scale variation is one of the challenges in object detection. In this paper, we design a Multi-Level Feature Fusion Pyramid Network (MLFFPN) that can fuse features with different receptive fields so as to produce reliable object representations robust against scale variation. Specifically, we perform feature extraction on the backbone network with convolutional kernels of different sizes, reconstructing the feature pyramids with the various receptive fields by adding top-down paths and lateral connections. Then, the reconstructed feature pyramids are fused. Finally, the bottom-up path enhancement is added for the final prediction. To verify the proposed method, we constructed a large-scale object detection dataset containing in total 225,944 instances and 16,000 images of 30 classes of common objects. In this study, we introduce MLFFPN into the object detection network and conduct a series of experiments on our datasets and MSCOCO datasets. Without bells and whistles, MLFFPN achieves a considerable detection improvement over the baseline network. [ABSTRACT FROM AUTHOR]

In recent years, e-commerce and online shopping have grown tremendously. The development of an image-based cloth retrieval system has become a major research topic in order to effectively and accurately recover required garments from a big database. In this paper, a piecewise feature extraction (PWF) framework is proposed to retrieve the cross-domain clothes. This feature selection strategy is presented to extract optimal features in order to enhance the detection rate while simultaneously simplifying image retrieval computation. First, the user-input query images are collected from online or snapshots. Further clutter backgrounds and different skin textures, of the image, are eliminated via modified GrabCut segmentation. Followed by, the PWF model retrieves the required cloth from the shop domain. In PWF cloth and sleeve size attributes are identified in a separate block with Hough transform and fuzzy rule. Consequently, high-level features are extracted by using a novel RetrieveNet. BAM (Bottleneck attention module) of the RetrieveNet emphasizes the network to focus on the domain features of the input cloth. Finally, the L2 norm is employed to determine the degree of similarity between the shop and query cloths. Based on the similarity, related clothes are retrieved. DARN and Deepfashion2, public datasets are exploited to validate the proposed model, showing that the top-50 image retrieval accuracy is 69.54% and 47.29%, respectively. The efficiency of the proposed clothes retrieval approach is demonstrated by the outcomes of our experiments. [ABSTRACT FROM AUTHOR]

Heart disease is a common condition worldwide and has become one of the leading causes of death worldwide. The electrocardiogram (PCG) is a safe, painless, and non-invasive test that captures bioacoustic information reflecting the function of the heart by capturing the acoustic signal of the patient's heart. Nowadays, based on biosignal processing and artificial intelligence technologies, automated heart sound classification is playing an increasingly important role in clinical applications. In this paper, we propose a new parallel CNNs-transformer network with multi-scale feature context aggregation (PCTMF-Net). It combines the advantages of CNNs and transformer to achieve efficient heart sound classification. In PCTMF-Net, firstly, the heart tone signal features are extracted using the second-order spectral analysis, and a transformer-based MHTE-4 (multi-head transformer encoder with four attention heads) is designed to encode and aggregate the contextual information, and then, two CNNs feature extractors are designed in parallel with MHTE-4 to capture the hierarchical features. Finally, the feature vectors obtained from CNNs and MHTE-4 through feature fusion in PCTMF-Net will be fed into the fully connected layer for predicting the classification results of heart sounds. In addition, we perform validation based on two publicly available mutually exclusive heart sound datasets and conduct extensive experiments and comparisons of existing algorithms under different metrics. The experimental results show that our proposed method achieves 99.36% accuracy on the Yaseen dataset and 93% accuracy on the PhysioNet dataset. It surpasses current algorithms in terms of accuracy, recall and F1-score metrics. The aim of this study is to apply these new techniques and methods to improve the diagnostic accuracy and validity of heart disease for clinical use. [ABSTRACT FROM AUTHOR]