Search

Fruit recognition and location are the premises of robot automatic picking. YOLOv3 has been used to detect different fruits in complex environment. However, for the object with definite features, the complex network structure will increase the computing time and may cause overfitting. Therefore, this paper has carried out a lightweight design for the YOLOv3. This paper proposed an improved T-Net to detect tomato images. Firstly, the T-Net reduces the residual network layers. This paper changed the number of cycles in each group of the residual unit to 1, 2, 2, 1, and 1. Second, two feature layers with different scales are selected according to the features of tomatoes. Meanwhile, the convolutional layer at the neck has been reduced by two layers. Finally, the location and approximate diameter of the ripe tomato are obtained by combining the node information of the Intel D435i camera and T-Net in the Robot Operation System. T-Net obtains mean average precision (mAP) of 99.2%, F1-score of 98.9%, precision of 99.0%, and recall of 98.8% at a detection rate of 104.2 FPS. The proposed T-Net has outperformed the YOLOv3 with 0.4%, 0.1%, and 0.2% increase in precision, mAP, and F1-score. The detection speed of T-Net is 1.8 times faster than YOLOv3. The mean errors of the center coordinates and diameter of the tomato are 8.5 mm and 2.5 mm, respectively. This model provides a method for efficient real-time detection and location of tomatoes. [ABSTRACT FROM AUTHOR]

Regularly detect the corrosion of metal structures and take countermeasures according to the degree of corrosion, which can reduce the potential safety hazards and avoid unnecessary economic losses. This paper presents a deep learning-based metal corrosion detection method, which is used to accurately segment the corrosion regions. This model incorporates Hybrid Attention Residual Block (HARB) and External Skip Connections (ESC) in the basic architecture of U-Net. HARB can reweight the features so that the network pays more attention to the corrosion regions and ignores other irrelevant regions. Shallow features and deep features are fused using ESC to make the features of network learning more effective and comprehensive, which is conducive to improving the segmentation accuracy. In order to further improve the semantic segmentation performance, the Attention Gate Residual Block (AGRB) is proposed, which can optimize segmentation by strengthening the regions to be segmented and reducing the activation value of the background, while overcoming the semantic and scale inconsistencies between input features. In this paper, comprehensive comparative experiments are conducted on the Metal Corrosion dataset. The experimental results show that the F1 and mIoU of the proposed method are 91.21% and 89.56%, respectively. Compared with U-Net, HEU-Net achieves a 7.91% improvement in F1 and a 10.16% improvement in mIoU. These results show that our method outperforms state-of-the-art models and can achieve better semantic segmentation results. [ABSTRACT FROM AUTHOR]

Image distortion is an inevitable part of the image acquisition process, which negatively affects the high-frequency contents of the images. Therefore, it is important to improve the high-frequency contents of the acquired degraded images in the imaging systems and devices. High-boost filtering is an effective method that is used by the scanning and printing devices for enhancing the high-frequency contents of the images and improving their visual quality. In view of this, in this paper, we first develop a residual block for the task of image super-resolution that employs the high-boost filtering operations. It is demonstrated that the super-resolution network, which is formed by a cascade of the proposed residual block, is able to provide a high performance. Further, in this paper, we propose a novel learning method that improves the generalization capability of our deep super-resolution network. Specifically, we generalize the mapping between the spaces of the degraded low-resolution image and the ground-truth image by employing the multiple supervised learning strategy. It is shown that the proposed multiple supervised learning strategy leads to obtaining the weights of our super-resolution network in such a way that its performance is still high when the images are degraded by a set of degradation parameters that is slightly different than that used for the training process. [ABSTRACT FROM AUTHOR]

Due to the emergence and advancement of deep learning technologies, scene text detection is becoming more widespread in various fields. However, due to the complexity of distances, angles and backgrounds, the adjacent texts in images have the problem that the detection boxes are far away from the texts, i.e., a position is not accurate enough. In this paper, we propose a text detection method centered on double ResNet-based and changed channels recursive feature pyramid, which integrates ResNet50-Mish and Res2Net50-Mish, as well as using recursive feature pyramid with changed channels. Firstly, scene images are fed into ResNet50-Mish and Res2Net50-Mish of double ResNet-based, and results are passed through a weight-based addition step to generate the fused feature maps. Secondly, the processed feature maps of double ResNet-based are sent into changed channels recursive feature pyramid to obtain feature maps with enhanced feature information. Also, the relevant segmentation results are then obtained by concatenating and convoluting. Finally, the results are given to progressive scale expansion algorithm to output the location of texts in images. The proposed model is trained and tested on ICDAR15 and CTW1500 benchmark datasets. In terms of precision values, our method outperforms or is comparable to state-of-the-art methods. In particular, experimental results achieve 91.53% precision on ICDAR15 dataset and 84.89% precision on CTW-1500 dataset. [ABSTRACT FROM AUTHOR]

Due to the rapid advancement of GPU computing, deep learning has lately been widely used in image denoising. Most deep learning methods require noise-free images as labels, which are often difficult or impossible to obtain. Therefore, denoising network models have to be trained with a pair of noisy and low-noise images. However, the restored images still face the problem of losing detail information. In this paper, we propose a novel denoising network model based on the concept of Noise2Noise (N2N), where pairs of noisy images are utilized to train a neural network that can learn the noise distribution relationship between them. This newly-proposed model (RA-UNet) draws inspiration from the classical UNet architecture and is designed with a multi-Residual convolutional block with Attention that can adapt different scales to mine the key information of images and recover clearer images. The denoising performance of RA-UNet is comparable and better than that of the conventional CBM3D, while the proposed RA-UNet performs significantly better on both PSNR and SSIM with less 2.5 % FLOPs and less 3 % running time compared to existing deep learning-based methods, such as DnCNN and ADNet. From the perspective of visual quality, RA-UNet can restore images with higher clarity. Compared with the UNet model, our model improves the average PSNR and SSIM obtained by testing Gaussian noise ( σ = 50 ) images on four classic datasets by 2.28 dB and 0.0552, respectively. [ABSTRACT FROM AUTHOR]

In recent years, deep metric learning and its probabilistic extensions claimed state-of-the-art results in the face verification task. Despite improvements in face verification, probabilistic methods received little attention in the research community and practical applications. Previous metric learning benchmarks avoided comparison with probabilistic methods, and it is still unclear whether they generalize well to image retrieval tasks beyond faces. In this paper, we, for the first time, perform an in-depth analysis and unified comparison of known probabilistic methods in verification and retrieval tasks. We study different design choices and argue that many of them have limited impact, while only a few probabilistic methods surpass modern metric learning approaches. Combining the best ideas of earlier works, we propose a simple modification of the top-performing method, achieving new state-of-the-art results among probabilistic methods. Finally, we study confidence prediction and show that it correlates with data quality, but contains little information about prediction error probability. We thus provide a new confidence evaluation benchmark and establish a baseline for future confidence prediction research. PyTorch implementation is publicly released. [ABSTRACT FROM AUTHOR]

This paper proposes a novel data-enhanced virtual texture generation network for use in deep learning detection systems. The current methods of data enhancement, such as image flipping, scaling ratios, or Generative Adversarial Networks, have limitations as they cannot determine characteristics beyond the training data. The proposed system uses the texture characteristics of a learning surface to generate surface textures through the Open Graphics Library, which can simulate material textures, light sources, and shadow effects. This enables the generation of required texture parameters for the Reinforcement Learning Network to conduct parameter search. The generated image is authenticated by a discriminator, and the reward score is fed back into the critic network to update the value network. The proposed system can complement the imbalance of defective data types, generate large quantities of random and non-defective data, and automatically classify and label during the generation process, reducing labor consumption and improving labeling accuracy. The study found that the proposed data enhancement method can increase the diversity of data characteristics, and the generated data can increase the recall rate of test and verification data sets. Specifically, the proposed system increased the recall rate of test data sets with different distributions from 78.21% to 82.40% and the recall rate of verification data sets with the same distribution from 81.64 to 91.94%. [ABSTRACT FROM AUTHOR]

Fabric defect detection is an important part of the textile industry to ensure product quality. To solve the problems such as the difficulty of detecting small defects and the coexistence of multi-scale defects in fabric defect detection, a fabric defect detection method based on the feature purification fusion structure is proposed in this paper. Specifically, we improve the feature extraction network to enhance the network's ability to focus on small defective features and effectively reduce the model parameters. The existing methods use direct fusion between multi-level feature maps, which will lead to feature confusion. Therefore, we propose the feature purification fusion structure (FPF), which includes the semantic information supplementation strategy (SIS) and the detail information supplementation strategy (DIS). SIS extracts valid information from the deep feature map and supplements it to the shallow feature map, weakening feature information irrelevant to the shallow feature map detection task. DIS adaptively supplements the feature information required by the deep feature map detection task from the shallow feature map. FPF improves the ability of the network to detect small defects and effectively mitigates the aliasing effect generated during feature fusion. The experimental results show that compared to the baseline model YOLOv5s algorithm, our model achieved a 6.8% improvement in detection accuracy, with an average inference frame rate of 37.6 FPS, demonstrating better detection performance in fabric defect detection. Furthermore, extending this model to aluminum profile defect datasets also demonstrates strong performance. [ABSTRACT FROM AUTHOR]

The feature matching algorithm based on deep learning has achieved superior performance compared to traditional algorithms in terms of both matching quantity and accuracy, but there are still some high-error matching results in complex scenes, which adversely affects the subsequent work. Based on SuperGlue, we propose an accurate feature matching algorithm via outlier filtering. Firstly, DBSCAN real-time superpixel segmentation (RTSS-DBSCAN) is used to divide the image into regions, and then the outlier filtering module is designed according to the local similarity principle of feature matching. On the premise of not affecting the correct matching results, the matching results with high errors will be filtered to improve the matching accuracy. Meanwhile, due to the lag of traditional Exponential Moving Average algorithm (EMA), an adaptive EMA is designed and integrated into the SuperGlue training process to further improve the training speed and matching accuracy. We evaluate the overall performance of the matching method using the AUC of pose error at the thresholds (5°, 10°, 20°), a common evaluation metric, to provide a more detailed and intuitive evaluation of the matching effectiveness using precision and recall. The experimental results show that the method in this paper can effectively filter the matching results with large errors and has high accuracy and robustness. The AUC of pose error at thresholds (5°, 10°, 20°) reaches 36.53, 56.23, and 73.68, and the precision and recall reach 80.07 and 91.52, respectively, which have better matching results compared with other algorithms. [ABSTRACT FROM AUTHOR]

With the development of deep learning, the performance and efficiency of text detection in natural scenes have been significantly improved. Due to the irregular geometric shape of natural scene text, it is challenging to detect text of arbitrary shape. Most of the existing methods are regression-based or segmentation-based methods. This paper presents an efficient framework to detect arbitrary shape text instances by combining regression-based and segmentation-based methods. Specifically, we use cubic non-uniform B-spline closed curve to fit the boundaries of arbitrary-shaped text instances. By adopting the anchor-free method as the regression detector to obtain the coordinates of B-spline curve control points, and using the segmentation method to obtain the knot vector value, our method not only uses the detection efficiency of regression method, but also combines the insensitivity of segmentation method to arbitrary shape text to improve the accuracy of text detection. Experiments on ICAR2015, CTW1500 and total-text benchmarks, including regular shape and arbitrary shape scene text in natural images, demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning-based stereo matching has made significant progress, but it still faces challenges: The disparity prediction error maps of current models show that errors are concentrated primarily on object boundaries. We find that executing the smooth L1 loss function on the entire region during stereo matching model training cannot effectively address the imbalance between edge regions and flat regions, resulting in poor disparity estimates for edge regions. In this paper, a new weighted smooth L1 loss function, which considers the loss function calculation on edge regions and can yield improved accuracy, is proposed. An improved bilateral grid upsampling module is also added to the training model, and a strategy is adopted to balance the computational consumption introduced by the new loss function-weighted item, allowing for real-time inference. Extensive experiments conducted on two datasets, i.e., Scene Flow and KITTI, verify the simplicity and effectiveness of this approach. Under the condition of 33 frames per second (FPS), the endpoint error of the proposed model can be improved to 0.63. In addition, the proposed edge-based loss function can be easily embedded into many existing stereo matching networks, such as GwcNet, AANet, and PSMNet. After embedding the proposed edge-based loss function, the reduction rates of the endpoint errors of the existing models can be improved to 3.5%, 11.6%, and 27.2% for GwcNet, AANet, and PSMNet, respectively. [ABSTRACT FROM AUTHOR]

3D shape processing is a fundamental computer application. Specifically, 3D mesh could provide a natural and detailed way for object representation. However, due to its non-uniform and irregular data structure, applying deep learning technologies to 3D mesh is difficult. Furthermore, previous deep learning approaches for 3D mesh mainly focus on local structural features and there is a loss of information. In this paper, to make better mesh shape awareness, a novel deep learning approach is proposed, which aims to full-use the information of mesh data and exploit comprehensive features for more accurate classification. To utilize self-attention mechanism and learn global features of mesh edges, we propose a novel attention-based structure with the edge attention module. Then, for local feature learning, our model aggregates edge features from adjacent edges. We refine the network by discarding pooling layers for efficiency. Thus, it captures comprehensive features from both local and global fields for better shape awareness. Moreover, we adopt spatial position encoding module based on spatial information of edges to enhance the model to better recognize edges and make full use of mesh data. We demonstrate effectiveness of our model in classification tasks with numerous experiments which show outperforming results on popular datasets. [ABSTRACT FROM AUTHOR]

Low-light image enhancement based on deep learning has achieved breakthroughs recently. However, the current methods based on deep learning have problems with inadequate resolution enhancement or inadequate contrast. To address these problems, this paper proposes a resolution and contrast fusion network with ConvLSTM (RCFNC) for low-light image enhancement. The network is mainly constructed by four parts, including resolution enhancement branch, contrast enhancement branch, multi-scale feature fusion block (MFFB), and convolution long short-time memory block (ConvLSTM). Specifically, to improve the resolution of the low-light image, a resolution enhancement branch consisting of multi-scale differential feature blocks is proposed, using residual features at different scales to enhance the spatial details of image. To enhance the contrast of the image, a contrast enhancement branch consisting of adaptive convolution residual blocks is introduced to learn the mapping relationship between global and local features in the image. In addition, a weighted fusion is performed using MFFB to better balance the resolution and contrast features obtained from the above branches. Finally, to improve the learning capability of the model, ConvLSTM is added to filter redundant information. Experiments on the LOL, MIT5K, and five benchmark datasets show that RCFNC outperforms current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Logistics parcel detection technology is critical for unmanned sorting. YOLOv5, the state-of-the-art (SOTA) object detection model, is a classic network widely used in engineering. However, on the premise of fast and accurate detection of logistics parcels, it confronts certain challenges of high computing load and parameter quantity. To address these issues, this paper proposes a two-scale lightweight deep learning model named SFYOLOv5. In this article, a lightweight feature extraction module called Pruned-Shuffle-Block (PSB) is proposed. Meanwhile, a double-layer pyramid structure for the medium and large target detection is created in accordance with the image size distribution of logistics parcels. With this structure, the Floating Point of Operations (FLOPs) and parameters in feature extraction were significantly reduced. In addition, a down sampling module named Focus for Downsampling (FFD) is designed and attention modules are introduced to extract high-level semantic information in logistics parcels. These modules not only compensate for the loss caused by down sampling but also improve the mean Average Precision (mAP). Finally, the comparison experiment is performed by using the self-built logistics parcel dataset. The results show that the mAP of the model reaches 99.1%, the number of model parameters decreased by 92.14%, and the FLOPs decreased by 89.57% compared with the existing SOTA model. This model can be used in logistics parcel intelligent sorting. [ABSTRACT FROM AUTHOR]

Multiple object tracking (MOT) in unmanned surface vehicle (USV) videos has many application scenarios in the military and civilian fields. State-of-the-art MOT methods first extract a set of detections from the video frames, then utilize IoU distance to associate the detections of current frame and tracklets of last frame, and finally adopt linear Kalman filter to estimate the current position of tracklets. However, some problems in USV videos seriously affect the tracking performance, such as low frame rate, wobble of observation platform, nonlinear motion of objects, small objects and ambiguous appearance. In this paper, we fully explore the motion cue in USV videos and propose a simple but effective tracker, named MotionTrack. Equipping with YOLOv7 as object detector, the data association of MotionTrack is mainly composed of cascade matching with Gaussian distance module and observation-centric Kalman filter module. We validate the effectiveness with extensive experiments on the recent Jari-Maritime-Tracking-2022 dataset, achieving new state-of-the-art 46.9 MOTA, 49.2 IDF1 with 35.2 FPS running speed on a single 3090 GPU. The source code, pretrained models with deploy versions are released at https://github.com/lzq11/MotionTrack. [ABSTRACT FROM AUTHOR]

In this paper, a novel data-centric approach is proposed for solving the 3D mesh segmentation problem. The method uses node2vec, a semi-supervised learning algorithm, to create vector embedding representations for each node in a 3D mesh graph. This makes the mesh data more compact and easier to process which is important for reducing computation costs. K-Means clustering is then used to cluster each node according to their node embedding information. This data-centric approach is more computationally efficient than other complex models such as CNN and RNN. The main contribution of this study is the development of a data-centric AI framework that combines node2vec embedding, machine learning, and deep learning techniques. The use of cosine similarity is also adapted to compare and evaluate the trained node embedding vectors with different hyperparameters. Additionally, a new algorithm is developed to determine the optimal cluster number using geodesic distance on the 3D mesh. Overall, this approach provides competitive results compared to existing mesh segmentation methods. [ABSTRACT FROM AUTHOR]

Deep learning shows great potential in low-light image enhancement, which can improve image brightness and contrast while keeping image natural. However, due to the lack of prior knowledge extracted manually or excessive amplification of noise, these methods result in poor quality of enhanced images. To solve these challenging problems, this paper proposes a dual-branch grayscale image enhancement network based on dense residual and attention mechanism. Low-light grayscale image is used as input. Firstly, features that fuse deep and shallow information are extracted through dense residual convolution branch network; secondly, texture features are extracted through U-Net branch network combined with attention mechanism; then the extracted features are integrated, and finally luminance is adjusted by a brightness adjustment module to output an enhanced grayscale image. In addition, a joint loss function is designed to measure the network training loss from brightness, texture, contrast and noise. A large number of quantitative and qualitative experiments on LOL and VE-LOL datasets show that the proposed method improves Peak Signal to Noise Ratio Index and Structural Similarity Index by 19.65 - 59.76% and 5.61 - 85.53%, respectively, compared with EnlightenGAN, KinD++, RUAS, LLFlow, etc. The proposed method is superior to the most famous methods, thanks to the deep feature extraction and fusion capability of dense residual convolutional network and texture extraction capability of U-Net network combined with attention mechanism. [ABSTRACT FROM AUTHOR]

In videos, anomaly detection is challenging due to its diverse nature in different application domains. Reconstruction and prediction-based methods have been widely employed to detect anomalies. Due to the generalization capability of a deep neural network, sometimes, it recreates irregular patterns along with regular ones. This paper presents a novel autoencoder-based framework called deep multiplicative attention-based autoencoder (DeMAAE) to detect anomalies in a video sequence. The global attention mechanism is used at the decoder side of DeMAAE for better feature learning during the decoding phase. An attention map is created by taking the dot product between all encoder's hidden states and the previously generated decoder's hidden state. After that, the final output of the decoder is determined by the context vector. The context vector is computed using the weighted summation of all encoder's hidden states and attention weight. DeMAAE delivers an improved runtime of 0.015 s (∼ 67 fps) for detecting anomalies during testing. Extensive experiments have been performed on the two diversified and widely used datasets (UCSD Pedestrian and CUHK Avenue) to compare the efficacy of DeMAAE with different state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, deep learning networks have achieved prominent success in the field of multi-exposure image fusion. However, it is still challenging to prevent color distortion and blurry edges which leads to bad visual effects. In this paper, we present a multi-scale attention-guided network for multi-exposure image fusion in a coarse-to-fine manner. The network generates multi-scale enhanced attention weight maps of images in different sizes which possess vital details and can emphasize essential regions of interest from both sides. The multi-scale structure can extract features on different scales, and the bilayer structure can extract features from different image sizes. Moreover, we designed a coarse-to-fine attention module to finally generate the weight maps; the module combines channel attention with spatial attention. Fused results will be generated under the guidance of the weight maps. Qualitative and quantitative experiments are performed on a publicly available dataset which shows our method outperforms the state-of-the-art methods in visual effect and objective analysis. Also, ablation experiments prove each part of our method has a great advantage in generating images with significant details, prominent targets, and faithful color. [ABSTRACT FROM AUTHOR]

Multi-exposure fusion (MEF) is widely researched and applied to high dynamic range (HDR) imaging, where one of the most challenging problems is the artifacts caused by the motion of objects between input images. Recently, deep learning methods have been widely applied for HDR imaging with excellent results, showing significant advantages. However, many methods cannot avoid artifacts due to inaccurate alignment before merging HDR images. In this paper, we propose an end-to-end network (C-ED-GMNET) with a channel and spatial attention network, an encoder–decoder network and a gradual merging network for generating artifact-free HDR images in dynamic scenes. The attention module consists of two submodules to identify useful features and exclude harmful components in inputs from both channel and spatial dimensions, respectively. The attention-guided feature maps are sent to the encoders for further feature extraction, and then, the outputs are sent to a gradual merging module consisting of two steps to generate deep features progressively. Besides, the differences between the merged image and the original images are identified by applying global residual learning with all the inputs, and the merged image feature is recovered by the decoder to obtain the final HDR image. Quantitative and qualitative experiments on two public datasets show that the proposed C-ED-GMNET produces better results than existing state-of-the-art methods and significantly reduces the runtime due to the encoders which reduce the amount of computation. [ABSTRACT FROM AUTHOR]

Nowadays, salient object detection methods based on deep learning have become a research focus. Therefore, how to reveal the representation mechanism and association rules of features at different levels and scales in order to improve the accuracy of salient object detection is a key issue to be solved. This paper proposes a salient object detection method to boost multi-scale information on multi-level aggregated features, which can accurately and flexibly aggregate multi-scale feature information via effective multi-level feature utilization. First, a scalable feature pyramid module is proposed, which can aggregate deep feature information in shallow features, thus obtaining aggregated features between different levels. Then, the global information enhancement module is built in the bottom-up network path to make up for the lost or weakened feature information in the process of multi-scale integration feature transmission. Next, internally convolve each level of aggregated feature via self-interactive module to enrich multi-scale information and improve the multi-scale representation ability of aggregated features. Finally, the global associativity loss function is designed to solve the noise caused by multi-scale variation so as to optimize the network training process, which effectively compensates for the deficiency of cross-entropy in the salient object detection task. The experimental results on four public datasets show that the performance of the proposed method has improved in contrast to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accurately localizing bare-hands is crucial for human–computer interaction systems. In real-time systems, computational time plays a pivotal role in achieving this task within a specified timeframe. The localization process may involve detection, tracking, or both, depending on the framework's needs. Most studies on bare-hand detection and tracking have been conducted in controlled environments. However, localizing bare-hands in uncontrolled environments are challenging due to the complexity of variations, such as changes in illumination, rotation, occlusion, scale, pose, speed, and impostor bare-hands. These factors can significantly impact the performance of the models, along with background feature domination effect and motion blur, which further complicate localization. To address these challenges, this paper presents a comprehensive analysis of the most significant deep learning-based hand localization models. We have customized an object detector as a bare-hand localization model by incorporating detection and tracking modules, providing computationally efficient performance while addressing the variations. To evaluate the models' efficiency across a range of scales, we have proposed a scale-based bare-hand database with varying scales from 20 to 200 cm. We have also evaluated these models on various bare-hand benchmark databases. [ABSTRACT FROM AUTHOR]

With the gradual growth of deep learning in machine vision, efficient extraction of 3D point clouds becomes significant. The raw data of the 3D point cloud are sparse, disordered, and immersed in noise, which makes it difficult to classify and segment. Whether 3D point clouds can be classified and segmented or not, the local feature is an essential ingredient. Therefore, this paper proposes a GateNet-based PointNet++ network (G-PointNet++). G-PointNet++ extracts local features more accurately than PointNet++ by suppressing irrelevant features and emphasizing important features. Meanwhile, it refines the feature adaptively. Besides, the SENet and attention mechanism are introduced into PointNet++. G-PointNet++ was evaluated on the public ModelNet dataset, ShapeNet dataset, and S3DIS dataset, and its effectiveness in classification and segmentation tasks was verified. In the classification task, G-PointNet++ achieves an overall classification accuracy (OA) of 95.5% on ModelNet10 and 93.3% on ModelNet40. In the segmentation task, the mIoU of G-PointNet++ reaches 85.5% on ShapeNet. These experiments show that G-PointNet++ achieves better performance and saves more time than PointNet++, and its overall accuracy is higher than that of PCT network on ModeNet40. [ABSTRACT FROM AUTHOR]

Rice which is a staple food crop in most Asian countries mainly suffers from higher yield loss due to different factors, and one of the common factors that affect rice yield is rice leaf disease. Many countries do not allow the plantation of new rice varieties unless they are disease-resistant. Hence, disease control serves as an important factor for rice production and this can be effectively accomplished by identifying the rice disease at an early stage which is required for timely pesticide application and disease control. The rice leaf diseases are mainly inspected by the farmers with their naked eye which is error-prone, time-consuming, and often leads to huge yield loss if predicted wrong. This process is effectively handled by different researchers via the use of computer vision and machine learning techniques. To overcome the drawbacks associated with manual processing and low recognition accuracy of different statistical and machine learning techniques, this paper presents a novel concept that uses 3D 2D CNN for feature extraction and an improved backtracking search (IBS) algorithm-optimized deep generative adversarial network (DGAN) for classification. The 3D2D deep convolutional neural network (DCNN) is formed by integrating the 3D fast-learning block with a 2DCNN for extracting the rice disease features such as lesions and shape. The IBS algorithm-optimized GAN architecture is used for rice disease classification and the IBS algorithm is mainly adopted to overcome the instability and overfitting issues that arise in the DGAN. The proposed IBS-optimized GAN architecture offers improved accuracy of 98.7% which is relatively higher than the existing techniques such as extreme gradient boosting (XGBoost), transfer learning techniques, and support vector machine (SVM). [ABSTRACT FROM AUTHOR]

Intelligent recognition of traffic signs and markings is an important component of autonomous driving and intelligent transportation systems. It is also an important theoretical basis for autonomous driving path planning. To address the low accuracy and poor real-time performance of the traffic signs and markings detection in complex and multivariate scenes, a lightweight convolutional neural network recognition method in multiple interference scenes is proposed. Firstly, based on Gamma correction and contrast limited histogram equalization algorithm, the traffic signs and markings image under the low illumination condition is enhanced adaptively. Then, the MobileNet-V2 fuse together with DeepLab-V3 + algorithm to segment traffic signs and markings in multiple scenes. Finally, an identification model is established to realize the adaptive recognition of traffic signs and markings, which are based on Lightweight Convolutional Neural Networks (Lw-CNN). Besides, the recognition method proposed in this paper is verified by the CSUST Chinese Traffic Sign Detection Benchmark (CCTSDB). The experimental results show that the Mean Intersection Over Union (MIOU) of the MobileNet-V2 and DeepLab-V3 + algorithm reaches 83.07%, it increased by 25.8% than before image enhancement. The recognition accuracy of algorithm based on lightweight convolutional neural network is 99.92%, higher than the algorithms of MobileNet-V2 and VGG16. [ABSTRACT FROM AUTHOR]

Unlike 2D images that are represented in regular grids, 3D point clouds are irregular and unordered, hence directly applying convolution neural networks (CNNs) to process point clouds is quite challenging. In this paper, we propose a novel deep neural network named AKNet to achieve point cloud semantic segmentation. The key to our AKNet is the attentive kernel convolution (AKConv), which is a deformed convolution operation for perceiving sufficient local context of 3D scenes. AKConv first constructs the Basic Weight Units that are robust to point's ordering. Then, for capturing the more distinctive local features, the convolution kernels of AKConv are associated with Attentive Weight Units through the self-attentive function acting on Basic Weight Units. Furthermore, 3D point clouds provide richer geometric shape information, which is helpful to recognize objects. However, inputting only raw point features to the convolution function could cause geometric information loss. Thus, we utilize augmented features as input of AKConv. Besides, to preserve the semantic information from the encoding to decoding layers, we introduce the backward encoding (BE) mechanism by utilizing higher-layer semantic features. We conduct experiments on three large-scale point clouds datasets. The experimental results demonstrate that our AKNet outperforms state-of-the-art (SOTA) networks. [ABSTRACT FROM AUTHOR]

Three-dimensional object detection plays a key role in autonomous driving, which becomes extremely challenging in occlusion situations. This paper presents a novel multimodal 3D object detection framework which fuses visual semantic information and depth point cloud information to accurately detect targets with distant object features and occlusion situations. The framework consists of the four steps. Firstly, an improved semantic segmentation network is used to extract semantic information of objects containing similar features. Secondly, semantic images and point clouds are combined to generate pixel-level fusion data so that the semantic information and training capability of sparse and far-point clouds can be improved. Thirdly, a deep learning-based point cloud classification network is used for training of the fused data to output accurate detection frames. Fourthly, an extended Kalman filter is incorporated into point cloud prediction for image-based object detection to further enhance the robustness of object detection. Both Cityscapes and KITTI datasets are used in ablation study and experiments to validate the effectiveness of the proposed framework. [ABSTRACT FROM AUTHOR]

Thangka is a kind of ancient painting that originated from Tibetan culture. Considering its value in cultural and historical fields, digital protection and restoration of Thangka images are important. In this paper, we propose an automatic colorization framework for a Thangka sketch. First, we propose a network structure with a feedback generator to restore the true color of a Thangka painting based on semantic matching. Second, we design a color gradient algorithm for Thangka coloring. Finally, we construct an automatic coloring framework for Thangka paintings that supports user customization. The experimental results show that the algorithm has a highly accurate response to a user's selection process. Compared with traditional methods, the proposed method improves the image quality by 22.3% on average. The data for our approach are publicly available at https://github.com/wangfubo123/SMAC-CGAN. [ABSTRACT FROM AUTHOR]

Micro-expression is an expression that reveals one's true feelings and can be potentially applied in various domains such as healthcare, safety interrogation, and business negotiation. The micro-expression recognition is thus far being judged manually by psychologists and trained experts, which consumes a lot of human effort and time. Recently, the development of the deep learning network has proven promising performance in many computer vision-related tasks. Amongst, micro-expression recognition adopts the deep learning methodology to improve the feature learning capability and model generalization. This paper introduces a lightweight apex-based enhanced network that improves by extending one of the state-of-the-art, shallow triple stream three-dimensional CNN. Concretely, the network is first pre-trained with a macro-expression dataset to encounter the small data problem. The features were extracted from CASME II, SMIC, and SAMM datasets. Moreover, thorough recognition results comparison of the datasets are the optical flow-guided features. Besides, an eye masking technique is introduced to reduce noise interference such as eye blinking and glasses reflection issues. The results obtained have an accuracy of 79.19 % and an F1-score of 75.9 % . Comprehensive experimentation had been conducted on the composite dataset that consists of is provided by comparing it with recent methods. Detailed qualitative and quantitative results are reported and discussed. [ABSTRACT FROM AUTHOR]

To solve the problems of complex surface texture of magnetic tile, cumbersome process of traditional detection algorithm and low detection accuracy, this paper proposes a deep learning-based detection model: RC-YOLOv5s. The model incorporates two new structures: Res-Head and Drop-CA, where Res-Head enhances the feature fusion and information exchange between different layer structures and Drop-CA alleviates the case that the model pays too much attention to the defect target and reduces the missed detection rate of the model. Compared with YOLOv5s, the detection accuracy of the proposed model is improved by 1.83%, the missing rate is reduced from 1.673 to 0.372%, and the average detection frame rate of a single image reaches 41.67, which meets the requirements of real-time and accuracy of tile surface detection. [ABSTRACT FROM AUTHOR]

Pedestrian tracking is highly relevant to the understanding of static and moving scenes in video sequences. The increasing demand for people's safety and security has resulted in more research on intelligent visual surveillance in a wide range of applications, such as moving human detection and tracking. With the great success of deep learning methods, researchers decided to switch from traditional methods based on hand-crafted feature extractors to recent deep-learning-based techniques in order to detect and track people. In this work, the topic of person detection using a Top-view moving fisheye camera is addressed using a deep learning detector combined with a centroid technique in order to track a selected person. Although the fisheye camera is a useful tool for video monitoring, most object detection techniques use classical perspective cameras, with (or without) deep learning. However, due to the distortions of fisheye images, we expect to have higher requirements and challenges on pedestrian detection using this type of device. In this paper, an end-to-end people detection learning method is proposed; it is based on a YOLOv3 detector that detects people using oriented bounding boxes. The proposed model customizes the traditional YOLOv3 for the detection of oriented bounding boxes, by regressing the angle of each bounding box using a periodic loss function. With rotation bounding box prediction, the new approach is efficient, reaching 98.1% of true detection. This detection model is combined with a centroid tracker in order to track a single person by identifying the trajectory, estimated angle of rotation and target distance. Finally, the proposed method is evaluated on a new available dataset where rotated bounding boxes represent annotations from several fisheye videos. [ABSTRACT FROM AUTHOR]

Blind image deblurring is a challenging image processing problem, and a proper solution for this problem has many applications in the real world. This is an ill-posed problem, as both the sharp image and blur kernel are unknown. The traditional methods based on maximum a posterior (MAP) apply heavy constraints on the latent image or blur kernel to find the solution. However, these constraints are not always effective; meanwhile, they are very time-consuming. Recently, new approaches based on deep learning have emerged. The methods based on this approach suffer from two problems: the need for a large number of images and kernels for training and also the dependency of the result on the training data. In this paper, we propose a multiscale method based on MAP framework for image motion deblurring. In this method, we represent the blurry image in different scales. We suggest segmenting the image of each scale using κ -means clustering. Using the image information at dominant edges guided by the segmented images, the blur kernel is estimated at each scale. The blur kernel at the finest level of the pyramid is estimated from the coarser levels in a coarse-to-fine manner. Unlike the existing MAP-based methods, the proposed method does not need mathematically complicated assumptions to estimate the intermediate latent image. So the proposed image deblurring is run fast. We evaluated the proposed method and compared it to the existing methods. The experimental results on real and synthetic blurry images demonstrate that the proposed scheme has promising results. The proposed method competes with the existing MAP-based methods for reconstructing qualitative sharp images, while the execution time for our method is considerably less. [ABSTRACT FROM AUTHOR]

Advances in communication and information technologies have allowed for improvements in the distribution and management of several types of imaging in digital medical environments. The scientific literature has reported data hiding methods that can contribute to improving medical image management and mitigate information security risks. This paper proposes a secure management scheme for retinal imaging based on deep learning, reversible data hiding and zero-watermarking. To create a proper link between a patient and their retinal image, a unique feature is obtained through retina vessel segmentation and optic disk detection using U-Net and RetinaNet deep learning architectures, respectively. The unique feature, in conjunction with a halftoned version of the patient's image, are employed to generate a zero-watermarking code using a zero-watermarking technique based on message digest, spread spectrum, and seam-carving methods. Finally, using a color channel of the retinal image, the zero-watermarking code is concealed in a reversible manner using a data hiding technique based on code division multiplexing. The proposed method ensures patient authentication and verification of integrity, and avoids detachment between the patient and their retinal image. Experimental results show the contribution of the proposed scheme to and its efficiency in retinal image management. [ABSTRACT FROM AUTHOR]

Handwritten character recognition (HCR) is a challenging task because of high intra-class dissimilarity and high inter-class similarity among the character images. Therefore, the features that are used to represent the character images should be such that they maximize the inter-class variations and minimize intra-class variations. In this paper, a multilevel approach for recognition of Meitei Mayek handwritten characters using fusion of features strategy is presented. The proposed methodology has two levels of recognition. The first level employs a simple CNN. A method to identify the structurally similar character classes based on the probability values of the softmax function is also proposed. The fused feature set is employed in the second-level recognition to distinguish between these character classes, thereby enhancing the discriminative power of the fused feature set by reducing the search space in the second level. The fused feature set is a combination of traditional handcrafted feature descriptors and deep features learned by CNN. The fused feature set is fed to an SVM for second-level recognition. The experimental results show a superior performance of the proposed methodology in identifying the structurally similar character classes. This achieves an overall recognition accuracy of 97.08%, which sets the benchmark on the concerned Meitei Mayek handwritten character dataset. The methodology also shows significant results on MNIST, DIDA and CArDIS datasets. [ABSTRACT FROM AUTHOR]

Kinship verification is a challenging computer vision task that aims to mainly answer the question: “Are these two persons blood relatives?”. It is an important area of research with many applications, including the identification of criminals or wanted suspects through their relatives, genealogical studies, face recognition improvements, scrapbook organization, character selection for movies, the search for missing family members, and social media analysis. Despite the significant advancements to date, automatic kinship verification remains challenging to accomplish. Moreover, the datasets used to evaluate the developed kinship verification methods reveal intrinsic problems, including changes in the subject’s facial information caused by age variation, gender and ethnic differences, as well as external difficulties that are mainly related to the acquisition parameters, such as the various imaging settings and the uncooperative subjects (datasets collected from different sources). Recently, due to the great success of the deep learning models for image and video classification, kinship verification methods have seen a significant shift from the classical metric learning and machine learning methods toward both deep learning and also hybrid methods (ensembles of different methods). Therefore, this paper presents an overview of kinship verification using facial information methods as well as the available image and video datasets used to evaluate them. Besides, an experimental study is carried out to highlight the most efficient methods. Finally, a discussion is provided to weigh up the strengths and limitations of the recent kinship verification approaches and suggest future research directions.Graphic abstract: Kinship verification is a challenging computer vision task that aims to mainly answer the question: “Are these two persons blood relatives?”. It is an important area of research with many applications, including the identification of criminals or wanted suspects through their relatives, genealogical studies, face recognition improvements, scrapbook organization, character selection for movies, the search for missing family members, and social media analysis. Despite the significant advancements to date, automatic kinship verification remains challenging to accomplish. Moreover, the datasets used to evaluate the developed kinship verification methods reveal intrinsic problems, including changes in the subject’s facial information caused by age variation, gender and ethnic differences, as well as external difficulties that are mainly related to the acquisition parameters, such as the various imaging settings and the uncooperative subjects (datasets collected from different sources). Recently, due to the great success of the deep learning models for image and video classification, kinship verification methods have seen a significant shift from the classical metric learning and machine learning methods toward both deep learning and also hybrid methods (ensembles of different methods). Therefore, this paper presents an overview of kinship verification using facial information methods as well as the available image and video datasets used to evaluate them. Besides, an experimental study is carried out to highlight the most efficient methods. Finally, a discussion is provided to weigh up the strengths and limitations of the recent kinship verification approaches and suggest future research directions. [ABSTRACT FROM AUTHOR]

Single-view 3D face reconstruction refers to recovering 3D information of a face, such as shape and texture, from a single image. With the wide application of deep learning in the image field, there have been a number of researches using this method to learn the 3D shape and texture of a face from image information. In this paper, we propose a self-supervised single-image 3D face reconstruction method based on the attention mechanism and attribute refinement, which incorporates the attention mechanism in the network structural model, allowing feature extraction to fuse the information of the channel domain and the spatial domain to enhance the feature extraction capability. Joint 2D image-level supervision and supervision between 3D attributes can better learn the 3D model of the face. In this paper, on the basis of using the traditional 2D image supervision, we design a variety of loss functions by combining the cyclic consistency, interpolation consistency, and landmark consistency to realize the 3D attribute level supervision. In order to strengthen the ability to characterize the details of the face, this paper proposes an attribute refinement network to enhance the ability of the model to reconstruct the details and make the reconstruction results more realistic. Based on the symmetry of the face, this paper constructs a deep learning network model to decouple the 3D information directly from the image, and finally realizes unsupervised 3D face reconstruction from a single image. [ABSTRACT FROM AUTHOR]

In the production process of HDMI cables, manual intervention is often required, resulting in low production efficiency and time-consuming. The paper presents a real-time vision-based automatic inspection system for HDMI cables to reduce the labor requirement in the production process. The system consists of hardware and software design. Since the wires in HDMI cables are tiny objects, the hardware design includes an image capture platform with a high-resolution camera and a ring light source to acquire high-resolution and high-quality images of the wires. The software design includes a data augmentation system and an automatic HDMI wire-split inspection system. The former aims to increase the number and diversity of training samples. The latter is designed to detect the coordinate position of the wire center and the corresponding Pin-ID (pid) number and output the results to the wire-bonding machine to perform subsequent tasks. In addition, a new HDMI cable dataset is created to train and evaluate a series of existing detection network models for this study. The experimental results show that the detection accuracy of the wire center using the existing YOLOv4 detector reaches 99.9%. Furthermore, the proposed system reduces the execution time by about 38.67% compared with the traditional manual wire-split inspection operation. [ABSTRACT FROM AUTHOR]

Image cartoonization, a special kind of style transformation, is a challenging image processing task. Most existing cartoonization methods aim at single-style transformation. While multiple models are trained to achieve multi-style transformation, which is time-consuming and resource-consuming. Meanwhile, existing multi-style cartoonization methods based on generative adversarial network require multiple discriminators to handle different styles, which increases the complexity of the network. To solve the above issues, this paper proposes an image cartoonization method for multi-style transformation based on style condition information, called MCLGAN. This approach integrates two key components for promoting multi-style image cartoonization. Firstly, we design a conditional generator and a multi-style learning discriminator to embed the style condition information into the feature space, so as to enhance the ability of the model in realizing different cartoon styles. Then the new loss mechanism, the conditional contrastive loss, is used strategically to strengthen the difference between different styles, thus effectively realizing multi-style image cartoonization. At the same time, MCLGAN simplifies the cartoonization process of different styles images, and only needs to train the model once, which significantly improves the efficiency. Numerous experiments verify the validity of our method as well as demonstrate the superiority of our method compared to previous methods. [ABSTRACT FROM AUTHOR]

Hypertensive retinopathy (HR) can potentially lead to vision loss if left untreated. Early screening and treatment are critical in reducing the risk of vision loss. The computer-aided diagnostic system presents an opportunity to improve the efficiency and reliability of HR screening and diagnosis, particularly given the shortage of specialized medical professionals and the challenges faced by primary care physicians in making precise diagnoses. A notable barrier to the development of such diagnostic algorithms is the lack of publicly available benchmarks and datasets. To address these issues, we organized a challenge named “HRDC—Hypertensive Retinopathy Diagnosis Challenge” in conjunction with the Computer Graphics International (CGI) 2023 conference. The challenge provided a fundus image dataset for two clinical tasks: hypertension classification and HR classification, with each task containing 1000 images. This paper presents a concise summary and analysis of the submitted methods and results for the two challenge tasks. For hypertension classification, the best performing algorithm achieved a Kappa score of 0.3819, an F1 score of 0.6337, and a specificity of 0.8472. For HR classification, the best performing algorithm achieved a Kappa score of 0.4154, an F1 score of 0.6122, and a specificity of 0.8444. We also explored an ensemble approach to the top-ranking methods, which further improved performance beyond the individual best performing algorithm for each task. The challenge results show that there is room for further optimization of these methods, but the insights and methodologies derived from this challenge provide valuable directions for developing more precise and reliable classification models for hypertension and HR. [ABSTRACT FROM AUTHOR]

This paper presents the first method for synthesizing seasonal transition of terrain textures for an input heightfield. Our method reproduces a seamless transition of terrain textures according to the seasons by learning measured data on the earth using a convolutional neural network. We attribute the main seasonal texture transition to vegetation and snow, and control the texture synthesis not only with the input heightfield but also with the annual temperature and precipitation based on Köppen’s climate classification as well as insolation at the location. We found that month-by-month synthesis yields incoherent transitions, while a naïve conditioning with explicit temporal information (e.g., month) degrades generalizability due to the north–south hemisphere difference. To address these issues, we introduce a simple solution—periodic conditioning on the annual data without explicit temporal information. Our experiments reveal that our method can synthesize plausible seasonal transitions of terrain textures. We also demonstrate large-scale texture synthesis by tiling the texture output. [ABSTRACT FROM AUTHOR]

As one of the major complications of diabetic retinopathy (DR), neovascularization of the optic disc (NVD) is a leading cause of visual impairment and blindness. Early identification and timely treatment of NVD are essential to prevent these risks. In this paper, we develop a deep learning (DL) system to identify, quantify, and visualize NVD from optical coherence tomography angiography (OCTA) images. Two datasets of OCTA images were used in this study to develop and evaluate the DL system: (1) 24,576 OCTA images collected from 96 patients with NVD; (2) 15,360 OCTA images from 60 NVD patients with NVD. The task of the DL system involved the detection of the optic disc boundary, the identification of the NVD regions, and the construction and calculation of 3D images for these regions. The DL system achieved promising results in the detection of the optic disc boundary and the identification of NVD regions. The accuracy of the DL system was significantly better than other DL algorithms and comparable to the performance of retina specialists. Furthermore, the DL system could provide a more intuitive 3D image for visualizing the NVD and its blood flow information. [ABSTRACT FROM AUTHOR]

Recognizing hand gestures poses a formidable challenge, particularly when dealing with semantic gestures that require disentanglement prior to recognition. This paper addresses the intricate issue of an additional stroke, commonly referred to as ‘movement epenthesis stroke,’ which emerges between continuous gestures. Our proposed system employs a multifaceted approach to tackle this challenge. Initially, the system extracts color-motion information to facilitate hand detection, subsequently employing a fusion of shape information and a modified Kanade–Lucas–Tomasi (KLT) feature tracker. This integration significantly mitigates the issue of occlusions. The identification of movement epenthesis is accomplished by analyzing the gesture trajectory using a speed profile. Furthermore, self-co-articulation strokes are discerned by leveraging slope-angle information. To enhance the recognition process, a carefully selected set of 40 features is extracted, which are then employed for recognizing the resulting meaningful gestures. These features serve as inputs to various classification models, including support vector machines (SVM), k-nearest neighbors (kNN), and extreme learning machines (ELM). Deep learning algorithms are judiciously deployed to recognize gesture trajectories, thus streamlining the time-consuming feature extraction process. The outcomes of individual classifiers are amalgamated, resulting in a classifier fusion model. This model is enhanced through majority voting and is used in conjunction with cross-validation results. The experimental analysis culminates in an impressive accuracy rate of 98.88% achieved by the classifier fusion model. This achievement surpasses the performance of individual classifiers, underscoring the effectiveness of our proposed methodology. [ABSTRACT FROM AUTHOR]

Jigsaw puzzle is a casual game that can be used for leisure and stress relief. This paper presents a novel algorithm for quantifying and estimating the time required for users to complete jigsaw puzzle games and providing game difficulty reference for game designers. Firstly, a difficulty quantification model is proposed. Then, based on observation and hypothesis, it is believed that jigsaw puzzle difficulty is related to elements such as texture in the puzzle. Finally, experimental validation demonstrates that jigsaw puzzle difficulty is related to the texture and number of repeated elements in the puzzle. The algorithm is tested on a large amount of jigsaw puzzle game datasets, subsequently verifying its effectiveness and accuracy. The main contribution of this algorithm is to provide a new quantitative evaluation method for jigsaw puzzle game difficulty, which can assist game designers in optimizing game difficulty and enhancing user experience. Our data, code, and model are available at CunHua-YYT/JigsawSort (github.com). [ABSTRACT FROM AUTHOR]

Face expression recognition in the wild faces challenges such as small data size, low quality images, and noisy labels. In order to solve these problems, this paper proposes a novel structural self-contrast learning method (SSCL) based on adaptively weighted negative samples. Firstly, two augmented variants are generated for each input sample as positive samples, which should be as similar in structure as possible. Secondly, EMD (Earth’s Mover Distance) is used to define the structural similarity between the two augmented samples and extract the features in a self-supervised way. Subsequently, the concept of negative samples is further introduced into SSCL, and the augmented variants of other categories are generated as negative samples, which makes the structural similarity between positive samples increase and between negative samples simultaneously decrease. Finally, an adaptive weighting method for the negative is proposed, which aims to solve the imbalance between positive samples and negative samples, mine difficult negative samples, and reduce their similarity with the input image in an adaptive way. The extensive experiments on benchmark datasets demonstrate that SSCL outperforms the state-of-the-art methods. The cross-dataset evaluation shows the superior generalization capability of the proposed method. The code will be available at https://github.com/zjhzhy/SSCL. [ABSTRACT FROM AUTHOR]

Point cloud completion represents a complex task that entails predicting the complete geometry of a 3D shape from a set representation of the partial observational points. This paper presents a novel point cloud completion method, called Cascaded Graph Convolutional Completion Network (CGCN). Our method leverages a cascaded encoder–decoder architecture to predict the missing points from the input, the partial points. These predicted missing points are then concatenated with the input points to form the complete shape. Our architecture consists of two main modules: a Multi-Level Feature Extraction Encoder (MFE) and a Folding-Refinement Decoder (FRD). The MFE is composed of the Edge Feature Extraction Module (EFE) and the Global Feature Extraction Module (GFE). The encoder initiates by using several EFEs to secure multi-level local features through the execution of graph convolutional operations on each point. It then employs a GFE to extract global features from the final level features. Consequently, the MFE aggregates features containing local information into global features, offering an output that carries both local and global features. Our FRD contains both a folding block and several refinement blocks. Initially, the folding block morphs 2D grids with features output by the MFE to predict the initial missing points. The decoder then iteratively applies a series of refinement blocks to refine these initial missing points and eventually obtain the output of the predicted missing points. The FRD merges multi-level features and global features to deliver the shape of the predicted missing part. To achieve the complete shape, we integrate the predicted missing part, derived from the FRD and the input points. We then sample this unified point cloud to secure the final output. The effectiveness and competitiveness of our model are validated through experiments on the ShapeNet dataset, using Chamfer Distance (CD) as metrics. [ABSTRACT FROM AUTHOR]

One of the vital characteristics that determine the health condition of a person is the shape and number of the red blood cells, white blood cells and platelets present in one’s blood. Any abnormality in these characteristics is an indication of the person suffering from diseases like anaemia, leukaemia or thrombocytosis. The counting of the blood cell is conventionally made by means of microscopic studies with the application of suitable chemical substances in the blood. The conventional methods pose challenges in the analysis in terms of manual labour and are time-consuming and costly tasks requiring highly skilled medical professionals. This paper proposes a novel scheme to analyse the blood sample of an individual by employing a visual question answering (VQA) system, which accepts a blood smear image as input and answers questions pertaining to the sample, viz. amount of blood cells, nature of abnormalities, etc. very quickly without requiring the service of a skilled medical professional. In VQA, the computer generates textual answers to questions about an input image. Solving this difficult problem requires visual understanding, question comprehension and deductive reasoning. The proposed approach exploits a convolutional neural network for question categorisation and an object detector with an attention mechanism for visual comprehension. The experiment has been conducted with two types of attention: (1) convolutional block attention module and (2) squeeze-and-excitation network which facilitates very fast and reliable results. A VQA dataset has been created for this study due to the unavailability of a public dataset, and the proposed system exhibited an accuracy of 94% for numeric response questions/yes or no type questions and has a BLEU score of 0.91. It is also observed that the attention-based object recognition model of the proposed system for counting the blood characteristics has an accuracy of 97%, 100% and 98% for red blood cell count, white blood cell count and platelet count, respectively, which is an improvement of 1%, 0.06% and 1.61% as compared to the state-of-the-art model. [ABSTRACT FROM AUTHOR]

In practical scenarios, reflections may impair the visual quality of images, bring negative impacts to both human perception and subsequent computer vision tasks. Removing reflections from an image poses considerable challenges due to the diverse nature of reflection content, often blended with background targets. To address the issue, this paper proposes a single-image reflection removal method, which is based on self-attention and local discrimination. By exploiting the mutual relationship between feature maps with a self-attention mechanism, the proposed method can adaptively re-weights each feature according to the input content, benefiting the separation of reflections. For further improvement, a scaled variance map is generated from local statistics and then used in a local discrimination loss to ensure the consistency within local regions. Experimental comparisons have demonstrated the effectiveness and superiority of the proposed method for single image reflection removal. The related code is released at https://github.com/HighColdMan/SIRR_SALD. [ABSTRACT FROM AUTHOR]

Transformer network structures have significantly improved the performance of computer vision (CV) tasks. However, due to the restriction of taking high-dimensional tokens as model inputs and the singularity of tokenization method, it is computationally intensive and easily deteriorates local fine-grained features when recognizing images based on Transformer architecture. In this paper, we incorporate a novel triple self-attention mechanism as a single encoder block and integrate it with the Transformer structure to introduce a new model, namely TripleFormer. Firstly, modeling information inside the window rather than the entire image, we present two sequence inputs from orthogonal perspectives, which are named Patch Attention In Spatial (PAIS) and Patch Attention In Channel (PAIC) for capturing more detailed features. We further partition the multi-channel of one feature map along spatial dimensions and compute attention belonging to same channels. In this way, the proposed encoder block incorporates local feature extraction and long-range visual dependencies to boost the feature learning capability. Finally, experiments on ImageNet-1K and CIFAR100 datasets exhibit the superiority of our proposed models as compared to other methods in terms of lower FLOPs and complexity while maintaining similar accuracy. In addition, our models demonstrate competitive performance on small-scale datasets in comparison to other pure Transformer models. [ABSTRACT FROM AUTHOR]

Over the last decade, convolutional neural networks (CNNs) have allowed remarkable advances in single image super-resolution (SISR). In general, recovering high-frequency features is crucial for high-performance models. High-frequency features suffer more serious damages than low-frequency features during downscaling, making it hard to recover edges and textures. In this paper, we attempt to guide the network to focus more on high-frequency features in restoration from both channel and spatial perspectives. Specifically, we propose a high-frequency channel attention (HFCA) module and a frequency contrastive learning (FCL) loss to aid the process. For the channel-wise perspective, the HFCA module rescales channels by predicting statistical similarity metrics of the feature maps and their high-frequency components. For the spatial perspective, the FCL loss introduces contrastive learning to train a spatial mask that adaptively assigns high-frequency areas with large scaling factors. We incorporate the proposed HFCA module and FCL loss into an EDSR baseline model to construct the proposed lightweight high-frequency channel contrastive network (HFCCN). Extensive experimental results show that it can yield markedly improved or competitive performances compared to the state-of-the-art networks of similar model parameters. [ABSTRACT FROM AUTHOR]

Semantic segmentation plays a crucial role in the fields of computer vision and computer graphics, with extensive applications in various practical scenarios. Significant progress has been made in semantic segmentation tasks using deep learning-based methods. However, most existing semantic segmentation algorithms focus on good weather conditions, and they face challenges in terms of accuracy and robustness when applied to rainy scenes due to factors such as raindrops, haze, and lighting variations. To address this issue, this paper proposes a rainy-day semantic segmentation method based on a two-stage progressive network. The proposed method consists of two modules: a rain removal module responsible for eliminating raindrops and haze from the input rainy images and restoring the basic structural information of the images, and a segmentation module that performs pixel-level semantic prediction on the rain-removed images. Specifically, the rain removal module introduces two progressive units with shared weights to gradually achieve rain removal. The segmentation module adopts an encoder–decoder architecture, utilizing down-sampling and deep asynchronous bottleneck units for encoding. It also introduces a dual attention-guided fusion module to aggregate channel attention information and spatial attention information, guiding the multiscale feature fusion process in the decoder. Experimental results demonstrate that this method effectively mitigates the influence of rain streaks on semantic segmentation, thereby improving segmentation performance and achieving more accurate and robust semantic segmentation results in rainy conditions. We will provide the code and datasets on https://github.com/zhang152267/TSPN. [ABSTRACT FROM AUTHOR]