Search

Image matching is an important task in computer vision. The detector-free dense matching method is an important research direction of image matching due to its high accuracy and robustness. The classical detector-free image matching methods utilize convolutional neural networks to extract features and then match them. Due to the lack of scale equivariance in CNNs, this method often exhibits poor matching performance when the images to be matched undergo significant scale variations. However, large-scale variations are very common in practical problems. To solve the above problem, we propose SeLFM, a method that combines scale equivariance and the global modeling capability of transformer. The two main advantages of this method are scale-equivariant CNNs can extract scale-equivariant features, while transformer also brings global modeling capability. Experiments prove that this modification improves the performance of the matcher in matching image pairs with large-scale variations and does not affect the general matching performance of the matcher. The code will be open-sourced at this link: https://github.com/LiaoYun0x0/SeLFM/tree/main [ABSTRACT FROM AUTHOR]

With the development of image feature matching technology, feature matching algorithms based on deep learning have achieved excellent results, but in scenarios with low texture or extreme perspective changes, the matching accuracy is still difficult to guarantee. In this paper, a superresolution reconstruction method based on a Residual-ESPCN (efficient subpixel convolutional neural network) approach is proposed based on LoFTR (local feature matching with transformers). The superresolution method is used to improve the interpolation method used in ASFF (adaptive spatial feature fusion) to increase the image resolution, enhance the detailed information of the image, and make the extracted features richer. Then, ASFF is introduced into the local feature extraction module of LoFTR, which can alleviate the inconsistency problem of information transmission between different scale features of the feature pyramid and lessen the amount of information lost during transmission from low- to high-resolution levels. Moreover, to improve the adaptability of the algorithm to different scenarios, OTSU is introduced to adaptively calculate the threshold of feature matching. The experimental results show that in different indoor or outdoor scenarios, our proposed algorithm for matching features can effectively improve the adaptability of feature matching and can achieve good results in terms of the area under the curve (AUC), accuracy and recall. [ABSTRACT FROM AUTHOR]

Aiming at the low tracking accuracy and longtime tracking problems of the traditional automatic mobile human body tracking methods, this research proposes an automatic tracking method of the moving human body based on remote sensing spatial information. The proposed automatic tracking method for the moving human body first utilizes remote sensing technology to obtain the spatial information of the moving human body and then constructs a human body target motion model. By segmenting and fusing human body motion images, automatic matching and tracking of human body motion is finally realized. The experimental results showed that the auto-tracking time of the moving human body was only 0.3 s, while the auto-tracking accuracy rate was as high as 98.29%. In summary, the method used in this research has a high human motion tracking recognition effect. In addition, the study still has shortcomings, such as how to maintain high accuracy tracking in low light and bad weather conditions, still need to be studied. [ABSTRACT FROM AUTHOR]

Robust matching, especially the number, precision and distribution of feature point matching, directly affects the effect of 3D reconstruction. However, the existing methods rarely consider these three aspects comprehensively to improve the quality of feature matching, which in turn affects the effect of 3D reconstruction. Therefore, to effectively improve the quality of 3D reconstruction, we propose a circle-based enhanced motion consistency and guided diffusion feature matching algorithm for 3D reconstruction named EMC+GD_C. Firstly, a circle-based neighborhood division method is proposed, which increases the number of initial matching points. Secondly, to improve the precision of feature point matching, on the one hand, we put forward the idea of enhancing motion consistency, reducing the mismatch of high similarity feature points by enhancing the judgment conditions of true and false matching points; on the other hand, we combine the RANSAC optimization method to filter out the outliers and further improve the precision of feature point matching. Finally, a novel guided diffusion idea combining guided matching and motion consistency is proposed, which expands the distribution range of feature point matching and improves the stability of 3D models. Experiments on 8 sets of 908 pairs of images in the public 3D reconstruction datasets demonstrate that our method can achieve better matching performance and show stronger stability in 3D reconstruction. Specifically, EMC+GD_C achieves an average improvement of 24.07% compared to SIFT-based ratio test, 9.18% to GMS and 1.94% to EMC+GD_G in feature matching precision. [ABSTRACT FROM AUTHOR]

In the domain of point cloud registration, the coarse-to-fine feature matching paradigm has received significant attention due to its impressive performance. This paradigm involves a two-step process: first, the extraction of multilevel features, and subsequently, the propagation of correspondences from coarse to fine levels. However, this approach faces two notable limitations. Firstly, the use of the Dual Softmax operation may promote one-to-one correspondences between superpoints, inadvertently excluding valuable correspondences. Secondly, it is crucial to closely examine the overlapping areas between point clouds, as only correspondences within these regions decisively determine the actual transformation. Considering these issues, we propose OAAFormer to enhance correspondence quality. On the one hand, we introduce a soft matching mechanism to facilitate the propagation of potentially valuable correspondences from coarse to fine levels. On the other hand, we integrate an overlapping region detection module to minimize mismatches to the greatest extent possible. Furthermore, we introduce a region-wise attention module with linear complexity during the fine-level matching phase, designed to enhance the discriminative capabilities of the extracted features. Tests on the challenging 3DLoMatch benchmark demonstrate that our approach leads to a substantial increase of about 7% in the inlier ratio, as well as an enhancement of 2%–4% in registration recall. Finally, to accelerate the prediction process, we replace the Conventional Random Sample Consensus (RANSAC) algorithm with the selection of a limited yet representative set of high-confidence correspondences, resulting in a 100 times speedup while still maintaining comparable registration performance. [ABSTRACT FROM AUTHOR]

With the rapid development and extensive application of next-generation image processing technologies, the manufacturing industry is increasingly adopting intelligent equipment. To meet the demands of high precision and high efficiency production, there has been a growing focus on researching 3D point cloud processing methods that go beyond traditional approaches. A fundamental and crucial challenge in the field of point cloud processing is establishing a point-to-point correspondence mapping between two point clouds, which relies on leveraging the local feature description information inherent in the point cloud. This paper thoroughly investigates novel local description methods based on point cloud processing. It addresses the issue of inadequate descriptive capability and robustness found in existing local description methods. Specifically, this study explores the encoding of point information in the neighborhood space and multi-view projection mapping, proposing a local point cloud description method based on depth grids. This method leverages a local reference frame and establishes a depth grid after obtaining the local reference frame through neighborhood projection and distance weighting. The contribution of neighboring points to the depth of the grid is calculated to obtain the eigenvalues. To enhance efficiency, the calculation of eigenvalues incorporates normalization and multi-view projection techniques. The proposed method is compared and evaluated against various local description methods to verify its effectiveness and accuracy. [ABSTRACT FROM AUTHOR]

Computer vision is a catch-all term for a variety of applications. This makes it a good research environment for new ideas and concepts. Feature extraction is considered an essential step in such applications. Several research studies introduced the Scale-Invariant Feature Transform (SIFT) as a benchmark method to extract visual features from objects inside images. This ensures the need for a deep study of SIFT in a variety of settings. Hence, this paper presents an assessment of SIFT from different perspectives that are not explicitly expressed in the literature. In addition, it presents an illustration of the majority of Oriented FAST and Rotated BRIEF (ORB) feature extraction characteristics to facilitate the choice procedure between SIFT and ORB. Several experimental cases are included, each of which evaluates the performance of such methods from distinct and different aspects. At first, the paper presents an assessment of these methods to identify objects inside geometrically–affine-transformed images. This is done by comparing how well their gathered feature descriptors from images perform against one another. Second, this paper presents an evaluation of the invariance of these methods to the changes in illumination. Furthermore, the computational and asymptotic complexity of such methods is investigated to examine its impact on the complexity of any feature-based system. Finally, the efficiency of these methods is verified by assessing their ability to support real-time applications, through the evaluation of their time and space complexities over all investigated test scenarios. [ABSTRACT FROM AUTHOR]

Semantic information associated Simultaneous Localization and Mapping (SIA-SLAM), a visual SLAM algorithm using semantic information association, is proposed to solve the problems that dynamic objects lead to the decreased accuracy of the localization and feature matching between two frames due to the lack of object semantic information. Firstly, a Solov2 instance segmentation network is used to obtain instance segmentation images, and the feature points are extracted from the RBG images simultaneously. Secondly, the feature points on dynamic objects are removed, and the semantic information of static objects is associated with the remaining feature points. Then, the static feature points are utilized to estimate the camera poses and update the static map point set. Finally, the camera poses are optimized by using closed-loop detection. When tracking the camera poses and inter-frame feature matching during the closed-loop detection, the semantic information of the feature points is checked first, and then the bag-of-words model is used for feature matching. The proposed SIA-SLAM algorithm is tested on the Technische Universität München (TUM) public dataset. As far as the absolute trajectory errors (ATE) are concerned, the Root Mean Square Errors (RMSE) and Standard Deviation (S.D.) improvement values can reach up to 98.15% and 98.18% in high dynamic scene of TUM dataset, respectively. The proposed SIA-SLAM algorithm is superior to other semantic SLAM algorithms which are tested in the specific datasets. Furthermore, the reliability and robustness of the SIA-SLAM algorithm are verified in a real scenario. The SIA-SLAM algorithm effectively improves the accuracy of the camera trajectory estimation and feature matching. [ABSTRACT FROM AUTHOR]

The feature matching algorithm based on deep learning has achieved superior performance compared to traditional algorithms in terms of both matching quantity and accuracy, but there are still some high-error matching results in complex scenes, which adversely affects the subsequent work. Based on SuperGlue, we propose an accurate feature matching algorithm via outlier filtering. Firstly, DBSCAN real-time superpixel segmentation (RTSS-DBSCAN) is used to divide the image into regions, and then the outlier filtering module is designed according to the local similarity principle of feature matching. On the premise of not affecting the correct matching results, the matching results with high errors will be filtered to improve the matching accuracy. Meanwhile, due to the lag of traditional Exponential Moving Average algorithm (EMA), an adaptive EMA is designed and integrated into the SuperGlue training process to further improve the training speed and matching accuracy. We evaluate the overall performance of the matching method using the AUC of pose error at the thresholds (5°, 10°, 20°), a common evaluation metric, to provide a more detailed and intuitive evaluation of the matching effectiveness using precision and recall. The experimental results show that the method in this paper can effectively filter the matching results with large errors and has high accuracy and robustness. The AUC of pose error at thresholds (5°, 10°, 20°) reaches 36.53, 56.23, and 73.68, and the precision and recall reach 80.07 and 91.52, respectively, which have better matching results compared with other algorithms. [ABSTRACT FROM AUTHOR]

Most arbitrary style transfer methods only consider transferring the features of the style and content images. Although the pixel-wise style transfer is achieved. It is limited to preserving the content structure, the model tends to transfer the style features, and the loss of image information occurs during the transfer process. The model incline to transfer the style features and preservation of the content structure is weak. The generated pictures will produce artifacts and patterns of style pictures. In this paper, an attention feature distribution matching method for arbitrary style transfer is proposed. In network architecture, a combination of the self-attention mechanism and second-order statistics is used to perform style transfer, and the style strengthen block enhances the style features of generated images. In the loss function, the traditional content loss is not used. We integrate the attention mechanism and feature distribution matching to construct the loss function. The constraints are strengthened to avoid artifacts in the generated image. Qualitative and quantitative experiments demonstrate the effectiveness of our method compared with state-of-the-art arbitrary style transfer in improving arbitrary style transfer quality. [ABSTRACT FROM AUTHOR]

The matching of infrared and visible images has a wide range of applications across various fields. However, the large difference between these two types of images poses a significant challenge to achieving accurate feature matching. In this paper, we introduce a novel feature matching method for infrared and visible images, named MIVI. Our proposed multi-stage matching architecture enables the model to capture both fine local feature details and remote dependencies, while our novel composite loss function optimizes the model at each stage and significantly improves the matching accuracy. Qualitative and quantitative experiments demonstrate that MIVI outperforms other excellent algorithms in terms of accuracy. The code will be released at: https://github.com/LiaoYun0x0/MIVI. [ABSTRACT FROM AUTHOR]

Local feature matching has been a critical task in computer vision applications. In the early days of computer vision, local feature matching relied heavily on detector-based methods, where keypoint detectors were used to extract and describe the salient features of an image. However, with the advent of deep learning, detector-free methods that do not rely on keypoint detection have become increasingly popular. These methods directly learn feature descriptors from the image data, leading to improved performance and faster computation times. In this review, we explore the evolution of local feature matching from detector-based to detector-free methods, discussing the advantages and disadvantages of each approach and highlighting the recent advancements in the field. We also discuss the challenges and opportunities for future research in this area. [ABSTRACT FROM AUTHOR]

Rigid point cloud registration is a crucial upstream task in computer vision, whose goal is to align two misaligned point clouds using a rigid transformation. Existing methods, which directly utilize extracted point features for computing point relationships, are likely to result in a wrong-matching relationship since two or more similar feature points in the source point cloud easily correspond to the same point in the target point cloud. To this end, this paper proposes a two-view point cloud registration network that better alleviates the problem of similar feature points from both the feature and geometry levels. Specifically, at the feature level, a residual correction unit is proposed to learn feature-aware coefficients from raw 3D point clouds to adaptively increase or decrease the difference between features. An attention mechanism is established in two-point clouds to capture the implicit feature relationship between the two-point clouds, gathering the information of another point cloud to enrich the feature information of its own point cloud. Second, at the geometry level, a dual-view graph topology fusion module is described. All points in the graph structure are no longer independent but connected by their geometry structure. Therefore, each point can aggregate neighbor information in the same point cloud and in different point clouds through the constructed single graph and interactive graph, so that each point can enhance the difference between points through its geometry structure. In order to fuse the information of a single graph and an interactive graph, a cross-attention module is proposed to supplement contextual information to obtain point features that are more suitable for matching. Experimental results demonstrate that our method achieves excellent results on complete and partial noisy point clouds. [ABSTRACT FROM AUTHOR]

With the exponential growth of various data interactions on network systems, network intrusions are also increasing. The emergence of edge computing technology brings a new solution to network security. However, due to the difficulty of processing massive and unbalanced data at the edge, higher accuracy requirements are necessary for deployed detection models. This paper proposes a multi-classification model for network intrusion detection based on reconstruction and feature matching. This model can be deployed on small-scale edge nodes, effectively identifying various attack behaviors through the utilization of reconstruction errors and adaptive scaling. Furthermore, we proposed a model transfer method based on feature matching to enhance the training and detection efficiency of multi-classification models under different data distribution conditions. The proposed model has been evaluated on the CICIDS2017 dataset in terms of accuracy, recall, precision and F1 score. The model demonstrates high accuracy for normal flows in the network, majority class attacks, and minority class attacks, achieving an overall multi-class accuracy of 99.81%, outperforming similar models. Furthermore, this model demonstrates faster convergence and training speed after feature matching, exhibiting better robustness and outstanding performance at the edge. [ABSTRACT FROM AUTHOR]

In order to meet the visual characteristics of human eyes, in this paper, we propose a ghost imaging method based on binocular vision matching fusion. The imaging process of the left-eye camera and the right-eye camera in binocular vision system is realized, using ghost imaging algorithm, and its weak signal imaging characteristics can effectively compensate the parallax between the left-eye camera and the right-eye camera. The Speeded Up Robust Features algorithm is used to match the feature points of the two sub-images. Combined with the fusion algorithm of New Sum of Modified Laplacian, the two sub-images after registration are fused to obtain a clearer image. Whether from the visual perception (human vision) or objective analysis (CC, SSIM, IE, RMSE, PSNR, and BER), it can be proved that the proposed scheme has better image quality, better fidelity and robustness, and is closer to the ideal image, which provides a new method for binocular vision imaging technology. [ABSTRACT FROM AUTHOR]

Recovering three-dimensional structure from images is one of the important researches in computer vision. The quality of feature matching is one of the keys to obtaining more accurate results. However, as different objects or different surfaces of objects have similar images with the same elements and different typography, the camera pose estimation will be wrong and the task will fail. This paper proposes a new mismatch elimination algorithm based on global topology consistency. We first formulate the matching task as a mathematical model based on the global constraints, then convert the feature matching into grid matching, calculate the confidence of the grids according to the changes in the angle and displacement between correspondence grid vectors, and remove the mismatches with low confidence. The experiments have demonstrated that our proposed method performs better than the state-of-the-art feature matching methods to accomplish outlier match rejection in the task of similar image matching and could be helped to obtain the correct camera pose to reconstruct more complete and more accurate object models. [ABSTRACT FROM AUTHOR]

Image matching is a crucial step in executing many complex visual tasks, which involve identifying the same or similar visual patterns across various images. Matching images between infrared and visible becomes quite challenging due to the significant non-linear intensity differences. In this paper, we propose using a lightweight network for feature matching of infrared and visible images, combining global and local feature information, and reducing computational costs, enabling real-time inference on most desktop-level GPUs. To fully leverage the powerful matching capabilities of existing state-of-the-art models, we introduce knowledge distillation to obtain more robust features. Moreover, to address the issue of insufficient datasets for network training in existing methods, we propose using image style transfer techniques to synthesize paired datasets of infrared and visible. Experimental results show that our method achieves results comparable to the most advanced methods in infrared and visible image matching. Furthermore, our method has a significant advantage in inference speed, which is beneficial for tasks that require real-time completion. [ABSTRACT FROM AUTHOR]

Nowadays, most artificial intelligence applications are feature-based. Hence, the feature extraction field is regarded as a broad, constantly evolving research field. Many of these applications, including image classification, retrieval, and others, depend on the feature extraction as a crucial step. The feature extraction algorithms encounter several difficulties related to image type, content, size, and others. These difficulties directly influence the efficiency, accuracy, scalability, and speed of applications. Thus, a thorough research of the entire domain of feature extraction methods is required to close the gap between the performance of such methods and the requirements of different applications, especially with the substitution of several feature detection and extraction methods. This paper is a step forward in accommodating and understanding the majority of the visual feature traits to make the choice between extraction methods easier. It introduces a detailed assessment of several feature extraction methods under different conditions and from different perspectives. Consequently, several contributions are included. First, the vulnerability of methods to image redundancy is evaluated through the measurement of similarity matching between calculated descriptors by each method for different types of image content. Second, the method strength is examined against object rotation and flipping within the images. Finally, the efficacy of object distinguishability is assessed in noisy and rotated flipped images. Furthermore, the computational complexity of the methods involved is examined to assess their influence on system complexity, and their efficiency is also ensured. [ABSTRACT FROM AUTHOR]

The aim of the reference-based image super-resolution (RefSR) is to reconstruct high-resolution (HR) when a reference (Ref) image with similar content as that of the low-resolution (LR) input is given. In the task, the quality of existing approaches degrades severely when there are several similar objects but different contents. Besides, not all similar information in the reference image is useful for the input image. Therefore, we propose high-to-low-level feature matching and complementary information fusion (HMCF) network for RefSR. The matching strategy adopts high-level to low-level feature matching to distinguish similar objects but different contents according to high-level semantics. The complementary information fusion module utilizes the channel and spatial attention to select the complement information for LR image and keeps the pixel consistency of input and Ref image. We perform extensive experiments to demonstrate that our proposed HMCF obtains the SOTA performance on the RefSR benchmarks and presents a high visual quality. [ABSTRACT FROM AUTHOR]

Feature matching for multimodal images is an important task in image processing. However, most methods perform image feature detection, description, and matching sequentially, resulting in a large loss, low matching accuracy, and slow performance. To tackle these challenges, we propose a detector-free method called FeMIP for feature matching of multimodal images. We design coarse matching and fine regression modules to implement accurate multimodal image feature matches in a coarse-to-fine manner. Furthermore, we add a novel data augmentation method enabling FeMIP to achieve feature matching faster and more accurately. The coarse-to-fine module automatically generates pixel-level labels on the original image, enabling FeMIP to perform pixel-level matching on data with only image-level labels. In addition, we use the principle of reinforcement learning to design a policy gradient method to improve the solution to the problem of discreteness in matching. Extensive experiments show that FeMIP has good generalization and achieves excellent matching performances. The code will be released at: https://github.com/LiaoYun0x0/FeMIP. [ABSTRACT FROM AUTHOR]

There is a long-standing problem of repeated patterns in correspondence problems, where mismatches frequently occur because of inherent ambiguity. The unique position information associated with repeated patterns makes coordinate representations a useful supplement to appearance representations for improving feature correspondences. However, the issue of appropriate coordinate representation has remained unresolved. In this study, we demonstrate that geometric-invariant coordinate representations, such as barycentric coordinates, can significantly reduce mismatches between features. The first step is to establish a theoretical foundation for geometrically invariant coordinates. We present a seed matching and filtering network (SMFNet) that combines feature matching and consistency filtering with a coarse-to-fine matching strategy in order to acquire reliable sparse correspondences. We then introduce Degree, a novel anchor-to-barycentric (A2B) coordinate encoding approach, which generates multiple affine-invariant correspondence coordinates from paired images. Degree can be used as a plug-in with standard descriptors, feature matchers, and consistency filters to improve the matching quality. Extensive experiments in synthesized indoor and outdoor datasets demonstrate that Degree alleviates the problem of repeated patterns and helps achieve state-of-the-art performance. Furthermore, Degree also reports competitive performance in the third Image Matching Challenge at CVPR 2021. This approach offers a new perspective to alleviate the problem of repeated patterns and emphasizes the importance of choosing coordinate representations for feature correspondences. [ABSTRACT FROM AUTHOR]

Visual object tracking using deep features has achieved great success, particularly when object appearances change in the presence of illumination variation, occlusion, in-plane rotation, scaling, and fast motion. In the state-of-the-art approaches, the common model for object tracking is developed to address appearance variations with coexisting challenges. When the object features differ from appearance variations, the common model approach is ineffective, with simultaneous challenges. To alleviate these limitations, in this paper, a visual object tracking framework is proposed that relies on an adaptive object appearance feature update and template overlap maximization. The tracked object location is identified by performing feature matching between the previous frame's tracked template and the scaled templates' feature vectors in the current frame. Feature vectors are formed using template color and pretrained deep features. To adapt to appearance variations, the proposed tracking model updates the tracked template feature vector and the spatial information for target tracking in the successive frames. The experimental results on challenging video sequences in object tracking benchmarks demonstrate that the proposed tracking model can track objects with precision of 88.7% at a 12 frames/second tracking speed. The qualitative analysis shows that the proposed tracking model outperforms the related conventional trackers. [ABSTRACT FROM AUTHOR]

Recognizing 3D objects based on local feature descriptors, in point cloud scenes with occlusion and clutter, is a very challenging task. Most existing 3D local feature descriptors rely on normal information to encode local features, however, they ignore the normal-sign-ambiguity issue, which greatly limits their descriptiveness and robustness. This paper proposes a method called VOxelization in Invariant Distance space for 3D object recognition. First, we propose a VOID descriptor that is invariant to normal-sign-ambiguity, and is also rotation-invariant, distinctive, robust, and efficient. Second, a VOID-based 3D object recognition method considering the self-similarity between local features is proposed to enhance the recognition performance. Five standard datasets are employed to validate our proposed method as well as comparison with the state-of-the-arts. The results suggest that: (1) VOID descriptor is invariant to normal-sign-ambiguity, distinctive, and robust; (2) VOID-based 3D object recognition achieves outstanding recognition performance, i.e., 99.47%, 93.07% and 99.18%, on the U3OR, Queen's and Ca' Foscari Venezia datasets, respectively. [ABSTRACT FROM AUTHOR]

Loop closure detection(LCD) is an essential part of the visual SLAM system, which can reduce the cumulative error caused by drift. LCD based on traditional methods adopts artificially designed image features. However, this method lacks semantic information and is vulnerable to the external lighting environment. Aiming at the problem of missing information in the image feature representation of LCD, this article proposes a feature extraction method based on multi-layer feature fusion of the lightweight network SqueezeNet. This method can reduce the loss of location and detail information and significantly improve the feature-matching accuracy and extraction rate. Then we employ the nonlinear KPCA method to reduce the dimension of the extracted image feature vectors. In addition, in order to prevent the error matching of adjacent images and the long matching time, we propose an adaptive range matching algorithm, which adaptively limits the matching range in the matching feature stage and jumps out of unnecessary candidate ranges by setting corresponding thresholds and dictionaries of candidate key frames. It not only improves the accuracy of LCD but also dramatically reduces the matching time. The extensive experiments on relevant datasets show that the proposed method has higher accuracy and rate than other methods of CNNs, achieving better robustness and real-time requirements and proving the method’s effectiveness for LCD. [ABSTRACT FROM AUTHOR]

Feature matching is determining true correspondences between image pairs, important for many computer vision applications. It is challenging to determine true correspondences quickly under scene changes in viewpoint, rotation, scaling and illumination. Higher accuracy and efficiency are required for feature matching. While other methods determine true correspondences by treating the images independently, we instead condition on image pairs to take account of the affine information between them.To achieve this, we propose AAM-ORB, an efficient and robust algorithm for feature matching in the scene-shift. The key to our approach is an affine attention module (AAM), which can condition the affine features on both images to boost robustness. AAM is integrated into the well-known ORB feature matching pipeline, resulting in a significant improvement. Although remarkably matching accuracy, AAM can reduce computation efficient. To overcome this, we select a grid-based motion statistics for separating true correspondences from false ones at high speed. Extensive experiments show that AAM-ORB surpasses state-of-the-art approaches for feature matching on benchmark datasets. Moreover, the proposed AAM-ORB has less time consumption. Finally, AAM-ORB achieves better performance and efficiency of feature matching under scene changes. [ABSTRACT FROM AUTHOR]

Aiming at the problems of cupping machine camera with a small field of view, complete and high-definition cupping spot characteristics need to be monitored in real time, so as to effectively ensure that the cupping machine can accurately control the cupping time to avoid harm to the human body and extracting the complete back contour for automatic acupuncture point positioning, a fast stitching method for multi-view image of cupping spots is proposed. The present study uses linear transformation and Gaussian smoothing to preprocess the images to enhance image details, improve contrast, and reduce the influence of tank occlusion; Then, we combine the BRISK algorithm and the match factor to match the image and estimate the overlapping area through establishing a binary tree model so as to improve the efficiency of feature point detection and matching. Experimental results show that compared with the AutoStitch, the image stitched which is high definition has no obvious seams, ghosting, and distortion. What's more, the algorithm in this paper is more efficient and the back contour is more complete. The algorithm in this paper has good real-time performance and the clarity of the cupping spots is high, which is conducive to the subsequent real-time detection of the cupping spots. At the same time, the back contour of the image stitched by the algorithm in this paper is complete, which is conducive to the subsequent automatic acupuncture point positioning. [ABSTRACT FROM AUTHOR]

The transformational and spatial proximities are important cues for identifying inliers from an appearance based match set because correct matches generally stay close in input images and share similar local transformations. However, most existing approaches only check one type of them or both types consecutively with manually set thresholds, and thus their matching accuracy and flexibility in handling large-scale images are limited. In this paper, we present an efficient clustering based approach to identify match inliers with both proximities simultaneously. It first projects the putative matches into a joint transformational-spatial space, where mismatches tend to scatter all around while correct matches gather together. A mode-seeking process based on joint kernel density estimation is then proposed to obtain significant clusters in the joint space, where each cluster contains matches mapping the same object across images with high accuracy. Moreover, kernel bandwidths for measuring match proximities are adaptively set during density estimation, which enhances its applicability for matching different images. Experiments on three standard datasets show that the proposed approach delivers superior performance on a variety of feature matching tasks, including multi-object matching, duplicate object matching and object retrieval. [ABSTRACT FROM AUTHOR]

In pattern matching, estimating the spatial distance between generated feature descriptors is known as "similarity matching". It is a fundamental procedure in many application disciplines. The primary axis for rapid and precise similarity matching is the extraction method, which can effectively represent images with low-dimensional descriptors. In this paper, we evaluate the Scale-Invariant Feature Transform (SIFT) due to its relevance and use in numerous applications. However, it has several limitations. Thus, the usage of Oriented FAST and Rotated BRIEF (ORB) is recommended. In the form of three scenarios, this paper introduces similarity matching performance assessment for each method from several perspectives not explicitly expressed in the literature. The first scenario is introduced to assess the extraction of the same features from multiple images. The method evaluation to identify the K most important keypoints is included in the second scenario. The sorting of the used images hinders the feature extraction method as well as the similarity matching. The third scenario involves evaluating the matching performance of visual features extracted with SIFT and ORB from real and sketched images. Furthermore, this paper introduces a step forward to fill the gap between sketch and real domains by converting real images to pencil-sketch images. Finally, the computational complexity of the overall test cases is computed and assessed. According to the results of real-to-pencil-sketch image matching, ORB outperforms SIFT in terms of the computed performance metrics as well as the correct matching by a factor of around two. Besides, SIFT has a temporal complexity that is nearly double that of ORB, which directly affects the capacity of any application relying on similarity matching. [ABSTRACT FROM AUTHOR]

In this paper, we propose an original algorithm allowing the registration of multimodal images. Indeed, the large and nonlinear intensity changes between different modalities often lead to mismatching of the interest points, which can alter the quality of the registration. Our method builds on the SURF for the identification and the description of the interest points. In order to boost the confidence on the subsequent matching operation of the registration, an applicable classification on both the distance and orientation SURF features is presented despite the low structural similarity of the multimodal acquisition. The proposed algorithm is tested and validated on a large dataset extracted from acquisitions on cultural heritage wall paintings that implement 4 imaging modalities covering UV, IR, visible and fluorescence. The comparison with conventional SIFT/SURF approaches, PIIFD, SURF-PIIFD-PRM,CFOG and HTCPR for registration highlights the effectiveness of the method and its robustness to outliers in SIFT/SURF descriptors. [ABSTRACT FROM AUTHOR]

Objective quality assessment of retargeted images aims to find the best retargeting method for showing an image on different display terminals. This paper uses a Radial Basis Function (RBF) neural network to assess the quality of retargeted images. First, invariant feature points in the original image and their counterparts in the retargeted image are matched in a spatial order-preserving manner. Feature points centered local patches are extracted from original and retargeted images. Then, multi-scale local structural similarity and multi-scale local HSV color histograms difference of matched local patches are measured. A saliency map is employed as the weights of the local patches for evaluating the structural distortion and content change. Finally, the overall assessment of the retargeted image quality can be obtained by the RBF neural network. Experimental results on a benchmark test show the high consistency between the proposed objective assessment and subjective evaluations, and our method is closer to the practical application due to its simplicity compared with those complex ones. [ABSTRACT FROM AUTHOR]

This paper studies graph clustering with application to feature matching and proposes an effective method, termed as GC-LAC, that can establish reliable feature correspondences and simultaneously discover all potential visual patterns. In particular, we regard each putative match as a node and encode the geometric relationships into edges where a visual pattern sharing similar motion behaviors corresponds to a strongly connected subgraph. In this setting, it is natural to formulate the feature matching task as a graph clustering problem. To construct a geometric meaningful graph, based on the best practices, we adopt a local affine strategy. By investigating the motion coherence prior, we further propose an efficient and deterministic geometric solver (MCDG) to extract the local geometric information that helps construct the graph. The graph is sparse and general for various image transformations. Subsequently, a novel robust graph clustering algorithm (D2SCAN) is introduced, which defines the notion of density-reachable on the graph by replicator dynamics optimization. Extensive experiments focusing on both the local and the whole of our GC-LAC with various practical vision tasks including relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multimodel fitting, demonstrate that our GC-LAC is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code for this work is publicly available at: https://github.com/YifanLu2000/GCLAC. [ABSTRACT FROM AUTHOR]

This paper is devoted to the important problem of subsea infrastructure inspection using autonomous underwater vehicles (AUVs). The accomplishment of the inspection mission requires high accuracy of AUV coordination with respect to objects, which is not always provided by standard hydroacoustic equipment. That is why we propose an approach based on stereo image processing that can provide submeter accuracy of AUV coordination required under subsea conditions. In the framework of a unified approach based on the use of an a priori formed point model of an object and the application of a structural coherence criterion, we propose and analyze several methods for recognition of underwater objects and coordinate referencing of the AUV to these objects. As a geometric model of an object, we consider its representation based on characteristic points, which determines the spatial structure of the object. In addition to the standard method for object model construction from preliminary measurements, a new method is proposed, which is based on automated technique of model generation from images offline. The proposed object recognition methods use the structural coherence criterion to match 3D point clouds with object models. The 3D point clouds are formed from stereo images captured by a camera using visual navigation. Feature points are generated and matched using the SURF detector and Harris corner detector. To improve the reliability of object identification, we propose joint processing of source images with their vectorized forms. The use of line segments obtained by vectorization significantly increases the number of identified characteristic points when matching a 3D point cloud with a model. Based on the identified points of an object, the matrix of AUV coordinate referencing to this object is computed. The efficiency of the considered methods is estimated and compared. In the computational experiments with model scenes, a virtual geometric model of a distributed subsea production system is used. The experiments on real-world data are carried out under laboratory conditions with the use of a Karmin2 stereo camera (Nerian's 3D stereo camera, baseline 25 cm). [ABSTRACT FROM AUTHOR]

This paper proposes an effective method, termed as motion-consistency driven matching (MCDM), for mismatch removal from given tentative correspondences between two feature sets. In particular, we regard each correspondence as a hypothetical node, and formulate the matching problem into a probabilistic graphical model to infer the state of each node (e.g., true or false correspondence). By investigating the motion consistency of true correspondences, a general prior is incorporated into our formulation to differentiate false correspondences from the true ones. The final inference is casted into an integer quadratic programming problem, and the solution is obtained by using an efficient optimization technique based on the Frank-Wolfe algorithm. Extensive experiments on general feature matching, as well as fundamental matrix estimation, relative pose estimation and loop-closure detection, demonstrate that our MCDM possesses strong generalization ability as well as high accuracy, which outperforms state-of-the-art methods. Meanwhile, due to the low computational complexity, the proposed method is efficient for practical feature matching tasks. [ABSTRACT FROM AUTHOR]

Feature matching can establish reliable points correspondences between two images, which plays a vital role in indirect Visual Simultaneous Localization and Mapping (VSLAM). The VSLAM system performs data association between image frames through feature matching. There are a large number of incorrect correspondences in the unfiltered matching results, and the accuracy of data association will directly determine the localization performance. This paper proposes a convolutional neural network with dense connectivity to eliminate the outliers in the matching results. It repeatedly concatenates low-level and high-level features into a dense feature map, which allows both local and global information to be used for mismatch removal. Furthermore, we introduce an outlier rejection module based on our network into the classic monocular visual odometry (VO) pipeline, in order to enhance the reliability of data association. Experiments on several benchmark datasets show that our method achieves favorable performance against other algorithms, especially under challenging scenes, such as low texture. Also, we verify the performance of our system on a mobile robot in real-world localization task. [ABSTRACT FROM AUTHOR]

Loop detection technology is an important part of the simultaneous localization and mapping system for eliminating the pose drift of robots during long-term movement. In order to solve the three main challenges of appearance-based methods, namely viewpoint changes, repeated textures, and large amounts of calculation, this paper proposes an unsupervised loop detection method that takes into account both the texture pattern and position information of feature points and avoids any pre-training steps. Since the relative rotation and translation of the robot between two frames forming loop closure are both very small, the proposed method constrains the matching range with an overlapped block strategy to not only improve the matching precision, but also reduce the cost of matching. Furthermore, the method introduces Gaussian functions to weight and fuse the matching score of each block. The proposed method is evaluated in detail on two different public datasets with various scenarios, and the results show that the proposed method performs better and more efficiently than existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Whole pixel registration of non-rigid images with high accuracy and efficiency is a challenging problem in computer vision. To address this issue, we propose a correspondence vector field (CVF) Interpolation approach based on sparse matching of feature seeds. First, we detect and match two types of feature seeds to improve the accuracy of the later dense CVF interpolation. The first type of feature seeds is to guarantee the accuracy of the motion boundary, while the second one is to achieve the uniform distribution of seeds, which is helpful to improve the effect of interpolation. Second, we regionally estimate the dense CVF using the proposed interpolation approach on this basis. At last, we realize the whole-pixel registration of non-rigid images to yield the image alignment. Different from the traditional CVF interpolation approaches based on optical flow field, ours is based on the sparse matching of feature seeds. Thus, it is not limited to the large displacements and tends to achieve the accurate matching of certain key points easily, which is critical to the final interpolation result. Qualitative and quantitative experimental results on several internationally used datasets demonstrate that our approach outperforms the state-of-the-art ones. [ABSTRACT FROM AUTHOR]

The protection and digitization of ancient Chinese Buddhist and Daoist temples is an urgent task due to their wooden structures and usual location on high mountainous sites, sensitive to various natural and manmade damages. Image-based 3D reconstruction technology presents a unique advantage owing to its easy and cheap data acquisition. However, due to the shape complexity, repeated structural and textural patterns, how to reliably establish feature correspondences across images is still a great challenging task. To address this issue, we propose a grid-and-multi-density-based feature matching method, called GMC_FM. Firstly, an image is partitioned into clusters based on grid and multi-density analysis. Secondly, an iterative clustering strategy is proposed to adaptively determine the suitable grid size. Thirdly, corresponding clusters from two images are determined by a proposed new matching measure. Finally, new features within the corresponding clusters are extracted by Canny operator and feature correspondences are established through a new similarity measure combining local geometry and gray-level distribution. Our proposed GMC_FM is evaluated on the Buddhist and Taoist Temple Dataset, and satisfactory feature matching results are obtained in terms of matching precision and robustness. [ABSTRACT FROM AUTHOR]

Camshift algorithm tracking is susceptible to interference when a tracking object is occluded or when its hue is similar to the background. An improved Camshift object-tracking algorithm combining AKAZE (Accelerated-KAZE) feature matching and Kalman filtering is proposed. First, the video channel is converted for processing. Second, AKAZE is used to match the object feature points and Kalman filtering is used to predict the next position. Then different scenes are judged by the threshold and the Camshift and Kalman tracking algorithms are used for object tracking, respectively. Finally, the improved Camshift algorithm is used to test the moving object in a variety of situations and compared with the traditional Camshift algorithm and the Kalman filter improved Camshift algorithm. Experimental results show that the improved joint tracking algorithm can continue tracking under full occlusion. The effective frame rate of recognition is increased by about 20%, and the single-frame image processing time is less than 35 ms, which can meet the real-time tracking requirements. [ABSTRACT FROM AUTHOR]

Human activity recognition using smartphone has been attracting great interest. Since collecting large amount of labeled data is expensive and time-consuming for conventional machine learning techniques, transfer learning techniques have been proposed for activity recognition. However, existing transfer learning techniques typically rely on feature matching based on global domain shift and lack considering the intra-class knowledge transfer. In this paper, a novel transfer learning technique is proposed for cross-domain activity recognition, which can properly integrate feature matching and instance reweighting across the source and target domain in principled dimensionality reduction. The experiments using three real datasets demonstrate that the proposed scheme can achieve much higher precision (92%), recall (91%), and F1-score (92%), in comparison with the existing schemes. [ABSTRACT FROM AUTHOR]

Generally, deep networks learn to recognize a category of objects by training on a large number of annotated images accurately. However, a meta-learning problem known as a low-shot image recognition task occurs when a few images with annotations are available for learning a recognition model for a single category. Consequently, the objects in testing/query and training/support image datasets are likely to vary in terms of size, location, style, and so on. In this paper, we propose a method, cascaded feature matching network (CFMN), to solve this problem. We train the meta-learner to learn a more fine-grained and adaptive deep distance metric using feature matching block, which aligns associated features together and naturally ignores non-discriminative features. By applying the proposed feature matching block in different layers of the network, multi-scale information among the compared images is incorporated into the final cascaded matching feature, which boosts the recognition performance and generalizes better by learning on relationships. Moreover, the experiments for few-shot learning (FSL) using two standard datasets: miniImageNet and Omniglot, confirm the effectiveness of our proposed method. Besides, the multi-label few-shot task is first studied on a new data split of the COCO dataset, which further shows the superiority of the proposed feature matching network when performing the FSL in complex images. [ABSTRACT FROM AUTHOR]

Structure from motion has attracted a lot of research in recent years, with new state-of-the-art approaches coming almost every year. One of its advantages over 3D reconstruction is that it can be used for any cameras (UAVs, depth sensor, light field) and produces relatively accurate point clouds and camera parameters. One of its disadvantages compared to other approaches is that it is computationally expensive. In this paper, we design a novel structure-from-motion framework to reduce the computational cost and implement a parallel bundle adjustment on GPU device for large-scale optimization. In our framework, the local bundle adjustment is added into the architecture of the incremental structure from motion; namely, the point clouds and camera's parameters are optimized when an additional number of images was added. Then, the purpose is not only to improve the quality of the produced point clouds but also to reduce computation time via parallel bundle adjustment. We conduct extensively experiments on several challenging datasets and make comparison with the state-of-the-art methods. Experimental results show that the proposed method has the best performance in terms of accuracy and efficiency. [ABSTRACT FROM AUTHOR]

This paper presents a GPU-based real-time approach for generating high-definition (HD) aerial image mosaics. The cumbersome process of registering HD images is addressed by a parallel scheme that rapidly matches binary features. The proposed feature matcher takes advantage of the fast ORB (oriented FAST and rotated BRIEF) descriptor and its attainable arrangement into hash tables. By exploiting the best functionalities of binary descriptors and hashing-based data structures, the process of creating HD mosaics is accelerated. On average, real-time performance of 14.5 ms is achieved in a frame-to-frame process, for input images of 2.7 K resolution (2704 × 1521). For evaluation purposes in terms of robustness and speed, we selected two image registration methods for comparison. The first method uses the feature extractor and matcher modules of the well-known ORB-SLAM. The second comparison is carried out against the standard KNN-based matcher of OpenCV. The experiments were conducted under different conditions and scenarios, and the proposed approach exhibits a speed-up of 10.5 times compared to ORB-SLAM-based approach and 36.5 times compared to the OpenCV matcher. Therefore, this research widens the range of applications for aerial mosaicing, since the proposed system is capable of creating high-detail panoramas of large sites while acquiring data. [ABSTRACT FROM AUTHOR]

As a fundamental and critical task in various visual applications, image matching can identify then correspond the same or similar structure/content from two or more images. Over the past decades, growing amount and diversity of methods have been proposed for image matching, particularly with the development of deep learning techniques over the recent years. However, it may leave several open questions about which method would be a suitable choice for specific applications with respect to different scenarios and task requirements and how to design better image matching methods with superior performance in accuracy, robustness and efficiency. This encourages us to conduct a comprehensive and systematic review and analysis for those classical and latest techniques. Following the feature-based image matching pipeline, we first introduce feature detection, description, and matching techniques from handcrafted methods to trainable ones and provide an analysis of the development of these methods in theory and practice. Secondly, we briefly introduce several typical image matching-based applications for a comprehensive understanding of the significance of image matching. In addition, we also provide a comprehensive and objective comparison of these classical and latest techniques through extensive experiments on representative datasets. Finally, we conclude with the current status of image matching technologies and deliver insightful discussions and prospects for future works. This survey can serve as a reference for (but not limited to) researchers and engineers in image matching and related fields. [ABSTRACT FROM AUTHOR]

We detect the trajectory of particles using the feature matching method to improve the resolution of particle streak velocimetry (PSV), which is used to measure the velocity of particles from a visualized path line. PSV has a more reliable performance in particle matching as compared to particle tracking velocimetry and is therefore less likely to cause erroneous matching even in high-density images. The center of gravity of the first and last trajectories is obtained to calculate the displacement. The trajectory of the particle is illuminated using a diode laser and imaged using a digital single-lens reflex camera; the trajectory is then divided into three parts and recorded in a single frame using coded illumination. The first and second trajectories are short, and the third trajectory is long. The asymmetry of the trajectories is then used to determine the flow direction. We first evaluate the detection rate by increasing the trajectory density of synthetic images. The image size was fixed at 500 × 500 pixels, and the number of trajectories was increased from 28 to 280, and the detection rate was examined. Then, we evaluated the accuracy of detection of the center of gravity of the first and last trajectories using the root mean square error. Finally, we used the coded illumination method to visualize the swirling flow inside a device to examine its applicability to real flows. [ABSTRACT FROM AUTHOR]

In this study, a hierarchical clustering matching (HCM) algorithm is proposed to match features with ambiguity due to repetitive patterns in visual odometry. Visual odometry is a real-time system that estimates the motions of camera setups, in which feature matching is a key step for tracking and relocalization. However, it is still difficult to remove outliers fast and reliably when a high proportion of outliers exist. The proposed HCM algorithm solves this problem by clustering accurate matches hierarchically. Dubious matches excluded from any clusters are removed during the iterations, which finally converges when new clusters no longer generate. Local geometric consistency and descriptor of features are both considered to be a metric to link two clusters using a centroid linkage criterion. Experimental results demonstrate that the proposed method works well on solving the problem mentioned above. Compared to state-of-the-art methods on feature matching, HCM performs much better on efficiency with comparable accuracy. [ABSTRACT FROM AUTHOR]

The point pair feature (PPF) algorithm is one of the best-performing 3D object recognition algorithms. However, the high dimensionality of its search space is a disadvantage of this algorithm. This high dimensionality means the feature matching process contains a large number of uninformative features, which reduces recognition speed. To solve this problem and improve the object recognition speed, this paper proposes a fast 3D object recognition algorithm based on the plane-constrained point pair features. By utilizing the property of the coplanar point pair features and the characteristics of the object placement plane, the proposed algorithm extracts the object placement plane through convex hull area calculation, eliminates irrelevant point pair features, and then performs object recognition with the reduced point pair feature descriptors for the feature matching. Experimental results demonstrate that the proposed algorithm significantly reduces the number of feature descriptors and accelerates the recognition speed of 3D objects in a complex background. Compared to the original point pair feature algorithm, the proposed method can achieve better performance and efficiency for 3D object recognition. [ABSTRACT FROM AUTHOR]

The goal of computer vision is to identify objects of interest from images. Copy-move forgery is the process of copying one or more parts of an image and moved into another part of an equivalent image. The detection of copy-move images has many limitations in recognition of objects. In this paper, the proposed work uses Speeded Up Robust Feature (SURF) feature extraction, and the specific object is recognized with the help of the support vector machine. When copy-move forgery was performed, some modifications were done to the image. For instance, turning, scaling, darkening, compression, and noise addition are applied to make effective impersonation forgeries. Here, feature matching process uses the image rotate function, which consists of bicubic and crop operations, and calculates the difference using the blend, scale and joint operation. The results show that forged images are extracted from a given set of test images. The test results exhibit that the proposed technique can get noteworthy and impressive results. [ABSTRACT FROM AUTHOR]

Feature matching aims at generating correspondences across images, which is widely used in many computer vision tasks. Although considerable progress has been made on feature descriptors and fast matching for initial correspondence hypotheses, selecting good ones from them is still challenging and critical to the overall performance. More importantly, existing methods often take a long computational time, limiting their use in real-time applications. This paper attempts to separate true correspondences from false ones at high speed. We term the proposed method (GMS) grid-based motion Statistics, which incorporates the smoothness constraint into a statistic framework for separation and uses a grid-based implementation for fast calculation. GMS is robust to various challenging image changes, involving in viewpoint, scale, and rotation. It is also fast, e.g., take only 1 or 2 ms in a single CPU thread, even when 50K correspondences are processed. This has important implications for real-time applications. What's more, we show that incorporating GMS into the classic feature matching and epipolar geometry estimation pipeline can significantly boost the overall performance. Finally, we integrate GMS into the well-known ORB-SLAM system for monocular initialization, resulting in a significant improvement. [ABSTRACT FROM AUTHOR]

In this paper, a novel video frame rate up-conversion (FRUC) technique based on adaptive motion compensation is presented by combining the semantic feature analysis for image sequences. Firstly, the image is divided into the sub-patch with the same size and their feature areextracted. And, semantic feature analysis is adopted to generate a more accurate motion vector field from the previous frame to the following frame, and then we select these suitable classification algorithms for patial-temporal patches to divide the frame into moving areas, static areas and fault areas. By using the semantic-based adaptive interpolation, we developed a hierarchical refinement strategy to adaptively correct these motion vector so as to get the interpolation frame. Experimental results show that the performance of our method is better than those of the popular I-FRUC, B-FRUC, F-FRUC and A-FRUC methods in both objective and subjective quality, which has relative advantage for engineering applications. [ABSTRACT FROM AUTHOR]

Domain adaptation and adversarial networks are two main approaches for transfer learning. Domain adaptation methods match the mean values of source and target domains, which requires a very large batch size during training. However, adversarial networks are usually unstable when training. In this paper, we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects. At the same time, our method improves the stability of training. Moreover, the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent. Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets. [ABSTRACT FROM AUTHOR]