Search

In this paper, we propose a real-time vibration extraction system, which extracts vibration component within a given frequency range from videos in real time, for realizing tremor suppression used in microsurgery assistance systems. To overcome the problems in our previous system based on the mean Lucas-Kanade (LK) optical flow of the whole frame, we have introduced a new architecture combining dense optical flow calculated with simple feature matching and block-based band-pass filtering using band-limited multiple Fourier linear combiner (BMFLC). As a feature of optical flow calculation, we use the simplified rotation-invariant histogram of oriented gradients (RIHOG) based on a gradient angle quantized to 1, 2, or 3 bits, which greatly reduces the usage of memory resources for a frame buffer. An obtained optical flow map is then divided into multiple blocks, and BMFLC is applied to the mean optical flow of each block independently. By using the L1-norm of adaptive weight vectors in BMFLC as a criterion, blocks belonging to vibrating objects can be isolated from background at low cost, leading to better extraction accuracy compared to the previous system. The whole system for 480p and 720p resolutions can be implemented on a single Xilinx Zynq-7000 XC7Z020 FPGA without any external memory, and can process a video stream supplied directly from a camera at 60fps., IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, E106A(3), pp.504-513; 2023

Feature matching is a fundamental task in the field of image processing, aimed at ensuring correct correspondence between two sets of features. Putative matches constructed based on the similarity of descriptors always contain a large number of false matches. To eliminate these false matches, we propose a remote sensing image feature matching method called LMC (local motion consistency), where local motion consistency refers to the property that adjacent correct matches have the same motion. The core idea of LMC is to find neighborhoods with correct motion trends and retain matches with the same motion. To achieve this, we design a local geometric constraint using a homography matrix to represent local motion consistency. This constraint has projective invariance and is applicable to various types of transformations. To avoid outliers affecting the search for neighborhoods with correct motion, we introduce a resampling method to construct neighborhoods. Moreover, we design a jump-out mechanism to exit the loop without searching all possible cases, thereby reducing runtime. LMC can process over 1000 putative matches within 100 ms. Experimental evaluations on diverse image datasets, including SUIRD, RS, and DTU, demonstrate that LMC achieves a higher F-score and superior overall matching performance compared to state-of-the-art methods.

Feature matching is a core step in multi-source remote sensing image registration approaches based on feature. However, for existing methods, whether traditional classical SIFT algorithm or deep learning-based methods, they essentially rely on generating descriptors from local regions of feature points, which can lead to low matching success rates due to various challenges, including gray-scale changes, content changes, local similarity, and occlusions between images. Inspired by the human approach of finding rough corresponding regions globally and then carefully comparing local regions, and the excellent global attention property of transformers, the proposed feature matching network adopts a coarse-to-fine matching strategy that utilizes both global and local information between images to predict corresponding feature points. Importantly, the network has great flexibility of matching corresponding points for any feature points and can be effectively trained without strong supervised signals of corresponding feature points and only require the true geometric transformation between images. The qualitative experiment illustrate the effectiveness of the proposed network by matching feature points extracted by SIFT or sampled uniformly. In the quantitative experiments, we used feature points extracted by SIFT, SuperPoint, and LoFTR as the keypoints to be matched. We then calculated the mean match success ratio (MSR) and mean reprojection error (MRE) of each method at different thresholds in the test dataset. Additionally, boxplot graphs were plotted to visualize the distributions. By comparing the MSR and MRE values as well as their distributions with other methods, we can conclude that the proposed method consistently outperforms the comparison methods in terms of MSR at different thresholds. Moreover, the MSR of the proposed method remains within a reasonable range compared to the MRE of other methods.

Multiple modality clustering seeks to partition objects via leveraging cross-modality relations to provide comprehensive descriptions of the same objects. Current clustering methods rely heavily on accurate affinity measurements among samples. The samplewise affinity is costive to be constructed yet easy to corrupt by the heterogeneous gap. In the era of big data, fast and accurate clustering of multiple modality data remains challenging. To fill the gap, we propose a novel approach to achieve the clustering by focusing on feature matching across different modalities instead of samplewise affinity. First, a feature matching matrix is calculated by measuring the potential featurewise correlations. The obtained matching matrix is decomposed into two bases corresponding to the column and row spaces of feature matching, acting as coded bases within feature spaces of the different modalities. Then, the sample assignment is obtained by jointly reconstructing the samples by the two bases. The feature matching potential and sample assignment are collaboratively learned by an alternating optimization scheme. The proposed method dramatically reduces the computational cost by avoiding the costive samplewise affinity estimation, without sacrificing accuracy. Extensive experiments on the synthetic and real-world datasets demonstrate its superior speed and high accuracy.

In this paper, we propose an iterative feature matching framework for self-supervised depth estimation in indoor scenes. Conventional methods usually leverage the structure-from-motion supervision to help the photometric optimization escape from the local minima, which have complex ego-motion and large regions with non-texture or repeated-texture. However, the supervision is limited as the reconstruction is usually sparse. To address this, we propose an iterative feature matching framework called IFMNet to jointly learn depths and search for correspondences. With the predicted depths from the previous iteration, we present an online optimized grid searching algorithm to find more accurate correspondences. Given these new correspondences, we compute the triangulated depths and improve the depth network with adaptive bin-wise online hard example mining. Experimental results on the NYU Depth V2 and SceneNet datasets verify the effectiveness of our approach.

In order to solve the problem of automatically completing safety detection for construction sites and give an alert based on high-speed image streams, this paper proposes a violation of rules and regulations (VoRR) recognition method on a construction site and gives a matching method by automatically obtaining a few samples. The proposed safety detection method consists of five parts, which are redundant information reduction, classification, feature extraction, matching, inference rule and alarm alert. Compared with existing safety detection methods, the accuracy of the proposed method is increased by more than 9%. It not only has better performance, but also has more functions: reminding and alarming. For the subsequent establishment of an unmanned supervision system model on a construction site, this research will provide a new method of decision support, target detection, and recognition in multiple different scenarios.

Image matching is a basic task in three-dimensional reconstruction, which, in recent years, has attracted extensive attention in academic and industrial circles. However, when dealing with large-scale image datasets, these methods have low accuracy and slow speeds. To improve the effectiveness of modern image matching methods, this paper proposes an image matching method for 3D reconstruction. The proposed method can obtain high matching accuracy through hash index in a very short amount of time. The core of hash matching includes two parts: creating the hash table and hash index. The former is used to encode local feature descriptors into hash codes, and the latter is used to search candidates for query feature points. In addition, the proposed method is extremely robust to image scaling and transformation by using various verifications. A comprehensive experiment was carried out using several challenging datasets to evaluate the performance of hash matching. Experimental results show that the HashMatch presents excellent results compared to the state-of-the-art methods in both computational efficiency and matching accuracy.

The ongoing digitization of historical photographs in archives allows investigating the quality, quantity, and distribution of these images. However, the exact interior and exterior camera orientations of these photographs are usually lost during the digitization process. The proposed method uses content-based image retrieval (CBIR) to filter exterior images of single buildings in combination with metadata information. The retrieved photographs are automatically processed in an adapted structure-from-motion (SfM) pipeline to determine the camera parameters. In an interactive georeferencing process, the calculated camera positions are transferred into a global coordinate system. As all image and camera data are efficiently stored in the proposed 4D database, they can be conveniently accessed afterward to georeference newly digitized images by using photogrammetric triangulation and spatial resection. The results show that the CBIR and the subsequent SfM are robust methods for various kinds of buildings and different quantity of data. The absolute accuracy of the camera positions after georeferencing lies in the range of a few meters likely introduced by the inaccurate LOD2 models used for transformation. The proposed photogrammetric method, the database structure, and the 4D visualization interface enable adding historical urban photographs and 3D models from other locations.

Different tile maps may use different coordinate systems, and it is difficult to superimpose maps from different sources. In this regard, we propose a progressive dynamic registration (PDR) method based on optimum features extracted from images of tile map screenshots of a certain scale. Using this method, we can automatically register maps in roughly the same area from different sources without knowing the map project information. The better features among feature points and feature surfaces are selected for image registration based on the richness of the features in the map, and a new matching filter that combines the characteristics of the tile map and the features is proposed. In the progressive registration process during map zooming, the result of the adjacent scales are used as reference values for coarse registration. After experimental verification in different areas, the root mean square error of PDR is below 2.5 in different maps and is better than that of the registration method using feature points only. Moreover, the registration accuracies of remote sensing maps in all areas and vector maps in nonurban areas are better than that of the method based on coordinate system transformation. The calculation results of PDR can register not only the tile maps but also other nontiled vector or remote sensing data.

Due to the periodicity of circuit boards, the registration algorithm based on keypoints is less robust in circuit board detection and is prone to misregistration problems. In this paper, the binary neighborhood coordinate descriptor (BNCD) is proposed and applied to circuit board image registration. The BNCD consists of three parts: neighborhood description, coordinate description, and brightness description. The neighborhood description contains the grayscale information of the neighborhood, which is the main part of BNCD. The coordinate description introduces the actual position of the keypoints in the image, which solves the problem of inter-period matching of keypoints. The brightness description introduces the concept of bright and dark points, which improves the distinguishability of BNCD and reduces the calculation amount of matching. Experimental results show that in circuit board image registration, the matching precision rate and recall rate of BNCD is better than that of classic algorithms such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF), and the calculation of descriptors takes less time.

In underwater scenes, degraded underwater images caused by wavelength-dependent light absorption and scattering present huge challenges to vision tasks. Underwater image enhancement has attracted much attention due to the significance of vision-based applications in marine engineering and underwater robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, almost all existing approaches focus only on the enhancement of independent images. Considering that images photographed in the same underwater scene usually share similar degradation, related images can provide rich complementary information for each other’s enhancement. In this paper, we propose an Underwater Image Co-enhancement Network (UICoE-Net) based on an encoder-decoder Siamese architecture. For joint learning, we introduced correlation feature matching units into the multiple layers of our Siamese encoder-decoder structure in order to communicate the mutual correlation of the two branches. Extensive experiments using the Underwater Image Enhancement Benchmark (UIEB), Underwater Image Co-enhancement Dataset (UICoD) collected from an underwater video dataset with ground-truth reference and Stereo Quantitative Underwater Image Dataset (SQUID) dataset demonstrate the effectiveness of our method.

Image matching between the optical and synthetic aperture radar (SAR) is one of the most fundamental problems for earth observation. In recent years, many researchers have used hand-made descriptors with their expertise to find matches between optical and SAR images. However, due to the large nonlinear radiation difference between optical images and SAR images, the image matching becomes very difficult. To deal with the problems, the article proposes an efficient feature matching and position matching algorithm (MatchosNet) based on local deep feature descriptor. First, A new dataset is presented by collecting a large number of corresponding SAR images and optical images. Then a deep convolutional network with dense blocks and cross stage partial networks is designed to generate deep feature descriptors. Next, the hard L2 loss function and ARCpatch loss function are designed to improve matching effect. In addition, on the basis of feature matching, the two-dimensional (2-D) Gaussian function voting algorithm is designed to further match the position of optical images and SAR images of different sizes. Finally, a large number of quantitative experiments show that MatchosNet has a excellent matching effect in feature matching and position matching. The code will be released at: https://github.com/LiaoYun0x0/Feature-Matching-and-Position-Matching-between-Optical-and-SAR.

Feature matching is a critical prerequisite in many applications of remote sensing, and its aim is to establish reliable correspondences between two sets of features. Existing attempts typically involve estimating the underlying image transformations to remove false matches in putative matches. However, the image transformation could vary with different application scenarios, which means that using a predefined geometrical model may lead to inferior matching accuracy, especially if the image transformation is nonrigid. This article casts the mismatch removal into a neighborhood consistency evaluation problem under a customized learning framework. With only seven training image pairs involving approximately 8000 putative matches, our method can handle different types of images or transformation models (affine, homography, piecewise-linear transformation, and others). Extensive experiments on feature matching and image registration are conducted to demonstrate the superiority of our method over the eight state-of-the-art competitors.

Unmanned aerial vehicle (UAV) hyperspectral imaging has been extensively applied in various fields. However, due to the limited imaging width, hyperspectral images (HSIs) captured by UAV need to be stitched so as to effectively cover the study area. In this paper, an effective seamless stitching method with deep feature matching and elastic warp is proposed for HSIs, which consists of the following major steps. First, for each input HSI, a single band grayscale image is obtained by fusing the bands corresponding to the red, green, and blue wavelengths. Second, the feature points of each HSI are obtained with a robust VGG-style network, and matched with a graph neural network. After point pairs are obtained, the next step is to estimate the transformation matrix of adjacent images, and a spectral correction method based on intrinsic decomposition is proposed to ensure the spectral consistency of adjacent images. In the final stage, a seam-cutting and multi-scale blending strategy is adopted to ensure the spatial consistency of the stitching results. Experimental results on real HSIs show that the proposed method is superior to six representative image stitching approaches.

In recent years, social platforms have become integrated in a variety of economic, political and cultural domains. Social media have become the primary outlets for many citizens to consume news and information, and, at the same time, to produce and share online a large amount of data and meta-data. This paper presents an innovative system able to analyze visual information shared by citizens on social media during extreme events for contributing to the situational awareness and supporting people in charge of coordinating the emergency management. The system analyzes all posts containing images shared by users by taking into account: (a) the event class and (b) the GPS coordinates of the geographical area affected by the event. Then, a Single Shot Multibox Detector (SSD) network is applied to select only the posted images correctly related to the event class and an advanced image processing procedure is used to verify if these images are correlated with the geographical area where the emergency event is ongoing. Several experiments have been carried out to evaluate the performance of the proposed system in the context of different emergency situations caused by earthquakes, floods and terrorist attacks.

To promote the development of deep learning for feature matching, image registration, and three-dimensional reconstruction, we propose a method of constructing a deep learning benchmark dataset for affine-invariant feature matching. Existing images often have large viewpoint differences and areas with weak texture, which may cause difficulties for image matching, with respect to few matches, uneven distribution, and single matching texture. To solve this problem, we designed an algorithm for the automatic production of a benchmark dataset for affine-invariant feature matching. It combined two complementary algorithms, ASIFT (Affine-SIFT) and LoFTR (Local Feature Transformer), to significantly increase the types of matching patches and the number of matching features and generate quasi-dense matches. Optimized matches with uniform spatial distribution were obtained by the hybrid constraints of the neighborhood distance threshold and maximum information entropy. We applied this algorithm to the automatic construction of a dataset containing 20,000 images: 10,000 ground-based close-range images, 6000 satellite images, and 4000 aerial images. Each image had a resolution of 1024 × 1024 pixels and was composed of 128 pairs of corresponding patches, each with 64 × 64 pixels. Finally, we trained and tested the affine-invariant deep learning model, AffNet, separately on our dataset and the Brown dataset. The experimental results showed that the AffNet trained on our dataset had advantages, with respect to the number of matching points, match correct rate, and matching spatial distribution on stereo images with large viewpoint differences and weak texture. The results verified the effectiveness of the proposed algorithm and the superiority of our dataset. In the future, our dataset will continue to expand, and it is intended to become the most widely used benchmark dataset internationally for the deep learning of wide-baseline image matching.

The semantic segmentation model usually provides pixel-wise category prediction for images. However, a massive amount of pixel-wise annotation images is required for model training, which is time-consuming and labor-intensive. An image-level categorical annotation is recently popular and attempted to overcome the above issue in this work. This is also termed weakly supervised semantic segmentation, and the general framework aims to generate pseudo masks with class activation mapping. This can be learned through classification tasks that focus on explicit features. Some major issues in these approaches are as follows: (1) Excessive attention on the specific area; (2) for some objects, the detected range is beyond the boundary, and (3) the smooth areas or minor color gradients along the object are difficult to categorize. All these problems are comprehensively addressed in this work, mainly to overcome the importance of overly focusing on significant features. The suppression expansion module is used to diminish the centralized features and to expand the attention view. Moreover, to tackle the misclassification problem, the saliency-guided module is adopted to assist in learning regional information. It limits the object area effectively while simultaneously resolving the challenge of internal color smoothing. Experimental results show that the pseudo masks generated by the proposed network can achieve 76.0%, 73.3%, and 73.5% in mIoU with the PASCAL VOC 2012 train, validation, and test set, respectively, and outperform the state-of-the-art methods.

The reliability of feature matching can decide the accuracy and robustness of the feature-based registration result. Aiming at the problem that the number of final feature matches preserved by many...

Camshift algorithm tracking is susceptible to interference when a tracking object is occluded or when its hue is similar to the background. An improved Camshift object-tracking algorithm combining AKAZE (Accelerated-KAZE) feature matching and Kalman filtering is proposed. First, the video channel is converted for processing. Second, AKAZE is used to match the object feature points and Kalman filtering is used to predict the next position. Then different scenes are judged by the threshold and the Camshift and Kalman tracking algorithms are used for object tracking, respectively. Finally, the improved Camshift algorithm is used to test the moving object in a variety of situations and compared with the traditional Camshift algorithm and the Kalman filter improved Camshift algorithm. Experimental results show that the improved joint tracking algorithm can continue tracking under full occlusion. The effective frame rate of recognition is increased by about 20%, and the single-frame image processing time is less than 35 ms, which can meet the real-time tracking requirements.

Image classification systems have been found vulnerable to adversarial attack, which is imperceptible to human but can easily fool deep neural networks. Recent researches indicate that regularizing the network by introducing randomness could greatly improve the model’s robustness against adversarial attack, but the randomness module would normally involve complex calculations and numerous additional parameters and seriously affect the model performance on clean data. In this paper, we propose a feature matching module to regularize the network. Specifically, our model learns a feature vector for each category and imposes additional restrictions on image features. Then, the similarity between image features and category features is used as the basis for classification. Our method does not introduce any additional network parameters than undefended model and can be easily integrated into any neural network. Experiments on the CIFAR10 and SVHN datasets highlight that our proposed module can effectively improve both clean data and perturbed data accuracy in comparison with the state-of-the-art defense methods and outperform the L2P method by 6.3$$\%$$%, 24$$\%$$%on clean and perturbed data, respectively, using ResNet-V2(18) architecture.

Feature matching in optical remote sensing images usually suffers from significant intensity differences and background variations. In this paper, an effective image matching approach is proposed t...

In order to solve the problems of small field of view and unclear detail features of monitoring images in coal mines, a multi-view image stitching method in mine is proposed. Firstly, an improved adaptive histogram equalization method with contrast limitation is used to pre-process the images to highlight the image details and improve the contrast. Secondly, the ORB algorithm is used to extract the image feature points, and the improved Brief algorithm is used to calculate the feature descriptors. Thirdly, the K-nearest neighbor(KNN) algorithm is used to achieve rough matching of feature point pairs, and the random sample consensus(RANSAC) algorithm is used to filter and eliminate the mismatched feature point pairs, and the optimal perspective transformation matrix is solved to transform the coordinates of the pixel points of the image to be matched. Finally, the hat function weighted average fusion algorithm is used to stitch and fuse the fixed image and the image to be matched. The experimental results show that compared with the speeded up robust features(SIFT) and KAZE algorithms, the ORB algorithm reduces the number of feature points extracted for a single image by 48% and 33% respectively, which improves the effective feature point extraction capability. The feature point extraction time is reduced by 17% and 34% respectively, which improves the calculation efficiency. The images stitched by this method avoid the phenomenon of cracks and black lines at the joints, the image transition is natural, and the definition is high.

Purpose Laser absolute distance measurement has the characteristics of high precision, wide range and non-contact. In laser ranging system, tracking and aiming measurement point is the precondition of automatic measurement. To solve this problem, this paper aims to propose a novel method. Design/methodology/approach For the central point of the hollow angle coupled mirror, this paper proposes a method based on correlation filtering and ellipse fitting. For non-cooperative target points, this paper proposes an extraction method based on correlation filtering and feature matching. Finally, a visual tracking and aiming system was constructed by combining the two-axis turntable, and experiments were carried out. Findings The target tracking algorithm has an accuracy of 91.15% and a speed of 19.5 frames per second. The algorithm can adapt to the change of target scale and short-term occlusion. The mean error and standard deviation of the center point extraction of the hollow Angle coupling mirror are 0.20 and 0.09 mm. The mean error and standard deviation of feature points matching for non-cooperative target were 0.06 mm and 0.16 mm. The visual tracking and aiming system can track a target running at a speed of 0.7 m/s, aiming error mean is 1.74 pixels and standard deviation is 0.67 pixel. Originality/value The results show that this method can achieve fast and high precision target tracking and aiming and has great application value in laser ranging.

From traditional machine learning to the latest deep learning classifiers, most models require a large amount of labeled data to perform optimal training and obtain the best performance. Yet, when limited training samples are available or when accompanied by noisy labels, severe degradation in accuracy can arise. The proposed work mainly focusses on these practical issues. Herein, standard datasets, i.e., Mini-ImageNet, CIFAR-FS, and CUB 200, are considered, which also have similar issues. The main goal is to utilize a few labeled data in the training stage, extracting image features and then performing feature similarity analysis across all samples. The highlighted aspects of the proposed method are as follows. (1) The main self-supervised learning strategies and augmentation techniques are exploited to obtain the best pretrained model. (2) An improved dual-model mechanism is proposed to train the support and query datasets with multiple training configurations. As examined in the experiments, the dual-model approach obtains superior performance of few-shot classification compared with all of the state-of-the-art methods.

To reduce the impact of volatility on photovoltaic (PV) power generation forecasting and achieve improved forecasting accuracy, this article provides an in-depth analysis of the characteristics of PV power outputs under typical weather conditions. The trend of PV power generation and the similarity between simultaneous outputs are found, and a hybrid prediction model based on feature matching, singular spectrum analysis (SSA) and a long short-term memory (LSTM) network is proposed. In this paper, correlation analysis is used to verify the trend of PV power generation; the similarity between forecasting days and historical meteorological data is calculated through grey relation analysis; and similar generated PV power levels are searched for phase feature matching. The input time series is decomposed by singular spectrum analysis; the trend component, oscillation component and noise component are extracted; and principal component analysis and reconstruction are carried out on each component. Then, an LSTM network prediction model is established for the reconstructed subsequences, and the external feature input is controlled to compare the obtained prediction results. Finally, the model performance is evaluated through the data of a PV power plant in a certain area. The experimental results prove that the SSA-LSTM model has the best prediction performance.

The affine scale-invariant feature transform (ASIFT) algorithm is a feature extraction algorithm with affinity and scale invariance, which is suitable for image feature matching using unmanned aerial vehicles (UAVs). However, there are many problems in the matching process, such as the low efficiency and mismatching. In order to improve the matching efficiency, this algorithm firstly simulates image distortion based on the position and orientation system (POS) information from real-time UAV measurements to reduce the number of simulated images. Then, the scale-invariant feature transform (SIFT) algorithm is used for feature point detection, and the extracted feature points are combined with the binary robust invariant scalable keypoints (BRISK) descriptor to generate the binary feature descriptor, which is matched using the Hamming distance. Finally, in order to improve the matching accuracy of the UAV images, based on the random sample consensus (RANSAC) a false matching eliminated algorithm is proposed. Through four groups of experiments, the proposed algorithm is compared with the SIFT and ASIFT. The results show that the algorithm can optimize the matching effect and improve the matching speed.

The problem of colour correction of underwater images concerns not only surveyors, who primarily use images for photogrammetric purposes, but also archaeologists, marine biologists, and many other experts of different disciplines whose aim is to study objects and lifeforms underwater. For this reason, this study aims to address and quantify the impact of an ad-hoc underwater image restoration algorithm regarding image feature matching and 3D reconstruction of the scene. The datasets used for the purpose of this study consist of the original and colour corrected images captured in two different sites located in Cyprus with different depths and different water conditions. The images were colour corrected following a simple mathematical model that utilizes the scene's geometry while considering also the absorption and backscattering of light in order to restore the missing colour information.

needs a solid validation that has verification and matching uploaded images. To solve this problem, this paper implementing a detection model using Faster R-CNN and a matching method using ORB (Oriented FAST and Rotated BRIEF) and KNN-BFM (K-Nearest Neighbor Brute Force Matcher). The goal of the implementations is to reach both an 80% mark of accuracy and prove matching using ORB only can be a replaced OCR technique. The implementation accuracy results in the detection model reach mAP (Mean Average Precision) of 94%. But, the matching process only achieves an accuracy of 43,46%. The matching process using only image feature matching underperforms the previous OCR technique but improves processing time from 4510ms to 60m). Image matching accuracy has proven to increase by using a high-quality dan high quantity dataset, extracting features on the important area of EKTP card images.

Robust estimation (RE) is a fundamental issue in robot vision and photogrammetry, which is the theoretical basis of geometric model estimation with outliers. However, M-estimations solved by iteratively reweighted least squares (IRLS) are only suitable for cases with low outlier rates (

To better cope with the significant nonlinear radiation distortions (NRD) and severe rotational distortions in multi-modal remote sensing image matching, this paper introduces a rotationally robust feature-matching method based on the maximum index map (MIM) and 2D matrix, which is called the rotation-invariant local phase orientation histogram (RI-LPOH). First, feature detection is performed based on the weighted moment equation. Then, a 2D feature matrix based on MIM and a modified gradient location orientation histogram (GLOH) is constructed and rotational invariance is achieved by cyclic shifting in both the column and row directions without estimating the principal orientation separately. Each part of the sensed image’s 2D feature matrix is additionally flipped up and down to obtain another 2D matrix to avoid intensity inversion, and all the 2D matrices are concatenated by rows to form the final 1D feature vector. Finally, the RFM-LC algorithm is introduced to screen the obtained initial matches to reduce the negative effect caused by the high proportion of outliers. On this basis, the remaining outliers are removed by the fast sample consensus (FSC) method to obtain optimal transformation parameters. We validate the RI-LPOH method on six different types of multi-modal image datasets and compare it with four state-of-the-art methods: PSO-SIFT, MS-HLMO, CoFSM, and RI-ALGH. The experimental results show that our proposed method has obvious advantages in the success rate (SR) and the number of correct matches (NCM). Compared with PSO-SIFT, MS-HLMO, CoFSM, and RI-ALGH, the mean SR of RI-LPOH is 170.3%, 279.8%, 81.6%, and 25.4% higher, respectively, and the mean NCM is 13.27, 20.14, 1.39, and 2.42 times that of the aforementioned four methods.

Remote-sensing developments such as UAVs heighten the need for hyperspectral image stitching techniques that can obtain information on a large area through various parts of the same scene. State-of-the-art approaches often suffer from accumulation errors and high computational costs for large-scale hyperspectral remote-sensing images. In this study, we aim to generate high-precision hyperspectral panoramas with less spatial and spectral distortion. We introduce a new stitching strategy and apply it to hyperspectral images. The stitching framework was built as follows: First, a single band obtained by signal-to-noise ratio estimation was chosen as the reference band. Then, a feature-matching method combining the SuperPoint and LAF algorithms was adopted to strengthen the reliability of feature correspondences. Adaptive bundle adjustment was also designed to eliminate misaligned artifact areas and occasional accumulation errors. Lastly, a spectral correction method using covariance correspondences is proposed to ensure spectral consistency. Extensive feature-matching and image-stitching experiments on several hyperspectral datasets demonstrate the superiority of our approach over the state of the art.

U današnje vrijeme ne možemo zamisliti svijet bez tehnologije i računala. Računala postaju sve moćnija i u stanju su izvršavati stvari koje se nisu činile mogućim nekoliko godina unazad. Za velik napredak zaslužno je područje umjetne inteligencije. Računalni vid je polje umjetne inteligencije koje kod računala imitira ljudsko oko. Neki od ključnih zadataka računalnog vida, baš kao i ljudskog vida, čine prepoznavanje i uparivanje značajki. Značajke su dijelovi slike koji se razlikuju od svoje okoline te privlače najveću pozornost i najkorisnije su za daljnju obradu. Prepoznavanje i uparivanje značajki temelj su mnogih složenijih zadataka kao što su rekonstrukcija slika, modeliranje 3D scena i autonomno kretanje robota. U radu su predstavljeni i uspoređeni SIFT i ORB, jedni od najkorištenijih klasičnih algoritama detekcije i uparivanja značajki. Koraci algoritama se sastoje od prepoznavanja ključnih točaka, izgradnje opisnika svake ključne točke te uparivanja značajki pomoću izgrađenih opisnika. Algoritmi SIFT i ORB su također uspoređeni s novijim pristupom problemu, algoritmima temeljenim na dubokom učenju. Nowadays, we cannot imagine our life without technology and computers. Computers are becoming more powerful and are able to do things that did not seem possible a few years ago. The field of artificial intelligence is responsible for the great progress. Computer vision is a field of artificial intelligence that mimics the human eye in computers. Some of the key tasks of computer vision, just like human vision, are recognizing and matching features. Features are parts of the image that differ from their surroundings and attract the most attention and are most useful for further processing. Recognizing and matching features is the foundation of many more complex tasks such as image reconstruction, 3D scene modeling and autonomous robot movement. The paper presents and compares SIFT and ORB, one of the most used classic algorithms for feature detection and matching. The steps of the algorithms consist of identifying keypoints, building a descriptor of each keypoint and pairing features using the built descriptors. SIFT and ORB algorithms are also compared to a newer approach to the problem, using algorithms based on deep learning.

Minutiae feature extraction and matching are not only two crucial tasks for identifying fingerprints, but also play an eminent role as core components of automated fingerprint recognition (AFR) systems, which first focus primarily on the identification and description of the salient minutiae points that impart individuality to each fingerprint and differentiate one fingerprint from another, and then matching their relative placement in a candidate fingerprint and previously stored fingerprint templates. In this paper, an automated minutiae extraction and matching framework is presented for identification and verification purposes, in which an adaptive scale-invariant feature transform (SIFT) detector is applied to high-contrast fingerprints preprocessed by means of denoising, binarization, thinning, dilation and enhancement to improve the quality of latent fingerprints. As a result, an optimized set of highly-reliable salient points discriminating fingerprint minutiae is identified and described accurately and quickly. Then, the SIFT descriptors of the local key-points in a given fingerprint are matched with those of the stored templates using a brute force algorithm, by assigning a score for each match based on the Euclidean distance between the SIFT descriptors of the two matched keypoints. Finally, a postprocessing dual-threshold filter is adaptively applied, which can potentially eliminate almost all the false matches, while discarding very few correct matches (less than 4%). The experimental evaluations on publicly available low-quality FVC2004 fingerprint datasets demonstrate that the proposed framework delivers comparable or superior performance to several state-of-the-art methods, achieving an average equal error rate (EER) value of 2.01%.

Human activity recognition using smartphone has been attracting great interest. Since collecting large amount of labeled data is expensive and time-consuming for conventional machine learning techniques, transfer learning techniques have been proposed for activity recognition. However, existing transfer learning techniques typically rely on feature matching based on global domain shift and lack considering the intra-class knowledge transfer. In this paper, a novel transfer learning technique is proposed for cross-domain activity recognition, which can properly integrate feature matching and instance reweighting across the source and target domain in principled dimensionality reduction. The experiments using three real datasets demonstrate that the proposed scheme can achieve much higher precision (92%), recall (91%), and F1-score (92%), in comparison with the existing schemes.

For many remote sensing image applications, orientation estimation is a crucial step for feature points extraction and matching, but it has attracted little attention. Due to the intensity differences between remote sensing image pairs, it is difficult to estimate the orientations of corresponding points accurately, resulting in performance degradation of feature matching. Thus, encountering the intensity differences, a robust spatial structure description for feature regions, and an effective calculation manner from description to orientation play a key role in accurate orientation estimation. To this end, in this letter, we first define a plausible orientation for feature points by the total gradient, offering an effective way to convert the gradient trend of the feature region to orientation. Therefore, we further propose a novel orientation estimation method, in which the rotation invariant gradient is introduced to improve the accuracy of gradient calculation and robustness of spatial structure description. Experimental results on multisensor remote sensing images demonstrate that our method increases the orientation estimation accuracy remarkably and outperforms other orientation estimation methods by a large margin, and effectively improves the performance of feature matching.

Imitating the visual characteristics of human eyes is one of the important tasks of digital image processing and computer vision. Feature correspondence of humanoid-eye binocular images is a prerequisite for obtaining the fused image. Human eyes are more sensitive to edge, because it contains much information. However, existing matching methods usually fail in producing enough edge corresponding pairs for humanoid-eye images because of viewpoint and view direction differences. To this end, we propose a novel and effective feature matching algorithm based on edge points. The proposed method consists of four steps. First, the SUSAN operator is employed to detect features, for its outstanding edge feature extraction capability. Second, the input image is constructed into a multi-scale structure based on image pyramid theory, which is then used to compute simplified SIFT descriptors for all feature points. Third, a novel multi-scale descriptor is constructed, by stitching the simplified SIFT descriptor of each layer. Finally, the similarity of multi-scale descriptors is measured by bidirectional matching, and the obtained preliminary matches are refined by subsequent procedures, to achieve accurate matching results. We respectively conduct qualitative and quantitative experiments, which demonstrate that our method can robustly match feature points in humanoid-eye binocular image pairs, and achieve favorable performance under illumination changes compared to the state-of-the-art.

This study aimed to achieve the accurate and real-time geographic positioning of UAV aerial image targets. We verified a method of registering UAV camera images on a map (with the geographic location) through feature matching. The UAV is usually in rapid motion and involves changes in the camera head, and the map is high-resolution and has sparse features. These reasons make it difficult for the current feature-matching algorithm to accurately register the two (camera image and map) in real time, meaning that there will be a large number of mismatches. To solve this problem, we used the SuperGlue algorithm, which has a better performance, to match the features. The layer and block strategy, combined with the prior data of the UAV, was introduced to improve the accuracy and speed of feature matching, and the matching information obtained between frames was introduced to solve the problem of uneven registration. Here, we propose the concept of updating map features with UAV image features to enhance the robustness and applicability of UAV aerial image and map registration. After numerous experiments, it was proved that the proposed method is feasible and can adapt to the changes in the camera head, environment, etc. The UAV aerial image is stably and accurately registered on the map, and the frame rate reaches 12 frames per second, which provides a basis for the geo-positioning of UAV aerial image targets.

The multiview low dynamic range images captured with sparse camera arrangement under ill-lighting conditions contain highlighted and shadow regions due to over-exposed and under-exposed regions. The processing of these images produces contrast distortion, and it is challenging to maintain relative brightness with color consistency. Moreover, the disparity map estimation faces the challenges of holes and artifacts due to a wide baseline and poor visibility, with a shared view of vision. In this article, we propose a multiview ghost-free image enhancement strategy for in-the-wild images with unknown exposure and geometry. We address the complex geometric alignment problem for a wide variational baseline among multiple sparsely arranged cameras. The features among multiple viewpoints are detected and matched for the image restoration. The restored image contains highlighted and shadow regions with a color imbalance problem. We synthesize virtual images following the intensity mapping function, which compensates for the relative brightness and color distortions. Finally, we fuse all the images to obtain high-quality images. The proposed method is more frequent and feasible for future multiview systems with varying baselines without relying on disparity maps. The experimental results demonstrate that the proposed method outperformed the state-of-the-art approaches.

Image registration is a fundamental and important task in remote sensing. In this article, we focus on feature-based image registration. Existing attempts often require estimating a transformation model or imposing relaxed geometric constraints to establish reliable feature correspondences. However, a parametric model cannot handle image pairs undergoing complex transformations, and relaxed methods discard a lot of structure information and the results are often coarse. To solve the above issues, we propose a local structure visualization descriptor to preserve the original structure information, and cast the feature matching task into an evaluation of the consensus of visual structure under a convolutional neural network. This strategy can effectively measure the similarity of neighborhood structure for mismatch removal. In summary, our method does not depend on a specific transformation model and can process arbitrary remote sensing images (e.g., different deformations, severe outliers, various rotations, and scaling changes). To demonstrate the robustness of our strategy for image registration, extensive experiments on various real remote sensing images for feature matching are conducted and compared against nine state-of-the-art methods, where our method gives the best performances in most scenarios.

We consider SAR-optical image matching problems, where correspondences are acquired from a pair of SAR and optical images. Recent methods for such a problem typically simplify the SAR-optical image matching to the SAR-SAR or optical-optical image matchings using supervised-image-synthesis methods. However, training supervised-image-synthesis needs plenty of aligned SAR-optical image pairs while gathering sufficient amounts of aligned multi-modal image pairs is challenging in remote sensing. In this work, we investigate the applicability of unsupervised-image-synthesis for SAR-optical image matching such that the unaligned SAR-optical images could be used. To this end, we apply feature matching loss to a well known unsupervised-image-synthesis method, i.e., CycleGAN, to enforce the feature matching consistency. Moreover, we develop a shared-matching-strategy to improve the results of SAR-optical image matching further. Qualitative comparisons against CycleGAN, StarGAN, and DualGAN demonstrate the superiority of our approach. Quantitative results show that, compared with CycleGAN, StarGAN, and DualGAN, our method obtains at least 2.6 times more qualified SAR-optical matchings.

Feature matching is one of the key techniques in many vision-based tasks, which aims to establish reliable correspondences between two sets of features. In this paper, we present a new feature matching method, which formulates the matching of two feature sets as a mathematical model based on two common consistency constraints. We first propose an advanced consensus of neighborhood topology, which can better exploit the consensus of topological structures to identify inliers. In order to have reliable neighborhood information for the feature points, a subset with high percentage inliers obtained by a guided matching strategy from the putative matches for the neighborhood construction is used. We demonstrate the advantages of our proposed method on various real image pairs. The results demonstrate that the proposed method is superior to the state-of-the-art feature matching methods.

A new airborne river surface flow measurement technique is presented, called Airborne Feature Matching Velocimetry (AFMV). It uses matchings of characteristic image features for orthorectification ...