Back to Search
Start Over
UMHE: Unsupervised Multispectral Homography Estimation
- Source :
- IEEE Sensors Journal; 2024, Vol. 24 Issue: 10 p17259-17268, 10p
- Publication Year :
- 2024
-
Abstract
- Multispectral image alignment plays a crucial role in exploiting complementary information between different spectral images. Homography-based image alignment can be a practical solution considering a tradeoff between runtime and accuracy. Existing methods, however, have difficulty with multispectral images due to the additional spectral gap or require expensive human labels to train models. To solve these problems, this article presents a comprehensive study on multispectral homography estimation in an unsupervised learning manner. We propose a curriculum data augmentation, an effective solution for models learning spectrum-agnostic representation by providing diverse input pairs. We also propose to use the phase congruency loss that explicitly calculates the reconstruction between images based on low-level structural information in the frequency domain. To encourage multispectral alignment research, we introduce a novel FLIR corresponding dataset that has manually labeled local correspondences between multispectral images. Our model achieves state-of-the-art alignment performance on the proposed FLIR correspondence dataset among supervised and unsupervised methods while running at 151 FPS. Furthermore, our model shows good generalization ability on the M3FD dataset without fine-tuning.
Details
- Language :
- English
- ISSN :
- 1530437X and 15581748
- Volume :
- 24
- Issue :
- 10
- Database :
- Supplemental Index
- Journal :
- IEEE Sensors Journal
- Publication Type :
- Periodical
- Accession number :
- ejs66398123
- Full Text :
- https://doi.org/10.1109/JSEN.2024.3383453