Search

Multimodal medical image fusion is a crucial task that combines complementary information from different imaging modalities into a unified representation, thereby enhancing diagnostic accuracy and treatment planning. While deep learning methods, particularly Convolutional Neural Networks (CNNs) and Transformers, have significantly advanced fusion performance, some of the existing CNN-based methods fall short in capturing fine-grained multiscale and edge features, leading to suboptimal feature integration. Transformer-based models, on the other hand, are computationally intensive in both the training and fusion stages, making them impractical for real-time clinical use. Moreover, the clinical application of fused images remains unexplored. In this paper, we propose a novel CNN-based architecture that addresses these limitations by introducing a Dilated Residual Attention Network Module for effective multiscale feature extraction, coupled with a gradient operator to enhance edge detail learning. To ensure fast and efficient fusion, we present a parameter-free fusion strategy based on the weighted nuclear norm of softmax, which requires no additional computations during training or inference. Extensive experiments, including a downstream brain tumor classification task, demonstrate that our approach outperforms various baseline methods in terms of visual quality, texture preservation, and fusion speed, making it a possible practical solution for real-world clinical applications. The code will be released at https://github.com/simonZhou86/en_dran., Comment: An extended version of the paper accepted at IEEE BIBM 2024

Estimating the landscape and soil freeze-thaw (FT) dynamics in the Northern Hemisphere is crucial for understanding permafrost response to global warming and changes in regional and global carbon budgets. A new framework is presented for surface FT-cycle retrievals using L-band microwave radiometry based on a deep convolutional autoencoder neural network. This framework defines the landscape FT-cycle retrieval as a time series anomaly detection problem considering the frozen states as normal and thawed states as anomalies. The autoencoder retrieves the FT-cycle probabilistically through supervised reconstruction of the brightness temperature (TB) time series using a contrastive loss function that minimizes (maximizes) the reconstruction error for the peak winter (summer). Using the data provided by the Soil Moisture Active Passive (SMAP) satellite, it is demonstrated that the framework learns to isolate the landscape FT states over different land surface types with varying complexities related to the radiometric characteristics of snow cover, lake-ice phenology, and vegetation canopy. The consistency of the retrievals is evaluated over Alaska, against in situ ground-based observations, showing reduced uncertainties compared to the traditional methods that use thresholding of the normalized polarization ratio.

Medical images play an important role in clinical applications. Multimodal medical images could provide rich information about patients for physicians to diagnose. The image fusion technique is able to synthesize complementary information from multimodal images into a single image. This technique will prevent radiologists switch back and forth between different images and save lots of time in the diagnostic process. In this paper, we introduce a novel Dilated Residual Attention Network for the medical image fusion task. Our network is capable to extract multi-scale deep semantic features. Furthermore, we propose a novel fixed fusion strategy termed Softmax-based weighted strategy based on the Softmax weights and matrix nuclear norm. Extensive experiments show our proposed network and fusion strategy exceed the state-of-the-art performance compared with reference image fusion methods on four commonly used fusion metrics., Comment: 8 pages, 8 figures, 3 tables

More than half of the global land area undergoes seasonal freeze/thaw (FT) transitions in spring. Spatial patterns and timing of spring thawing influence eco-hydrological processes and landscape moisture availability over arctic and boreal ecosystems. The seasonal progression of spring thawing coincides with warmer temperatures, snowmelt, and a rapid increase in soil moisture, which initiates the growing season for ecosystem productivity. In this study, we utilize complementary satellite observations to determine the pattern and order of occurrence in landscape thawing, soil moisture increase, and ecosystem productivity that collectively define the eco-hydrological spring onset across Alaska and Northern Canada. Satellite data utilized include landscape FT status from SMAP and AMSR-2, OCO-2 derived solar-induced chlorophyll fluorescence (GOSIF), and gross primary production (GPP) and soil moisture from SMAP. The resulting spring onset maps showed spring thawing as the precursor to growing season onset, indicated by a rapid rise in available soil moisture and GPP. Our results indicated an average spring transition period of 3±2 (SD) weeks between initial landscape thawing and growing season onset. A rapid increase in soil moisture generally followed landscape thawing but occurred before the subsequent seasonal rise in GPP. Spring onset generally occurred earlier in boreal forest (DOY 102±14) than arctic tundra (DOY 124±22).