Search

This paper introduces a new discrete Hough transform, DHT, that pre-computes discrete line information (rules) and uses this information to detect line segments in the image. Pre-computing line information removes the need for run-time line calculations and the associated parameters. The proposed approach does not depend on the parameterization of a straight line and is formulated based on the discrete domain. This new DHT is compared with selected existing techniques to demonstrate the large reduction in computation time achieved by this new approach, while not sacrificing accuracy. [ABSTRACT FROM AUTHOR]

Motion provides extra information that can aid in the recognition of objects. One of the most commonly seen objects is, perhaps, the human body. Yet little attention has been paid to the analysis of human motion. One of the key steps required for a successful motion analysis system is the ability to track moving objects. In this paper, we describe a new system called Log-Tracker, which was recently developed for tracking the motion of the different parts of the human body. Occlusion of body parts is termed a forking condition. Two classes of forks as well as the attributes required to classify them are described. Experimental results from two gymnastics sequences indicate that the system is able to track the body parts even when they are occluded for a short period of time. Occlusions that extend for a long period of time still pose problems to Log-Tracker. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural network (CNN) based on the encoder-decoder structure have been successfully applied to image dehazing. However, these CNN based dehazing methods have two limitations: First, these dehazing models are large in size with enormous parameters, which not only consumes much GPU memory, but also is hard to train from scratch. Second, these models, which ignore the structural information at different resolutions of intermediate layers, cannot capture informative texture and edge information for dehazing by stacking more layers. In this paper, we propose a light-weight end-to-end network named the residual dense pyramid network (RDPN) to address the above problems. To exploit the structural information at different resolutions of intermediate layers fully, a new residual dense pyramid (RDP) is proposed as a building block. By introducing a dense information fusion layer and the residual learning module, the RDP can maximize the information flow and extract local features. Furthermore, the RDP further learns the structural information from intermediate layers via a multiscale pyramid fusion mechanism. To reduce the number of network parameters and to ease the training process, we use one RDP in the encoder and two RDPs in the decoder, following a multilevel pyramid pooling layer for incorporating global context features before estimating the final result. The extensive experimental results on a synthetic dataset and real-world images demonstrate that the new RDPN achieves favourable performance compared with some state-of-the-art methods, e.g., the recent densely connected pyramid dehazing network, the all-in-one dehazing network, the enhanced pix2pix dehazing network, pixel-based alpha blending, artificial multi-exposure image fusions and the genetic programming estimator, in terms of accuracy, run time and number of parameters. To be specific, RDPN outperforms all of the above methods in terms of PSNR by at least 4.25 dB. The run time of the proposed method is 0.021 s, and the number of parameters is 1,534,799, only 6% of that used by the densely connected pyramid dehazing network. [ABSTRACT FROM AUTHOR]

Please refer to full text. [ABSTRACT FROM AUTHOR]

Match cost functions are common elements of every stereopsis algorithm that are used to provide a dissimilarity measure between pixels in different images. Global stereopsis algorithms incorporate assumptions about the smoothness of the resulting distance map that can interact with match cost functions in unpredictable ways. In this paper, we present a large-scale study on the relative performance of a structured set of match cost functions within several global stereopsis frameworks. We compare 272 match cost functions that are built from component parts in the context of four global stereopsis frameworks with a data set consisting of 57 stereo image pairs at three different variances of synthetic sensor noise. From our analysis, we infer a set of general rules that can be used to guide derivation of match cost functions for use in global stereopsis algorithms. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) using deep convolutional neural networks (CNNs) achieves the state-of-the-art performance. Most existing SISR models mainly focus on pursuing high peak signal-to-noise ratio (PSNR) and neglect textures and details. As a result, the recovered images are often perceptually unpleasant. To address this issue, in this paper, we propose a texture and detail-preserving network (TDPN), which focuses not only on local region feature recovery but also on preserving textures and details. Specifically, the high-resolution image is recovered from its corresponding low-resolution input in two branches. First, a multi-reception field based branch is designed to let the network fully learn local region features by adaptively selecting local region features in different reception fields. Then, a texture and detail-learning branch supervised by the textures and details decomposed from the ground-truth high resolution image is proposed to provide additional textures and details for the super-resolution process to improve the perceptual quality. Finally, we introduce a gradient loss into the SISR field and define a novel hybrid loss to strengthen boundary information recovery and to avoid overly smooth boundary in the final recovered high-resolution image caused by using only the MAE loss. More importantly, the proposed method is model-agnostic, which can be applied to most off-the-shelf SISR networks. The experimental results on public datasets demonstrate the superiority of our TDPN on most state-of-the-art SISR methods in PSNR, SSIM and perceptual quality. We will share our code on https://github.com/tocaiqing/TDPN. [ABSTRACT FROM AUTHOR]

Progressive cerebral degeneration in amyotrophic lateral sclerosis (ALS) remains poorly understood. Here, three‐dimensional (3D) texture analysis was used to study longitudinal gray and white matter cerebral degeneration in ALS from routine T1‐weighted magnetic resonance imaging (MRI). Participants were included from the Canadian ALS Neuroimaging Consortium (CALSNIC) who underwent up to three clinical assessments and MRI at four‐month intervals, up to 8 months after baseline (T0). Three‐dimensional maps of the texture feature autocorrelation were computed from T1‐weighted images. One hundred and nineteen controls and 137 ALS patients were included, with 81 controls and 84 ALS patients returning for at least one follow‐up. At baseline, texture changes in ALS patients were detected in the motor cortex, corticospinal tract, insular cortex, and bilateral frontal and temporal white matter compared to controls. Longitudinal comparison of texture maps between T0 and Tmax (last follow‐up visit) within ALS patients showed progressive texture alterations in the temporal white matter, insula, and internal capsule. Additionally, when compared to controls, ALS patients had greater texture changes in the frontal and temporal structures at Tmax than at T0. In subgroup analysis, slow progressing ALS patients had greater progressive texture change in the internal capsule than the fast progressing patients. Contrastingly, fast progressing patients had greater progressive texture changes in the precentral gyrus. These findings suggest that the characteristic longitudinal gray matter pathology in ALS is the progressive involvement of frontotemporal regions rather than a worsening pathology within the motor cortex, and that phenotypic variability is associated with distinct progressive spatial pathology. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a new robust approach for multi-view L 2 triangulation based on optimal inlier selection and 3D structure refinement. The proposed method starts with estimating the scale of noise in image measurements, which affects both the quantity and the accuracy of reconstructed 3D points but is overlooked or ignored in existing triangulation pipelines. A new residual-consensus scheme within which the uncertainty of epipolar transfer is analytically characterized by deriving its closed-form covariance is developed to robustly estimate the noise scale. Different from existing robust triangulation pipelines, the issue of outliers is addressed by directly searching for the optimal 3D points that are within either the theoretical correct error bounds calculated by second-order cone programming (SOCP) or the efficiently calculated approximate ranges. In particular, both the inlier selection and 3D structure refinement are realized in an optimal fashion using Differential Evolution (DE) optimization which allows flexibility in the design of the objective function. To validate the performance of the proposed method, extensive experiments using both synthetic data and real image sequences were carried out. Comparing with state-of-the-art robust triangulation strategies, the proposed method can consistently identify more reliable inliers and hence, reconstruct more unambiguous 3D points with higher accuracy than existing methods. [Copyright &y& Elsevier]

Due to the scattering of light and the influence of different water types, underwater images usually suffer from different type of hybrid degradation, e.g. color distortion, blurred details and low contrast. Existing underwater image enhancement methods are weak at handling hybrid degradation simultaneously, resulting in low quality results. Inspired by the fact that wavelet-based enhancement methods can correct color and enhance details in frequency domain and the color compensation prior can compensate missing color information in spatial domain, we design the Multi-level Wavelet-based Underwater Image Enhancement Network (MWEN) with the color compensation prior to enhance image in both frequency domain and spatial domain. Specifically, we integrate the multi-level wavelet transform and the color compensation prior into a multi-stage enhancement framework, where each stage consists of a Multi-level Wavelet-based Enhancement Module (MWEM), a Color Compensation Prior Extraction Module (CCPEM) and a color filter with prior-aware weights. The MWEM decomposes image features into low frequency and high frequency by a wavelet transform, and then enhances them by a low frequency enhancement branch and several high frequency enhancement branches, respectively. The low frequency reduces the color distortion of different water types using Instance Normalization for style transfer, while the high frequency enhancement enhances sparse details using a non-local sparse attention mechanism. After the inverse wavelet transform, the preliminary enhanced result by the MWEM is obtained. Then, the color filter whose weights are customized by the color compensation information extracted from the CCPEM dynamically is applied to output of the MWEM for color compensation. Such an operation enables network to adapt to hybrid degradation and achieve better performance. The experiments demonstrate MWEN outperforms existing UIE methods quantitatively and qualitatively. • A new Multi-level Wavelet-based Underwater Image Enhancement Network is proposed. • A new Multi-level Wavelet-based Enhancement Module is proposed. • A new Color filter inspired by color compensation prior is proposed. [ABSTRACT FROM AUTHOR]

Underwater images suffer from different types of degradation, where color degradation occurs in the spatial domain and edge degradation in the frequency domain. The high-quality underwater image enhancement represents a crucial milestone in advancing computer vision systems tailored for marine environments. This foundational endeavor encompasses a wide array of applications in computer vision tasks, including underwater inspection, underwater archaeology, and environmental monitoring. However, current convolutional neural network (CNN)-based pyramid frameworks primarily focus on capturing local features, often overlooking the significance of global semantic features that play a crucial role in understanding underwater scenes. Moreover, these frameworks handle spatial and frequency features independently, failing to enhance images by exploring correlation among domain-specific attributes for enabling information consistency. Besides, optimizing the model using a loss function with the same domain attributes from the ground truth may not lead to a better generalization ability. To solve these problems, we propose a new Convolution-Transformer Blend Pyramid Network (CTPN), which consists of a spatial branch and several frequency branches. The CTPN has four key components: a Swin transformer encoder, a CNN-Transformer aggregated encoder–decoder (CTED) and a blend pyramid framework. The Swin transformer encoder is employed to capture global semantic features, benefiting from its ability to extract long-range and global dependencies among features. The CTED fuses local features captured by CNN layers and global semantic features captured by the Swin transformer encoder in the spatial branch, with the help of the Cross-Model Fusion Module (CFM) and Skip-Aggregation Module (SAM). Subsequently, a blend pyramid framework is designed which not only progressively expands the transformed information of the previous domain branch to the current domain branch via the CTED-based refining operation, but also utilizes the proposed Domain Affinity Block (DAB) to explore the connection between domain attributes, ensuring information consistency. The experimental results demonstrate that the proposed method outperforms existing underwater image enhancement methods quantitatively and qualitatively. • A Convolution-Transformer blend pyramid network with spatial and frequency branches is designed for image enhancement. • A Convolution-Transformer aggregated encoder–decoder fuses local and global features. • A blend pyramid framework with the Domain Affinity Block is designed for enabling information consistent. [ABSTRACT FROM AUTHOR]

Image dehazing is an improtant image pre-processing step for many computer vision tasks with many proposed methods using convolutional neural networks. Ordinary Differential Equations (ODE) are a powerful mathematical tool in social and natural science, leading to better understanding, prediction, and use of information systems. Because prior knowledge has demonstrated its effectiveness in many practical domains, we design a Visual Attention Network (VAN) and an ODE-inspired Network (ODEN) based on prior knowledge. Then, we develop a multi-model fuzzy fusion strategy, which integrates results predicted by the prior-based Visual Attention Network (VAN) and the ODE-inspired Network (ODEN) into a single network to leverage their respective strengths in improving the dehazing performance. First, to utilize the haze-related prior for dehazing, the VAN removes haze with the help of the haze attention map. Then, to improve performance of stacked residual blocks inspired by the first-order Euler method in ODE-based methods, the ODEN is built by only 3 Runge–Kutta Modules (RKM), each of which not only is related to the stable fourth-order Runge–Kutta method implemented in the Runge–Kutta Block (RKB), but also combines the RKB with an attention mechanism. Finally, an attention based fusion mechanism is used to fuse results estimated by the ODEN and the VAN based on the multi-level features extracted by a pretrained ResNeXt. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art methods in terms of visual effects and accuracy. The average PSNR and SSIM on three public datasets are 31.36 and 0.9766, respectively, which are better than the compared state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a novel, simple, yet powerful and robust method for rotation invariant texture classification. Like the Local Binary Patterns (LBP), the proposed method considers at each pixel a neighboring function defined on a circle of radius R. We define local frequency components as the magnitude of the coefficients of the 1D Fourier transform of the neighboring function. By applying different bandpass filters on the 2D Fourier transform of the local frequency components, we define our Local Frequency Descriptors (LFD). The LFD features are added dynamically from low frequencies to high. The features defined in this paper are invariant to rotation. As well, they are robust to noise. The experimental results on the Outex, CUReT, and KTH-TIPS datasets show that the proposed method outperforms state-of-the-art texture analysis methods. The results also show that the proposed method is very robust to noise. [Copyright &y& Elsevier]

• An end-to-end network with a parallel spatial/channel-wise attention block is proposed for image dehazing. • The spatial attention module and the channelwise attention module are combined in parallel for obtaining powerful features. • The accuracy of the dehazing results is improved by using one parallel spatial/channel-wise attention block in the network. Image dehazing is a very important pre-processing step to many computer vision tasks such as object recognition and tracking. However, it is a challenging problem because the physical parameters of imaging, e.g. the depth information of scene pixels and the attenuation model, are usually unknown. Based on a physical model, different methods have been proposed to recover these parameters. Existing convolutional neural networks (CNNs) based methods try to solve the image dehazing problem using an end-to-end network to learn a direct mapping between a hazy image and its corresponding clear image. But the representational ability, spatial variant ability and dehazing capability of these network models are hindered by treating all the spatial and channel-wise features indiscriminately. Hence, we propose an end-to-end dehazing network with a parallel spatial/channel-wise attention block for capturing more informative spatial and channel-wise features respectively. Specifically, based on the encoder-decoder framework with a pyramid pooling operation, a novel parallel spatial/channel-wise attention block is proposed and applied to the end of the encoder for guiding the decoder to reconstruct better clear images. In the spatial/channel-wise attention block, the spatial attention module and the channel-wise attention module are connected in parallel, where the spatial attention module highlights important spatial positions of features. Meanwhile, the channel-wise module exploits inter-dependencies among the channel-wise features. Extensive experiments demonstrate that our network with a parallel spatial /channel-wise attention block can achieve better accuracy and visual results over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Amyotrophic Lateral Sclerosis (ALS) is a complex neurodegenerative disorder characterized by motor neuron degeneration. Significant research has begun to establish brain magnetic resonance imaging (MRI) as a potential biomarker to diagnose and monitor the state of the disease. Deep learning has emerged as a prominent class of machine learning algorithms in computer vision and has shown successful applications in various medical image analysis tasks. However, deep learning methods applied to neuroimaging have not achieved superior performance in classifying ALS patients from healthy controls due to insignificant structural changes correlated with pathological features. Thus, a critical challenge in deep models is to identify discriminative features from limited training data. To address this challenge, this study introduces a framework called S F 2 F o r m e r , which leverages the power of the vision transformer architecture to distinguish ALS subjects from the control group by exploiting the long-range relationships among image features. Additionally, spatial and frequency domain information is combined to enhance the network's performance, as MRI scans are initially captured in the frequency domain and then converted to the spatial domain. The proposed framework is trained using a series of consecutive coronal slices and utilizes pre-trained weights from ImageNet through transfer learning. Finally, a majority voting scheme is employed on the coronal slices of each subject to generate the final classification decision. The proposed architecture is extensively evaluated with multi-modal neuroimaging data (i.e., T1-weighted, R2*, FLAIR) using two well-organized versions of the Canadian ALS Neuroimaging Consortium (CALSNIC) multi-center datasets. The experimental results demonstrate the superiority of the proposed strategy in terms of classification accuracy compared to several popular deep learning-based techniques. • We propose a novel vision transformer model to classify ALS from healthy controls. • We analyze two independent and extensive datasets of 120 and 232 MRI scans. • We leverage multi-center and multi-modal neuroimaging data (T1W, R2*, and FLAIR). • The proposed method demonstrates state-of-the-art classification accuracy. [ABSTRACT FROM AUTHOR]

In an underwater imaging system, a refractive interface is introduced when a camera looks into the water-based environment, resulting in distorted images due to the refraction of light. Simply ignoring the refraction effect or using the lens radial distortion model causes erroneous 3D reconstruction. This paper deals with a general underwater imaging setup using two cameras, of which each camera is placed in a separate waterproof housing with a flat glass window. In order to handle refraction properly, a simplified refractive camera model is used in this paper. Based on two new concepts, namely the Ellipse of Refrax (EoR) and the Refractive Depth (RD) of a scene point, we derive two new formulations of the underwater known rotation structure and motion (SaM) problem. One gives a globally optimal solution and the other is robust to outliers. The constraint of known rotation is further relaxed by incorporating the robust known rotation SaM into a new hybrid optimization framework. Our method is able to simultaneously perform underwater camera calibration and 3D reconstruction automatically without using any calibration object or additional calibration device. In order to evaluate the performance and practical applicability of our method, extensive experiments using synthetic data, synthetically rendered images and real underwater images were carried out. The experimental results demonstrate that the proposed method can significantly improve the accuracy of the reconstructed 3D structure (within 0.78 mm for an object of dimension over 200 mm compared with the ground truth model captured by a land-based system) and of the system parameters for underwater applications. Compared with bundle adjustment using the refractive camera model initialized with traditional 3D reconstruction methods, our proposed optimization method has significantly better completeness and accuracy and lower 3D errors in the reconstructed models. [ABSTRACT FROM AUTHOR]

Occlusion poses as a critical challenge in computer vision for a long time. Recently, the technique of synthetic aperture photography using a camera array has been regarded as a promising way to address the problem of occluded object imaging. Unfortunately, the expensive cost of a standard camera array system with the required calibration procedure still limits the widespread popularity of this technique. To make synthetic aperture photography practical, in this paper, we propose a novel approach based on a mobile camera-IMU system moving on a planar surface. The contributions of our work include: (1) with geometric property assumptions, we mathematically prove that the transformation matrices for synthetic aperture image generation can be factorized into an infinite homography and a shift; (2) the IMU data are utilized to estimate the homography by selecting the plane at infinity as the reference plane, which is scene independent and does not require any calibration; (3) compared to the conventional camera array based approaches, the proposed method effectively improves the solution for synthetic aperture images capturing and generation by using a more flexible, convenient and inexpensive system. Extensive experimental results with qualitative and quantitative evaluations have demonstrated the see-through-occlusion performance with satisfactory accuracy of the proposed approach in different scenarios. [ABSTRACT FROM AUTHOR]

• We proposed a novel method termed the "PKG-Net" to accurately predict Alzheimer's disease. • The input is designed to be multi-dimensional and collaboratively represents lesion area from multiple scales via the pyramid representation. • The joint loss function is utilized to improves the practicability of the proposed network and its stability in training. • Our method demonstrates good generalization ability, and has achieved excellent results of 97.28% in accuracy on the ADNI dataset. Recently, deep learning based Computer-Aided Diagnosis methods have been widely utilized due to their highly effective diagnosis of patients. Although Convolutional Neural Networks (CNNs) are capable of extracting the latent structural characteristics of dementia and of capturing the changes of brain anatomy in Magnetic Resonance Imaging (MRI) scans, the high-dimensional input to a deep CNN usually makes the network difficult to train, and affects its diagnostic accuracy. In this paper, a novel method called the hierarchical pseudo-3D convolution neural network based on a kernel attention mechanism with a new global context block, which is abbreviated as "PKG-Net", is proposed to accurately predict Alzheimer's disease even when the input features are complex. Specifically, the proposed network first extracts multi-scale features from pre-processed images. Second, the attention mechanism and global context blocks are applied to combine features from different layers to hierarchically transform the MRI into more compact high-level features. Then, a joint loss function is used to train the proposed network to generate more distinguishing features, which improve the generalization performance of the network. In addition, we combine our method with different architectures. Extensive experiments are conducted to analyze the performance of the PKG-Net with different hyper-parameters and architectures. Finally, in order to verify the effectiveness of our method on Alzheimer's disease diagnosis, we carry out extensive experiments on the ADNI dataset, and compare the results of our method with that of existing methods in terms of accuracy, recall and precision. Furthermore, our network can fully take advantage of the deep 3D convolutional neural network for automatic feature extraction and representation, and thus can avoid the limitation of low processing efficiency caused by the preprocessing procedure in which a specific area needs to be annotated in advance. Finally, we evaluate our proposed framework using two public datasets, ADNI-1 and ADNI-2, and the experimental results show that our proposed framework can achieve superior performance over state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Abstract: Metric 3D reconstruction from two uncalibrated images involves estimating both the camera parameters and the 3D structure of a scene, which is known to be sensitive to the quality of image correspondences. In this paper, the above problem is recast as a single constrained optimization problem, which can be efficiently solved by a new modified Constrained Adaptive Differential Evolution optimizer, within which noticeable acceleration of convergence rate has been achieved by incorporating geometrically meaningful evolutionary operations. The proposed approach avoids solving the inverse 3D reconstruction problem by directly searching for the globally optimal 3D structure while satisfying the epipolar geometry and the cheirality constraints. Given a set of outlier affected noisy image correspondences, the camera calibration and scene structure can be simultaneously estimated in our global optimization framework. Extensive experimental validation on both synthetic data and real images were carried out. The performance of our proposed method is compared with that of four well-known fundamental matrix estimation methods, each of which is combined with analytical focal length estimation, optimal triangulation and bundle adjustment optimization. Statistical analysis of the results demonstrates that our method can significantly improve the accuracy of camera calibration and scene reconstruction. The stable, accurate numerical performance as well as fast convergence rate make our method a practical one. [Copyright &y& Elsevier]

This paper presents an automatic real-time video matting system. The proposed system consists of two novel components. In order to automatically generate trimaps for live videos, we advocate a Time-of-Flight (TOF) camera-based approach to video bilayer segmentation. Our algorithm combines color and depth cues in a probabilistic fusion framework. The scene depth information returned by the TOF camera is less sensitive to environment changes, which makes our method robust to illumination variation, dynamic background and camera motion. For the second step, we perform alpha matting based on the segmentation result. Our matting algorithm uses a set of novel Poisson equations that are derived for handling multichannel color vectors, as well as the depth information captured. Real-time processing speed is achieved through optimizing the algorithm for parallel processing on graphics hardware. We demonstrate the effectiveness of our matting system on an extensive set of experimental results. [ABSTRACT FROM AUTHOR]

• The proposed network contains a feature extraction network, a recurrent refinement network and an encoder-decoder network. • The recurrent refinement network generates and refines the haze attention map using low-level and high-level features as input alternatively. • The encoder-decoder network predicts the dehazing result with the guidance of a haze attention map and the fused multi-level features. Image dehazing is very important for many computer vision tasks. However, typical CNN-based methods learn a direct mapping from a hazy image to a clear image, ignoring relevant haze priors and multi-level features. In this paper, a new Visual Attention Dehazing Network (VADN) with multi-level refinement and fusion is proposed, which leverages a haze attention map as a haze relevant prior and learns complementary haze information among multi-level features. The VADN contains a feature extraction network, a recurrent refinement network and an encoder-decoder network. The feature extraction network captures the multi-level features. The recurrent refinement network generates and refines the haze attention map by taking low-level features and high-level features as inputs alternatively. Then, the haze attention map is injected into the encoder-decoder network to obtain the clear image with the help of complementary information learned from informative multi-level features. The experimental results demonstrate that the average PSNR of VADN is 32.50 dB which outperforms most state-of-the-art methods by up to 5.14 dB. Besides, the run time of VADN is 0.067 s, only 55 % of the run time spent by the recent enhanced pix2pix dehazing network. [ABSTRACT FROM AUTHOR]

• The first time to address the missing information problem caused by heavy occlusion in Synthetic Aperture Imaging. • Our method generates a realistic all-in-focus synthetic aperture image where the information of the occluded region is completely restored. • Extensive experiments on public datasets and our own datasets demonstrate the superior performance of the proposed method over state-of-the-art Synthetic Aperture Imaging methods. Occlusions handling poses a significant challenge to many computer vision and pattern recognition applications. Recently, Synthetic Aperture Imaging (SAI), which uses more than two cameras, is widely applied to reconstruct occluded objects in complex scenes. However, it usually fails in cases of heavy occlusions, in particular, when the occluded information is not captured by any of the camera views. Hence, it is a challenging task to generate a realistic all-in-focus synthetic aperture image which shows a completely occluded object. In this paper, semantic inpainting using a Generative Adversarial Network (GAN) is proposed to address the above-mentioned problem. The proposed method first computes a synthetic aperture image of the occluded objects using a labeling method, and an alpha matte of the partially occluded objects. Then, it uses energy minimization to reconstruct the background by focusing on the background depth of each camera. Finally, the occluded regions of the synthesized image are semantically inpainted using a GAN and the results are composited with the reconstructed background to generate a realistic all-in-focus image. The experimental results demonstrate that the proposed method can handle heavy occlusions and can produce better all-in-focus images than other state-of-the-art methods. Compared with traditional labeling methods, our method can quickly generate label for occlusion without introducing noise. To the best of our knowledge, our method is the first to address missing information caused by heavy occlusions in SAI using a GAN. [ABSTRACT FROM AUTHOR]

• Data from multiple centers for ALS research has some challenges such as intensity variations in MRI scans. • A gradient-based texture method called M-CoHOG is proposed to discriminate ALS patients from controls. • A feature normalization method along with an ensemble classifier is also proposed to accommodate the multicenter issues. • Our method achieves state-of-the-art results by accommodating the variation in multicenter data. • Thus, texture analysis using M-CoHOG shows promise as a potential biomarker for ALS. Gradient-based texture analysis methods have become popular in computer vision and image processing and has many applications including medical image analysis. This motivates us to develop a texture feature extraction method to discriminate Amyotrophic Lateral Sclerosis (ALS) patients from controls. But, the lack of data in ALS research is a major constraint and can be mitigated by using data from multiple centers. However, multi-center data gives some other challenges such as differing scanner parameters and variation in intensity of the medical images, which motivate the development of the proposed method. To investigate these challenges, we propose a gradient-based texture feature extraction method called Modified Co-occurrence Histograms of Oriented Gradients (M-CoHOG) to extract texture features from 2D Magnetic Resonance Images (MRI). We also propose a new feature-normalization technique before feeding the normalized M-CoHOG features into an ensemble of classifiers, which can accommodate for variation of data from different centers. ALS datasets from four different centers are used in the experiments. We analyze the classification accuracy of single center data as well as that arising from multiple centers. It is observed that the extracted texture features from downsampled images are more significant in distinguishing between patients and controls. Moreover, using an ensemble of classifiers shows improvement in classification accuracy over a single classifier in multi-center data. The proposed method outperforms the state-of-the-art methods by a significant margin. [ABSTRACT FROM AUTHOR]

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL.

Deep learning has become a leading subset of machine learning and has been successfully employed in diverse areas, ranging from natural language processing to medical image analysis. In medical imaging, researchers have progressively turned towards multi-center neuroimaging studies to address complex questions in neuroscience, leveraging larger sample sizes and aiming to enhance the accuracy of deep learning models. However, variations in image pixel/voxel characteristics can arise between centers due to factors including differences in magnetic resonance imaging scanners. Such variations create challenges, particularly inconsistent performance in machine learning-based approaches, often referred to as domain shift, where the trained models fail to achieve satisfactory or improved results when confronted with dissimilar test data. This study analyzes the performance of multiple disease classification tasks using multi-center MRI data obtained from three widely used scanner manufacturers (GE, Philips, and Siemens) across several deep learning-based networks. Furthermore, we investigate the efficacy of mitigating scanner vendor effects using ComBat-based harmonization techniques when applied to multi-center datasets of 3D structural MR images. Our experimental results reveal a substantial decline in classification performance when models trained on one type of scanner manufacturer are tested with data from different manufacturers. Moreover, despite applying ComBat-based harmonization, the harmonized images do not demonstrate any noticeable performance enhancement for disease classification tasks., (© 2023. Springer Nature Limited.)

Conventional deconvolution methods utilize hand-crafted image priors to constrain the optimization. While deep-learning-based methods have simplified the optimization by end-to-end training, they fail to generalize well to blurs unseen in the training dataset. Thus, training image-specific models is important for higher generalization. Deep image prior (DIP) provides an approach to optimize the weights of a randomly initialized network with a single degraded image by maximum a posteriori (MAP), which shows that the architecture of a network can serve as the hand-crafted image prior. Unlike conventional hand-crafted image priors, which are obtained through statistical methods, finding a suitable network architecture is challenging due to the unclear relationship between images and their corresponding architectures. As a result, the network architecture cannot provide enough constraint for the latent sharp image. This paper proposes a new variational deep image prior (VDIP) for blind image deconvolution, which exploits additive hand-crafted image priors on latent sharp images and approximates a distribution for each pixel to avoid suboptimal solutions. Our mathematical analysis shows that the proposed method can better constrain the optimization. The experimental results further demonstrate that the generated images have better quality than that of the original DIP on benchmark datasets.

In medical research and clinical applications, the utilization of MRI datasets from multiple centers has become increasingly prevalent. However, inherent variability between these centers presents challenges due to domain shift, which can impact the quality and reliability of the analysis. Regrettably, the absence of adequate tools for domain shift analysis hinders the development and validation of domain adaptation and harmonization techniques. To address this issue, this paper presents a novel Domain Shift analyzer for MRI (DSMRI) framework designed explicitly for domain shift analysis in multi-center MRI datasets. The proposed model assesses the degree of domain shift within an MRI dataset by leveraging various MRI-quality-related metrics derived from the spatial domain. DSMRI also incorporates features from the frequency domain to capture low- and high-frequency information about the image. It further includes the wavelet domain features by effectively measuring the sparsity and energy present in the wavelet coefficients. Furthermore, DSMRI introduces several texture features, thereby enhancing the robustness of the domain shift analysis process. The proposed framework includes visualization techniques such as t-SNE and UMAP to demonstrate that similar data are grouped closely while dissimilar data are in separate clusters. Additionally, quantitative analysis is used to measure the domain shift distance, domain classification accuracy, and the ranking of significant features. The effectiveness of the proposed approach is demonstrated using experimental evaluations on seven large-scale multi-site neuroimaging datasets.

Recent developments in Convolutional Neural Networks (CNNs) have made them one of the most powerful image dehazing methods. In particular, the Residual Networks (ResNets), which can avoid the vanishing gradient problem effectively, are widely deployed. To understand the success of ResNets, recent mathematical analysis of ResNets reveals that a ResNet has a similar formulation as the Euler method in solving the Ordinary Differential Equations (ODE's). Hence, image dehazing which can be formulated as an optimal control problem in dynamical systems can be solved by a single-step optimal control method, such as the Euler method. This optimal control viewpoint provides a new perspective to address the problem of image restoration. Motivated by the advantages of multi-step optimal control solvers in ODE's, which include better stability and efficiency than single-step solvers, e.g. Euler, we propose the Adams-based Hierarchical Feature Fusion Network (AHFFN) for image dehazing with modules inspired by a multi-step optimal control method named the Adams-Bashforth method. Firstly, we extend a multi-step Adams-Bashforth method to the corresponding Adams block, which achieves a higher accuracy than that of single-step solvers because of its more effective use of intermediate results. Then, we stack multiple Adams blocks to mimic the discrete approximation process of an optimal control in a dynamical system. To improve the results, the hierarchical features from stacked Adams blocks are fully used by combining Hierarchical Feature Fusion (HFF) and Lightweight Spatial Attention (LSA) with Adams blocks to form a new Adams module. Finally, we not only use HFF and LSA to fuse features, but also highlight important spatial information in each Adams module for estimating the clear image. The experimental results using synthetic and real images demonstrate that the proposed AHFFN obtains better accuracy and visual results than that of state-of-the-art methods., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Background and Purpose: This study sought to evaluate the relationship of progressive corticospinal tract (CST) degeneration with survival in patients with amyotrophic lateral sclerosis (ALS)., Methods: Forty-one ALS patients and 42 healthy controls were prospectively recruited from the Canadian ALS Neuroimaging Consortium. Magnetic resonance imaging scanning and clinical evaluations were performed on participants at three serial visits with 4-month intervals. Texture analysis was performed on T1-weighted magnetic resonance imaging scans and the texture feature 'autocorrelation' was quantified. Whole-brain group-level comparisons were performed between patient subgroups. Linear mixed models were used to evaluate longitudinal progression. Region-of-interest and 3D voxel-wise Cox proportional-hazards regression models were constructed for survival prediction. For all survival analyses, a second independent cohort was used for model validation., Results: Autocorrelation of the bilateral CST was increased at baseline and progressively increased over time at a faster rate in ALS short survivors. Cox proportional-hazards regression analyses revealed autocorrelation of the CST as a significant predictor of survival at 5 years follow-up (hazard ratio 1.28, p = 0.005). Similarly, voxel-wise whole-brain survival analyses revealed that increased autocorrelation of the CST was associated with shorter survival. ALS patients stratified by median autocorrelation in the CST had significantly different survival times using the Kaplan-Meier curve and log-rank tests (χ 2  = 7.402, p = 0.007)., Conclusions: Severity of cerebral degeneration is associated with survival in ALS. CST degeneration progresses faster in subgroups of patients with shorter survival. Neuroimaging holds promise as a tool to improve patient management and facilitation of clinical trials., (© 2023 European Academy of Neurology.)

This paper proposes a new glass segmentation method utilizing paired RGB and thermal images. Due to the large difference between the transmission property of visible light and that of the thermal energy through the glass where most glass is transparent to the visible light but opaque to thermal energy, glass regions of a scene are made more distinguishable with a pair of RGB and thermal images than solely with an RGB image. To exploit such a unique property, we propose a neural network architecture that effectively combines an RGB-thermal image pair with a new multi-modal fusion module based on attention, and integrate CNN and transformer to extract local features and non-local dependencies, respectively. As well, we have collected a new dataset containing 5551 RGB-thermal image pairs with ground-truth segmentation annotations. The qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach on fusing RGB and thermal data for glass segmentation. Our code and data are available at https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.

Transformer-based architectures start to emerge in single image super resolution (SISR) and have achieved promising performance. However, most existing vision Transformer-based SISR methods still have two shortcomings: (1) they divide images into the same number of patches with a fixed size, which may not be optimal for restoring patches with different levels of texture richness; and (2) their position encodings treat all input tokens equally and hence, neglect the dependencies among them. This paper presents a HIPA, which stands for a novel Transformer architecture that progressively recovers the high resolution image using a hierarchical patch partition. Specifically, we build a cascaded model that processes an input image in multiple stages, where we start with tokens with small patch sizes and gradually merge them to form the full resolution. Such a hierarchical patch mechanism not only explicitly enables feature aggregation at multiple resolutions but also adaptively learns patch-aware features for different image regions, e.g., using a smaller patch for areas with fine details and a larger patch for textureless regions. Meanwhile, a new attention-based position encoding scheme for Transformer is proposed to let the network focus on which tokens should be paid more attention by assigning different weights to different tokens, which is the first time to our best knowledge. Furthermore, we also propose a multi-receptive field attention module to enlarge the convolution receptive field from different branches. The experimental results on several public datasets demonstrate the superior performance of the proposed HIPA over previous methods quantitatively and qualitatively. We will share our code and models when the paper is accepted.