Search

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL.

Deep learning has become a leading subset of machine learning and has been successfully employed in diverse areas, ranging from natural language processing to medical image analysis. In medical imaging, researchers have progressively turned towards multi-center neuroimaging studies to address complex questions in neuroscience, leveraging larger sample sizes and aiming to enhance the accuracy of deep learning models. However, variations in image pixel/voxel characteristics can arise between centers due to factors including differences in magnetic resonance imaging scanners. Such variations create challenges, particularly inconsistent performance in machine learning-based approaches, often referred to as domain shift, where the trained models fail to achieve satisfactory or improved results when confronted with dissimilar test data. This study analyzes the performance of multiple disease classification tasks using multi-center MRI data obtained from three widely used scanner manufacturers (GE, Philips, and Siemens) across several deep learning-based networks. Furthermore, we investigate the efficacy of mitigating scanner vendor effects using ComBat-based harmonization techniques when applied to multi-center datasets of 3D structural MR images. Our experimental results reveal a substantial decline in classification performance when models trained on one type of scanner manufacturer are tested with data from different manufacturers. Moreover, despite applying ComBat-based harmonization, the harmonized images do not demonstrate any noticeable performance enhancement for disease classification tasks., (© 2023. Springer Nature Limited.)

Conventional deconvolution methods utilize hand-crafted image priors to constrain the optimization. While deep-learning-based methods have simplified the optimization by end-to-end training, they fail to generalize well to blurs unseen in the training dataset. Thus, training image-specific models is important for higher generalization. Deep image prior (DIP) provides an approach to optimize the weights of a randomly initialized network with a single degraded image by maximum a posteriori (MAP), which shows that the architecture of a network can serve as the hand-crafted image prior. Unlike conventional hand-crafted image priors, which are obtained through statistical methods, finding a suitable network architecture is challenging due to the unclear relationship between images and their corresponding architectures. As a result, the network architecture cannot provide enough constraint for the latent sharp image. This paper proposes a new variational deep image prior (VDIP) for blind image deconvolution, which exploits additive hand-crafted image priors on latent sharp images and approximates a distribution for each pixel to avoid suboptimal solutions. Our mathematical analysis shows that the proposed method can better constrain the optimization. The experimental results further demonstrate that the generated images have better quality than that of the original DIP on benchmark datasets.

In medical research and clinical applications, the utilization of MRI datasets from multiple centers has become increasingly prevalent. However, inherent variability between these centers presents challenges due to domain shift, which can impact the quality and reliability of the analysis. Regrettably, the absence of adequate tools for domain shift analysis hinders the development and validation of domain adaptation and harmonization techniques. To address this issue, this paper presents a novel Domain Shift analyzer for MRI (DSMRI) framework designed explicitly for domain shift analysis in multi-center MRI datasets. The proposed model assesses the degree of domain shift within an MRI dataset by leveraging various MRI-quality-related metrics derived from the spatial domain. DSMRI also incorporates features from the frequency domain to capture low- and high-frequency information about the image. It further includes the wavelet domain features by effectively measuring the sparsity and energy present in the wavelet coefficients. Furthermore, DSMRI introduces several texture features, thereby enhancing the robustness of the domain shift analysis process. The proposed framework includes visualization techniques such as t-SNE and UMAP to demonstrate that similar data are grouped closely while dissimilar data are in separate clusters. Additionally, quantitative analysis is used to measure the domain shift distance, domain classification accuracy, and the ranking of significant features. The effectiveness of the proposed approach is demonstrated using experimental evaluations on seven large-scale multi-site neuroimaging datasets.

Amyotrophic Lateral Sclerosis (ALS) is a complex neurodegenerative disorder characterized by motor neuron degeneration. Significant research has begun to establish brain magnetic resonance imaging (MRI) as a potential biomarker to diagnose and monitor the state of the disease. Deep learning has emerged as a prominent class of machine learning algorithms in computer vision and has shown successful applications in various medical image analysis tasks. However, deep learning methods applied to neuroimaging have not achieved superior performance in classifying ALS patients from healthy controls due to insignificant structural changes correlated with pathological features. Thus, a critical challenge in deep models is to identify discriminative features from limited training data. To address this challenge, this study introduces a framework called SF 2 Former, which leverages the power of the vision transformer architecture to distinguish ALS subjects from the control group by exploiting the long-range relationships among image features. Additionally, spatial and frequency domain information is combined to enhance the network's performance, as MRI scans are initially captured in the frequency domain and then converted to the spatial domain. The proposed framework is trained using a series of consecutive coronal slices and utilizes pre-trained weights from ImageNet through transfer learning. Finally, a majority voting scheme is employed on the coronal slices of each subject to generate the final classification decision. The proposed architecture is extensively evaluated with multi-modal neuroimaging data (i.e., T1-weighted, R2*, FLAIR) using two well-organized versions of the Canadian ALS Neuroimaging Consortium (CALSNIC) multi-center datasets. The experimental results demonstrate the superiority of the proposed strategy in terms of classification accuracy compared to several popular deep learning-based techniques., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.)

Recent developments in Convolutional Neural Networks (CNNs) have made them one of the most powerful image dehazing methods. In particular, the Residual Networks (ResNets), which can avoid the vanishing gradient problem effectively, are widely deployed. To understand the success of ResNets, recent mathematical analysis of ResNets reveals that a ResNet has a similar formulation as the Euler method in solving the Ordinary Differential Equations (ODE's). Hence, image dehazing which can be formulated as an optimal control problem in dynamical systems can be solved by a single-step optimal control method, such as the Euler method. This optimal control viewpoint provides a new perspective to address the problem of image restoration. Motivated by the advantages of multi-step optimal control solvers in ODE's, which include better stability and efficiency than single-step solvers, e.g. Euler, we propose the Adams-based Hierarchical Feature Fusion Network (AHFFN) for image dehazing with modules inspired by a multi-step optimal control method named the Adams-Bashforth method. Firstly, we extend a multi-step Adams-Bashforth method to the corresponding Adams block, which achieves a higher accuracy than that of single-step solvers because of its more effective use of intermediate results. Then, we stack multiple Adams blocks to mimic the discrete approximation process of an optimal control in a dynamical system. To improve the results, the hierarchical features from stacked Adams blocks are fully used by combining Hierarchical Feature Fusion (HFF) and Lightweight Spatial Attention (LSA) with Adams blocks to form a new Adams module. Finally, we not only use HFF and LSA to fuse features, but also highlight important spatial information in each Adams module for estimating the clear image. The experimental results using synthetic and real images demonstrate that the proposed AHFFN obtains better accuracy and visual results than that of state-of-the-art methods., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

Background and Purpose: This study sought to evaluate the relationship of progressive corticospinal tract (CST) degeneration with survival in patients with amyotrophic lateral sclerosis (ALS)., Methods: Forty-one ALS patients and 42 healthy controls were prospectively recruited from the Canadian ALS Neuroimaging Consortium. Magnetic resonance imaging scanning and clinical evaluations were performed on participants at three serial visits with 4-month intervals. Texture analysis was performed on T1-weighted magnetic resonance imaging scans and the texture feature 'autocorrelation' was quantified. Whole-brain group-level comparisons were performed between patient subgroups. Linear mixed models were used to evaluate longitudinal progression. Region-of-interest and 3D voxel-wise Cox proportional-hazards regression models were constructed for survival prediction. For all survival analyses, a second independent cohort was used for model validation., Results: Autocorrelation of the bilateral CST was increased at baseline and progressively increased over time at a faster rate in ALS short survivors. Cox proportional-hazards regression analyses revealed autocorrelation of the CST as a significant predictor of survival at 5 years follow-up (hazard ratio 1.28, p = 0.005). Similarly, voxel-wise whole-brain survival analyses revealed that increased autocorrelation of the CST was associated with shorter survival. ALS patients stratified by median autocorrelation in the CST had significantly different survival times using the Kaplan-Meier curve and log-rank tests (χ 2  = 7.402, p = 0.007)., Conclusions: Severity of cerebral degeneration is associated with survival in ALS. CST degeneration progresses faster in subgroups of patients with shorter survival. Neuroimaging holds promise as a tool to improve patient management and facilitation of clinical trials., (© 2023 European Academy of Neurology.)

This paper proposes a new glass segmentation method utilizing paired RGB and thermal images. Due to the large difference between the transmission property of visible light and that of the thermal energy through the glass where most glass is transparent to the visible light but opaque to thermal energy, glass regions of a scene are made more distinguishable with a pair of RGB and thermal images than solely with an RGB image. To exploit such a unique property, we propose a neural network architecture that effectively combines an RGB-thermal image pair with a new multi-modal fusion module based on attention, and integrate CNN and transformer to extract local features and non-local dependencies, respectively. As well, we have collected a new dataset containing 5551 RGB-thermal image pairs with ground-truth segmentation annotations. The qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach on fusing RGB and thermal data for glass segmentation. Our code and data are available at https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.

Transformer-based architectures start to emerge in single image super resolution (SISR) and have achieved promising performance. However, most existing vision Transformer-based SISR methods still have two shortcomings: (1) they divide images into the same number of patches with a fixed size, which may not be optimal for restoring patches with different levels of texture richness; and (2) their position encodings treat all input tokens equally and hence, neglect the dependencies among them. This paper presents a HIPA, which stands for a novel Transformer architecture that progressively recovers the high resolution image using a hierarchical patch partition. Specifically, we build a cascaded model that processes an input image in multiple stages, where we start with tokens with small patch sizes and gradually merge them to form the full resolution. Such a hierarchical patch mechanism not only explicitly enables feature aggregation at multiple resolutions but also adaptively learns patch-aware features for different image regions, e.g., using a smaller patch for areas with fine details and a larger patch for textureless regions. Meanwhile, a new attention-based position encoding scheme for Transformer is proposed to let the network focus on which tokens should be paid more attention by assigning different weights to different tokens, which is the first time to our best knowledge. Furthermore, we also propose a multi-receptive field attention module to enlarge the convolution receptive field from different branches. The experimental results on several public datasets demonstrate the superior performance of the proposed HIPA over previous methods quantitatively and qualitatively. We will share our code and models when the paper is accepted.

Progressive cerebral degeneration in amyotrophic lateral sclerosis (ALS) remains poorly understood. Here, three-dimensional (3D) texture analysis was used to study longitudinal gray and white matter cerebral degeneration in ALS from routine T1-weighted magnetic resonance imaging (MRI). Participants were included from the Canadian ALS Neuroimaging Consortium (CALSNIC) who underwent up to three clinical assessments and MRI at four-month intervals, up to 8 months after baseline (T 0 ). Three-dimensional maps of the texture feature autocorrelation were computed from T1-weighted images. One hundred and nineteen controls and 137 ALS patients were included, with 81 controls and 84 ALS patients returning for at least one follow-up. At baseline, texture changes in ALS patients were detected in the motor cortex, corticospinal tract, insular cortex, and bilateral frontal and temporal white matter compared to controls. Longitudinal comparison of texture maps between T 0 and T max (last follow-up visit) within ALS patients showed progressive texture alterations in the temporal white matter, insula, and internal capsule. Additionally, when compared to controls, ALS patients had greater texture changes in the frontal and temporal structures at T max than at T 0 . In subgroup analysis, slow progressing ALS patients had greater progressive texture change in the internal capsule than the fast progressing patients. Contrastingly, fast progressing patients had greater progressive texture changes in the precentral gyrus. These findings suggest that the characteristic longitudinal gray matter pathology in ALS is the progressive involvement of frontotemporal regions rather than a worsening pathology within the motor cortex, and that phenotypic variability is associated with distinct progressive spatial pathology., (© 2021 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.)

Intensity inhomogeneity and noise are two common issues in images but inevitably lead to significant challenges for image segmentation and is particularly pronounced when the two issues simultaneously appear in one image. As a result, most existing level set models yield poor performance when applied to this images. To this end, this paper proposes a novel hybrid level set model, named adaptive variational level set model (AVLSM) by integrating an adaptive scale bias field correction term and a denoising term into one level set framework, which can simultaneously correct the severe inhomogeneous intensity and denoise in segmentation. Specifically, an adaptive scale bias field correction term is first defined to correct the severe inhomogeneous intensity by adaptively adjusting the scale according to the degree of intensity inhomogeneity while segmentation. More importantly, the proposed adaptive scale truncation function in the term is model-agnostic, which can be applied to most off-the-shelf models and improves their performance for image segmentation with severe intensity inhomogeneity. Then, a denoising energy term is constructed based on the variational model, which can remove not only common additive noise but also multiplicative noise often occurred in medical image during segmentation. Finally, by integrating the two proposed energy terms into a variational level set framework, the AVLSM is proposed. The experimental results on synthetic and real images demonstrate the superiority of AVLSM over most state-of-the-art level set models in terms of accuracy, robustness and running time.

Single image super-resolution (SISR) using deep convolutional neural networks (CNNs) achieves the state-of-the-art performance. Most existing SISR models mainly focus on pursuing high peak signal-to-noise ratio (PSNR) and neglect textures and details. As a result, the recovered images are often perceptually unpleasant. To address this issue, in this paper, we propose a texture and detail-preserving network (TDPN), which focuses not only on local region feature recovery but also on preserving textures and details. Specifically, the high-resolution image is recovered from its corresponding low-resolution input in two branches. First, a multi-reception field based branch is designed to let the network fully learn local region features by adaptively selecting local region features in different reception fields. Then, a texture and detail-learning branch supervised by the textures and details decomposed from the ground-truth high resolution image is proposed to provide additional textures and details for the super-resolution process to improve the perceptual quality. Finally, we introduce a gradient loss into the SISR field and define a novel hybrid loss to strengthen boundary information recovery and to avoid overly smooth boundary in the final recovered high-resolution image caused by using only the MAE loss. More importantly, the proposed method is model-agnostic, which can be applied to most off-the-shelf SISR networks. The experimental results on public datasets demonstrate the superiority of our TDPN on most state-of-the-art SISR methods in PSNR, SSIM and perceptual quality. We will share our code on https://github.com/tocaiqing/TDPN.

Most existing trackers use bounding boxes for object tracking. However, the background contained in the bounding box inevitably decreases the accuracy of the target model, which affects the performance of the tracker and is particularly pronounced for non-rigid objects. To address the above issue, this paper proposes a novel hybrid level set model, which can robustly address the issue of topology changing, occlusions and abrupt motion in non-rigid object tracking by accurately tracking the object contour. In particular, an appearance model is first obtained by repeatedly training and relabeling the initial labeled frame using competing one-class SVMs. Then, by integrating the trained appearance model, an edge detector and image spatial information into the level set model, a new hybrid level set model is presented, which accurately locates the object contour and feeds back to the competing one-class SVMs to update the appearance model of the next frame. In addition, a motion model is defined to predict the accurate location of the object when occlusion and abrupt motion occur in the next frame. Finally, the experimental results on state-of-the-art benchmarks demonstrate the feasibility and effectiveness of the proposed model and the superiority of the proposed method over existing trackers in terms of accuracy and robustness.

Background: Several neuroimaging studies report structural alterations of the trigeminal nerve in trigeminal neuralgia (TN). Less attention has been paid to structural brain changes occurring in TN, even though such changes can influence the development and response to treatment of other headache and chronic pain conditions. The purpose of this study was to apply a novel neuroimaging technique-texture analysis-to identify structural brain differences between classical TN patients and healthy subjects. Methods: We prospectively recruited 14 medically refractory classical TN patients and 20 healthy subjects. 3-Tesla T1-weighted brain MRI scans were acquired in all participants. Three texture features (autocorrelation, contrast, energy) were calculated within four a priori brain regions of interest (anterior cingulate, insula, thalamus, brainstem). Voxel-wise analysis was used to identify clusters of texture difference between TN patients and healthy subjects within regions of interest ( p < 0.001, cluster size >20 voxels). Median raw texture values within clusters were also compared between groups, and further used to differentiate TN patients from healthy subjects (receiver-operator characteristic curve analysis). Median raw texture values were correlated with pain severity (visual analog scale, 1-100) and illness duration. Results: Several clusters of texture difference were observed between TN patients and healthy subjects: right-sided TN patients showed reduced autocorrelation in the left brainstem, increased contrast in the left brainstem and right anterior insula, and reduced energy in right and left anterior cingulate, right midbrain, and left brainstem. Within-cluster median raw texture values also differed between TN patients and healthy subjects: TN patients could be segregated from healthy subjects using brainstem autocorrelation ( p = 0.0040, AUC = 0.84, sensitivity = 89%, specificity = 70%), anterior insula contrast ( p = 0.0002, AUC = 0.92, sensitivity = 78%, specificity = 100%), and anterior cingulate energy ( p = 0.0004, AUC = 0.92, sensitivity = 78%, specificity = 100%). Additionally, anterior insula contrast and duration of TN were inversely correlated ( p = 0.030, Spearman r = -0.73). Conclusions: Texture analysis reveals distinct brain abnormalities in TN, which relate to clinical features such as duration of illness. These findings further implicate structural brain changes in the development and maintenance of TN., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Danyluk, Ishaque, Ta, Yang, Wheatley, Kalra and Sankar.)

Background: Texture analysis (TA) is an image-analysis technique that detects complex intervoxel statistical patterns. 3D TA has shown potential in detecting cerebral degeneration not perceptible to the human eye in many neurological disorders. The reliability of this method's application in a multicenter study is unknown., Purpose: To assess the intrasite and intersite reliability of a novel 3D TA method from data acquired systematically from the Canadian ALS Neuroimaging Consortium (CALSNIC)., Study Type: Prospective multicenter data with harmonized MR sequence parameters acquired from five sites., Population: Six healthy subjects., Field Strength: 3T 3D-MPRAGE and 3D-SPGR T 1 -weighted MRI of the brain., Assessment: Voxel-based 3D TA was performed on the whole brain to produce texture maps., Statistical Tests: Intra- and intersite reliability of texture features was assessed using a two-way mixed-effects model for intraclass correlation coefficients (ICC). ICCs were calculated in a region-of-interest (ROI) analysis of predetermined anatomically relevant areas. A voxelwise approach was used to assess the whole brain., Results: In the ROI analyses, intrasite reliability was excellent (ICC > 0.75) across most regions and texture features (autocorrelation [autoc], contrast [contr], energy [energ]). Intersite reliability was excellent for most regions with autoc, ranging from fair to excellent for contr, and ICCs ranging from poor to good (<0.40-0.75) for energ. Voxelwise analyses revealed a large range in ICC across the brain for both intrasite and intersite ICCs (0.0-0.90), with higher reliability in the cortical gray matter compared with deeper subcortical structures., Data Conclusion: Overall, the reliability of 3D TA was highly dependent on texture feature, region studied, and method of analysis (ROI or voxelwise). Intrasite reproducibility was good to excellent, and better than intersite. ROI-based analyses present higher reliability in comparison with voxelwise analyses. Autoc has overall excellent reliability. These factors might be considered when designing future 3D TA studies., Level of Evidence: 2 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2020;51:1200-1209., (© 2019 International Society for Magnetic Resonance in Medicine.)

Gradient-based texture analysis methods have become popular in computer vision and image processing and has many applications including medical image analysis. This motivates us to develop a texture feature extraction method to discriminate Amyotrophic Lateral Sclerosis (ALS) patients from controls. But, the lack of data in ALS research is a major constraint and can be mitigated by using data from multiple centers. However, multi-center data gives some other challenges such as differing scanner parameters and variation in intensity of the medical images, which motivate the development of the proposed method. To investigate these challenges, we propose a gradient-based texture feature extraction method called Modified Co-occurrence Histograms of Oriented Gradients (M-CoHOG) to extract texture features from 2D Magnetic Resonance Images (MRI). We also propose a new feature-normalization technique before feeding the normalized M-CoHOG features into an ensemble of classifiers, which can accommodate for variation of data from different centers. ALS datasets from four different centers are used in the experiments. We analyze the classification accuracy of single center data as well as that arising from multiple centers. It is observed that the extracted texture features from downsampled images are more significant in distinguishing between patients and controls. Moreover, using an ensemble of classifiers shows improvement in classification accuracy over a single classifier in multi-center data. The proposed method outperforms the state-of-the-art methods by a significant margin., (Copyright © 2019 Elsevier Ltd. All rights reserved.)

Objective: Susceptibility-weighted imaging (SWI) has been used to identify neurodegeneration in amyotrophic lateral sclerosis (ALS) through qualitative gross visual comparison of signal intensity. The aim of this study was to quantitatively identify cerebral degeneration in ALS on SWI using texture analysis. Methods: SW images were acquired from 17 ALS patients (58.4 ± 10.3 years, 13M/4F, ALSFRS-R 41.2 ± 4.1) and 18 healthy controls (56.3 ± 17.6 years, 9M/9F) at 4.7 tesla. Textures were computed within the precentral gyrus and basal ganglia and compared between patients and controls using ANCOVA with age and gender as covariates. Texture features were correlated with clinical measures in patients. Texture features found to be significantly different between patients and controls in the precentral gyrus were then used in a whole-brain 3D texture analysis. Results: The texture feature autocorrelation was significantly higher in ALS patients compared to healthy controls in the precentral gyrus and basal ganglia ( p  < 0.05). Autocorrelation correlated significantly with clinical measures such as disease progression rate and finger tapping speed ( p  < 0.05). Whole brain 3D texture analysis using autocorrelation revealed differences between ALS patients and controls within the precentral gyrus on SWI images ( p  < 0.001). Conclusion: Texture analysis on SWI can quantitatively identify cerebral differences between ALS patients and controls.

The purpose of this study was to investigate whether textures computed from T1-weighted (T1W) images of the corticospinal tract (CST) in amyotrophic lateral sclerosis (ALS) are associated with degenerative changes evaluated by diffusion tensor imaging (DTI). Nineteen patients with ALS and 14 controls were prospectively recruited and underwent T1W and diffusion-weighted magnetic resonance imaging. Three-dimensional texture maps were computed from T1W images and correlated with the DTI metrics within the CST. Significantly correlated textures were selected and compared within the CST for group differences between patients and controls using voxel-wise analysis. Textures were correlated with the patients' clinical upper motor neuron (UMN) signs and their diagnostic accuracy was evaluated. Voxel-wise analysis of textures and their diagnostic performance were then assessed in an independent cohort with 26 patients and 13 controls. Results showed that textures autocorrelation, energy, and inverse difference normalized significantly correlated with DTI metrics (p < .05) and these textures were selected for further analyses. The textures demonstrated significant voxel-wise differences between patients and controls in the centrum semiovale and the posterior limb of the internal capsule bilaterally (p < .05). Autocorrelation and energy significantly correlated with UMN burden in patients (p < .05) and classified patients and controls with 97% accuracy (100% sensitivity, 92.9% specificity). In the independent cohort, the selected textures demonstrated similar regional differences between patients and controls and classified participants with 94.9% accuracy. These results provide evidence that T1-based textures are associated with degenerative changes in the CST., (© 2018 Wiley Periodicals, Inc.)

Introduction: Currently, there are no tools that can accurately predict which patients with mild cognitive impairment (MCI) will progress to Alzheimer's disease (AD). Texture analysis uses image processing and statistical methods to identify patterns in voxel intensities that cannot be appreciated by visual inspection. Our main objective was to determine whether MRI texture could be used to predict conversion of MCI to AD., Methods: A method of 3-dimensional, whole-brain texture analysis was used to compute texture features from T1-weighted MR images. To assess predictive value, texture changes were compared between MCI converters and nonconverters over a 3-year observation period. A predictive model using texture and clinical factors was used to predict conversion of patients with MCI to AD. This model was then tested on ten randomly selected test groups from the data set., Results: Texture features were found to be significantly different between normal controls (n = 225), patients with MCI (n = 382), and patients with AD (n = 183). A subset of the patients with MCI were used to compare between MCI converters (n = 98) and nonconverters (n = 106). A composite model including texture features, APOE -ε4 genotype, Mini-Mental Status Examination score, sex, and hippocampal occupancy resulted in an area under curve of 0.905. Application of the composite model to ten randomly selected test groups (nonconverters = 26, converters = 24) predicted MCI conversion with a mean accuracy of 76.2%., Discussion: Early texture changes are detected in patients with MCI who eventually progress to AD dementia. Therefore, whole-brain 3D texture analysis has the potential to predict progression of patients with MCI to AD.

Objective: To evaluate cerebral degenerative changes in ALS and their correlates with survival using 3D texture analysis., Methods: A total of 157 participants were included in this analysis from four neuroimaging studies. Voxel-wise texture analysis on T1-weighted brain magnetic resonance images (MRIs) was conducted between patients and controls. Patients were divided into long- and short-survivors using the median survival of the cohort. Neuroanatomical differences between the two survival groups were also investigated., Results: Whole-brain analysis revealed significant changes in image texture (FDR P  <   0.05) bilaterally in the motor cortex, corticospinal tract (CST), insula, basal ganglia, hippocampus, and frontal regions including subcortical white matter. The texture of the CST correlated ( P  <   0.05) with finger- and foot-tapping rate, measures of upper motor neuron function. Patients with a survival below the media of 19.5 months demonstrated texture change (FDR P  <   0.05) in the motor cortex, CST, basal ganglia, and the hippocampus, a distribution which corresponds to stage 4 of the distribution TDP-43 pathology in ALS. Patients with longer survival exhibited texture changes restricted to motor regions, including the motor cortex and the CST., Interpretation: Widespread gray and white matter pathology is evident in ALS, as revealed by texture analysis of conventional T1-weighted MRI. Length of survival in patients with ALS is associated with the spatial extent of cerebral degeneration.

Background: Evidence of cerebral degeneration is not apparent on routine brain MRI in amyotrophic lateral sclerosis (ALS). Texture analysis can detect change in images based on the statistical properties of voxel intensities. Our objective was to test the utility of texture analysis in detecting cerebral degeneration in ALS. A secondary objective was to determine whether the performance of texture analysis is dependent on image resolution., Methods: High-resolution (0.5×0.5 mm2 in-plane) coronal T2-weighted MRI of the brain were acquired from 12 patients with ALS and 19 healthy controls on a 4.7 Tesla MRI system. Image data sets at lower resolutions were created by down-sampling to 1×1, 2×2, 3×3, and 4×4 mm2. Texture features were extracted from a slice encompassing the corticospinal tract at the different resolutions and tested for their discriminatory power and correlations with clinical measures. Subjects were also classified by visual assessment by expert reviewers., Results: Texture features were different between ALS patients and healthy controls at 1×1, 2×2, and 3×3 mm2 resolutions. Texture features correlated with measures of upper motor neuron function and disability. Optimal classification performance was achieved when best-performing texture features were combined with visual assessment at 2×2 mm2 resolution (0.851 area under the curve, 83% sensitivity, 79% specificity)., Conclusions: Texture analysis can detect subtle abnormalities in MRI of ALS patients. The clinical yield of the method is dependent on image resolution. Texture analysis holds promise as a potential source of neuroimaging biomarkers in ALS.

Noise and constant empirical motion constraints affect the extraction of distinctive spatiotemporal features from one or a few samples per gesture class. To tackle these problems, an adaptive local spatiotemporal feature (ALSTF) using fused RGB-D data is proposed. First, motion regions of interest (MRoIs) are adaptively extracted using grayscale and depth velocity variance information to greatly reduce the impact of noise. Then, corners are used as keypoints if their depth, and velocities of grayscale and of depth meet several adaptive local constraints in each MRoI. With further filtering of noise, an accurate and sufficient number of keypoints is obtained within the desired moving body parts (MBPs). Finally, four kinds of multiple descriptors are calculated and combined in extended gradient and motion spaces to represent the appearance and motion features of gestures. The experimental results on the ChaLearn gesture, CAD-60 and MSRDailyActivity3D datasets demonstrate that the proposed feature achieves higher performance compared with published state-of-the-art approaches under the one-shot learning setting and comparable accuracy under the leave-one-out cross validation., Competing Interests: The authors declare no conflict of interest.

Introduction: Routine MR images do not consistently reveal pathological changes in the brain in ALS. Texture analysis, a method to quantitate voxel intensities and their patterns and interrelationships, can detect changes in images not apparent to the naked eye. Our objective was to evaluate cerebral degeneration in ALS using 3-dimensional texture analysis of MR images of the brain., Methods: In a case-control design, voxel-based texture analysis was performed on T1-weighted MR images of 20 healthy subjects and 19 patients with ALS. Four texture features, namely, autocorrelation, sum of squares variance, sum average, and sum variance were computed. Texture features were compared between the groups by statistical parametric mapping and correlated with clinical measures of disability and upper motor neuron dysfunction., Results: Texture features were different in ALS in motor regions including the precentral gyrus and corticospinal tracts. To a lesser extent, changes were also found in the thalamus, cingulate gyrus, and temporal lobe. Texture features in the precentral gyrus correlated with disease duration, and in the corticospinal tract they correlated with finger tapping speed., Conclusions: Changes in MR image textures are present in motor and non-motor regions in ALS and correlate with clinical features. Whole brain texture analysis has potential in providing biomarkers of cerebral degeneration in ALS.

This paper presents a novel voxel-based method for texture analysis of brain images. Texture analysis is a powerful quantitative approach for analyzing voxel intensities and their interrelationships, but has been thus far limited to analyzing regions of interest. The proposed method provides a 3D statistical map comparing texture features on a voxel-by-voxel basis. The validity of the method was examined on artificially generated effects as well as on real MRI data in Alzheimer's Disease (AD). The artificially generated effects included hyperintense and hypointense signals added to T1-weighted brain MRIs from 30 healthy subjects. The AD dataset included 30 patients with AD and 30 age/sex matched healthy control subjects. The proposed method detected artificial effects with high accuracy and revealed statistically significant differences between the AD and control groups. This paper extends the usage of texture analysis beyond the current region of interest analysis to voxel-by-voxel 3D statistical mapping and provides a hypothesis-free analysis tool to study cerebral pathology in neurological diseases.

This paper presents a method for robust volumetric texture classification. It also proposes 2D and 3D gradient calculation methods designed to be robust to imaging effects and artifacts. Using the proposed 2D method, the gradient information is extracted on the XYZ orthogonal planes at each voxel and used to form a local coordinate system. The local coordinate system and the local 3D gradient computed by the proposed 3D gradient calculator are then used to define volumetric texture features. It is shown that the presented gradient calculation methods can be efficiently implemented by convolving with 2D and 3D kernels. The experimental results demonstrate that the proposed gradient operators and the texture features are robust to imaging effects and artifacts, such as blurriness and noise in 2D and 3D images. The proposed method is compared with three state-of- the-art volumetric texture classification methods the 3D gray level cooccurance matrix, 3D local binary patterns, and second orientation pyramid on magnetic resonance imaging data of the brain. The experimental results show the superiority of the proposed method in accuracy, robustness, and speed.

This paper presents a novel rotation invariant method for texture classification based on local frequency components. The local frequency components are computed by applying 1-D Fourier transform on a neighboring function defined on a circle of radius R at each pixel. We observed that the low frequency components are the major constituents of the circular functions and can effectively represent textures. Three sets of features are extracted from the low frequency components, two based on the phase and one based on the magnitude. The proposed features are invariant to rotation and linear changes of illumination. Moreover, by using low frequency components, the proposed features are very robust to noise. While the proposed method uses a relatively small number of features, it outperforms state-of-the-art methods in three well-known datasets: Brodatz, Outex, and CUReT. In addition, the proposed method is very robust to noise and can remarkably improve the classification accuracy especially in the presence of high levels of noise.

In an underwater imaging system, a perspective camera is often placed outside a tank or in waterproof housing with a flat glass window. The refraction of light occurs when a light ray passes through the water-glass and air-glass interface, rendering the conventional multiple view geometry based on the single viewpoint (SVP) camera model invalid. While most recent underwater vision studies mainly focus on the challenging topic of calibrating such systems, no previous work has systematically studied the influence of refraction on underwater three-dimensional (3D) reconstruction. This paper demonstrates the possibility of using the SVP camera model in underwater 3D reconstruction through theoretical analysis of refractive distortion and simulations. Then, the performance of the SVP camera model in multiview underwater 3D reconstruction is quantitatively evaluated. The experimental results reveal a rather surprising and useful yet overlooked fact that the SVP camera model with radial distortion correction and focal length adjustment can compensate for refraction and achieve high accuracy in multiview underwater 3D reconstruction (within 0.7 mm for an object of dimension 200 mm) compared with the results of land-based systems. Such an observation justifies the use of the SVP camera model in underwater application for reconstructing reliable 3D scenes. Our results can be used to guide the selection of system parameters in the design of an underwater 3D imaging setup.

Match cost functions are common elements of every stereopsis algorithm that are used to provide a dissimilarity measure between pixels in different images. Global stereopsis algorithms incorporate assumptions about the smoothness of the resulting distance map that can interact with match cost functions in unpredictable ways. In this paper, we present a large-scale study on the relative performance of a structured set of match cost functions within several global stereopsis frameworks. We compare 272 match cost functions that are built from component parts in the context of four global stereopsis frameworks with a data set consisting of 57 stereo image pairs at three different variances of synthetic sensor noise. From our analysis, we infer a set of general rules that can be used to guide derivation of match cost functions for use in global stereopsis algorithms.

This paper presents a new technique, called aura 3D textures, for generating solid textures based on input examples. Our method is fully automatic and requires no user interactions in the process. Given an input texture sample, our method first creates its aura matrix representations and then generates a solid texture by sampling the aura matrices of the input sample constrained in multiple view directions. Once the solid texture is generated, any given object can be textured by the solid texture. We evaluate the results of our method based on extensive user studies. Based on the evaluation results using human subjects, we conclude that our algorithm can generate faithful results of both stochastic and structural textures with an average successful rate of 76.4 percent. Our experimental results also show that the new method outperforms Wei and Levoy's method and is comparable to that proposed by Jagnow et al. [21].

In this paper, we present a novel method for synchronizing multiple (more than two) uncalibrated video sequences recording the same event by free-moving full-perspective cameras. Unlike previous synchronization methods, our method takes advantage of tri-view geometry constraints instead of the commonly used two-view one for their better performance in measuring geometric alignment when video frames are synchronized. In particular, the tri-ocular geometric constraint of point/line features, which is evaluated by tri-focal transfer, is enforced when building the timeline maps for sequences to be synchronized. A hierarchical approach is used to reduce the computational complexity. To achieve subframe synchronization accuracy, the Levenberg-Marquardt method-based optimization is performed. The experimental results on several synthetic and real video datasets demonstrate the effectiveness and robustness of our method over previous methods in synchronizing full-perspective videos.

An efficient unambiguous stereo matching technique is presented in this paper. Our main contribution is to introduce a new reliability measure to dynamic programming approaches in general. For stereo vision application, the reliability of a proposed match on a scanline is defined as the cost difference between the globally best disparity assignment that includes the match and the globally best assignment that does not include the match. A reliability-based dynamic programming algorithm is derived accordingly, which can selectively assign disparities to pixels when the corresponding reliabilities exceed a given threshold. The experimental results show that the new approach can produce dense (> 70 percent of the unoccluded pixels) and reliable (error rate < 0.5 percent) matches efficiently (< 0.2 sec on a 2GHz P4) for the four Middlebury stereo data sets.