Search

We consider the problem of forecasting a high-dimensional time series that can be modeled as matrices where each column denotes a measurement and use low-rank matrix factorization for predicting future values or imputing missing ones. We define and analyze our problem in the online setting in which the data arrive as a stream and only a single pass is allowed. We present and analyze new matrix factorization techniques that can learn low-dimensional embeddings effectively in an online manner. Based on these embeddings, we derive a recursive minimum mean square error estimator based on an autoregressive model. Experiments with two real datasets of tens of millions of measurements show the benefits of the proposed approach. [ABSTRACT FROM AUTHOR]

We consider the nonlinear Kalman filtering problem using Kullback–Leibler (KL) and $\alpha$-divergence measures as optimization criteria. Unlike linear Kalman filters, nonlinear Kalman filters do not have closed form Gaussian posteriors because of a lack of conjugacy due to the nonlinearity in the likelihood. In this paper, we propose novel algorithms to approximate this posterior by optimizing the forward and reverse forms of the KL divergence, as well as the $\alpha$-divergence that contains these two as limiting cases. Unlike previous approaches, our algorithms do not make approximations to the divergences being optimized, but use Monte Carlo techniques to derive unbiased algorithms for direct optimization. We assess performance on radar and sensor tracking, and options pricing, showing general improvement over the extended, unscented, and ensemble Kalman filters, as well as competitive performance with particle filtering. [ABSTRACT FROM PUBLISHER]

Background: Cancer is a complex disease driven by somatic genomic alterations (SGAs) that perturb signaling pathways and consequently cellular function. Identifying patterns of pathway perturbations would provide insights into common disease mechanisms shared among tumors, which is important for guiding treatment and predicting outcome. However, identifying perturbed pathways is challenging, because different tumors can have the same perturbed pathways that are perturbed by different SGAs. Here, we designed novel semantic representations that capture the functional similarity of distinct SGAs perturbing a common pathway in different tumors. Combining this representation with topic modeling would allow us to identify patterns in altered signaling pathways. Results: We represented each gene with a vector of words describing its function, and we represented the SGAs of a tumor as a text document by pooling the words representing individual SGAs. We applied the nested hierarchical Dirichlet process (nHDP) model to a collection of tumors of 5 cancer types from TCGA. We identified topics (consisting of co-occurring words) representing the common functional themes of different SGAs. Tumors were clustered based on their topic associations, such that each cluster consists of tumors sharing common functional themes. The resulting clusters contained mixtures of cancer types, which indicates that different cancer types can share disease mechanisms. Survival analysis based on the clusters revealed significant differences in survival among the tumors of the same cancer type that were assigned to different clusters. Conclusions: The results indicate that applying topic modeling to semantic representations of tumors identifies patterns in the combinations of altered functional pathways in cancer. Keywords: Cancer, Topic modeling, Semantic representation, Cancer genomics. [ABSTRACT FROM AUTHOR]

We develop a nested hierarchical Dirichlet process (nHDP) for hierarchical topic modeling. The nHDP generalizes the nested Chinese restaurant process (nCRP) to allow each word to follow its own path to a topic node according to a per-document distribution over the paths on a shared tree. This alleviates the rigid, single-path formulation assumed by the nCRP, allowing documents to easily express complex thematic borrowings. We derive a stochastic variational inference algorithm for the model, which enables efficient inference for massive collections of text documents. We demonstrate our algorithm on 1.8 million documents from The New York Times and 2.7 million documents from Wikipedia. [ABSTRACT FROM AUTHOR]

We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled \(k \) -space data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prior for representing an image patch as a sparse combination of dictionary elements. The size of the dictionary and patch-specific sparsity pattern are inferred from the data, in addition to other dictionary learning variables. Dictionary learning is performed directly on the compressed image, and so is tailored to the MRI being considered. In addition, we investigate a total variation penalty term in combination with the dictionary learning model, and show how the denoising property of dictionary learning removes dependence on regularization parameters in the noisy setting. We derive a stochastic optimization algorithm based on Markov chain Monte Carlo for the Bayesian model, and use the alternating direction method of multipliers for efficiently performing total variation minimization. We present empirical results on several MRI, which show that the proposed regularization framework can improve reconstruction accuracy over other methods. [ABSTRACT FROM PUBLISHER]

We develop an active learning algorithm for kernel-based linear regression and classification. The proposed greedy algorithm employs a minimum-entropy criterion derived using a Bayesian interpretation of ridge regression. We assume access to a matrix, Φ ϵ ℝN × N, for which the (i, j) th element is defined by the kernel function K (γi, γj) ϵ ℝ, with the observed data γi ϵ ℝd. We seek a model, M : γi → yi, where yi is a real-valued response or integer-valued label, which we do not have access to a priori. To achieve this goal, a submatrix, Φ Il, Ib ϵ ℝn×m, is sought that corresponds to the intersection of n rows and m columns of Φ, indexed by the sets Il and Ib, respectively. Typically m ≪ N and n ≪ N. We have two objectives: (i) Determine the m columns of Φ, indexed by the set Ib, that are the most informative for building a linear model, M : [1Φi,Ib]T → yi, without any knowledge of {yi}i=1N and (ii) using active learning, sequentially determine which subset of n elements of {yi}i=1N should be acquired; both stopping values, ∣Ib∣ = m and ∣Il∣ = n, are also to be inferred from the data. These steps are taken with the goal of minimizing the uncertainty of the model parameters, x, as measured by the differential entropy of its posterior distribution. The parameter vector x ϵ ℝm, as well as the model bias η ϵ ℝ, is then learned from the resulting problem, yIl = ΦIl, Ib x + η1 + ϵ. The remaining N - n responses/labels not included in yIl can be inferred by applying x to the remaining N - n rows of Φ :, Ib. We show experimental results for several regression and classification problems, and compare to other active learning methods. [ABSTRACT FROM AUTHOR]

The number of states in a hidden Markov model (HMM) is an important parameter that has a critical impact on the inferred model. Bayesian approaches to addressing this issue include the nonparametric hierarchical Dirichlet process, which does not extend to a variational Bayesian (VB) solution. We present a fully conjugate, Bayesian approach to determining the number of states in a HMM, which does have a variational solution. The infinite-state HMM presented here utilizes a stick-breaking construction for each row of the state transition matrix, which allows for a sparse utilization of the same subset of observation parameters by all states. In addition to our variational solution, we discuss retrospective and collapsed Gibbs sampling methods for MCMC inference. We demonstrate our model on a music recommendation problem containing 2250 pieces of music from the classical, jazz, and rock genres. [ABSTRACT FROM AUTHOR]

A new hierarchical nonparametric Bayesian framework is proposed for the problem of multi-task learning (MTL) with sequential data. The models for multiple tasks, each characterized by sequential data, are learned jointly, and the intertask relationships are obtained simultaneously. This MTL setting is used to analyze and sort large databases composed of sequential data, such as music clips. Within each data set, we represent the sequential data with an infinite hidden Markov model (iHMM), avoiding the problem of model selection (selecting a number of States). Across the data sets, the multiple iHMMs are learned jointly in a MTL setting, employing a nested Dirichlet process (nDP). The nDP-iHMM MTL method allows simultaneous task-level and data-level clustering, with which the individual iHMMs are enhanced and the between-task similarities are learned. Therefore, in addition to improved learning of each of the models via appropriate data sharing, the learned sharing mechanisms are used to infer interdata relationships of interest for data search. Specifically, the MTL-Iearned task-level sharing mechanisms are used to define the affinity matrix in a graph-diffusion sorting framework. To speed up the MCMC inference for large databases, the nDP-iHMM is truncated to yield a nested Dirichlet-distribution based HMM representation, which accommodates fast variational Bayesian (VB) analysis for large-scale inference, and the effectiveness of the framework is demonstrated using a database composed of 2500 digital music pieces. [ABSTRACT FROM AUTHOR]

We develop a hidden Markov mixture model based on a Dirichlet process (DP) prior, for representation of the statistics of sequential data for which a single hidden Markov model (HMM) may not be sufficient. The DP prior has an intrinsic clustering property that encourages parameter sharing, and this naturally reveals the proper number of mixture components. The evaluation of posterior distributions for all model parameters is achieved in two ways: 1) via a rigorous Markov chain Monte Carlo method; and 2) approximately and efficiently via a variational Bayes formulation. Using DP HMM mixture models in a Bayesian setting, we propose a novel scheme for music analysis, highlighting the effectiveness of the DP HMM mixture model. Music is treated as a time-series data sequence and each music piece is represented as a mixture of HMMs. We approximate the similarity of two music pieces by computing the distance between the associated HMM mixtures. Experimental results are presented for synthesized sequential data and from classical music clips. Music similarities computed using DP HMM mixture modeling are compared to those computed from Gaussian mixture modeling, for which the mixture modeling is also performed using DP. The results show that the performance of DP HMM mixture modeling exceeds that of the DP Gaussian mixture modeling. [ABSTRACT FROM AUTHOR]

The article reports on using machine learning methods to identify patterns in time-dependent spectral properties of seismic signals and enable detection of changes in faulting processes. These methods characterize subtle changes in seismic source properties, with applications to tectonic and geothermal seismicity. The study examined 46,000 earthquakes from the Geysers geothermal field (CA).

Replies to comments made by Ginny Childs on an article in the 'Journal of the American Medical Association,' about facial injuries due to pet ferrets.

We develop a Bayesian nonparametric approach to a general family of latent class problems in which individuals can belong simultaneously to multiple classes and where each class can be exhibited multiple times by an individual. We introduce a combinatorial stochastic process known as the negative binomial process ( NBP ) as an infinite-dimensional prior appropriate for such problems. We show that the NBP is conjugate to the beta process, and we characterize the posterior distribution under the beta-negative binomial process (BNBP ) and hierarchical models based on the BNBP (the HBNBP ). We study the asymptotic properties of the BNBP and develop a three-parameter extension of the BNBP that exhibits power-law behavior. We derive MCMC algorithms for posterior inference under the HBNBP , and we present experiments using these algorithms in the domains of image segmentation, object recognition, and document analysis. [ABSTRACT FROM PUBLISHER]

Nonparametric Bayesian methods are considered for recovery of imagery based upon compressive, incomplete, and/or noisy measurements. A truncated beta-Bernoulli process is employed to infer an appropriate dictionary for the data under test and also for image recovery. In the context of compressive sensing, significant improvements in image recovery are manifested using learned dictionaries, relative to using standard orthonormal image expansions. The compressive-measurement projections are also optimized for the learned dictionary. Additionally, we consider simpler (incomplete) measurements, defined by measuring a subset of image pixels, uniformly selected at random. Spatial interrelationships within imagery are exploited through use of the Dirichlet and probit stick-breaking processes. Several example results are presented, with comparisons to other methods in the literature. [ABSTRACT FROM PUBLISHER]

Nonparametric Bayesian methods are employed to constitute a mixture of low-rank Gaussians, for data \mbi x\in\BBR^N that are of high dimension N but are constrained to reside in a low-dimensional subregion of \BBR^N. The number of mixture components and their rank are inferred automatically from the data. The resulting algorithm can be used for learning manifolds and for reconstructing signals from manifolds, based on compressive sensing (CS) projection measurements. The statistical CS inversion is performed analytically. We derive the required number of CS random measurements needed for successful reconstruction, based on easily-computed quantities, drawing on block-sparsity properties. The proposed methodology is validated on several synthetic and real datasets. [ABSTRACT FROM AUTHOR]

We consider the problem of inferring and modeling topics in a sequence of documents with known publication dates. The documents at a given time are each characterized by a topic and the topics are drawn from a mixture model. The proposed model infers the change in the topic mixture weights as a function of time. The details of this general framework may take different forms, depending on the specifics of the model. For the examples considered here, we examine base measures based on independent multinomial-Dirichlet measures for representation of topic-dependent word counts. The form of the hierarchical model allows efficient variational Bayesian inference, of interest for large-scale problems. We demonstrate results and make comparisons to the model when the dynamic character is removed, and also compare to latent Dirichlet allocation (LDA) and Topics over Time (TOT). We consider a database of Neural Information Processing Systems papers as well as the US Presidential State of the Union addresses from 1790 to 2008. [ABSTRACT FROM AUTHOR]

Background: Nonparametric Bayesian techniques have been developed recently to extend the sophistication of factor models, allowing one to infer the number of appropriate factors from the observed data. We consider such techniques for sparse factor analysis, with application to gene-expression data from three virus challenge studies. Particular attention is placed on employing the Beta Process (BP), the Indian Buffet Process (IBP), and related sparseness-promoting techniques to infer a proper number of factors. The posterior density function on the model parameters is computed using Gibbs sampling and variational Bayesian (VB) analysis. Results: Time-evolving gene-expression data are considered for respiratory syncytial virus (RSV), Rhino virus, and influenza, using blood samples from healthy human subjects. These data were acquired in three challenge studies, each executed after receiving institutional review board (IRB) approval from Duke University. Comparisons are made between several alternative means of per-forming nonparametric factor analysis on these data, with comparisons as well to sparse-PCA and Penalized Matrix Decomposition (PMD), closely related non-Bayesian approaches. Conclusions: Applying the Beta Process to the factor scores, or to the singular values of a pseudo-SVD construction, the proposed algorithms infer the number of factors in gene-expression data. For real data the "true" number of factors is unknown; in our simulations we consider a range of noise variances, and the proposed Bayesian models inferred the number of factors accurately relative to other methods in the literature, such as sparse-PCA and PMD. We have also identified a "pan-viral" factor of importance for each of the three viruses considered in this study. We have identified a set of genes associated with this pan-viral factor, of interest for early detection of such viruses based upon the host response, as quantified via gene-expression data. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have shown their advantages in the single image de-raining task. However, most existing CNNs-based methods utilize only local spatial information without considering long-range contextual information. In this paper, we propose a graph convolutional networks (GCNs)-based model to solve the above problem. We specifically design two graphs to extract representations from new dimensions. The first graph models the global spatial relationship between pixels in the feature, while the second graph models the interrelationship across the channels. By integrating conventional CNNs and our GCNs into a single framework, the proposed method is able to explore comprehensive feature representations from three aspects, i.e., local spatial patterns, global spatial coherence and channel correlation. To better exploit the explored rich feature representations, we further introduce a simple yet effective recurrent operations to perform the de-raining process in a successive manner. Benefiting from the rich information exploration and exploitation, our method achieves state-of-the-art results on both synthetic and real-world data sets. [ABSTRACT FROM AUTHOR]

Compressed sensing is utilized with the aims of reconstructing an MRI using a fraction of measurements to accelerate magnetic resonance imaging called compressed sensing magnetic resonance imaging (CS-MRI). Conventional optimization-based CS-MRI methods use random under-sampling patterns and model the MRI data in the image domain as the classic CS-MRI paradigm. Instead, we design a uniform under-sampling strategy and explore the potential of modeling the MRI data directly in the measured Fourier domain. We propose a dual-domain deep lattice network (DD-DLN) for CS-MRI with variable density uniform under-sampling. We train the networks to learn the mapping between both image and frequency domains. We observe the dual networks have complementary advantages, which motivates their combination via a lattice structure. Experiments show that the proposed DD-DLN model provides promising performance in CS-MRI under the designed variable density uniform under-sampling. [ABSTRACT FROM AUTHOR]

We present a weakly-supervised technique for learning to remove rain from images without using synthetic rain software. The method is based on a two-stage data distillation approach, which requires only some unpaired rainy and clean images to generate supervision. First, a rainy image is paired with a coarsely derained version using on a simple filtering technique (“rain-to-clean”). Then a clean image is randomly matched with the rainy soft-labeled pair. Through a shared deep neural network, the rain that is removed from the first image is then added to the clean image to generate a second pair (“clean-to-rain”). The neural network simultaneously learns to map both images such that high resolution structure in the clean images can inform the deraining of the rainy images. Demonstrations show that this approach can address those visual characteristics of rain not easily synthesized by software in the usual way. [ABSTRACT FROM AUTHOR]

The segmentation of brain tissue in MRI is valuable for extracting brain structure to aid diagnosis, treatment and tracking the progression of different neurologic diseases. Medical image data are volumetric and some neural network models for medical image segmentation have addressed this using a 3D convolutional architecture. However, this volumetric spatial information has not been fully exploited to enhance the representative ability of deep networks, and these networks have not fully addressed the practical issues facing the analysis of multimodal MRI data. In this paper, we propose a spatially-weighted 3D network (SW-3D-UNet) for brain tissue segmentation of single-modality MRI, and extend it using multimodality MRI data. We validate our model on the MRBrainS13 and MALC12 datasets. This unpublished model ranked first on the leaderboard of the MRBrainS13 Challenge. [ABSTRACT FROM AUTHOR]

Compressive sensing enables fast magnetic resonance imaging (MRI) reconstruction with undersampled k-space data. However, in most existing MRI reconstruction models, the whole MR image is targeted and reconstructed without taking specific tissue regions into consideration. This may fails to emphasize the reconstruction accuracy on important and region-of-interest (ROI) tissues for diagnosis. In some ROI-based MRI reconstruction models, the ROI mask is extracted by human experts in advance, which is laborious when the MRI datasets are too large. In this paper, we propose a deep neural network architecture for ROI MRI reconstruction called ROIRecNet to improve reconstruction accuracy of the ROI regions in under-sampled MRI. In the model, we obtain the ROI masks by feeding an initially reconstructed MRI from a pre-trained MRI reconstruction network (RecNet) to a pre-trained MRI segmentation network (ROINet). Then we fine-tune the RecNet with a binary weighted ℓ 2 loss function using the produced ROI mask. The resulting ROIRecNet can offer more focus on the ROI. We test the model on the MRBrainS13 dataset with different brain tissues being ROIs. The experiment shows the proposed ROIRecNet can significantly improve the reconstruction quality of the region of interest. [ABSTRACT FROM AUTHOR]

Compressed sensing (CS) theory assures us that we can accurately reconstruct magnetic resonance images using fewer k-space measurements than the Nyquist sampling rate requires. In traditional CS-MRI inversion methods, the fact that the energy within the Fourier measurement domain is distributed non-uniformly is often neglected during reconstruction. As a result, more densely sampled low frequency information tends to dominate penalization schemes for reconstructing MRI at the expense of high frequency details. In this paper, we propose a new framework for CS-MRI inversion in which we decompose the observed k-space data into "subspaces" via sets of filters in a lossless way, and reconstruct the images in these various spaces individually using off-the-shelf algorithms. We then fuse the results to obtain the final reconstruction. In this way, we are able to focus reconstruction on frequency information within the entire k-space more equally, preserving both high and low frequency details. We demonstrate that the proposed framework is competitive with state-of-the-art methods in CS-MRI in terms of quantitative performance, and often improves an algorithm's results qualitatively compared with its direct application to k-space. Unlabelled Image • The MRI measurements distribute non-uniformly in k-space. • A divide-and-conquer (DAC) approach for compressed sensing MRI is proposed. • The DAC approach can fully utilize the statistical characteristics of each k-space subspace. • The DAC approach often improves an undersampled MRI algorithm's results. [ABSTRACT FROM AUTHOR]

The anisotropy of radar cross section of scatterers makes the scatterer trajectory matrix incomplete in sequential inverse synthetic aperture radar images. As a result, factorization methods cannot be directly applied to reconstruct the 3-D geometry of scatterers without additional consideration. We propose a modified expectation-maximization (EM) algorithm to retrieve the complete scatterer trajectory matrix. First, we derive the motion dynamics of the projected scatterer, which approximates an ellipse. Then, based on the estimated ellipse parameters using the known data of each scatterer trajectory, we use the Kalman filter to initialize the missing data. To address the limitations of a traditional EM, which only considers the rank-deficient characteristics of the scatterer trajectory matrix, we propose to augment EM by using both the known rank-deficient and elliptical motion characteristics. Experimental results on simulated data verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

We propose a straightforward and efficient fusion-based method for enhancing weakly illumination images that uses several mature image processing techniques. First, we employ an illumination estimating algorithm based on morphological closing to decompose an observed image into a reflectance image and an illumination image. We then derive two inputs that represent luminance-improved and contrast-enhanced versions of the first decomposed illumination using the sigmoid function and adaptive histogram equalization. Designing two weights based on these inputs, we produce an adjusted illumination by fusing the derived inputs with the corresponding weights in a multi-scale fashion. Through a proper weighting and fusion strategy, we blend the advantages of different techniques to produce the adjusted illumination. The final enhanced image is obtained by compensating the adjusted illumination back to the reflectance. Through this synthesis, the enhanced image represents a trade-off among detail enhancement, local contrast improvement and preserving the natural feel of the image. In the proposed fusion-based framework, images under different weak illumination conditions such as backlighting, non-uniform illumination and nighttime can be enhanced. [ABSTRACT FROM AUTHOR]

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets. [ABSTRACT FROM AUTHOR]

Presents several letters to the editor about pediatric diseases and care. Enzymuria as a marker of renal injury and disease; Discussion on the trauma-associated periorbital cellulitis due to Haemophilus influenza; Difference between chlamydial pneumonia of infancy and Wilson-Mikity syndrome.