Search

The inductive bias of the convolutional neural network (CNN) can be a strong prior for image restoration, which is known as the Deep Image Prior (DIP). Recently, DIP is utilized in unsupervised dynamic MRI reconstruction, which adopts a generative model from the latent space to the image space. However, existing methods usually use a pyramid-shaped CNN generator shared by all frames, embedding the temporal modeling within the latent space, which may hamper the model expression capability. In this work, we propose a novel scheme for dynamic MRI representation, named ``Graph Image Prior'' (GIP). GIP adopts a two-stage generative network in a new modeling methodology, which first employs independent CNNs to recover the image structure for each frame, and then exploits the spatio-temporal correlations within the feature space parameterized by a graph model. A graph convolutional network is utilized for feature fusion and dynamic image generation. In addition, we devise an ADMM algorithm to alternately optimize the images and the network parameters to improve the reconstruction performance. Experiments were conducted on cardiac cine MRI reconstruction, which demonstrate that GIP outperforms compressed sensing methods and other DIP-based unsupervised methods, significantly reducing the performance gap with state-of-the-art supervised algorithms. Moreover, GIP displays superior generalization ability when transferred to a different reconstruction setting, without the need for any additional data.

The development of the manufacturing systems has made it increasingly necessary to monitor the data generated by multiple interconnected subsystems with rapid incoming of samples. Based on incremental Singular Value Decomposition (ISVD), we develop a general online monitoring approach for the relationship of data generated from two interconnected subsystems, where each subsystem produces big data streams with several variation patterns in normal working condition. When special situation happens and new associations occur, a very small amount of computation is sufficient to update the system status and compute the control statistics by using this approach. The proposed method reduces computational overhead and retains only a small number of pairs of possible dependent patterns at each step. The validation of the method through simulation studies and a case study on semiconductor manufacturing processes further supports its effectiveness.

In this work, we proposed a continuous-acquisition strategy using a gradient echo (GRE) inversion recovery sequence based on spiral trajectories to simultaneously obtain the $T_1$ mapping and CINE imaging. The acquisition is using a free-breathing and ungated fashion. An approach based on variational auto-encoder(VAE) is used for the motion estimation from the centered k-space data. The motion signal is then used to train a deep manifold reconstruction algorithm for image reconstruction. Once the network is trained, we can excite the latent vectors (the estimated motion signals and the contrast signal) in any way as we wanted to generate the image frames in the time series. We can estimate the $T_1$ mapping using the generated image frames where only contrast is varying. We can also generate the breath-hold CINE in different contrast.

We introduce an unsupervised motion-compensated reconstruction scheme for high-resolution free-breathing pulmonary MRI. We model the image frames in the time series as the deformed version of the 3D template image volume. We assume the deformation maps to be points on a smooth manifold in high-dimensional space. Specifically, we model the deformation map at each time instant as the output of a CNN-based generator that has the same weight for all time-frames, driven by a low-dimensional latent vector. The time series of latent vectors account for the dynamics in the dataset, including respiratory motion and bulk motion. The template image volume, the parameters of the generator, and the latent vectors are learned directly from the k-t space data in an unsupervised fashion. Our experimental results show improved reconstructions compared to state-of-the-art methods, especially in the context of bulk motion during the scans.

Deep learning algorithms that rely on extensive training data are revolutionizing image recovery from ill-posed measurements. Training data is scarce in many imaging applications, including ultra-high-resolution imaging. The deep image prior (DIP) algorithm was introduced for single-shot image recovery, completely eliminating the need for training data. A challenge with this scheme is the need for early stopping to minimize the overfitting of the CNN parameters to the noise in the measurements. We introduce a generalized Stein's unbiased risk estimate (GSURE) loss metric to minimize the overfitting. Our experiments show that the SURE-DIP approach minimizes the overfitting issues, thus offering significantly improved performance over classical DIP schemes. We also use the SURE-DIP approach with model-based unrolling architectures, which offers improved performance over direct inversion schemes.

Free-breathing cardiac MRI schemes are emerging as competitive alternatives to breath-held cine MRI protocols, enabling applicability to pediatric and other population groups that cannot hold their breath. Because the data from the slices are acquired sequentially, the cardiac/respiratory motion patterns may be different for each slice; current free-breathing approaches perform independent recovery of each slice. In addition to not being able to exploit the inter-slice redundancies, manual intervention or sophisticated post-processing methods are needed to align the images post-recovery for quantification. To overcome these challenges, we propose an unsupervised variational deep manifold learning scheme for the joint alignment and reconstruction of multislice dynamic MRI. The proposed scheme jointly learns the parameters of the deep network as well as the latent vectors for each slice, which capture the motion-induced dynamic variations, from the k-t space data of the specific subject. The variational framework minimizes the non-uniqueness in the representation, thus offering improved alignment and reconstructions.

Melts of 3-dimensional dendritic beads-springs, namely coarse-grained soft-clusters, are studied by molecular dynamics simulations. The goal is to elucidate the unique dynamics of giant molecules, or generally speaking, 3-dimensional architectured polymers. When constituted by more than the critical number around 200 beads, soft-clusters cannot diffuse or relax far above their glass transition temperature, although relaxation can happen on the level of beads. Each soft-cluster can only rotate in the cage formed by neighboring soft-clusters. Such a non-diffusing state would transform to the liquid state at exceptionally high temperature, e.g. 10 times the glass transition temperature. Agreeing with experiments, 3D hierarchies lead to unique dynamics, especially their divergent relaxation times with the number of beads. These unique dynamics are in sharp contrast with 1-dimensional chain-like polymers. We name such a special state as 'cooperative glass', because of the 'cooperation' of the 3D-connected beads. The design of soft-clusters may also resemble cooperative rearranging regions where cooperativeness is contributed by low temperature, thus offer further insights into the glass problem.

We introduce a generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The model assumes that the images in the dataset are non-linear mappings of low-dimensional latent vectors. We use the deep convolutional neural network (CNN) to represent the non-linear transformation. The parameters of the generator as well as the low-dimensional latent vectors are jointly estimated only from the undersampled measurements. This approach is different from traditional CNN approaches that require extensive fully sampled training data. We penalize the norm of the gradients of the non-linear mapping to constrain the manifold to be smooth, while temporal gradients of the latent vectors are penalized to obtain a smoothly varying time-series. The proposed scheme brings in the spatial regularization provided by the convolutional network. The main benefit of the proposed scheme is the improvement in image quality and the orders-of-magnitude reduction in memory demand compared to traditional manifold models. To minimize the computational complexity of the algorithm, we introduce an efficient progressive training-in-time approach and an approximate cost function. These approaches speed up the image reconstructions and offers better reconstruction performance.

We introduce a novel generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The proposed generative framework represents the image time series as a smooth non-linear function of low-dimensional latent vectors that capture the cardiac and respiratory phases. The non-linear function is represented using a deep convolutional neural network (CNN). Unlike the popular CNN approaches that require extensive fully-sampled training data that is not available in this setting, the parameters of the CNN generator as well as the latent vectors are jointly estimated from the undersampled measurements using stochastic gradient descent. We penalize the norm of the gradient of the generator to encourage the learning of a smooth surface/manifold, while temporal gradients of the latent vectors are penalized to encourage the time series to be smooth. The main benefits of the proposed scheme are (a) the quite significant reduction in memory demand compared to the analysis based SToRM model, and (b) the spatial regularization brought in by the CNN model. We also introduce efficient progressive approaches to minimize the computational complexity of the algorithm.

Current deep learning-based manifold learning algorithms such as the variational autoencoder (VAE) require fully sampled data to learn the probability density of real-world datasets. Once learned, the density can be used for a variety of tasks, including data imputation. However, fully sampled data is often unavailable in a variety of problems, including the recovery of dynamic and high-resolution MRI data considered in this work. To overcome this problem, we introduce a novel variational approach to learn a manifold from undersampled data. The VAE uses a decoder fed by latent vectors, drawn from a conditional density estimated from the fully sampled images using an encoder. Since fully sampled images are not available in our setting, we approximate the conditional density of the latent vectors by a parametric model whose parameters are estimated from the undersampled measurements using back-propagation. We use the framework for the joint alignment and recovery of multislice free breathing and ungated cardiac MRI data from highly undersampled measurements. Most of the current self-gating and manifold cardiac MRI approaches consider the independent recovery of images from each slice; these methods are not capable of exploiting the inter-slice redundancies in the datasets and require sophisticated post-processing or manual approaches to align the images from different slices. By contrast, the proposed scheme is able to align the multislice data and exploit the redundancies. Experimental results demonstrate the utility of the proposed scheme in dynamic imaging alignment and reconstructions.

Several imaging algorithms including patch-based image denoising, image time series recovery, and convolutional neural networks can be thought of as methods that exploit the manifold structure of signals. While the empirical performance of these algorithms is impressive, the understanding of recovery of the signals and functions that live on manifold is less understood. In this paper, we focus on the recovery of signals that live on a union of surfaces. In particular, we consider signals living on a union of smooth band-limited surfaces in high dimensions. We show that an exponential mapping transforms the data to a union of low-dimensional subspaces. Using this relation, we introduce a sampling theoretical framework for the recovery of smooth surfaces from few samples and the learning of functions living on smooth surfaces. The low-rank property of the features is used to determine the number of measurements needed to recover the surface. Moreover, the low-rank property of the features also provides an efficient approach, which resembles a neural network, for the local representation of multidimensional functions on the surface.

We introduce a sampling theoretic framework for the recovery of smooth surfaces and functions living on smooth surfaces from few samples. The proposed approach can be thought of as a nonlinear generalization of union of subspace models widely used in signal processing. This scheme relies on an exponential lifting of the original data points to feature space, where the features live on union of subspaces. The low-rank property of the features are used to recover the surfaces as well as to determine the number of measurements needed to recover the surface. The low-rank property of the features also provides an efficient approach which resembles a neural network for the local representation of multidimensional functions on the surface; the significantly reduced number of parameters make the computational structure attractive for learning inference from limited labeled training data.

We introduce a continuous domain framework for the recovery of a planar curve from a few samples. We model the curve as the zero level set of a trigonometric polynomial. We show that the exponential feature maps of the points on the curve lie on a low-dimensional subspace. We show that the null-space vector of the feature matrix can be used to uniquely identify the curve, given a sufficient number of samples. The worst-case theoretical guarantees show that the number of samples required for unique recovery depends on the bandwidth of the underlying trigonometric polynomial, which is a measure of the complexity of the curve. We introduce an iterative algorithm that relies on the low-rank property of the feature maps to recover the curves when the samples are noisy or when the true bandwidth of the curve is unknown. We also demonstrate the preliminary utility of the proposed curve representation in the context of image segmentation.