Search

The article discusses the theoretical results about dictionary learning in signal processing. It explores several perspectives of dictionary learning as an unsupervised learning or a matrix factorization problem including non-convex optimization problem, symmetries and consistency results. Also mentioned is a manifold interpretation of sparse coding with projective geometry.

The article discusses the theoretical results of compressed sensing and sparse recovery in parsimony. It focuses on methods based on the l1-norm, especially on the basis pursuit formulation of structured sparsity-inducing norms. Also mentioned are assumptions on the practice of sparse recovery such as support recovery and sign consistency, code estimation and signal denoising.

The article focuses on the dictionary learning introduced by Bruno Olshausen and David J. Field. The scientific method is used for signal and image processing and neuroscience which are related to classical theories of the mammalian visual cortex. Also mentioned are matrix factorization point of view, risk minimization point of view and constrained variants. [ABSTRACT FROM AUTHOR]

The article introduces the era of modern parsimony in signal processing and statistics. It features the l1-norm and other variants which are applied to geophysics with the Lasso estimator of Tisbshirani, statistics and signal processing with basis pursuit formulation of Chen and colleagues. The era is also said to match the use of convex optimization techniques that solve feature selection or sparse decomposition problems.

The article discusses the use of sparse estimation within the wavelet framework in signal processing which was developed in the 1990s. The monograph focuses on wavelet thresholding to exploit the wave structure and parsimony to improve thresholding estimators and define groups of coefficients. It is also applied in continuous signals and discrete signals.

We consider a class of learning problems regularized by a structured sparsity-inducing norm defined as the sum of l2- or l∞-norms over groups of variables. Whereas much effort has been put in developing fast optimization techniques when the groups are disjoint or embedded in a hierarchy, we address here the case of general overlapping groups. To this end, we present two different strategies: On the one hand, we show that the proximal operator associated with a sum of l∞-norms can be computed exactly in polynomial time by solving a quadratic min-cost flow problem, allowing the use of accelerated proximal gradient methods. On the other hand, we use proximal splitting techniques, and address an equivalent formulation with non-overlapping groups, but in higher dimension and with additional constraints. We propose efficient and scalable algorithms exploiting these two strategies, which are significantly faster than alternative approaches. We illustrate these methods with several problems such as CUR matrix factorization, multi-task learning of tree-structured dictionaries, background subtraction in video sequences, image denoising with wavelets, and topographic dictionary learning of natural image patches. [ABSTRACT FROM AUTHOR]

Sparse coding consists in representing signals as sparse linear combinations of atoms selected from a dictionary. We consider an extension of this framework where the atoms are further assumed to be embedded in a tree. This is achieved using a recently introduced tree-structured sparse regularization norm, which has proven useful in several applications. This norm leads to regularized problems that are difficult to optimize, and in this paper, we propose efficient algorithms for solving them. More precisely, we show that the proximal operator associated with this norm is computable exactly via a dual approach that can be viewed as the composition of elementary proximal operators. Our procedure has a complexity linear, or close to linear, in the number of atoms, and allows the use of accelerated gradient techniques to solve the tree-structured sparse approximation problem at the same computational cost as traditional ones using the ℓ1-norm. Our method is efficient and scales gracefully to millions of variables, which we illustrate in two types of applications: first, we consider fixed hierarchical dictionaries of wavelets to denoise natural images. Then, we apply our optimization tools in the context of dictionary learning, where learned dictionary elements naturally self-organize in a prespecified arborescent structure, leading to better performance in reconstruction of natural image patches. When applied to text documents, our method learns hierarchies of topics, thus providing a competitive alternative to probabilistic topic models. [ABSTRACT FROM AUTHOR]

The article focuses on face recognition in visual recognition. It presents the sparse-representation based classifier (SRC) which sparsely decomposes a set of training signals. Also mentioned are various extensions to solve the problem of face recognition which include occlusion and image misalignment.

The article focuses on the natural extensions of dictionary learning into video processing. It states that each frame should be processed independently by achieving temporal consistency between frames because of a noisy video sequence. It also suggests to adapt the inpainting and color video denoising tasks which include processing three-dimensional patches at the same time.

The article focuses on inverting nonlinear local transformations as one of the sparse models for imaging processing. The method makes an assumption on the relation between the image patches aside from a common sparse representation onto two different dictionaries. It also illustrates the ability of the previous sparse models for learning nonlinear mappings to reconstruct grayscale images from binary images.

The article focuses on image inpainting as one of the sparse models for imaging processing. It states that the method can only hold holes that are smaller than the patch size and unstructured noise. It also suggests that the missing pixels are recovered using the dictionary learning approach wherein the brick texture in the image house cannot be seen by human eyes.

The article discusses the early concepts of parsimony in statistics and information theory in 1970s and 1980s. Topics covered include the approximate computational techniques for the ordinary least squares problem, such as criterions for controlling the trade-off between the sparsity of the solution Ø and the adequacy of the fit to training data and Cp-statisctics, the Akaike information criterion and the Bayesian information criterion. Also mentioned is the forward selection technique.

The article focuses on the monograph for dictionary learning and sparse estimation. Topics discussed include various applications of dictionary learning in computer vision and image processing, as well as sparse estimation tools in signal processing and statistics. Also mentioned are the new application of sparse estimation in neuroscience and omission of dictionary learning in hyperspectral data and to disparity maps for stereo imaging.

The article focuses on various unsupervised learning techniques in discovering the structures of natural images. Topics discussed include matrix factorization and sparse coding. Also mentioned are emerging techniques including the Gaussian mixture model (GMM), restricted Boltzmann machines and convolutional neural networks.

The article presents other optimization techniques for sparse estimation. Topics discussed include proximal splitting techniques, stochastic optimization techniques, and omitted quasi-Newton methods. These optimization techniques solve convex sparse estimation problems, large-scale problems for machine learning, and the curvature of the objective function.

People that the author would like to thank for their assistance in the creation of the monograph are mentioned.

Modeling data with linear combinations of a few elements from a learned dictionary has been the focus of much recent research in machine learning, neuroscience, and signal processing. For signals such as natural images that admit such sparse representations, it is now well established that these models are well suited to restoration tasks. In this context, learning the dictionary amounts to solving a large-scale matrix factorization problem, which can be done efficiently with classical optimization tools. The same approach has also been used for learning features from data for other purposes, e.g., image classification, but tuning the dictionary in a supervised way for these tasks has proven to be more difficult. In this paper, we present a general formulation for supervised dictionary learning adapted to a wide variety of tasks, and present an efficient algorithm for solving the corresponding optimization problem. Experiments on handwritten digit classification, digital art identification, nonlinear inverse image problems, and compressed sensing demonstrate that our approach is effective in large-scale settings, and is well suited to supervised and semi-supervised classification, as well as regression tasks for data that admit sparse representations. [ABSTRACT FROM AUTHOR]