Search

Sparse coding is a matrix factorization technique. It models a target signal as a sparse linear combination of atoms (elementary signals) drawn from a dictionary (a fixed collection). Sparse coding has become a popular paradigm in signal processing, statistics, and machine learning. This chapter introduces compressed sensing, sparse representation/sparse coding, tensor compressed sensing, and sparse PCA.

SVM [12, 201] is one of the most popular nonparametric classification algorithms. It is optimal and is based on computational learning theory [200, 202]. The goal of SVM is to minimize the VC dimension by finding the optimal hyperplane between classes, with the maximal margin, where the margin is defined as the distance of the closest point in each class to the separating hyperplane. It has a general-purpose linear learning algorithm and a problem-specific kernel that computes the inner product of input data points in a feature space. The key idea of SVM is to project the training set in a high-dimensional space into a lower-dimensional feature space by means of a set of nonlinear kernel functions, where the projections of the training examples are always linearly separable in the feature space. The hippocampus, a brain region critical for learning and memory processes, has been reported to possess pattern separation function similar to SVM [6].

Hardware and parallel implementations can substantially speed up machine learning algorithms to extend their widespread applications. In this chapter, we first introduce various circuit realizations for popular neural network learning methods. We then introduce their parallel implementations on graphic processing units (GPUs), systolic arrays of processors, and parallel computers.

Binary sequences with low aperiodic autocorrelation levels, defined in terms of the peak sidelobe level (PSL) and/or merit factor, have many important engineering applications, such as radars, sonars, spread spectrum communications, system identification, and cryptography. Searching for low autocorrelation binary sequences (LABS) is a notorious combinatorial problem, and has been chosen to form a benchmark test for constraint solvers. Due to its prohibitively high complexity, an exhaustive search solution is impractical, except for relatively short lengths. Many suboptimal algorithms have been introduced to extend the LABS search for lengths of up to a few hundred. In this paper, we address the challenge of discovering even longer LABS by proposing an evolutionary algorithm (EA) with a new combination of several features, borrowed from genetic algorithms, evolutionary strategies (ES), and memetic algorithms. The proposed algorithm can efficiently discover long LABS of lengths up to several thousand. Record-breaking minimum peak sidelobe results of many lengths up to 4096 have been tabulated for benchmarking purposes. In addition, our algorithm design can be easily adapted to tackle various extensions of the LABS problem, say, with a generic sidelobe criterion and/or for possibly nonbinary sequences.

Many information processing problems can be transformed into some form of eigenvalue or singular value problems. Eigenvalue decomposition (EVD) and singular value decomposition (SVD) are usually used for solving these problems. In this paper, we give an introduction to various neural network implementations and algorithms for principal component analysis (PCA) and its various extensions. PCA is a statistical method that is directly related to EVD and SVD. Minor component analysis (MCA) is a variant of PCA, which is useful for solving total least squares (TLSs) problems. The algorithms are typical unsupervised learning methods. Some other neural network models for feature extraction, such as localized methods, complex-domain methods, generalized EVD, and SVD, are also described. Topics associated with PCA, such as independent component analysis (ICA) and linear discriminant analysis (LDA), are mentioned in passing in the conclusion. These methods are useful in adaptive signal processing, blind signal separation (BSS), pattern recognition, and information compression.

The radial basis function (RBF) network has its foundation in the conventional approximation theory. It has the capability of universal approximation. The RBF network is a popular alternative to the well-known multilayer perceptron (MLP), since it has a simpler structure and a much faster training process. In this paper, we give a comprehensive survey on the RBF network and its learning. Many aspects associated with the RBF network, such as network structure, universal approimation capability, radial basis functions, RBF network learning, structure optimization, normalized RBF networks, application to dynamic system modeling, and nonlinear complex-valued signal processing, are described. We also compare the features and capability of the two models.

One of the main benefits of the cyclostationary beamforming algorithms is their ability to extract signals from co-channel interference with only a knowledge of the cycle frequency. In this paper, we study the popular cyclostationary beamformers, and propose five new algorithms, namely, the adaptive cyclic adaptive beamforming (ACAB), adaptive cross-SCORE (ACS), constrained least-squares (CLS), adaptive phase-SCORE (APS), and maximal constrained autocorrelation (MCA) algorithms. All these algorithms are adaptive and have a computational complexity of O(n 2) complex multiplications, where n is the number of array elements. A comparative study of these algorithms is made based on numerical simulations. Each of these algorithms has specific application scenarios. The ACS and the APS algorithms are particularly suited for very adverse signal environments. The ACAB, MCA and cyclic adaptive beamforming (CAB, from the work of Wu and Wong) algorithms can provide good performance in the case of medium or weak interference, while the CLS algorithm is especially suitable for weak interference. The CAB algorithm is shown to be a special case of the least-square self-coherent restoral (LS-SCORE) algorithm. Some insights as to how one can assign carrier frequency and symbol rate during digital modulation are also suggested. The proposed adaptive algorithms are easy to implement, and thus are very promising for applications in wireless and mobile communications.

Tabu search is a single-solution-based stochastic metaheuristic global optimization method. It is a hill-climbing method that imitates human memory structure to improve decision-making. Scatter search is a population-based metaheuristic algorithm. Scatter search and its generalized form called path relinking are intimately related to tabu search, and they derive additional advantages by using adaptive memory mechanism.

This chapter introduces dozens of metaheuristic optimization algorithms that are related to physics, natural phenomena, chemistry, biogeography, and mathematics.

Human being is the most intelligent creature on this planet. This chapter introduces various search metaheuristics that are inspired by various behaviors of human creative problem-solving process.

This chapter continues to introduce topics on EAs. Convergence of EAs is first analyzed by using scheme theorem, building-block hypothesis, and then by using finite and infinite population models. Various parallel implementations of EAs are then described in detail. Some other associated topics including coevolution and fitness approximation are finally introduced.

Estimation of distribution algorithm (EDA) is a most successful paradigm of EAs. EDAs are derived by inspirations from evolutionary computation and machine learning. This chapter describes EDAs as well as several classical EDA implementations.

Ants are capable of finding the shortest path between the food and the colony using a pheromone-laying mechanism. ACO is a metaheuristic optimization approach inspired by this foraging behavior of ants. This chapter is dedicated to ACO.

This chapter treats several hard problems associated with metaheuristic optimization, namely, dynamic, multimodal, and constrained optimization problems.

This chapter describes bacterial foraging algorithm inspired by the social foraging behavior of Escherichia coli present in human intestine. Several algorithms inspired by molds, algae, and tumor cells are also introduced.

Immature dendritic cells (iDCs) have been shown to be able to induce peripheral T-cell tolerance through distinct pathways. Here, we investigated the tolerogenic property of recipient iDCs whose maturation was arrested by a dominant negative mutant of inhibitor of nuclear factor kappa-B kinase 2 (dnIKK2) gene. We found that dnIKK2-iDCs presented a typical semi-mature morphology and expressed lower levels of CD80 and CD86, slightly higher MHC-II than untransfected iDCs. The expression of these molecules had no significant change even dnIKK2-iDCs were pulsed by donor antigen. In primary mixed leukocyte reaction (MLR), dnIKK2-iDCs exhibited impaired ability to stimulate allogeneic T-cells, but induced CD4(+)CD25(-) T-cell formation. In co-culture MLR, these CD4(+)CD25(-) T-cells suppressed T-cell alloreaction in an antigen-specific manner. Besides, CD4(+)CD25(-) T-cells inhibited IL-2 and IFN-γ release, whereas promoted IL-10 and TGF-β secretion. These data suggested recipient dnIKK2-iDCs could maintain peripheral tolerance through down-regulating costimulatory molecule expressions and inducing CD4(+)CD25(-) T-cell formation.

The radiation characteristics of a uniform circular array with linear centrally-fed dipole elements, used for smart antenna systems, are analyzed in this paper. The radiation patterns of the array and the gain of the array over a single element are derived based on the thin-wire model. Numerical simulations using the thin-wire model and the method of moments (MoM) have been conducted and compared for a half-wavelength dipole array. Simulations using the MoM demonstrate that the wire diameter of the elements has no significant effect on normalized azimuthal and elevation patterns for a given pattern design. Normalized azimuthal patterns by the thin-wire model and the MoM are in good agreement with each other, while normalized elevation patterns by the thin-wire model are inaccurate since mutual coupling is not considered in the model. Wire diameter affects the radiation resistance and thus determines the amplitude of the radiation field.

We propose an urban traffic state detection method based on support vector machine (SVM) and multilayer perception (MLP). Fusing the SVM and MLP classifiers into a cascade two-tier classifier improves the accuracy of the traffic state classification. A cascade two-tier classifier MLP-SVM, a single SVM classifier and a single MLP classifier are then fused to further improve the final detection accuracy. We also implement a Dempster-Shafer (D-S) theory of evidence based classifier. Finally, fusion strategies at the training and implementation phases are proposed to improve the detection accuracy.

Biometrics are the personal or physical characteristics of a person. These biometric identities are usually used for identification or verification. Biometric recognition systems are increasingly being deployed as a more natural, more secure, and more efficient means than the conventional password-based method for the recognition of people. Many biometric verification systems have been developed for global security.

The brain is a strongly recurrent structure. This massive recurrence suggests a major role of self-feeding dynamics in the processes of perceiving, acting and learning, and in maintaining the organism alive