Search

We summarize the main results of a recent theory—developed by the authors—establishing fundamental lower bounds on the connectivity and memory requirements of deep neural networks as a function of the complexity of the function class to be approximated by the network. These bounds are shown to be achievable. Specifically, all function classes that are optimally approximated by a general class of representation systems—so-called affine systems —can be approximated by deep neural networks with minimal connectivity and memory requirements. Affine systems encompass a wealth of representation systems from applied harmonic analysis such as wavelets, shearlets, ridgelets, α-shearlets, and more generally α-molecules. This result elucidates a remarkable universality property of deep neural networks and shows that they achieve the optimum approximation properties of all affine systems combined. Finally, we present numerical experiments demonstrating that the standard stochastic gradient descent algorithm generates deep neural networks which provide close-to-optimal approximation rates at minimal connectivity. Moreover, stochastic gradient descent is found to actually learn approximations that are sparse in the representation system optimally sparsifying the function class the network is trained on.

Stability analysis of various neural networks have been successfully applied in many fields such as parallel computing and pattern recognition. This paper is concerned with a class of stochastic Markovian jump neural networks. The general mean-square stability of Backward Euler-Maruyama method for stochastic Markovian jump neural networks is discussed. The sufficient conditions to guarantee the general mean-square stability of Backward Euler-Maruyama method are given.

We study the problem of separation of extracellularly recorded spikes by means of an artificial neural network (NN) and its modifications - wavelet neural networks (WNN). Advantages of networks over the standard approaches such as, e.g., the principal component analysis (PCA) are discussed. Application of neural networks seems to be a highly efficient way to improve classification provided by PCA.

To solve the problem that recognition rates of speech recognition systems decrease in the noisy environment presently, a novel wavelet-RBF network model is presented in this paper. The model combines the time-frequency localization characteristic of wavelet and RBF neural network with the best classification capacity and ability of identification. A speech recognition system is mapped through wavelet-RBF network, which helps to overcome the defects of ANN such as the difficulty of rationally determining the network structure and the existence of partial optimal points. The experimental results show that the wavelet-RBF network is better than RBF network in SNRs and recognition rates.

We establish stability results for both competitive neural networks with only neural activity levels and also with different time-scales under parameter perturbations and determine conditions that ensure the existence of exponentially stable equilibria of the perturbed neural system. The perturbed neural system is modeled as nonlinear perturbations to a known nonlinear idealized system and is represented by both a short-term memory subsystem and also by a two time-scale subsystem. Based on the theory of sliding mode control, we can determine for the simple competitive model a reduced-order system and show that if it is asymptotically stable then the full system will also be asymptotically stable. In addition, a region of attraction can be found. For the two time-scales neural systems, we derive a Lyapunov function for the coupled system and a maximal upper bound for the fast time scale associated with the neural activity state.

In this paper computational intelligence, referring here to the synergy of neural networks and genetic algorithms, is deployed in order to determine a near-optimal neural network for the classification of dark formations in oil spills and look-alikes. Optimality is sought in the framework of a multi-objective problem, i.e. the minimization of input features used and, at the same time, the maximization of overall testing classification accuracy. The proposed method consists of two concurrent actions. The first is the identification of the subset of features that results in the highest classification accuracy on the testing data set i.e. feature selection. The second parallel process is the search for the neural network topology, in terms of number of nodes in the hidden layer, which is able to yield optimal results with respect to the selected subset of features. The results show that the proposed method, i.e. concurrently evolving features and neural network topology, yields superior classification accuracy compared to sequential floating forward selection as well as to using all features together. The accuracy matrix is deployed to show the generalization capacity of the discovered neural network topology on the evolved sub-set of features.

Modern computers can be made more friendly and otherwise improved by making them behave more like humans. Perhaps we can learn how to do this from biology in which human brains evolved over a long period of time. Therefore, we first explain a commonly used biological neural network (BNN) model, the Wilson-Cowan neural oscillator, that has cross-coupled excitatory (positive) and inhibitory (negative) neurons. The two types of neurons are used for frequency modulation communication between neurons which provides immunity to electromagnetic interference. We then evolve, for the first time, an artificial neural network (ANN) to perform the same task. Two dynamical feed-forward artificial neural networks use cross-coupling feedback (like that in a flip-flop) to form an ANN nonlinear dynamic neural oscillator with the same equations as the Wilson-Cowan neural oscillator. Finally we show, through simulation, that the equations perform the basic neural threshold function, switching between stable zero output and a stable oscillation, that is a stable limit cycle. Optical implementation with an injected laser diode and future research are discussed.

Neural network based image processing algorithms present numerous advantages due to their supervised adjustable weight and bias coefficients. Among various neural network architectures, dynamic neural networks, Hopfield and Cellular neural networks have been found inherently suitable for filtering applications. These kind of neural networks present two important features; supervised learnable and optimization properties. Using these properties, dynamic neural filtering technique has been developed based on Hopfield neural networks. The filtering structure involves adjustable a filter mask and 2D convolution operation instead of weight matrix operations. To improve the supervised training properties, Widrow-recurrent learning algorithm has been proposed in this paper. Since the proposed learning algorithm requires less computation, consumption time in the training stage has been decreased considerably compared to previous reported supervised techniques for dynamic neural filtering.© (2005) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper examines the forecasting performance of multi-layer feed forward neural networks in modeling a particular foreign exchange rates, i.e. Japanese Yen/US Dollar. The effects of two learning methods, Back Propagation and Genetic Algorithm, in which the neural network topology and other parameters fixed, were investigated. The early results indicate that the application of this hybrid system seems to be well suited for the forecasting of foreign exchange rates. The Neural Networks and Genetic Algorithm were programmed using MATLAB®.

There exist a common belief that random sequences are produced from very complicated phenomena, making impossible the construction of accurate mathematical models. It has been recently shown that under specific conditions the exact solutions to some chaotic functions can be generalized to produce truly random sequences. This establishes a transition from chaos to stochastic dynamics. Using this result we can obtain explicit output expressions for stochastic dynamics problems like those posed by stochastic resonant nonlinear systems. We show that in this kind of systems the phenomenon of noise-induced disorder-order can be more efficiently described with an information-theory approach through the determination of a parameter that measures the complexity of the dynamics. The Stochastic Resonance (SR) is just an example of the principal phenomenon wherein the complex stochastic dynamics is converted into a simpler one. Then we show the opposite phenomenon whereby the autonomous (without input noise) transition from chaotic order to stochastic disorder is achieved by a static non-invertible non-linearity. We build electronic systems to simulate and produce experimentally all these phenomena.

A novel Artificial Neural Network (ANN) is presented, which has been designed for computationally intensive problems, and applied to the optimization of electromagnetic devices such as antennas and microwave devices. The ANN exploits a unique number representation in conjunction with a more standard neural network architecture. An ANN consisting of hetero-associative memory provided a very efficient method of computing the necessary geometrical values for the devices, when used in conjunction with a new randomization process. The number representation used provides significant insight into this new method of fault-tolerant computing. Further work is needed to evaluate the potential of this new paradigm.

This paper focuses on application of artificial neural networks (ANN) in land suitability evaluation. There are some problems in applying fuzzy system to land suitability evaluation such as self-adjusting ability of the membership functions and rules of fuzzy evaluation system. In this paper, the model of fuzzy neural network is designed for land suitability evaluation. This model is the result of integrated fuzzy system and artificial neural network. This fuzzy neural network model has five layers. The learning algorithm of the model has been designed based on the principle of error back propagation of neural networks. The learning strategy, algorithm and efficiency of the model have been tested and the results of test are satisfied.

A novel method of optimizing feed-forward neural networks using cascaded genetic algorithm is proposed in this paper. It adopts a hybrid encoding method, which architectures and connection weights vector of neural networks are encoded into binary code and real-value code respectively. The proposed optimizing method includes two cascaded evolutionary procedures in which the first mainly plays the role of fast search in constrained area and the second extends global exploration ability. The proposed method has represented a particular compromise between exploitation and exploration of searching optimized neural networks and enhanced the global search ability while using less computation. The experimental results have shown its good performance.

The structures of optical neural nets (NN) based on new matrix-tensor equivalental models (MTEMs) and algorithms are described in this article. MTE models are neuroparadigm of non-iterative type, which is a generalization of Hopfield and Hamming networks. The adaptive multi-layer networks, auto-associative and hetero-associative memory of 2-D images of high order can be built on the basis of MTEMs. The capacity of such networks in comparison with capacity of Hopfield networks is increased. The results of modeling show that the number of neurons in neural network MTEMs is 10-20 thousand and more. The problems of trailing of such networks, different modifications, including networks with double adaptive-equivalent autoweighing of weights, organization of computing process in different modes of network are discussed. The basic components of networks: matrix-tensor 'equivalentors' and variants of their realization on the basis of liquid-crystal structures and optical multipliers with spatial and time integration are considered. The efficiency of proposed optical neural networks on the basis of MTEMs is evaluated for both variants on the level of 10 9 connections per second. Modified optical connections are realized as liquid-crystal television screens.

The paper presents a new way to study the results obtained by back-scattering of light in tissue through artificial intelligence. The artificial neural networks' (ANN) ability to extract significant information from an initial set of data allows both an interpolation, in the a priori defined points, and an extrapolation outside of the range bordered by the extreme points from the initial training set. The data obtained from EMPHO Spectrophotometer were used for neural networks learning. Specific aspects related to the training procedure and parameter fitting are presented. The evaluation of the computing effort shows some way for future optimizations.

We discuss a way to implement predictor for stochastic process with stationary increments by using optical neural network.© (2001) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Neural networks, especially in nonlinear system identification and control applications, are typically considered to be black-boxes which are difficult to analyze and understand mathematically. Due to this reason, an in- depth mathematical analysis offering insight into the different neural network transformation layers based on a theoretical transformation scheme is desired, but up to now neither available nor known. In previous works it has been shown how proven engineering methods such as dimensional analysis and the Laplace transform may be used to construct a neural controller topology for time-invariant systems. Using the knowledge of neural correspondences of these two classical methods, the internal nodes of the network could also be successfully interpreted after training. As further extension to these works, the paper describes the latest of a theoretical interpretation framework describing the neural network transformation sequences in nonlinear system identification and control. This can be achieved By incorporation of the method of exact input-output linearization in the above mentioned two transform sequences of dimensional analysis and the Laplace transformation. Based on these three theoretical considerations neural network topologies may be designed in special situations by pure translation in the sense of a structural compilation of the known classical solutions into their correspondent neural topology. Based on known exemplary results, the paper synthesizes the proposed approach into the visionary goals of a structural compiler for neural networks. This structural compiler for neural networks is intended to automatically convert classical control formulations into their equivalent neural network structure based on the principles of equivalence between formula and operator, and operator and structure which are discussed in detail in this work.

This paper presents a new method to identify the vibration model for large smart structures. A large structure typically has low frequency vibrations, unique distributed dynamic characteristics, and no simple control model. Study of composite aircraft fuselage vibration suppression required the identification of a distributed parameter control method. Artificial neural networks were used to identify structural vibration characteristics in distinct neighborhoods. Neural networks were trained and compared using a plate model.

Image compression is an important area of multimedia investigation and neural network methods have attracted more and more attentions for using in image coding. Recently a random neural network model, which has the solutions with product form in steady state (i.e. the steady state probability distribution of network can always be expressed as the product of the probabilities of the states of each neuron) on some conditions, was brought forward. Among the diverse random neural network models, the feed-forward one is very practicable because its solutions exist and are unique. In this paper, a new learning method for feed-forward random neural network, which can be implemented easier than the learning algorithm of the RNN presented by Gelenbe, was presented. Using the new learning formulas we developed, we designed a new image coding method, which applies the random neural network method in classical DCT- based coding framework. The experimental results show that our new method could gain a lot in PSNR (1 approximately 2dB) compared with standard neural network coding methods. In conclusion, we stated that the DCT-based image compression method using random neural network is an efficient algorithm for image coding.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper, we analyze decision boundaries of radial basis function (RBF) neural networks when the RBF neural networks are used as a classifier. We divide the working mechanism of the neural network into two parts: dimension expansion by hidden neurons and linear decision boundary formation by output neurons. First, we investigate the dimension expansion from the input space to the hidden neuron space and then address several properties of decision boundaries in the hidden neuron space that is defined by the outputs of the hidden neurons. Finally, we present a thorough analysis how the number of hidden neurons influences decision boundaries in the input space with illustrations, providing a helpful insight into how RBF networks define complex decision boundaries.

For nonlinear and adaptive control of smart structures direct and indirect neural network control strategies have been suggested. In indirect neural network control the identified plant models are usually implemented as black-box neural networks using no a priori knowledge. Designing a neural network for system identification using dimensional analysis results in neural networks, where in contrary to black-box solutions no dimensionally inhomogeneous states can occur. Furthermore, the generalization and learning properties of neural networks designed using dimensional analysis are usually improved compared to conventional black-box networks. This work describes a technique of using neural networks for system identification and control, where the neural network has been constructed according to a dimensional analysis of the governing equations.

The Forward Generating Neural Network is a special network for solving mapping problems. Its background,characteristics and training are introduced. The main advantages of this neural network are its rapid training and theaccuracy of its output. Keywords: Neural Network, Forward Generating, Training 1. INTRODUCTION Neural network training has attracted the efforts of many researchers and there are many methods for solving theproblem. Backpropagation and counterpropagation are two important methods. However, in many cases, training still takes too much computer time and this limits the application of neural networks. Thus many researchers have continued their efforts to devise more efficient training methods and network structure. In this paper, we propose a method to solve the training problem with a network called Forward Generating Neural Network, PGN Net for short. 2. FORWARD GENERATING NEURAL NETWORK (PGN NET) The architecture of PGN Net is shown in Fig. I . This is a network with two layers. In Fig. I WF, denotes the

Applications of neural networks have been reported on a lot in recent years, but the research on how to find reliable guidelines to design neural networks is still in its infancy. This work intends to provide some ideas on how to find useful predefined network structures for at least certain parts of the neural net. By breaking off to a certain extend the so-called black-box character of the neural net, the performance of the networks can be improved and the solutions of the nets get more transparent and understandable at the same time. Additionally, the ability of the neural nets to generalize from some training patterns to unlearned data regions is improved substantially. In this work two commonly used engineering principles such as the technique of dimensional analysis and the Laplace- transformation are used to identify suitable topologies for neural networks. The integration of the dimensional analysis in the context of feed-forward neural networks is presented. In the second part of this work the use of the Laplace- transformation in neural networks is demonstrated. Even though at the moment the application of this technique has been shown in a linear time-invariant process, a future use of this method in nonlinear system is considered.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this article we developed a method for optimizing the structure of a fuzzy artificial neural networks through genetic algorithms. This genetic algorithm is used by optimizing the number of weight connections in a neural network structure, by the evolution of those structures as individuals in a population. It is found that the optimization of the neural network provides higher confidence accuracy of the suggested solution in a case based diagnostic system. The computational cost of the optimized network also improved considerably high.

A new model of feed-back neural network is presented in this paper and a simulative research work has been done on its dynamics behavior. The result show that equilibrium point and limit cycle may exist simultaneously when three neurons are in connection, additional chaotic phenomenon can occur with four neurons being in connection and the network characteristic has a close relationship with self-feedback intensity.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In the research, we introduced an artificial neural network model named as coupled lattice neural network to reconstruct an original image from a degrade one in the blind deconvolution, where the original image and blurring function are not known. In the coupled lattice neural network, each neuron connects with its nearest neighbor neurons, the neighborhood corresponds to the weights of the neural network and is defined by a finite domain. Outputs of neurons shows the intensity distribution of an estimated original image. Weights of each neuron correspond to an estimated blur function and are the same for different neurons. The coupled lattice neural network includes two main operations, one is a nearest neighbor coupling or diffusion, the other is a local nonlinear reflection and learning. First a rule for a blur function growing is introduced. Then the coupled lattice neural network implements an estimated original image evolving based on an estimated blur function. Moreover we define a growing error criterion to control the evolution of the coupled lattice neural network. Whenever the error criterion is minimized, the coupled neural network gets stable, then outputs of the neural network correspond to the reconstructed original image, the weights are the blur function. In addition we demonstrate a method for the option of initial state variables of the coupled lattice neural network. The new approach to blind deconvolution can recover a digital binary image successful. Moreover the coupled lattice neural network can be used in the reconstruction of a gray-scale image.

There are may phase unwrapping algorithms have been developed in the recent years. Some of them work well in no or less noisy situations. But when encountering noisy data which is the most occurring situation these algorithms may exhibit insufficiency and diversity and usually cause the result unusable. Artificial neural networks have some features in tackling these problems. Especially random artificial neural network is more powerful and robust in solving combinatorial optimization problems. A random artificial neural network model has been used in demodulation of 'wrapped phase' in one and two dimensions of noisy interference pattern.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Images are statistical in nature due to random changes and noise,therefore it is sometimes an advantage to treat image functions as a realization of a stochastic process. An advantage of stochastic random field models is that they need only a few parameters to describe a region or texture. In this paper textural images are modeled as a realization of Markov Random Fields such as binomial and autoregressive Markov random fields. Parameters of each model are estimated and considered as features of the textural images. The extracted features are incorporated in either a probabilistic neural network (PNN) or a deterministic back propagation neural networks for the purpose of classification and differentiation between various textural images. The PNN and the learning algorithm are discussed in this paper in details. To train back propagation neural network, a hybrid training algorithm is proposed. This hybrid algorithm takes advantage of both simulated annealing and deterministic learning algorithms. The former algorithm is more reliable since it locates a more likely global minimum, but it is slow. The latter algorithm is fast but less reliable, and it can converge to a local minimum. There are many practical uses for the above proposed textural analysis tool such as remote sensing, mineralogical analysis, and medical image processing. In this paper, successful applications of the present stochastic model to synthetic texture images and MRI tongue and brain images are described.

In this paper, we describe a stochastic algorithm which can be used for evaluation of energy function of Hopfield neural networks designed for solving optimization problems. To demonstrate this capability we used the Hopfield neural network described by T. P. Troudet to optimize the operation of a crossbar packet switch. The evaluation of energy function of a Hopfield neural network requires determination of synaptic weights from the energy function which represents the optimization problem under consideration. This process involves several iterations of adjusting the coefficients of the energy function and calculating the synaptic weights from the coefficients of the energy function. The proposed stochastic algorithm allows the designer of a Hopfield network to directly evaluate the energy function and adjust the coefficients without going through the step to determine the synaptic weights. In this paper, we also present the simulation results for evaluation of the energy function described by T. P. Troudet using the stochastic algorithm.

fragments, such as line interseclions, line ends, fractures, etc. The approach proposed allows us todevelop simple neural nets with virtually any form of receplive fields The processing method issuggested in which only those simple features survive that have specific relalive posilions.Keywords: neural network, fealure selection, oplical fihiralion.1.INTRODUcTIONIn many praclical cases it is necessary to select some simple features from input contour image,such as ends, fractures, intersections, orientalions of lines, etc. The paper offers a neural networkalgorithm for solving such problems and its optoelecironic implementanon. When dealing with thealgorithms, all optical connections are supposed to have positive weights.2 .NEURAL

One of the most important properties of neural networks is generality, as the same network can be trained to solve rather different tasks, depending on the training data. This is also one of the most prominent problems when practical real world problems are solved by neural networks, as existing domain knowledge is difficult to incorporate into the models. In this contribution we present methods for adding prior knowledge to neural network modeling. The approach is based on training the knowledge on the network of hard-coding the knowledge in advance to the connections or weights. The knowledge is specified as target values or constraints for different order partial derivatives of the network. This approach can be viewed as a flexible regularization method that controls directly the characteristics of the resulting mapping. The proposed algorithms have been implemented in a neural network modeling tool that supports modular network design and domain knowledge representation with fuzzy-like terms. In this paper we present examples of the effect of incorporating different degrees of information from the modular structure and the functional behavior of the target processes in the model building and training.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A new approach to the identification of dynamical systems by means of evolved neural networks is presented. We implement two functional neural networks: polynomials and orthogonal basis functions. The functional neural networks contain four parameters that need to be optimized: the weights, training parameters, network topology and scaling factors. An approach to the solution of this combinatorial problem is to genetically evolve functional neural networks. This paper presents a preliminary analysis of the proposed method to automatically select network parameters. The networks are encoded as chromosomes that are evolved during the identification by means of genetic algorithms. Experimental results show that the method is effective for the identification of dynamical systems.

Analogue implementation of Artificial Neural Networks (ANN) especially as CMOS integrated circuits show several attractive features. During the last decade, numerous works show that small size analogue ANN operate correctly. However, today the efforts are focused on real industrial size application of ANN that will require large networks. On the other hand, all of the presented implementations of ANN have been supposed to be working in ideal conditions but real applications will be subject to some global perturbations. Especially in the case of the analogue and mixed digital/analogue implementation, the behavior analysis of the neural network with perturbation conditions is thus inevitable. Unfortunately, very few papers analyze the behavior of analogue neural network with global perturbations. We have investigated modeling and experimental validation of the behavior of analogue ANN in the case of a global perturbation of the network. We have analyzed the behavior of a CMOS analogue implementation of synchronous Boltzmann Machine model when the neural circuit is subject to perturbations. The perturbations we have considered concern the supply voltage of the neural circuit and ambient temperature in which the circuit operates. In this paper we present the analysis of the behavior of the analogue implementation of synchronous Boltzmann Machine with electrical and thermal perturbations. Simulation and experimental results have been exposed.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Neural network training requires a large quantity of samples and consumes a great deal of computing time. Despite of this, we still do not know the generalization capacity of a trained neural network in a certain domain. In thispaper, we propose an algorithm for training neural networks to approximate polynomials. This algorithm can workwith relatively small sample set and predict the generalization capacity of the learned neural network. Simulationresults demonstrate the property of this algorithm.KEYWORD S: neural network, learning algorithm, generalization capacity 1. INTRODUCTION Neural networks are often used to approximate nonlinear functions. From a theoretical point of view, neural network approx?rnation methods have less theoretical support in comparison with regressive analysis or polynomial approximation. However, with their simple structures and potential parallel processing mechanism, neural networksare widely used in many complicated problems. Therefore, the approximation or generalization capacity of a neural

Current interests include the use of adaptive optics systems to image celestial and earth orbiting objects from the ground. Methods, such as the minimum variance reconstructor, have been developed to improve the imaging performance of adaptive optics systems. However, one tool available which has not been fully investigated is the artificial neural network. Neural networks provide nonlinear solutions to adaptive optics problems while offering the possibility to adapt to changing seeing conditions. In this paper we address the use of neural networks for three tasks: (1) to reduce the wave front sensor (WFS) noise variance, (2) to estimate the Fried coherence length, ro, and (3) to estimate the variance of the WFS noise. All of these tasks are accomplished using only the noisy WFS measurements as input. Where appropriate, we compare to classical statistics based methods to determine if neural networks offer true benefits in performance. We find that neural networks perform well in all three tasks. While a statistics based method is found to perform better than a neural network in reducing WFS noise variance, neural networks perform better than the statistics based methods in estimating ro and the variance of the WFS noise.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A VLSI implementable, massively parallel, digital, stochastic neural network architecture with on-chip learning is described which can be used to address video/image compression applications. Color information in images is usually encoded as a means to reduce the data needed to transmit high resolution video images over a lower bandwidth communication system. A neural net approach can be used for such a compression scheme since it can be trained to map a set of patterns from a k-dimensional space to a 1D space very easily. The training algorithm is implemented as a cross correlation between previously calculated weights and global errors. These simple calculations are performed by each processing unit with information available in their local memories, thus, no transfer of information between neurons is necessary. This feature allows for the synchronous updating of all the weights, which optimizes the inherent parallel nature of the neural network architecture, making the design an excellent candidate for VLSI implementation as an SIMD architecture. By incorporating the learning portion on-chip, an appreciable reduction in computing time is possible. The stochastic nature of the learning algorithm, and a simulated 'annealing' process, allows it to convergence to a global minimum. The architecture is synthesized from an HDL description using powerful design and analysis tools which allow for many performance trade- offs and fault coverage to be done by the compiler before the final design is sent for fabrication.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The possibilities of artificial neural networks in computer-aided evaluation of holographic interference patterns are shown at three examples: demodulation of the interference phase modulo 2(pi) in one and two dimensions by a time-discrete recurrent Hopfield network, fringe tracking by a self organizing feature map network of the Kohonen-type, and the automatic detection of partial patterns due to material defects by a multilayer network trained by backpropagation.

Neural networks are applied to the Rayleigh discrimination task. Network performance is compared to results obtained previously using human viewers, and to the best machine approximation to the ideal observer found in an earlier investigation. We find that simple preprocessing of the input image, in this case by projection, greatly improves network convergence and only results obtained on projections are presented here. It is shown that back propagation neural networks significantly outperform a standard nonadaptive linear machine also operating on the projections. In addition, this back propagation neural network performs competitively to a nonadaptive machine that uses the complete two-dimensional information, even though some relevant information is destroyed in the projection process. Finally, improved performance on this Rayleigh task is found for nonlinear (over linear) neural network decision strategies.

This paper presents a new recursive neural network to solve optimization problems. It is made of binary neutrons with feedbacks. From a random initial state, the dynamics alternate successively several pulsation and constraint satisfaction phases. Applying the previous neural network has the following advantages: (1) There is no need to precisely adjust some parameters in the motion equations to obtain good feasible solutions. (2) During each constraint satisfaction phase, the network converges to a feasible solution. (3) The convergence time in the constraint satisfaction phases is very fast: only a few updates of each neuron are necessary. (4) The end user can limit the global response time of the network which regularly provides feasible solutions. This paper describes such a neural network to solve a complex real time resource allocation problem and compare the performances to a simulated annealing algorithm. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper we present a neural generator method that uses a neural network to generate initial search points for a discrete heuristic. We demonstrate the method for the subset-sum problem (SSP), and consider the SSP to be typical of the sub-problems that a scheduling algorithm must solve while on route to solving an entire scheduling problem. The neural generator method hinges on using the continuous valued activations of the neural system to select a corner of the n-cube that can be used to initialize a discrete search. This can be done at each neural iteration, resulting in many discrete searches over the source of a single neural run. Without the discrete heuristic, the selected corners can be interpreted as instantaneous neural solutions and the best-so-far tabulated as the neural system runs. This allows the neural system to be terminated without losing the full effort of the run, and should the network be run until convergence, the best-so-far result is at least as good as the convergent corner, and usually better. With the discrete heuristic, a search is launched from the instantaneous neural solutions, greatly improving the overall results (again keeping the best-so-far). The results are presented for an n equals 25 SSP. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Dynamic learning of random stimuli can be described as a random walk among the stable synaptic values. It is shown that prototype extraction can take place in material attractor neural networks when the stimuli are correlated and hierarchically organized. The network learns a set of attractors representing the prototypes in a completely unsupervised fashion and is able to modify its attractors when the input statistics change. Learning and forgetting rates are computed. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We describe and discuss an electronic implementation of an attractor neural network with plastic synapses. The synaptic dynamics are unsupervised and autonomous, in that they are driven exclusively and perpetually by neural activities. The latter follow the network activity via the developing synapses and the influence of external stimuli. Such a network self- organizes and is a device which converts the gross statistical characteristics of the stimulus input stream into a set of attractors (reverberations). To maintain for a long time the acquired memory the analog synaptic efficacies are discretized by a stochastic refresh mechanism. The discretized synaptic memory has indefinitely long life time in the absence of activity in the network. It is modified only by the arrival of new stimuli. The stochastic refresh mechanism produces transitions at low probability which ensures that transient stimuli do not create significant modifications and that the system has large palimpsestic memory. The electronic implementation is completely analog, stochastic and asynchronous. The circuitry of the first prototype is discussed in some detail as well as the tests performed on it. In carrying out the implementation we have been guided by biological considerations and by electronic constraints. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The higher order neural network used for pattern recognition and optimization is studied in this paper and the results in two different aspects have been obtained. (1) Theoretically, the capacity formula of the Hopfield neural network with the second order weights has been obtained. Compared with the first order network, the capacity of the second order network is about three times greater than that of the first order one and the cost to reach such an efficiency is to add higher order weights. The simulated experiments according to the theory by digital computer are satisfied. (2) Theoretically, the method of how to solve the optimization problems, whose energy functions are more general than Lyapunov function, has been put forward at the end of this paper. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

similar to Successive Over Relaxation (SOR). The network is trained to find the appropriate relaxation parameter. A derivation of the algorithm and its relation to the SOR algorithm is given. The performance of the standard SOR and Jacobi methods are compared with the neural network for two sample problems. 1. INTRODUCTION A simple neural network with no hidden units and no nonlinearities (i.e. the identity function for its activation function) performs a vector-matrix multiplication. For the problem of solving a linear system, Ax=b (1)Iterative solutions of this system can be visualized as recurrent networks consisting of two levels. Theiterative algorithm). The second level corresponds to the calculation of the vector-matrix product which equals the given right hand side, b, when the solution process has converged. The standard Jacobi method can be expressed in terms of a neural network architecture easily. However, without the use ofany adjustable parameters, neural networks provide a parallel solution technique.The most common iterative solution technique for linear systems that utilizes a parameter whichmake a neural network architecture for this algorithm quite cumbersome. We have, therefore, developed

The problem of the design of unerring neural networks with block structure is considered. The bottleneck is building faultless blocks without spurious states. All the commonly used rules of recurrent neural network learning don't exclude the possibility of spurious state apparition in the stable state neighborhoods. In recognition problems these spurious states cause false solutions deteriorating over the network and destroying its robustness. The problem of finding the number and localizing the stable states in neural networks with polynomial capacity without oscillations is considered. A method is proposed to detect spurious states that are close to the stable states of a given network. The method is based on random sampling. It is fast and almost always finds dangerous spurious states. The algorithm is of Las Vegas type and can find the centers and approximate sizes of all the basins of attraction that are great enough. The evaluation of the test time and the probability of the test correctness are given. The number of operations is polynomial unless the capacity is exponential. Some approaches to spurious state suppression are proposed. The results of experiments with some neural network models are briefly described.

In this paper we present a hybrid neural method that uses a neural network to generate initial search points for a discrete heuristic. We demonstrate the method for the subset-sum problem (SSP). The method hinges on using the continuous valued activations of the neural system to select a corner of the n-cube that can be used to initialize a discrete search. This can be done at each neural iteration, resulting in many discrete searches over the coarse of a single neural run. Without the discrete heuristic, the selected corners can be interpreted as instantaneous neural solutions and the best-so-far tabulated as the neural system runs. This allows the neural system to be terminated without losing the full effort of the run, and should the network be run until convergence, the best-so-far result is at least as good as the convergent corner, and usually better. With the discrete heuristic, a search is launched from the instantaneous neural solutions, greatly improving the overall results (again keeping the best-so-far). The results are presented for an n equals 25 SSP, with comparisons to simulated annealing and genetic approaches.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper, a bipolar neural network of weighted neurons' connection Tj, was presented, based on the systemenergy evolution in neural networks dynamic course. The weighted connection matrix consists of a monotoneincreasing even function 1(x), which makes every stored patterns as the stable attractors of network system.KEYWORDS: Neural Network, Associative Memory, Storage Capacity 1. INTRODUCTION Since the system energy function(or Lyapunov function) was introduced into the field of neural network by Hopfield'] , a new upsurge has been arisen in the research of neural network's dynamics system. The energy functionwas successful in determining the storage stability of the electronics circuit of neural networks2' and optimizing the TSP problem31 , which has inspired many people to study the properties of neural network and to set up several ide-alized neural network models46 . In this paper, the viewpoint of system energy evolution has been used to constructa weighted bipolar neural network with high capacity of stable storage.

The implementation of artificial neural networks (ANN) as CMOS analog integrated circuits shows several attractive features. Stochastic models, especially the Boltzmann Machine, show a number of many attractive features. Recent studies on artificial models point out that classification is their most successful application field, and that real pattern recognition tasks, and especially image processing by artificial neural networks will require large networks. All of the presented implementations of ANN are supposed to be working in ideal conditions but real applications are subject to perturbations. For a digital implementation of ANN perturbation effects could be neglected in a firth order approximation. But for the analog and mixed digital/analog implementation cases, the behavior analysis of the neural network with perturbation conditions is inevitable. Unfortunately, very few papers analyze the behavior of analog neural networks with perturbation or their limitations. In this paper we present the analysis of a Boltzmann Machine model's behavior with physical temperature perturbation. The relation between the T parameter of the Boltzmann Machine model and the physical temperature of circuit has been established. Simulation results are presented and temperature effects compensation is discussed.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper we investigate the application of an artificial neural network to the computation of the instantaneous observer's heading in a static environment. This parameter has an important role in vision-based collision avoidance systems and relies upon accurately locating the focus of expansion (FOE) associated with the radial optical flow pattern arising as a consequence of the translational component of motion. The approach proposed in this paper is based on a feed-forward neural network able to compute the image coordinates of the FOE. It is assumed that the input signals are supplied to the network by a sensorial module computing the optical flow associated with a sequence of time-varying images of the viewed scene. A number of experiments have been performed both for theoretical and for realistic optical flow fields. In this latter real-world experiments the input sensorial module has been simulated through an Hopfield network. Experimental results show that the proposed neural architecture is able to recover the FOE position of testing flow fields with a mean error of 0.1 pixels for exact theoretical motion fields. Moreover it seems resistant to noise and its performances appear appreciable also in real world contexts.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.