Search

Recently, removing rain streaks from a single image has attracted a lot of attention because rain streaks can severely degrade the perceptual quality of the image and cause many practical vision systems to fail. Single image deraining can be served as a pre-processing step to improve the performance of high-level vision tasks such as object detection and video surveillance. In this paper, we propose recurrent scale-guide networks for single image deraining. Although the multi-scale strategy has been successfully applied to many computer vision problems, the correlation between different scales has not been explored in most existing methods. To overcome this deficiency, we propose two types of scale-guide blocks and develop two combinations between the blocks. One type of scale-guide block is that small scale guides the large, and the other is that large scale guides the small. Moreover, we extend the single-stage deraining model to the multi-stage recurrent framework and introduce the Long Short-Term Memory (LSTM) to link every stage. Extensive experiments verify that the scale-guide manner boosts the deraining performance and the recurrent style improves the deraining results. Experimental results demonstrate that the proposed method outperforms other state-of-the-art deraining methods on three widely used datasets: Rain100H, Rain100L, and Rain1200. The source codes can be found at https://supercong94.wixsite.com/supercong94. [ABSTRACT FROM AUTHOR]

• Network models were used to study the possible functions of nonspecific top-down inputs. • Nonspecific top-down signals can suppress undesired bottom-up connections. • Synchronous oscillatory activity is essential for learning of bottom-up connections. • Inputs to excitatory and inhibitory neurons have distinct roles. In the brain, high-order and low-order areas are connected via bottom-up connections (from low-order to high-order areas) and top-down connections (from high-order to low-order areas). While bottom-up signals are thought to be critical in generating perception, functions of top-down signals have not been clearly delineated. One popular theory is that top-down inputs modify the activity of specific cell assemblies to modulate responses to bottom-up inputs. However, a different line of studies proposes that not all top-down inputs are specifically delivered. As the leading theories cannot account for nonspecific top-down inputs, we seek potential functions of nonspecific top-down signals using network models in our study. Our simulation results suggest that top-down inputs can regulate low-order area responses by providing temporal information even without spatial specificity. Specifically, the temporal information in nonspecific top-down inputs can weaken the undesired bottom-up connections, contributing to bottom-up connections' learning. Further, we found that cortical rhythms (synchronous oscillatory neural responses) are critical in the proposed learning process of bottom-up connections in our model. [ABSTRACT FROM AUTHOR]

In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and machine translation. However this success in modelling short term dependencies has not successfully transitioned to application areas such as trajectory prediction, which require capturing both short term and long term relationships. In this paper, we propose a Tree Memory Network (TMN) for jointly modelling both long term relationships between multiple sequences and short term relationships within a sequence, in sequence-to-sequence mapping problems. The proposed network architecture is composed of an input module, controller and a memory module. In contrast to related literature which models the memory as a sequence of historical states, we model the memory as a recursive tree structure. This structure more effectively captures temporal dependencies across both short and long term time periods through its hierarchical structure. We demonstrate the effectiveness and flexibility of the proposed TMN in two practical problems: aircraft trajectory modelling and pedestrian trajectory modelling in a surveillance setting. In both cases the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in depth analysis on the evolution of the memory module content over time and provide visual evidence on how the proposed TMN is able to map both short and long term relationships efficiently via a hierarchical structure. [ABSTRACT FROM AUTHOR]

Abstract: A fast learning neuroevolutionary algorithm for both feedforward and recurrent networks is proposed. The method is inspired by the well known and highly effective Cartesian genetic programming (CGP) technique. The proposed method is called the CGP-based Artificial Neural Network (CGPANN). The basic idea is to replace each computational node in CGP with an artificial neuron, thus producing an artificial neural network. The capabilities of CGPANN are tested in two diverse problem domains. Firstly, it has been tested on a standard benchmark control problem: single and double pole for both Markovian and non-Markovian cases. Results demonstrate that the method can generate effective neural architectures in substantially fewer evaluations in comparison to previously published neuroevolutionary techniques. In addition, the evolved networks show improved generalization and robustness in comparison with other techniques. Secondly, we have explored the capabilities of CGPANNs for the diagnosis of Breast Cancer from the FNA (Finite Needle Aspiration) data samples. The results demonstrate that the proposed algorithm gives 99.5% accurate results, thus making it an excellent choice for pattern recognitions in medical diagnosis, owing to its properties of fast learning and accuracy. The power of a CGP based ANN is its representation which leads to an efficient evolutionary search of suitable topologies. This opens new avenues for applying the proposed technique to other linear/non-linear and Markovian/non-Markovian control and pattern recognition problems. [Copyright &y& Elsevier]

We present a simple autoassociative neural network model of cortical adaptation that replicates a high-level category aftereffect. We study in our model the effect of a priming stimulus on retrieval from a briefly presented partial cue. We also explore, in our general framework, which learning conditions permit correct pattern retrieval. [Copyright &y& Elsevier]

Abstract: There is growing evidence that neural circuits may employ statistical algorithms for inference and learning. Many such algorithms can be derived from independence diagrams (graphical models) showing causal relationships between random variables. A general algorithm for inference in graphical models is belief propagation, where nodes in a graphical model determine values for random variables by combining observed values with messages passed between neighboring nodes. We propose that small groups of synaptic connections between neurons in cortex correspond to causal dependencies in an underlying graphical model. Our results suggest a new probabilistic framework for computation in the neocortex. [Copyright &y& Elsevier]

Abstract: It has been hypothesized that increasing CS longevity affects performance on trace conditioning. Using a hippocampal model, we find that increasing CS and US longevity increases the learnable trace interval. Our simulations show that, over a modest range, the maximal learnable trace interval is approximately a linear function of CS/US longevity. [Copyright &y& Elsevier]

We present an analysis of the range of values of synaptic connections to enhance the storage properties of neural networks. Random patterns are shown to a random connected system, imposing specific activities over neuron pairs that might be connected evoking long-term synapse modifications. Two approaches are given to quantify quality retrieval. The first focuses on the noise context and addresses a simple instrument to directly quantify the retrieval, while the second one focuses on the learning rule considering the information content of the network. We find that increasing the variability among the synapse values within a given range is followed by an increase of the retreival quality. [Copyright &y& Elsevier]

A bistable system exhibits two distinct response levels, so it may act as a switch. Bistability is a common feature of many dynamical systems, including single and interconnected neurons. In general, it provides a simple model mechanism for the gating of neural activity, which may be ubiquitously important for the operation of real nervous systems. Here we explore the range of conditions under which bistability is possible in recurrent neural networks. In particular, we explore whether oscillations affect the capacity of a circuit to produce bistable behavior. To this end, we simulate and compare three types of networks: one with homogeneous parameters (so individual units are identical), another with heterogeneous parameters, and a third that is forced to produce oscillatory activity. We use an evolutionary strategy to generate networks of each type that are bistable in the presence of noise. We find that all evolved networks reach a nearly equal capacity to sustain bistability in the presence of high noise, suggesting that such capacity is not greatly limited by other constraints that may be placed upon a network. [Copyright &y& Elsevier]

Self-organization constitutes an important paradigm in machine learning with successful applications e.g. in data- and web-mining. Most approaches, however, have been proposed for processing data contained in a fixed and finite dimensional vector space. In this article, we will focus on extensions to more general data structures like sequences and tree structures. Various modifications of the standard self-organizing map (SOM) to sequences or tree structures have been proposed in the literature some of which are the temporal Kohonen map, the recursive SOM, and SOM for structured data. These methods enhance the standard SOM by utilizing recursive connections. We define a general recursive dynamic in this article which provides recursive processing of complex data structures by recursive computation of internal representations for the given context. The above mentioned mechanisms of SOMs for structures are special cases of the proposed general dynamic. Furthermore, the dynamic covers the supervised case of recurrent and recursive networks. The general framework offers an uniform notation for training mechanisms such as Hebbian learning. Moreover, the transfer of computational alternatives such as vector quantization or the neural gas algorithm to structure processing networks can be easily achieved. One can formulate general cost functions corresponding to vector quantization, neural gas, and a modification of SOM. The cost functions can be compared to Hebbian learning which can be interpreted as an approximation of a stochastic gradient descent. For comparison, we derive the exact gradients for general cost functions. [Copyright &y& Elsevier]

This work proposes a new recurrent polynomial neural network that utilises both the temporal dynamics of the image formation process and the multi-linear interactions between the pixels for 1D/2D predictive image coding. The network consists of a layer of summing units followed by a product unit and incorporates a feedback link from the output to the input layer. It is trained using a small size training set through dynamic backpropagation. Its performance is evaluated on a database of 15 images and compared to the higher-order neural network, the feedforward pi-sigma neural network, and the standard linear predictor. [Copyright &y& Elsevier]

In the last few years several kinds of recurrent neural networks (RNNs) have been proposed for solving linear and nonlinear optimization problems. In this paper, we provide a survey of RNNs that can be used to solve both the constrained quadratic optimization problem related to support vector machine (SVM) learning, and the SVM model selection by automatic hyperparameter tuning. The appeal of this approach is the possibility of implementing such networks on analog VLSI systems with relative easiness. We review several proposals appeared so far in the literature and test their behavior when applied to solve a telecommunication application, where a special purpose adaptive hardware is of great interest. [Copyright &y& Elsevier]

Recurrent neural networks possess interesting universal approximation capabilities, making them good candidates for time-series modeling. Unfortunately, long-term dependencies are difficult to learn if gradient descent algorithms are employed. We support the view that it is easier for these algorithms to find good solutions if time-delayed connections are included in the recurrent networks. The algorithm we present here allows one to choose the right locations and delays for such connections. As we show on several univariate benchmarks and one multivariate real-world problem, this algorithm produces very good results while keeping the total number of connections in the recurrent network to a minimum. [Copyright &y& Elsevier]

The hippocampus is needed to store memories that are reconfigurable. We have previously shown that a hippocampal-like computational model solves the transverse patterning (TP) problem and transitive inference problems. Here, we show that the same model with the same parameters which solve the TP problem and transitive inference problems reproduces another interesting problem—transverse non-patterning (TNP) problem investigated by Alvarado and Rudy (J. Exp. Psychol.: Anim. Behav. Process 18 (1992) 145). By turning TNP into a problem of sequence learning (stimuli–decision–outcome), a sequence learning, hippocampal-like neural network finds that the TNP problem is unlearnable with the progressive learning paradigm. This unlearnability is what Alvarado and Rudy observed. Thus, both rats and the model learn TP, but fail to learn TNP. [Copyright &y& Elsevier]