Search

Primary visual cortical complex cells are thought to serve as invariant feature detectors and to provide input to higher cortical areas. We propose a single model for learning the connectivity required by complex cells that integrates two factors that have been hypothesized to play a role in the development of invariant feature detectors: temporal slowness and sparsity. This model, the generative adaptive subspace self-organizing map (GASSOM), extends Kohonen's adaptive subspace self-organizing map (ASSOM) with a generative model of the input. Each observation is assumed to be generated by one among many nodes in the network, each being associated with a different subspace in the space of all observations. The generating nodes evolve according to a first-order Markov chain and generate inputs that lie close to the associated subspace. This model differs from prior approaches in that temporal slowness is not an externally imposed criterion to be maximized during learning but, rather, an emergent property of the model structure as it seeks a good model of the input statistics. Unlike theASSOM,theGASSOMdoes not require an explicit segmentation of the input training vectors into separate episodes. This enables us to apply this model to an unlabeled naturalistic image sequence generated by a realistic eyemovement model.We show that the emergence of temporal slowness within the model improves the invariance of feature detectors trained on this input.

The Adaptive Subspace Self Organized Map (ASSOM) is a model that incorporates sparsity, nonlinear pooling, topological organization and temporal continuity to learn invariant feature detectors, each corresponding to one node of the network. Temporal continuity is implemented by grouping inputs into training episodes. Each episode contains samples from one invariance class and is mapped to a particular node during training. However, this explicit grouping makes application of this algorithm for natural image sequences difficult, since the grouping is generally not known a priori. This work proposes a probabilistic generative model of the ASSOM that addresses this problem. Each node of the ASSOM generates input vectors from one invariance class. Training sequences are generated by nodes that are chosen according to a Markov process. We demonstrate that this model can learn invariant feature detectors similar to those found in the primary visual cortex from an unlabeled sequence of input images generated by a realistic model of eye movements. Performance is comparable to the original ASSOM algorithm, but without the need for explicit grouping into training episodes.

The Adaptive Subspace Self Organized Map (ASSOM) is a model that incorporates sparsity, nonlinear pooling, topological organization and temporal continuity to learn invariant feature detectors, each corresponding to one node of the network. Temporal continuity is implemented by grouping inputs into training episodes. Each episode contains samples from one invariance class and is mapped to a particular node during training. However, this explicit grouping makes application of this algorithm for natural image sequences difficult, since the grouping is generally not known a priori. This work proposes a probabilistic generative model of the ASSOM that addresses this problem. Each node of the ASSOM generates input vectors from one invariance class. Training sequences are generated by nodes that are chosen according to a Markov process. We demonstrate that this model can learn invariant feature detectors similar to those found in the primary visual cortex from an unlabeled sequence of input images generated by a realistic model of eye movements. Performance is comparable to the original ASSOM algorithm, but without the need for explicit grouping into training episodes.