Search

A current limitation in our understanding of the visual system is its function under natural viewing conditions, especially in the context of dynamic, human behavior.While there have been many advances in understanding the spatial response properties of visual neurons in relation to static stimuli such as natural images, understanding of the corresponding temporal properties has been limited by the lack of high-fidelity datasets that document the properties of the signal that reaches the human retina under natural conditions. In this thesis, I describe the design and construction of a mobile tracking device that can re-create the signal present on the retina of a human as they perform everyday tasks. This device is used to collect high fidelity video and tracking data from human subjects performing a set of tasks that sample the everyday human environment and behavioral repertoire.This new dataset makes it possible to characterize the spatiotemporal statistics of natural time-varying signals as they occur on the retina. Here I examine the spatio-temporal power spectrum, which is of interest as a natural scene statistic in part because it is the Fourier transform of the autocorrelation function. In the absence of ego motion (movement of head and body), the spatiotemporal power spectrum of the dynamic environment has similar power-law structure to that previously reported for Hollywood movies. Head and eye motion modulate the spatiotemporal signal, boosting mid- and high-range temporal frequencies, such that the visual input on the retina is nearly whitened. This can be beneficial for reducing signal redundancy and maximizing the use of available bandwidth in the optic nerve. The phase spectrum, which compliments the power spectrum, also carries relevant information about natural image statistics. Despite the strong perceptual signal carried by the classically defined global phase, I show that it has limited utility to differentiate natural images from noise. However, phase congruency, a locally-defined property of the phase, shows marked differences between the distributions of natural images and noise, as well as differences within separate categories of natural scenes. Finally, I explore the relationship between natural signals and the human visual system by optimizing a neural network to carry the most amount of information through a bottleneck inspired by the human optic nerve, while limiting the energy utilized by neural spikes. I show that a previously proposed model exhibits computational instabilities that hinder the use of autodifferentiation software in training this model, and I offer methods of addressing them. I also show that this model can be reformulated with a restructuring of the network, from a single layer model to an autoencoder framework, avoiding computational instabilities altogether. I conclude with a summary of contributions, as well as a discussion of future areas of exploration.

A primary focus of neuroscience is understanding how information about the world is encoded in the activity of neurons. The classic approach to studying neural representation is to observe how activity in a neural system is modulated when presented with stimuli that vary along a specific, hypothesis-driven dimension (e.g. luminance, contrast, orientation, spatial frequency). This approach offers straight-forward interpretation and has allowed neuroscientists to develop computational models of the stimulus-response function for neurons in early processing stages of the visual system. However, the interpretations of these studies often do not generalize to other stimuli such as natural images, and the corresponding computational models often fail to explain later processing stages where the stimulus-response function is strikingly nonlinear. What is needed is an alternative approach that can a) inform new hypotheses without relying on the intuitions of the experimenter, b) generate tractable and interpretable nonlinear models of neural function based on these hypotheses, and c) directly assess model accuracy at predicting neural function under natural conditions. In this dissertation I propose a simple but powerful framework that uses statistical feature learning to automatically derive potential forms of neural representation and generates predictive and testable models of neural function based on these potential representations. The framework is general and can potentially be applied to data acquired using any physiological measurement technique (e.g. neurophysiology, neuroimaging, behavioral) in order to model neural representation throughout the brain. I demonstrate the potential of the proposed framework by first modeling the neural representation of image structure in early cortical stages of the macaque visual system (areas V1 and V2). I then apply the framework to investigate high-level scene processing in the human visual system. In both case studies, I am able to objectively derive accurate and interpretable nonlinear models of neural representation. In each study, I am also able to replicate and extend findings from a multitude of classic neurophysiology and neuroimaging studies using a single, simple experiment based on natural visual stimuli.

Sighted animals must survive in an environment that is diverse yet highly structured. Neural-coding models predict that the visual system should allocate its computational resources to exploit regularities in the environment, and that this allocation should facilitate perceptual judgments. Here we use three approaches (natural scenes statistical analysis, a reanalysis of single-unit data from alert behaving macaque, and a behavioral experiment in humans) to address the question of how the visual system maximizes behavioral success by taking advantage of low-level regularities in the environment. An analysis of natural scene statistics reveals that the probability distributions for light increments and decrements are biased in a way that could be exploited by the visual system to estimate depth from relative luminance. A reanalysis of neurophysiology data from Samonds et al. (2012) shows that the previously reported joint tuning of V1 cells for relative luminance and binocular disparity is well matched to a predicted distribution of binocular disparities produced by natural scenes. Finally, we show that a percept of added depth can be elicited in images by exaggerating the correlation between luminance and depth. Together, the results from these three approaches provide further evidence that the visual system allocates its processing resources in a way that is driven by the statistics of the natural environment.