Search

Recent studies have explored finite-time dissipation-minimizing protocols for stochastic thermodynamic systems driven arbitrarily far from equilibrium, when granted full external control to drive the system. However, in both simulation and experimental contexts, systems often may only be controlled with a limited set of degrees of freedom. Here, going beyond slow- and fast-driving approximations employed in previous studies, we obtain exact finite-time optimal protocols for this unexplored limited-control setting. By working with deterministic Fokker-Planck probability density time evolution, we can frame the work-minimizing protocol problem in the standard form of an optimal control theory problem. We demonstrate that finding the exact optimal protocol is equivalent to solving a system of Hamiltonian partial differential equations, which in many cases admit efficiently calculatable numerical solutions. Within this framework, we reproduce analytical results for the optimal control of harmonic potentials, and numerically devise novel optimal protocols for two anharmonic examples: varying the stiffness of a quartic potential, and linearly biasing a double-well potential. We confirm that these optimal protocols outperform other protocols produced through previous methods, in some cases by a substantial amount. We find that for the linearly biased double-well problem, the mean position under the optimal protocol travels at a near-constant velocity. Surprisingly, for a certain timescale and barrier height regime, the optimal protocol is also non-monotonic in time., Comment: Main text: 11 pages, 3 figures. Supplementary materials: 10 pages, 3 figures

Brains can gracefully weed out irrelevant stimuli to guide behavior. This feat is believed to rely on a progressive selection of task-relevant stimuli across the cortical hierarchy, but the specific across-area interactions enabling stimulus selection are still unclear. Here, we propose that population gating, occurring within A1 but controlled by top-down inputs from mPFC, can support across-area stimulus selection. Examining single-unit activity recorded while rats performed an auditory context-dependent task, we found that A1 encoded relevant and irrelevant stimuli along a common dimension of its neural space. Yet, the relevant stimulus encoding was enhanced along an extra dimension. In turn, mPFC encoded only the stimulus relevant to the ongoing context. To identify candidate mechanisms for stimulus selection within A1, we reverse-engineered low-rank RNNs trained on a similar task. Our analyses predicted that two context-modulated neural populations gated their preferred stimulus in opposite contexts, which we confirmed in further analyses of A1. Finally, we show in a two-region RNN how population gating within A1 could be controlled by top-down inputs from PFC, enabling flexible across-area communication despite fixed inter-areal connectivity.

It is well known that sparse coding models trained on natural images learn basis functions whose shapes resemble the receptive fields (RFs) of simple cells in the primary visual cortex (V1). However, few studies have considered how these basis functions develop during training. In particular, it is unclear whether certain types of basis functions emerge more quickly than others, or whether they develop simultaneously. In this work, we train an overcomplete sparse coding model (Sparsenet) on natural images and find that there is indeed order in the development of its basis functions, with basis functions tuned to lower spatial frequencies emerging earlier and higher spatial frequency basis functions emerging later. We observe the same trend in a biologically plausible sparse coding model (SAILnet) that uses leaky integrate-and-fire neurons and synaptically local learning rules, suggesting that this result is a general feature of sparse coding. Our results are consistent with recent experimental evidence that the distribution of optimal stimuli for driving neurons to fire shifts towards higher frequencies during normal development in mouse V1. Our analysis of sparse coding models during training yields an experimentally testable prediction for V1 development that this shift may be due in part to higher spatial frequency RFs emerging later, as opposed to a global shift towards higher frequencies across all RFs, which may also play a role. We also find that at least two explanations could account for the order of RF development: 1) high frequency RFs require more information to be specified accurately, and thus may require more visual experience in order to learn, and 2) early development of low frequency RFs improves the sparseness and fidelity of the visual representation more than early development of high frequency RFs.Author summaryWe are interested in how visual neurons learn representations of the natural world. In particular, we want to know whether certain visual features are learned by the visual cortex earlier in development than others. To address this question, we turn to a class of algorithms that can learn to represent natural scenes in a sparse fashion, with only a few neurons active at any given time (population sparseness). While sparse coding has been used extensively to model the response properties of neurons in the visual cortex, we use it here to arrive at a quantitative description of the way neurons might learn to encode visual information during development. We find that receptive fields (RFs) tuned to lower spatial frequencies develop earlier in our sparse coding models compared to high frequency RFs. If our prediction is accurate, such a description would provide a general framework for understanding the development of the functional properties of V1 neurons and serve as a guide for future experimental studies. It could also lead to new computational models that learn from input statistics, as well as advances in the design of devices that can augment or replace human vision.

Engineered swift equilibration (ESE) is a class of driving protocols that enforce an equilibrium distribution with respect to external control parameters at the beginning and end of rapid state transformations of open, classical non-equilibrium systems. ESE protocols have previously been derived and experimentally realized for Brownian particles in simple, one-dimensional, time-varying trapping potentials; one recent study considered ESE in two-dimensional Euclidean configuration space. Here we extend the ESE framework to generic, overdamped Brownian systems in arbitrary curved configuration space and illustrate our results with specific examples not amenable to previous techniques. Our approach may be used to impose the necessary dynamics to control the full temporal configurational distribution in a wide variety of experimentally realizable settings., Comment: v4: 6+2 pages, 1 figure; correction of typo in funding information

Non-invasive and label-free spectral microscopy (spectromicroscopy) techniques can provide quantitative biochemical information complementary to genomic sequencing, transcriptomic profiling, and proteomic analyses. However, spectromicroscopy techniques generate high-dimensional data; acquisition of a single spectral image can range from tens of minutes to hours, depending on the desired spatial resolution and the image size. This substantially limits the timescales of observable transient biological processes. To address this challenge and move spectromicroscopy towards efficient real-time spatiochemical imaging, we developed a grid-less autonomous adaptive sampling method. Our method substantially decreases image acquisition time while increasing sampling density in regions of steeper physico-chemical gradients. When implemented with scanning Fourier Transform infrared spectromicroscopy experiments, this grid-less adaptive sampling approach outperformed standard uniform grid sampling in a two-component chemical model system and in a complex biological sample, Caenorhabditis elegans. We quantitatively and qualitatively assess the efficiency of data acquisition using performance metrics and multivariate infrared spectral analysis, respectively., Holman et al. develop a grid-less autonomous adaptive sampling method to explore high-dimensional spatiochemical experimental systems. Their method greatly decreases image acquisition time while improving spatial resolution, and when implemented with FTIR, it outperforms existing standard grid sampling approaches. They further show its utility for a complex biological sample, C. elegans.

Simultaneous recordings from the cortex have revealed that neural activity is highly variable, and that some variability is shared across neurons in a population. Further experimental work has demonstrated that the shared component of a neuronal population’s variability is typically comparable to or larger than its private component. Meanwhile, an abundance of theoretical work has assessed the impact shared variability has upon a population code. For example, shared input noise is understood to have a detrimental impact on a neural population’s coding fidelity. However, other contributions to variability, such as common noise, can also play a role in shaping correlated variability. We present a network of linear-nonlinear neurons in which we introduce a common noise input to model, for instance, variability resulting from upstream action potentials that are irrelevant for the task at hand. We show that by applying a heterogeneous set of synaptic weights to the neural inputs carrying the common noise, the network can improve its coding ability as measured by both Fisher information and Shannon mutual information, even in cases where this results in amplification of the common noise. With a broad and heterogeneous distribution of synaptic weights, a population of neurons can remove the harmful effects imposed by afferents that are uninformative about a stimulus. We demonstrate that some nonlinear networks benefit from weight diversification up to a certain population size, above which the drawbacks from amplified noise dominate over the benefits of diversification. We further characterize these benefits in terms of the relative strength of shared and private variability sources. Finally, we studied the asymptotic behavior of the mutual information and Fisher information analytically in our various networks as a function of population size. We find some surprising qualitative changes in the asymptotic behavior as we make seemingly minor changes in the synaptic weight distributions.

Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from many random initial points. One thread of work has focused on explaining this phenomenon by characterizing the local curvature near critical points of the loss function, where the gradients are near zero, and demonstrating that neural network losses enjoy a no-bad-local-minima property and an abundance of saddle points. We report here that the methods used to find these putative critical points suffer from a bad local minima problem of their own: they often converge to or pass through regions where the gradient norm has a stationary point. We call these gradient-flat regions, since they arise when the gradient is approximately in the kernel of the Hessian, such that the loss is locally approximately linear, or flat, in the direction of the gradient. We describe how the presence of these regions necessitates care in both interpreting past results that claimed to find critical points of neural network losses and in designing second-order methods for optimizing neural networks., 18 pages, 5 figures

For the benefit of designing scalable, fault resistant optical neural networks (ONNs), we investigate the effects architectural designs have on the ONNs' robustness to imprecise components. We train two ONNs -- one with a more tunable design (GridNet) and one with better fault tolerance (FFTNet) -- to classify handwritten digits. When simulated without any imperfections, GridNet yields a better accuracy (~98%) than FFTNet (~95%). However, under a small amount of error in their photonic components, the more fault tolerant FFTNet overtakes GridNet. We further provide thorough quantitative and qualitative analyses of ONNs' sensitivity to varying levels and types of imprecisions. Our results offer guidelines for the principled design of fault-tolerant ONNs as well as a foundation for further research.

Considerable progress has recently been made with geometrical approaches to understanding and controlling small out-of-equilibrium systems, but a mathematically rigorous foundation for these methods has been lacking. Towards this end, we develop a perturbative solution to the Fokker-Planck equation for one-dimensional driven Brownian motion in the overdamped limit enabled by the spectral properties of the corresponding single-particle Schr\"odinger operator. The perturbation theory is in powers of the inverse characteristic timescale of variation of the fastest varying control parameter, measured in units of the system timescale, which is set by the smallest eigenvalue of the corresponding Schr\"odinger operator. It applies to any Brownian system for which the Schr\"odinger operator has a confining potential. We use the theory to rigorously derive an exact formula for a Riemannian "thermodynamic" metric in the space of control parameters of the system. We show that up to second-order terms in the perturbation theory, optimal dissipation-minimizing driving protocols minimize the length defined by this metric. We also show that a previously proposed metric is calculable from our exact formula with corrections that are exponentially suppressed in a characteristic length scale. We illustrate our formula using the two-dimensional example of a harmonic oscillator with time-dependent spring constant in a time-dependent electric field. Lastly, we demonstrate that the Riemannian geometric structure of the optimal control problem is emergent; it derives from the form of the perturbative expansion for the probability density and persists to all orders of the expansion., Comment: 14 pages; edited for clarity, matches published version

Retinal ganglion cell outputs are less correlated across space than are natural scenes, and it has been suggested that this decorrelation is performed in the retina in order to improve efficiency and to benefit processing later in the visual system. However, sparse coding, a successful computational model of primary visual cortex, is achievable under some conditions with highly correlated inputs: most sparse coding algorithms learn the well-known sparse features of natural images and can output sparse, high-fidelity codes with or without a preceding decorrelation stage of processing. We propose that sparse coding with biologically plausible local learning rules does require decorrelated inputs, providing a possible explanation for why whitening may be necessary early in the visual system.

Neurons recorded in behaving animals often do not discernibly respond to sensory input and are not overtly task-modulated. These non-classically responsive neurons are difficult to interpret and are typically neglected from analysis, confounding attempts to connect neural activity to perception and behavior. Here, we describe a trial-by-trial, spike-timing-based algorithm to reveal the coding capacities of these neurons in auditory and frontal cortex of behaving rats. Classically responsive and non-classically responsive cells contained significant information about sensory stimuli and behavioral decisions. Stimulus category was more accurately represented in frontal cortex than auditory cortex, via ensembles of non-classically responsive cells coordinating the behavioral meaning of spike timings on correct but not error trials. This unbiased approach allows the contribution of all recorded neurons – particularly those without obvious task-related, trial-averaged firing rate modulation – to be assessed for behavioral relevance on single trials., eLife digest Neurons encode information in the form of electrical signals called spikes. Certain neurons increase the rate at which they produce spikes under specific circumstances, e.g., whenever an animal hears a particular sound. These neurons are said to be 'classically responsive'. But not all neurons behave in this way. Others produce spikes at a variable rate that does not obviously relate to the animal's behavior. These neurons are said to be 'non-classically responsive'. They are often omitted from analyses, despite typically outnumbering their classically responsive counterparts. So, what are these neurons doing? To find out, Insanally et al. trained rats to respond to sounds. The animals learned to poke their nose into a window whenever they heard a specific tone, and to avoid responding whenever they heard any other tone. As the rats performed the task, Insanally et al. recorded from neurons in two areas of the brain, the frontal cortex and the auditory cortex. A computer then analyzed the activity of individual neurons during each trial. As expected, the firing rate of non-classically responsive cells did not relate to the animals' behavior. But the timing of this firing did. The interval between spikes contained information about which tone the animals had heard and/or how they had responded. The cells worked together in groups to encode this information. Over the course of each trial, every neuron in the group varied the interval between its spikes. Eventually, the group reached a consensus, with all neurons using the same interval to represent information relevant to the task. Groups of neurons in the frontal cortex encoded more information about the category of the tone than those in the auditory cortex. By including all neurons – both classically and non-classically responsive – this model offers a more comprehensive view of how neural activity relates to behavior. This may in turn help us understand the variable and complex neural activity seen in people with sensory and cognitive disorders.

Active biological systems reside far from equilibrium, dissipating heat even in their steady state, thus requiring an extension of conventional equilibrium thermodynamics and statistical mechanics. In this Letter, we have extended the emerging framework of stochastic thermodynamics to active matter. In particular, for the active Ornstein-Uhlenbeck model, we have provided consistent definitions of thermodynamic quantities such as work, energy, heat, entropy, and entropy production at the level of single, stochastic trajectories and derived related fluctuation relations. We have developed a generalization of the Clausius inequality, which is valid even in the presence of the non-Hamiltonian dynamics underlying active matter systems. We have illustrated our results with explicit numerical studies.

Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with these constraints has not been explored. We provide upper and lower bounds on the entropy for the {\em minimum} entropy distribution over arbitrarily large collections of binary units with any fixed set of mean values and pairwise correlations. We also construct specific low-entropy distributions for several relevant cases. Surprisingly, the minimum entropy solution has entropy scaling logarithmically with system size for any set of first- and second-order statistics consistent with arbitrarily large systems. We further demonstrate that some sets of these low-order statistics can only be realized by small systems. Our results show how only small amounts of randomness are needed to mimic low-order statistical properties of highly entropic distributions, and we discuss some applications for engineered and biological information transmission systems., Comment: 34 pages, 7 figures

Sparse coding models of natural scenes can account for several physiological properties of primary visual cortex (V1), including the shapes of simple cell receptive fields (RFs) and the highly kurtotic firing rates of V1 neurons. Current spiking network models of pattern learning and sparse coding require direct inhibitory connections between the excitatory simple cells, in conflict with the physiological distinction between excitatory (glutamatergic) and inhibitory (GABAergic) neurons (Dale's Law). At the same time, the computational role of inhibitory neurons in cortical microcircuit function has yet to be fully explained. Here we show that adding a separate population of inhibitory neurons to a spiking model of V1 provides conformance to Dale's Law, proposes a computational role for at least one class of interneurons, and accounts for certain observed physiological properties in V1. When trained on natural images, this excitatory–inhibitory spiking circuit learns a sparse code with Gabor-like RFs as found in V1 using only local synaptic plasticity rules. The inhibitory neurons enable sparse code formation by suppressing predictable spikes, which actively decorrelates the excitatory population. The model predicts that only a small number of inhibitory cells is required relative to excitatory cells and that excitatory and inhibitory input should be correlated, in agreement with experimental findings in visual cortex. We also introduce a novel local learning rule that measures stimulus-dependent correlations between neurons to support “explaining away” mechanisms in neural coding.

Recent years have witnessed major advances in our understanding of nonequilibrium processes. The Jarzynski equality, for example, provides a link between equilibrium free energy differences and finite-time, nonequilibrium dynamics. We propose a generalization of this relation to non-Hamiltonian dynamics, relevant for active matter systems, continuous feedback, and computer simulation. Surprisingly, this relation allows us to calculate the free energy difference between the desired initial and final equilibrium states using arbitrary dynamics. As a practical matter, this dissociation between the dynamics and the initial and final states promises to facilitate a range of techniques for free energy estimation in a single, universal expression., 10 pages, no figures

The design of efficient quantum information processing will rely on optimal nonequilibrium transitions of driven quantum systems. Building on a recently developed geometric framework for computing optimal protocols for classical systems driven in finite time, we construct a general framework for optimizing the average information entropy for driven quantum systems. Geodesics on the parameter manifold endowed with a positive semidefinite metric correspond to protocols that minimize the average information entropy production in finite time. We use this framework to explicitly compute the optimal entropy production for a simple two-state quantum system coupled to a heat bath of bosonic oscillators, which has applications to quantum annealing.

Nonequilibrium physics encompasses a broad range of natural and synthetic small-scale systems. Optimizing transitions of such systems will be crucial for the development of nanoscale technologies and may reveal the physical principles underlying biological processes at the molecular level. Recent work has demonstrated that when a thermodynamic system is driven away from equilibrium then the space of controllable parameters has a Riemannian geometry induced by a generalized inverse diffusion tensor. We derive a simple, compact expression for the inverse diffusion tensor that depends solely on equilibrium information for a broad class of potentials. We use this formula to compute the minimal dissipation for two model systems relevant to small-scale information processing and biological molecular motors. In the first model, we optimally erase a single classical bit of information modelled by an overdamped particle in a smooth double-well potential. In the second model, we find the minimal dissipation of a simple molecular motor model coupled to an optical trap. In both models, we find that the minimal dissipation for the optimal protocol is inversely proportional to protocol duration, as expected, though the dissipation for the erasure model takes a different form than what we found previously for a similar system.

Objective: We present a device that combines principles of ultrasonic echolocation and spatial hearing to provide human users with environmental cues that are 1) not otherwise available to the human auditory system, and 2) richer in object and spatial information than the more heavily processed sonar cues of other assistive devices. The device consists of a wearable headset with an ultrasonic emitter and stereo microphones with affixed artificial pinnae. The goal of this study is to describe the device and evaluate the utility of the echoic information it provides. Methods: The echoes of ultrasonic pulses were recorded and time stretched to lower their frequencies into the human auditory range, then played back to the user. We tested performance among naive and experienced sighted volunteers using a set of localization experiments, in which the locations of echo-reflective surfaces were judged using these time-stretched echoes. Results: Naive subjects were able to make laterality and distance judgments, suggesting that the echoes provide innately useful information without prior training. Naive subjects were generally unable to make elevation judgments from recorded echoes. However, trained subjects demonstrated an ability to judge elevation as well. Conclusion: This suggests that the device can be used effectively to examine the environment and that the human auditory system can rapidly adapt to these artificial echolocation cues. Significance: Interpreting and interacting with the external world constitutes a major challenge for persons who are blind or visually impaired. This device has the potential to aid blind people in interacting with their environment.

Many models of cortical dynamics have focused on the high-firing regime, in which neurons are driven near their maximal rate. Here we consider the responses of neurons in auditory cortex under typical low-firing rate conditions, when stimuli have not been optimized to drive neurons maximally. We used whole-cell patch-clamp recordingin vivoto measure subthreshold membrane potential fluctuations in rat primary auditory cortex in both the anesthetized and awake preparations. By analyzing the subthreshold membrane potential dynamics on single trials, we made inferences about the underlying population activity. We found that, during both spontaneous and evoked responses, membrane potential was highly non-Gaussian, with dynamics consisting of occasional large excursions (sometimes tens of millivolts), much larger than the small fluctuations predicted by most random walk models that predict a Gaussian distribution of membrane potential. Thus, presynaptic inputs under these conditions are organized into quiescent periods punctuated by brief highly synchronous volleys, or “bumps.” These bumps were typically so brief that they could not be well characterized as “up states” or “down states.” We estimate that hundreds, perhaps thousands, of presynaptic neurons participate in the largest volleys. These dynamics suggest a computational scheme in which spike timing is controlled by concerted firing among input neurons rather than by small fluctuations in a sea of background activity.

The high variability of cortical sensory responses is often assumed to impose a major constraint on efficient computation. In the auditory cortex, however, response variability can be very low. We have used in vivo whole cell patch-clamp methods to study the trial-to-trial variability of the subthreshold fluctuations in membrane potential underlying tone-evoked responses in the auditory cortex of anesthetized rats. Using methods adapted from classical quantal analysis, we partitioned this subthreshold variability into a private component (which includes synaptic, thermal, and other sources local to the recorded cell) and a shared component arising from network interactions. Here we report that this private component is remarkably small, usually about 1–3 mV, as quantified by the variance divided by the mean of the ensemble of tone-evoked response heights. The shared component can be much larger, and shows more heterogeneity across the population, ranging from about 0 to 10 mV. The remarkable fact that, at least 5 synapses from the auditory periphery, this variability remains so small raises the possibility that the intervening neural circuitry is organized so as to prevent private noise from accumulating as neural signals propagate to the cortex.

© 2015 Marzen, DeWeese and Crutchfield. The mutual information between stimulus and spike-train response is commonly used to monitor neural coding efficiency, but neuronal computation broadly conceived requires more refined and targeted information measures of input-output joint processes. A first step toward that larger goal is to develop information measures for individual output processes, including information generation (entropy rate), stored information (statistical complexity), predictable information (excess entropy), and active information accumulation (bound information rate). We calculate these for spike trains generated by a variety of noise-driven integrate-and-fire neurons as a function of time resolution and for alternating renewal processes. We show that their time-resolution dependence reveals coarse-grained structural properties of interspike interval statistics; e.g., τ-entropy rates that diverge less quickly than the firing rate indicated by interspike interval correlations. We also find evidence that the excess entropy and regularized statistical complexity of different types of integrate-and-fire neurons are universal in the continuous-time limit in the sense that they do not depend on mechanism details. This suggests a surprising simplicity in the spike trains generated by these model neurons. Interestingly, neurons with gamma-distributed ISIs and neurons whose spike trains are alternating renewal processes do not fall into the same universality class. These results lead to two conclusions. First, the dependence of information measures on time resolution reveals mechanistic details about spike train generation. Second, information measures can be used as model selection tools for analyzing spike train processes.

We demonstrate a method for the selective encapsulation of carbon nanotubes in thin parylene films using iron as a sacrificial lift-off layer. The iron serves as an inhibitor of parylene deposition and prevents the parylene molecules from linking, thus facilitating selective area coating after lift-off.

We present closed-form expressions for the entropy rate, statistical complexity, and predictive information for the spike train of a single neuron in terms of the first passage time probability distribution. Our analysis applies to any one-dimensional neural model where observation of a spike causes the neuron to reset'' to some membrane voltage and in which any noise term is uncorrelated in time. We then use these formulae to study the linear leaky integrate-and- fire and quadratic integrate-and-fire neurons driven by white noise in the naturally spiking regime. The statistical complexity is simply related to the interspike interval's mean and coefficient of variation. The excess entropy, or the total predictive information, is highest for neural spike trains with low interspike interval coefficients of variation. Both the statistical complexity and the excess entropy arise naturally in the context of predictive rate-distortion and could be useful for characterizing predictor'' neurons that learn to predict other neurons.

SummaryAnimals can selectively respond to a target sound despite simultaneous distractors, just as humans can respond to one voice at a crowded cocktail party. To investigate the underlying neural mechanisms, we recorded single-unit activity in primary auditory cortex (A1) and medial prefrontal cortex (mPFC) of rats selectively responding to a target sound from a mixture. We found that prestimulus activity in mPFC encoded the selection rule—which sound from the mixture the rat should select. Moreover, electrically disrupting mPFC significantly impaired performance. Surprisingly, prestimulus activity in A1 also encoded selection rule, a cognitive variable typically considered the domain of prefrontal regions. Prestimulus changes correlated with stimulus-evoked changes, but stimulus tuning was not strongly affected. We suggest a model in which anticipatory activation of a specific network of neurons underlies the selection of a sound from a mixture, giving rise to robust and widespread rule encoding in both brain regions.Video Abstract

Information theory provides a powerful framework to analyse how neurons represent sensory stimuli or other behavioural variables. A recurring question regards the amount of information conveyed by a specific neuronal response. Here we show that the commonly used definition for this quantity has a serious flaw: the information accumulated during subsequent observations of neural activity fails to combine additively. Additivity is a highly desirable property, both on theoretical grounds and for the practical purpose of analysing population codes. We propose an alternative measure for the information per observation and prove that this is the only definition that satisfies additivity. The old and the new definitions measure very different aspects of the neural code, which is illustrated with visual responses from a motion-sensitive neuron in the primate cortex. Our analysis allows additional interpretation of several published results, which suggests that the neurons studied are operating far from their information capacity.

We continually rely on our ability to segregate the myriad sounds in our environment--phones ringing, people talking--into separate "auditory streams," each originating from a different source. In this issue of Neuron, Micheyl et al. provide the most direct evidence to date linking single-unit spiking responses from auditory cortex to the perception of distinct auditory streams.

Information erasure inevitably leads to the generation of heat. Minimizing this dissipation will be crucial for developing small-scale information processing systems, but little is known about the optimal procedures required. We have obtained closed-form expressions for maximally efficient erasure cycles for deletion of a classical bit of information stored by the position of a particle diffusing in a double-well potential. We find that the extra heat generated beyond the Landauer bound is proportional to the square of the Hellinger distance between the initial and final states divided by the cycle duration, which quantifies how far out of equilibrium the system is driven. Finally, we demonstrate close agreement between the exact optimal cycle and the protocol found using a linear response framework.

Recent experiments show that the neural codes at work in a wide range of creatures share some common features. At first sight, these observations seem unrelated. However, we show that these features arise naturally in a linear filtered threshold crossing (LFTC) model when we set the threshold to maximize the transmitted information. This maximization process requires neural adaptation to not only the DC signal level, as in conventional light and dark adaptation, but also to the statistical structure of the signal and noise distributions. We also present a new approach for calculating the mutual information between a neuron's output spike train and any aspect of its input signal which does not require reconstruction of the input signal. This formulation is valid provided the correlations in the spike train are small, and we provide a procedure for checking this assumption. This paper is based on joint work (DeWeese [1], 1995). Preliminary results from the LFTC model appeared in a previous proceedings (DeWeese [2], 1995), and the conclusions we reached at that time have been reaffirmed by further analysis of the model.

The dynamics of subthreshold membrane potential provide insight into the organization of activity in neural circuits. In many brain areas, membrane potential is bistable, transiting between a relatively hyperpolarized down state and a depolarized up state. These up and down states, which have been proposed to play a number of computational roles, have mainly been studied in anesthetized and in vitro preparations. Here, we have used intracellular recordings to characterize the dynamics of membrane potential in the auditory cortex of awake rats. We find that long up states are rare in the awake auditory cortex, with only 0.4% of up states >500 ms. Most neurons displayed only brief up states (bumps) and spent on average ∼1% of recording time in up states >500 ms. We suggest that the near absence of long up states in awake auditory cortex may reflect an adaptation to the rapid processing of auditory stimuli.

A deeper understanding of nonequilibrium phenomena is needed to reveal the principles governing natural and synthetic molecular machines. Recent work has shown that when a thermodynamic system is driven from equilibrium then, in the linear response regime, the space of controllable parameters has a Riemannian geometry induced by a generalized friction tensor. We exploit this geometric insight to construct closed-form expressions for minimal-dissipation protocols for a particle diffusing in a one-dimensional harmonic potential, where the spring constant, inverse temperature, and trap location are adjusted simultaneously. These optimal protocols are geodesics on the Riemannian manifold and reveal that this simple model has a surprisingly rich geometry. We test these optimal protocols via a numerical implementation of the Fokker-Planck equation and demonstrate that the friction tensor arises naturally from a first-order expansion in temporal derivatives of the control parameters, without appealing directly to linear response theory.

The opacity of typical objects in the world results in occlusion --- an important property of natural scenes that makes inference of the full 3-dimensional structure of the world challenging. The relationship between occlusion and low-level image statistics has been hotly debated in the literature, and extensive simulations have been used to determine whether occlusion is responsible for the ubiquitously observed power-law power spectra of natural images. To deepen our understanding of this problem, we have analytically computed the 2- and 4-point functions of a generalized "dead leaves" model of natural images with parameterized object transparency. Surprisingly, transparency alters these functions only by a multiplicative constant, so long as object diameters follow a power law distribution. For other object size distributions, transparency more substantially affects the low-level image statistics. We propose that the universality of power law power spectra for both natural scenes and radiological medical images -- formed by the transmission of x-rays through partially transparent tissue -- stems from power law object size distributions, independent of object opacity., 20 pages, 4 figures. Matches the version accepted to Phys Rev E

We have developed a sparse mathematical representation of speech that minimizes the number of active model neurons needed to represent typical speech sounds. The model learns several well-known acoustic features of speech such as harmonic stacks, formants, onsets and terminations, but we also find more exotic structures in the spectrogram representation of sound such as localized checkerboard patterns and frequency-modulated excitatory subregions flanked by suppressive sidebands. Moreover, several of these novel features resemble neuronal receptive fields reported in the Inferior Colliculus (IC), as well as auditory thalamus and cortex, and our model neurons exhibit the same tradeoff in spectrotemporal resolution as has been observed in IC. To our knowledge, this is the first demonstration that receptive fields of neurons in the ascending mammalian auditory pathway beyond the auditory nerve can be predicted based on coding principles and the statistical properties of recorded sounds., Author Summary The receptive field of a neuron can be thought of as the stimulus that most strongly causes it to be active. Scientists have long been interested in discovering the underlying principles that determine the structure of receptive fields of cells in the auditory pathway to better understand how our brains process sound. One possible way of predicting these receptive fields is by using a theoretical model such as a sparse coding model. In such a model, each sound is represented by the smallest possible number of active model neurons chosen from a much larger group. A primary question addressed in this study is whether the receptive fields of model neurons optimized for natural sounds will predict receptive fields of actual neurons. Here, we use a sparse coding model on speech data. We find that our model neurons do predict receptive fields of auditory neurons, specifically in the Inferior Colliculus (midbrain) as well as the thalamus and cortex. To our knowledge, this is the first time any theoretical model has been able to predict so many of the diverse receptive fields of the various cell-types in those areas.

Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function. We propose a new parameter fitting method, minimum probability flow (MPF), which is applicable to any parametric model. We demonstrate parameter estimation using MPF in two cases: a continuous state space model, and an Ising spin glass. In the latter case, MPF outperforms current techniques by at least an order of magnitude in convergence time with lower error in the recovered coupling parameters.

Sparse coding algorithms trained on natural images can accurately predict the features that excite visual cortical neurons, but it is not known whether such codes can be learned using biologically realistic plasticity rules. We have developed a biophysically motivated spiking network, relying solely on synaptically local information, that can predict the full diversity of V1 simple cell receptive field shapes when trained on natural images. This represents the first demonstration that sparse coding principles, operating within the constraints imposed by cortical architecture, can successfully reproduce these receptive fields. We further prove, mathematically, that sparseness and decorrelation are the key ingredients that allow for synaptically local plasticity rules to optimize a cooperative, linear generative image model formed by the neural representation. Finally, we discuss several interesting emergent properties of our network, with the intent of bridging the gap between theoretical and experimental studies of visual cortex., Author Summary In a sparse coding model, individual input stimuli are represented by the activities of model neurons, the majority of which are inactive in response to any particular stimulus. For a given class of stimuli, the neurons are optimized so that the stimuli can be faithfully represented with the minimum number of co-active units. This has been proposed as a model for visual cortex. While it has previously been demonstrated that sparse coding model neurons, when trained on natural images, learn to represent the same features as do neurons in primate visual cortex, it remains to be demonstrated that this can be achieved with physiologically realistic plasticity rules. In particular, learning in cortex appears to occur by the modification of synaptic connections between neurons, which must depend only on information available locally, at the synapse, and not, for example, on the properties of large numbers of distant cells. We provide the first demonstration that synaptically local plasticity rules are sufficient to learn a sparse image code, and to account for the observed response properties of visual cortical neurons: visual cortex actually could learn a sparse image code.

The canonical model of primary visual cortex (V1) is that it forms a linear generative model of the image stimulus presented to the eyes. Thus, for a given image with pixel values Xj, the representation Xj*=∑ibiψij is formed by multiplying the activity of each neuron (bi) by the feature that neuron encodes (ψij), and summing over all neurons. We call this a cooperative representation, since it involves all of the neurons collectively forming a single representation. Over time, the network is thought to adapt so as to minimize, on average, the mean-squared error between the representation X* and the input X, ||X-X*||2. Performing gradient descent on this error function yields the usual learning rule Δψij= α bi(Xj- ∑ibiψij ), where α is some small positive constant called the learning rate. Typically, the features ψij are interpreted as the receptive fields (RF’s; features to which a neuron responds) of the neurons; indeed, there is strong evidence [1] that the feature encoded by a neuron is very similar to its RF. In that case, the value ψij can be thought of as the strength of the synaptic connection between input pixel value Xj, and neuron i. With that interpretation in mind, it is clear that the canonical learning rule Δψij= α bi(Xj- ∑ibiψij ), used by most previous work in this field [1,2], fails to be biologically realistic because the rule for updating one synaptic strength ψij requires knowledge of the strengths of many synaptic connections, all on different neurons (with indices i), and it is not clear that such information is available to each individual synapse in the brain. We consider instead a Hebbian learning rule that respects synaptic locality, Δψij= α bi(Xj- biψij ) [3]. In this case, the information required to change the strength of synapse ψij consists solely of the pre-synaptic activity Xj, the post-synaptic activity bi, and the current strength of the synaptic connection ψij. While this rule respects the locality of synaptic information, it does not appear to perform gradient descent on the desired error function ||Xj- ∑ibiψij||2. Instead, our local rule can be seen as gradient descent on the error function ∑i||Xj- biψij ||2, which is the sum over all neurons of the error between each neuron’s own internal representation of the input, biψij, and the input image. In other words, a network that follows Oja’s [3] local learning rule is a solipsistic one: each neuron makes its own individual representation of the input, and learning optimizes each of those representations individually. We have proven that, if neuronal activities {bi} are uncorrelated, and sufficiently sparse (the majority of the bi’s are zero for any given image), the local and non-local learning rules are approximately equal, when averaged over many image presentations: = α ≈ α . This suggests a previously undiscovered role for independence and sparseness in visual cortex: these properties allow the neuronal network to (approximately) form the optimal cooperative representation, despite the locality of its learning rules. The same proof applies other neuronal networks that form linear generative models. We will present the details of our proof, and an example network (similar to that of [4]) of leaky integrate-and-fire neurons that learns a sparse image code using the local learning rule Δψij= α bi(Xj- biψij ). In our network, inhibitory inter-neuronal connections and variable firing thresholds keep the neuronal activities uncorrelated and sparse throughout the learning process. When trained on natural scenes, this network learns the same diversity of receptive fields as do previous non-local algorithms [1,2].

Until recently there have been no convincing quantitative measurements on the rates of information transmission in real neurons. Here we review the theoretical basis for making such measurements, together with the data which demonstrate remarkably high information rates in a variety of systems. In fact these rates are within a factor of two of the absolute physical limits set by the entropy of neural spike trains. Theses observations lead to sharp theoretical questions about the structure of the code and the strategy for adapting the code to different ensembles of input signals.

Neurons in the auditory cortex can lock with millisecond precision to the fine timingof acoustic stimuli, but it is not known whether this precise spike timing can be usedto guide decisions. We used chronically implanted microelectrode pairs to stimulateneurons in the rat auditory cortex directly. Here we demonstrate that rats canexploit differences in the timing of cortical activity as short as three milliseconds toguide decisions.

In vivo whole-cell patch-clamp recording provides a means for measuring membrane currents and potentials from individual cells in the intact animal. Patch-clamp methods have largely been developed in vitro. This body of work has contributed enormously to the understanding of many important phenomena in excitable cells--including synaptic plasticity in the mammalian central nervous system, and the behavior of individual protein channels. In recent years, an increasing number of groups have applied whole-cell recording techniques in the intact animal. Such in vivo studies offer the tantalizing possibility of uncovering the underlying principles and mechanisms of neural interactions within the natural context of fully intact biological networks. This unit focuses on strategies for overcoming the specific technical challenges posed by in vivo whole-cell recording. A straightforward procedure is described for obtaining whole-cell records from the cortex of the anesthetized rat; this procedure has also been applied successfully to awake animals and other rodent species with minor modifications.

Neurons are often assumed to operate in a highly unreliable manner: a neuron can signal the same stimulus with a variable number of action potentials. However, much of the experimental evidence supporting this view was obtained in the visual cortex. We have, therefore, assessed trial-to-trial variability in the auditory cortex of the rat. To ensure single-unit isolation, we used cell-attached recording. Tone-evoked responses were usually transient, often consisting of, on average, only a single spike per stimulus. Surprisingly, the majority of responses were not just transient, but were also binary, consisting of 0 or 1 action potentials, but not more, in response to each stimulus; several dramatic examples consisted of exactly one spike on 100% of trials, with no trial-to-trial variability in spike count. The variability of such binary responses differs from comparably transient responses recorded in visual cortical areas such as area MT, and represent the lowest trial-to-trial variability mathematically possible for responses of a given firing rate. Our study thus establishes for the first time that transient responses in auditory cortex can be described as a binary process, rather than as a highly variable Poisson process. These results demonstrate that cortical architecture can support a more precise control of spike number than was previously recognized, and they suggest a re-evaluation of models of cortical processing that assume noisiness to be an inevitable feature of cortical codes.