Search

Understanding how the brain processes dynamic natural stimuli remains a fundamental challenge in neuroscience. Current dynamic neural encoding models either take stimuli as input but ignore shared variability in neural responses, or they model this variability by deriving latent embeddings from neural responses or behavior while ignoring the visual input. To address this gap, we propose a probabilistic model that incorporates video inputs along with stimulus-independent latent factors to capture variability in neuronal responses, predicting a joint distribution for the entire population. After training and testing our model on mouse V1 neuronal responses, we found that it outperforms video-only models in terms of log-likelihood and achieves further improvements when conditioned on responses from other neurons. Furthermore, we find that the learned latent factors strongly correlate with mouse behavior, although the model was trained without behavior data.

A high degree of structural complexity arises in dynamic neuronal dendrites due to extensive branching patterns and diverse spine morphologies, which enable the nervous system to adjust function, construct complex input pathways and thereby enhance the computational power of the system. Owing to the determinant role of dendrite morphology in the functionality of the nervous system, recognition of pathological changes due to neurodegenerative disorders is of crucial importance. We show that the statistical analysis of a temporary signal generated by cargos that have diffusively passed through the complex dendritic structure yields vital information about dendrite morphology. As a feasible scenario, we propose engineering mRNA-carrying multilamellar liposomes to diffusively reach the soma and release mRNAs, which encodes a specific protein upon encountering ribosomes. The concentration of this protein over a large population of neurons can be externally measured, as a detectable temporary signal. By developing a stochastic coarse-grained approach for first-passage through dendrites, we connect the model parameters to the characteristics of the evolving signal and, on the other hand, map them to the key morphological properties affected by neurodegenerative diseases -- including the density and size of spines, the extent of the tree, and the segmental increase of dendrite diameter towards soma. Thus, we establish a direct link between the dendrite morphology and the statistical characteristics of the detectable signal. Our approach provides a fast noninvasive measurement technique to indirectly extract vital information about the morphological evolution of dendrites in the course of neurodegenerative disease progression., Comment: 13 pages, 7 figures

Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the Sensorium 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net.

Deep predictive models of neuronal activity have recently enabled several new discoveries about the selectivity and invariance of neurons in the visual cortex. These models learn a shared set of nonlinear basis functions, which are linearly combined via a learned weight vector to represent a neuron's function. Such weight vectors, which can be thought as embeddings of neuronal function, have been proposed to define functional cell types via unsupervised clustering. However, as deep models are usually highly overparameterized, the learning problem is unlikely to have a unique solution, which raises the question if such embeddings can be used in a meaningful way for downstream analysis. In this paper, we investigate how stable neuronal embeddings are with respect to changes in model architecture and initialization. We find that $L_1$ regularization to be an important ingredient for structured embeddings and develop an adaptive regularization that adjusts the strength of regularization per neuron. This regularization improves both predictive performance and how consistently neuronal embeddings cluster across model fits compared to uniform regularization. To overcome overparametrization, we propose an iterative feature pruning strategy which reduces the dimensionality of performance-optimized models by half without loss of performance and improves the consistency of neuronal embeddings with respect to clustering neurons. This result suggests that to achieve an objective taxonomy of cell types or a compact representation of the functional landscape, we need novel architectures or learning techniques that improve identifiability. We will make our code available at publication time.

Single camera 3D pose estimation is an ill-defined problem due to inherent ambiguities from depth, occlusion or keypoint noise. Multi-hypothesis pose estimation accounts for this uncertainty by providing multiple 3D poses consistent with the 2D measurements. Current research has predominantly concentrated on generating multiple hypotheses for single frame static pose estimation or single hypothesis motion estimation. In this study we focus on the new task of multi-hypothesis motion estimation. Multi-hypothesis motion estimation is not simply multi-hypothesis pose estimation applied to multiple frames, which would ignore temporal correlation across frames. Instead, it requires distributions which are capable of generating temporally consistent samples, which is significantly more challenging than multi-hypothesis pose estimation or single-hypothesis motion estimation. To this end, we introduce Platypose, a framework that uses a diffusion model pretrained on 3D human motion sequences for zero-shot 3D pose sequence estimation. Platypose outperforms baseline methods on multiple hypotheses for motion estimation. Additionally, Platypose also achieves state-of-the-art calibration and competitive joint error when tested on static poses from Human3.6M, MPI-INF-3DHP and 3DPW. Finally, because it is zero-shot, our method generalizes flexibly to different settings such as multi-camera inference.

We consider the probability distributions of the subsystem (staggered) magnetization in ordered and disordered models of quantum magnets in D dimensions. We focus on Heisenberg antiferromagnets and long-range transverse-field Ising models as particular examples. By employing a range of self-consistent time-dependent mean-field approximations in conjunction with Holstein-Primakoff, Dyson-Maleev, Schwinger boson and modified spin-wave theory representations we obtain results in thermal equilibrium as well as during non-equilibrium evolution after quantum quenches. To extract probability distributions we derive a simple formula for the characteristic function of generic quadratic observables in any Gaussian theory of bosons., Comment: 54 pages, 35 figures

The presence of long-lived oscillations in the expectation values of local observables after quantum quenches has recently attracted considerable attention in relation to weak ergodicity breaking. Here we focus on an alternative mechanism that gives rise to such oscillations in a class of systems that support kinematically protected gapped excitations at zero temperature. An open question in this context is whether such oscillations will ultimately decay. We resolve this issue by considering spin models that can be mapped to systems of weakly interacting fermions, which in turn are amenable to an analysis by standard methods based on the BBGKY hierarchy. We find that there is a time scale beyond which the oscillations start to decay, which grows as the strength of the quench is made small., Comment: 14 pages, 13 Figures

These are notes based on lectures given at the 2021 summer school on Fundamental Problems in Statistical Physics XV. Their purpose is to give a very brief introduction to Generalized Hydrodynamics, which provides a description of the large scale structure of the dynamics in quantum integrable models. The notes are not meant to be comprehensive or provide an overview of all relevant literature, but rather give an exposition of the key ideas for non-experts, using a simple fermionic tight-binding model as the main example., Comment: 24 pages, 6 figures

Markov-chain Monte Carlo (MCMC), the field of stochastic algorithms built on the concept of sampling, has countless applications in science and technology. The overwhelming majority of MCMC algorithms are time-reversible and satisfy the detailed-balance condition, just like physical systems in thermal equilibrium. The underlying Markov chains typically display diffusive dynamics, which leads to a slow exploration of sample space. Significant speed-ups can be achieved by non-reversible MCMC algorithms exhibiting non-equilibrium dynamics, whose steady states exactly reproduce the target equilibrium states of reversible Markov chains. Such algorithms have had successes in applications but are generally difficult to analyze, resulting in a scarcity of exact results. Here, we introduce the "lifted" TASEP (totally asymmetric simple exclusion process) as a paradigm for lifted non-reversible Markov chains. Our model can be viewed as a second-generation lifting of the reversible Metropolis algorithm on a one-dimensional lattice and is exactly solvable by an unusual kind of coordinate Bethe ansatz. We establish the integrability of the model and present strong evidence that the lifting leads to faster relaxation than in the KPZ universality class., Comment: 11 pages, 8 figures

Understanding how biological visual systems process information is challenging due to the complex nonlinear relationship between neuronal responses and high-dimensional visual input. Artificial neural networks have already improved our understanding of this system by allowing computational neuroscientists to create predictive models and bridge biological and machine vision. During the Sensorium 2022, we introduced benchmarks for vision models with static input. However, animals operate and excel in dynamic environments, making it crucial to study and understand how the brain functions under these conditions. Moreover, many biological theories, such as predictive coding, suggest that previous input is crucial for current input processing. Currently, there is no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we propose the Sensorium 2023 Benchmark Competition with dynamic input. It includes the collection of a new large-scale dataset from the primary visual cortex of ten mice, containing responses from over 78,000 neurons to over 2 hours of dynamic stimuli per neuron. Participants in the main benchmark track will compete to identify the best predictive models of neuronal responses for dynamic input. We will also host a bonus track in which submission performance will be evaluated on out-of-domain input, using withheld neuronal responses to dynamic input stimuli whose statistics differ from the training set. Both tracks will offer behavioral data along with video stimuli. As before, we will provide code, tutorials, and strong pre-trained baseline models to encourage participation. We hope this competition will continue to strengthen the accompanying Sensorium benchmarks collection as a standard tool to measure progress in large-scale neural system identification models of the entire mouse visual hierarchy and beyond., Comment: arXiv admin note: substantial text overlap with arXiv:2206.08666

Knowledge distillation (KD) is a simple and successful method to transfer knowledge from a teacher to a student model solely based on functional activity. However, current KD has a few shortcomings: it has recently been shown that this method is unsuitable to transfer simple inductive biases like shift equivariance, struggles to transfer out of domain generalization, and optimization time is magnitudes longer compared to default non-KD model training. To improve these aspects of KD, we propose Hard Augmentations for Robust Distillation (HARD), a generally applicable data augmentation framework, that generates synthetic data points for which the teacher and the student disagree. We show in a simple toy example that our augmentation framework solves the problem of transferring simple equivariances with KD. We then apply our framework in real-world tasks for a variety of augmentation models, ranging from simple spatial transformations to unconstrained image manipulations with a pretrained variational autoencoder. We find that our learned augmentations significantly improve KD performance on in-domain and out-of-domain evaluation. Moreover, our method outperforms even state-of-the-art data augmentations and since the augmented training inputs can be visualized, they offer a qualitative insight into the properties that are transferred from the teacher to the student. Thus HARD represents a generally applicable, dynamically optimized data augmentation technique tailored to improve the generalization and convergence speed of models trained with KD.

In a recent numerical study by Haldar et al. (Phys. Rev. X 11, 031062) it was shown that signatures of proximate quantum critical points can be observed at early and intermediate times after certain quantum quenches. Said work focused mainly on the case of the axial next-nearest neighbour Ising (ANNNI) model. Here we construct a simple time-dependent mean-field theory that allows us to obtain a quantitatively accurate description of these quenches at short times, which for reasons we explain remains a fair approximation at late times (with some caveats). Our approach provides a simple framework for understanding the reported numerical results as well as fundamental limitations on detecting quantum critical points through quench dynamics. We moreover explain the origin of the peculiar oscillatory behaviour seen in various observables as arising from the formation of a long-lived bound state., Comment: 25 pages, 13 figures

Due to depth ambiguities and occlusions, lifting 2D poses to 3D is a highly ill-posed problem. Well-calibrated distributions of possible poses can make these ambiguities explicit and preserve the resulting uncertainty for downstream tasks. This study shows that previous attempts, which account for these ambiguities via multiple hypotheses generation, produce miscalibrated distributions. We identify that miscalibration can be attributed to the use of sample-based metrics such as minMPJPE. In a series of simulations, we show that minimizing minMPJPE, as commonly done, should converge to the correct mean prediction. However, it fails to correctly capture the uncertainty, thus resulting in a miscalibrated distribution. To mitigate this problem, we propose an accurate and well-calibrated model called Conditional Graph Normalizing Flow (cGNFs). Our model is structured such that a single cGNF can estimate both conditional and marginal densities within the same model - effectively solving a zero-shot density estimation problem. We evaluate cGNF on the Human~3.6M dataset and show that cGNF provides a well-calibrated distribution estimate while being close to state-of-the-art in terms of overall minMPJPE. Furthermore, cGNF outperforms previous methods on occluded joints while it remains well-calibrated., Comment: Under review

We demonstrate that nonlinear response functions in many-body systems carry a sharp signature of interactions between gapped low-energy quasiparticles. Such interactions are challenging to deduce from linear response measurements. The signature takes the form of a divergent-in-time contribution to the response -- linear in time in the case when quasiparticles propagate ballistically -- that is absent for free bosonic excitations. We give an intuitive semiclassical picture of this singular behaviour, validated against exact results from a form-factor expansion of the Ising chain and tDMRG simulations in a non-integrable model -- the spin-1 AKLT chain. We comment on extensions of these results to more general settings, finite temperature, and higher dimensions., Comment: Updated version with additional numerical data. 6 pages, 3 figures (+7 pages of supplementary material)

The neural underpinning of the biological visual system is challenging to study experimentally, in particular as the neuronal activity becomes increasingly nonlinear with respect to visual input. Artificial neural networks (ANNs) can serve a variety of goals for improving our understanding of this complex system, not only serving as predictive digital twins of sensory cortex for novel hypothesis generation in silico, but also incorporating bio-inspired architectural motifs to progressively bridge the gap between biological and machine vision. The mouse has recently emerged as a popular model system to study visual information processing, but no standardized large-scale benchmark to identify state-of-the-art models of the mouse visual system has been established. To fill this gap, we propose the Sensorium benchmark competition. We collected a large-scale dataset from mouse primary visual cortex containing the responses of more than 28,000 neurons across seven mice stimulated with thousands of natural images, together with simultaneous behavioral measurements that include running speed, pupil dilation, and eye movements. The benchmark challenge will rank models based on predictive performance for neuronal responses on a held-out test set, and includes two tracks for model input limited to either stimulus only (Sensorium) or stimulus plus behavior (Sensorium+). We provide a starting kit to lower the barrier for entry, including tutorials, pre-trained baseline models, and APIs with one line commands for data loading and submission. We would like to see this as a starting point for regular challenges and data releases, and as a standard tool for measuring progress in large-scale neural system identification models of the mouse visual system and beyond., Comment: NeurIPS 2022 Competition Track

We consider fermions defined on a continuous one-dimensional interval and subject to weak repulsive two-body interactions. We show that it is possible to perturbatively construct an extensive number of mutually compatible conserved charges for any interaction potential. However, the contributions to the densities of these charges at second order and higher are generally non-local and become spatially localized only if the potential fulfils certain compatibility conditions. We prove that the only solutions to the first of these conditions are the Cheon-Shigehara potential (fermionic dual to the Lieb-Liniger model) and the Calogero-Sutherland potentials. We use our construction to show how Generalized Hydrodynamics (GHD) emerges from the Bogoliubov--Born--Green--Kirkwood--Yvon hierarchy, and argue that GHD in the weak interaction regime is robust under non-integrable perturbations., Comment: 7+8 pages, no figures; v2 Property 1 amended; v3 improved presentation, as appears in PRL

In one dimensional quantum gases there is a well known "duality" between hard core bosons and non-interacting fermions. However, at the field theory level, no exact duality connecting strongly interacting bosons to weakly interacting fermions is known. Here we propose a solution to this long standing problem. Our derivation relies on regularizing the only point-like interaction between fermions in 1D that induces a discontinuity in the wave function proportional to its derivative. In contrast to all known regularisations our potential is weak for small interaction strengths. Crucially, this allows one to apply standard methods of diagrammatic perturbation theory to strongly interacting bosons. As a first application we compute the finite temperature spectral function of the Cheon-Shigehara model, the fermionic model dual to the celebrated Lieb-Liniger model., Comment: 5 pages + supplemental material. Title changed and typos corrected

Deep neural networks set the state-of-the-art across many tasks in computer vision, but their generalization ability to image distortions is surprisingly fragile. In contrast, the mammalian visual system is robust to a wide range of perturbations. Recent work suggests that this generalization ability can be explained by useful inductive biases encoded in the representations of visual stimuli throughout the visual cortex. Here, we successfully leveraged these inductive biases with a multi-task learning approach: we jointly trained a deep network to perform image classification and to predict neural activity in macaque primary visual cortex (V1). We measured the out-of-distribution generalization abilities of our network by testing its robustness to image distortions. We found that co-training on monkey V1 data leads to increased robustness despite the absence of those distortions during training. Additionally, we showed that our network's robustness is very close to that of an Oracle network where parts of the architecture are directly trained on noisy images. Our results also demonstrated that the network's representations become more brain-like as their robustness improves. Using a novel constrained reconstruction analysis, we investigated what makes our brain-regularized network more robust. We found that our co-trained network is more sensitive to content than noise when compared to a Baseline network that we trained for image classification alone. Using DeepGaze-predicted saliency maps for ImageNet images, we found that our monkey co-trained network tends to be more sensitive to salient regions in a scene, reminiscent of existing theories on the role of V1 in the detection of object borders and bottom-up saliency. Overall, our work expands the promising research avenue of transferring inductive biases from the brain, and provides a novel analysis of the effects of our transfer., Comment: NeurIPS 2021 camera ready (final)

We consider the dynamics of fluctuations in the quantum asymmetric simple exclusion process (Q-ASEP) with periodic boundary conditions. The Q-ASEP describes a chain of spinless fermions with random hoppings that are induced by a Markovian environment. We show that fluctuations of the fermionic degrees of freedom obey evolution equations of Lindblad type, and derive the corresponding Lindbladians. We identify the underlying algebraic structure by mapping them to non-Hermitian spin chains and demonstrate that the operator space fragments into exponentially many (in system size) sectors that are invariant under time evolution. At the level of quadratic fluctuations we consider the Lindbladian on the sectors that determine the late time dynamics for the particular case of the quantum symmetric simple exclusion process (Q-SSEP). We show that the corresponding blocks in some cases correspond to known Yang-Baxter integrable models and investigate the level-spacing statistics in others. We carry out a detailed analysis of the steady states and slow modes that govern the late time behaviour and show that the dynamics of fluctuations of observables is described in terms of closed sets of coupled linear differential-difference equations. The behaviour of the solutions to these equations is essentially diffusive but with relevant deviations, that at sufficiently late times and large distances can be described in terms of a continuum scaling limit which we construct. We numerically check the validity of this scaling limit over a significant range of time and space scales. These results are then applied to the study of operator spreading at large scales, focusing on out-of-time ordered correlators and operator entanglement., Comment: 66 pages, 12 figures

We consider the XY spin chain with arbitrary time-dependent magnetic field and anisotropy. We argue that a certain subclass of Gaussian states, called Coherent Ensemble (CE) following [1], provides a natural and unified framework for out-of-equilibrium physics in this model. We show that $all$ correlation functions in the CE can be computed using form factor expansion and expressed in terms of Fredholm determinants. In particular, we present exact out-of-equilibrium expressions in the thermodynamic limit for the previously unknown order parameter one-point function, dynamical two-point function and equal-time three-point function., Comment: 44 pages

We study quantum gravity effects on radiation properties of thin accretion disks around a renormalization group improved (RGI-) Schwarzschild black hole. In the infrared (IR) limit of the asymptotically safe theory with higher derivatives, the running Newton coupling G(r) depends on a free parameter which encodes the quantum effects on the spacetime geometry. By varying this parameter, modifications to thermal properties of the disk as the time averaged energy flux, the disk temperature, the differential luminosity, and the conversion efficiency of accreting mass into radiation, are obtained. In addition to a shifting of the radius of the innermost stable circular orbit (ISCO) toward small values, we find an increase of the maximum values of these thermal properties and a greater efficiency than in the classical relativistic regime. We discuss astrophysical applications of these results by using observational data of the stellar-mass black hole candidate LMC X-3. Our findings could, in principle, be used to identify quantum gravity effects through astrophysical observations., Comment: 15 pages, 9 figures, 2 tables, discussion expanded, references added, conclusions unchanged, accepted for publication in EPJC

We consider a Lindblad equation that for particular initial conditions reduces to an asymmetric simple exclusion process with additional loss and gain terms. The resulting Lindbladian exhibits operator-space fragmentation and each block is Yang-Baxter integrable. For particular loss/gain rates the model can be mapped to free fermions. We determine the full quantum dynamics for an initial product state in this case., Comment: 27 pages, 5 figures. Explanations clarified from previous version, minor notation changes, minor typos fixed. Accepted for publication in JSTAT

We review the issue of steady spherically symmetric accretion onto a renormalization group improved Schwarzschild space-time which is solution to an asymptotically safe theory (AS) containing high-derivative terms. We use a Hamiltonian dynamical system approach for the analysis of the accretion of four types of isothermal test fluids: ultra-stiff fluid, ultra-relativistic fluid, radiation fluid, and sub-relativistic fluid. An important outcome of our study is that, contrary to what claimed in a recent work, there exist physical solutions for the accretion of an ultra-relativistic fluid in AS which include subsonic, supersonic and transonic regimes. We also study quantum corrections to the known stability of the accretion in general relativity (GR). To this end we use a perturbative procedure based on the continuity equation with the mass accretion rate being the perturbed quantity. Two classes of perturbations are studied: standing waves and travelling waves. We find that quantum gravity effects either enhance or diminish the stability of the accretion depending on the type of test fluid and depending on the radial distance to the central object., Comment: 20 pages, 11 figures, accepted for publication in Chinese Physics C

We consider the time evolution of local observables after an interaction quench in the repulsive Lieb-Liniger model. The system is initialized in the ground state for vanishing interaction and then time-evolved with the Lieb-Liniger Hamiltonian for large, finite interacting strength $c$. We employ the Quench Action approach to express the full time evolution of local observables in terms of sums over energy eigenstates and then derive the leading terms of a $1/c$ expansion for several one and two-point functions as a function of time $t > 0$ after the quantum quench. We observe delicate cancellations of contributions to the spectral sums that depend on the details of the choice of representative state in the Quench Action approach and our final results are independent of this choice. Our results provide a highly non-trivial confirmation of the typicality assumptions underlying the Quench Action approach., Comment: 42 pages