Search

Stress-strain relations for random packings of entangling chains under triaxial compression can exhibit strain stiffening and sustain stresses several orders-of-magnitude beyond typical granular materials. X-ray tomography reveals the transition to this strong strain stiffening occurs when chains are long enough to entangle an average of about one chain each, which results in system-filling clusters of entangled chains. The number of entanglements is nearly proportional to the area surrounded by entangling particles with an excluded volume effect. A tendency was found for chain links to stretch when the packing was strained. The slope of the stress-strain relation of the packing can be calculated from a mean-field model consisting of the product of the effective extensional modulus of the chain, packing fraction, probability of stretched links, and the ratio of strain of stretched links to packing strain. The stress-strain model requires as input measurements of the ratio between local particle deformation and global average strain, and the probability of stretching for non-rigid particles. This results in a quadratic prediction for the stress-strain curve, with a curvature that agrees with experiments within the model uncertainties. This model explains that the strength of these packings comes from stretching of the links of chains, but only when the system-filling network of entanglements provides constraints that prevents failure by shear banding, so that particles must be deformed to move further under strain. In this model, the increasing slope of the stress-strain curve is mainly due to the fraction of stretched links increasing with strain. This model for the stress-strain relation is shown to be generalizable to different shapes of entangling particles by applying it to staples., Comment: 21 pages, 24 figures, submitted to Physical Review E

The advent of Large Language Models (LLMs) and Generative AI has revolutionized natural language applications across various domains. However, high-stakes decision-making tasks in fields such as medical, legal and finance require a level of precision, comprehensiveness, and logical consistency that pure LLM or Retrieval-Augmented-Generation (RAG) approaches often fail to deliver. At Elemental Cognition (EC), we have developed a neuro-symbolic AI platform to tackle these problems. The platform integrates fine-tuned LLMs for knowledge extraction and alignment with a robust symbolic reasoning engine for logical inference, planning and interactive constraint solving. We describe Cora, a Collaborative Research Assistant built on this platform, that is designed to perform complex research and discovery tasks in high-stakes domains. This paper discusses the multi-step inference challenges inherent in such domains, critiques the limitations of existing LLM-based methods, and demonstrates how Cora's neuro-symbolic approach effectively addresses these issues. We provide an overview of the system architecture, key algorithms for knowledge extraction and formal reasoning, and present preliminary evaluation results that highlight Cora's superior performance compared to well-known LLM and RAG baselines.

We report superluminal jet motion with an apparent speed of $\beta_\mathrm{app}=1.65\pm0.57$ in the radio-quiet (RQ) low ionisation nuclear emission line region (LINER) galaxy, KISSR872. This result comes from two epoch phase-referenced very long baseline interferometry (VLBI) observations at 5 GHz. The detection of bulk relativistic motion in the jet of this extremely radio faint AGN, with a total 1.4 GHz flux density of 5 mJy in the 5.4 arcsec resolution Very Large Array (VLA) FIRST survey image and 1.5 mJy in the $\sim5$ milli-arcsec resolution Very Long Baseline Array (VLBA) image, is the first of its kind in a RQ LINER galaxy. The presence of relativistic jets in lower accretion rate objects like KISSR872, with an Eddington ratio of 0.04, reveals that even RQ AGN can harbor relativistic jets, and evidentiates their universality over a wide range of accretion powers., Comment: 9 pages, 4 figures, accepted for publication in ApJ

Training networks consisting of biophysically accurate neuron models could allow for new insights into how brain circuits can organize and solve tasks. We begin by analyzing the extent to which the central algorithm for neural network learning -- stochastic gradient descent through backpropagation (BP) -- can be used to train such networks. We find that properties of biophysically based neural network models needed for accurate modelling such as stiffness, high nonlinearity and long evaluation timeframes relative to spike times makes BP unstable and divergent in a variety of cases. To address these instabilities and inspired by recent work, we investigate the use of "gradient-estimating" evolutionary algorithms (EAs) for training biophysically based neural networks. We find that EAs have several advantages making them desirable over direct BP, including being forward-pass only, robust to noisy and rigid losses, allowing for discrete loss formulations, and potentially facilitating a more global exploration of parameters. We apply our method to train a recurrent network of Morris-Lecar neuron models on a stimulus integration and working memory task, and show how it can succeed in cases where direct BP is inapplicable. To expand on the viability of EAs in general, we apply them to a general neural ODE problem and a stiff neural ODE benchmark and find again that EAs can out-perform direct BP here, especially for the over-parameterized regime. Our findings suggest that biophysical neurons could provide useful benchmarks for testing the limits of BP-adjacent methods, and demonstrate the viability of EAs for training networks with complex components.

This paper explores the potential of the transformer models for learning Granger causality in networks with complex nonlinear dynamics at every node, as in neurobiological and biophysical networks. Our study primarily focuses on a proof-of-concept investigation based on simulated neural dynamics, for which the ground-truth causality is known through the underlying connectivity matrix. For transformer models trained to forecast neuronal population dynamics, we show that the cross attention module effectively captures the causal relationship among neurons, with an accuracy equal or superior to that for the most popular Granger causality analysis method. While we acknowledge that real-world neurobiology data will bring further challenges, including dynamic connectivity and unobserved variability, this research offers an encouraging preliminary glimpse into the utility of the transformer model for causal representation learning in neuroscience., Comment: Accepted to the NeurIPS 2023 Workshop on Causal Representation Learning

In theoretical neuroscience, recent work leverages deep learning tools to explore how some network attributes critically influence its learning dynamics. Notably, initial weight distributions with small (resp. large) variance may yield a rich (resp. lazy) regime, where significant (resp. minor) changes to network states and representation are observed over the course of learning. However, in biology, neural circuit connectivity could exhibit a low-rank structure and therefore differs markedly from the random initializations generally used for these studies. As such, here we investigate how the structure of the initial weights -- in particular their effective rank -- influences the network learning regime. Through both empirical and theoretical analyses, we discover that high-rank initializations typically yield smaller network changes indicative of lazier learning, a finding we also confirm with experimentally-driven initial connectivity in recurrent neural networks. Conversely, low-rank initialization biases learning towards richer learning. Importantly, however, as an exception to this rule, we find lazier learning can still occur with a low-rank initialization that aligns with task and data statistics. Our research highlights the pivotal role of initial weight structures in shaping learning regimes, with implications for metabolic costs of plasticity and risks of catastrophic forgetting., Comment: Published at ICLR 2024

The generalization capacity of deep neural networks has been studied in a variety of ways, including at least two distinct categories of approaches: one based on the shape of the loss landscape in parameter space, and the other based on the structure of the representation manifold in feature space (that is, in the space of unit activities). Although these two approaches are related, they are rarely studied together explicitly. Here, we present an analysis that bridges this gap. We show that in the final phase of learning in deep neural networks, the compression of the manifold of neural representations correlates with the flatness of the loss around the minima explored by SGD. This correlation is predicted by a relatively simple mathematical relationship: a flatter loss corresponds to a lower upper bound on the compression metrics of neural representations. Our work builds upon the linear stability insight by Ma and Ying, deriving inequalities between various compression metrics and quantities involving sharpness. Empirically, our derived inequality predicts a consistently positive correlation between representation compression and loss sharpness in multiple experimental settings. Overall, we advance a dual perspective on generalization in neural networks in both parameter and feature space.

In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for $\textit{recurrent}$ neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.

The core challenge in numerous real-world applications is to match an inquiry to the best document from a mutable and finite set of candidates. Existing industry solutions, especially latency-constrained services, often rely on similarity algorithms that sacrifice quality for speed. In this paper we introduce a generic semantic learning-to-rank framework, Self-training Semantic Cross-attention Ranking (sRank). This transformer-based framework uses linear pairwise loss with mutable training batch sizes and achieves quality gains and high efficiency, and has been applied effectively to show gains on two industry tasks at Microsoft over real-world large-scale data sets: Smart Reply (SR) and Ambient Clinical Intelligence (ACI). In Smart Reply, $sRank$ assists live customers with technical support by selecting the best reply from predefined solutions based on consumer and support agent messages. It achieves 11.7% gain in offline top-one accuracy on the SR task over the previous system, and has enabled 38.7% time reduction in composing messages in telemetry recorded since its general release in January 2021. In the ACI task, sRank selects relevant historical physician templates that serve as guidance for a text summarization model to generate higher quality medical notes. It achieves 35.5% top-one accuracy gain, along with 46% relative ROUGE-L gain in generated medical notes.

Student-teacher relationships are important to student outcomes and may be especially pivotal at the high school transition and for minoritized racial/ethnic groups. Although interventions exist to improve student-teacher relationships, none have been shown to be effective among high school students or in narrowing racial/ethnic disparities in student outcomes. This study was conducted to examine the effects of an equity-explicit student-teacher relationship intervention (Equity-Explicit Establish Maintain Restore, or E-EMR) for high school teachers and students. A cluster-randomized pilot trial was conducted with 94 ninth grade teachers and 417 ninth grade students in six high schools. Teachers in three schools were randomized to receive E-EMR training and follow-up supports for one year. Teachers in three control schools conducted business as usual. Student-teacher relationships, sense of school belonging, academic motivation, and academic engagement were collected via student self-report in September and January of their ninth-grade year. Longitudinal models revealed non-significant main effects of E-EMR. However, there were targeted benefits for students who started with low scores at baseline, for Asian, Latinx, multicultural, and (to a lesser extent) Black students. We also found some unexpected effects, where high-performing and/or advantaged groups in the E-EMR condition had less favorable outcomes at post, compared to those in the control group, which may be a result of the equity-explicit focus of E-EMR. Implications and directions for future research are discussed. [This is the online first version of an article published in "School Mental Health."]

Rayleigh-B{\'e}nard convection experiments were done with two adjacent cubic cells with a partial wall in between to force the generation of two interacting convection rolls. Observed stable states include both counter-rotating and co-rotating states. The stability of each of these states and their dynamics were modeled by stochastic ordinary differential equations of motion in terms of the orientation, amplitude, and mean temperature of each convection roll. The form of the interaction terms is predicted based on an effective turbulent diffusion of temperature between the adjacent rolls. Predictions are made for stable fixed points of the co- and counter-rotating states. This suggests that the same turbulent thermal diffusivity that describes macroscopically averaged heat transport also controls the interactions between neighboring convection rolls. The surprising stability of co-rotating states is due to the temperature difference between the neighboring rolls becoming large enough that the heat flux between the rolls stabilizes the temperature profile of aligned co-rotating states. This temperature difference can be driven by heating the plates of the two cells to different mean temperatures. This shifts the orientations of the rolls of counter-rotating states in opposite directions, and for large temperature differences only co-rotating states are stable Spontaneous switching between co-rotating and counter-rotating states is also observed. Switching to counter-rotating states occurs mainly due to cessation (a significant weakening of a convection roll), which reduces damping on changes in orientation, allowing the orientation to change rapidly due to diffusive fluctuations. Switching to co-rotating states is mainly driven by smaller diffusive fluctuations, which have a positive feedback that destabilizes the counter-rotating state., Comment: 28 pages, 17 figures

The spectacular successes of recurrent neural network models where key parameters are adjusted via backpropagation-based gradient descent have inspired much thought as to how biological neuronal networks might solve the corresponding synaptic credit assignment problem. There is so far little agreement, however, as to how biological networks could implement the necessary backpropagation through time, given widely recognized constraints of biological synaptic network signaling architectures. Here, we propose that extra-synaptic diffusion of local neuromodulators such as neuropeptides may afford an effective mode of backpropagation lying within the bounds of biological plausibility. Going beyond existing temporal truncation-based gradient approximations, our approximate gradient-based update rule, ModProp, propagates credit information through arbitrary time steps. ModProp suggests that modulatory signals can act on receiving cells by convolving their eligibility traces via causal, time-invariant and synapse-type-specific filter taps. Our mathematical analysis of ModProp learning, together with simulation results on benchmark temporal tasks, demonstrate the advantage of ModProp over existing biologically-plausible temporal credit assignment rules. These results suggest a potential neuronal mechanism for signaling credit information related to recurrent interactions over a longer time horizon. Finally, we derive an in-silico implementation of ModProp that could serve as a low-complexity and causal alternative to backpropagation through time., Comment: NeurIPS 2022 Camera Ready

To unveil how the brain learns, ongoing work seeks biologically-plausible approximations of gradient descent algorithms for training recurrent neural networks (RNNs). Yet, beyond task accuracy, it is unclear if such learning rules converge to solutions that exhibit different levels of generalization than their nonbiologically-plausible counterparts. Leveraging results from deep learning theory based on loss landscape curvature, we ask: how do biologically-plausible gradient approximations affect generalization? We first demonstrate that state-of-the-art biologically-plausible learning rules for training RNNs exhibit worse and more variable generalization performance compared to their machine learning counterparts that follow the true gradient more closely. Next, we verify that such generalization performance is correlated significantly with loss landscape curvature, and we show that biologically-plausible learning rules tend to approach high-curvature regions in synaptic weight space. Using tools from dynamical systems, we derive theoretical arguments and present a theorem explaining this phenomenon. This predicts our numerical results, and explains why biologically-plausible rules lead to worse and more variable generalization properties. Finally, we suggest potential remedies that could be used by the brain to mitigate this effect. To our knowledge, our analysis is the first to identify the reason for this generalization gap between artificial and biologically-plausible learning rules, which can help guide future investigations into how the brain learns solutions that generalize., Comment: NeurIPS 2022 Camera Ready Version

Responding to an Illinois law that took effect in 2010, the Performance Evaluation Reform Act (PERA), Chicago Public Schools (CPS) created and implemented the REACH (Recognizing Educators Advancing Chicago's Students) educator evaluation and support system. REACH was piloted in 2012-13. It is still being implemented as of the 2019-20 school year and will continue to be implemented through June 2024, per the teachers' contract agreed upon in fall 2019. The evaluation system includes classroom observation ratings of teacher practice, as well as up to two student growth measures. This brief addresses the research question: What are teachers' and administrators' perceptions and opinions of REACH after five years of implementation? It draws from teacher and administrator survey reports of whether REACH influenced instructional change and student learning, the intended end goals of the policy. This brief is part of a series of three to extend previous research from the University of Chicago Consortium on School Research (UChicago Consortium) on early REACH implementation and the REACH evaluation system five years after the 2012-13 rollout. [For "Teacher Evaluation in CPS: What Makes Evaluator Feedback Useful? Research Brief," see ED608091. For "Teacher Evaluation in CPS: REACH Ratings and Teacher Mobility. Research Brief," see ED608092.]

Emerging literature has highlighted the importance of discerning general and strategic organizational context (OC) factors (e.g., leadership and climate) and their interaction effect on individual implementation behaviors (e.g., attitudes toward evidence-based practices; EBPs) in youth mental healthcare. This study aimed to examine how leadership and climate (general and strategic) are associated with implementer attitudes toward EBPs across the individual and organizational levels and their interaction effect in schools. A series of multilevel models (MLMs) were fitted on a diverse sample of schools actively implementing universal prevention programs for youth mental health (441 implementers from 52 schools). The organization-level aggregates and individual educators' perceptions of general and strategic leadership and climate, and their interaction terms, were entered as level-2 and level-1 predictors of four attitudinal dimensions (Requirement, Openness, Appeal, and Divergence) based on their level of measurement. At the organizational level, higher levels of strategic leadership and climate, but not their general counterparts, were consistently associated with more favorable attitudes in all four dimensions. At the individual/within-school level, higher levels of perceived general and strategic leadership and climate were associated with more favorable attitudes of Requirement and Openness. At the organizational/between-school level, general climate moderated the positive effect of strategic climate on implementers' perception of appeal and divergence of EBPs. Our findings indicate that leaders should make data-based decisions to allocate resources on strategic and/or general leadership and climate to foster favorable staff attitudes toward EBPs based on the level of measurement, implementation-specificity, and attitudinal dimensions. [This is the online version of an article published in "Administration and Policy in Mental Health and Mental Health Services Research."]

Quantum Amplitude Estimation (QAE) -- a technique by which the amplitude of a given quantum state can be estimated with quadratically fewer queries than by standard sampling -- is a key sub-routine in several important quantum algorithms, including Grover search and Quantum Monte-Carlo methods. An obstacle to implementing QAE in near-term noisy intermediate-scale quantum (NISQ) devices has been the need to perform Quantum Phase Estimation (QPE) -- a costly procedure -- as a sub-routine. This impediment was lifted with various QPE-free methods of QAE, wherein Grover queries of varying depths / powers (often according to a "schedule") are followed immediately by measurements and classical post-processing techniques like maximum likelihood estimation (MLE). Existing analyses as to the optimality of various query schedules in these QPE-free QAE schemes have hitherto assumed noise-free systems. In this work, we analyse QPE-free QAE under common noise models that may afflict NISQ devices and report on the optimality of various query schedules in the noisy regime. We demonstrate that, given an accurate noise characterization of one's system, one must choose a schedule that balances the trade-off between the greater ideal performance achieved by higher-depth circuits, and the correspondingly greater accumulation of noise-induced error.

We experimentally test the effects of tilting a turbulent Rayleigh-B{\'e}nard convection cell on the dynamics of the large-scale circulation (LSC) orientation $\theta_0$. The probability distribution of $\theta_0$ is measured, and used to obtain a tilt-induced potential acting on $\theta_0$, which is used in a low-dimensional model of diffusion of $\theta_0$ in a potential. The form of the potential is sinusoidal in $\theta_0$, and linear in tilt angle for small tilt angles, which is explained by a simple geometric model of the vector direction of the mean buoyancy force acting on the LSC. However, the magnitude of the tilt-induced forcing is found to be two orders of magnitude larger than previously predicted. When this parameter is adjusted to match values obtained from the probability distribution of $\theta_0$, the diffusive model can quantitatively predict effects of tilt on $\theta_0$. In particular, tilt causes a change in potential barrier height between neighboring corners of a cubic cell, and changes in the barrier-crossing rate for $\theta_0$ to escape a corner are predicted with an accuracy of $\pm30\%$. As a cylindrical cell is tilted, the tilt-induced potential provides a restoring force which induces oscillations when it exceeds the strength of damping; this critical tilt angle is predicted within 20\%, and the prediction is consistent with measured oscillation frequencies. These observations show that a self-consistent low-dimensional model can be extended to include the dynamics of $\theta_0$ due to tilt. However, the underprediction of the effect of tilt on $\theta_0$ warrants revisiting the predicted magnitude., Comment: 12 pages, 8 figures, submitted to Physics of Fluids

We investigate dilation-induced surface deformations in a Discontinuous Shear Thickening (DST) suspension to determine the relationship between dilation and stresses in DST. Video is taken at two observation points on the surface of the suspension in a rheometer while shear and normal stresses are measured. A roughened surface of the suspension is observed as particles poke through the liquid-air interface, corresponding to dilation. Dilation events are found to be intermittent and localized spatially. Shear and normal stresses also fluctuate between high- and low-stress states, and dilation is observed frequently in the high stress state. On the other hand, a complete lack of dilation is observed when the stresses remain at low values for a several seconds. Dilation is most prominent while the stresses grow from the low-stress state to the high-stress state, and the dilated region tends to span the entire surface by the end of the stress growth period. Dilation is found only at stresses and shear rates in and above the shear thickening range. These observed relations between surface dilation and stresses confirm that dilation and stresses are coupled in the high-stress state of DST., Comment: 14 pages, 9 figures

In 1982 John Hopfield published a neural network model for memory retrieval, a model that became a cornerstone in theoretical neuroscience. A key ingredient of the Hopfield model was the use of a network dynamics that is governed by a Lyapunov function. In a recent paper, Krotov and Hopfield showed how a Lyapunov function governs a biological plausible learning rule for the neural networks' connectivity. By doing so, they bring an intriguing approach to classification tasks, and show the relevance of the broader framework across decades in the field., Comment: Trends in Neurosciences (2020)

We present observations of oscillations in the shape of the temperature profile of the large-scale circulation (LSC) of turbulent Rayleigh-B{\'e}nard convection. Temperature measurements are broken down into Fourier moments as a function of $\theta-\theta_0$, where $\theta$ is the azimuthal angle in a horizontal plane at mid-height, and $\theta_0$ is the LSC orientation. The oscillation structure is dominated by a 3rd order sine moment and 3rd order cosine moment in a cubic cell. In contrast, these moments are not found to oscillate in a cylindrical cell. This geometry-dependent behavior can be explained by a model that assumes that the heat transported by the LSC is conducted from the thermal boundary layers, and is proportional to pathlength of the LSC along boundary layers at the top and bottom plates. In a non-circular cross-section cell, oscillations of the LSC orientation $\theta_0$ result in an oscillation in the container shape in the reference frame of the LSC, resulting in an oscillation in the pathlength of the LSC at a given $\theta-\theta_0$. In a square-cross-section cell, this model predicts the dominant 3rd order sine moment and 3rd order cosine moment with magnitudes within 50\% of measured values, when using the amplitude of the oscillation of $\theta_0$ as input. A cylindrical cell is special in that the pathlength is independent of $\theta_0$, and so these oscillating moments are not induced. In a cylindrical cell, the model reproduces the sinusoidal mean temperature profile with a sloshing oscillation dominated by the 2nd order sine moment, consistent with previous observations in that geometry., Comment: 13 figures, 18 pages

We compute the firing rate of a leaky integrate-and-fire (LIF) neuron with stochastic conductance-based inputs in the limit when synaptic decay times are much shorter than the membrane time constant. A comparison of our analytical results to numeric simulations is presented for a range of biophysically-realistic parameters., Comment: 5 pages, 2 figures

We test the ability of a low-dimensional turbulence model to predict how dynamics of large-scale coherent structures such as convection rolls change in different cell geometries. We performed Rayleigh-B\'enard convection experiments in a cubic container, in which there is a single convection roll known as the large-scale circulation (LSC). The model describes the motion of the orientation $\theta_0$ of the LSC as diffusion in a potential which is predicted as a function of the shape of the cell from an approximate solutions of the Navier-Stokes equations. The model predicts advected oscillation modes, driven by a restoring force created by the non-circular cell cross-section. We observe the predicted lowest-wavenumber mode in which the LSC orientation $\theta_0$ oscillates around a corner, and a slosh angle $\alpha$ rocks back and forth, which is distinct from the higher-wavenumber advected twisting and sloshing oscillations found in cylindrical cells. The potential has quadratic minima near each corner with the same curvature in both the LSC orientation $\theta_0$ and slosh angle $\alpha$, as predicted. The new oscillation mode around corners is found above a critical Ra $=4\times10^8$, which appears in the model as a crossing of an underdamped-overdamped transition. The natural frequency of the potential, oscillation period, power spectrum, and critical Ra for oscillations are all within a factor of 3 of model predictions for the Rayleigh number range $8\times10^7 \le Ra \le 3\times 10^9$. However, these uncertainties in model parameters are too large to correctly predict whether the system is in the underdamped or overdamped state at a given Ra. The success of the model at predicting the potential and flow modes for a cubic cell suggests that such a modeling approach could be applied more generally to different cell geometries that support a single convection roll., Comment: 38 pages, 27 figures