Search

Establishing a unified theory of cognition has been a major goal of psychology. While there have been previous attempts to instantiate such theories by building computational models, we currently do not have one model that captures the human mind in its entirety. Here we introduce Centaur, a computational model that can predict and simulate human behavior in any experiment expressible in natural language. We derived Centaur by finetuning a state-of-the-art language model on a novel, large-scale data set called Psych-101. Psych-101 reaches an unprecedented scale, covering trial-by-trial data from over 60,000 participants performing over 10,000,000 choices in 160 experiments. Centaur not only captures the behavior of held-out participants better than existing cognitive models, but also generalizes to new cover stories, structural task modifications, and entirely new domains. Furthermore, we find that the model's internal representations become more aligned with human neural activity after finetuning. Taken together, Centaur is the first real candidate for a unified model of human cognition. We anticipate that it will have a disruptive impact on the cognitive sciences, challenging the existing paradigm for developing computational models.

The Bible, a product of an extensive and intricate process of oral-written transmission spanning centuries, obscures the contours of its earlier recensions. Debate rages over determining the existing layers and identifying the date of composition and historical background of the biblical texts. Traditional manual methodologies have grappled with authorship challenges through scrupulous textual criticism, employing linguistic, stylistic, inner-biblical, and historical criteria. Despite recent progress in computer-assisted analysis, many patterns still need to be uncovered in Biblical Texts. In this study, we address the question of authorship of biblical texts by employing statistical analysis to the frequency of words using a method that is particularly sensitive to deviations in frequencies associated with a few words out of potentially many. We aim to differentiate between three distinct authors across numerous chapters spanning the first nine books of the Bible. In particular, we examine 50 chapters labeled according to biblical exegesis considerations into three corpora (D, DtrH, and P). Without prior assumptions about author identity, our approach leverages subtle differences in word frequencies to distinguish among the three corpora and identify author-dependent linguistic properties. Our analysis indicates that the first two authors (D and DtrH) are much more closely related compared to P, a fact that aligns with expert assessments. Additionally, we attain high accuracy in attributing authorship by evaluating the similarity of each chapter with the reference corpora. This study sheds new light on the authorship of biblical texts by providing interpretable, statistically significant evidence that there are different linguistic characteristics of biblical authors and that these differences can be identified.

We present a procedure based on higher criticism (Dohono \& Jin 2004) to address the sparse multi-sensor quickest change-point detection problem. Namely, we aim to detect a change in the distribution of the multi-sensor that might affect a few sensors out of potentially many, while those affected sensors, if they exist, are unknown to us in advance. Our procedure involves testing for a change point in individual sensors and combining multiple tests using higher criticism. As a by-product, our procedure also indicates a set of sensors suspected to be affected by the change. We demonstrate the effectiveness of our method compared to other procedures using extensive numerical evaluations. We analyze our procedure under a theoretical framework involving normal data sensors that might experience a change in both mean and variance. We consider individual tests based on the likelihood ratio or the generalized likelihood ratio statistics and show that our procedure attains the information-theoretic limits of detection. These limits coincide with existing litereature when the change is only in the mean., Comment: Authors are listed in alphabetical order

Large Language Models (LLMs) vary in their abilities on a range of tasks. Initiatives such as the Open LLM Leaderboard aim to quantify these differences with several large benchmarks (sets of test items to which an LLM can respond either correctly or incorrectly). However, high correlations within and between benchmark scores suggest that (1) there exists a small set of common underlying abilities that these benchmarks measure, and (2) items tap into redundant information and the benchmarks may thus be considerably compressed. We use data from n > 5000 LLMs to identify the most informative items of six benchmarks, ARC, GSM8K, HellaSwag, MMLU, TruthfulQA and WinoGrande (with d = 28,632 items in total). From them we distill a sparse benchmark, metabench, that has less than 3% of the original size of all six benchmarks combined. This new sparse benchmark goes beyond point scores by yielding estimators of the underlying benchmark-specific abilities. We show that these estimators (1) can be used to reconstruct each original individual benchmark score with, on average, 1.24% root mean square error (RMSE), (2) reconstruct the original total score with 0.58% RMSE, and (3) have a single underlying common factor whose Spearman correlation with the total score is r = 0.94., Comment: accepted for publication at ICLR 2025

Controlling zero bias anomalies in magnetic atoms provides a promising strategy to engineer tunable quantum many-body excitations. Here we show how two different quantum impurities featuring spinaron and Kondo excitations can be controlled via quantum confinement engineering by using circular quantum corrals on a Ag(111) surface. In corrals built from both Ag and Co adatoms, the width of the zero bias anomaly in the central Co adatom oscillates as a function of corral radius with a period of half of the Ag(111) surface state wavelength. Parameters extracted for Co/Ag(111) show only small differences in extracted spinaron zero-bias anomaly between corral walls built from Ag or Co adatoms. In quantum corrals occupied with metal-free phthalocyanine, a $S=1/2$ Kondo system, we observe notable changes in the zero bias anomaly lineshape as a function of corral radius. Our results offer insight into many-body Kondo and spinaron resonances where the electronic density is controlled by confinement engineering.

We propose a new method to compare survival data based on Higher Criticism (HC) of P-values obtained from many exact hypergeometric tests. The method can accommodate censorship and is sensitive to moderate differences in some unknown and relatively few time intervals, attaining much better power against such differences than the log-rank test and other tests that are popular under non-proportional hazard alternatives. We demonstrate the usefulness of the HC-based test in detecting rare differences compared to existing tests using simulated data and using actual gene expression data. Additionally, we analyze the asymptotic power of our method under a piece-wise homogeneous exponential decay model with rare and weak departures, describing two groups experiencing failure rates that are usually identical over time except in a few unknown instances in which the second group's failure rate is higher. Under an asymptotic calibration of the model's parameters, the HC-based test's power experiences a phase transition across the plane involving the rarity and intensity parameters that mirrors the phase transition in a two-sample rare and weak normal means setting. In particular, the phase transition curve of our test indicates a larger region in which it is fully powered than the corresponding region of the log-rank test. %The latter attains a phase transition curve that is analogous to a test based on Fisher's combination statistic of the hypergeometric P-values. %To our knowledge, this is the first analysis of a rare and weak signal detection model that involves individually dependent effects in a non-Gaussian setting., Comment: Part of this work was presented at JSM 2023

We propose an unsupervised method to extract keywords and keyphrases from texts based on a pre-trained language model (LM) and Shannon's information maximization. Specifically, our method extracts phrases having the highest conditional entropy under the LM. The resulting set of keyphrases turns out to solve a relevant information-theoretic problem: if provided as side information, it leads to the expected minimal binary code length in compressing the text using the LM and an entropy encoder. Alternately, the resulting set is an approximation via a causal LM to the set of phrases that minimize the entropy of the text when conditioned upon it. Empirically, the method provides results comparable to the most commonly used methods in various keyphrase extraction benchmark challenges.

We propose a method to determine whether a given article was written entirely by a generative language model or perhaps contains edits by a different author, possibly a human. Our process involves multiple tests for the origin of individual sentences or other pieces of text and combining these tests using a method that is sensitive to rare alternatives, i.e., non-null effects are few and scattered across the text in unknown locations. Interestingly, this method also identifies pieces of text suspected to contain edits. We demonstrate the effectiveness of the method in detecting edits through extensive evaluations using real data and provide an information-theoretic analysis of the factors affecting its success. In particular, we discuss optimality properties under a theoretical framework for text editing saying that sentences are generated mainly by the language model, except perhaps for a few sentences that might have originated via a different mechanism. Our analysis raises several interesting research questions at the intersection of information theory and data science., Comment: Accepted for publication in Harvard Data Science Review: https://hdsr.mitpress.mit.edu/pub/f90vid3h/release/1?readingCollection=5531bd2e

We study the problem of testing the goodness of fit of occurrences of items from many categories to an identical Poisson distribution over the categories. As a class of alternative hypotheses, we consider the removal of an $\ell_p$ ball, $p \leq 2$, of radius $\epsilon$ from a hypercube around the sequence of uniform Poisson rates. When the expected number of samples $n$ and number of categories $N$ go to infinity while $\epsilon$ is small, the minimax risk asymptotes to $2\Phi(-n N^{2-2/p} \epsilon^2/\sqrt{8N})$; $\Phi(x)$ is the normal CDF. This result allows the comparison of the many estimators previously proposed for this problem at the constant level, rather than at the rate of convergence of the risk or the scaling order of the sample complexity. The minimax test relies exclusively on collisions in the small sample limit but behaves like the chisquared test otherwise. Empirical studies over a range of problem parameters show that the asymptotic risk estimate is accurate in finite samples and that the minimax test is significantly better than the chisquared test or a test that only uses collisions. Our analysis combines standard ideas from non-parametric hypothesis testing with new results in the low count limit of multiple Poisson distributions, including the convexity of certain kernels and a central limit theorem of linear test statistics.

Regression calibration is a popular approach for correcting biases in estimated regression parameters when exposure variables are measured with error. This approach involves building a calibration equation to estimate the value of the unknown true exposure given the error-prone measurement and other confounding covariates. The estimated, or calibrated, exposure is then substituted for the true exposure in the health outcome regression model. When used properly, regression calibration can greatly reduce the bias induced by exposure measurement error. Here, we first provide an overview of the statistical framework for regression calibration, specifically discussing how a special type of error, called Berkson error, arises in the estimated exposure. We then present practical issues to consider when applying regression calibration, including: (1) how to develop the calibration equation and which covariates to include; (2) valid ways to calculate standard errors (SE) of estimated regression coefficients; and (3) problems arising if one of the covariates in the calibration model is a mediator of the relationship between the exposure and outcome. Throughout the paper, we provide illustrative examples using data from the Hispanic Community Health Study/Study of Latinos (HCHS/SOL) and simulations. We conclude with recommendations for how to perform regression calibration.

Abundant in coastal areas, sea turtles are affected by high-intensity acoustic anthropogenic sounds. In this paper, we offer a pattern analysis-based detection approach to serve as a warning system for the existence of nearby sea turtles. We focus on the challenge of overcoming the low signal-to-clutter ratio (SCR) caused by reverberations. Assuming that, due to low SCR, target reflections within the point cloud are received in groups, our detector searches for patterns through clustering to identify possible 'blobs' in the point cloud of reflections, and to classify them as either clutter or a target. Our unsupervised clustering is based on geometrical and spectral constraints over the blob's member relations. In turn, the classification of identified blobs as either a target or clutter is based on features extracted from the reflection pattern. To this end, assuming reflections from a sea turtle are stable but include spectral diversity due to distortions within the turtles body, we quantify the stability of the blob's members and their spectral entropy. We test our detector in both modeled simulations, and at sea, for the detection of sea turtles released after rehabilitation. The results show robustness to highly-fluctuating target intensity and ability to detect at low SCR.

Atomic-scale manipulation in scanning tunneling microscopy has enabled the creation of quantum states of matter based on artificial structures and extreme miniaturization of computational circuitry based on individual atoms. The ability to autonomously arrange atomic structures with precision will enable the scaling up of nanoscale fabrication and expand the range of artificial structures hosting exotic quantum states. However, the \textit{a priori} unknown manipulation parameters, the possibility of spontaneous tip apex changes, and the difficulty of modeling tip-atom interactions make it challenging to select manipulation parameters that can achieve atomic precision throughout extended operations. Here we use deep reinforcement learning (DRL) to control the real-world atom manipulation process. Several state-of-the-art reinforcement learning techniques are used jointly to boost data efficiency. The reinforcement learning agent learns to manipulate Ag adatoms on Ag(111) surfaces with optimal precision and is integrated with path planning algorithms to complete an autonomous atomic assembly system. The results demonstrate that state-of-the-art deep reinforcement learning can offer effective solutions to real-world challenges in nanofabrication and powerful approaches to increasingly complex scientific experiments at the atomic scale.

Neuroscience has recently made much progress, expanding the complexity of both neural-activity measurements and brain-computational models. However, we lack robust methods for connecting theory and experiment by evaluating our new big models with our new big data. Here, we introduce new inference methods enabling researchers to evaluate and compare models based on the accuracy of their predictions of representational geometries: A good model should accurately predict the distances among the neural population representations (e.g. of a set of stimuli). Our inference methods combine novel 2-factor extensions of crossvalidation (to prevent overfitting to either subjects or conditions from inflating our estimates of model accuracy) and bootstrapping (to enable inferential model comparison with simultaneous generalization to both new subjects and new conditions). We validate the inference methods on data where the ground-truth model is known, by simulating data with deep neural networks and by resampling of calcium imaging and functional MRI data. Results demonstrate that the methods are valid and conclusions generalize correctly. These data analysis methods are available in an open-source Python toolbox (rsatoolbox.readthedocs.io)., Comment: revision submitted to Elife, 40 pages 9 figures