Search

Objective Social determinants of health (SDOH) are nonclinical, socioeconomic conditions that influence patient health and quality of life. Identifying SDOH may help clinicians target interventions. However, SDOH are more frequently available in narrative notes compared to structured electronic health records. The 2022 n2c2 Track 2 competition released clinical notes annotated for SDOH to promote development of NLP systems for extracting SDOH. We developed a system addressing 3 limitations in state-of-the-art SDOH extraction: the inability to identify multiple SDOH events of the same type per sentence, overlapping SDOH attributes within text spans, and SDOH spanning multiple sentences. Materials and Methods We developed and evaluated a 2-stage architecture. In stage 1, we trained a BioClinical-BERT-based named entity recognition system to extract SDOH event triggers, that is, text spans indicating substance use, employment, or living status. In stage 2, we trained a multitask, multilabel NER to extract arguments (eg, alcohol “type”) for events extracted in stage 1. Evaluation was performed across 3 subtasks differing by provenance of training and validation data using precision, recall, and F1 scores. Results When trained and validated on data from the same site, we achieved 0.87 precision, 0.89 recall, and 0.88 F1. Across all subtasks, we ranked between second and fourth place in the competition and always within 0.02 F1 from first. Conclusions Our 2-stage, deep-learning-based NLP system effectively extracted SDOH events from clinical notes. This was achieved with a novel classification framework that leveraged simpler architectures compared to state-of-the-art systems. Improved SDOH extraction may help clinicians improve health outcomes.

We present a computational model capable of simulating aspects of human knowledge for thousands of real-world concepts. Our approach involves a pretrained transformer network that is further fine-tuned on large data sets of participant-generated feature norms. We show that such a model can successfully extrapolate from its training data, and predict human knowledge for new concepts and features. We apply our model to stimuli from 25 previous experiments in semantic cognition research and show that it reproduces many findings on semantic verification, concept typicality, feature distribution, and semantic similarity. We also compare our model against several variants, and by doing so, establish the model properties that are necessary for good prediction. The success of our approach shows how a combination of language data and (laboratory-based) psychological data can be used to build models with rich world knowledge. Such models can be used in the service of new psychological applications, such as the modeling of naturalistic semantic verification and knowledge retrieval, as well as the modeling of real-world categorization, decision-making, and reasoning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

People make judgments about thousands of different objects and concepts on a day-to-day basis; however, capturing the knowledge that subserves these judgments has been difficult. Recent advances in computational linguistics are filling this gap, as the statistics of language use yield rich, distributed semantic representations for natural objects and concepts. These representations have been shown to predict semantic and linguistic judgments, such as judgments of meaning and relatedness, and more recently, high-level judgments, including probability judgment and forecasting, stereotyping and various types of social judgment, consumer choice, and perceptions of risk. Distributed semantic representations are now a key component of computational models that represent knowledge, make evaluations and attributions, and give responses, in a human-like manner.

The prediction of everyday human behavior is a central goal in the behavioral sciences. However, efforts in this direction have been limited, as (1) the behaviors studied in most surveys and experiments represent only a small fraction of all possible behaviors, and (2) it has been difficult to generalize data from existing studies to predict arbitrary behaviors, owing to the difficulty in adequately representing such behaviors. Our paper addresses each of these problems. First, by sampling frequent verb phrases in natural language and refining these through human coding, we compile a dataset of nearly 4,000 common human behaviors. Second, we use distributed semantic models to obtain vector representations for our behaviors, and combine these with demographic and psychographic data, to build supervised, deep neural network models of behavioral propensities for a representative sample of the US population. Our best models achieve high accuracy rates when predicting propensities for novel (out-of-sample) participants as well as novel behaviors. This work lays the foundation for new predictive theories of everyday behavior, improving the generality and naturalism of research in the behavioral sciences.

Every day, people make countless judgments and decisions, from evaluating the tastiness of a food, to predicting whether an earthquake is likely to strike their state. In order to formally study such judgments and decisions we need: (a) a theory of our knowledge of the natural entities we make judgments and decisions about, and (b) a theory of how that knowledge is used to make judgments and decisions. In this chapter, we review Distributed Semantic (DS) representations which deliver dense, cheap representations for millions of words and concepts, and thus offer theories of (a). We first introduce technical details of DS models, and then review applications to behavior, including semantic judgment, associative judgment, and high-level judgment. We suggest that the promise of existing DS representations has yet to be fully realized, as they are often not combined with cognitively plausible process models, i.e. theories of (b). We suggest some possibilities along these lines, as well as some ongoing and potential future judgment and decision making applications involving very recent DS models that offer distributed representations for phrases and sentences.

New languages emerge through interactions among people, yet the role of social network structure in language emergence is not clear, despite research from experimental semiotics, observational fieldwork, and computational modeling. To better understand the effects of social network structure on the formation of conventional referring expressions, we use a silent gesture paradigm that combines the methodological control of experimental semiotics and computational simulations with the naturalistic affordances of the human body, physical environment, and interpersonal communication. We elicited gestural referring expressions from hearing participants randomly assigned to either a richly- or sparsely-connected communicative network. Results demonstrate greater conventionalization among participants in the richly-connected condition, although this effect disappears after accounting for between-condition differences in overall number of communicative interactions. These results provide the first experimental demonstration that communicative network structure causally impacts the conventionalization of referring expressions in human participants, using a communicative modality in which human language naturally arises.

Similarity is one of the most important relations humans perceive, arguably subserving category learning and categorization, generalization and discrimination, judgment and decision making, and other cognitive functions. Researchers have proposed a wide range of representations and processes that could be at play in similarity judgment, yet have not comprehensively compared the power of these representations and processes for predicting similarity within and across different semantic categories. We performed such a comparison by pairing eight prominent vector semantic representations with seven established similarity metrics that could operate on these representations, as well as supervised methods for dimensional weighting in the similarity function. This approach yields a factorial model structure with 56 distinct representation-process pairs, which we tested on a novel dataset of similarity judgments between pairs of co-hyponymic words in eight categories. We found that cosine similarity and Pearson correlation were the overall best performing unweighted similarity functions, and that word vectors derived from free association norms often outperformed word vectors derived from text (including those specialized for similarity). Importantly, models that used human similarity judgments to learn category-specific weights on dimensions yielded substantially better predictions than all unweighted approaches across all types of similarity functions and representations, although dimension weights did not generalize well across semantic categories, suggesting strong category context effects in similarity judgment. We discuss implications of these results for cognitive modeling and natural language processing, as well as for theories of representations and processes involved in similarity.

Social determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes.A retrospective cohort study.Data were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes.We developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the "Social Work" category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures - convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) - for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC).All three DNN models accurately classified all SDOH categories (minimum micro-F1 = 0.632, minimum macro-AUC = 0.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC = 0.907). The BERT model most effectively identified the "occupational" category (F1 = 0.774, AUC = 0.965) and least effectively identified the "non-SDOH" category (F = 0.491, AUC = 0.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = 0.87 vs. cTAKES F1 = 0.06), and outperformed logistic regression (micro-F1 = 0.649, macro-AUC = 0.696) and random forest (micro-F1 = 0.502, macro-AUC = 0.523) trained on bag-of-words.Our study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.

Information stored in memory influences the formation of preferences and beliefs in most everyday decision tasks. The richness of this information, and the complexity inherent in interacting memory and decision processes, makes the quantitative model-driven analysis of such decisions very difficult. In this article we present a general framework that can address the theoretical and methodological barriers to building formal models of naturalistic memory-based decision making. Our framework implements established theories of memory search and decision making within a single integrated cognitive system, and uses computational language models to quantify the thoughts over which memory and decision processes operate. It can thus describe both the content of the information that is sampled from memory, as well as the processes involved in retrieving and evaluating this information in order to make a decision. Furthermore, our framework is tractable, and the parameters that characterize memory-based decisions can be recovered using thought listing and choice data from existing experimental tasks, and in turn be used to make quantitative predictions regarding choice probability, length of deliberation, retrieved thoughts, and the effects of decision context. We showcase the power and generality of our framework by applying it to naturalistic binary choices from domains such as risk perception, consumer behavior, financial decision making, ethical decision making, legal decision making, food choice, and social judgment. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Psychologists collect similarity data to study a variety of phenomena including categorization, generalization and discrimination, and representation itself. However, collecting similarity judgments between all pairs of items in a set is expensive, spurring development of techniques like the Spatial Arrangement Method (SpAM; Goldstone, Behavior Research Methods, Instruments, & Computers, 26, 381-386, 1994), wherein participants place items on a two-dimensional plane such that proximity reflects perceived similarity. While SpAM greatly hastens similarity measurement, and has been successfully used for lower-dimensional, perceptual stimuli, its suitability for higher-dimensional, conceptual stimuli is less understood. In study 1, we evaluated the ability of SpAM to capture the semantic structure of eight different categories composed of 20-30 words each. First, SpAM distances correlated strongly (r = .71) with pairwise similarity judgments, although below SpAM and pairwise judgment split-half reliabilities (r's > .9). Second, a cross-validation exercise with multidimensional scaling fits at increasing latent dimensionalities suggested that aggregated SpAM data favored higher (> 2) dimensional solutions for seven of the eight categories explored here. Third, split-half reliability of SpAM dissimilarities was high (Pearson r = .90), while the average correlation between pairs of participants was low (r = .15), suggesting that when different participants focus on different pairs of stimulus dimensions, reliable high-dimensional aggregate similarity data is recoverable. In study 2, we show that SpAM can recover the Big Five factor space of personality trait adjectives, and that cross-validation favors a four- or five-dimension solution on this dataset. We conclude that SpAM is an accurate and reliable method of measuring similarity for high-dimensional items like words. We publicly release our data for researchers.

Recent advances in machine learning, combined with the increased availability of large natural language datasets, have made it possible to uncover semantic representations that characterize what people know about and associate with a wide range of objects and concepts. In this paper, we examine the power of word embeddings, a popular approach for uncovering semantic representations, for studying high-level human judgment. Word embeddings are typically applied to linguistic and semantic tasks, however we show that word embeddings can be used to predict complex theoretically- and practically-relevant human perceptions and evaluations in domains as diverse as social cognition, health behavior, risk perception, organizational behavior, and marketing. By learning mappings from word embeddings directly onto judgment ratings, we outperform a similarity-based baseline as well as common metrics of human inter-rater reliability. Word embeddings are also able to identify the concepts that are most associated with observed perceptions and evaluations, and can thus shed light on the psychological substrates of judgment. Overall, we provide new methods and insights for predicting and understanding high-level human judgment, with important applications across the social and behavioral sciences.

It is largely acknowledged that natural languages emerge from not just human brains, but also from rich communities of interacting human brains (Senghas, 2005). Yet the precise role of such communities and such interaction in the emergence of core properties of language has largely gone uninvestigated in naturally emerging systems, leaving the few existing computational investigations of this issue at an artificial setting. Here we take a step towards investigating the precise role of community structure in the emergence of linguistic conventions with both naturalistic empirical data and computational modeling. We first show conventionalization of lexicons in two different classes of naturally emerging signed systems: (1) protolinguistic “homesigns” invented by linguistically isolated Deaf individuals, and (2) a natural sign language emerging in a recently formed rich Deaf community. We find that the latter conventionalized faster than the former. Second, we model conventionalization as a population of interacting individuals who adjust their probability of sign use in response to other individuals' actual sign use, following an independently motivated model of language learning (Yang 2002, 2004). Simulations suggest that a richer social network, like that of natural (signed) languages, conventionalizes faster than a sparser social network, like that of homesign systems. We discuss our behavioral and computational results in light of other work on language emergence, and other work of behavior on complex networks.

Why do languages have the words they have, and not some other set of words? While certainly there is some arbitrariness in the lexicon (English ‘frog’ vs. Spanish ‘rana’), there is just as surely some systematicity or functionality in it as well. What exactly might the nature of this systematicity or functionality be? For example, might the lexicon be efficiently adapted for communication, learning, memory storage, retrieval, or other cognitive functions? This paper critically reviews evidence that natural language lexicons efficiently carve up semantic fields (e.g., color, space, kinship) and have phonological forms that are similarly efficient when the aggregate lexicon is considered. The paper also suggests additional ways functionalism in lexicons might be assessed, and speculates on how functional lexicons may have arisen.

The last decade has seen an explosion of studies examining language emergence experimentally (a literature known as Experimental Semiotics). While revealing, most studies in this domain typically have a complex set of constraints on behavior which make formal analysis of the results challenging. Wishing to take a dynamical systems approach in which we could comprehensively analyze the typology of behavioral outcomes, we devised a coordination task based on Roberts and Goldstone (2011)’s number summing game and studied the process by which the groups arrived at a coordinating scheme. While we have not yet seen evidence for traditional hallmarks of natural language in our game, we believe two features of the results offer a helpful perspective on language evolution: (i) the variation across groups is organized around equivalence classes of strategies (egalitarian points) that stem from the combination of the task structure with the physical capabilities of the organisms, suggesting a dynamicalsystems-based framework for understanding parametric variation in natural languages different from conceptions of Universal Grammar as an information structure and (ii) individual behavior appears to be a sum of impulses that generally approximate but also tend to resist the egalitarian points, suggesting that construing linguistic structure as self-organized, rather than as biologically specified principles and parameters may help address language emergence.