Search

Adults often learn new semantic information in face-to-face communication with other adults (e.g., teachers, colleagues). More knowledgeable individuals provide an ensemble of multimodal behaviours that can shape the information that their interlocutors learn. Using the naturalistic ECOLANG corpus of dyadic conversations, we ask whether multimodal behaviours (pitch, speaking rate, representational gestures, points, object manipulations, and gaze) support adults' semantic learning of unknown objects above and beyond verbal properties of utterances (number of utterances, lexical diversity, mean length of utterances, concreteness) and learners' individual differences (vocabulary, working memory). We found that individual differences, pointing and object manipulations affected learning, with verbal and multimodal factors also interacting to predict adult semantic learning. Our results highlight the relevance of accounts of multimodal learning in adulthood and the importance of considering naturalistic interaction in its complexity to understand the factors that influence adult learning.

Alongside the linguistic input, young children leverage multimodal cues (e.g., prosody, gestures) to learn novel words in face-to-face interactions. It is unclear whether multimodal cues play a similar role in adults. Here, we used ECOLANG, a corpus of semi-naturalistic dyadic conversations where English-speaking adults incidentally learned about unknown objects and their names by interacting with a partner who knew those objects. We examined whether multimodal cues (prosodic, indexical, and iconic) predicted learners' ability to learn the objects' names, above and beyond individual differences and linguistic predictors. We found that the number of repetitions of the label predicted word learning. Additionally, learners with lower working memory abilities benefited from speakers producing representational gestures while labelling the unknown objects. We discuss implications for theories of word learning and approaches of situated cognition.

Representational gestures are co-speech gestures that carry semantic content related to the content of speech. Previous studies focusing on adult-adult conversation have investigated the temporal alignment of gestures and speech finding that the overwhelming majority of representational gestures are produced right before the lexical content they refer to (their lexical affiliate, LA). However, nothing is yet known about whether caregivers would also time their gestures in the same way in naturalistic interactions. We annotated representational gestures from a large corpus (ECOLANG) of semi-naturalistic conversations between caregivers and their 3-4 year old children (n = 899 gestures from n=36 caregivers). We found that, just as in adult-directed language (ADL), representational gestures in child-directed language (CDL) were more tightly linked to the onset of LAs than the onset of the utterance in which LAs were produced (hence planned when full events are encoded); with the overall majority of the representational gestures starting before their LAs. We further found that age of acquisition (AoA) rating of the LA had a significant effect on the speech-gesture latency. We found that for words acquired earlier, the gesture’s stroke (the meaningful part of a gesture) tended to be produced before the LA’s onset; for the later acquired word, the stroke tended to be produced at the same time or after the onset of the LA in speech. Our findings suggest that: (1) Regardless of their addressee, speakers always time the production of representational gestures to specific conceptual/linguistic units, rather than the full event/utterance. (2) In contrast to ADL, caregivers’ gestures may support addresses’ linguistic processing not only by supporting word prediction (of likely better-known words), but also by supporting the learning of conceptual features (of likely less well-known words).

Studies have shown the importance of caregivers’ multimodal behaviours (e.g., prosody, gestures, gaze) on children’s word learning. However, most studies focus on only one specific behaviour (e.g., only prosody). Here, we investigate which multimodal behaviours used by caregivers best predict children’s word learning and vocabulary growth. Using data from the ECOLANG corpus, we analysed caregiver behaviour in semi-naturalistic interactions with their child (3 to 4 years old) in which they talked about known and unknown toys. We analysed caregivers’ (n=36) use of multimodal cues while labelling the objects, specifically their use of yes/no questions, pitch, representational gestures, pointing, object manipulations and gaze. Caregivers’ pitch, use of yes/no questions and pointing predicted children’s word learning. In particular, higher pitch when labelling unknown toys predicted immediate word learning. The degree to which caregivers used higher pitch when producing the label for known compared to unknown toys predicted both immediate learning and vocabulary growth. Furthermore, the degree to which caregivers used yes/no questions more for unknown toys predicted immediate learning, while the frequency of yes/no questions when naming unknown toys predicted vocabulary growth. Lastly, caregiver pointing also predicted immediate label learning and vocabulary growth, but in the opposite direction from prosody: the more they pointed towards known toys, the better children’s learning of novel toy labels. Other behaviours did not predict word learning. Overall, these results provide evidence for the important role of multimodal caregiver behaviours, particularly prosody, on children’s lexical development.

Research about explanation processes is gaining relevance because of the increased popularity of artificial systems required to explain their function or outcome. Following an interactive approach, not only explainers but also explainees contribute to successful interactions. However, little is known about how explainees actively guide explanation processes and how their involvement relates to learning. We explored the occurrence and type of explainees’ questions in 20 adult–adult explanation dialogues about unknown present and absent objects. Crucially, we related the question types to the explainees’ subsequent recall of the unknown object labels. We found that explainees asked different types of questions, especially about the object’s label and facts. Questions about the object’s function were asked more when objects were present. In addition, requests for labeling were linked to better recall. The results contribute to designing explainable AI that aim to provide relevant explanations and to further experimental approaches to study explanations.

Theories suggest that representational gestures depicting properties of referents in accompanying speech could facilitate language production and comprehension. In order to shed light on how gesture and speech are coordinated during production, we investigate whether representational gestures are time-locked to the onset of utterances (hence planned when full events are encoded) or Lexical Affiliates (LAs; words most closely aligned with the gesture meaning; hence planned when individual concepts are encoded) in a large corpus of naturalistic conversation (n = 1803 gestures from n = 24 speakers). Our data shows that representational gestures are more tightly tied to LA onsets than utterance onsets, which is consistent with theories of multimodal communication in which gestures aid conceptual packaging or retrieval of individual concepts rather than events. We also demonstrate that in naturalistic speech, representational gestures tend to precede their LAs by around 370ms, which means that they could plausibly allow for an addressee to predict upcoming words (ter Bekke, Drijvers & Holler, 2021; Ferré, 2010; Habets et al., 2011).

Communication comprises a wealth of multimodal signals (e.g., gestures, eye gaze, intonation) in addition to speech and there is a growing interest in the study of multimodal language by psychologists, linguists, neuroscientists and computer scientists. The ECOLANG corpus provides audiovisual recordings and ELAN annotations of multimodal behaviours (speech transcription, gesture, object manipulation, and eye gaze) by British and American English-speaking adults engaged in semi-naturalistic conversation with their child (N = 38, children 3-4 years old, face-blurred) or a familiar adult (N = 31). Speakers were asked to talk about objects to their interlocutors. We further manipulated whether the objects were familiar or novel to the interlocutor and whether the objects could be seen and manipulated (present or absent) during the conversation. These conditions reflect common interaction scenarios in real-world communication. Thus, ECOLANG provides ecologically-valid data about the distribution and co-occurrence of multimodal signals across these conditions for cognitive scientists and neuroscientists interested in addressing questions concerning real-world language acquisition, production and comprehension, and for computer scientists to develop multimodal language models and more human-like artificial agents. [ABSTRACT FROM AUTHOR]

Most language use is displaced, referring to past, future, or hypothetical events, posing the challenge of how children learn what words refer to when the referent is not physically available. One possibility is that iconic cues that imagistically evoke properties of absent referents support learning when referents are displaced. In an audio‐visual corpus of caregiver–child dyads, English‐speaking caregivers interacted with their children (N = 71, 24–58 months) in contexts in which the objects talked about were either familiar or unfamiliar to the child, and either physically present or displaced. The analysis of the range of vocal, manual, and looking behaviors caregivers produced suggests that caregivers used iconic cues especially in displaced contexts and for unfamiliar objects, using other cues when objects were present. [ABSTRACT FROM AUTHOR]

Research about explanation processes is gaining relevance because of the increased popularity of artificial systems required to explain their function or outcome. Following an interactive approach, not only explainers, but also explainees contribute to successful interactions. However, little is known about how explainees actively guide explanation processes and how their involvement relates to learning. We explored the occurrence and type of explainees' questions in 20 adult — adult explanation dialogues about unknown present and absent objects. Crucially, we related the question types to the explainees' subsequent recall of the unknown object labels. We found that explainees asked different types of questions, especially about the object's label and facts. Questions about the object's function were asked more when objects were present. In addition, requests for labelling were linked to better recall. The results contribute to designing explainable AI that aims to provide relevant and adaptive explanations and to further experimental approaches to study explanations. [ABSTRACT FROM AUTHOR]

What aspects of infants' prelinguistic communication are most valuable for learning to speak, and why? We test whether early vocalizations and gestures drive the transition to word use because, in addition to indicating motoric readiness, they (a) are early instances of intentional communication and (b) elicit verbal responses from caregivers. In study 1, 11 month olds (N = 134) were observed to coordinate vocalizations and gestures with gaze to their caregiver's face at above chance rates, indicating that they are plausibly intentionally communicative. Study 2 tested whether those infant communicative acts that were gaze‐coordinated best predicted later expressive vocabulary. We report a novel procedure for predicting vocabulary via multi‐model inference over a comprehensive set of infant behaviours produced at 11 and 12 months (n = 58). This makes it possible to establish the relative predictive value of different behaviours that are hierarchically organized by level of granularity. Gaze‐coordinated vocalizations were the most valuable predictors of expressive vocabulary size up to 24 months. Study 3 established that caregivers were more likely to respond to gaze‐coordinated behaviours. Moreover, the dyadic combination of infant gaze‐coordinated vocalization and caregiver response was by far the best predictor of later vocabulary size. We conclude that practice with prelinguistic intentional communication facilitates the leap to symbol use. Learning is optimized when caregivers respond to intentional vocalizations with appropriate language. [ABSTRACT FROM AUTHOR]

Prosocial behavior, including instrumental helping, emerges early in development, but the role parental attitudes and practices take in shaping the emergence of early helping across different cultural contexts is not well understood. We took a longitudinal approach to investigate maternal socialization of early helping across two different cultural groups. Participants were mother-infant dyads from urban/suburban York, United Kingdom (43 infants: 21 females, 22 males) and the rural Masindi District, Uganda (39 infants: 22 females, 17 males). We examined cultural variations in mother's helping-related parenting practices toward 14- and 18-month-olds and infants' actual helping in experimental tasks at 18 months. We then asked whether maternal parenting practices and socialization goals predicted individual variation in infant helping. We found that U.K. mothers scaffolded infant helping using a larger range of strategies than Ugandan mothers, but expecting an infant to help was more common in Uganda than in the United Kingdom. Moreover, we found that the Ugandan infants were more likely and often quicker to help an adult in need than the U.K. infants. Finally, we found that maternal scaffolding behaviors positively predicted individual variation in infant helping at 18 months in the United Kingdom, but not in Uganda. By contrast, maternal alignment with relational socialization goals at 11 months positively predicted infant helping at 18 months in the Ugandan, but not in the U.K., sample. These results indicate that early instrumental helping behavior varies across societies and that maternal socialization goals and scaffolding behaviors can shape infant helping in culturally specific ways. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

In psychology, researchers often predict a dependent variable (DV) consisting of multiple measurements (e.g., scale items measuring a concept). To analyze the data, researchers typically aggregate (sum/average) scores across items and use this as a DV. Alternatively, they may define the DV as a common factor using structural equation modeling. However, both approaches neglect the possibility that an independent variable (IV) may have different relationships to individual items. This variance in individual item slopes arises because items are randomly sampled from an infinite pool of items reflecting the construct that the scale purports to measure. Here, we offer a mixed-effects model called random item slope regression, which accounts for both similarities and differences of individual item associations. Critically, we argue that random item slope regression poses an alternative measurement model to common factor models prevalent in psychology. Unlike these models, the proposed model supposes no latent constructs and instead assumes that individual items have direct causal relationships with the IV. Such operationalization is especially useful when researchers want to assess a broad construct with heterogeneous items. Using mathematical proof and simulation, we demonstrate that random item slopes cause inflation of Type I error when not accounted for, particularly when the sample size (number of participants) is large. In real-world data ( n = 564 participants) using commonly used surveys and two reaction time tasks, we demonstrate that random item slopes are present at problematic levels. We further demonstrate that common statistical indices are not sufficient to diagnose the presence of random item slopes. (PsycInfo Database Record (c) 2023 APA, all rights reserved).