Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study

Current tools for objectively measuring young children’s observed behaviors are expensive, time-consuming, and require extensive training and professional administration. The lack of scalable, reliable, and validated tools impacts access to evidence-based knowledge and limits our capacity to collect population-level data in non-clinical settings. To address this gap, we developed mobile technology to collect videos of young children while they watched movies designed to elicit autism-related behaviors and then used automatic behavioral coding of these videos to quantify children’s emotions and behaviors. We present results from our iPhone study Autism & Beyond, built on ResearchKit’s open-source platform. The entire study—from an e-Consent process to stimuli presentation and data collection—was conducted within an iPhone-based app available in the Apple Store. Over 1 year, 1756 families with children aged 12–72 months old participated in the study, completing 5618 caregiver-reported surveys and uploading 4441 videos recorded in the child’s natural settings. Usable data were collected on 87.6% of the uploaded videos. Automatic coding identified significant differences in emotion and attention by age, sex, and autism risk status. This study demonstrates the acceptability of an app-based tool to caregivers, their willingness to upload videos of their children, the feasibility of caregiver-collected data in the home, and the application of automatic behavioral encoding to quantify emotions and attention variables that are clinically meaningful and may be refined to screen children for autism and developmental disorders outside of clinical settings. This technology has the potential to transform how we screen and monitor children’s development.
Mobile technologies: revolutionizing behavioral assessments of young children A phone-based app that assesses the behavior of young children in their homes can be used to determine their risk of autism. Autism is the most common neurodevelopmental disorder in the US. Although some of the signs of autism can be identified within the first months of life, many children wait years to be diagnosed. A study led by Richard Bloomfield, Geraldine Dawson, Helen Egger, and Guillermo Sapiro, Duke University, examined caregiver-collected smartphone videos that quantify the emotion and attention of children aged between 1 and 6 years watching movies known to elicit autism-related behaviors. Automatic coding identified significant differences in emotion and attention by age, sex, and autism risk status. These findings highlight the feasibility of using this approach to identify autism symptoms and, potentially, those of other behavioral disorders.