Is a Picture Worth 51 Million Words? A Text Analysis of Public User Reviews of Schools. Technical Report

Massive online user review platforms, with their star ratings and text reviews, are reshaping the information available for consumer and public service decisions. We study the leading K-12 schooling platform, GreatSchools, applying machine learning (Natural Language Processing, NLP) to 600,000 reviews that encompass the vast majority of the nation's traditional public, charter, and private schools (84,009 schools in total), supplemented with qualitative analysis of a subsample of reviews. Encompassing more than fifty million words of text, our initial analysis pre-specified eight broad topics and 27 sub-topics and coded review words into these categories. We find that parents write the vast majority of reviews and tend to write more about School Staff and School Culture than students. More generally, text reviews vary in important ways across user types (parents, students, teachers, principals), school sector (traditional public, charter, and private schools), grade level, and demographics. The partial correlation between topics and star ratings also differs across user types and sectors. Taken together, these results suggest that user reviews are less useful than they appear and less useful than with other kinds of products. Our analysis points to design features that might improve their usefulness. The variation in content and value of the reviews also has methodological implications as it shows how NLP can complement qualitative research methods with such large volumes of text.