Search

The present study examined the extent to which adaptive feedback and just-in-time writing strategy instruction improved the quality of high school students' persuasive essays in the context of the Writing Pal (W-Pal). W-Pal is a technology-based writing tool that integrates automated writing evaluation into an intelligent tutoring system. Students wrote a pretest essay, engaged with W-Pal's adaptive instruction over the course of four training sessions, and then completed a posttest essay. For each training session, W-Pal differentiated strategy instruction for each student based on specific weaknesses in the initial training essays prior to providing the opportunity to revise. The results indicated that essay quality improved overall from pretest to posttest with respect to holistic quality, as well as several specific dimensions of essay quality, particularly for students with lower literacy skills. Moreover, students' scores on some of the training essays improved from the initial to revised version on the dimensions of essay quality that were targeted by instruction, whereas scores did not improve on the dimensions that were not targeted by instruction. Overall, the results suggest that W-Pal's adaptive strategy instruction can improve the quality of students' essays overall, as well as more specific dimensions of essay quality. [For the corresponding grantee submission, see ED619500.]

The constructed responses individuals generate while reading can provide insights into their coherence-building processes. The current study examined how the cohesion of constructed responses relates to performance on an integrated writing task. Participants (N = 95) completed a multiple document reading task wherein they were prompted to think aloud, self-explain, or evaluate the sources while reading and then write an integrated essay based on their reading. Natural Language Processing techniques were used to analyze the cohesion of the constructed responses at both within- and across-text levels. Both within- and across-text cohesion indices were positively related to essay quality; however, across-text cohesion indices exhibited stronger effects. Overall, this study provides evidence that the cohesion of constructed responses can serve as a proxy of the coherence of the mental representations that readers construct during multiple document processing. [This article was published in "Discourse Processes" (EJ1357567).]

Successful text comprehension requires readers to engage in a number of coherence-building processes. This study examined how analyzing the cohesion of students 'constructed responses can be used to evaluate these coherence-building processes and the extent to which they vary across readers' individual differences and across types of texts. We posed two primary research questions: (1) Can we predict individual differences in working memory and reading skill based on the cohesion of students' constructed responses to text?; and (2) Do the relations between individual differences and cohesion vary as a function of genre? Participants (n = 119) generated constructed responses while reading history and science texts and completed reading skill and working memory assessments. The current study leveraged natural language processing (NLP) techniques to analyze the cohesion of readers' constructed responses, using cohesion as a proxy for assessing the coherence of their mental representations of the texts. Cohesion was measured at the sentence, paragraph, and synonym levels. Machine learning models showed that linguistic indices related to cohesion were significant predictors of both working memory and reading skill. Additional quantitative and qualitative inspection revealed that the relations between individual differences and coherence-building processes varied depending on the text's genre. These findings indicate that the interaction between genre and individual differences may be used to model coherence-building processes during reading.

Analyzing constructed responses, such as think-alouds or self-explanations, can reveal valuable information about readers' comprehension strategies. The current study expands on the extant work by (1) investigating combinations and patterns of comprehension strategies that readers use and (2) examining the extent to which these patterns relate to individual differences and comprehension outcomes. We leveraged archival data from three datasets (n = 472) to examine how comprehension strategy use varied across datasets, texts, and populations (high school, undergraduate). Students' self-explanations were coded for strategy use and then further analyzed in terms of combinations and patterns of strategies. Our analyses revealed that almost all readers primarily engaged in paraphrasing and/or the combination of paraphrasing and bridging, with few instances of elaboration. Further, the combination of paraphrasing and bridging was the best predictor of performance on a comprehension test. In terms of patterns, switching between strategies was not correlated to reading comprehension and was negatively correlated with the combination of paraphrasing and bridging. Understanding which strategy combinations and patterns are optimal can be used to inform adaptive instruction and feedback that can aid in more individualized support for readers. [For the corresponding grantee submission, see ED630658.]

For decades, educators have relied on readability metrics that tend to oversimplify dimensions of text difficulty. This study examines the potential of applying advanced artificial intelligence methods to the educational problem of assessing text difficulty. The combination of hierarchical machine learning and natural language processing (NLP) is leveraged to predict the difficulty of practice texts used in a reading comprehension intelligent tutoring system, iSTART. Human raters estimated the text difficulty level of 262 texts across two text sets (Set A and Set B) in the iSTART library. NLP tools were used to identify linguistic features predictive of text difficulty and these indices were submitted to both flat and hierarchical machine learning algorithms. Results indicated that including NLP indices and machine learning increased accuracy by more than 10% as compared to classic readability metrics (e.g., Flesch-Kincaid Grade Level). Further, hierarchical outperformed non-hierarchical (flat) machine learning classification for Set B (72%) and the combined set A + B (65%), whereas the nonhierarchical approach performed slightly better than the hierarchical approach for Set A (79%). These findings demonstrate the importance of considering deeper features of language related to text difficulty as well as the potential utility of hierarchical machine learning approaches in the development of meaningful text difficulty classification. [This article was published in "International Journal of Artificial Intelligence in Education" (EJ1271922).]

Literacy is a critically important and contemporary issue for educators, scientists, and politicians. Efforts to overcome the challenges associated with illiteracy, and the subsequent development of literate societies, are closely related to those of poverty reduction and sustainable human development. In this paper, the authors examine literacy from the lens of text and discourse theorists who focus on the higher-order "comprehension" processes involved in literacy. Discourse processing models make the assumption that comprehension emerges from the construction of a mental model of the text, which relies on the reader generating inferences to connect ideas within the text and to what the reader already knows. The article provides a broad overview of the theoretical models that drive research on text comprehension and production, as well as how this research shapes literacy instruction and effective interventions. The authors focus on two interventions with proven success in improving deep comprehension and writing, iSTART and the Writing Pal. Increasing literacy across the world call for a greater focus on theory driven strategy interventions to be integrated within classrooms and community at large.

Few studies have explored how general skills in both reading and writing influence performance on integrated, source-based writing. The goal of the present study was to consider the relative contributions of reading and writing ability on multiple-document integrative reading and writing tasks. Students in the U.S. (n=94) completed two tasks in which they read text sets about a socioscientific issue, generated constructed responses while reading, and then composed integrated essays. They also completed individual difference measures (general knowledge, reading skill, reading strategy use) and wrote independent essays to assess their writing ability. Mixed effect models revealed that general knowledge and reading skills contributed to integrated essay performance, but that once general writing ability was entered into the model, it became the strongest predictor of integrated writing scores. These results suggest the need for deeper consideration of the role of writing skills in integrated reading and writing tasks.

The benefits of writing strategy feedback are well established. This study examined the extent to which adding spelling and grammar checkers support writing and revision in comparison to providing writing strategy feedback alone. High school students (n = 119) wrote and revised six persuasive essays in Writing Pal, an automated writing evaluation and tutoring system. All participants received automated strategy feedback after writing the first draft of their essays. Half of the participants were also given access to spelling and grammar checkers while writing. Spelling and grammar feedback on its own had no effect on the quality of students' first draft. Linear mixed effects models revealed improvements from initial draft to revision on most subscales. The addition of spelling and grammar feedback contributed small but significant gains after revision on five subscales (i.e., mechanics, word choice, voice, conclusion, and organization) but no other aspects of the students' essays. Qualitative exploration of exemplar students' revision moves revealed how students incorporated both strategy and spelling and grammar feedback into their revisions. Findings from this study demonstrate that strategy feedback with an opportunity to revise contributed to improved essay quality, but that spelling and grammar feedback provided modest, complementary benefits.

The goal of this study was to assess the relationships between computational approaches to analyzing constructed responses made during reading and individual differences in the foundational skills of reading in college readers. We also explored if these relationships were consistent across texts and samples collected at different institutions and texts. The study made use of archival data that involved college participants who produced typed constructed responses under thinking aloud instructions reading history and science texts. They also took assessments of vocabulary knowledge and proficiency in comprehension. The protocols were analyzed to assess two different ways to determine their cohesion. One approach involved assessing how readers established connections with themselves (i.e., to other constructed responses they produced). The other approach involved assessing connections between the constructed responses and the texts that were read. Additionally, the comparisons were made by assessing both lexical (i.e., word matching) and semantic (i.e., high dimensional semantic spaces) comparisons. The result showed that both approaches for analyzing cohesion and making the comparisons were correlated with vocabulary knowledge and comprehension proficiency. The implications of the results for theory and practice are discussed.

Summary writing is a useful instructional tool for learning. However, summary writing is a challenge to many students. This mixed-method study examined the potential of the Student Mental Model Analyzer for Research and Teaching (SMART) system to help students produce summaries that reflect key concepts and relations in a text. SMART uses the students' summary to generate a multi-dimensional 3S (surface, structure, semantic) evaluation of the students' mental model. This model is then used to drive feedback to help students revise their summary. The current study is an initial investigation examining whether writing and revising in SMART improves students' summary quality. Students (n = 38) in a graduate-level online course used SMART for seven reading assignments. The 38 students submitted a total of 357 summaries in response to the seven readings. In 47 cases, students produced both an initial draft and a modified revision. These 47 cases were selected for analysis. In the quantitative phase, MANOVA results indicated that students' summaries improved along the 3S dimensions from initial draft to revision. In the qualitative phase, inspection of exemplar cases revealed how students' mental models changed towards more robust and cohesive knowledge structure for texts.

During reading, students construct mental models of what they read. Summaries can be used to evaluate the latent knowledge structure of these mental models. We used indices from Student Mental Model Analyzer for Research and Teaching (SMART) to explore the potential of a global index, Graph Centrality (GC), as a measure to describe mental model structure and its relation to the quality of student summaries (e.g., the amount of content-coverage). Students (n = 73) in an online graduate-level course wrote and revised summaries of their course readings. Data preview left the total count of 32 cases to evaluate how students' mental representations changed from initial to final version. These summaries were analyzed using indices derived from the 3S model (surface, structure, semantic) as well as a measure of GC. The results of this initial investigation are promising, demonstrating that Graph Centrality captures important differences in students' summaries, including revision behaviors to the wholistic structure of mental models, modification trajectories toward a cohesive and solid mental representation that is semantically similar to the expert model.

Purpose: Complex features of science texts present idiosyncratic challenges for middle grade readers, especially in a post-Common Core educational world where students' learning is dependent on understanding informational text. The primary aim of this study was to explore how middle school readers process science texts and whether such comprehension processes differed due to features of complexity in two science texts. Method: Thirty 7th grade students read two science texts with different profiles of text complexity in a think-aloud task. Think-aloud protocols were coded for six comprehension processes: connecting inferences, elaborative inferences, evaluative comments, metacognitive comments, and associations. We analyzed the quantity and type of comprehension processes generated across both texts in order to explore how features of text complexity contributed to the comprehension processes students produced while reading. Results: Students made significantly more elaborative and connecting inferences when reading a text with deep cohesion, simple syntax, and concrete words, while students made more evaluative comments, paraphrases, and metacognitive comments when reading a text with referential cohesion, complex syntax, and abstract words. Conclusions: The current study provides exploratory evidence for features of text complexity affecting the type of comprehension processes middle school readers generate while reading science texts. Accordingly, science classroom texts and materials can be evaluated for word, sentence, and passage features of text complexity in order to encourage deep level comprehension of middle school readers.

The use of active comprehension strategies that encourage students to explain what they have read can improve students' comprehension of complex scientific texts. Most research has focused on either strategies that are engaged during reading (online) or those used after reading (offline)--often ignoring potential interactions that might occur in authentic learning. This study uses a 2(online: think-aloud, self-explain) × 3(offline: reread, free recall, explanatory retrieval) design with a 7-day delayed comprehension test to examine how online and offline explanation strategies affect comprehension. Linear mixed effects modeling will be used to examine both main effects and interactions.

iSTART-E is a web-based intelligent tutor developed for Spanish-speaking students to improve their reading comprehension through self-explanation strategy training. This study examined the effects of a blended comprehension strategy intervention on students' reading comprehension skill. Chilean high school students (N = 22) completed nine iSTART-E sessions and a face-to-face classroom lesson that included integrative video and additional examples of self-explanation. Survey data indicated that students thought that iSTART-E was useful and that they had improved their reading skills. Critically, this was supported by objective assessment--students' standardized reading comprehension test (Lectum) performance improved from pre-training to post-training. These findings demonstrate promise for the use of iSTART-E as a computer-supported learning environment for Spanish readers.