Search

Educators use reading comprehension assessments to summarize academic achievement, make decisions in diagnostic evaluations, and identify intervention needs. A challenge, however, with using different assessments in practice is that student performance may change depending on which assessment is administered. This article guides educators in evaluating student performance across multiple assessments, specifically when making decisions for students with learning disabilities (LD) in reading. First, this article provides educators with guidance in establishing a student's foundational skills, specifically those that may contribute to low performance on reading comprehension assessments. Next, the article presents steps for examining the texts and assessment methods commonly used in measures of reading comprehension. The article concludes with recommendations for evaluating student performance when considering a student's foundational skills and characteristics of reading comprehension assessments.

This study investigated the dependability of reading comprehension scores across different text genres and response formats for readers with varied language knowledge. Participants included 78 fourth-graders in an urban elementary school. A randomized and counterbalanced 3 × 2 study design investigated three response formats (open-ended, multiple-choice, retell) and two text genres (narrative, expository) from the Qualitative Reading Inventory (QRI-5) reading comprehension test. Standardized language knowledge measures from the Woodcock Johnson III Tests of Achievement (Academic Knowledge, Oral Comprehension, Picture Vocabulary) defined three reader profiles: (a) < 90 as emerging, (b) 90-100 as basic, and (c) > 100 as proficient. Generalizability studies partitioned variance in scores for reader, text genre, and response format for all three groups. Response format accounted for 42.8 to 62.4% of variance in reading comprehension scores across groups, whereas text genre accounted for very little variance (1.2-4.1%). Single scores were well below a 0.80 dependability threshold (absolute phi coefficients = 0.06-0.14). Decision studies projecting dependability achieved with additional scores varied by response format for each language knowledge group, with very low projected dependability on open-ended and multiple-choice scores for readers with basic language knowledge. Multiple-choice scores had similarly low projected dependability levels for readers with emerging language knowledge. Findings evidence interactions between reader language knowledge and response format in reading comprehension assessment practices. Implications underscore the limitations of using a single score to classify readers with and without proficiency in foundational skills.

Researchers have increasingly investigated sources of variance in reading comprehension test scores, particularly with students with reading difficulties (RD). The purpose of this meta-analysis was to determine if the achievement gap between students with RD and typically developing (TD) students varies as a function of different reading comprehension response formats (e.g., multiple choice, cloze). A systematic literature review identified 82 eligible studies. All studies administered reading comprehension assessments to students with RD and TD students in Grades K-12. Hedge's g standardized mean difference effect sizes were calculated, and random effects robust variance estimation techniques were used to aggregate average weighted effect sizes for each response format. Results indicated that the achievement gap between students with RD and TD students was larger for some response formats (e.g., picture selection ES[subscript g] = -1.80) than others (e.g., retell ES[subscript g] = -0.60). Moreover, for multiple-choice, cloze, and open-ended question response formats, single-predictor metaregression models explored potential moderators of heterogeneity in effect sizes. No clear patterns, however, emerged in regard to moderators of heterogeneity in effect sizes across response formats. Findings suggest that the use of different response formats may lead to variability in the achievement gap between students with RD and TD students.

Educators use reading comprehension assessments to summarize academic achievement, make decisions in diagnostic evaluations, and identify intervention needs. A challenge, however, with using different assessments in practice is that student performance may change depending on which assessment is administered. This article guides educators in evaluating student performance across multiple assessments, specifically when making decisions for students with learning disabilities (LD) in reading. First, this article provides educators with guidance in establishing a student's foundational skills, specifically those that may contribute to low performance on reading comprehension assessments. Next, the article presents steps for examining the texts and assessment methods commonly used in measures of reading comprehension. The article concludes with recommendations for evaluating student performance when considering a student's foundational skills and characteristics of reading comprehension assessments. [ABSTRACT FROM AUTHOR]

Researchers have increasingly investigated sources of variance in reading comprehension test scores, particularly with students with reading difficulties (RD). The purpose of this meta-analysis was to determine if the achievement gap between students with RD and typically developing (TD) students varies as a function of different reading comprehension response formats (e.g., multiple choice, cloze). A systematic literature review identified 82 eligible studies. All studies administered reading comprehension assessments to students with RD and TD students in Grades K-12. Hedge's g standardized mean difference effect sizes were calculated, and random effects robust variance estimation techniques were used to aggregate average weighted effect sizes for each response format. Results indicated that the achievement gap between students with RD and TD students was larger for some response formats (e.g., picture selection ESg = -1.80) than others (e.g., retell ESg = -0.60). Moreover, for multiple-choice, cloze, and open-ended question response formats, single-predictor metaregression models explored potential moderators of heterogeneity in effect sizes. No clear patterns, however, emerged in regard to moderators of heterogeneity in effect sizes across response formats. Findings suggest that the use of different response formats may lead to variability in the achievement gap between students with RD and TD students. [ABSTRACT FROM AUTHOR]