The deceptive mean: Conceptual scoring of cloze entries differentially advantages more able readers.

The ‘cloze’ test is one possible investigative instrument for predicting text comprehensibility. Conceptual coding of student replacement of deleted words has been considered to be more valid than exact coding, partly because conceptual coding seemed fairer to poorer readers. This paper reports a quantitative study of 447 Australian secondary school student responses to three differing cloze tests covering identical content. The study sought to better understand the impact of cloze coding decisions on estimation of text difficulty and/or reader competence. Findings reveal the following: (a) conceptual coding increases reader cloze score differentially; (b) the increase in score from conceptual coding is greater for more able readers; (c) conceptual coding differentially advantages more able readers of less difficult text. This study is significant because instructors, researchers and developers often make decisions based on interpretation of mean scores from grouped cloze test results and such summaries of result distributions can obscure differential advantages such as those made clearer by this study. Estimates of text comprehensibility based on mean scores of conceptually coded cloze tests may overestimate group competence and underestimate text difficulty for some groups of readers. This contradicts an apparent consensus in the literature dealing with deletion-based tests. [ABSTRACT FROM AUTHOR]