Mastery Measurement in Mathematics and the Goldilocks Effect

Math curriculum-based measurement (CBM) is an essential tool for multi-tiered systems of support decision making, but the reliability of math CBMs has received little research, particularly using more rigorous methods such as generalizability (G) theory. Math CBM is historically organized into two domains: mastery measures and general outcome measures. This paper details 17 concurrent G and dependability studies in a partially crossed design investigating the reliability of mastery measure CBMs for students (N = 263) in Grades K, 1, 3, 5, and 7. This study extends prior research by including novel grade levels and more rigorous math content; using generated rather than static measures; embedding a replication; examining bias by race and sex; and evaluating a simpler scoring method of answers correct as compared to digits correct. Most of the variance in scores was accounted for by student. Probe form effects accounted for less than 5% of the variance for 16 of 17 measures and results replicated across days. G coefficients exceeded 0.75 on the first trial for 14 of 17 measures. G studies were repeated by race, sex, and scoring metric. Overall, 1-4 min of assessment was sufficient to meet reliability thresholds, which exceeds prior findings for general outcome measures. This study supports the reliability of mastery measurement in math CBM and as a precise tool to be used in the screening process.