Back to Search
Start Over
Examining Human and Automated Ratings of Elementary Students' Writing Quality: A Multivariate Generalizability Theory Application
- Source :
-
American Educational Research Journal . Dec 2022 59(6):1122-1156. - Publication Year :
- 2022
-
Abstract
- We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3-5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.
Details
- Language :
- English
- ISSN :
- 0002-8312 and 1935-1011
- Volume :
- 59
- Issue :
- 6
- Database :
- ERIC
- Journal :
- American Educational Research Journal
- Publication Type :
- Academic Journal
- Accession number :
- EJ1355666
- Document Type :
- Journal Articles<br />Reports - Research
- Full Text :
- https://doi.org/10.3102/00028312221106773