Examining Human and Automated Ratings of Elementary Students' Writing Quality: A Multivariate Generalizability Theory Application
Chen, D., Hebert, M., & Wilson, J. (2022). Examining human and automated ratings of elementary students' writing quality: A multivariate generalizability theory application. American Educational Research Journal, 59(6), 1122-1156. https://doi.org/10.3102/00028312221106773
This article uses multivariate generalizability theory to compare the reliability of human hand-scoring and automated essay scoring for elementary students' writing. Using essays from grade 3-5 students across multiple genres, the study examines how the two scoring methods perform for struggling and non-struggling writers and considers how they might be combined to improve writing assessment.
Key takeaways
- Both hand-scoring and automated essay scoring were highly reliable overall.
- Automated essay scoring was more reliable for non-struggling writers.
- Hand-scoring was more reliable for struggling writers.
- The findings support blending human and automated scoring for writing assessment.
Why it matters
Provides an empirical comparison of human and automated writing scores for elementary students and offers guidance on using both approaches to optimize reliability in writing assessment.
Coauthors
Dandan Chen, Michael Hebert, Joshua Wilson
Topic tags
External links