Comparing the accuracy of different scoring methods for identifying sixth graders at risk of failing a state writing assessment
Wilson, J., Olinghouse, N. G., McCoach, D. B., Santangelo, T., & Andrada, G. N. (2016). Comparing the accuracy of different scoring methods for identifying sixth graders at risk of failing a state writing assessment. Assessing Writing, 27, 11-23. https://doi.org/10.1016/j.asw.2015.06.003
This study compared several writing scoring methods for their ability to identify sixth graders at risk of failing a state writing assessment. Using benchmark writing data from fall 2012 and state assessment results from spring 2013, the authors evaluated prior writing achievement, human holistic scoring, automated essay scoring, and related writing indicators using ROC and AUC classification analyses.
Key takeaways
- Prior writing achievement and Project Essay Grade automated essay scoring showed the highest classification accuracy.
- Combining predictors produced only small improvements over the best single measures.
- Choice of scoring method meaningfully affected how well students at risk of failure were identified.
- Automated essay scoring may be a practical tool for early identification of struggling writers.
Why it matters
Provides evidence on the relative classification accuracy of common writing score types for screening upper-elementary and middle-school students at risk of failing a state writing assessment, with direct implications for assessment and intervention decisions.
Coauthors
Joshua Wilson, Natalie G. Olinghouse, D. Betsy McCoach, Tanya Santangelo, Gilbert N. Andrada
Topic tags
External links