Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy
Wilson, J., & Roscoe, R. D. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87-125. https://doi.org/10.1177/0735633119830764
This study evaluated automated writing evaluation using multiple outcomes, including writing self-efficacy, holistic writing quality, state English language arts performance, and teachers' perceptions of social validity. After controlling for pretest measures, students using AWE showed stronger writing self-efficacy and higher state test performance, even though holistic writing quality did not differ from the comparison condition.
Key takeaways
- AWE use did not significantly improve holistic writing quality after controlling for pretest measures.
- Students in the AWE condition reported higher writing self-efficacy.
- Students in the AWE condition performed better on the state English language arts test.
- Writing self-efficacy partially mediated the effect of condition on state test performance.
- Teachers reported positive perceptions of AWE's social validity.
- Evaluating AWE with multiple metrics provides a more complete picture of effectiveness than relying on a single outcome.
Why it matters
This article sharpened Dr. Wilson's research agenda by showing that AWE efficacy should be judged through multiple lenses, including motivation, assessment outcomes, and teacher perceptions.
Coauthors
Joshua Wilson, Rod D. Roscoe
Topic tags
External links