Exploring the Effectiveness of Large-Scale Automated Writing Evaluation Implementation on State Test Performance Using Generalised Boosted Modelling
Huang, Y., & Wilson, J. (2025). Exploring the effectiveness of large-scale automated writing evaluation implementation on state test performance using generalised boosted modelling. Journal of Computer Assisted Learning, 41, e70009. https://doi.org/10.1111/jcal.70009
This study examined the first-year statewide implementation of the Utah Compose automated writing evaluation system and its association with students' English Language Arts state test performance. Using statewide data from students in Grades 4-11 in Utah (N = 337,473) and a quasi-experimental design with generalized boosted modelling for propensity score weighting, the authors found that more frequent AWE use was associated with improved ELA performance, with some variation across student demographic groups.
Key takeaways
- Large-scale AWE use was associated with higher state ELA performance.
- The analysis used a quasi-experimental design with propensity score weighting to strengthen causal interpretation.
- Effects differed across some student demographic groups.
- The study provides evidence from a naturalistic K-12 statewide implementation, not just small classroom trials.
Why it matters
Provides large-scale quasi-experimental evidence on the effectiveness of a statewide automated writing evaluation system in U.S. K-12 education.
Coauthors
Yue Huang, Joshua Wilson
Topic tags
External links