Validity of automated essay scores for elementary-age English language learners: Evidence of bias?
Wilson, J., & Huang, Y. (2024). Validity of automated essay scores for elementary-age English language learners: Evidence of bias? Assessing Writing, 60, Article 100815. https://doi.org/10.1016/j.asw.2024.100815
This study examined whether automated essay scores from the MI Write system show evidence of bias for elementary-age English language learners. Using data from 2,829 students in grades 3-5, the authors compared the predictive validity of automated and human scores on an English-learner-focused writing test and a state writing test, and found that automated scoring was not uniquely biased relative to human scoring, though both methods were less predictive for English language learners than for non-ELL students on the state test.
Key takeaways
- For English language learners, automated MI Write scores and human scores predicted both writing outcomes with similar accuracy.
- Automated scoring did not appear uniquely biased relative to human scoring for this subgroup.
- Both automated and human scores were less strongly related to the state writing test for English language learners than for non-ELL peers.
- The findings suggest MI Write reproduces existing human-scoring biases rather than introducing a distinct new bias.
Why it matters
Provides subgroup-focused validity evidence for automated essay scoring in elementary writing assessment by comparing predictive bias in automated and human scoring for English language learners.
Coauthors
Joshua Wilson, Yue Huang
Topic tags
External links