Score Generalizability in Writing Assessment

Reliability of writing scores has always been a§longstanding concern in second-language assessment.§Due to the myriad sources of variability§contributing to writing scores, such as tasks,§raters, scoring rubrics, genres, topics, and traits§to be evaluated, writing exams have relatively low§score reliability. Classical methods of estimating§reliability fall short of accounting for different§sources of error since only one undifferentiated§error term is provided in this context.§Generalizability theory is perceived as a better§alternative to these classical methods given its§ability to provide information about the different§sources of error. In addition, it gives insights into§how these variables interact in an assessment§context. More importantly, generalizability theory§can propose possible assessment scenarios to achieve§maximum measurement precision. This book focuses on§two of these facets, tasks and raters, in order to§investigate their relative contribution to writing§scores using generalizability analysis. The book§adopts an interdisciplinary perspective where both§applied linguistics and psychometrics research is§brought together.