Rater training

The current study was carried out because of difficulties language advisers were experiencing with utilising the marking sheets to draw students’ attention to their strengths and weaknesses. A selection of 66 marking sheets with detailed comments from a variety of experienced raters was analysed and coded by two independent researchers. Themes were established regarding features that make a comment valuable or not valuable. Some of those same comments were then shared with students to determine whether or not they agreed with the advisers’ assessment.

18p nguathienthan9 08-12-2020 14 3   Download

Their ratings before and after the training session on the VSTEP.3-5 speaking rating scales were then compared. Particularly, dimensions of score reliability, criterion difficulty, rater severity, rater fit, rater bias, and score band separation were analyzed. Positive results were detected when the post-training ratings were shown to be more reliable, consistent, and distinguishable. Improvements were more noticeable for the score band separation and slighter in other aspects.

14p nguathienthan9 08-12-2020 29 2   Download