Validation of a Learning Outcome Survey (WOLOS) using the Rasch Model: An Implication on Washback Study

Abstract

The assessment reform which has enveloped every part of the world warrants an evaluation of teaching and learning practices through washback study. This is due to the fact that washback is the phenomenon of how testing influences the teaching and learning. Malaysia has adopted Outcome Based Education policy and therefore, the efficacy of its assessment system, Outcome Based Assessment, is deemed pivotal to be evaluated. Against this backdrop, the Washback on Learning Outcome Survey (WOLOS) was developed and validated by means of qualitative (semi-structured interview) and quantitative analysis (Item Objective Congruence) and Rasch Measurement Model). Responses to 150 items by 65 participants from one public university in Malaysia were subjected to the Rasch analysis to ascertain the psychometric properties of the WOLOS.  Five criteria within reliability (person and item reliability), validity (separation index, item polarity and item fit) and precision of measurement were evaluated to ensure the usefulness of measurement in WOLOS. Some items were deleted.  Subsequently, reanalysis of the criteria provided evidence that WOLOS can be considered a psychometrically reliable instrument for the evaluation of impact of assessment practices on student learning outcomes.