Validation of the performance of Tshivenda learners in PIRLS 2006

Abstract

The aim of this study is to validate the Tshivenda learner performance in the Progress in International Reading Literacy Study (PIRLS) 2006 in which there is an anomaly in the Tshivenda language group. By comparing the performance of the Tshivenda learners to that of learners who wrote the PIRLS 2006 test in the other official languages, the notion of performance is related to equivalence in translation in that, if the learners wrote equivalent instruments across all official South African languages, then it is possible that the difference in performance was related to different translation equivalence. Therefore, the validation of the learner performance in this study is directly linked to the validation of the translation. The South African national results of PIRLS 2006 revealed that the Tshivenda language speakers, who had written the PIRLS tests in a secondary language, achieved higher scores than those Tshivenda speakers who had written the tests in their mother tongue (Tshivenda). This result was considered an anomaly. This research investigated the role of translation as an influencing factor in learner comprehension, which may have contributed to this anomaly. Some of the procedures and standards set in place for PIRLS 2006 related to translation and verification were examined. Issues of language and culture, with specific reference to the availability of media in Tshivenda are discussed in the literature. Further investigation was conducted into what translation entails including translation and back-translation, equivalence and non-equivalence as well as the comprehension processes required by each of the four released PIRLS 2006 texts. This study is a secondary analysis of data gathered for PIRLS 2006. Permission to use the data was given in 2011 by the Centre for Evaluation and Assessment at the University of Pretoria, the PIRLS National Centre. Details of the original sampling, collection and analysis methods are provided as part of the discussion on the quality assurance, validity and reliability of the original study. The secondary analysis of the data utilised a mixed methods approach which involved Classical Test Theory and Content Analysis in order to accurately explore this data. The results of this study indicated that, despite the fact that the back-translation revealed many errors, the translation did not affect the learners’ level of comprehension.