A conceptual framework for the design and development of scholarly assessment instruments in higher education

Abstract

Assessment illiteracy amongst higher education lecturers undermines effective assessment practices. The non-mandatory nature of assessor training that appears amongst non-educationally trained lecturers in South African higher education diminishes the quality of assessment practices to that of being unscholarly, unprincipled and disorganised. Grounded in Design Science Research the main aim of this research study was to conceptualise a framework for the design and development of scholarly assessment instruments in higher education, validated by an expert panel and informed by the General Systems Theory (GST) of Von Bertalanffy (1968). It was argued that strengthening the assessment literacy of lecturers might promote scholarly thinking when reflecting on, designing and developing an assessment instrument.

Sixteen lecturers from four academic departments residing in two different schools/faculties were sampled and data was collected using a mixed methods research design. The research participants were subjected to two cognitive mapping sessions, two semi-structured interviews and a workshop intervention resulting in the development of seven interconnected and interdependent information and technology artifacts, namely, an Assessment Literacy Baseline Assessment (ALBA) diagnostic tool, a Scholarly Interpretation Mapping Method (SIMM) consisting of scholarly interpretation steps, a scholarly visualisation method and contextualised taxonomy handouts, a Scholarly Assessment Instrument Design (SAID) workshop, a Scholarly Assessment Instrument Design and Development Application (SAIDA) and a Scholarly Principled Approach to the Practice of Assessment Instrument Design and Development (SPPAIDD).

In a higher education institution (such as the selected university), where assessment literacy training is not compulsory, it was found that customised, contextualised assessment instrument design and development advising and training, embedded in a model of cognition using an established learning taxonomy and the amplification of lecturers’ cognition through the visualisation of the assessment instrument design and development process via a software application, can strengthen lecturers’ assessment literacy levels. It is concluded that a scholarly approach to the design and development of assessment instruments is an under-researched phenomenon in the South African higher education context and that a conceptual framework for the promotion of a scholarly approach to the practice of assessment instrument design and development, is long overdue.