An Automated Feedback System for Written Exams

Abstract

Summative assessments, particularly exams, are the dominant method for evaluating student performance in higher education. Although feedback is the single most influential factor in promoting learning, exams are often excluded from the good practices highlighted in literature. The gap stems from challenges such as time constraints, large student numbers, and institutional policies. While automation could improve feedback processes, most current automated feedback systems (AFS) are teacher-centred and do not align with good feedback practices. The present paper uses a design science research approach to develop a student-centred AFS for exams in different domains. The system provides a structured setup process to guide lecturers in providing high-quality feedback. We utilise expert knowledge in the form of Bloom's taxonomy and task types, as well as student data such as points, to generate feedback for each student providing a performance overview and suggestions how to improve their exam performance and learning strategies. In addition, the system delivers detailed feedback on topics on which the student performed poorly. In an initial evaluation of the student-centred AFS, three lecturers successfully used the AFS to generate feedback reports to 1323 students from two large-scale bachelor courses. Survey results indicate that the student-centred AFS delivers high-quality, timely, and personalised feedback at scale, helping students to adapt learning strategies and to identify deficits. The present work thus contributes to solving the challenges of feedback in higher education in general and to solving the exam-specific feedback gap. Additionally, our AFS that is easy to adapt for non-experts. We demonstrate the various design features required for such an AFS, including an adaptable domain model utilising Bloom’s taxonomy and customisable task types, to ensure applicability across diverse educational domains.