AI, Instructor, Observer, and Peer Feedback in Online Microteaching: A Multi-Rater Comparison Study
DOI:
https://doi.org/10.34190/ecel.25.1.5040Keywords:
online microteaching, AI assessment, peer feedback, multi-rater comparison, EFL teacher educationAbstract
Feedback is central to the effectiveness of microteaching as a teacher training method, yet how different feedback sources compare when evaluating the same performance is an underdeveloped area. In online environments, peer feedback, instructor assessment, external observation, and AI-generated evaluation can all operate alongside one another. Understanding which sources produce meaningful and consistent judgements has practical consequences for teacher educators. This paper presents a case study conducted at Constantine the Philosopher University in Nitra, Slovakia, within a master's-level course on the Didactics of Distance Online Education. Students of the course each delivered an online microteaching session via Google Meet, incorporating at least one AI tool connected to their lesson objective. All sessions were recorded and re-analysed by an external observer. Five rater sources evaluated each microteaching against a shared four-criterion rubric: course peers, the course instructor, the external observer, and two AI models applied under three distinct evaluation conditions: isolated, sequential, and simultaneous. The study investigates how these rater sources compare in their assessments of the same teaching events and whether AI-generated feedback converges with trained human judgement depending on evaluation structure. Findings reveal differences between rater sources. Peer feedback shows a pattern of inflation and an inverse relationship with expert judgement, raising questions about its reliability as a standalone formative instrument. AI evaluation is highly sensitive to evaluation conditions. Isolated assessment produces variable and unreliable results. Sequential evaluation, in which all presenters are assessed in a single continuous conversation, converges closely with the judgements of trained human raters. This convergence was confirmed across both AI models tested. These findings have implications for feedback design in online teacher education. Rater source is as consequential as rubric design. Peer feedback requires pedagogical framing and should not function as a standalone assessment instrument. AI can serve as a supplementary tool when deployment conditions are understood in advance. Pedagogical expert judgement remains the validity benchmark against which any automated or peer-based system must be measured. The study contributes empirical data to a field where AI tools are being adopted faster than the research can evaluate them.