Limes Master Edition: Using Behavioural Data to Refine Game Mechanics in Research

Abstract

Educational video games are increasingly used in experimental research to examine their effects on learning and motivation. However, many studies focus primarily on outcome measures such as pre–post knowledge tests or self-reported engagement, while paying limited attention to behavioural data generated during gameplay. As a result, it often remains unclear which specific game features contribute to learning outcomes. This paper argues that behavioural logging within educational games can play an important methodological role in experimental research by revealing how learners interact with key game mechanics and by informing the iterative refinement of these mechanics. I illustrate this argument using Limes, a Unity-based educational video game designed to teach Roman history, as a methodological case study based on previously conducted research. The paper draws on findings from two previously conducted experimental studies investigating different aspects of the game design. In these studies, behavioural logs captured players’ interactions with several gameplay features, including optional Codex entries that provided additional historical information. Analyses indicated that engagement with Codex content was positively associated with knowledge acquisition, highlighting the Codex system as a learning-relevant feature within the game environment. Based on these insights, the Codex system was iteratively redesigned during the continued development of the game, leading to a revised version (Limes Master Edition) with more precise behavioural tracking. While earlier versions tracked Codex engagement at a general level, the revised version implemented a logging architecture capable of recording interactions with individual Codex entries. This modification enables future research to analyse detailed patterns of information-seeking behaviour and to better understand how optional learning resources are used during gameplay. Using Limes as a case study, this paper discusses challenges that arise when educational games are used as experimental research tools. It illustrates how behavioural evidence from experiments can guide the refinement and instrumentation of game features. Building on this example, the paper proposes guidelines for researchers who develop educational games for experimental studies, outlining how behavioural data can inform the iterative development of game mechanics and logging systems. By linking experimental findings with design refinements, this approach supports a systematic investigation of how specific game features contribute to learning outcomes.