Evaluating Cybersecurity Awareness in Employees Using Gameplay: Data and Machine Learning Models

Abstract

Cyber-attacks continue to pose persistent challenges within professional environments. Human error remains a critical vulnerability, frequently leading to security breaches through credential misuse and social engineering tactics. Traditional cybersecurity training approaches often lack effectiveness when not adapted to the dynamic threat landscape. This study presents CyberEmployee, a serious game developed to enhance cybersecurity awareness among employees through interactive learning. The objective is to assess employees’ awareness levels by analysing gameplay data using machine learning techniques. Data were collected via the game's integrated scoreboard, which tracked user behaviors and performance patterns. The resulting dataset was analysed using multiple machine learning algorithms, including Random Forest, Support Vector Machines (SVM), XGBoost, K-Nearest Neighbors (KNN), and Logistic Regression. Experimental results demonstrated accuracy rates ranging from 86% to 100% and F1-scores from 75% to 100%. The highest performance—100% accuracy and 100% F1-score—was achieved using the Random Forest and XGBoost models. This analysis indicates that ensemble learning methods outperform other classifiers in employee classification. Furthermore, gameplay duration and player score were identified as key predictive features. These findings indicate the potential of serious games combined with machine learning for data-driven cybersecurity training frameworks.