Exploring Trainees’ Behaviour in Hands-on Cybersecurity Exercises Through Data Mining

Abstract

Despite the rising number of cybersecurity professionals, the demand for more experts in this field is still substantial. Cybersecurity professionals must also possess up-to-date knowledge and skills to counter cybersecurity threats’ dynamicity and rapidly evolving nature. Hands-on cybersecurity training is mandatory to practice various tools and improve one’s technical cybersecurity skills. Generally, an interactive learning environment is set, where trainees perform sophisticated tasks by accessing complete operating systems, applications, and networks. One of the main challenges that cybersecurity organizations are facing today is the generation of massive data through practice exercises.  So, it becomes a problem to convert this data into knowledge to improve the overall quality of the learning system. The large amount of interaction data and its complexity also limit us to do automated analysis. Thus, these challenges for cybersecurity learners can be addressed through appropriate educational data analysis by having insights or testing hypotheses or models on a proper dataset. Revealing the patterns, rules, item sets and time taken by trainees while using any command line tool could help the trainer to assess the trainees and to provide feedback. Therefore, in this paper we are analyzing the frequency patterns and timing information captured from the trainees’ command line log to reveal their solving techniques, easy and struggling stages, slipups, and individual performance.  Through our study, we aim to show how education and training providers can foresee learners who are less likely to succeed in a task or exhibit low performance, which can impede learning proficiency. With this knowledge, organizations and trainers can identify trainees who require additional attention or support. It may also be able to identify elements related to an organization like training aids, training methodology, etc. that need improvement. This study demonstrates the utility of data-mining techniques, specifically rule mining and sequential mining, to empower training designers to delve into datasets derived from cyber security training exercises.