Evaluation of AI Agent Accelerated Cyber Operations Planning

Abstract

As societies become increasingly digital, they are more exposed to cyber threats that have the potential to harm human life and damage critical infrastructure and other assets. To counter these fast-paced threats, Defensive Cyber Operations (DCO) leaders must enhance their capabilities for rapid decision-making and response. Artificial Intelligence (AI), as a radical levelling technology, has the potential to accelerate DCOs; however, the existing solutions frequently focus on narrow technical use-cases and lack emphasis on the leadership dimension of DCO. The purpose of this paper is to address that gap by researching how AI can accelerate one of the most relevant DCO use-cases identified in the author’s earlier research, course of action recommendation, especially in operations planning. The study is based on case study methodology, where AI agent-generated operations plans are compared to real-life DCO operations plans made by leading experts in the world’s most complex defensive cyber exercise, NATO Locked Shields 2025. The study focuses on two of the most critical decisions a DCO leader needs to make during the primary process of DCO: Prioritization of defended capabilities and assets, and the right resourcing and allocations. The selected exercise provided an excellent platform for this study to compare multiple human-made plans to machine-made plans, as 17 world-class blue teams were given the same exercise scenario and operation order. As a result, this paper demonstrates that with proper architecture and context engineering, AI can significantly accelerate DCO leaders’ decision-making in operations planning, while human-machine teaming is still needed to navigate a complex operating environment where cyber operations are typically conducted. The main contributions of this paper are 1. evaluation of an AI agent’s performance in DCO operations planning in comparison to human experts, and 2. construction of a reference architecture for the DCO planning agent. Future research can be built to improve the results of the reference architecture. As AI’s capabilities are developing rapidly, it is expected that the capabilities of autonomic AI agents will increase.