This is an abridged version of a talk I authored and presented to a government customer in November 2023.

I’ve written about Attack Simulation before, and it’s a topic that comes up on a regular basis with large customers that I speak to. I’ve used various open source tools to help generate attack data quickly, without the heavy lifting involved with manually building servers, installing an OS, installing agents, and performing an attack.

But what is Attack Simulation? Why might an organisation, or a newcomer working in a cyber security defence role, utilise this set of tools?

Attack Simulation – Lowering the barrier to entry

Attack Simulation is one of the many tools in the toolkit for a red team, traditionally highly specialised, highly trained, and costly. These teams would conduct rigorous exercises to simulate the tactics, techniques and procedures of threat actors, with the aim to penetrate an organisation’s systems or networks. One challenge lay in this exclusivity – a select few were entrusted with the responsibility off utilising these advanced toolsets in their quest to improve an organisations defences.

With the release of a number of Attack Simulation toolkits over the last few years, the barrier to entry has been significantly lowered. Many junior analysts are struggling with a lack of time, budget, and complex systems that they are required to learn. We know that the cyber security workforce is extremely short on talent, with a shortfall of 2.6 million professionals in APAC alone. These junior analysts can use these capabilities to rapidly build, iterate and learn using advanced attack data that would be extremely hard to come by in the real world.

An experienced defender can use attack simulation to learn generate data from techniques used by threat actors in their industry vertical, or geographic part of the world, in order to validate that their security detections are configured correctly, and would likely detect a real attack.

This equal footing helps foster new talent, promotes ingenuity, and taps into diverse pockets of an organisation that have significant value to add, but may not have formal training, or decades of experience.

Fostering a proactive cybersecurity culture

Significant advantages to individuals and teams exist beyond just technical capabilities and training. Leveraging Attack Simulation further allows an organisation to go on the front foot with their defences, and is a key pillar of a threat informed defence, and an assume breach mindset.

Initially developed by MITRE, a “threat-informed defense applies a deep understanding of adversary tradecraft and technology to protect against, detect, and mitigate cyber-attacks.”

Assume Breach is the recognition that “security teams should not operate under the assumption that a breach will happen, but when“. Attack Simulation allows for the defender to put themselves in the mindset of that attacker that they assume has already been on, or is actively on their systems.

Cyber threats are continually evolving, with new threat actors, new 0-days and new geopolitical events driving significant change. Attack Simulation allows for a defender to actively test their countermeasures dynamically, potentially via entirely automated means, where defence strategies can rapidly evolve with the changing threat landscape.

Threat feeds, weather they be open source, commercial or internally generated, are used to determine appropriate attacks to simulate. Often these feeds and the attacks to be simulated are mapped to a framework, such as MITRE ATT&CK.

The simulated attack is performed with the view to determining if the attack would be detected in the real world. Should no attack be detected from the simulation, a process of detection engineering is performed to appropriately find the signal from the noise.

Reporting allows the organisation to track key metrics such as the percentage of simulated attacks that have been detected, allowing for a steady improvement in security posture.

Quantifying unknowns

You have great protections against many threats. Your various security systems are finding events, malicious actors, malware, and insider threats on a regular basis. But what about all those things you’re looking for, but have never detected in your environment? That new 0-day that was released and is propagating in the wild, but you haven’t seen any alerts. Is it because you haven’t been hit yet? Or because your detections are somehow flawed? What are your SOC teams actually doing with those alerts when they arrive – are they being appropriately investigated?

The key to quantifying these unknowns is to test, and to be able to test rapidly and repeatedly when the circumstances change (which is of course near constant).

Simulate an adversary targeting your network, and actively test your defences to start quantifying these unknowns, and closing gaps. Use open source and commercial tools to do this, starting small, then iterating and growing as you start to see success.

Cost-Effective Risk Mitigation at Scale

Every organisation I speak to has cost pressures across the board, with the cyber part of the organisation certainly not immune.

The cost of data breaches continues to climb annually, with the average cost in 2023 at $4.45M USD, not to mention the impacts on publicly traded companies. CISOs have even been subject to fraud charges for failing to mitigate and disclose cyber risks.

Proactive use of simulation tools to identify and address vulnerabilities provides an organisation with significantly enhanced defences, reducing the likelihood of an an expensive compromise. It allows CISOs to prove their teams cyber defences have been tested appropriately.

The use of Attack Simulation does not diminish the need for comprehensive red team assessments, instead complementing these capabilities through regular, repeated testing of defences.

Closing Thoughts: A Unified Front Against Cyber Threats

There are significant benefits in utilising these tools for both experienced cyber professionals, and new entrants to the cyber security industry, as well as the organisations they work for.

For junior analysts, utilise technology as a force multiplier to help you abstract away from the undifferentiated heavy lifting often involved in learning new cyber security tools.

These tools help you get results quickly, fail fast, and then iterate rapidly, allowing you to spend time on what matters – learning how to detect and defend against an attack.

Experienced professionals can start to answer some difficult questions – our detections for known threats are working, but how do we quantify the unknowns? How do we know if we would catch a new attack that we’ve never spotted in the wild? How do our SOC teams actually respond to a new alert – do they mark it as a false positive and move on, or do they appropriately triage?

Appendix: List of Attack Simulation toolkits I’ve used

There are many different tools out there. I’ve written about Splunk Attack Range, but this is far from the only capability, and I’d encourage you to explore all options.

​https://github.com/splunk/attack_range

​https://github.com/mitre/caldera

​https://github.com/center-for-threat-informed-defense/adversary_emulation_library

​https://github.com/guardicore/monkey