Jump to Content
Security & Identity

The empty chair: Guess who’s missing from your cybersecurity tabletop exercise?

June 27, 2024
https://storage.googleapis.com/gweb-cloudblog-publish/images/GettyImages-1363104923.max-2600x2600.jpg
Bill Reid

Security Advisor, Office of the CISO, Google Cloud

Alishia Hui

Senior Consultant, Mandiant

Hear monthly from our Cloud CISO in your inbox

Get the latest on security from Cloud CISO Phil Venables.

Subscribe

Imagine for a moment that you’re the chief executive of a hospital. The calendar you had at 6:59 a.m. has been completely overhauled three minutes later: Your hospital’s computers and network are down, and it looks like the cause might be a cyberattack.

You’re facing numerous disruptions to key operational systems, including a shutdown of your hospital’s HVAC. You and your executive team are forced to look at workarounds and coordination with neighboring facilities to provide time-sensitive patient care while dealing with an active attacker in the network.

The team struggles to manage the chaotic situation, balancing patient care and ongoing operations, technical response and remediation efforts, and keeping a consistent narrative to inform everyone involved and affected by the disruption.

Too many organizations overlook potential participants who are responsible for critical business segments: those running operational technology (OT) and industrial control systems (ICS).

Just as quickly as the crisis started, it’s over. Fortunately, it existed only in the confines of the conference room where you were participating in a role-playing scenario designed to help you, other executives, and your board of directors understand their responsibilities during a cyberattack.

Known as a tabletop exercise, or TTX, these workshops are one of the most effective ways to increase awareness of cybersecurity issues. They can be especially useful tools for helping an organization’s leadership and board members understand and test how the company would react when faced with a cyberattack.

TTXs involve a realistic, current scenario crafted to represent what would be a typical threat facing the company. Often designed by the IT security team, they usually borrow from current threat research and event activity that has occurred among peer organizations. Mixed in amongst the scenario are a series of “injects,” plot twists designed to simulate the nature of real events, where further investigation uncovers more complexity and the event becomes harder to fix.

Participants apply their organization’s incident response and business continuity plans as a way of testing them, with a specific emphasis on intra-organizational coordination and communication, and the proscribed ways in which communication should happen externally with the media and customers.

While the security team usually convenes the TTX, they sometimes use an outside party to help run them so that they can participate. Other participants invited often include IT, human resources, legal, finance, operations, marketing, communications, and frontline staff leaders.

The scope of the exercise involves some form of cybersecurity breach of a system, often with a malicious threat actor gaining unauthorized access to an endpoint device, and then traversing to networks and other systems such as cloud infrastructure. The initial vector of the compromise is often a stolen credential, an unpatched system, or social engineering of some form to evade second factor authentication.

These exercises are incredibly useful, and should be part of every organization’s security toolkit, because they test incident response plans and allow practice of crisis communication in a lower risk situation. However, too many organizations overlook potential participants who are responsible for critical business segments: those running operational technology (OT) and industrial control systems (ICS).

https://storage.googleapis.com/gweb-cloudblog-publish/images/CISA_yearly_disclosed_ICS_and_medical_vuln.max-1300x1300.png

CISA yearly disclosed ICS and medical vulnerabilities between 2010 and 2023.

In a recent conversation I had with several healthcare CEOs, they talked about how much they had learned from recently-completed tabletop exercises in their organizations. When pressed, they all sheepishly admitted that they had neglected to include the head of biomedical engineering, who is responsible for all the medical devices, and the head of building operations, who looks after all the building management systems including air handling and lighting. No matter how educational their TTXs were, these omissions are a missed opportunity to bring the whole organization up to speed on realistic, likely cyberattack vectors.

OT and ICS: We are all 'factories'

Most companies don’t think enough about cybersecurity for OT and ICS, yet they depend on these physical systems to operate their business. OT and ICS are especially vital to hospitals and other healthcare facilities to literally operate, since they require the correct air pressure and temperature to perform surgeries.

While IT-based threats continue to dominate the conversation, the threat landscape for OT and ICS is challenging and of increasing concern. For example, threat intelligence from Mandiant (part of Google Cloud), which tracks cyber threat activity and incidents affecting OT and ICS environments, shows that the annual number of incidents we tracked affecting OT and ICS systems has effectively doubled since 2020.

This increase is primarily driven by the increasing number of threat actors engaging in opportunistic targeting of internet-accessible OT and by the resurgence of state-sponsored cyber physical attacks following Russia’s invasion of Ukraine in 2022.

Similarly, our analysis of vulnerability advisories published by the Cybersecurity and Infrastructure Security Agency (CISA) shows a significant upward trend in the number of ICS and medical vulnerabilities disclosed over time as these systems gain more interest from government, industry, security researchers, and threat actors.

There has been so much concern with medical device security that Congress amended the Food, Drug, and Cosmetic Act to add Section 524B, which requires newly-approved medical devices to design with security in mind, identify and mitigate vulnerabilities before and after release into market, and to provide software bills of materials to further aid the detection and patching of emergent vulnerabilities.

Filling the empty seat: How we can be more OT and ICS inclusive

The Mandiant team has worked with many customers to help them identify threats and vulnerabilities, and to help with OT and ICS incident response. They’ve also helped many organizations create tabletop exercises that are inclusive of these technologies and the risks they face in operations.

When designing a tabletop exercise, Mandiant will initially work with a customer to identify the potential attendees for the exercise, the desired scope and subject of the exercise, and any specific objectives for this exercise. This phase is critical to ensure that the appropriate participants are engaged, both for the design and the exercise itself, and customer context, threat intelligence, and operational context are leveraged to design an exercise that includes OT and ICS components if desired. Questions and impacts to consider include:

Question

Impact

Critical OT/ICS components: What OT and ICS devices do we have that contribute to key business functions? What would be the operational impact if these devices were to fail? 

OT and ICS components can often be overlooked when designing these exercises. Considering their involvement in key business processes can help identify additional scenarios with specific response nuances that may not have been considered before.

Incorporation of trusted agents and participant identification: Who is the subject matter expert for these devices? Who makes decisions related to devices or processes that may have potential cyber-physical or physical impacts?

By involving a SME who is familiar with the devices and their granular involvement and integration in key processes and the overall environment in the TTX design, the exercise can be customized to better test the organization’s crisis preparedness. This SME should be engaged during the exercise as well, since the context and decisions they provide can inform other processes (such as incident response, business continuity, communications, and fiscal responsibility).

Threat intelligence and operational concerns: What is the operational team most concerned about? What threat intelligence reports align with these concerns? Are there any other topical threats that are aligned with our industry vertical or similar organizations?

Threat intelligence should be leveraged to ensure that scenarios are relevant and incorporate the tactics, techniques, and procedures currently used by threat actors. TTX often don’t consider attack vectors such as a building management system, a legacy device in a manufacturing environment, or an interconnected ICS device at a remote plant — but attackers do.

Exercise scenario scope: In addition to concerns around confidentiality and integrity, what injects or scenario developments have we considered that will test responses pertaining to safety and reliability? What processes and devices support human safety directly (such as management and regulation of heat or electricity) or indirectly (such as creation or distribution of medicine)?

Timely response, safety considerations, manual workarounds, and availability of key components are often a larger priority in OT-driven scenarios, and should be incorporated accordingly. 

Consideration for convergent scenarios, where an impact to IT and OT is observed, engages both IT and OT participants while also demonstrating the importance of communication and collaboration in order to manage concurrent response efforts in both environments.

Exercise and incident experience: Has the IT and OT team previously conducted tabletop exercises or undergone a recent incident?

To get the most out of an exercise involving both IT and OT participants, consideration should be made for conducting an initial exercise or a set of exercises with individual teams to allow team members to talk through their specific processes and response nuances. A joint exercise will generally focus on the broader organization-wide response, key decisions, and coordination and communication efforts.

Answering these questions will inform key design decisions, helping to design an exercise that is both topical and engaging for participants.

Mandiant recently conducted a tabletop exercise for an organization in the utilities industry, where participants from the IT and OT teams were engaged to walk through the technical response efforts for a simulated incident with potential cyber-physical impacts. The trusted agent was a key member of the OT team who was able to walk the team through the core infrastructure, processes to deliver power, and available monitoring tooling within the OT environment. This enabled the delivery of a successful exercise, where the following topics were explored:

  • Impact to customers in the immediate to short-term, and manual workarounds that would be required to reduce the possible impacts to customer safety.
  • Ability to validate the integrity of control systems and additional steps that would be required.
  • Existing information sharing channels that were lacking between the IT and OT teams.
  • Key information that must be provided by incident responders to support decision making and business continuity needs.
  • Key incident response activities to contain and eradicate the attacker.
  • Proactive communication to minimize disruptions and ensure a smooth workflow.

Takeaways following the exercise included the definition of additional processes to improve communication and early incident notification, identification of additional steps in existing incident response playbooks, clarification of specific manual workarounds, and further refinement of incident management processes. In addition, the exercise provided an opportunity to raise awareness of OT among responders, consider specific response nuances, and highlight the importance of security within the OT environment.

Enabling and intentionally incorporating OT representation and participants in tabletop exercises can be invaluable to improve an organization’s preparedness in the case of an incident that can have tangible impacts on the well-being of many. If you’d like to learn more about tabletop exercises, you can contact our experts in Google Cloud’s Office of the CISO and Mandiant.

Posted in