Threat-Led Penetration Testing for DORA – How does it work?

The long-awaited draft regulatory technical standards (RTS) on threat-led penetration testing (TLPT) have finally been released, bringing clarity to the specific requirements for financial organizations. Published on July 17th, 2024, this key document is part of the second batch of policy products under the Digital Operational Resilience Act (DORA), issued by the three European Supervisory Authorities—European Banking Authority, European Insurance and Occupational Pensions Authority, and European Securities and Markets Authority.

Regulatory technical standards present the criteria for identifying financial entities required to perform threat-led penetration testing and specify the requirements for internal testers, scope, methodology, testing phases, results, remediation, and necessary supervisory cooperation for TLPT implementation and mutual recognition.

This article summarizes the technical standards of TLPT, including the process, methodology, organizational arrangements and testing phases. We also explain what threat-led pentesting is and how it differs from the regular pentesting that organizations in the financial sector are used to performing.

Here are the main takeaways from the document:

• TLPT for DORA will follow the EU TIBER framework for threat-led pentesting, except for the use of purple teams and internal testers.
• DORA TLPT will have to be performed at least every 3 years or more if the supervising authorities require it.
• Only certain financial institutions will have to perform TLPT.
• Purple teaming exercise is currently strongly encouraged but not a mandatory element in the original TIBER-EU framework. However, the RTS makes purple teaming mandatory in the TLPT closure phase, similarly to the replay workshop.
• Financial entities will be able to use external and internal testers for the red-teaming exercise. The threat intelligence provider always has to be external. As an additional safeguard, every 3rd test will have to use an external red team.

What is TLPT?

Threat-Led Penetration Testing (TLPT) is an advanced cybersecurity assessment that involves intelligence-driven simulations of real-world cyberattacks on an organization. Unlike traditional penetration testing, TLPT is conducted in secrecy, with the organization’s defense team unaware that a test is taking place. This covert approach ensures an authentic evaluation of the organization’s ability to detect and respond to sophisticated attacks. Based on detailed threat intelligence, a red team replicates the tactics of specific threat actors, targeting a wide range of systems, processes, and personnel.

TLPT technical standards for Digital Operational Resilience Act (DORA)

TLPT standards for DORA were developed in accordance with the TIBER-EU framework. However, they differ in a few aspects, such as the use of internal testers and the obligatory performance of a purple teaming exercise.

Financial entities required to perform TLPT

DORA covers a range of financial entities such as credit institutions, payment institutions, electronic money institutions, investment firms, crypto-asset service providers, central securities depositories, trading venues, and many more. All of them are required to undergo penetration testing assessments. However, only the ones considered significant and possessing mature IT systems are required to undergo TLPT.

The Digital Operational Resilience Act requires the following financial entities to perform Threat Led Penetration Testing (TLPT):

1. Credit Institutions: Identified as Global Systemically Important Institutions (G-SIIs) or Other Systemically Important Institutions (O-SIIs).
2. Payment Institutions: Exceeding EUR 150 billion in payment transactions for each of the previous two financial years.
3. Electronic Money Institutions: Exceeding EUR 150 billion in payment transactions or EUR 40 billion in outstanding electronic money.
4. Central Securities Depositories and Central Counterparties.
5. Trading Venues: With the highest market share nationally in specific securities or derivatives for the past two years or with over 5% market share at the EU level.
6. Insurance and Reinsurance Undertakings: Meeting specific thresholds for gross written premiums, technical provisions, and total assets.

These entities are identified based on their systemic importance, ICT risk profile, and potential impact on financial stability. Additional entities can be opted in if they meet specific impact-related criteria or opted out if a detailed assessment indicates that TLPT is not justified.

The largest, most significant and most advanced entities may be required to go beyond the elements outlined in the regulatory technical standards. The RTS is to be understood as the minimum requirement for conducting TLPTs under DORA.

ICT third party providers

According to Article 23 of DORA,

“financial entities shall identify all relevant underlying ICT processes, systems and technologies supporting critical functions and services, including those contracted to ICT third-party service providers. Where ICT third-party service providers are included in the remit of the threat-led penetration testing, the financial entity shall take the necessary measures to ensure their participation.“

ESAs Report on the landscape of ICT third-party providers in the EU analyses the types of ICT third-party providers that are used critically in the financial sector. They are:

• Software and application services
• Network infrastructure services
• Data centers
• ICT consultancy & managed ICT services
• Information security & cybersecurity services
• Cloud service providers
• Data analysis and other data services

Testing methodology and process

The main participants of a Threat-Led Penetration Test (TLPT) include the financial entity undergoing the test, with a control team managing the process and a blue team defending the entity’s systems without prior knowledge of the test.

Additionally, the TLPT authority oversees and validates the testing process, often through a dedicated TLPT cyber team. External testers (red team) simulate real-world cyberattacks, and threat intelligence providers gather and analyze relevant threat data to create realistic attack scenarios. Each participant plays a crucial role in ensuring the TLPT’s effectiveness in identifying and mitigating potential vulnerabilities.

The TLPT process is structured into three main phases: preparation, testing, and closure.

Preparation phase

• Notification and initiation: Financial entities receive a notification from the TLPT authority indicating the requirement to perform a TLPT. Within three months of notification, entities must submit TLPT initiation documents, including a project charter, control team lead contact details, intended use of testers, communication channels, and a code name for the TLPT.
• Control Team formation: A control team is established, including a lead responsible for the TLPT’s day-to-day management. The team handles internal and external communications, risk management, and procurement of suitable threat intelligence providers and penetration testers.
• Scope specification: Within six months, the control team must submit a scope specification document detailing the critical or important functions to be tested. The document should be approved by the financial entity’s management body and validated by the TLPT authority.

Testing phase

The testing phase is divided into two main activities: threat intelligence gathering and red team testing.

• Threat intelligence gathering: The threat intelligence provider collects and analyzes relevant threat intelligence, identifying potential attack surfaces and threat scenarios. Next, the control team, testers, and TLPT authority select at least three attack scenarios for the test, with one scenario potentially being non-threat-led for forward-looking analysis.
• Red Team testing: Testers develop a red team test plan based on the scope specification and threat intelligence report that has to be reviewed and approved by the control team and TLPT authority.

The active red team testing phase lasts at least twelve weeks. During this time, testers simulate real-life cyber-attacks on the financial entity’s live production systems. Testers report weekly to the control team and TLPT authority, and adjustments to the test plan are made as necessary.

Closure Phase

The closure phase focuses on analyzing the results, replaying actions, and developing remediation plans. At this stage, the blue team is notified that TLPT has been performed. The use of the purple team is mandatory at this phase.

Reporting: Testers submit a red team test report to the control team within four weeks of completing the active testing phase. The control team shares the report with the blue team, which then submits a blue team test report within ten weeks.

Replay and Purple Teaming: The control team organizes replay and purple teaming exercises to review offensive and defensive actions taken during the TLPT. Purple teaming is a collaborative testing activity that involves both the red team and the blue team.

Remediation: Based on the TLPT findings, the control team prepares a remediation plan that addresses identified shortcomings, including a root cause analysis, proposed measures, prioritization, and implementation responsibilities.

What are the requirements for testers to carry out TLPT?

Requirements for testers are laid out in Article 27 of DORA. However, the technical standard further specifies the requirements. The external testers used for TLPT must:

• Provide certifications that align with recognized market standards for penetration testing and red team activities.
• Be fully covered by professional indemnity insurance, including coverage for risks related to misconduct and negligence.
• Provide at least five references from previous assignments specifically related to penetration testing and red team testing.
• Include a manager with at least five years of experience in penetration testing and red team testing.
• Include at least two additional testers, each with a minimum of two years of experience in penetration testing and red team testing.
• Have a broad and appropriate range of professional knowledge and skills, including understanding the financial entity’s business, skills in reconnaissance, risk management, exploit development, physical penetration, social engineering, and vulnerability analysis, and adequate communication skills to clearly present and report the engagement’s results.
• Have combined participation in at least five previous penetration and red team testing assignments.
• Must not be employed by or provide services to a provider that simultaneously performs blue team tasks for the financial entity, ICT third-party service provider, or an ICT intra-group service provider involved in the TLPT.
• Be separated from any staff of the same provider who are simultaneously providing threat intelligence services for the same TLPT.

In exceptional cases, financial entities may hire external testers and threat intelligence providers who don’t meet all the requirements as long as they mitigate and document the associated risks.

Use of internal testers

Unlike the TIBER-EU framework on which it is based, DORA permits the use of internal testers under specific conditions:

• Financial entities must establish a policy for managing internal testers, including criteria for assessing suitability, competence, and potential conflicts of interest.
• The internal testing team should consist of a test lead and at least two members, all employed by the financial entity or an ICT intra-group service provider for the preceding 12 months. Internal testers must receive training in penetration testing and red team testing.
• Entities must ensure that using internal testers does not negatively impact their defensive capabilities or the availability of resources for ICT-related tasks. Documentation of the use of internal testers is required in the TLPT initiation documents, red team test report, and the final TLPT summary report.

Financial entities using internal testers must ensure that:

• They use external testers upon every third test
• Testers undergo prior supervisory approval
• An external threat intelligence provider is used

Risk management for TLPT

Effective risk management is essential due to the inherent risks of threat-led penetration testing on live production systems:

• The control team must conduct a thorough risk assessment, considering potential impacts on data integrity, system functionality, and service continuity. Risk assessments should be reviewed and updated throughout the TLPT process.
• Measures must be taken to mitigate identified risks, including ensuring the suitability and competence of threat intelligence providers and testers. Specific measures include verifying certifications, references, and professional indemnity insurance for external providers.
• Testers and threat intelligence providers must conduct restoration procedures to secure data, remove malware, and restore systems to their original state post-testing.

Joint and pooled TLPTs

DORA and the technical standards allow for the use of pooled and joint threat-led assessments. Pooled TLPTs involve multiple financial entities and an ICT third-party service provider to efficiently test shared systems, while Joint TLPTs involve several financial entities using the same ICT intra-group service provider or belonging to the same group with common ICT systems.

Both approaches aim to streamline testing processes, reduce costs, and ensure thorough evaluation of interconnected ICT environments. In pooled tests, a designated financial entity leads, ensuring coordination, while joint tests emphasize collaboration among entities and authorities to achieve comprehensive and effective resilience assessments.

Conclusion

DORA was developed to ensure the digital operational resilience of the EU financial sector through consistent rules on reporting major ICT-related incidents, ICT third-party risk management, an ICT risk management framework and digital operational resilience testing.

While financial organizations prepare for the implementation of the Digital Operational Resilience Act, the publication of technical standards regarding threat-led penetration testing provides valuable guidance. The document was modified after consultations with the affected parties, particularly on reporting timelines and experience requirements for red team testers. These adjustments make the regulation more practical and relevant for real-world application.

TLPT is an advanced cybersecurity assessment that is a valuable part of critical financial organizations’ security strategies. By integrating threat intelligence and realistic attack simulations, organizations can better identify and address vulnerabilities to mitigate the ICT risks that the EU financial system continues to face.

The Draft Regulatory Technical Standards for TLPT under DORA, following adoption by the European Commission, are expected to be applied on 17 January 2025.

FAQ