Our client requested an in-depth security control assessment to determine their organization’s coverage against the 280+ tactics, techniques, and procedures (TTPs) cataloged by MITRE ATT&CK. After evaluation of the security controls, the coverage observations were captured and a custom Splunk application was developed to display the security control effectiveness via a heatmap of the MITRE ATT&CK framework. This allowed the organization to easily visualize their security control effectiveness and quickly identify areas of improvement.
Roles and Responsibilities:
- Lead Cybersecurity Engineer – As lead, my responsibility was to co-ordinate with our Cyber Threat Intelligence (CTI) lead and project manager for project execution. Prior to application development, we held several meetings to take client requirements for application function, evaluated the security controls in the environment, and incorporated CTI data from the Active Persistent Threat (APT) groups that the organization had already profiled into our assessment of the organization’s security posture.
- Security Tool Owner and Technical Account Manager (TAM) Interviews – In order to verify security tool coverage with high fidelity, the owners of each security tool were interviewed to understand tool deployment and configuration, giving us a granular look into the tool’s effectiveness against the TTPs in the MITRE ATT&CK framework. In instances where tool capability could not be fully established, the client’s TAM was also interviewed for further clarity.
- Security Tool Effectiveness Quantification – As each security tool in the environment was profiled, an “Effectiveness Score” was assigned to the tool to determine its efficacy against the specific MITRE ATT&CK tactic in question. Each tool was compared against the 280+ MITRE ATT&CK TTPs and assigned a score 0 – 5 regarding coverage, where 0 meant the tool was not capable of detecting the technique, 1 was not very effective, and 5 was complete coverage (full effectiveness).
- Splunk Application Development/Sole Developer – A custom Splunk application was developed to fully support project delivery and create a visual representation of our security assessment outputs. I wrote this application in Python 3 using the pandas library to create a visualized matrix that represents the MITRE ATT&CK framework. To create the heatmap, the quantification scores were imported and assigned different colors, ranging from red to green (least effective to most effective). Lastly, the CTI data from threat groups were also incorporated which ultimately allowed me to create the application with three main views:
- Overall Effectiveness Overview – The MITRE ATT&CK matrix was displayed and the effectiveness scores from all tools were shown to visibly show which techniques the organization had coverage for and if so, what level of coverage did they have.
- Tool Effectiveness Overview – A dropdown was created to select each of the profiled tools in the environment, and the matrix was displayed to convey which techniques the tool had coverage for as well as the level to which it offered coverage.
- APT Group Overview – APT groups were imported to profile the threat actors that were of concern to the organization, as well as the TTPs that these threat groups used. This dropdown allowed the organization to see, at a quick glance, which threat groups are being tracked and visualize their coverage against the TTPs use by those groups.
- Documentation and Future Proofing – After development of the Splunk application, documentation was created to provide the organization with the necessary details to extend and expand the application should they require additional functionality. Additionally, an “updater” module was built into the app to ensure that as new TTPs were introduce by MITRE, they were incorporated into the application.
- Ex. During the engagement, MITRE released the “Impact” Tactic and the application was able to pull in these techniques and update the matrix for all of the views.
