"MITRE's MDR Stress-Test Winners Combine Human Intelligence and AI for Stronger Cybersecurity"
Cyberattacks have succeeded by exploiting gaps in corporate Information Technology (IT) environments, endpoints, and identities through social engineering and spear-phishing. They often immediately launch persistent threats and then steal credentials to move laterally across networks undetected. This breach sequence was chosen by MITRE for its first-ever closed-book titled "MITRE ATT&CK Evaluations for Security Service Providers." The ATT&CK evaluation is designed to assess providers' cybersecurity effectiveness. In order to keep evaluations open and fair, MITRE Engenuity ATT&CK evaluations are based on a knowledge base of tactics, techniques, and sub-techniques. The most widely used framework for evaluating enterprise systems and software security is MITRE's ATT&CK Matrix for Enterprise. Historically, MITRE ATT&CK evaluations have informed security vendors, prior to active testing, about the intrusion and breach attempts that will be tested and why. Vendors have been known to game evaluations with that advance information, resulting in inaccurate results. In a closed-book evaluation, vendors are unaware of the threats they will face during the test. "MITRE ATT&CK Evaluations for Security Service Providers" is the first closed-book evaluation designed to put vendors' Managed Services or Managed Detection and Response (MDR) solutions through a stress test. Closed-book assessments provide the most accurate picture of a security vendor's performance in a customer environment. According to Michael Sentonas, CrowdStrike's CTO, the closed book test provides an opportunity to demonstrate how security platforms operate against adversary tradecraft in a real-world setting because vendors have no prior knowledge to guide their actions. MITRE's assessment of MDRs is especially pertinent given that chronic cybersecurity skills shortages put organizations at greater risk of breach. The MITRE Security Service Providers evaluation lasted five days and had a reporting window of 24 hours. Sixteen MDR vendors who took part in the program had no prior knowledge of the adversary or its tactics, techniques, or procedures (TTPs). They were graded on a scale of 10 steps, each consisting of 76 events, including 10 unique ATT&CK tactics and 48 unique ATT&CK techniques. This article continues to discuss MITRE's first-ever closed-book MITRE Security Service Providers evaluation, combining human intelligence with Artificial Intelligence (AI) and Machine Learning (ML) in delivering the best results, and the importance of AI-assisted threat intelligence for an MDR.