Spotlight on Lablet Research #14 - A Human Agent-Focused Approach to Security Modeling
Spotlight on Lablet Research #14 -
Project: A Human Agent-Focused Approach to Security Modeling
Lablet: University of Illinois at Urbana-Champaign
The aim of this project, which concluded in 2020, was to make fundamental advances in scientifically-motivated techniques to aid risk assessment for computer security through the development of a general-purpose, easy-to-use formalism. This formalism will allow for realistic modeling of cyber systems and all human agents that interact with the system with the ultimate goal of generating quantitative results that will help system architects make better design decisions.
The hypothesis is that models that incorporate all human agents who interact with the system will produce insightful metrics. System architects can leverage the results to build more resilient systems that are able to achieve their mission objectives despite attacks.
Researchers began by conducting a literature review with the goal of constructing a high-quality case study to exercise the human-centric cybersecurity modeling formalism being developed. Their case study focused on comparing the security and usability of different password policies (e.g., password length, time until the password expires, etc.), which a hypothetical institution may enact. They constructed submodels of the institutions, their employees and customers, and the adversaries. Then, they composed these submodels and studied the interaction to give insight into the relative strengths and weaknesses of the password policies. The model was validated by using previously-conducted studies of human behavior with regard to passwords.
UIUC researchers extended their work focused on a metamodeling-based approach to sensitivity analysis and uncertainty quantification in complex security models. Many realistic security models run slowly and have input variables whose values are uncertain, making it difficult to conduct sensitivity analysis and uncertainty quantification. It is possible to create metamodels of the base security model that trade some accuracy for speed using machine learning techniques. Researchers had earlier investigated this method by applying it to a previously-published work that models the growth of peer-to-peer botnets, and they then applied it to two new models to test its general applicability.
The researchers also investigated two ways to solve an issue with applying the metamodeling approach to certain models that contained a mix of quantitative and qualitative input variables. The two approaches were one-hot encoding and splitting. They implemented the two approaches and evaluated them on an AMI ADVISE model, and found, at least in that one case, that splitting substantially outperformed one-hot encoding. This work can help modelers apply the metamodeling approach that was developed to a broader class of security models. The metamodeling approach helps modelers perform sensitivity analysis and uncertainty quantification on complex slow-running security models that contain uncertain input variables.
Researcher Michael Rausch and Principal Investigator William Sanders were awarded the Best Paper Award for "Sensitivity Analysis and Uncertainty Quantification of State-Based Discrete-Event Simulation Models through a Stacked Ensemble Metamodels," at the 17th Annual International Conference on Quantitative Evaluation of SysTems (QEST), 2020.