An Assessment Methodology, Models for National Security Systems

Abstract:

The assurance of National Security Systems (NSS), like all computer systems, is measured, or assessed, by a variety of methodologies and assessors. Most assessors know that the level of assurance required for each system is dependent upon dynamic factors such as attack vector persistence, operational environment, and probability of a successful attack, regardless of its complexity or connectivity. This paper presents a methodology that implements mathematical models that are simple enough for non-mathematicians to use, can be integrated into existing acceptance and certification methodologies, or can be implemented standalone, and is based upon lessons learned from over a decade of direct, real world, assessment experience.  

In assessments, evidence must be collected and assessed against a model.  In existing assessment methodologies, that model is a complete, live implementation of a system in a single operational environment. Each operational environment may contain multiple situational instances or states of physical characterizations (such as an aircraft in flight vice parked). 

This paper presents mathematical models detailing various individual factors that contribute to an aggregate measure, including the operational environment states, flaws, countermeasures, vulnerabilities, threats, probabilities, attack vectors, impacts, and risk. The models provide a basis for a new assessment methodology that can be combined with the current and future assessment methodologies, improve confidence in the system by requiring an independent assessor to be integrated into the development process to achieve greater insight, and improve cost savings by preventing duplicate assessments and reducing the time it takes to conduct assessments by allowing future assessments to build on the findings of past assessments.  

This methodology provides the ability to model system states to characterize dynamic aspects of the system and environment. Computers alter states every time a decision is completed. So, computers and networks exist in fluidity, each constantly changing. The models need to represent systems in multiple states based on dynamic aspects, analogous to the modeling used in weather forecast models, nuclear explosions, and disease infection rates. This type of modeling provides objective evidence throughout the assessment.

The proposed methodology provides to the assessor mechanisms to map the evidence to mathematical models to assessor’s findings. Currently, assessors must rely on documentation provided by the vendor, which can be biased. An Information Systems Security Engineer (ISSE) is key to the entire methodology as it removes any possible bias from a vendor, design team, program manager, command, etc, and the ISSE can provide mathematical foundations supporting evidence creation. The use of the models increases objectiveness, repeatability, and knowledge of system robustness from ISSE to risk acceptor, as well as ISSE to ISSE. 

The methodology can be implemented at any time within the development lifecycle of a system.  The earlier in the lifecycle the methodology is implemented, the greater the applicability of evidence that is available to the ISSE.  In addition, the methodology strongly integrates the ISSE with system’s developers and engineers.  An ISSE that is involved in the system development processes starting at design conception, can increase the measure of confidence in the assurance of the system by identifying applicable supplementary artifacts, and through the use of subject matter expertise, increase the quality of all assurance evidence. 

Individual models will be iteratively addressed so that the ISSE is able to represent each impression of the system’s capabilities, correlate the models to the evidence, and provide a level of assessment detail that has heretofore not been provided. As the ISSE’s knowledge of the system increases, the content of these models will go from generalized to specific as the assessment progresses. These individual models will build into the overall assessment model. The individual models will be iteratively developed, fulfilling the needs of the assessor to represent their initial impression of the system’s capabilities, represent the system’s capabilities as it is assessed, and finally, to representatively correlate or map the completed models to the empirical evidence of the assessment.

Within the proposed methodology, there are multiple stages, with each stage correlating to the progression of the assessor’s exposure to the system. At each stage, the ISSE iterates the individual models to represent their impression of the system’s capabilities. As each assessment is individualistic, the number of stages and the stage at which a model is created will vary wildly based upon the system functionality, and the point in the lifecycle in which the system enters the methodology, and the information available at that the time. 

Bio:

Jennifer Guild is a PhD candidate at the University of Idaho who is employed as a computer scientist by the US Navy. She specializes in the assessment of complex systems, such as Cross Domain Solutions. Ms. Guild received an MS in Computer Science from the US Naval Postgraduate School.