Effective response to security attacks often requires a combination of both automated and human-mediated actions. Currently we lack adequate methods to reason about such human-system coordination, including ways to determine when to allocate tasks to each party and how to gain assurance that automated mechanisms are appropriately aligned with organizational needs and policies. In this project, we develop a model-based approach to (a) reason about when and how systems and humans should cooperate with each other, (b) improve human understanding and trust in automated behavior through self-explanation, and (c) provide mechanisms for humans to correct a system’s automated behavior when it is inappropriate. We will explore the effectiveness of the techniques in the context of coordinated system-human approaches for mitigating advanced persistent threats (APTs).

Building on prior work that we have carried out in this area, we will show how probabilistic models and model checkers can be used both to synthesize complex plans that involve a combination of human and automated actions, as well as to provide human understandable explanations of mitigation plans proposed or carried out by the system. Critically, these models capture an explicit value system (in a multi-dimensional utility space) that forms the basis for determining courses of action. Because the value system is explicit we believe that it will be possible to provide a rational explanation of the principles that led to a given system plan. Moreover, our approach will allow the user to make corrective actions to that value system (and hence, future decisions) when it is misaligned. This will be done without a user needing to know the mathematical form of the revised utility reward function.

Machine-learning algorithms, especially classifiers, are becoming prevalent in safety and security-critical applications. The susceptibility of some types of classifiers to being evaded by adversarial input data has been explored in domains such as spam filtering, but with the rapid growth in adoption of machine learning in multiple application domains amplifies the extent and severity of this vulnerability landscape. We propose to (1) develop predictive metrics that characterize the degree to which a neural-network-based image classifier used in domains such as face recognition (say, for surveillance and authentication) can be evaded through attacks that are both practically realizable and inconspicuous, and (2) develop methods that make these classifiers, and the applications that incorporate them, robust to such interference. We will examine how to manipulate images to fool classifiers in various ways, and how to do so in a way that escapes the suspicion of even human onlookers. Armed with this understanding of the weaknesses of popular classifiers and their modes of use, we will develop explanations of model behavior to help identify the presence of a likely attack; and generalize these explanations to harden models against future attacks.

Critical infrastructure is increasingly comprised of distributed, inter--‐dependent components and information that is vulnerable to sophisticated, multi--‐stage cyber--‐attacks.  These attacks are difficult to understand as isolated incidents, and thus to improve understanding and response, organizations must rapidly share high quality threat, vulnerability and exploit--‐related, cyber--‐security information.  However, pervasive and ubiquitous computing has blurred the boundary between work--‐related and personal data.  This includes both the use of workplace computers for personal purposes, and the increase in publicly available, employee information that can be used to gain unauthorized access to systems through attacks targeted at employees. 
 

To address this challenge, we envision a two part solution that includes: (1) the capability to assign information category tags to data “in transit” and “at rest” using an ontology that describes what information is personal and non--‐personal; and (2) a scoring algorithm that computes the “privacy risk” of some combination of assigned tags.

Anonymity is a basic right and a core aspect of Internet. Recently, there has been tremendous interest in anonymity and privacy in social networks, motivated by the natural desire to share one’s opinions without the fear of judgment or personal reprisal (by parents, authorities, and the public). We propose to study the fundamental questions associated with building such a semi-distributed, anonymous messaging platform, which aims to keep anonymous the identity of the source who initially posted a message as well as the identity of the relays who approved and propagated the message.

• Community Development - The goal is to build an extended and vibrant interdisciplinary community of science of security researchers, research methodologists, and practitioners (Carver, Williams).
• Community Resources - To create and maintain a repository of defensible scientific methods for security research (Carver, Williams).
• Oversight for the Application of Defensible Scientific Research Methodologies - To encourage the application of scientifically defensible research through various methods of consultation and feedback (Carver).
• Usable Data Sharing - To enable open, efficient, and secure sharing of data and experimental results for experimentation among SoS researchers (Al-Shaer).

• Contributions to Developing a Science of Security - We will design and implement an evaluation process for assessing the effectiveness and impact of the Lablet's research and community development activities (McGowen, Stallings, & Wright).
• Contributions to Security Science Research Methodology - We will examine both the impact of Lablet work on the maturity of the SoS field and the methodological rigor of the Lablet research projects themselves (McGowen, Carver).
• Development of a Community of Practice for the Science of Security - We will develop methods to assess whether Lablet activities are contributing to the development of a sustainable community of practice for the SoS field (McGowen, Stallings, Carver, & Wright).

The patterns and characteristics of security incidents are a significant driver of security technology innovation. Patterns are detected by analyzing repositories of malware/viruses/worms, incidents affecting control/SCADA systems, general security alerts and updates, and data breaches. For most types of privacy incidents there are no repositories. Privacy incidents that do not involve a security breach, such as cyber-bullying/slander/stalking, revenge porn, social media oversharing, data reidentification and surveillance, are not represented in the current repositories.  Our project is building the first comprehensive encyclopedia and database of privacy incidents. This publicly-accessible repository will enable tracking of incident rates and characteristics such as involved entities and incident root causes. The repository will provide a resource for privacy researchers to investigate the patterns of a broad range of privacy incidents, and the incident patterns surfaced by the database will help inform privacy technology development globally.

Security-Metrics-Driven-Evaluation, Design, Development and Deployment. Our research evaluates security pattern selection and application by designers in response to attack patterns. The evaluation is based on formal models of attack scenarios that are used to measure security risk and promote risk reduction strategies based on assurance cases constructed by the analyst. The aim is to improve the usability of formal methods for studying security design and composition.

Understanding and Accounting for Human Behavior. Our research is based on theory in psychology concerning how designers comprehend and interpret their environment, how they plan and project solutions into the future, with the aim of better understanding how these activities exist in designing more secure systems. These are not typical models of attackers and defenders, but models of developer behavior, including our ability to influence that behavior with interventions.