One major shortcoming of the current "notice and consent" privacy framework is that the constraints for data usage stated in policies—be they stated privacy practices, regulation, or laws—cannot easily be compared against the technologies that they govern. To that end, we are developing a framework to automatically compare policy against practice. Broadly, this involves identifying the relevant data usage policies and practices in a given domain, then measuring the real-world exchanges of data restricted by those rules. The results of such a method will then be used to measure and predict the harms brought onto the data’s subjects and holders in the event of its unauthorized usage. In doing so, we will be able to infer which specific protected pieces of information, individual prohibited operations on that data, and aggregations thereof pose the highest risks compared to other items covered by the policy. This will shed light on the relationship between the unwanted collection of data, its usage and dissemination, and resulting negative consequences.

We have built infrastructure into the Android operating system, whereby we have heavily instrumented both the permission-checking APIs and included network-monitory functionality. This allows us to monitor when an application attempts to access protected data (e.g., PII, persistent identifiers, etc.) and what it does with it. Unlike static analysis techniques, which only detect the potential for certain behaviors (e.g., data exfiltration), executing applications with our instrumentation yields real-time observations of actual privacy violations. The only drawback, however, is that applications need to be executed, and broad code coverage is desired. To date, we have demonstrated that many privacy violations are detectable when application user interfaces are “fuzzed” using random input. However, there are many open research questions about how we can yield better code coverage to detect a wider range of privacy- related events, while doing so in a scalable manner. Towards that end, we plan to virtualize our privacy testbed and integrate crowd-sourcing. By doing this, we will develop new methods for performing privacy experiments that are repeatable, rigorous, and gen- eralizable. The results of these experiments can then be used to implement data-driven privacy controls, address gaps in regulation, and enforce existing regulations.

This project seeks to aid security analysts in identifying and protecting against accidental and malicious actions by users or software through automated reasoning on unified representations of user expectations and software implementation to identify misuses sensitive to usage and machine context.

Systems that are technically secure may still be exploited if users behave in unsafe ways. Most studies of user behavior are in controlled laboratory settings or in large-scale between-subjects measurements in the field. Both methods have shortcomings: lab experiments are not in natural environments and therefore may not accurately capture real world behaviors (i.e., low ecological validity), whereas large-scale measurement studies do not allow the researchers to probe user intent or gather explanatory data for observed behaviors, and they offer limited control for confounding factors. The SBO addresses the gap through a panel of participants consenting to our observing their daily computing behavior in situ, so we can understand what constitutes “insecure” behavior. We use the security behavior observatory to attempt to answer a number of research questions, including 1) What are risk indicators of a user’s propensity to be infected by malware?  2) Why do victims fail to update vulnerable software in a timely manner? 3) How can user behavior be modeled with respect to security and privacy “in the wild”?

Typically, securing software is the responsibility of the software developer. The customer or end-user of the software does not control or direct the steps taken by the developer to employ best practice coding styles or mechanisms to ensure software security and robustness. Current systems and tools also do not provide the end-user with an ability to determine the level of security in the software they use. At the same time, any flaw or security vulnerabilities ultimately affect the end-user of the software. Therefore, our overall project aim is to provide greater control to the end-user to actively assess and secure the software they use.

Our project goal is to develop a high-performance framework for client-side security assessment and enforcement for binary software. Our research is developing new tools and techniques to: (a) assess the security level of binary executables, and (b) enhance the security level of binary software, when and as desired by the user to protect the binary against various classes of security issues. Our approach combines static and dynamic techniques to achieve efficiency, effectiveness, and accuracy.

CPS systems routinely employ custom off-the-shelf (COTS) applications and binaries to realize their overall system goals. COTS applications for CPS are typically programmed using unsafe languages, like C/C++ or assembly. Such programs are often plagued with memory and other vulnerabilities that attackers can exploit to compromise the system.

There are many issues that need to be explored and resolved to provide security in this environment. For instance; (a) different systems may desire distinct and customizable levels of protection (for the same software), (b) different systems may have varying tolerances to the performance and/or timing penalties imposed by existing security solutions, and hence a solution applicable in one case may not be appropriate to a different system, (c) multiple solutions to the same vulnerability/attack may impose varying levels of security and performance penalties. Such tradeoffs and comparisons with other potential solutions are typically unknown or unavailable to users, and (d) solutions to newly discovered attacks and improvements to existing solutions continue to be devised. There is currently no efficient mechanism to retrofit existing application binaries with new security patches with minimal disruption to system operation.

The goal of this research is to design a mechanism to: (a) analyze and quantify the level of security provided and performance penalty imposed by different solutions to various security risks affecting native binaries, and (b) to study and build an architecture that can efficiently and adaptively patch vulnerabilities or retrofit COTS applications with chosen security mechanisms with minimal disruption.

Successful completion of this project will result in:

• Exploration and understanding of the security and performance tradeoffs imposed by different proposed solutions to important software problems.
• Discovery of (a set of) mechanisms to reliably retrofit desired compiler-level, instrumentation-based, or other user-defined security solutions into existing binaries.
• Study and resolution of the issues involved in the design and construction of an efficient production-quality framework to realize the proposed goals.

CPS systems routinely employ custom off-the-shelf (COTS) applications and binaries to realize their overall system goals. COTS applications for CPS are typically programmed using unsafe languages, like C/C++ or assembly. Such programs are often plagued with memory and other vulnerabilities that attackers can exploit to compromise the system.

There are many issues that need to be explored and resolved to provide security in this environment. For instance; (a) different systems may desire distinct and customizable levels of protection (for the same software), (b) different systems may have varying tolerances to the performance and/or timing penalties imposed by existing security solutions, and hence a solution applicable in one case may not be appropriate to a different system, (c) multiple solutions to the same vulnerability/attack may impose varying levels of security and performance penalties. Such tradeoffs and comparisons with other potential solutions are typically unknown or unavailable to users, and (d) solutions to newly discovered attacks and improvements to existing solutions continue to be devised. There is currently no efficient mechanism to retrofit existing application binaries with new security patches with minimal disruption to system operation.

The goal of this research is to design a mechanism to: (a) analyze and quantify the level of security provided and performance penalty imposed by different solutions to various security risks affecting native binaries, and (b) to study and build an architecture that can efficiently and adaptively patch vulnerabilities or retrofit COTS applications with chosen security mechanisms with minimal disruption.

Despite the success of Contextual Integrity (see project "Operationalizing Contextual Integrity"), its uptake by computer scientists has been limited due to the philosophical framework not meeting them on their terms. In this project we will both refine Contextual Integrity (CI) to better fit the problems computer scientists face and to express it in the mathematical terms they expect.

According to the theory of CI, informational norms are specific to social contexts (e.g., healthcare, education, commercial marketplace, political citizenship, etc.). Increasing interest in context as a factor in computer science research marks important progress toward a more nuanced interpretation of privacy. It is clear, however, that context takes on many meanings across these research projects. As noted above, Contextual Integrity is committed to context as social domain, or sphere, while some works have used the term to mean situation, physical surroundings, or even technical platform. In this project, we will disentangle the many meanings of context and expand the CI framework using formal models to show how these meanings are logically linked. We are exploring how precisely differentiating between situation and sphere can make CI more actionable. For example, this differentiation will help disentangle cases where a single situation participates in more than one sphere, or when information flows inappropriately from one situation to another. To make the de-conflated notions of context crisp, we are developing formal models for each notion of contexts with clear explanations of which applies in which setting. We are attempting to model the central notion of concept found in CI using Markov Decision Processes to capture that most contexts are organized around some goal (e.g., healthcare).

Privacy skeptics have cited variations across nations, cultures, and even individuals as proof that privacy is not a fundamental, but more like a preference. The lesson for designers, for example, is to assess preferences in order to succeed within the marketplace of their targeted users. The explanation CI offers is that differences in privacy norms are due to differences in societal structures and the function of values of specific contexts within those structures. But, because societies change over time, sometimes radically through revolutionary shifts, a theory of privacy must allow for changes in privacy norms. In the present time, revolutionary shifts are being forced by computer science and technology. Take, for example, a social platform such as a classroom discussion board and assume one has implemented Contextual Integrity, preventing flows from taking place that conflict with educational privacy norms. Assume, also, that norms change over time due to changes in technical practices and the educational system itself (e.g., the introduction of MOOCs). How might such systems adapt? We are laying the groundwork for understanding this problem by developing formal models of context and norm drift over time. We will augment the formal models of context mentioned above with with notions of change drawing inspiration from temporal logics.

CI and differential privacy (DP) both claim to define privacy as it applies to data flow. The former, as we have seen, offers a systematic account for what people mean when protesting that privacy is under threat, or is violated by systems that collect, accumulate, and analyze data; the latter offers a mathematical property of operations that process data as a definition of privacy that is robust, meaningful, and mathematically rigorous. For this project, another driving question is the relationship between CI and DP. For example, DP may be understood as one kind of transmission principle, but DP does not capture other socially meaningful transmission principles, such as reciprocity, confidentiality, and notice. Thus, we are also cataloging the wide range of transmission principles relevant to privacy and showing where DP is a useful mathematical expression. This will allow us to derive other mathematically rigorous specifications for other transmission principles.

Machine-learning algorithms, especially classifiers, are becoming prevalent in safety and security-critical applications. The susceptibility of some types of classifiers to being evaded by adversarial input data has been explored in domains such as

spam filtering, but with the rapid growth in adoption of machine learning in multiple application domains amplifies the extent and severity of this vulnerability landscape. We propose to (1) develop predictive metrics that characterize the degree to which a

neural-network-based image classifier used in domains such as face recognition (say, for surveillance and authentication) can be evaded through attacks that are both practically realizable and inconspicuous, and (2) develop methods that make these classifiers, and the applications that incorporate them, robust to such interference. We will examine how to manipulate images to fool classifiers in various ways, and how to do so in a way that escapes the suspicion of even human onlookers. Armed with this

understanding of the weaknesses of popular classifiers and their modes of use, we will develop explanations of model behavior to help identify the presence of a likely attack; and generalize these explanations to harden models against future attacks.

We have developed the SURE platform, a modeling and simulation integration testbed for evaluation of resilience for complex CPS [1].  Our previous efforts resulted in a web-based collaborative design environment for attack-defense scenarios supported by a cloud-deployed simulation engine for executing and evaluating the scenarios. The goal of this project is to extend these design and simulation capabilities for better understanding the security and resilience aspects of CPS systems. These improvements include the first class support for the design of experiments (exploring different parameters and/or strategies),  alternative CPS domains (connected vehicles, railway systems, smart grid), incorporating models of human behavior, and executing multistage games. 

[1] Xenofon Koutsoukos, Gabor Karsai, Aron Laszka, Himanshu Neema, Bradley Potteiger, Peter Volgyesi, Yevgeniy Vorobeychik, and Janos Sztipanovits. "SURE: A Modeling and Simulation Integration Platform for Evaluation of SecUre and REsilient Cyber-Physical Systems", Proceedings of the IEEE, 106(1), 93-112, January 2018.

The goals of this project are to develop the principles and methods for designing and analyzing resilient CPS architectures that deliver required service in the face of compromised components. A fundamental challenge is to understand the basic tenets of CPS resilience and how they can be used in developing resilient architectures. The proposed approach integrates redundancy, diversity, and hardening methods for designing either passive resilience methods that are inherently robust against attacks and active resilience methods that allow responding to attacks.