Methods, approaches, and tools to identify the correct conceptualization of privacy early in the design and engineering process are important. For example, early whole body imaging technology for airport security were analyzed by the Department of Homeland Security through a Privacy Impact Assessment, focusing on the collection of personally identifiable information finding that the images of persons’ individual bodies were not detailed enough to constitute PII, and would not pose a privacy problem. Nevertheless, many citizens, policymakers, and organizations subsequently voiced strong privacy ob- jections: the conception of privacy as being about the collection of PII did not cover the types of privacy concerns raised by stakeholders, leading to expensive redesigns to ad- dress the correct concepts of privacy (such as having the system display an outline of a generic person rather than an image of the specific person being scanned). In this project, we will investigate current tools, methods and approaches being utilized by engineers and designers to identify and address privacy risks and harms.

To help address gaps and shortcomings that we find in current tools and approaches, we are adapting design research techniques—traditionally used to help designers and engineers to explore and define problem spaces in grounded, inductive, and generative ways—to specifically address privacy. This builds on a tradition of research termed "values in design," which seeks to identify values and create systems that better recognize and address them. Design methods, including card activities, design scenarios, design workbooks, and design probes, can be used by engineers or designers of systems, and/or can be used with other stakeholders of systems (such as end-users). These methods help foster discussion of values, chart the problem space of values, and are grounded by specific contexts or systems. These methods can be deployed during early ideation stages of a design process, during or after the design process as an analytical tool, or as part of training and educating. We suggest that design approaches can help explore and define the problem space of privacy and identify and define privacy risks (including, but also going beyond unauthorized use of data), leveraging the contextual integrity framework.

As part of this project, we are creating, testing, validating, and deploying a set of privacy-focused tools and approaches that can be used to help train engineers and designers to identify, define and analyze the privacy risks that need to be considered when designing a system, as part of privacy engineering.

1. Select [Edit]
2. Please add the project description here.
3. Add the names of the PIs, Co-PIs, and Researchers below (they must have accounts on the VO)
4. Select the correct hard problem from the vocabulary terms on the right

This project considers models for secure collaboration and contracts in a decentralized environment among parties that have not established trust. A significant example of this is blockchain programming, with platforms such as Ethereum and HyperLedger. There are many documented defects in secure collaboration mechanisms, and some have been exploited to steal money.  Our approach builds two kinds of models to address these defects: typestate models to mitigate re-entrancy-related vulnerabilities, and linear types to model and statically detect an important class of errors involving money and other transferrable resources.

The project research will include both technical and usability assessment of these two ideas. The technical assessment addresses the feasibility of sound and composable static analyses to support these two semantic innovations. The usability assessment focuses on the ability of programmers to use Obsidian effectively to write secure programs with little training.  A combined assessment would focus on whether programmers are more likely to write correct, safe code with Obsidian than with Solidity, and with comparable or improved productivity.

Effective response to security attacks often requires a combination of both automated and human-mediated actions. Currently we lack adequate methods to reason about such human-system coordination, including ways to determine when to allocate tasks to each party and how to gain assurance that automated mechanisms are appropriately aligned with organizational needs and policies. In this project, we develop a model-based approach to (a) reason about when and how systems and humans should cooperate with each other, (b) improve human understanding and trust in automated behavior through self-explanation, and (c) provide mechanisms for humans to correct a system’s automated behavior when it is inappropriate. We will explore the effectiveness of the techniques in the context of coordinated system-human approaches for mitigating advanced persistent threats (APTs).

Building on prior work that we have carried out in this area, we will show how probabilistic models and model checkers can be used both to synthesize complex plans that involve a combination of human and automated actions, as well as to provide human understandable explanations of mitigation plans proposed or carried out by the system. Critically, these models capture an explicit value system (in a multi-dimensional utility space) that forms the basis for determining courses of action. Because the value system is explicit we believe that it will be possible to provide a rational explanation of the principles that led to a given system plan. Moreover, our approach will allow the user to make corrective actions to that value system (and hence, future decisions) when it is misaligned. This will be done without a user needing to know the mathematical form of the revised utility reward function.

Machine-learning algorithms, especially classifiers, are becoming prevalent in safety and security-critical applications. The susceptibility of some types of classifiers to being evaded by adversarial input data has been explored in domains such as spam filtering, but with the rapid growth in adoption of machine learning in multiple application domains amplifies the extent and severity of this vulnerability landscape. We propose to (1) develop predictive metrics that characterize the degree to which a neural-network-based image classifier used in domains such as face recognition (say, for surveillance and authentication) can be evaded through attacks that are both practically realizable and inconspicuous, and (2) develop methods that make these classifiers, and the applications that incorporate them, robust to such interference. We will examine how to manipulate images to fool classifiers in various ways, and how to do so in a way that escapes the suspicion of even human onlookers. Armed with this understanding of the weaknesses of popular classifiers and their modes of use, we will develop explanations of model behavior to help identify the presence of a likely attack; and generalize these explanations to harden models against future attacks.