The winning paper of the 5th Competition is You Get Where You’re Looking For: The Impact of Information Sources on Code Security (Free Open Access Copy) by Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L. Mazurek, Christian Stransky. These researchers are at CISPA, Saarland University in Germany and at The University of Maryland, College Park in the United States. The paper was presented at the 2016 IEEE Symposium on Security and Privacy ("Oakland").

This paper helps answer the question of why are software developers writing programs that have security vulnerabilities. The paper presents scientific evidence that confirms anecdotal stories that are in the programming community. Specifically, the researchers investigate how different information sources available to the developer influence the developer's abilities to quickly program and to program securely. They studied 54 developers (in Germany and the United States) in a controlled laboratory setting where they had them write security- and privacy relevant code under time constraints. They examined four conditions: 1) Developers were allowed to use any source; 2) Stack Overflow only; 3) Official Android Documentation only; and 4) books only. The results found that "Official API documentation is secure but hard to use, while informal documentation such as Stack Overflow is more accessible but often leads to insecurity. Interestingly, books (the only paid resource) perform well both for security and functionality. However, they are rarely used (in our study, one free choice participant used a book)."

This paper was selected for excelling at multiple attributes of high quality scientific work and reporting. First the authors developed laboratory study to control factors so they could accurately measure the information source variable and help determine the root cause of software vulnerabilities. These choices were based on their preliminary research in Android App developers where determined the best variable to measure. The research also included work to validate the results and they examined the limitations of their study. The paper did a thorough job explaining the research method which helps other researchers duplicate and build upon this work. The paper also has some actionable scientific based advice on developing better materials to have developers write more secure programs. This paper adds scientific knowledge to our understanding of how developers rely on information sources and the impact to the introduction of insecure software code.

Yasemin Acar is pursuing a Master's degree in mathematics and a PhD in Computer Science at Leibniz University of Hannover, Germany. Her research is focused on identifying causes that prevent developers from writing secure code, and helping them overcome those challenges. Her recent projects include evaluating the usability of cryptographic APIs and developing usable documentation for security-relevant APIs.

Michael Backes is the designated founding director of the CISPA Helmholtz Center for Information Security. He holds the chair for Information Security and Cryptography at Saarland University. Moreover, he is the speaker of the Collaborative Research Center on Online Privacy and of the CISPA-Stanford Center for Cybersecurity Research. He authored more than 200 scientific publications and received various scientific awards, in particular the ERC Synergy Grant (Europe's most distinguished research award), the ERC Starting Grant, the Microsoft Privacy Enhancing Technology Award, the Max Planck Fellowship, the IBM Faculty Award as well as the IBM Outstanding Achievement Award.

Sascha Fahl is head of the Information Security Institute in the Computer Science Department at the Leibniz University Hannover, Germany. He studies the intersection of computer security and privacy with human factors. Sascha is particularly interested in investigating end users, administrators, developers and designers of computer systems and their interdependencies with computer security and privacy mechanisms. His research involves large-scale analyses of the Internet and software repositories to understand the huge challenges humans face when interacting with computer security and privacy mechanisms. To understand root causes, evaluate existing mechanisms and investigate novel ideas, he conducts all kinds of user studies with end users, administrators, and developers of these systems. Sascha received his Ph.D. in Computer Science in 2016.

Michelle Mazurek is an Assistant Professor in the Computer Science Department and the Institute for Advanced Computer Studies at the University of Maryland, College Park. Her research aims to improve security- and privacy-related decision making by understanding people's needs and then building sound tools and systems. Recent projects include analyzing how users learn and process security advice; contrasting user expectations with app behavior in Android apps; examining convenience/security tradeoffs in end-to-end encryption; and examining how and why developers make security and privacy mistakes. Mazurek received her Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University in 2014.

Christian Stransky is pursuing his PhD in computer science at Saarland University, Germany. His research is targeted at identifying and understanding usability issues in the toolchain and libraries of developers that result in security or privacy problems. His recent projects include evaluating the usability of security APIs, exploring different samples and their scientific merit for developer studies and providing a tool for other researchers to conduct developer studies. 

In order to encourage the development of the scientific foundations of cybersecurity, the National Security Agency (NSA) established The Annual Best Scientific Cybersecurity Paper Competition. NSA invites nominations of papers that show an outstanding contribution to cybersecurity science. A set of Distinguished Experts will review the nominations according to the criteria below. Awardees will be invited to NSA to receive the award and present the winning paper to an audience of cybersecurity experts.

Papers published in peer-reviewed journals, magazines, or technical conferences are eligible for nomination. The date of the publication must be between January 1st 2016 and December 31st 2016. Nominations should include, in 500 words or less, a nomination statement describing the scientific contribution of the paper and explaining why this paper merits the award. A strong nomination statement is desired and will be used as part of the criteria when evaluating paper submissions. Nominated papers must be available in English and pdf format. Nominations must be submitted via this site. The nominator may not be an author or co-author of the nominated paper. If a paper includes a reviewer as a co-author it may not be considered for an award. Papers may come from any field of cybersecurity research. (Please refer to the SoS-VO discussion forum What is Security Science?)

A set of distinguished experts will review the submitted nominations and provide individual assessments to the NSA Research Directorate.

The following individuals served as distinguished experts for the 5th annual competition:

For additional information on the review team.

The NSA Research Directorate will recommend awardees to the NSA Director of Research, whose decision will be final. Considerations in the evaluation of the nominated paper may include:

Scientific merit and significance of the work reported, the degree to which the paper exemplifies how to perform and report scientific research in cybersecurity.

Submission Period Begins: December 15, 2016
Submission Period for Entries Ends: March 31, 2017 11:59 PM, EST.
Evaluation Process for Entries Begins: April 1, 2017
Winners Notified: By September 15, 2017
Winners Announced: Fall 2017