The winning paper of the 5th Competition is You Get Where You’re Looking For: The Impact of Information Sources on Code Security (Free Open Access Copy) by Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L. Mazurek, Christian Stransky. These researchers are at CISPA, Saarland University in Germany and at The University of Maryland, College Park in the United States. The paper was presented at the 2016 IEEE Symposium on Security and Privacy ("Oakland").

This paper helps answer the question of why are software developers writing programs that have security vulnerabilities. The paper presents scientific evidence that confirms anecdotal stories that are in the programming community. Specifically, the researchers investigate how different information sources available to the developer influence the developer's abilities to quickly program and to program securely. They studied 54 developers (in Germany and the United States) in a controlled laboratory setting where they had them write security- and privacy relevant code under time constraints. They examined four conditions: 1) Developers were allowed to use any source; 2) Stack Overflow only; 3) Official Android Documentation only; and 4) books only. The results found that "Official API documentation is secure but hard to use, while informal documentation such as Stack Overflow is more accessible but often leads to insecurity. Interestingly, books (the only paid resource) perform well both for security and functionality. However, they are rarely used (in our study, one free choice participant used a book)."

This paper was selected for excelling at multiple attributes of high quality scientific work and reporting. First the authors developed laboratory study to control factors so they could accurately measure the information source variable and help determine the root cause of software vulnerabilities. These choices were based on their preliminary research in Android App developers where determined the best variable to measure. The research also included work to validate the results and they examined the limitations of their study. The paper did a thorough job explaining the research method which helps other researchers duplicate and build upon this work. The paper also has some actionable scientific based advice on developing better materials to have developers write more secure programs. This paper adds scientific knowledge to our understanding of how developers rely on information sources and the impact to the introduction of insecure software code.

In order to encourage the development of the scientific foundations of cybersecurity, the National Security Agency (NSA) established The Annual Best Scientific Cybersecurity Paper Competition. NSA invites nominations of papers that show an outstanding contribution to cybersecurity science. A set of Distinguished Experts will review the nominations according to the criteria below. Awardees will be invited to NSA to receive the award and present the winning paper to an audience of cybersecurity experts.

Papers published in peer-reviewed journals, magazines, or technical conferences are eligible for nomination. The date of the publication must be between January 1st 2016 and December 31st 2016. Nominations should include, in 500 words or less, a nomination statement describing the scientific contribution of the paper and explaining why this paper merits the award. A strong nomination statement is desired and will be used as part of the criteria when evaluating paper submissions. Nominated papers must be available in English and pdf format. Nominations must be submitted via this site. The nominator may not be an author or co-author of the nominated paper. If a paper includes a reviewer as a co-author it may not be considered for an award. Papers may come from any field of cybersecurity research. (Please refer to the SoS-VO discussion forum What is Security Science?)

A set of distinguished experts will review the submitted nominations and provide individual assessments to the NSA Research Directorate.

The following individuals served as distinguished experts for the 5th annual competition:

For additional information on the review team.

The NSA Research Directorate will recommend awardees to the NSA Director of Research, whose decision will be final. Considerations in the evaluation of the nominated paper may include:

Scientific merit and significance of the work reported, the degree to which the paper exemplifies how to perform and report scientific research in cybersecurity.

Submission Period Begins: December 15, 2016
Submission Period for Entries Ends: March 31, 2017 11:59 PM, EST.
Evaluation Process for Entries Begins: April 1, 2017
Winners Notified: By September 15, 2017
Winners Announced: Fall 2017