5th Annual Best Scientific Cybersecurity Paper Competition Winners

 

Winning Paper | Award Ceremony | Review Team

The fifth NSA Competition for Best Scientific Cybersecurity Paper recognizes the best scientific cybersecurity paper published in 2016. Papers were nominated between January 1, 2016 through December 31, 2016 and 38 nominations were received. One paper was selected for recognition.

Winning Paper

The winning paper of the 5th Competition is You Get Where You're Looking For: The Impact of Information Sources on Code Security (Free Open Access Copy) by Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L. Mazurek, Christian Stransky. These researchers are at CISPA, Saarland University in Germany and at The University of Maryland, College Park in the United States. The paper was presented at the 2016 IEEE Symposium on Security and Privacy ("Oakland").

This paper helps answer the question of why are software developers writing programs that have security vulnerabilities. The paper presents scientific evidence that confirms anecdotal stories that are in the programming community. Specifically, the researchers investigate how different information sources available to the developer influence the developer's abilities to quickly program and to program securely. They studied 54 developers (in Germany and the United States) in a controlled laboratory setting where they had them write security- and privacy relevant code under time constraints. They examined four conditions: 1) Developers were allowed to use any source; 2) Stack Overflow only; 3) Official Android Documentation only; and 4) books only. The results found that "Official API documentation is secure but hard to use, while informal documentation such as Stack Overflow is more accessible but often leads to insecurity. Interestingly, books (the only paid resource) perform well both for security and functionality. However, they are rarely used (in our study, one free choice participant used a book)."

This paper was selected for excelling at multiple attributes of high quality scientific work and reporting. First the authors developed laboratory study to control factors so they could accurately measure the information source variable and help determine the root cause of software vulnerabilities. These choices were based on their preliminary research in Android App developers where determined the best variable to measure. The research also included work to validate the results and they examined the limitations of their study. The paper did a thorough job explaining the research method which helps other researchers duplicate and build upon this work. The paper also has some actionable scientific based advice on developing better materials to have developers write more secure programs. This paper adds scientific knowledge to our understanding of how developers rely on information sources and the impact to the introduction of insecure software code.

Yasemin Acar is pursuing a Master's degree in mathematics and a PhD in Computer Science at Leibniz University of Hannover, Germany. Her research is focused on identifying causes that prevent developers from writing secure code, and helping them overcome those challenges. Her recent projects include evaluating the usability of cryptographic APIs and developing usable documentation for security-relevant APIs.

Michael Backes is the designated founding director of the CISPA Helmholtz Center for Information Security. He holds the chair for Information Security and Cryptography at Saarland University. Moreover, he is the speaker of the Collaborative Research Center on Online Privacy and of the CISPA-Stanford Center for Cybersecurity Research. He authored more than 200 scientific publications and received various scientific awards, in particular the ERC Synergy Grant (Europe's most distinguished research award), the ERC Starting Grant, the Microsoft Privacy Enhancing Technology Award, the Max Planck Fellowship, the IBM Faculty Award as well as the IBM Outstanding Achievement Award.

Sascha Fahl is head of the Information Security Institute in the Computer Science Department at the Leibniz University Hannover, Germany. He studies the intersection of computer security and privacy with human factors. Sascha is particularly interested in investigating end users, administrators, developers and designers of computer systems and their interdependencies with computer security and privacy mechanisms. His research involves large-scale analyses of the Internet and software repositories to understand the huge challenges humans face when interacting with computer security and privacy mechanisms. To understand root causes, evaluate existing mechanisms and investigate novel ideas, he conducts all kinds of user studies with end users, administrators, and developers of these systems. Sascha received his Ph.D. in Computer Science in 2016.

Michelle Mazurek is an Assistant Professor in the Computer Science Department and the Institute for Advanced Computer Studies at the University of Maryland, College Park. Her research aims to improve security- and privacy-related decision making by understanding people's needs and then building sound tools and systems. Recent projects include analyzing how users learn and process security advice; contrasting user expectations with app behavior in Android apps; examining convenience/security tradeoffs in end-to-end encryption; and examining how and why developers make security and privacy mistakes. Mazurek received her Ph.D. in Electrical and Computer Engineering from Carnegie Mellon University in 2014.

Christian Stransky is pursuing his PhD in computer science at Saarland University, Germany. His research is targeted at identifying and understanding usability issues in the toolchain and libraries of developers that result in security or privacy problems. His recent projects include evaluating the usability of security APIs, exploring different samples and their scientific merit for developer studies and providing a tool for other researchers to conduct developer studies.

 

Award Ceremony  

On October 27, 2017, the Research Directorate at NSA hosted the 5th Annual Best Scientific Cybersecurity Paper Competition awards ceremony. The winners are Mr. Doowon Kim, and Prof. Michelle Mazurek from the University of Maryland and Ms. Yasemin Acar, Dr. Sascha Fahl and Mr. Christian Stransky from the Universitat Des Saarlandes (Saarland University) whose paper was entitled "You Get Where You're Looking For: The Impact of Information Sources on Code Security."

Dr. Adam Tagert, the Science of Security Technical Director, gave the welcoming remarks, stating that this ceremony would "acknowledge these special award winners." In addition to the award, Dr. Tagert pointed out that the winning paper was also presented at the 2016 IEEE Symposium on Security and Privacy. He also gratefully acknowledged the help from the 11-member Distinguished Expert Reviewers, with a special nod to Prof. Jean Camp from Indiana University who was in attendance.

Dr. Deborah Frincke, NSA's Director of Research, presented the Opening Remarks that emphasized how research is valued. She noted that, after all, the Director of Research has a seat at NSA's Board of Directors. To the winners, she confirmed that papers like theirs "influence the outside world" in part by demonstrating how science can be used as a "common language and rigor to approach problems." The competition also serves as a thermometer that gauges the maturity of security research, as she is seeing improvement over the years. In this case, the winning team set an example for others to follow as they have shown "what 'good' looks like."

Both Ms. Acar and Mr. Stransky gave a brief presentation on their research which was inspired by a common problem. When software developers get "stuck", they often turn to resources such as Stack Overflow to find solutions. Unfortunately, many of the posted solutions are not necessarily secure. The research explores developers' problem solving choices, and the impact on the software ecosystem. They noticed that an unsettling number of Android apps used readily available, and insecure code snippets. After describing their methodology of subjecting Android developers to various security-relevant tasks and varying their choices of resources (Stack Overflow, official documentation, books, and free choice), they reviewed their findings on the impacts to both functional correctness, and security correctness. They concluded that project managers should "take developers offline and give them a book," and added that while professionals tended to produce functional code more reliably, they were no better at security.

Dr. Carl Landwehr then moderated a Q&A panel discussion with the awardees. They were asked what kinds of blind alleys they might have gone down. They didn't expect how difficult it would be to recruit enough Android developers, and to get their development system to run on different systems, with different restrictions. The discussion led to conclusions about documentation, how it needs more troubleshooting to be an effective, preferred resource, and that the team is working with Google to improve their documentation. The team was then asked about the generalizability of their findings. They responded that there needs to be a change in mindset, so that security needs to be treated as a common goal, and that "documentation matters" - but nobody likes to write it. The team noted that it might be necessary to treat developers like end-users in that most don't know enough about security. Dr. Frincke asked if there were other human nature traps. The team felt that people don't search for optimal solutions and take security advice from odd sources that discourages deeper learning.

Dr. George Coker, Chief of Information Assurance Research, gave the closing remarks. He concluded that the scientific approach to Science of Security is advancing as demonstrated by the research quality improving year over year. He thanked the winners (who stood out above the other 37 nominees), the expert reviewers, and Dr. Frincke. Lastly, he pointed out that the nominations for the 6th annual competition will be due in December.

Review Team

NSA Competition Leads

  • Dr. Deborah Frincke - Director of Research, NSA
  • Dr. Adam Tagert - Science of Security, NSA Trusted Systems Research Group

Distinguished Expert Reviewers

 

About the 5th Annual Paper Competition

The Best Scientific Cybersecurity Paper Competition is sponsored yearly by NSA's Research Directorate and reflects the Agency's desire to increase scientific rigor in the cybersecurity field. This competition was established to recognize current research that exemplifies the development of scientific rigor in cybersecurity research. SoS is a broad enterprise, involving both theoretical and empirical work across a diverse set of topics. While there can only be one best paper, no single paper can span the full breadth of SoS topics. Nevertheless, work in all facets of security science is both needed and encouraged.

Links