SunDew: Systematic Automated Security Testing
Presented as part of the 2017 HCSS conference.
ABSTRACT
SunDew is a new automated test generation framework developed at Google, focused on finding security bugs in C/C++ code. It combines the strengths of multiple test generation techniques under a single cohesive platform. It leverages the vast amount of computational resources avail- able at Google to massively parallelize the automated test generation and triage.
By using a portfolio of test generation techniques, SunDew aims to overcome the coverage saturation (or plateau) that occurs with any indi- vidual technique. This saturation manifests as the inability of the tech- nique to discover unexplored parts of a program after a certain number of generated tests. A portfolio of techniques, on the other hand, provides a diversity of test generation strategies that complement each other.
SunDew embeds the most recent advances in automated test case gen- eration, which provide precision and thoroughness. For example, symbolic execution uses powerful constraint solvers to generate tests that precisely follow desired program branches. This approach allows symbolic execution to reach code executed under very specific input preconditions that would be difficult to discover randomly. At the same time, recent improvements to coverage-guided automated fuzzing, such as AFL or LibFuzzer, generates tests faster than symbolic execution. Thus, SunDew alternates these approaches by using coverage-guided fuzzing to quickly bring the coverage to a first saturation level, then using symbolic execution to refine the search for harder-to-reach code. This, in turn, may provide additional inputs for coverage-guided fuzzers, etc.
As part of SunDew, we also developed a number of format-aware fuzzers, that rely on, amongst other things, machine learning to generate language-aware fuzzers. The SunDew architecture follows a distributed continuous pipeline pattern. It allows a performance-based dynamic resource allocation for the various test generation techniques. This allows us to maximize the combined output of the test suite generation and to avoid long plateaus in the coverage growth of the test suite. We discuss the application of SunDew on a variety of fuzzing targets of interest.
--
Franjo Ivancic is a Staff Software Engineer at Google in New York focusing on application security, program analysis and software development productivity tools. Prior to joining Google, he was a Senior Researcher at NEC Labs in the Systems Analysis and Verification group in Princeton, NJ, from 2003-2013. He received a Ph.D. in Computer and Information Science from the University of Pennsylvania in Philadelphia, PA, in 2003.