Cognitive Aid for Vulnerability Analysis (CAVA)

Rajan Bhattacharyya is Director of R&D in the Intelligent Systems Laboratory at HRL Laboratories. His research interests include neurocognitive modeling, brain-machine interfaces, and AI. Rajan received a PhD in Computation and Neural Systems from Caltech, and a Bachelor's of Science in Electrical Engineering and Computer Science from UC Berkeley.

Distribution A: Approved for Public Release, Distribution Unlimited. 
The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

Abstract

Software systems continue to become critical elements in DoD missions through the adoption of analytics, automation, and autonomy. Analyzing security vulnerabilities in software require special knowledge, skills, and cognition from extensive prior experience which is currently unmatched by pure automation approaches in vulnerability detection. The cyber-analyst workforce does not match the needs in terms of scale and speed for DoD to secure critical software, and machine automation does not match expert analyst cognition, agility, and expertise in finding vulnerabilities. To further human machine teaming in cyber operations, we have investigated machine awareness of the human analyst, in the Cognitive Aid for Vulnerability Analysis (CAVA) project through a combination of R&D in human subjects experiments, cognitive science, machine learning, and graph analytics to extract the context of the analyst, including their cognitive states, intent, expertise, and confidence. CAVA was funded under the DARPA CHESS (Computers and Humans Enhancing Software Security) program to investigate the cognitive states and workflow in detail during program analysis for vulnerability discovery, with HRL Laboratories LLC, Carnegie Mellon University, and the Naval Information Warfare Center (NIWC) Pacific. To achieve that goal, reverse engineers analyzed binaries from challenge problems using our custom instrumented version of NSA’s Ghidra platform, which captures and timestamps all the interactions that analyst’s have during task performance. Further, we sense the analysts gaze (eye position and pupil diameter) and neurophysiological activity (brain metabolism and electrical activity) and decode the signals into key cognitive states such as workload, engagement, novelty, attention, and confidence using our machine learning algorithms. We assemble this large set of multimodal observations into temporal graphs, with different layers for cognitive state, user behavior, and task performance, in order to capture the evolution of the reverse engineering activities. This computational graph analytics approach allows us to capture similarities and differences, such as the difference between novice and expert reverse engineers. Further, we can exploit these graphs to due operations such as task classification, even in this highly variable and complex real-world environment. Together, these technologies provide a hierarchical understanding of the analyst context which can enable the machine to be aware of the analyst at the level of the task, behavior, and down to their underlying cognition.