WIP: Large Language Models for Static Analysis

 

Static analysis, the process of examining code without executing it, is crucial for identifying software issues. Yet, static analysis is hampered by its complexity and the need for customization for different targets. Traditional static analysis tools require extensive human effort and are often limited to specific target programs and programming languages. Recent advancements in large language models (LLMs), such as GPT-4 and Llama, offer new capabilities for software engineering tasks. However, their application in static analysis, especially in understanding complex code structures, remains under-explored.

This paper introduces a novel approach named E&V, which leverages LLMs to perform static analysis. Specifically, E&V employs LLMs to simulate the execution of pseudo-code, effectively conducting static analysis encoded in the pseudo-code with minimal human effort, thereby improving the accuracy of results. E&V includes a verification process for Pseudo-code Execution without needing an external oracle. This process allows E&V to mitigate hallucinations of LLMs and enhance the accuracy of static analysis results.We have implemented E&V in a prototype tool designed for triaging crashes through backward taint analysis. This prototype, paired with GPT-4-32k, has been applied to triage 170 recently fixed Linux kernel bugs across seven bug categories. Our experiments demonstrate that the prototype correctly identifies the blamed function in 81.2% of the cases. Additionally, we observe that our novel verification process significantly improves the accuracy, increasing it from 28.2% to 81.2%.

Yu Hao is a Computer Science Ph.D. candidate at University of California, Riverside. My research focuses on system security and program analysis. My research improves multiple program analysis and testing techniques (e.g., fuzzing, symbolic execution, static analysis and large language models), thereby combining the advantages of those techniques to systematically and automatically improve the security for real world software system (e.g., Linux kernel, LLMs, concurrent program). My research has led to multiple papers published in several prestigious conferences (e.g., S&P, CCS, NDSS, ICSE, FSE) and journal (e.g., TSE). The open source tools attract interest from and applied in academia, community and industry.