Business Processes and Business Logic (BL) systems that govern and operationalize these processes are critical to both the US economy as well as the DoD mission. Logic faults in these systems have the potential to severely impact production of essential products or impede procurement of critical parts and services. For instance, a mismatch between a part tolerance in a manufacturing specification and the implementation of that specification in a manufacturing plan can cause numerous downstream effects, including shipment delays, recalls, or failures. The goal of the DARPA BPL program is to develop methods and tools to protect these systems by identifying logic faults before they can have an impact.

Identifying logic faults requires overcoming several challenges, such as disparate system representations and unstructured specifications. Our team’s approach to overcoming these challenges, called Model-Enhanced Target-Adaptive Business Process Logic (META-BPL), builds upon a number of sophisticated technologies, some of which are products of decades of Vanderbilt’s research in model-based engineering, and others a product of recent breakthroughs in the form of Large Language Models (LLMs). META-BPL uses a combination of LLMs, formally-defined modeling languages, and traditional computational methods to provide a system that can ingest BL data and workflows, create structured representations, and analyze the BL system for logic faults.

In the META-BPL system, LLMs enable two types of analysis. The first is broad, high-level reasoning across many types of system artifacts through a retrieval-augmented generation (RAG) system. The second is the targeted extraction of specifications from unstructured representations. The extracted specifications are combined with formal models of the underlying system, and traditional algorithmic methods are used to analyze this combined representation. This “old meets new” approach combines the power of LLMs to translate messy, ambiguous inputs into structured insights that can then be tested, validated, and acted upon by more traditional computational methods.

We will describe the challenges in designing and building our system, our system architecture, and show a demo of our system. We will highlight the application of our methods and tools to program-generated evaluation challenges, as well as extensions developed for transition to external industrial partners.