{"id":1320,"date":"2025-09-29T07:49:50","date_gmt":"2025-09-29T07:49:50","guid":{"rendered":"https:\/\/scipapermill.com\/index.php\/2025\/09\/29\/formal-verification-in-the-age-of-ai-ensuring-trustworthiness-from-code-to-cyber-physical-systems\/"},"modified":"2025-12-28T22:06:10","modified_gmt":"2025-12-28T22:06:10","slug":"formal-verification-in-the-age-of-ai-ensuring-trustworthiness-from-code-to-cyber-physical-systems","status":"publish","type":"post","link":"https:\/\/scipapermill.com\/index.php\/2025\/09\/29\/formal-verification-in-the-age-of-ai-ensuring-trustworthiness-from-code-to-cyber-physical-systems\/","title":{"rendered":"Formal Verification in the Age of AI: Ensuring Trustworthiness from Code to Cyber-Physical Systems"},"content":{"rendered":"

Latest 50 papers on formal verification: Sep. 29, 2025<\/h3>\n

The relentless march of AI and ML innovation brings unprecedented capabilities, but also complex challenges, particularly concerning reliability, safety, and trustworthiness. In critical domains\u2014from autonomous vehicles and medical devices to cybersecurity and smart contracts\u2014even a minor flaw can have catastrophic consequences. This makes formal verification, a set of techniques for mathematically proving the correctness of systems, more vital than ever. Recent research highlights a burgeoning field where cutting-edge AI meets rigorous formal methods, promising a future of verifiable and robust AI-powered systems. This post delves into recent breakthroughs that are bridging this critical gap.<\/p>\n

The Big Idea(s) & Core Innovations<\/h3>\n

At the heart of recent advancements is the idea of deeply integrating formal verification into the AI\/ML lifecycle, from initial design to runtime operations. A recurring theme is the leverage of Large Language Models (LLMs) to automate and streamline traditionally manual and labor-intensive formal methods. For instance, the VeriSafe Agent<\/strong>, presented by Jungjae Lee and colleagues from KAIST<\/em>, introduces a novel system for mobile GUI agents that translates natural language user instructions into formally verifiable specifications. This autoformalization<\/code> enables pre-action verification, achieving up to 98.33% accuracy in detecting erroneous actions, a significant leap over purely LFM-based methods, as highlighted in their paper, \u201cVeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verification\u201d<\/a>.<\/p>\n

Similarly, the \u201cWhat You Code Is What We Prove: Translating BLE App Logic into Formal Models with LLMs for Vulnerability Detection\u201d<\/a> paper demonstrates how LLMs can translate Bluetooth Low Energy (BLE) application logic into formal models, significantly improving automated vulnerability detection. This is complemented by Preguss<\/strong>, a framework from Zhejiang University<\/em> researchers led by Zhongyi Wang, as detailed in \u201cPreguss: It Analyzes, It Specifies, It Verifies\u201d<\/a>. Preguss uses LLMs to automate the generation and refinement of formal specifications for large-scale software, synergizing static analysis with deductive verification by breaking down programs into manageable units. In the realm of hardware, UC Irvine<\/em> researchers in \u201cProof2Silicon: Prompt Repair for Verified Code and Hardware Generation via Reinforcement Learning\u201d<\/a> present Proof2Silicon<\/strong>, a reinforcement learning framework that uses prompt repair to generate formally verified code and hardware. This approach effectively bridges LLMs with formal specifications and reactive synthesis, promising high-quality, trustworthy outputs.<\/p>\n

Another significant thrust involves applying formal methods to complex, dynamic systems. For example, Carnegie Mellon University<\/em> researchers in \u201cFormal Verification and Control with Conformal Prediction\u201d<\/a> explore conformal prediction<\/code> to quantify uncertainty and ensure safety in learning-enabled autonomous systems (LEASs), offering a lightweight, data-driven alternative to traditional model-based approaches. In robotic systems, \u201cAS2FM: Enabling Statistical Model Checking of ROS 2 Systems for Robust Autonomy\u201d<\/a> introduces a framework for statistical model checking of ROS 2 systems to enhance autonomy and reliability through probabilistic models. Meanwhile, Bitdefender and INRIA<\/em> researchers, in \u201cBridging Threat Models and Detections: Formal Verification via CADP\u201d<\/a>, leverage attack trees<\/code> and a novel language (GTDL) with the CADP toolset to formally verify cybersecurity detection rules, identifying crucial gaps automatically. Even the core Bitcoin consensus protocol is being scrutinized, with University of Bologna<\/em> and Inria<\/em> researchers in \u201cFormal Modeling and Verification of the Algorand Consensus Protocol in CADP\u201d<\/a> demonstrating its vulnerabilities to adversarial conditions through formal analysis.<\/p>\n

Under the Hood: Models, Datasets, & Benchmarks<\/h3>\n

These innovations are often enabled by specialized tools, models, and datasets:<\/p>\n