Adversarial Attacks: Navigating the Evolving Landscape of AI Security and Robustness

Latest 100 papers on adversarial attacks: Aug. 25, 2025

The landscape of AI is rapidly evolving, and with it, the sophistication of adversarial attacks. These subtle, often imperceptible, manipulations pose a significant threat across diverse domains—from compromising autonomous vehicles to undermining financial systems and even influencing critical medical decisions. As models become more powerful and integrated into our daily lives, understanding and mitigating these vulnerabilities is paramount. This post dives into recent breakthroughs, exploring novel attack vectors and ingenious defense strategies that are shaping the future of AI security.### The Big Idea(s) & Core Innovationsresearch highlights a multi-faceted approach to both attacking and defending AI systems, pushing the boundaries of what was previously thought possible. A key theme emerging is the exploitation of inherent system properties, not just direct input manipulation. For instance, a groundbreaking study from University of California, Berkeley and NVIDIA, “Robustness of deep learning classification to adversarial input on GPUs: asynchronous parallel accumulation is a source of vulnerability“, reveals how asynchronous parallel floating-point reductions (APFPR) on GPUs can induce misclassification even without input perturbations, essentially creating a new class of hardware-based adversarial attacks. This highlights a critical, often overlooked, layer of vulnerability. Complementing this, research from Tsinghua University in “Robust Graph Contrastive Learning with Information Restoration” focuses on enhancing graph neural network (GNN) robustness by restoring lost information during training, improving resilience against both data corruption and adversarial assaults.the realm of language models, Sun Yat-sen University’s “No Query, No Access” introduces VDBA, a highly effective black-box attack on LLMs that operates without model or training data access, leveraging only victim texts. This demonstrates a disturbing level of exploitability with minimal information. Similarly, The Hong Kong Polytechnic University’s “PLA: Prompt Learning Attack against Text-to-Image Generative Models” presents a prompt learning attack (PLA) that expertly bypasses safety mechanisms in black-box text-to-image (T2I) models through gradient-based training of adversarial prompts. Indiana University’s “CAIN: Hijacking LLM-Humans Conversations via Malicious System Prompts” takes this further, showcasing how malicious system prompts can hijack AI-human conversations, leading to targeted harmful answers while maintaining benign behavior elsewhere. From a defense perspective, Nanyang Technological University’s “Cowpox: Towards the Immunity of VLM-based Multi-Agent Systems” introduces COWPOX, a novel distributed immunity mechanism using ‘curing samples’ to neutralize and prevent the spread of infectious jailbreak attacks in VLM-based multi-agent systems, providing a much-needed layer of collective safety awareness. For specific domain, National University of Singapore and **A*STAR’s “Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees” highlights vulnerabilities in learning-to-defer (L2D) systems and proposes SARD, a robust algorithm with theoretical guarantees for reliable task allocation under attack.computer vision, Beijing Jiaotong University’s “SAM Encoder Breach by Adversarial Simplicial Complex Triggers Downstream Model Failures” reveals how the Segment Anything Model (SAM) encoder can be breached by transferable adversarial examples, posing systemic risks to downstream applications. Countering this, Technical University of Munich’s “Set-Based Training for Neural Network Verification” proposes a novel set-based training procedure that uses gradient sets to directly control output enclosures, improving both robustness and formal verification efficiency. For financial applications, “Distributional Adversarial Attacks and Training in Deep Hedging” from Imperial College London and University of St. Gallen addresses vulnerabilities in deep hedging strategies to distributional shifts, introducing an adversarial training framework for enhanced resilience.### Under the Hood: Models, Datasets, & Benchmarksresearch actively introduces and leverages new tools and datasets to rigorously test and improve AI model robustness:Hardware-Level Attacks: “Robustness of deep learning classification to adversarial input on GPUs…” utilizes GNN datasets and models to benchmark vulnerabilities stemming from asynchronous parallel floating-point reductions. Code is available at https://www.github.com/minnervva/fpna-robustness.Multi-Agent Reinforcement Learning (MARL): Papers like “Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space” from KTH Royal Institute of Technology and “Constrained Black-Box Attacks Against Multi-Agent Reinforcement Learning” use various MARL benchmark environments (e.g., RWARE) to evaluate attack efficacy. Code for detection is at https://github.com/kiarashkaz/PGC-Adversarial and for black-box attacks at https://github.com/AmineAndam04/black-box-marl.Multimodal Entity Linking (MEL): “On Evaluating the Adversarial Robustness of Foundation Models for Multimodal Entity Linking” releases the first MEL adversarial example dataset and introduces LLM-RetLink, a retrieval-augmented approach.Vision Transformers (ViTs): “ViT-EnsembleAttack: Augmenting Ensemble Models for Stronger Adversarial Transferability in Vision Transformers” from Huazhong University of Science and Technology introduces ensemble strategies for ViTs, with code at https://github.com/Trustworthy-AI-Group/TransferAttack.Diffusion Models for Adversarial Generation: Papers like “TAIGen: Training-Free Adversarial Image Generation via Diffusion Models” and “Generating Adversarial Point Clouds Using Diffusion Model” leverage diffusion models for efficient, high-quality adversarial example generation. The latter’s code is at https://github.com/AdvPC/Generating-Adversarial-Point-Clouds-Using-Diffusion-Model.Language Model Security & Jailbreaks: “Mitigating Jailbreaks with Intent-Aware LLMs” introduces INTENT-FT for robust LLM defense. Surveys like “Security Concerns for Large Language Models: A Survey” highlight a range of LLMs and datasets, with a curated list at https://github.com/xingjunm/Awesome-Large-Model-Safety. 360 AI Security Lab’s “PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking” evaluates against benchmarks like SafeBench and MM-SafetyBench.Cybersecurity & IoT: “Reducing False Positives with Active Behavioral Analysis for Cloud Security” provides a framework for CSPM using AWS experiments. Code is available at https://github.com/27dikshant/Noisecut. “Enhancing IoT Intrusion Detection Systems through Adversarial Training” leverages adversarial training for IoT IDS, and DINA from National Chengchi University (https://arxiv.org/pdf/2508.05671) introduces a dual defense framework for NLP, with code at https://github.com/DINA-Project/dina-framework.Image Forgery Detection: “ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack” introduces ForensicsSAM, leveraging large vision foundation models with code at https://github.com/siriusPRX/ForenssSAM.Medical AI: “Adversarial Attacks on Reinforcement Learning-based Medical Questionnaire Systems…” extensively evaluates attacks on RL-based medical systems using the National Healthcare Interview Survey (NHIS) dataset and provides a medical constraint framework for clinical plausibility. Code at https://github.com/ArtiFy/medical-attack-framework.Novel Defenses: Monash University’s “MUNBa: Machine Unlearning via Nash Bargaining” leverages game theory to improve machine unlearning robustness. “Contrastive ECOC: Learning Output Codes for Adversarial Defense” from National Central University provides code at https://github.com/YuChou20/Automated-Codebook-Learning-with-Error-Correcting-Output-Code-Technique.### Impact & The Road Aheadadvancements underscore a critical and evolving arms race in AI. The ability of researchers to uncover novel vulnerabilities, from hardware-level floating-point arithmetic issues to sophisticated prompt engineering, pushes the envelope of what constitutes a “secure” AI system. The development of physically realizable attacks, such as “PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems” from Xi’an Jiaotong University, and “Towards Physically Realizable Adversarial Attacks in Embodied Vision Navigation“, highlights the immediate, real-world implications for safety-critical applications like autonomous driving and robotics. Similarly, the study of attacks on EEG systems with “Radio Adversarial Attacks on EMG-based Gesture Recognition Networks” by ShanghaiTech University signals emerging threats to bio-signal processing and human-computer interfaces. rise of advanced defense mechanisms, such as Nanyang Technological University’s COWPOX for multi-agent systems and Tsinghua University’s RCR-AF for enhanced generalization and robustness, shows promising directions towards building more resilient AI. The theoretical underpinning provided by papers like “Adversarial flows: A gradient flow characterization of adversarial attacks” from Deutsches Elektronen-Synchrotron DESY** deepens our understanding of attack dynamics, paving the way for more principled defenses. Looking ahead, the focus will undoubtedly remain on developing multi-layered, adaptive defenses that can keep pace with increasingly intelligent and stealthy adversarial strategies. The insights gained from this vibrant research landscape are not just about patching vulnerabilities but about fundamentally rethinking how we design, train, and deploy AI systems that are truly robust, reliable, and trustworthy in an unpredictable world. The journey towards truly secure and robust AI is far from over, and these papers are charting an exciting course forward.