Adversarial Attacks: Navigating the Shifting Sands of AI/ML Security
Latest 10 papers on adversarial attacks: Nov. 22, 2026
Adversarial attacks continue to be a significant challenge in AI/ML, pushing the boundaries of model robustness and trustworthiness. From subtle perturbations that fool vision systems to insidious backdoors in image-to-image networks and complex attacks on large language models, the landscape of AI security is constantly evolving. Recent research has delved into understanding these vulnerabilities and proposing innovative mitigation strategies. Let’s explore some of the latest breakthroughs and their implications for the future of secure AI.
The Big Idea(s) & Core Innovations
The central theme across these papers is the increasingly sophisticated nature of adversarial attacks and the corresponding need for more robust, often causally-aware or feedback-driven, defenses. A groundbreaking contribution comes from Wenbo Jiang et al. from the University of Electronic Science and Technology of China and affiliated institutions, who introduce I2I Backdoor, the first backdoor attack specifically targeting image-to-image (I2I) networks. This innovation leverages a targeted universal adversarial perturbation (UAP) as a trigger, demonstrating how an I2I model can maintain clean performance while emitting a chosen target image for a triggered input. What’s particularly alarming is its ability to propagate through upstream models, causing downstream classifiers and detectors to misbehave even when unknown to the attacker. Their work highlights that common input defenses often fall short, necessitating backdoor-specific evaluations.
Moving into the realm of robotics, Keke Tang et al. from Guangzhou University and collaborators expose a novel vulnerability in vision-language-action (VLA) models with ExecGap. Instead of solely focusing on prediction errors, ExecGap exploits the ‘command-execution gap’—the mismatch between a VLA model’s predicted task-space command and the robot’s actual realized motion. By steering adversarial patches towards kinematically inefficient or dynamically weak actuation directions, they achieve greater command-execution mismatch and weaker robot motion without altering the instruction or controller. This is a critical insight for embodied AI, underscoring that physical realization constraints are a new attack surface.
Meanwhile, in the domain of large language models (LLMs), Peng Yu et al. from Shanghai Jiao Tong University and others address persistent issues in LLM safety alignment with DACE (Diversity-Driven Adversarial Co-Evolution). DACE combats attack strategy collapse and defense adversarial forgetting in RL-based co-evolution. Their innovation lies in coupling a strategy-level diversity reward for the attacker with an uncertainty-aware Bayesian replay for the defender, significantly improving robustness to out-of-distribution attacks and increasing attacker strategy coverage. This approach fundamentally shifts how we think about non-stationary adversarial training for LLMs.
For wireless communication systems, Ho Viet Duc Luong et al. from Hanoi University of Science and Technology introduce evolutionary-based methods, PAGA and PAMFGA, for physical attacks on end-to-end autoencoder communications. They reframe physical-layer attacks as power-constrained, stochastic optimization problems, using genetic algorithms and multifactorial evolutionary frameworks to generate universal perturbations that can survive channel propagation. This highlights a critical, often overlooked, vulnerability in learned communication systems, urging for security-aware designs at the physical layer.
Further demonstrating the breadth of adversarial research, Han Wu et al. from Newcastle University explore Adversarial Tracking in autonomous vehicles. Their work reveals that merely perturbing the object detector is sufficient to severely degrade an entire tracking-by-detection pipeline, even with a single universal perturbation applied across frames. The staggering impact, with HOTA dropping drastically and MOTA plunging to negative values, emphasizes that autonomous driving security must be evaluated at the tracking level, not just detection accuracy.
In the medical imaging field, Zahid Mansur et al. from IIT Jodhpur uncover Cracks in Embeddings, evaluating the robustness of JEPA-based chest X-ray encoders. They found that strong clean-image performance and self-supervised pretraining do not guarantee robustness against various perturbations and black-box attacks. Their practical solution: adversarially fine-tuning a lightweight predictor and classifier without changing the pretrained encoder, effectively restoring embedding consistency and AUROC, proving explicit exposure to perturbed data is key.
Finally, two papers tackle the fundamental aspect of feature relevance and input cleaning. Mohammad Meymani and Roozbeh Razavi-Far from the University of New Brunswick systematically evaluate feature reduction techniques for adversarial robustness. Their study reveals that conventional methods like PCA and RFE can worsen vulnerability, especially under Universal Adversarial Perturbations, while permutation importance ranking and univariate mutual information offer more consistent gains. This challenges the intuition that “less is more” for robustness. Complementing this, Nasir Khan et al. from KTH Royal Institute of Technology and collaborators propose Causal AI-Driven Secure mmWave Beam Alignment with Defensive Variational Autoencoders (DACE). They use DirectLiNGAM for causal feature selection to significantly reduce beam-sweeping overhead and combine it with a DVAE to reconstruct clean RSSI representations from adversarially perturbed measurements, demonstrating robust beam alignment under attack. This shows how causal reasoning can enhance both efficiency and security.
Under the Hood: Models, Datasets, & Benchmarks
This research leverages and introduces a variety of essential resources:
- Models: The CODEX code-davinci-002 LLM, OpenVLA, OpenVLA-OFT, and π0 (for robotics), DPIR, SCUNet, MPRNet, MIRNet, and ESRGAN (I2I models), YOLOv3, YOLOv4, and YOLOX (object detectors), along with various JEPA-based encoders (CheXWorld, X-WIN, RadJEPA, I-JEPA).
- Datasets: APT malware dataset, ImageNet (for DFC), LIBERO benchmark (for ExecGap), GeoQuery and Scholar text-to-SQL benchmarks (for LLMs), NIH ChestX-ray14 and LIDC-IDRI (medical imaging), and the CARLA simulator-based synthetic autonomous-driving dataset (for adversarial tracking). The DeepMIMO dataset and Boston5G ray-tracing scenario were used for mmWave beam alignment.
- Benchmarks & Metrics: Robustness is quantified using diverse metrics: MOTA, HOTA, IDF1 for tracking; macro AUROC for medical imaging; robust execution accuracy for LLMs; block error rate for communication systems; and a novel mitigation score for feature reduction studies.
- Code: Several papers provide open-source code for their contributions:
- DACE (Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment): https://github.com/PursuitYP/DACE-NeurIPS
- Adversarial Tracking (attack code): https://github.com/wuhanstudio/adversarial-tracking
- CARLA tracking dataset generation code: https://github.com/wuhanstudio/carla-tracking-dataset
- DirectLiNGAM implementation: https://github.com/ashishb/DirectLiNGAM
Impact & The Road Ahead
The implications of this research are profound for the broader AI/ML community. The I2I Backdoor attack underscores the need for security assessments across the entire AI pipeline, not just at the final classification stage. ExecGap’s insights into robotic embodiment highlight the necessity of considering physical constraints as attack surfaces in safety-critical autonomous systems. DACE offers a powerful framework for robust LLM alignment, tackling the persistent challenges of attacker diversity and defender memory, pushing towards more reliable and less harmful AI assistants. The work on physical attacks in wireless communications serves as a stark reminder that even physical layers are vulnerable to intelligent adversaries, demanding integrated security-aware designs for 6G and beyond.
The findings in adversarial tracking demand that autonomous vehicle developers prioritize tracking-level robustness over mere detection accuracy. For medical AI, the lessons from JEPA encoders emphasize that strong clean performance doesn’t equate to robustness; explicit adversarial training or fine-tuning is crucial for reliable clinical deployment. Lastly, the nuanced understanding of feature reduction techniques and causal AI-driven defense for mmWave communications offer pathways to develop more efficient and secure systems, highlighting that informed design choices, rather than brute force, are key to bolstering defenses.
The road ahead involves developing adaptive, multi-layered defenses that not only detect and mitigate known attacks but also anticipate novel ones by understanding the underlying mechanisms of vulnerability. This body of work collectively paints a picture of an AI security landscape that is complex, dynamic, and critically important, driving innovation towards AI systems that are not just intelligent, but truly trustworthy.
Share this content:
Discover more from SciPapermill
Subscribe to get the latest posts sent to your email.
Post Comment