Diffusion Models: The New Architects of AI — From Pixels to Particles and Beyond

Latest 100 papers on diffusion model: May. 16, 2026

Diffusion models continue to redefine the boundaries of AI, moving beyond stunning image generation to tackle complex challenges across diverse fields like robotics, medicine, quantum computing, and even climate science. Recent research underscores a pivotal shift: these models are becoming more efficient, more controllable, and increasingly robust, unlocking unprecedented capabilities and addressing fundamental limitations. Let’s dive into some of the latest breakthroughs.

The Big Idea(s) & Core Innovations

At the heart of recent advancements is the drive to imbue diffusion models with greater control, realism, and efficiency. A recurring theme is the move towards decoupling complex generation tasks and integrating physical or geometric priors. For instance, researchers from Alibaba Group and University of Washington in their paper, RefDecoder: Enhancing Visual Generation with Conditional Video Decoding, highlight a systematic bottleneck where VAE decoders in video pipelines remain unconditional. They propose RefDecoder, a plug-and-play solution that injects high-fidelity reference signals directly into the decoder, significantly improving spatial detail and temporal consistency without retraining the diffusion model itself.

In video generation, achieving long-range consistency and real-time performance is crucial. Imperial College London’s RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO addresses a history supervision gap in autoregressive video diffusion, enabling end-to-end supervision through cached history and reformulating consistency sampling for direct online RL optimization. Complementing this, AGI Lab, Westlake University, and StepFun’s Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity introduces a training-free framework that leverages the functional heterogeneity of attention heads, assigning specialized KV cache strategies to extend video generation to minute-long durations. Meanwhile, Tsinghua University and Baidu Inc.’s SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer pushes the boundaries of video editing with a mask-free, one-step diffusion transformer, leveraging theoretical insights into conditional optimal transport to achieve ultra-fast subtitle removal.

Control over generated content is also seeing significant strides. Tel-Aviv University and NVIDIA Research’s Compositional Video Generation via Inference-Time Guidance uses a lightweight compositional classifier trained on cross-attention maps to steer denoising towards better compositional faithfulness without fine-tuning. For intricate image editing, Inner Mongolia University of Technology’s ClickRemoval: An Interactive Open-Source Tool for Object Removal in Diffusion Models provides a training-free, click-based object removal tool that leverages attention redirection for natural background restoration. Adding to the toolkit for creative control, UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation from University of Science and Technology of China and Kuaishou Technology (https://yiyanxu.github.io/UniCustom/) addresses the “grounding-binding gap” in multi-reference generation by fusing semantic and appearance features before VLM encoding, leading to more robust identity preservation.

The theoretical underpinnings are also evolving. Fudan University and Alibaba Group’s DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models extends on-policy distillation to continuous-state diffusion processes, deriving a closed-form KL objective that unifies SDE and ODE refinement for multi-task training. In the realm of language, Sun Yat-sen University and Australian National University’s Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space reformulates language generation as a stochastic optimal control problem, introducing Manta-LM which uses Flow Matching in a rectified latent space for high-fidelity, efficient parallel sampling with self-correction capabilities. Complementing this, EPFL and Microsoft AI’s Language Modeling with Hyperspherical Flows introduces S-FLM, operating on unit hypersphere embeddings for scalable, efficient language modeling. Furthermore, University of California Riverside and New York University’s Discrete Stochastic Localization for Non-autoregressive Generation develops a continuous-state framework for discrete sequence generation where the denoiser becomes SNR-invariant, enabling multiple inference paths from a single model.

Addressing a critical security gap, Fujian University of Technology and University of Macau’s DiffusionHijack: Supply-Chain PRNG Backdoor Attack on Diffusion Models and Quantum Random Number Defense reveals a novel PRNG hijacking attack and proposes a hardware QRNG defense, highlighting the need for robust randomness in generative AI. On the safety front, Yonsei University’s Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models proposes an inference-time defense using adaptive steering and remasking to suppress harmful content in DLMs.

Under the Hood: Models, Datasets, & Benchmarks

These innovations are often built upon or contribute to new models, datasets, and evaluation metrics:

• RefDecoder: A drop-in replacement for existing pretrained video VAE decoders (Wan 2.1, VideoVAE+) leveraging reference attention.
• RAVEN: Introduces a training-time test framework and CM-GRPO (Consistency-model GRPO) which reformulates consistency sampling. Evaluated on VBENCH I2V metrics.
• DiffusionOPD: Multi-task training paradigm, showing benefits over multi-task RL baselines on aesthetics, OCR, and GenEval benchmarks.
• CVG: Inference-time guidance for Wan2.2 and CogVideoX using a VLM-based compositional classifier trained on cross-attention maps.
• MicroscopyMatching: Leverages pre-trained latent diffusion models (LDMs) for segmentation, tracking, and counting across 20 benchmark datasets and 200 real-world experiments. Hugging Face Space for demo.
• ACE-LoRA: Introduces CIE-Bench, the first comprehensive benchmark for continual image editing, and uses Adaptive Orthogonal Decoupling and Rank-Invariant Historical Information Compression.
• SEDiT: Uses a Diffusion Transformer architecture with a conditional video branch. Trains on a comprehensive data synthesis pipeline and introduces VSR-Bench-400 benchmark. Project page: http://zheng222.github.io/SEDiT_project.
• HDRFace: Injects high-dimensional visual representations from DINOv3 into conditional branches of generative models like SD V2.1-base and Qwen-Image, and introduces SDFM module.
• CamProbe: A training-free camera control method for video diffusion models that serves as an analytical probe to study LTX-2.3 and HunyuanVideo-1.5 models. Code: https://github.com/xrchitech/CamProbe.
• VMU-Diff: Combines Vision Mamba State-Space (VMSS) blocks with conditional diffusion for precipitation nowcasting, using radar and Himawari-8 satellite data.
• D-CoCoA: Uncertainty quantification for Large Language Diffusion Models (LLaDA-1.5, Dream) using denoising trajectory signals.
• Manta-LM: Uses Flow Matching within a rectified latent control space and Transformer as Global Integral Operator. Evaluated on LAMBADA, WikiText-2/103, PTB, 1BW, QQP, Quasar-T, Wiki-Auto, CCD.
• Head Forcing: Training-free KV cache strategies for autoregressive video DiTs.
• ClickRemoval: Built on pretrained Stable Diffusion models (SD1.5, SD2.1, SDXL1.0) with an attention redirection framework. Code: https://github.com/zld-make/ClickRemoval.
• Mirage: Framework for discovering semantic adversarial attacks against online HD mapping in autonomous vehicles, using conditional diffusion models trained on nuScenes. Code: https://anonymous.4open.science/r/MIRAGE-F7A9/.
• AnyBand-Diff: Spectral-prior-guided diffusion framework for remote sensing imagery with Physics-Guided Sampling and Dual Stochastic Masking. Uses Pavia University and Washington DC hyperspectral datasets.
• IG-Diff: Illumination-guided diffusion model for complex night scene restoration, trained with a comprehensive data synthesis pipeline.
• D2-CDIG: Remote sensing image generation with dual-prior control (DEM and cloud-fog) using a dual-branch ControlNet. Uses Landsat-8 and RSICD Dataset.
• MM-SOLD: A training-free generative sampler based on score-smoothed Langevin dynamics and moment-matched interacting particles.
• SubDAPS++: Adapts pixel-space diffusion methods (DPS, DAPS) to dynamic resolution frameworks for image restoration. Code: https://github.com/StarNextDay/SubDAPS.git.
• TeDiO: Training-free inference-time method for Wan2.1 and CogVideoX that regularizes temporal attention patterns.
• PanoPlane: Training-free panoramic completion for 3D Gaussian Splatting, guided by Layout Anchored Attention Steering and evaluated on Replica, ScanNet++, and Matterport3D.
• Support Before Frequency: Theoretical and empirical study of discrete diffusion models on FineWeb and synthetic regular languages.
• Covariance-aware sampling: Novel first-order sampler for pixel-space Diffusion Models using Tweedie's formula and structured Fourier-space decomposition. Python implementation in Appendix C.
• MoZoo: Video diffusion framework for animal fur and muscle simulation, introducing MoZoo-Data (62K clips), MoZooBench (120 mesh-video pairs), Role-Aware RoPE, and Asymmetric Decoupled Attention. Project: https://dongxialiu15.github.io/MoZoo/.
• Realiz3D: Fine-tunes diffusion models for photorealistic 3D generation using Domain Shifters and layer-aware training strategies.
• DDiff: Dual ascent optimization framework for inverse problems using pretrained diffusion models as priors.
• CMC: Decoupled framework for trajectory-controlled human motion generation with Selective Inpainting Mechanism, achieving SOTA on HumanML3D and KIT datasets. Project: https://cdlchoi.github.io/cmcpage.
• FMD/L2Plan: Diffusion-based Fluence Map Diffusion (FMD) model and LSTM-based L2O for VMAT radiotherapy planning, validated on REQUITE dataset.
• Medical I2I Translation Benchmark: Compares GANs and latent diffusion models (SRGAN, Diffusion Models) across SynthRAD2023/2025, BraTS2023, autoPET, EnhancePET datasets.
• MCB: Marginal-conditioned bridge sampling for Flow Language Models (FLMs) on LM1B and OWT. Code: github.com/imbirik/mcb.
• HIR-ALIGN: Plug-and-play target-adaptive augmentation framework for hyperspectral image restoration using diffusion models with improved unCLIP and warp-based spectral transfer.
• Qwen-Image-VAE-2.0: Suite of high-compression VAEs (f16, f32) for latent diffusion models, with Global Skip Connections, DINOv2 features, and OmniDoc-TokenBench benchmark. Code: https://github.com/alibaba/OmniDoc-TokenBench.
• Latent Reuse Limits: Theoretical analysis of latent reuse in diffusion models, using principal-angle misalignment and ambient noise.
• Sec2Drum-DAC: Conditional latent-diffusion model for symbolic-to-audio drum rendering, using PCA-DAC latent target and RVQ cross-entropy regularization.
• Diffusion Encoder: A novel diffusion model as an encoder in an autoencoder framework, using alternating training scheme.
• HyperLoRA: Hypernetwork-driven Low-Rank Adaptation for stylized text-to-motion generation on HumanML3D and 100STYLE datasets. Code: https://github.com/junhyukjeon/style-salad.
• Img2CADSeq: Multi-stage pipeline generating standard STEP CAD files from images using hierarchical codebook and VQ-Diffusion. Introduces CAD-220K and PrintCAD datasets. Code: https://github.com/Rilpraa0110/Img2CADSeq.
• PGM: Proximal-based Generative Modeling for Bayesian inverse problems using Moreau score and proximal operators. Code: https://github.com/boyangzhang2000/PGM.
• Physics-Guided Generative Optimization: Combines conditional diffusion model, PINN critic, and GNN encoder for co-optimizing Trotter-Suzuki decomposition in quantum computing. Code: https://github.com/mindmemory-ai/pinn_diffusion_trotter_suzuki.git.
• DiffST: Efficient diffusion-based framework for space-time video super-resolution using cross-frame context aggregation and video representation guidance. Code: https://github.com/zhengchen1999/DiffST.
• Heavy Tails Diffusion: Theoretical and empirical study of DDPM (Gaussian) and DLPM (α-stable) models on KDD Cup 99 and Wildfires datasets.
• Adaptive Steering and Remasking: Inference-time defense for LLaDA (7B) and Dream (8B) models using Contrastive Safety Direction (CSD). Code: https://github.com/leeyejin1231/DLM_Steering_Remasking.
• Efficiency Gap in Byte Modeling: Compute-matched scaling study of byte vs BPE tokenization across AR and MDM objectives on Slimpajama-627B.
• Orthrus: Dual-architecture framework combining AR LLMs with a lightweight diffusion module for parallel token generation (e.g., Qwen3 model family). Code: https://github.com/chiennv2000/orthrus.
• Generative Motion In-betweening: Combines latent diffusion models (LDM) with motion implicit neural representations (INR) using Implicit Manifold Guidance (IMG). Code: https://github.com/Coondinator/NINB.
• CRAFT: Clinical Reward-Aligned Finetuning for medical image synthesis using Clinical Alignment Score (CAS) and multimodal LLMs/VLMs. Code: https://anonymous.4open.science/r/CRAFT-07B4.
• DIVER: Dual-stage dataset distillation framework using pretrained diffusion models (DiT-XL/2) to recover semantics from distilled datasets. Code: https://github.com/einsteinxia/DIVER.
• Critical Slowing Down: Theoretical and experimental investigation of diffusion models for sampling Gaussian O(n) models.
• RIGVid: System for robotic manipulation by imitating AI-generated videos (Kling), using FoundationPose tracking and VLM-based filtering. Project: https://rigvid-robot.github.io/.
• Score-Difference Flow: Theoretical framework unifying denoising diffusion models and GANs via score-difference (SD) flow.
• GaitProtector: Training-free gait de-identification framework using pretrained 3D diffusion priors for silhouette-based gait sequences.
• GeoQuery: Geometry-guided diffusion framework for sparse-view 3D reconstruction with 3D Gaussian Splatting, addressing query contamination. Code: https://github.com/xcao21/GeoQuery.
• FlowSR: Rectified flow and HR-regularized consistency learning for image super-resolution, leveraging Real-ESRGAN and Qwen2-VL.
• DriftXpress: Accelerated drifting models for one-step generative modeling using projected RKHS fields and Nyström landmarks. Code: github.com/Mortrest/DriftXpress.
• Neutron Source Distributions: Comparative study of generative models (VAE, NF, GAN, DM) for neutron source modeling, integrated into Vitess Monte Carlo software.
• SAEParate: Disentangled Sparse Autoencoder Representations with latent-space contrastive objective for diffusion model unlearning on UnlearnCanvas benchmark.
• L2P: Latent-to-Pixel transfer paradigm leveraging Z-Image LDM with large-patch tokenization for native 4K pixel generation.
• RealDiffusion: Training-free physics-informed attention for multi-character storybook generation using Stable Diffusion XL. Code: https://github.com/ShmilyQi-CN/RealDiffusion.
• Few-Shot Synthetic Data: Lightweight augmentation pipeline using LoRA adapter on FLUX.2-dev for rare-class detection in medical and industrial imaging.
• Multi-view Latent Priors: Geometry-Guided Gated Transformer (G3T) and Action Manifold Learning (AML) for robotic manipulation in LIBERO, LIBERO-Plus, RoboTwin 2.0. Project: https://junjxiao.github.io/Multi-view-VLA.github.io/.
• Monotonic Sampling Necessity: Systematic empirical test of monotonic vs non-monotonic noise schedules across DDPM, Flow Matching, EDM.
• DiffSegLung: Unsupervised lung pathology segmentation with Diffusion Radiomic Distillation using 3D diffusion U-Net. Code: https://anonymous.4open.science/r/DiffLungSeg-CEF1/README.md.
• W-Flow: One-step generative modeling via Wasserstein Gradient Flows and Sinkhorn divergence, achieving SOTA on ImageNet 256x256. Project: https://hanjq17.github.io/W-Flow/.
• Single-Shot HDR Recovery: Conditional video generation using Stable Video Diffusion fine-tuned for exposure bracket generation.
• FERMI: Feature-mapping framework for membership inference attacks against tabular diffusion models (TabDDPM, TabSyn, TabDiff) on California, Instacart, Berka datasets.
• OptDiff: Principled design of diffusion-based optimizers for inverse problems (MRI, deblurring, SR) using RED-diff framework.
• STRIDE: Training-free diversity guidance via PCA-Directed Feature Perturbation in single-step diffusion models (FLUX.1-schnell, SD3.5 Turbo).
• ZeroIDIR: Zero-reference illumination degradation image restoration with perturbed consistency diffusion models and adaptive gamma correction. Code: https://github.com/JianghaiSCU/ZeroIDIR.
• GDPD: Generative Diffusion Prior Distillation for long-context knowledge transfer in time series classification on UCR and UEA archives. Code: https://github.com/hewadehigaha/GDPD_ICLR26.
• Dynamic Full-body Motion Agent: Two-stage framework for human-object interaction using human motion diffusion model (MDM) and Composer-Blended HOI Execution. Project: https://yurangja99.github.io/dynamic-hoi/.
• Tractability Landscape: Theoretical framework analyzing diffusion alignment with KL vs Wasserstein distance and different reward types.
• Couple to Control: Framework for joint initial noise design in diffusion models (SD1.5, SDXL, SD3) for diverse image generation.
• SPADE: Conditional diffusion models for forward surrogate modeling in offline black-box optimization, with Calibrated Diffusion Estimation and Support-Proximity Regularization. Code: https://github.com/HarryYoung2018/spade.
• TMPO: Trajectory Matching Policy Optimization for diverse and efficient diffusion alignment using Softmax Trajectory Balance objective.
• FlashClear: Efficient diffusion-based object removal framework with Region-aware Adversarial Distillation (RAD) and Foreground-Prioritized Asymmetric Attention and Caching (FPAC).
• Diff-CAST: Diffusion-guided motion prior framework for quadrupedal locomotion with Symmetric Augmented Command Conditioning (SACC) and Constrained RL.
• P-Flow: Proxy-gradient Flows for linear inverse problems using flow matching and Gaussian spherical projection.
• SPOT: Inference-time framework for safe text-to-image generation via Selective Prompt Projection using LLM and VLM cascade.
• ELF: Continuous diffusion language model using Flow Matching in embedding space, with shared-weight network design and classifier-free guidance. Code: https://github.com/lillian039/ELF.
• Confidence-Guided Diffusion Augmentation: Framework for Bangla handwritten compound character recognition with SE-enhanced U-Net and classifier-confidence-based filtering on AIBangla dataset.
• EditMGT: First Masked Generative Transformer (MGT) based image editing framework with multi-layer attention consolidation and region-hold sampling. Project: https://weichow23.github.io/EditMGT.
• Elucidating Representation Degradation: Elucidated Representation Diffusion (ERD) framework addressing representation degradation in diffusion training. Code: https://github.com/LilYau350/Elucidated-Representation-Diffusion.
• Kernel-Gradient Drifting Models: One-step generative modeling using kernel gradients for Riemannian manifolds and discrete data. Code: https://anonymous.4open.science/r/kernel-grad-drift-B4D5.
• U2Diffine: Heteroscedastic Diffusion model for multi-agent trajectory completion with bi-variate Gaussian modeling and RankNN for error probabilities.
• diffGHOST: Conditional diffusion model for synthetic mobility trajectories with VAE latent space segmentation and targeted noise addition for K-anonymity.
• GenMed: Pairwise generative reformulation of medical diagnostic tasks using diffusion models for joint distribution modeling and test-time output optimization.
• GG-PA: Training-free framework for composing diffusion priors with physical context via generative Gibbs sampling. Code: https://github.com/wwz171/gg-pa-released.git.
• PoGMDM: Product-of-Gaussian-Mixture Diffusion Models for joint nonlinear MRI reconstruction and coil sensitivities.
• Branding Injection: Explores security vulnerability in multi-phase image generation-and-edit workflows using FLUX.1-dev and Gemini 2.5 Flash.
• SemanticREPA: Improves human image animation via Semantic Representation Alignment using structure and ID alignment modules.
• IMDM: Infinite Mask Diffusion Model for few-step distillation in language models, overcoming theoretical factorization error bounds. Project: https://Ugness.github.io/official_imdm.
• Masked Diffusion Language Models Training: Introduces bell-shaped time sampling for faster convergence.
• AID: Amortized Inpainting with Diffusion using a reusable guidance module and continuous-time actor-critic algorithm.
• AsymFlow: Rank-asymmetric velocity parameterization for flow-based generation, achieving SOTA on ImageNet 256×256 and finetuning latent flow models to pixel space. Project: https://hanshengchen.com/asymflow.
• GTA: Geometry-Then-Appearance Video Diffusion for image-to-3D world generation, using random latent shuffle and test-time scaling. Project: https://hanxinzhu-lab.github.io/GTA/.

Impact & The Road Ahead

The collective thrust of this research points toward a future where generative AI is not just powerful but also highly adaptable, safe, and efficient. We’re seeing a move from models that generate in isolation to ecosystems of models that interact, adapt, and learn from each other. The ability to integrate physical priors (AnyBand-Diff, RealDiffusion, PoGMDM, Physics-Guided Generative Optimization) is ushering in a new era of AI for scientific discovery and engineering, offering models that are not only realistic but also physically plausible. This is critical for high-stakes applications like medical imaging (GenMed, CRAFT, DiffSegLung, FMD/L2Plan, Medical I2I Translation Benchmark), where human-level fidelity and interpretability are paramount.

Efficiency gains, such as one-step sampling (W-Flow, FlowSR), faster training (Masked Diffusion Language Models Training), and memory optimization (P-Flow, L2P), are democratizing access to powerful generative models, making them usable on less hardware and in real-time scenarios. The focus on robust control (Couple to Control, CVG), security (DiffusionHijack, Mirage, Branding Injection), and privacy (FERMI, GaitProtector, diffGHOST) is equally vital, building trust and ensuring ethical deployment. We’re moving towards generative models that understand and interact with the world in a more nuanced way, whether it’s through multi-agent trajectory modeling (U2Diffine), multi-view robotic manipulation (Multi-view Latent Priors, RIGVid), or creating entire 3D worlds (GTA, PanoPlane). The future of AI is not just generating images, but generating intelligence itself, making diffusion models the versatile architects of tomorrow’s AI systems.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.