O(N) Complexity and Beyond: Scaling AI/ML for the Next Generation

Latest 50 papers on computational complexity: Nov. 23, 2025

The relentless pursuit of efficiency in AI/ML is driving a fascinating new wave of research, particularly around computational complexity. In an era where models grow ever larger and data becomes ubiquitous, finding ways to achieve more with less computational overhead is paramount. This digest explores recent breakthroughs that push the boundaries of what’s possible, moving towards linear or even sub-linear complexity, unlocking new capabilities from physics simulations to medical imaging, and robust control systems.

The Big Idea(s) & Core Innovations

At the heart of many recent advancements is the idea of reducing complexity through clever structural design and problem reformulation. For instance, in OptimizedDP: An Efficient, User-friendly Library For Optimal Control and Dynamic Programming by Minh Bui et al. from Simon Fraser University, the authors tackle high-dimensional dynamic programming (DP) by implementing level-set methods and HeteroCL optimizations. This significantly reduces computational time for solving Hamilton-Jacobi PDEs, a notoriously complex task. Similarly, in the realm of trajectory optimization, I. M. Ross from the Department of Mechanical and Aerospace Engineering in Hessians in Birkhoff-Theoretic Trajectory Optimization reveals how the specialized structure of the Birkhoff Hessian allows for decoupling problem-specific and method-specific components, promising a path to solve million-point trajectory optimization problems from O(N³) to O(N log N) with Chebyshev methods.

A groundbreaking shift in handling information processing is evident in papers exploring alternative computational paradigms. In Combinatorial Optimization using Comparison Oracles by Vincent Cohen-Addad et al. (Google Research, NYU, CMU, etc.), a novel comparison oracle model achieves a general query complexity bound of eO(n²) for finding optimal feasible sets, a significant theoretical breakthrough. This is achieved through a new Global Subspace Learning (GSL) framework. For learning PDE solution operators, SVD-NO: Learning PDE Solution Operators with SVD Integral Kernels by Noam Koren et al. (Technion, EPFL, Eindhoven University of Technology) leverages Singular Value Decomposition to achieve high expressivity and efficiency, outperforming existing Fourier and graph-based neural operators. This work is grounded in functional analysis, allowing the model to learn the kernel’s singular functions directly from data.

The challenge of scalability in large-scale machine learning is directly addressed by several works. Joohwan Ko et al. from KAIST and the University of Pennsylvania, in Provably Scalable Black-Box Variational Inference with Structured Variational Families [https://arxiv.org/pdf/2401.10989], rigorously prove that structured scale matrices can reduce the iteration complexity of Black-Box Variational Inference (BBVI) from O(N²) to O(N). This makes large-scale Bayesian inference practical. Similarly, Shengfei Wei et al. (National University of Defense Technology) introduce A General Anchor-Based Framework for Scalable Fair Clustering [https://arxiv.org/pdf/2511.09889], achieving linear-time scalability for fair clustering by using representative anchors, while preserving fairness. This is a critical step for deploying fair ML models on massive datasets.

Efficient signal processing and communication systems are also seeing significant gains. Neural Networks-Enabled Channel Reconstruction for Fluid Antenna Systems: A Data-Driven Approach by BrooklynSEUPHD demonstrates how neural networks can replace traditional channel reconstruction methods, reducing complexity to between linear and quadratic order in model-free fluid antenna systems. In massive MIMO, Finite-Precision Conjugate Gradient Method for Massive MIMO Detection [https://arxiv.org/pdf/2504.09820] explores how finite-precision conjugate gradient methods can reduce computational costs without sacrificing accuracy, a crucial aspect for real-time communication.

Under the Hood: Models, Datasets, & Benchmarks

Innovations in computational complexity often go hand-in-hand with new models, optimized algorithms, and benchmarking. Here’s a closer look at the resources enabling these advancements:

• OptimizedDP Library: The paper OptimizedDP: An Efficient, User-friendly Library For Optimal Control and Dynamic Programming introduces a Python library with a HeteroCL backend, available on GitHub, for solving high-dimensional optimal control problems via dynamic programming. It uses level-set methods and value iteration algorithms.
• GPR Framework: Graded Projection Recursion (GPR): A Framework for Controlling Bit-Complexity of Algebraic Packing by Jeffrey Uhlmann (University of Missouri – Columbia) proposes a theoretical framework for controlling bit-complexity, particularly in matrix multiplication, achieving a true O(n²) bit-complexity.
• SPAttention: Making Every Head Count: Sparse Attention Without the Speed-Performance Trade-off [https://arxiv.org/pdf/2511.09596] introduces a novel sparse attention mechanism for large language models. This method reorganizes computations through Principled Structural Sparsity, achieving O(N²) computational complexity while outperforming existing sparse attention methods. Code is available on GitHub.
• AFCF Framework: A General Anchor-Based Framework for Scalable Fair Clustering [https://arxiv.org/pdf/2511.09889] presents a general anchor-based framework for fair clustering, capable of achieving linear-time scalability for arbitrary fair clustering algorithms. The authors provide code at GitHub.
• GNN-PE Framework: Efficient Distributed Exact Subgraph Matching via GNN-PE: Load Balancing, Cache Optimization, and Query Plan Ranking [https://arxiv.org/pdf/2511.09052] introduces a distributed extension of the GNN-PE framework, incorporating dynamic load balancing, multi-GPU collaborative caching, and PE-score-driven query plan ranking for subgraph matching. This achieves 1-2 orders of magnitude improvement in query latency.
• RN-SDEs: RN-SDEs: Limited-Angle CT Reconstruction with Residual Null-Space Diffusion Stochastic Differential Equations [https://arxiv.org/pdf/2409.13930] uses residual null-space diffusion and SDEs for high-quality CT reconstruction from sparse data.
• MambaTrack3D: In MambaTrack3D: A State Space Model Framework for LiDAR-Based Object Tracking under High Temporal Variation [https://doi.org/10.1007/s11263], a state-space model framework for robust LiDAR-based object tracking is introduced, leveraging the power of SSMs to handle dynamic environments.
• MPCM-Net: MPCM-Net: Multi-scale network integrates partial attention convolution with Mamba for ground-based cloud image segmentation [https://arxiv.org/pdf/2511.11681] combines partial attention convolution with the Mamba architecture for improved cloud image segmentation. Code is available on GitHub.
• ReassembleNet: ReassembleNet: Learnable Keypoints and Diffusion for 2D Fresco Reconstruction [https://arxiv.org/pdf/2505.21117] integrates learnable keypoints and diffusion processes for 2D fresco reconstruction, using a semi-synthetic dataset to address data scarcity.

Impact & The Road Ahead

The implications of these advancements are far-reaching. By tackling computational complexity head-on, researchers are not only making current AI/ML applications more efficient but also paving the way for entirely new capabilities. Imagine real-time multi-agent systems navigating complex urban environments with provable safety guarantees, as explored in Game-Theoretic Safe Multi-Agent Motion Planning with Reachability Analysis for Dynamic and Uncertain Environments (Extended Version) by Consolini, Locatelli, and Saccani (University of Bologna, Politecnico di Milano, Université de Toulouse) [https://arxiv.org/pdf/2511.12160]. Or the ability to process ultra-high-definition medical images faster and more accurately, as demonstrated by Xingchi Chen et al. with 4KDehazeFlow: Ultra-High-Definition Image Dehazing via Flow Matching [https://arxiv.org/pdf/2511.09055].

These papers highlight a clear trend: moving beyond brute-force computation towards architecturally aligned, theoretically sound, and data-driven optimizations. From Abbas Tariverdi’s Robust Self-Triggered Control Approaches Optimizing Sensors Utilization with Asynchronous Measurements [https://arxiv.org/pdf/2511.16253] showing up to 74% reduction in sensor utilization, to DNA Storage in the Short Molecule Regime by Ran Tamir et al. (Universitat Politècnica de Catalunya, Technion, University of Cambridge) [https://arxiv.org/pdf/2511.14284] proving optimal information density with low-complexity partition coding for future DNA-based storage. The future of AI/ML is not just about bigger models, but smarter, more efficient ones, capable of robust, scalable performance across an ever-expanding array of real-world challenges. The relentless pursuit of O(N) and beyond in complexity is truly reshaping the landscape of intelligent systems.