O(N) & O(N²): Unlocking Efficiency and Scale in AI/ML and Beyond

Latest 46 papers on computational complexity: Apr. 25, 2026

The relentless pursuit of efficiency and scalability is a constant in AI/ML, driving innovation across diverse fields from autonomous systems to fundamental computational theory. The ability to manage computational complexity, particularly to push beyond quadratic (O(N²)) bottlenecks towards linear (O(N)) or even sub-linear performance, is a game-changer. Recent research highlights a fascinating spectrum of advancements, from new algorithms for real-time applications to theoretical breakthroughs redefining what’s possible. Let’s dive into some of these exciting developments.

The Big Idea(s) & Core Innovations

The central theme across these papers is the ingenious ways researchers are tackling the inherent computational challenges of complex problems. Many innovations revolve around rethinking problem formulations and leveraging new architectural paradigms to drastically reduce complexity. For instance, in Large Language Models (LLMs), the quadratic scaling of attention with context length (N) is a major hurdle. To address this, Jinyu Guo, Zhihan Zhang et al. from the University of Electronic Science and Technology of China and other institutions introduce DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing. Their core insight is to reframe attention as an approximate nearest-neighbor search using asymmetric deep hashing, replacing floating-point operations with efficient bitwise comparisons. This novel approach slashes complexity to O(N).

Similarly, in video understanding, self-attention’s quadratic cost is a bottleneck for temporal action detection. Zepeng Sun et al., in LiquidTAD: An Efficient Method for Temporal Action Detection via Liquid Neural Dynamics, replace self-attention with parallelized liquid neural dynamics, achieving O(N) linear complexity. This continuous-time approach, inspired by biological systems, dramatically reduces parameters while maintaining accuracy. This echoes the trend seen in medical imaging where Ahmed Marouane Djouamaa et al., with RF-HiT: Rectified Flow Hierarchical Transformer for General Medical Image Segmentation, achieve O(N) linear complexity for hierarchical transformers for fast inference in generative models.

Another significant innovation lies in decoupling and specializing processing pathways. For blood glucose forecasting, Ebrahim Farahmand et al. from Arizona State University in Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Blood Glucose Forecasting use Variational Mode Decomposition to split glucose signals into low- and high-frequency components, processed by separate LSTMs and Transformers. This specialization, combined with knowledge distillation, slashes parameters by 62%. Tejeswar Pokuri and Shivarth Rai from Manipal Institute of Technology adopt a similar philosophy for underwater image enhancement with Hero-Mamba: Mamba-based Dual Domain Learning for Underwater Image Enhancement. Their dual-domain learning processes spatial (RGB) and spectral (FFT) features in parallel using Mamba-based state space models, leveraging the linear complexity of Mamba for global context.

In distributed systems, Anil Jangam et al. from Cisco Systems, Inc., in Predictive Bayesian Arbitration: A Scalable Noisy-OR Model with Service Criticality Awareness, use an Adaptive Bayesian Noisy-OR model for predictive failure arbitration. This method maintains O(N) complexity while dynamically learning cascade dependencies, moving from reactive to proactive system management.

From a theoretical standpoint, Piotr Kawalek and Jacek Krzaczkowski from TU Wien and Maria Curie-Skłodowska University delve into foundational aspects with Complexity Classes Arising from Circuits over Finite Algebraic Structures. They establish a unifying algebraic framework, proving that circuits over nilpotent Malcev algebras recognize the CC0 class, and solvable Malcev algebras recognize the nuItDet class (iterated determinants), essentially showing that “circuits over solvable algebras are iterated determinants in disguise.” This fundamental work bridges universal algebra and circuit complexity, characterizing the computational power of various algebraic structures.

Under the Hood: Models, Datasets, & Benchmarks

These innovations are often underpinned by novel architectures, optimized data handling, or rigorous evaluation on established benchmarks. Here’s a look at the key resources driving these advancements:

• LLM Efficiency: DASH-KV uses the LongBench benchmark and evaluates on Qwen2-7B-Instruct, Llama-3.1-8B-Instruct, and Qwen2.5-14B-Instruct models. Its core relies on asymmetric deep hashing and a dynamic mixed-precision attention mechanism.
• Medical Image Segmentation: RF-HiT leverages an O(N) hierarchical Hourglass Transformer backbone with rectified flow and a Hierarchical Feature Encoder (HFE), tested on ACDC and BraTS 2021 datasets.
• Underwater Image Enhancement: Hero-Mamba features Mamba-based SS2D blocks, a physics-guided ColorFusion block, and an MS-Fusion block, benchmarked on UIEB and LSUI datasets.
• Temporal Action Detection: LiquidTAD utilizes parallelized ActionLiquid blocks based on closed-form continuous-time liquid neural dynamics, evaluated on THUMOS-14, ActivityNet-1.3, and Ego4D datasets using I3D, SlowFast, TSP, and InternVideo features.
• Blood Glucose Forecasting: GlucoNet employs Variational Mode Decomposition (VMD), LSTM for trends, and knowledge-distilled Transformer for fluctuations, validated on OhioT1DM (2018/2020) and AZT1D datasets.
• Smart Building Occupancy: M. Farjadnia et al. from KTH Royal Institute of Technology compare Logistic Regression, SVM, and LSTM with attention using data from the KTH Live-In Lab and IDA ICE simulation environment in Generalizability of Learning-based Occupancy Detection in Residential Buildings.
• Distributed Energy Resources (DER) Coordination: Ge Chen et al. from Great Bay University and Sichuan University present a two-layer Eulerian coordination framework using finite-volume discretization with flux-lifting for population-size independent complexity, detailed in A Convexified Eulerian Framework for Scalable Coordination of Massive DER Populations.
• Cell-Free XL-MIMO: Dogon Kim et al. from Jeonbuk National University and Hanyang University propose A-FP, an accelerated fractional programming algorithm for efficient uplink reception, with fronthaul overhead independent of antenna count, as described in Efficient Design of Fronthaul-Constrained Uplink Reception for Cell-Free XL-MIMO.
• LLM Optimization: Tianhao Tang et al. from HKUST and PolyU introduce GRACE: A Dynamic Coreset Selection Framework for Large Language Model Optimization, using k-NN graphs and gradient-based importance scores for adaptive coreset selection on MathInstruct, BioInstruct, and DialogSum benchmarks.
• Vision Foundation Model Acceleration: Jiawei Fan et al. from Intel Labs China present Chain-of-Models Pre-Training (CoM-PT), leveraging inverse weight initialization and feature distillation across ViT and Swin transformer families for training acceleration.
• Sparse Signal Recovery: Tatsuki Tokumura et al. from Nagoya Institute of Technology propose DU-PSISTA (Deep Unfolded-Periodic Sketched ISTA), combining periodic sketching with deep unfolding to optimize step sizes and thresholding for reduced computational complexity.
• Multichannel Audio Processing: Yuancheng Luo et al. from Amazon.com present a Constraint Optimized Multichannel Mixer-limiter Design (2507.06769) formulated as a quadratic program, using asymmetric COLA windows and variable/constraint reduction techniques for real-time audio.
• Medical Image Segmentation: Libin Lan et al. from Chongqing University of Technology and Southeast University introduce MSLAU-Net: A Hybrid CNN-Transformer Network for Medical Image Segmentation with a Multi-Scale Linear Attention (MSLA) module, evaluated on Synapse, ACDC, and CVC-ClinicDB.
• Multi-View 3D Object Detection: Danish Nazir et al. from Volkswagen AG and TU Braunschweig propose a method for Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning, featuring dynamic layer-wise token selection and parameter-efficient fine-tuning, tested on the NuScenes dataset.
• Fungal Automata: E. Formenti et al. in Complexity of Fungal Automaton Prediction analyze freezing totalistic 1D rules to show P-completeness for the majority rule at radius 1.5. This is a theoretical work without external code or datasets.
• Gram-Legendre Curves: Filip Chudy and Paweł Woźny from the University of Wrocław present methods for Evaluation of Gauss-Legendre curves, with O(n² + dn) algorithms for single point and O(Mdn + dn²) for multipoint evaluation, including code on GitHub.
• Financial Market Design: Daniel Aronoff and Robert Townsend from MIT propose A Smart-Contract to Resolve Multiple Equilibrium in an Intermediated Repo Trade for the US Treasuries repo market, using zero-knowledge proofs for privacy-preserving auditing.

Impact & The Road Ahead

The collective impact of this research is profound, pushing the boundaries of what’s computationally feasible across numerous domains. In real-time AI, the shift to O(N) complexity in LLMs and temporal action detection opens doors for on-device deployment of advanced models, making AI more accessible and sustainable. The advancements in medical imaging and healthcare AI promise more efficient diagnostics and personalized patient care through lightweight, accurate models.

For autonomous systems, innovations in game-theoretic navigation, multi-view 3D object detection, and efficient DER coordination pave the way for safer, more robust, and economically viable smart infrastructure. The theoretical work on computational complexity classes for algebraic structures and quantum Hamiltonians provides foundational insights, guiding future algorithm design by clarifying the inherent limits and potential breakthroughs in problem solving.

The increasing emphasis on interpretable AI and explainable results, as seen in time series forecasting and conceptual clustering, also promises to build greater trust and transparency in AI decision-making. The emergence of hybrid photonic-electronic systems signals a future where specialized hardware could fundamentally reshape computational paradigms, delivering unprecedented speed and energy efficiency for demanding tasks like matrix multiplication.

Looking ahead, we can expect continued exploration of novel architectural designs that leverage the strengths of different computational models (e.g., CNN-Transformer hybrids, Mamba-based systems). The integration of physics-guided priors and generative learning will likely become more prevalent, enabling models to learn from less data and generalize more robustly. Furthermore, the drive for on-device intelligence will necessitate relentless pursuit of extreme efficiency, pushing algorithms to their theoretical limits. This vibrant research landscape promises an exciting future where AI continues to grow not just in capability, but also in practicality and accessibility, truly transforming our world one efficient computation at a time.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.