O(N) and O(T) Scalability: Unlocking Efficiency in the Age of AI

Latest 50 papers on computational complexity: Dec. 21, 2025

The relentless march of AI and Machine Learning innovation continually pushes the boundaries of what’s possible, yet often collides with the formidable wall of computational complexity. From processing massive datasets to enabling real-time intelligent systems, the demand for efficiency that scales linearly—or even sub-linearly—with data size or sequence length is paramount. This blog post dives into recent breakthroughs from a collection of cutting-edge research papers that are reshaping this landscape, focusing on approaches that achieve remarkable O(N) or O(T) computational complexity, making advanced AI more accessible and practical.

The Big Idea(s) & Core Innovations

Many of the recent advancements coalesce around the central theme of linearizing complexity where quadratic or higher-order scaling once dominated. In the realm of large language models, the paper Multiscale Aggregated Hierarchical Attention (MAHA): A Game Theoretic and Optimization Driven Approach to Efficient Contextual Modeling in Large Language Models by Caner Erden from Sakarya University of Applied Science, Türkiye, introduces MAHA. This groundbreaking attention mechanism transforms quadratic complexity to near-linear by using multiscale decomposition and an optimization-driven aggregation, offering a mathematically rigorous approach to dynamic context balancing. This directly tackles the scalability bottleneck for long-context tasks in LLMs.

Similarly, in computer vision, MMMamba from researchers at Xiamen University, HKUST, USTC, and Huawei Research, detailed in their paper MMMamba: A Versatile Cross-Modal In Context Fusion Framework for Pan-Sharpening and Zero-Shot Image Enhancement, leverages the Mamba architecture to achieve efficient cross-modal fusion with linear computational complexity. This enables zero-shot image super-resolution and pan-sharpening, a significant leap from traditional methods. Parallel to this, in Generative AI for Video Translation: A Scalable Architecture for Multilingual Video Conferencing, a team from Yildiz Technical University proposes a system-level framework that reduces video translation complexity from O(N²) to O(N) through a novel Token Ring mechanism and Segmented Batched Processing, crucial for real-time multilingual conferencing. Furthermore, Efficient Action Counting with Dynamic Queries by researchers from Peking University reduces temporal repetition counting complexity from O(T²C) to O(TC) by introducing dynamic action queries and inter-query contrastive learning, enhancing efficiency for video analysis.

Beyond vision and language, efficiency gains are also making waves in signal processing and scientific computing. For temporal graph link prediction, Efficient Neural Common Neighbor for Temporal Graph Link Prediction by Peking University researchers introduces TNCN, a model with linear complexity that significantly outperforms GNN baselines in speed while achieving state-of-the-art accuracy. Meanwhile, in advanced wireless communications, Rotatable IRS-Assisted 6DMA Communications: A Two-timescale Design proposes a two-timescale approach for Intelligent Reflecting Surfaces (IRS) to dynamically adapt to channel conditions, enhancing efficiency in 6DMA networks. Even in core optimization, Accelerated Decentralized Constraint-Coupled Optimization: A Dual² Approach from The Hong Kong University of Science and Technology introduces iD2A and MiD2A, offering linear and asymptotic convergence for decentralized constraint-coupled problems, significantly lowering communication and computational complexities.

Under the Hood: Models, Datasets, & Benchmarks

The innovations highlighted are not just theoretical; they are often accompanied by new models, datasets, or benchmarks that validate their efficacy and provide resources for the community:

• MAHA (https://github.com/can-erderen/MAHA-Project): A novel attention mechanism for LLMs, demonstrating near-linear complexity for long-context tasks. Its effectiveness is shown through theoretical proofs and practical implementations that balance local and global context.
• MMMamba (https://github.com/Gracewangyy/MMMamba): Utilizes the Mamba architecture for cross-modal fusion, with an in-context conditioning paradigm and multimodal interleaved scanning, enabling zero-shot image super-resolution and efficient pan-sharpening.
• YOLO11-4K (https://github.com/ultralytics/ultralytics): Introduced in YOLO11-4K: An Efficient Architecture for Real-Time Small Object Detection in 4K Panoramic Images by UNSW, this one-stage detector addresses 4K panoramic image challenges. It features lightweight GhostConv/C3Ghost modules and a P2 detection head, achieving real-time inference at 21.4 ms per 4K frame, nearly five times faster than YOLO11. It’s benchmarked on the newly annotated CVIP360 dataset (https://github.com/huma-96/CVIP360_BBox_Annotations) for high-resolution small object detection.
• StageVAR (https://github.com/sen-mao/StageVAR): A framework for accelerating visual autoregressive models (StageVAR: Stage-Aware Acceleration for Visual Autoregressive Models) by Nankai University, City University of Hong Kong, MBZUAI, and Linköping University. It leverages stage-aware design, semantic irrelevance, and low-rank features to achieve up to 3.4x speedup on GenEval and DPG benchmarks.
• DB2-TransF (https://github.com/SteadySurfdom/DB2-TransF): A time series forecasting model from G B Pant and IIT Dharwad (DB2-TransF: All You Need Is Learnable Daubechies Wavelets for Time Series Forecasting) replaces self-attention with learnable Daubechies wavelets, achieving superior accuracy and efficiency across 13 diverse datasets.
• FRWKV (https://github.com/yangqingyuan-byte/FRWKV): Presented by Northeastern University (FRWKV: Frequency-Domain Linear Attention for Long-Term Time Series Forecasting), this model integrates frequency-domain analysis with linear attention, offering O(T) complexity and significant accuracy gains for long-term time series forecasting across eight benchmarks.
• CAPRMIL (https://github.com/mandlos/CAPRMIL): A Multiple Instance Learning (MIL) framework from the National and Kapodistrian University of Athens and CentraleSupélec (CAPRMIL: Context-Aware Patch Representations for Multiple Instance Learning) that achieves linear computational complexity with respect to bag size, significantly improving efficiency for whole-slide image analysis.
• T-SKM-Net (https://github.com/IDO-Lab/T-SKM-Net): A trainable neural network framework for linear constraint satisfaction from Zhejiang University (T-SKM-Net: Trainable Neural Network Framework for Linear Constraint Satisfaction via Sampling Kaczmarz-Motzkin Method), ensuring zero constraint violations and high computational efficiency.
• AHPP (https://github.com/longlonglin/AHPP): A scalable similarity search algorithm for large attributed bipartite graphs by Tsinghua University researchers (Scalable Similarity Search over Large Attributed Bipartite Graphs), demonstrating 1-2 orders of magnitude speedup over existing methods.

Impact & The Road Ahead

These advancements herald a new era of scalable AI, where the benefits of complex models can be realized in real-world scenarios without prohibitive computational costs. The shift towards O(N) or O(T) complexity across diverse domains like image compression (TreeNet: A Light Weight Model for Low Bitrate Image Compression), generative AI, computer vision, and time series forecasting is enabling faster inference, reduced energy consumption, and broader deployment on edge devices. For instance, the acceleration in generative models could lead to instant, high-quality content creation, while efficient object detection in 4K panoramic images will power next-generation autonomous systems and AR/VR experiences.

Looking ahead, the emphasis on linear scalability will continue to drive algorithmic innovation. Future research will likely explore further optimization in hardware-software co-design, novel architectures that inherently possess linear complexity, and broader applications of these efficient techniques to even more challenging real-world problems. The ultimate goal remains to make sophisticated AI not just powerful, but also ubiquitously accessible and sustainable. The journey to a truly efficient AI future is well underway, and these papers are lighting the path forward.