Energy Efficiency: Driving the Next Generation of AI/ML with Hardware-Software Co-Design
Latest 33 papers on energy efficiency: Jul. 25, 2026
The relentless growth of AI/ML, particularly with large models, has thrown a spotlight on a critical bottleneck: energy consumption. From powering massive data centers to enabling intelligence on tiny edge devices, the demand for more energy-efficient solutions is paramount. Recent breakthroughs, as highlighted in a collection of cutting-edge research, are tackling this challenge head-on, showcasing innovative approaches that blend hardware and software co-design, novel architectures, and smart algorithms to significantly reduce power while boosting performance.
The Big Idea(s) & Core Innovations
At the heart of these advancements lies the principle of doing more with less. Many papers converge on the idea of reducing redundant computations and data movement, which are major energy drains. For instance, the BRIM accelerator by authors from the University of Southern California and Yale University, in their paper BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator, targets the workload imbalance in sparse inference. Their Cyclic-Balanced Pruning (CBP) and Pairwise Slot Donation mechanisms boost PE utilization to over 90%, achieving up to 1.63x energy efficiency for CNNs, ViTs, and LLMs. This addresses the insight that independently varying weight and activation sparsity patterns create bottlenecks, which can be mitigated through offline optimization and runtime redistribution.
Similarly, in NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference, researchers from Arizona State University and Hewlett Packard Enterprise Labs introduce NIFA, an FPGA architecture integrating ADC-free, ACAM-based analog in-memory computing (IMC). This allows native nonlinear computation within the IMC core, yielding up to 40x higher energy efficiency for CNNs and 1.9x for Transformer workloads, largely by eliminating ADCs which account for 70% of IMC block area/power.
For LLMs specifically, CIMERA, presented by researchers from the National University of Singapore in CIMERA: Compute-in-Interconnect and Memory with Reconfigurable Precision for LLM Inference, leverages OS FeFET NVM arrays and a 2D-mesh Inter-PE Computational Network (IPCN) for in-network computation. This ground-breaking chiplet-based design achieves up to 25x and 10x higher energy efficiency than Nvidia H100 for 1B and 13B LLMs, respectively, fundamentally by co-locating compute with memory and optimizing data flow. Another notable LLM innovation is Squid by Nexa AI, detailed in Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Models. They introduce a novel decoder-decoder architecture that treats long context as a distinct modality, compressing it into memory tokens using a compact 0.5B decoder, leading to a 10-fold energy efficiency improvement and 5-fold latency reduction for on-device LLMs.
Even in classical signal processing, the drive for efficiency is strong. New Number Formats for FFT IP Cores in Optical OFDM Transceivers from RPTU University Kaiserslautern-Landau and Radyalis LLC shows how custom reduced-precision floating-point formats for FFT cores can achieve 19.8% power reduction and 12.0% area reduction over traditional fixed-point designs, by tailoring number formats to application value distributions.
Edge devices are also seeing significant gains. In Taming the Security-Energy Paradox: A Green AI Approach to Optimized Android Malware Detection, authors from MIE-SPPU Institute of Higher Education demonstrate how INT8 quantization reduces MLP model size by ~3.5x for Android malware detection, cutting energy to just 0.0189 mJ per inference while maintaining over 99.2% accuracy. This resolves the security-energy paradox, making real-time, low-impact mobile security a reality.
Neuromorphic computing offers a radically different paradigm. CIS-Fuse, a spiking neural network for image fusion from Nanyang Technological University and others, as described in Current Injection Spiking Neural Network for Infrared and Visible Image Fusion, achieves fusion quality comparable to state-of-the-art ANNs with ~10x lower inference energy by performing cross-modal integration at the membrane-potential level before binary spike firing. Similarly, the NeuroPnP algorithm by the AI for Space Group at Adelaide University, detailed in Robust PnP on a Neuromorphic Processor for Object Pose Estimation, executes geometric optimization on Intel Loihi 2 with ~99% power reduction over CPU, proving that even non-learning tasks can be energy-efficiently neuromorphic.
Under the Hood: Models, Datasets, & Benchmarks
The innovations discussed rely on a mix of established and newly introduced resources:
- Hardware Platforms: Intel Loihi 2 neuromorphic processor (for NeuroPnP), AMD Kintex UltraScale+ FPGAs (for SparHiXcel-v2), STM32N6 microcontroller with integrated NPU (for ARGO eyewear), 3D FeFET-NAND and DRAM (for ThAME, CIMERA), 3D NAND flash (for D-NOVA, Cross-Domain Acceleration). Specific technologies like 12 nm FinFET (for FFT IP cores) are also critical.
- Models & Architectures: Multi-Layer Perceptrons (MLP) for malware detection, YOLOv11 for object detection (ARGO), Vision Transformers (Opto-ViT-v2), Spiking Neural Networks (CIS-Fuse, SpikeDS, NeuroPnP), Mixture of Experts (MoE) LLMs (ThAME, eNPU), RISC-V Vector Extension (SEAM-V), Reconfigurable-precision Compute-in-Memory (CIMERA), Hyperdimensional Computing (HDC) for mass spectrometry (Cross-Domain Acceleration).
- Datasets & Benchmarks: TUANDROMD and DREBIN (malware detection), RoadScene, MSRS, M3FD, FMB (image fusion), WOTR (urban obstacle detection for ARGO), VTAB-1K, FGVC (ViT fine-tuning), The Pile, Natural Questions, BookCorpus, arXiv (LLM training/evaluation for Squid), NQ, HotpotQA, FEVER, FiQA, wiki_en, MIRACL-English (for D-NOVA in-storage retrieval), E-POSE, FRESH (event-based pose estimation), Intel Berkeley Research Lab (WSN anomaly detection), SPEC CPU2017 (RTM cache evaluation), Nyx cosmology, Miranda turbulence, Hurricane weather (scientific compression for FLARE).
- Code Repositories: Many projects emphasize reproducibility, with some papers directly linking to codebases or frameworks: Hugging Face (all-MiniLM-L6-v2), TensorFlow Lite (TFLite), Skyfield Python library, Python PuLP (for intent-based satellite networks), ST Edge AI v4.0.0, Ultralytics YOLO framework (ARGO), Lava framework, sinabs, Spiking Jelly (neuromorphic), gem5, DESTINY (RTM caches), Open-Sora, Latte, Open-Sora Plan, Wan 2.1, HunyuanVideo, Vchitect-2.0, CogVideoX-5B (video diffusion), OpenHWGroup (RISC-V extensions), OptiCommPy (optical communications), and the RoBust platform for morphing hexarotors. This commitment to open resources accelerates future research.
Impact & The Road Ahead
The implications of this research are far-reaching. We’re seeing AI become more ubiquitous, from privacy-preserving smart eyewear (ARGO in Fully-sensorized smart-eyewear platform for on-device Machine Learning) to ultra-reliable satellite-integrated rural connectivity (Large Language Model Assisted Intent-Based Satellite-Integrated Access and Backhaul FWA for Rural Areas). The energy-efficiency gains translate directly into longer battery life for mobile devices, more sustainable data centers, and the ability to deploy complex AI in resource-constrained environments like drones and space applications.
Beyond current applications, these advancements lay the groundwork for future AI systems. The ability to perform complex LLM inference on-device (Squid) opens up new possibilities for personalized, always-on AI assistants. Novel hardware architectures like ThAME (ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts) and CIMERA pave the way for LLMs that are not just powerful but also economically and environmentally sustainable. The cross-platform comparisons in Cross-Domain Acceleration of Open Modification Search reveal that memory- and storage-centric architectures can deliver >100x speedup and >40,000x energy efficiency over GPUs for certain data-intensive workloads, pointing to a future where specialized hardware dominates. This is further exemplified by D-NOVA from the University of California, San Diego in D-NOVA: In-Storage Retrieval Accelerator via Dual-Bound 3D NAND-Optimized Similarity Search with Vector Adaptation, which achieves true in-storage vector search within 3D NAND memory for RAG, reducing energy by 71x and improving speed by 41.7x.
Challenges remain, such as optimizing global building area estimation (as analyzed in Global Building Area Estimation Products: How Accurate Are They?), managing dynamic environments for drone sensing (Reinforcement Learning for Delivery Drone-Based Participatory Sensing in Dynamic Environments), and ensuring reliable communication in complex wireless networks (HARQ for Slow Fluid Antenna Multiple Access, Cooperative MARL-Based Energy-Efficient Power Control for Two-Hop Relaying Networks, Energy Minimization Oriented Resource Allocation for Integrated Sensing and Communication in Marine IoT Networks). However, the common thread is a deep commitment to energy efficiency. This vibrant research landscape signals a future where AI’s immense potential is realized responsibly, driving innovation across diverse fields while minimizing its ecological footprint.
Share this content:
Discover more from SciPapermill
Subscribe to get the latest posts sent to your email.
Post Comment