LLM Compression: Cracking the Code on Efficiency, Robustness, and Fairness

Latest 50 papers on model compression: Sep. 21, 2025

The relentless rise of AI, particularly large language models (LLMs) and complex deep neural networks (DNNs), has introduced a fascinating paradox: incredible capabilities balanced against immense computational demands. This isn’t just about raw power; it’s about deploying these intelligent systems efficiently, robustly, and fairly in a world constrained by energy, memory, and latency. The latest research in model compression offers a thrilling glimpse into how we’re tackling these challenges head-on.

The Big Idea(s) & Core Innovations

Recent breakthroughs highlight a multi-faceted approach to model compression, moving beyond mere size reduction to consider performance, security, and ethical implications. A central theme is the synergistic integration of various compression techniques, often leading to superior results than individual methods alone.

For instance, the paper, “SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression” by Mohammad Mozaffari, Amir Yazdanbakhsh, and Maryam Mehri Dehnavi from the University of Toronto, Google DeepMind, and NVIDIA Research, introduces SLIM. This unified framework masterfully combines quantization, sparsity, and low-rank approximation in a one-shot manner, sidestepping the need for extensive retraining. This approach not only boosts accuracy by up to 5.66% over prior methods but also significantly reduces memory and accelerates inference.

Complementing this, the work by Chao Zeng and affiliated researchers from ByteDance Inc. in “GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference” presents GQSA. This framework leverages group sparsity and structured quantization to achieve a better accuracy-speed trade-off, proving especially effective for edge device deployment by optimizing GEMV operations.

Pruning, another cornerstone of compression, sees innovative advancements. “GAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings” by Yixuan Tang and Yi Yang from The Hong Kong University of Science and Technology introduces GAPrune, a framework that leverages Fisher Information and gradient alignment. This allows for pruning that enhances domain-specific capabilities rather than hindering them, demonstrating that careful compression can lead to more specialized and efficient models.

Beyond just efficiency, robustness and fairness are emerging as critical considerations. P. Kassianik, E. J. Hu, and others from FoundationAI and ICL in their paper, “AQUA-LLM: Evaluating Accuracy, Quantization, and Adversarial Robustness Trade-offs in LLMs for Cybersecurity Question Answering”, highlight a crucial trade-off: while quantization boosts efficiency, it can compromise adversarial robustness, particularly in sensitive domains like cybersecurity. This is echoed in “Model Compression vs. Adversarial Robustness: An Empirical Study on Language Models for Code” by Md. Abdul Awal and co-authors from the University of Saskatchewan, who show that compressed code language models, especially those using knowledge distillation, are more vulnerable to adversarial attacks. On the fairness front, Nannan Huang and colleagues from RMIT University, Australia, in “Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions”, introduce HGLA pruning, which actively improves or maintains fairness during compression, addressing critical ethical concerns.

New paradigms are also emerging for specialized applications. “MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration” by Alex M. R. 09 introduces MaRVIn, an open-source framework that tightly integrates mixed-precision neural networks with RISC-V hardware for significant energy efficiency gains. Similarly, in “An Efficient GNNs-to-KANs Distillation via Self-Attention Dynamic Sampling with Potential for Consumer Electronics Edge Deployment”, Can Cui and co-authors from Dalian Jiaotong University leverage knowledge distillation to efficiently transfer knowledge from GNNs to the more efficient Kolmogorov-Arnold Networks (KANs), ideal for consumer electronics.

Under the Hood: Models, Datasets, & Benchmarks

These innovations are often built upon or validated using diverse models, datasets, and benchmarks, showcasing real-world applicability:

• LLMs & Transformers: LLaMA-2-7B, Llama-3.1-8B, Qwen2.5-7B, Pythia, CodeGen, GPT-Neo, DeepSeek-V3-0324, Kimi-K2-Instruct, Qwen3-235B-A22B-2507, Whisper Medium model. Many papers, like “CALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression” by MUCHAMMAD DANIYAL KAUTSAR et al. (IEEE Transactions on AI, Meta, Google Research, UC Berkeley, Stanford, MIT) focus directly on compressing these large models without compromising performance.
• Vision Models: CNNs, Vision Transformers (ViT), Video Diffusion Models (VDMs). The paper “MOR-VIT: Efficient Vision Transformer with Mixture-of-Recursions” by YiZhou Li from XJTLU introduces MoR-ViT, demonstrating a 70% parameter reduction and 2.5x inference acceleration on ImageNet-1K, outperforming DynamicViT and TinyViT.
• Neuromorphic Hardware: Intel Loihi 2, featured in “Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity” by Alessandro Pierro and colleagues from Intel Corporation, where unstructured sparsity leads to 42x lower latency and 149x lower energy consumption for RNNs.
• Quantum Computing: D-Wave’s annealing hardware is used in “Is Quantum Optimization Ready? An Effort Towards Neural Network Compression using Adiabatic Quantum Computing” by **Zhehui Wang and co-authors from A*STAR, Singapore**, to explore adiabatic quantum computing for pruning-quantization.
• Benchmarking & Toolkits: LLMC+ is introduced in “LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit” by Chengtao Lv and colleagues from Nanyang Technological University and SenseTime Research. This framework allows for systematic study of token-level and model-level compression in VLMs. Code for several methods is publicly available, such as GAPrune on github.com/yixuantt/GAPrune and SLiM on github.com/Mohammad-Mozaffari/slim.

Impact & The Road Ahead

This wave of research profoundly impacts the practicality and reach of AI. Efficient model deployment on resource-constrained edge devices – from smartphones to autonomous vehicles to consumer electronics – is now more feasible than ever. Consider how “Compressing CNN models for resource-constrained systems by channel and layer pruning” by A. Sadaqa and D. Liu and “OWLed: Outlier-weighed Layerwise Pruning for Efficient Autonomous Driving Framework” by Jiaxi Li from USTC are making CNNs and autonomous driving systems lighter and faster without compromising critical performance.

Beyond just speed, these advancements are opening doors to new applications. The concept of Agentic AI discussed in “Toward Edge General Intelligence with Agentic AI and Agentification: Concepts, Technologies, and Future Directions” by Y. Zhang and collaborators from Tsinghua University and Stanford University relies heavily on model compression to enable autonomous, memory-enabled, and context-aware systems at the edge. Even more intriguingly, “CognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning” by A. A. Cifuentes-Cuadros and others demonstrates how model compression underpins real-time brain-computer interfaces, pushing the boundaries of assistive technology.

The road ahead involves a continued push for data-free compression techniques, as seen in “Forget the Data and Fine-Tuning! Just Fold the Network to Compress” by Dong Wang and colleagues from Graz University of Technology, which can drastically simplify the compression pipeline. Further exploration into privacy-preserving compression, as highlighted by “How Quantization Impacts Privacy Risk on LLMs for Code?” by Md Nazmul Haque and colleagues from North Carolina State University, will be crucial as AI permeates sensitive domains. Finally, achieving better convergence and accuracy in distributed and federated learning through sophisticated compression, as detailed in the dissertation “Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization” by Kai Yi from KAUST, promises to unlock scalable and secure AI across decentralized networks.

This flurry of innovation underscores a dynamic field where researchers are not just making models smaller, but smarter, safer, and more universally deployable. The future of efficient and responsible AI is being built layer by layer, with each compression breakthrough bringing us closer to ubiquitous, powerful intelligence.