Deep Neural Networks: From Zero-Storage to Unbreakable Security, and Beyond!

Latest 16 papers on deep neural networks: Sep. 27, 2026

Deep Neural Networks continue to push the boundaries of AI, but the rapid advancements come with their own set of challenges, from computational overheads and security vulnerabilities to the need for greater interpretability and robust performance in real-world settings. This digest dives into a fascinating collection of recent research that tackles these critical issues head-on, showcasing innovative solutions across diverse subfields.

The Big Idea(s) & Core Innovations

One of the most mind-bending innovations comes from the realm of parameter storage. Imagine neural networks that don’t store their weights. The paper, “Orbital Error Dynamics: Self-Organized Criticality, Ephemeral Parameter Resonance, and Non-Linear Biological Ontologies in Zero-Storage Neural Synthesis” by Volkan Dağlı et al. from Anadolu University and others, introduces Orbital Error Dynamics (OED). This framework generates neural network parameters as transient topological resonances from the Mandelbrot quadratic polynomial map on-demand, rather than storing them statically. This revolutionary O(1) memory footprint (just a 24-byte seed!) challenges traditional O(W) storage, suggesting that intelligence might reside in active boundary dynamics at the ‘edge of chaos.’

Security is another major theme, with new threats and defense strategies emerging. A critical new attack, T-Backdoor, presented by Abdullah Arafat Miah et al. from the University of Rhode Island in their paper “T-Backdoor: Exploiting Temporal Redundancy in Neuromorphic Data for Spike-preserving Backdoor Attacks on SNNs,” targets Spiking Neural Networks (SNNs). This purely temporal backdoor attack uses Rate, Latency, and Jitter triggers that preserve spike distributions, making them nearly undetectable by existing defense mechanisms and achieving near-perfect attack success rates. This highlights a significant security gap in neuromorphic AI. On the defense side, Joseph Bingham from Technion – Israel Institute of Technology, in “Activation-Energy Pruning for Spiking Neural Networks: Unsupervised Personalization via Spike-Count Saliency,” demonstrates that activation-energy pruning improves SNN accuracy (up to 98.4% at 80% sparsity on N-MNIST) where gradient-based methods fail, effectively identifying “lottery ticket” subnetworks.

Model extraction, a critical privacy threat, is also seeing advancements. Fei Yuan et al. from Shandong University and Tsinghua University introduce GuidedRay in “GuidedRay: Diversity-Guided Direction Discovery for Targeted Hard-Label Black-Box Attacks,” which significantly improves targeted black-box adversarial attacks by efficiently discovering diverse adversarial directions using augmented target-class references. This drastically cuts down queries needed for initialization. Complementing this, “End-to-End Hard-Label Cryptanalytic Model Extraction Using Efficient Sign Recovery” by Akira Ito et al. from Tohoku University and NTT, introduces a ‘cosine method’ for sign recovery in hard-label model extraction, achieving higher accuracy without additional queries. Furthering this, Shi Tang et al. from Shandong University and Tsinghua University propose Normal Alignment in “Normal Alignment: Improved Cryptanalytic Sign Recovery on Hard-Label Networks,” a statistical method for hard-label sign recovery that leverages decision-facet normals, achieving 100% dual point utilization and enabling polynomial-time full sign recovery.

Beyond security, interpretability and robust optimization are key. Alexander Caicedo et al. from Pontificia Universidad Javeriana, in “NObSP: Functional Decomposition of Neural Networks via Oblique Subspace Projections,” propose NObSP, a framework for functional decomposition of neural network predictions that provides explicit per-feature contribution functions and handles feature superposition via oblique projections, offering both local and global interpretability. Meanwhile, Alejandro Calatrava et al., in “SAM-on-the-Curve: Sharpness-Aware Mode Connectivity for Robust Weight-Space Interpolation,” introduce Sharp Mode Connectivity (SMC), combining Sharpness-Aware Minimization (SAM) with Mode Connectivity to find low-loss and uniformly flat weight-space paths, leading to significant robustness improvements under distribution shifts and even achieving negative loss barriers.

Finally, practical applications and foundational theory are not forgotten. Cristian Comellas et al. from Universitat de les Illes Balears, in “A Study of the Limits of Collaborative DCT-Based Image Denoising via Interpretable Neural Networks,” present DeepBM3D, a differentiable architecture for collaborative image denoising that integrates classical BM3D principles with learned Wiener weights, achieving competitive performance at low/moderate noise while preserving interpretability. Kexuan Li from Bristol Myers Squibb offers a theoretical advancement in “Generalized Deep Regression for Repeated Measurements,” a framework for generalized deep regression with ReLU networks that achieves optimal convergence rates for repeated measurements, effectively separating between-unit and within-unit variations. Shiva Moshtagh et al. from Arizona State University, in “Machine Learning-Based State Estimation for an Actual Transmission System Using Field PMU Data,” validate DNN-based state estimation on a real-world transmission system with field PMU data, demonstrating accurate and significantly faster (0.15-0.2ms inference) estimates even with low observability. For edge computing, Mahdi Taheri et al. from Humboldt University of Berlin, in “REQAP: Resilient Weight Packing and Quantization for Edge DNN Acceleration,” introduce a reliability-aware methodology for DNN accelerators, achieving significant memory and MAC operation reductions while enhancing fault resilience through selective MSB protection without hardware changes. Completing the picture, Hao Fang et al. from Tsinghua University provide a “Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses” offering a comprehensive taxonomy and analysis of model inversion attacks, highlighting the evolving threat landscape in privacy. Lastly, Alessandro Bombini from Istituto Nazionale di Fisica Nucleare offers valuable “Lecture notes on Physics Informed Neural Networks, Neural Operators, and their applications” covering theoretical foundations and practical implementations of PINNs and Neural Operators for PDEs.

Under the Hood: Models, Datasets, & Benchmarks

These advancements are often powered by novel architectural choices, specific datasets, or rigorous benchmarks:

• OED (Zero-Storage AI): Relies on the Mandelbrot quadratic polynomial map to procedurally derive synaptic weights from a 24-byte seed. Evaluated on Two-Moons dataset.
• T-Backdoor (SNN Security): Leverages the temporal dimension of neuromorphic data on N-MNIST, CIFAR10-DVS, and N-Caltech101 datasets, utilizing the SpikingJelly framework for SNN training. Code available: https://github.com/SiSL-URI/T-Backdoor.
• Activation-Energy Pruning (SNN Optimization): Evaluated on N-MNIST and CIFAR-100, addressing the unique dynamics of Leaky Integrate-and-Fire (LIF) neurons. Code available: https://github.com/JosephBingham/snn_fp.
• GuidedRay (Black-Box Attacks): Tested on CIFAR-10, CIFAR-100, and ImageNet, using ResNet-50 and DenseNet-121 models. Code available: https://github.com/sudyuan/GuidedRay.
• DeepBM3D (Image Denoising): Fully differentiable architecture leveraging non-local patch grouping via PSAL and learned Wiener weights in the DCT domain. Evaluated on DIV2K, Flickr2K, Waterloo Exploration Database, BSD100, Kodak, McMaster, Set14, and Brodatz textures.
• NObSP (Interpretability): Decomposes networks using oblique subspace projections to recover functional forms. Evaluated on tabular and vision benchmarks, including TinyImageNet. Code available: https://github.com/santialferez/nobsp.
• SMC (Robust Optimization): Integrates Sharpness-Aware Minimization (SAM) with Mode Connectivity for robust weight-space paths. Validated across ResNet-18, VGG16-BN, and ViT-Tiny on CIFAR-10 and CIFAR-10-C (blur corruptions).
• Generalized Deep Regression: Employs ReLU deep neural networks for repeated measurements, with a potential application to the Capital Bikeshare dataset.
• DNN-SE (Power Systems): Validated on a large-scale US transmission system (System S) with field PMU data. Uses DNNs for voltage and phase angle estimation.
• REQAP (Edge Acceleration): Co-designs mixed-precision quantization and register packing for systolic-array-based accelerators. Evaluated with AlexNet on Fashion MNIST and VGG-11, ResNet-18 on CIFAR-10.
• Model Extraction (Hard-Label): New cosine method for sign recovery, demonstrated on MNIST and Fashion-MNIST with deep ReLU MLPs. Normal Alignment also on CIFAR-10 and MNIST.
• FedASAP (Federated Learning): Two-stage structured adaptive pruning using activation statistics on MICCAI WMH and FeTS datasets for lesion segmentation. Code available: https://github.com/Tarun2201/FedASAP_development.
• PINNs & Neural Operators: Comprehensive lecture notes covering DeepONets, Kolmogorov-Arnold Networks (KANs/PIKANs), and Fourier Neural Operators (FNOs). Implementation guidance with PyTorch and NVIDIA PhysicsNeMo. Code available: https://github.com/androbomb/PINN_Course_2026.

Impact & The Road Ahead

These diverse breakthroughs point to an exciting future for deep neural networks. Zero-storage AI could redefine how we think about model deployment on resource-constrained devices, potentially enabling truly edge-native, ephemeral intelligence. The advancements in SNN security, both in understanding new threats like T-Backdoor and developing robust defenses through activation-energy pruning, are crucial for the safe development of neuromorphic computing. Similarly, the enhanced model extraction techniques underscore the urgent need for more robust privacy-preserving ML, with the comprehensive survey providing a roadmap for future research in this area.

On the practical side, the DeepBM3D work shows that classical signal processing wisdom can be effectively blended with deep learning for interpretable and efficient solutions. The validation of DNN-based state estimation on real power grids opens doors for ultra-fast, data-driven control of critical infrastructure. REQAP’s focus on resilient, quantized edge acceleration directly addresses the deployment challenges of AI in the IoT era. Meanwhile, interpretability frameworks like NObSP are vital for building trust and understanding in complex AI systems, moving us closer to AI that can truly explain its reasoning. Finally, the theoretical underpinnings provided by the work on functional composition and generalized deep regression, alongside the educational resources on PINNs and Neural Operators, ensure that the field continues to build on a strong scientific foundation. The journey toward more efficient, secure, interpretable, and powerful deep neural networks is accelerating, promising transformative impacts across science, industry, and daily life.

Please leave this field empty

Hi there 👋

Get a roundup of the latest AI paper digests in a quick, clean weekly email.

Check your inbox or spam folder to confirm your subscription.