Explainable AI: Demystifying Models from Pixels to Policies and Beyond
Latest 16 papers on explainable ai: May. 2, 2026
Explainable AI (XAI) continues to be a pivotal frontier in artificial intelligence, promising to lift the veil of opacity from complex models and foster trust, particularly in high-stakes domains. Recent research underscores a crucial shift: moving beyond mere technical transparency to genuinely understanding and influencing human perception, learning, and decision-making. This digest explores cutting-edge advancements, from rethinking XAI evaluation to quantum-powered interpretability and novel applications in cybersecurity, mental health, and even creative AI.
The Big Ideas & Core Innovations
The overarching theme in recent XAI breakthroughs is a multifaceted approach to interpretability. One critical insight comes from the paper, “Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings”, by Inês Oliveira e Silva and colleagues from the University of Porto and Feedzai. They reveal a profound disconnect: traditional quantitative XAI metrics like sparsity and faithfulness often fail to correlate with human-perceived clarity or confidence. Alarmingly, their large-scale study on fraud detection showed that explanations consistently boosted human confidence without improving accuracy, highlighting a significant automation bias risk. This emphasizes the need for human-centered XAI evaluation.
Building on the human element, “CoAX: Cognitive-Oriented Attribution eXplanation User Model of Human Understanding of AI Explanations” from National University of Singapore, Singapore researchers Louth Bin Rawshan, Zhuoyu Wang, and Brian Y. Lim, introduces CoAX. This cognitive model simulates how humans reason with attribution XAI, demonstrating that a ‘Attribution sum’ strategy is common for SHAP-like explanations, while LIME-style explanations lead to diverse, sometimes flawed, reasoning. CoAX achieves a remarkable 98.8% correlation with human decisions, outperforming traditional ML proxies and offering a cheaper way to test XAI hypotheses at scale. Similarly, the position paper “Using Learning Theories to Evolve Human-Centered XAI: Future Perspectives and Challenges” by Karina Cortiñas-Lorenzo and Gavin Doherty from Trinity College Dublin advocates for reframing XAI through learning theories, emphasizing that fostering active learning and reflection, rather than just providing information, is key to mitigating risks like over-reliance.
On the technical front, several papers introduce novel methods for generating robust and relevant explanations. For instance, “Binary Spiking Neural Networks as Causal Models” by Aditya Kar, Emiliano Lorini, and Timothée Masquelier from Institut de Recherche en Informatique de Toulouse (IRIT), France, maps Binary Spiking Neural Networks (BSNNs) to binary causal models, allowing for the computation of abductive explanations. Crucially, their method guarantees that explanations contain only causally relevant features, a significant improvement over methods like SHAP, which they show can incorrectly identify irrelevant features. This formal rigor is a leap forward for trustworthy XAI in spiking networks.
Another innovative approach comes from Francesco Aldo Venturella and his team from BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain, in their paper “Towards interpretable AI with quantum annealing feature selection”. They propose a quantum annealing-based feature selection method for interpreting CNNs, formulating it as a QUBO problem that identifies the most informative and non-redundant feature maps, demonstrating improved class disentanglement over GradCAM. This signals a promising avenue for leveraging quantum computing for advanced interpretability.
Applications of XAI are expanding rapidly. In cybersecurity, “eDySec: A Deep Learning-based Explainable Dynamic Analysis Framework for Detecting Malicious Packages in PyPI Ecosystem” by Sk Tanzir Mehedi et al. from Queensland University of Technology, Australia, uses dynamic behavioral analysis and XAI (SHAP, LIME) to achieve 99% accuracy in detecting malicious Python packages, identifying Process_Operations and IO_Operations as key features. Meanwhile, “SDNGuardStack: An Explainable Ensemble Learning Framework for High-Accuracy Intrusion Detection in Software-Defined Networks” from University of Barishal, Bangladesh also integrates SHAP for transparent intrusion detection in Software-Defined Networks, achieving near-perfect accuracy and pinpointing critical features like Flow ID and Bwd Header Len.
Even in niche but vital areas like mental health and food quality, XAI is making strides. Yusif Ibrahimov et al. from the University of York survey “Explainable AI for Mental Disorder Detection on Social Media: A Survey and Outlook”, highlighting the growth of LLM-driven approaches and the need for clinically meaningful explanations. For the food industry, Leonardo Arrighi et al. from University of Trieste, Italy, in “Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review”, review over 100 studies, showcasing how SHAP and Grad-CAM enhance transparency in tasks like contamination detection and freshness assessment.
Beyond traditional applications, XAI is being pushed into interactive and creative realms. “Learning-to-Explain through 20Q Gaming: An Explainable Recommender for Cybersecurity Education” by Mary Nusrat et al. from University of North Texas transforms cybersecurity training into an interactive 20 Questions game, where a reinforcement learning agent provides transparent, contextualized explanations. “Tell Me Why: Designing an Explainable LLM-based Dialogue System for Student Problem Behavior Diagnosis” by Zhilin Fan et al. from Beijing Normal University, China, introduces an LLM-based diagnostic system for student behavior that uses hierarchical attribution to generate natural-language explanations, significantly increasing teacher trust. Furthermore, “AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe” by Adam Cole and Mick Grierson from University of the Arts London allows artists to manipulate cross-attention maps in Video Diffusion Transformers, revealing that cross-attention acts more as a spatial distributor than a geometry engine, opening new artistic possibilities and pushing the boundaries of XAI for creative exploration. Finally, for time series, Annemarie Jutte et al. from Saxion University of Applied Sciences, The Netherlands, introduce C-SHAP, a concept-based XAI method that provides high-level temporal explanations (e.g., trend, bias), aligning explanations with human intuition in domains like human activity recognition and predictive maintenance.
Under the Hood: Models, Datasets, & Benchmarks
These advancements are powered by significant contributions to models, datasets, and evaluation methodologies:
- CoAX Model: A cognitive model based on instance-based learning, designed to simulate human interpretation of XAI explanations. Tested against human decision data.
- Binary Spiking Neural Networks (BSNNs) & Binary Causal Models (BCMs): A formal mapping and logic-based framework using SAT and SMT solvers (e.g., Z3 solver) for causally relevant explanations in BSNNs. Benchmarked on MNIST.
- Quantum Annealing-based Feature Selection: A novel QUBO formulation combining GradCAM importance with cosine similarity for interpreting CNNs. Utilizes ResNet-18 on the STL-10 dataset with D-Wave Ocean. Code available at https://github.com/checc1/FS_QA.
- eDySec Framework: A deep learning framework (MLP, CNN, LSTM, Transformer) with FLAML-based feature selection for malicious PyPI package detection. Uses the QUT-DV25 dataset. Code available at https://github.com/tanzirmehedi/eDySec.
- SDNGuardStack: An ensemble learning model (Decision Tree, Extra Trees, MLP base learners with LightGBM meta-learner) for SDN intrusion detection. Evaluated on the InSDN dataset.
- BayesL: A logical framework for verifying Bayesian Networks, implemented with an open-source tool (https://zenodo.org/records/19834264) capable of dynamic auxiliary variable construction for model checking.
- FAIR_XAI: Investigates Vision-Language Models (VLMs) like Phi-3.5-Vision and Qwen2-VL for zero-shot depression classification, benchmarked across AFAR-BSFT and E-DAIC datasets.
- Hierarchical Clustering for Speaker Recognition: Applies SLINK and HDBSCAN to speaker recognition network representations (ResNet34 on VoxCeleb datasets) for semantic interpretation, introducing Liebig’s score (L-score).
- LLM-based Dialogue System for Education: Fine-tuned Qwen2.5-3B-Instruct with a hierarchical attribution method. Code at https://github.com/zhilinfan/AIED2026-Explainable-Dialogue-System.
- AttentionBender: An inference-time network bending tool for Video Diffusion Transformers (e.g., WAN 2.1). Visualizations and results at https://attention-bender.netlify.app/.
- C-SHAP for Time Series: Extends SHAP with time series decomposition (DWT using PyWavelets) for concept-based explanations. Applied to OPPORTUNITY and Turbofan datasets.
Impact & The Road Ahead
The implications of this research are profound. The critical audit of Shapley benchmarks in “Rethinking XAI Evaluation” serves as a stark warning: we must move beyond purely technical XAI metrics and embrace human-centered evaluation to prevent automation bias and ensure trust in high-stakes decisions. This resonates with the call from “Using Learning Theories to Evolve Human-Centered XAI” to design XAI for learning, not just information transfer, and from CoAX on understanding actual human reasoning with explanations.
The development of causally relevant explanations for Spiking Neural Networks via “Binary Spiking Neural Networks as Causal Models” and quantum-powered interpretability in “Towards interpretable AI with quantum annealing feature selection” signifies a push towards more rigorous, computationally advanced, and trustworthy XAI. These methods promise to unlock deeper insights into complex models, paving the way for AI that is not just powerful but also truly transparent and verifiable.
In practical domains, the integration of XAI into cybersecurity (eDySec, SDNGuardStack), mental health (“Explainable AI for Mental Disorder Detection on Social Media”), and food quality (“Explainable Artificial Intelligence Techniques for Interpretation of Food Models”) will foster greater adoption and reliability of AI systems where errors have serious consequences. The ability to explain why a malicious package was flagged or a mental health condition detected will be crucial for professional users.
Furthermore, the emergence of XAI in interactive systems like the 20Q cybersecurity game and the LLM-based educational dialogue system demonstrates XAI’s potential to transform learning and human-AI collaboration. AttentionBender even redefines XAI for artistic exploration, turning black-box models into malleable creative mediums.
The road ahead for XAI will involve deeper integration of human cognitive models, further development of causally sound explanation methods, and a relentless focus on evaluation that measures genuine human utility and learning outcomes. As AI becomes more ubiquitous, XAI’s role in building trustworthy, understandable, and beneficial systems will only grow in importance, guiding us towards an era of truly intelligent and accountable AI.
Share this content:
Post Comment