Wojciech Samek

[introductory/intermediate] Explainable AI: Concepts, Methods and Applications

Summary

Being able to explain the predictions of machine learning models is important in critical applications such as medical diagnosis or autonomous systems. The rise of deep nonlinear ML models has led to massive gains in terms of predictivity. Yet, we do not want such high accuracy to come at the expense of explainability. As a result, the field of Explainable AI (XAI) has emerged and has produced a collection of methods that are capable of explaining complex and diverse ML models. This tutorial will give a structured overview of the approaches that have been proposed for XAI. In particular, it will present motivations for such methods, their advantages/disadvantages and their theoretical underpinnings. It will also show how these techniques can be extended and applied in a way that they deliver maximum usefulness in real-world scenarios.

Syllabus

The first part of the tutorial will present motivations for XAI, in particular it will give examples where simple non-XAI validation techniques can strongly mislead the user in his assessment of the model performance. The second part will present several XAI methods that successfully cope with highly nonlinear ML models used in practice and discuss their theoretical underpinnings. The third part will present recent developments in XAI.

The topics covered are:

• Motivations: Black-box models and the “Clever Hans” effect
• Explainable AI: methods for explaining deep neural networks
• Unifying views on explanation methods & theoretical underpinnings
• Evaluating explanations
• Applications of XAI
• Explaining beyond deep networks, single-feature attributions, and individual predictions
• XAI-Based model improvement

References

http://www.heatmapping.org

W Samek, G Montavon, S Lapuschkin, C Anders, KR Müller. Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications, Proceedings of the IEEE, 109(3):247-278, 2021. https://doi.org/10.1109/JPROC.2021.3060483

G Montavon, W Samek, KR Müller. Methods for Interpreting and Understanding Deep Neural Networks, Digital Signal Processing, 73:1-15, 2018. https://doi.org/10.1016/j.dsp.2017.10.011

G Montavon, S Lapuschkin, A Binder, W Samek, KR Müller. Explaining NonLinear Classification Decisions with Deep Taylor Decomposition, Pattern Recognition, 65:211–222, 2017. http://dx.doi.org/10.1016/j.patcog.2016.11.008

L Arras, A Osman, W Samek. CLEVR-XAI: A Benchmark Dataset for the Ground Truth Evaluation of Neural Network Explanations, Information Fusion, 2022. https://doi.org/10.1016/j.inffus.2021.11.008

J Sun, S Lapuschkin, W Samek, A Binder. Explain and Improve: LRP-Inference Fine Tuning for Image Captioning Models, Information Fusion, 77:233-246, 2022. https://doi.org/10.1016/j.inffus.2021.07.008

CJ Anders, L Weber, D Neumann, W Samek, KR Müller, S Lapuschkin. Finding and Removing Clever Hans: Using Explanation Methods to Debug and Improve Deep Models, Information Fusion, 77:261-295, 2021. https://doi.org/10.1016/j.inffus.2021.07.015

W Samek, G Montavon, A Vedaldi, LK Hansen, KR Müller (Eds.) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer LNCS, 11700, 2019. https://link.springer.com/book/10.1007/978-3-030-28954-6

C Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1, 206–215 (2019). https://doi.org/10.1038/s42256-019-0048-x

S M Lundberg, G Erion, H Chen et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2, 56–67 (2020). https://doi.org/10.1038/s42256-019-0138-9

F Doshi-Velez, B Kim. Towards A Rigorous Science of Interpretable Machine Learning. arXiv:1702.08608. https://arxiv.org/abs/1702.08608

P Schramowski, W Stammer, S Teso, S. et al. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nat Mach Intell 2, 476–486 (2020). https://doi.org/10.1038/s42256-020-0212-3

Pre-requisites

Basic understanding of machine learning and deep learning.

Short bio

Wojciech Samek is head of the Department of Artificial Intelligence and the Explainable AI Group at Fraunhofer Heinrich Hertz Institute (HHI), Berlin, Germany. He studied computer science at Humboldt University of Berlin, Heriot-Watt University and University of Edinburgh and received the Dr. rer. nat. degree with distinction from the Technical University of Berlin in 2014. During his studies he was awarded scholarships from the German Academic Scholarship Foundation and the DFG Research Training Group GRK 1589/1, and was a visiting researcher at NASA Ames Research Center, Mountain View, USA. Wojciech is associated faculty at the BIFOLD – Berlin Institute for the Foundation of Learning and Data, the ELLIS Unit Berlin and the DFG Graduate School BIOQIC, and member of the scientific advisory board of IDEAS NCBR. Furthermore, he is an editorial board member of PLoS ONE, Pattern Recognition and IEEE TNNLS and an elected member of the IEEE MLSP Technical Committee. He is recipient of multiple best paper awards, including the 2020 Pattern Recognition Best Paper Award, and part of the expert group developing the ISO/IEC MPEG-17 NNR standard. He is the leading editor of the Springer book “Explainable AI: Interpreting, Explaining and Visualizing Deep Learning” and organizer of various special sessions, workshops and tutorials on topics such as explainable AI, neural network compression, and federated learning. Wojciech has co-authored more than 150 peer-reviewed journal and conference papers; some of them listed as “Highly Cited Papers” (i.e., top 1%) in the field of Engineering.