Sergei V. Gleyzer

[introductory/intermediate] Machine Learning Fundamentals and Their Applications to Very Large Scientific Data: Rare Signal and Feature Extraction, End-to-End Deep Learning, Uncertainty Estimation and Realtime Machine Learning Applications in Software and Hardware

Summary

Deep learning has become one of the most widely used tools in modern science and engineering, leading to breakthroughs in many areas ranging from computer vision to natural language processing to physics and medicine. This mini-course will introduce the basics of machine learning and classification theory based on statistical learning and describe two classes of popular algorithms in depth: decision and rule-based methods (decision trees, decision rules, bagging and boosting, random forests) and deep neural network-based models of various types (fully-connected, convolutional, recurrent/LSTM, graph neural networks and transformers). The course will focus on practical applications in analysis of large scientific data, interpretability, uncertainty estimation and how to best extract meaningful features, while implementing realtime deep learning algorithms in software and hardware. No previous machine learning background is required.

Syllabus

• Introduction to Machine Learning: Theoretical Foundation, Classification Theory
• Practical Applications and Examples in Sciences and Engineering with Large Scientific Data
• Tree-based Algorithms: decision trees, rules, bagging, boosting, random forests
• Deep Learning Methods: theory, loss functions, fully-connected networks, convolutional, recurrent, graph neural networks and geometric deep learning, transformers
• Fundamentals of Feature Extraction and End-to-end Deep Learning
• Uncertainty Estimation and Machine Learning Model Interpretations
• Symbolic machine learning
• Realtime Implementation of Deep Learning in Software and Hardware

References

I. Goodfellow, Y. Bengio and A. Courville, “Deep Learning” MIT Press 2016

G. James et al., “Introduction to Statistical Learning” Springer 2013

A. Géron, “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems” O’Reilly 2019

C.M. Bishop, “Pattern Recognition and Machine Learning” Springer 2006

J.R. Quinlan, “C4.5: Programs for Machine Learning” Morgan Kaufmann 1992

Pre-requisites

None.

Short bio

Sergei Gleyzer is a particle physicist working at the interface of machine learning and physics and astronomy towards more intelligent systems that can extract meaningful information from the data collected by the Large Hadron Collider (LHC), the world’s highest-energy particle physics experiment located at the CERN laboratory, near Geneva, Switzerland and Vera Rubin Observatory in Chile. He is a co-discoverer of the Higgs Boson and founder of several major machine learning initiatives such as the Inter-experimental Machine Learning Working Group and Compact Muon Solenoid experiment’s Machine Learning Forum. Professor Gleyzer is working on applying advanced machine learning methods to searches for new physics, such as dark matter. He also teaches a popular machine learning course at the University of Alabama and is the founder of the Machine Learning for Science (ML4SCI) Open Source Foundation.