Timothy Hospedales
[intermediate/advanced] Deep Meta-Learning
Summary
In the conventional supervised deep learning paradigm, one provides a labelled dataset and learning algorithm and trains a predictive model which should generalise to new data. This paradigm can work very well, but often performs poorly in less than ideal scenarios, most famously when the training data is insufficient. This seminar introduces deep meta-learning (aka learning-to-learn), where the learning algorithm itself is learned by a higher level meta-learner, rather than being completely user-specified. In this seminar, we will introduce the key concepts and algorithms in meta-learning. We will then explore the ways in which meta-learning techniques can be used, from improving data-efficient learning, to a range of other practical challenges such as fast optimisation, reinforcement learning, learning label-noise, train-test domain-shift, adversarial defense, self-supervision, semi-supervised learning, standard supervised learning, and more.
Syllabus
- Introduction to meta-learning
- Key concepts and algorithms
- Few-shot learning
- Other applications
- Outlook
References
Hospedales et al, Meta-learning in neural networks, PAMI 2021. https://ieeexplore.ieee.org/abstract/document/9428530
Hospedales, Meta-learning in neural networks, Blog. https://research.samsung.com/blog/Meta-Learning-in-Neural-Networks
Finn et al, Model-agnostic meta-learning for fast adaptation of deep networks, ICML 2017, https://arxiv.org/abs/1703.03400
Cubuk et al, AutoAugment: Learning Augmentation Policies from Data, https://arxiv.org/abs/1805.09501
Pre-requisites
Machine learning basics (regularisation, overfitting, underfitting, etc). Supervised deep learning.
Short bio
Timothy Hospedales is a full professor of Artificial Intelligence at the University of Edinburgh and Programme Director for Machine Learning and Data Intelligence at Samsung AI Centre Cambridge. He has worked extensively on methods for meta-learning and learning with limited data, with applications in computer vision, reinforcement learning, and beyond. He has published numerous papers in these areas in top venues such as CVPR, NeurIPS, ICML, ICLR, AAAI and PAMI, several of which have won best paper prizes and nominations. He is an Associate Editor of IEEE Transaction on Pattern Analysis and Machine Intelligence.
