Awni Hannun
[intermediate] An Introduction to Speech Recognition and Weighted Finite-State Automata [virtual]
Summary
In the first part of this course I will introduce a modern end-to-end automatic speech recognition system. We will cover the Connectionist Temporal Classification (CTC) loss function in detail as well as beam search decoding. In the second part of the course I will introduce weighted finite-state automata and their application to machine learning with a particular focus on speech recognition. I will motivate the use of automata in machine learning and proceed with an introduction to acceptors, transducers, and their associated properties. We will then go into detail on many of the core operations of weighted automata. Following this, we will move closer to the research frontier by explaining automatic differentiation and its use with weighted automata. We will end with some extended examples to gain deeper familiarity with weighted automata, their operations, and their use in machine learning.
Syllabus
Modern Automatic Speech Recognition
- Introduction
- Deep Dive: Connectionist Temporal Classification
- Deep Dive: Decoding with Beam Search
Weighted Finite-State Transducers
- Acceptors and Transducers
- Basic Operations
- Advanced Operations
- Differentiation with Automata
- Extended Examples
References
Sequence Modeling with CTC, Awni Hannun, 2017, https://distill.pub/2017/ctc/
An Introduction to Weighted Automata in Machine Learning, Awni Hannun, 2021, https://www.awnihannun.com/writing/automata_ml.html
Pre-requisites
Experience in speech recognition or natural language processing.
Short bio
Awni Hannun is a research scientist and founding member of the Zoom AI research lab. His research focuses on low-resource machine learning, sequential and structured data, and applications to speech and audio understanding. He also researches practical, privacy preserving machine learning. Prior to Zoom he was a research scientist at Facebook AI Research and Baidu Research. He completed his Ph.D. in computer science from Stanford University.
