DeepLearn 2023 Winter
8th International School
on Deep Learning
Bournemouth, UK · January 16-20, 2023
Registration
Downloads
  • Call DeepLearn 2023 Winter
  • Poster DeepLearn 2023 Winter
  • Lecture Materials
  • Home
  • Schedule
  • Lecturers
  • News
  • Accommodation
  • Info
    • Travel from London to Bournemouth
    • Sponsoring
    • Code of conduct
    • Visa
    • Testimonials
  • Home
  • Schedule
  • Lecturers
  • News
  • Accommodation
  • Info
    • Travel from London to Bournemouth
    • Sponsoring
    • Code of conduct
    • Visa
    • Testimonials
Zhiting Hu & Eric P. Xing

Zhiting Hu & Eric P. Xing

University of California, San Diego & Carnegie Mellon University

A “Standard Model” for Machine Learning with All Experiences [virtual]

Summary

In handling a wide variety of experience ranging from data instances, knowledge, constraints, to rewards, adversaries, and lifelong interaction in an ever-growing spectrum of tasks, contemporary Machine Learning (ML) and Artificial Intelligence (AI) research has resulted in a multitude of learning paradigms, optimization methods, and model architectures. Despite the continual progresses on all different fronts, the disparate narrowly focused methods also make standardized, composable, and reusable development of ML approaches difficult, and preclude the opportunity to build AI agents that panoramically learn from all types of experience, on diverse computing devices. The first part of the course covers a standardized ML formalism, in particular a ‘standard equation’ of the learning objective, that offers a unifying understanding of many important ML algorithms in the supervised, unsupervised, knowledge-constrained, reinforcement, adversarial, and online learning paradigms, respectively—those diverse algorithms are encompassed as special cases due to different choices of modeling components. The framework also provides guidance for mechanical design of new ML approaches and serves as a promising vehicle toward panoramic machine learning with all experience. The second part will focus on the system and scalability aspects of today’s ML. We’ll identify research and practical pain points in large model training and serving, and introduce new algorithmic techniques and system architectures for addressing the training and serving of popular big models, such as GPT-3, PaLM, and vision transformers.Through this part, we hope to lower the technical barrier of using big models in ML research and bring the big models to the masses.

Syllabus

  • Lecture 1: A ‘Standard Model’ of Machine Learning (I)
  • Lecture 2: A ‘Standard Model’ of Machine Learning (II)
  • Lecture 3: Techniques and Systems to Train and Serve Big Models

References

[1] Hu & Xing. “Toward a ‘Standard Model’ of Machine Learning”. Harvard Data Science Review. 2022. https://hdsr.mitpress.mit.edu/pub/zkib7xth/release/2

[2] Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning. OSDI’2022.

Pre-requisites

Introductory course on machine learning.

Short bios

Dr. Zhiting Hu is an Assistant Professor in Halicioglu Data Science Institute at UC San Diego. He received his Bachelor’s degree in Computer Science from Peking University in 2014, and his Ph.D. in Machine Learning from Carnegie Mellon University in 2020. His research interests lie in the broad area of machine learning, artificial intelligence, natural language processing, and ML systems. In particular, he is interested in principles, methodologies, and systems of training AI agents with all types of experience (data, symbolic knowledge, rewards, adversaries, lifelong interplay, etc), and their applications in controllable text generation, healthcare, and other application domains. His research was recognized with the best demo nomination at ACL2019 and an outstanding paper award at ACL2016.

Dr. Hao Zhang is a postdoc researcher at UC Berkeley. He completed his Ph.D. at Carnegie Mellon University. His research interests are in the intersection of machine learning and systems, with the focus on improving the performance and ease-of-use of today’s distributed ML systems. Hao’s research has been recognized with an NVIDIA pioneer research award at NeurIPS’17, and the Jay Lepreau best paper award at OSDI’21. Hao’s open-source artifacts have been used by organizations such as AI2, Meta, and Google, and parts of Hao’s research have been commercialized at multiple start-ups including Petuum and AnyScale.

Dr. Eric P. Xing is a Professor of Computer Science at Carnegie Mellon University, and the Founder, CEO, and Chief Scientist of Petuum Inc., a 2018 World Economic Forum Technology Pioneer company that builds standardized artificial intelligence development platform and operating system for broad and general industrial AI applications. He completed his undergraduate study at Tsinghua University, and holds a PhD in Molecular Biology and Biochemistry from the Rutgers University, and a PhD in Computer Science from UC Berkeley. His main research interests are the development of machine learning and statistical methodology, and large-scale computational systems and architectures, for solving problems involving automated learning, reasoning, and decision-making in high-dimensional, multimodal, and dynamic possible worlds in artificial, biological, and social systems. Prof. Xing currently serves or has served the following roles: associate editor of the Journal of the American Statistical Association (JASA), Annals of Applied Statistics (AOAS), IEEE Journal of Pattern Analysis and Machine Intelligence (PAMI) and the PLoS Journal of Computational Biology; action editor of the Machine Learning Journal (MLJ) and Journal of Machine Learning Research (JMLR); member of the United States Department of Defense Advanced Research Projects Agency (DARPA) Information Science and Technology (ISAT) advisory group. He is a recipient of the National Science Foundation (NSF) Career Award, the Alfred P. Sloan Research Fellowship, the United States Air Force Office of Scientific Research Young Investigator Award, the IBM Open Collaborative Research Faculty Award, as well as several best paper awards. Prof. Xing is a board member of the International Machine Learning Society; he has served as the Program Chair (2014) and General Chair (2019) of the International Conference of Machine Learning (ICML); he is also the Associate Department Head of the Machine Learning Department, founding director of the Center for Machine Learning and Health at Carnegie Mellon University; he is a Fellow of AAAI, IEEE, and ASA.

Other Courses

deeplearn-speaker-yi-maYi Ma
Daphna WeinshallDaphna Weinshall
Eric P. XingEric P. Xing
Matias Carrasco KindMatias Carrasco Kind
deeplearn-speaker-nitesh-chawlaNitesh Chawla
Sumit ChopraSumit Chopra
Luc De RaedtLuc De Raedt
Marco DuarteMarco Duarte
Joao GamaJoão Gama
Claus HornClaus Horn
deeplearn-speaker-nathalie-japkowiczNathalie Japkowicz
deeplearn-speaker-gregor-kasieczkaGregor Kasieczka
Karen LivescuKaren Livescu
deeplearn-speaker-david-mcallersterDavid McAllester
deeplearn-speaker-dhabaleswar-k-pandaDhabaleswar K. Panda
Fabio RoliFabio Roli
Bracha ShapiraBracha Shapira
deeplearn-speaker-kunal-tawarKunal Talwar
Tinne TuytelaarsTinne Tuytelaars
deeplearn-speaker-lyle-ungarLyle Ungar
speakers-bram-van-ginnekenBram van Ginneken
deeplearn-speaker-yudong-zhangYu-Dong Zhang

DeepLearn 2022 Winter

CO-ORGANIZERS

Bournemouth University
Department of Computing and Informatics

Universitat Rovira i Virgili, Tarragona

Institute for Research Development, Training and Advice – IRDTA, Brussels/London

Active links
  • DeepLearn 2023 Summer – 10th International Gran Canaria School on Deep Learning
  • BigDat 2023 Summer – 7th International School on Big Data
  • DeepLearn 2023 Spring – 9th International School on Deep Learning
Past links
  • DeepLearn 2022 Autumn
  • DeepLearn 2022 Summer
  • DeepLearn 2022 Spring
  • DeepLearn 2021 Summer
  • DeepLearn 2019
  • DeepLearn 2018
  • DeepLearn 2017
© IRDTA 2022. All Rights Reserved.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Cookie SettingsAccept All
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-advertisement1 yearThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Advertisement".
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
PHPSESSIDsessionThis cookie is native to PHP applications. The cookie is used to store and identify a users' unique session ID for the purpose of managing user session on the website. The cookie is a session cookies and is deleted when all the browser windows are closed.
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
CookieDurationDescription
_ga2 yearsThis cookie is installed by Google Analytics. The cookie is used to calculate visitor, session, campaign data and keep track of site usage for the site's analytics report. The cookies store information anonymously and assign a randomly generated number to identify unique visitors.
_gat_gtag_UA_74880351_91 minuteThis cookie is set by Google and is used to distinguish users.
_gid1 dayThis cookie is installed by Google Analytics. The cookie is used to store information of how visitors use a website and helps in creating an analytics report of how the website is doing. The data collected including the number visitors, the source where they have come from, and the pages visted in an anonymous form.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT
Powered by CookieYes Logo