Irwin King

[intermediate/advanced] Deep Learning on Graphs

Summary

Graphs are versatile and ubiquitous data structures widely adopted in many applications including social networks, chemical molecules, traffic roads, knowledge graphs, etc. In this tutorial, we plan to introduce some interesting topics recently developed in graph representation learning. First we will present some fundamental knowledge about graph neural networks with a few popular GNN models. Then we will examine heterogeneous graph representation learning in more detail by putting forward metapath-based graph neural networks and their applications. Next, we plan to focus on hyperbolic graph representations by demonstrating the efficacy of hyperbolic neural network models on hierarchical data. Finally, we will explore a model about semi-supervised multi-label graph learning.

Syllabus

• Graph neural networks basics
• Introduction to graph neural networks
• Heterogeneous graphs
• Heterogeneous graph and metapath
• Metapath-based graph neural networks
• Hyperbolic graphs
• Hyperbolic representations on graphs
• Hyperbolic graph neural networks on temporal graphs
• Graph semi-supervised learning
• Classic semi-supervised learning
• Semi-supervised learning on graph multi-label learning

References

Basic Graph Neural Networks

• Semi-Supervised Classification with Graph Convolutional Networks. Thomas N. Kipf and Max Welling. ICLR 2017.
• Inductive Representation Learning on Large Graphs. William L. Hamilton, Zhitao Ying and Jure Leskovec. NIPS 2016.
• Graph Attention Networks. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò and Yoshua Bengio. ICLR 2017.
• How Powerful are Graph Neural Networks? Keyulu Xu, Weihua Hu, Jure Leskovec and Stefanie Jegelka. ICLR 2019.
• Simplifying Graph Convolutional Networks. Felix Wu, Amauri H. Souza Jr., Tianyi Zhang, Christopher Fifty, Tao Yu and Kilian Q. Weinberger. ICML 2019.

Heterogeneous Graph Representation Learning

• Metapath2vec: Scalable Representation Learning for Heterogeneous Networks. Yuxiao Dong, Nitesh V. Chawla and Ananthram Swami. KDD2017.
• Heterogeneous Graph Attention Network. Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui and Philip S. Yu. WWW 2019.
• MAGNN: Metapath Aggregated Graph Neural Network for HeterogeneousGraph Embedding. Xinyu Fu, Jiani Zhang, Ziqiao Meng and Irwin King. WWW 2020.

Hyperbolic Graph Representation Learning

• Poincaré Embeddings for Learning Hierarchical Representations. Maximilian Nickel and Douwe Kiela. NIPS 2017.
• Hyperbolic Graph Convolutional Neural Networks. Ines Chami, Zhitao Ying, Christopher R ́e and Jure Leskovec. NIPS 2019.
• Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space. Menglin Yang, Min Zhou, Marcus Kalander, Zengfeng Huang and Irwin King. KDD 2021.
• Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation. Yankai Chen, Menglin Yang, Yingxue Zhang, Mengchen Zhao, Ziqiao Meng, Jianye Hao, Irwin King. WSDM 2022.

Graph Multi-label Learning

• Min Shi, Yufei Tang, and Xingquan Zhu. 2020. MLNE: Multi-Label Network Embedding. IEEE Trans. Neural Networks Learn. Syst. 31, 9 (2020), 3682–3695.
• Semi-supervised Multi-label Learning for Graph-structured Data. Zixing Song, Ziqiao Meng, Yifei Zhang and Irwin King. CIKM 2021.

Pre-requisites

Machine learning and deep learning.

Short bio

Prof. Irwin King is the Chair and Professor of Computer Science & Engineering at The Chinese University of Hong Kong. His research interests include machine learning, social computing, AI, web intelligence, data mining, and multimedia information processing. In these research areas, he has over 350 technical publications in journals and conferences. He is an Associate Editor of the Journal of Neural Networks (NN). He is an IEEE Fellow, an ACM Distinguished Member, and a Fellow of Hong Kong Institute of Engineers (HKIE). He has served as the President of the International Neural Network Society (INNS), General Co-chair of The WebConf 2020, ICONIP 2020, WSDM 2011, RecSys 2013, ACML 2015, and in various capacities in a number of top conferences and societies such as WWW, NIPS, ICML, IJCAI, AAAI, INNS, APNNS, etc. He is the recipient of the ACM CIKM2019 Test of Time Award, the ACM SIGIR 2020 Test of Time Award, and 2020 APNNS Outstanding Achievement Award for his contributions made in social computing with machine learning. In early 2010 while on leave with AT&T Labs Research, San Francisco, he taught classes as a Visiting Professor at UC Berkeley. He received his B.Sc. degree in Engineering and Applied Science from California Institute of Technology (Caltech), Pasadena and his M.Sc. and Ph.D. degree in Computer Science from the University of Southern California (USC), Los Angeles.