Savannah Thais

[intermediate] Applications of Graph Neural Networks: Physical and Societal Systems

Summary

Graph Neural Networks (GNNs) have gained significant attention in recent years due to their ability to model and learn from a broad range of geometrically or relationally structured data and their wide range of applications to a range of complex systems. This course will provide a comprehensive overview of GNNs, beginning from the foundations of graph convolutional and message passing networks and exploring more advanced or novel GNNs including graph transformers, equivariant GNNs, generative graph models, heterogenous and dynamic graphs, and more. We will then explore various applications of GNNs in real-world scenarios, including social network analysis, traffic prediction, drug discovery, urban planning, protein folding, particle and astrophysics data processing, and many others. We will discuss the benefits and limitations of graphs as data representation and GNNs as an architecture class including expressivity, scalability, and interpretability/explainability.

Syllabus

• Graphs as a data structure, why they are useful.
• Introduction to graph neural networks: graph convolutions and message passing, inherent permutation equivariance.
• Advanced concepts in GNNs: transformers, equivariance, heterogenous graphs, dynamic graphs, generative models.
• Exploration of science applications: particle and astrophysics, molecular modeling, materials science, physical dynamics and simulation.
• Exploration of social systems applications.
• Discussion of incorporating existing knowledge and inductive bias into GNN models.
• Benefits and limitations of graphs and GNNS.

References

Representation Learning on Graphs: Methods and Applications. William Hamilton, Rex Ying, Jure Leskovec (2017).

Semi-Supervised Classification with Graph Convolutional Networks. Thomas N. Kipf, Max Welling.

Everything is Connected: Graph Neural Networks. Petar Veličković.

Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges. Savannah Thais, Paolo Calafiura, Grigorios Chachamis, Gage DeZoort, Javier Duarte, Sanmay Ganguly, Michael Kagan, Daniel Murnane, Mark S. Neubauer, Kazuhiro Terao.

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković.

Graph Neural Networks with Learnable Structural and Positional Representations. Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson..

Graph neural networks for materials science and chemistry. Patrick Reiser, Marlen Neubert, André Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer & Pascal Friederich.

Interaction networks for learning about objects, relations and physics. Peter Battaglia et al. Advances in Neural Information Processing Systems 29 (2016).

On the equivalence between graph isomorphism testing and function approximation with GNNs. Zhengdao Chen, Soledad Villar, Lei Chen, Joan Bruna.

Graph neural networks: A review of methods and applications. Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun.

Non-Existence of Stable Social Groups in Information-Driven Networks. Augustin Chaintreau, Guillaume Ducoffe, Dorian Mazauric.

Unequal Opportunities in Multi-hop Referral Programs. Yiguang Zhang, Augustin Chaintreau.

Pre-requisites

Participants should have prior knowledge of machine learning, deep learning, and linear algebra. Interest in complex systems would also be useful but is not required.

Short bio

Savannah Thais is a Research Scientist in the Columbia University Data Science Institute with a focus on machine learning (ML). She is interested in complex system modeling and in understanding what types of information are measurable or modelable and what impacts designing and performing measurements have on systems and societies. Her recent work has focused on geometric deep learning, methods to incorporate physics-based inductive biases into ML models, regulation of emerging technology, social determinants of health, and community education. Dr. Thais is the founder and Research Director of Community Insight and Impact, a non-profit organization focused on data-driven community needs assessments for vulnerable populations and effective resource allocation. She was the ML Knowledge Convener for the CMS Experiment from 2020-2022, currently serves on the Executive Board of Women in Machine Learning and the Executive Committee of the APS Group on Data Science, and is a Founding Editor of the Springer AI Ethics journal.