Jayashree Kalpathy-Cramer
[introductory/intermediate] Multimodal AI for Healthcare
Summary
Machine learning, specifically deep learning, has the potential to greatly transform healthcare. We will begin with highlighting a few applications in radiology, oncology and ophthalmology. Although these techniques have shown remarkable performance for many tasks including medical image analysis, we will share some of the challenges that we have faced in developing robust and trustworthy algorithms including a lack of repeatability, explainability, generalizability, and the potential for bias. After reviewing these applications, challenges and mitigating strategies, we will discuss recent advances in multimodal approaches in these clincial areas.
Syllabus
- Brief introduction to medical imaging
- Application of machine learning in medical imaging including in radiology, oncology, and ophthalmology
- Introduction to MONAI framework
- Multimodal machine learning with imaging and medical records
- Challenges faced in real life deployment of machine learning algorithms including robustness, repeatability, privacy, fairness and bias
References
https://github.com/Project-MONAI/MONAI
Ahmed SR, Egemen D, Befano B, Rodriguez AC, Jeronimo J, Desai K, Teran C, Alfaro K, Fokom-Domgue J, Charoenkwan K, Mungo C, Luckett R, Saidu R, Raiol T, Ribeiro A, Gage JC, de Sanjose S, Kalpathy-Cramer J, Schiffman M. Assessing generalizability of an AI-based visual test for cervical cancer screening. PLOS Digit Health. 2024 Oct 2;3(10):e0000364. doi: 10.1371/journal.pdig.0000364. PMID: 39356713; PMCID: PMC11446437.
Pati S, Kumar S, Varma A, Edwards B, Lu C, Qu L, Wang JJ, Lakshminarayanan A, Wang SH, Sheller MJ, Chang K, Singh P, Rubin DL, Kalpathy-Cramer J, Bakas S. Privacy preservation for federated learning in health care. Patterns (N Y). 2024 Jul 12;5(7):100974. doi: 10.1016/j.patter.2024.100974. PMID: 39081567; PMCID: PMC11284498.
Coyner AS, Murickan T, Oh MA, Young BK, Ostmo SR, Singh P, Chan RVP, Moshfeghi DM, Shah PK, Venkatapathy N, Chiang MF, Kalpathy-Cramer J, Campbell JP. Multinational External Validation of Autonomous Retinopathy of Prematurity Screening. JAMA Ophthalmol. 2024 Apr 1;142(4):327-335. doi: 10.1001/jamaophthalmol.2024.0045. PMID: 38451496; PMCID: PMC10921347.
Tan TF, Thirunavukarasu AJ, Campbell JP, Keane PA, Pasquale LR, Abramoff MD, Kalpathy-Cramer J, Lum F, Kim JE, Baxter SL, Ting DSW. Generative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology: Clinical Applications and Challenges. Ophthalmol Sci. 2023 Sep 9;3(4):100394. doi: 10.1016/j.xops.2023.100394. PMID: 37885755; PMCID: PMC10598525.
Pre-requisites
Machine learning basics, Python for the hands-on portion.
Short bio
Jayashree Kalpathy-Cramer is the endowed chair in Ophthalmic Data Sciences and the founding chief of the Division of Artificial Medical Intelligence in the Department of Ophthalmology at the University of Colorado (CU) School of Medicine. She is also the Director for Health Informatics at the Colorado Clinical and Translational Sciences Institute. She leads the development and translation of novel artificial intelligence (AI) methods into effective patient care practices at the Sue Anschutz-Rodgers Eye Center. Her research interests span the spectrum from novel algorithm development to clinical deployment. She is passionate about the potential that machine learning and artificial intelligence have to improve the access and the quality of healthcare in the US and worldwide. Dr. Kalpathy-Cramer spent almost a decade in the semiconductor industry before a pivot to academia and healthcare. She has authored over 250 peer-reviewed publications, has written over a dozen book chapters and is a co-inventor on 15 patents.
