Sean Benson

[intermediate] Deep Learning for a Better Understanding of Cancer

Summary

My lectures will provide a brief introduction to computer vision and convolutional neural networks, before moving on to the challenges faced when training using medical imaging, namely heterogeneous datasets, the challenging dataset sizes and the relatively high expense of labels. I will move on to how semi-supervised and unsupervised methods are being used in combination with transfer learning in order to make the best use of available data and explain modern techniques for the combination of different data sources, with a focus on relevant clinical endpoints such as segmentation, tumour characterisation, and followup predictions.

Syllabus

• Convolutional neural networks (CNNs)
• Transfer learning techniques
• Supervised and unsupervised loss functions
• Application of CNNs to medical image segmentation
• Tumour characterisation and classification
• Data source combinations
• Outcome and recurrence predictions

References

[1] R. A. Weinberg, The Biology of Cancer, 0 dr. W.W. Norton & Company, 2013 [Online]. Beschikbaar op: https://www.taylorfrancis.com/books/9781317963462. [Geraadpleegd: 17 oktober 2022]

[2] F. Cardoso e.a., ‘Early breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up’, Annals of Oncology, vol. 30, nr. 8, pp. 1194–1220, aug. 2019, doi: 10.1093/annonc/mdz173. [Online]. Beschikbaar op: https://linkinghub.elsevier.com/retrieve/pii/S0923753419312876. [Geraadpleegd: 17 oktober 2022]

[3] P. Herent e.a., ‘Detection and characterization of MRI breast lesions using deep learning’, Diagnostic and Interventional Imaging, vol. 100, nr. 4, pp. 219–225, apr. 2019, doi: 10.1016/j.diii.2019.02.008. [Online]. Beschikbaar op: https://linkinghub.elsevier.com/retrieve/pii/S2211568419300567. [Geraadpleegd: 17 oktober 2022]

[4] S. Woo, J. Park, J.-Y. Lee, en I. S. Kweon, ‘CBAM: Convolutional Block Attention Module’. arXiv, 18 juli 2018 [Online]. Beschikbaar op: http://arxiv.org/abs/1807.06521. [Geraadpleegd: 17 oktober 2022]

[5] L. Itti, C. Koch, en E. Niebur, ‘A model of saliency-based visual attention for rapid scene analysis’, IEEE Trans. Pattern Anal. Machine Intell., vol. 20, nr. 11, pp. 1254–1259, nov. 1998, doi: 10.1109/34.730558. [Online]. Beschikbaar op: http://ieeexplore.ieee.org/document/730558/. [Geraadpleegd: 17 oktober 2022]

[6] W. Jin, X. Li, en G. Hamarneh, ‘One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images’. arXiv, 11 juli 2021 [Online]. Beschikbaar op: http://arxiv.org/abs/2107.05047. [Geraadpleegd: 17 oktober 2022]

[7] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, en D. Batra, ‘Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization’, Int J Comput Vis, vol. 128, nr. 2, pp. 336–359, feb. 2020, doi: 10.1007/s11263-019-01228-7. [Online]. Beschikbaar op: http://arxiv.org/abs/1610.02391. [Geraadpleegd: 17 oktober 2022]

[8] G. Habib e.a., ‘Automatic Breast Lesion Classification by Joint Neural Analysis of Mammography and Ultrasound’. arXiv, 23 september 2020 [Online]. Beschikbaar op: http://arxiv.org/abs/2009.11009. [Geraadpleegd: 17 oktober 2022]

[9] M. T. Ribeiro, S. Singh, en C. Guestrin, ‘“Why Should I Trust You?”: Explaining the Predictions of Any Classifier’. arXiv, 9 augustus 2016 [Online]. Beschikbaar op: http://arxiv.org/abs/1602.04938. [Geraadpleegd: 17 oktober 2022]

[10] S. Lundberg en S.-I. Lee, ‘A Unified Approach to Interpreting Model Predictions’. arXiv, 24 november 2017 [Online]. Beschikbaar op: http://arxiv.org/abs/1705.07874. [Geraadpleegd: 17 oktober 2022]

[11] F. Milletari, N. Navab, en S.-A. Ahmadi, ‘V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation’. arXiv, 15 juni 2016 [Online]. Beschikbaar op: http://arxiv.org/abs/1606.04797. [Geraadpleegd: 17 oktober 2022]

[12] M. Descoteaux, L. Maier-Hein, A. Franz, P. Jannin, D. L. Collins, en S. Duchesne, Red., Medical Image Computing and Computer Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part I, vol. 10433. Cham: Springer International Publishing, 2017 [Online]. Beschikbaar op: https://link.springer.com/10.1007/978-3-319-66182-7. [Geraadpleegd: 17 oktober 2022]

[13] O. Ronneberger, P. Fischer, en T. Brox, ‘U-Net: Convolutional Networks for Biomedical Image Segmentation’. arXiv, 18 mei 2015 [Online]. Beschikbaar op: http://arxiv.org/abs/1505.04597. [Geraadpleegd: 17 oktober 2022]

[14] F. Isensee e.a., ‘nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation’. arXiv, 27 september 2018 [Online]. Beschikbaar op: http://arxiv.org/abs/1809.10486. [Geraadpleegd: 17 oktober 2022]

[15] D. Veiga-Canuto e.a., ‘Comparative Multicentric Evaluation of Inter-Observer Variability in Manual and Automatic Segmentation of Neuroblastic Tumors in Magnetic Resonance Images’, Cancers, vol. 14, nr. 15, p. 3648, jul. 2022, doi: 10.3390/cancers14153648. [Online]. Beschikbaar op: https://www.mdpi.com/2072-6694/14/15/3648. [Geraadpleegd: 17 oktober 2022]

[16] K. V. Sarma e.a., ‘Harnessing clinical annotations to improve deep learning performance in prostate segmentation’, PLoS ONE, vol. 16, nr. 6, p. e0253829, jun. 2021, doi: 10.1371/journal.pone.0253829. [Online]. Beschikbaar op: https://dx.plos.org/10.1371/journal.pone.0253829. [Geraadpleegd: 17 oktober 2022]

[17] X. Luo, J. Chen, T. Song, en G. Wang, ‘Semi-supervised Medical Image Segmentation through Dual-task Consistency’, AAAI, vol. 35, nr. 10, pp. 8801–8809, mei 2021, doi: 10.1609/aaai.v35i10.17066. [Online]. Beschikbaar op: https://ojs.aaai.org/index.php/AAAI/article/view/17066. [Geraadpleegd: 17 oktober 2022]

[18] S. Frank, ‘Tissue Segmentation from Whole-Slide Images Using Lightweight Neural Networks’, In Review, preprint, dec. 2020 [Online]. Beschikbaar op: https://www.researchsquare.com/article/rs-122564/v1. [Geraadpleegd: 17 oktober 2022]

[19] W.-Y. Chuang e.a., ‘Successful Identification of Nasopharyngeal Carcinoma in Nasopharyngeal Biopsies Using Deep Learning’, Cancers, vol. 12, nr. 2, p. 507, feb. 2020, doi: 10.3390/cancers12020507. [Online]. Beschikbaar op: https://www.mdpi.com/2072-6694/12/2/507. [Geraadpleegd: 17 oktober 2022]

[20] B. Ehteshami Bejnordi e.a., ‘Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer’, JAMA, vol. 318, nr. 22, p. 2199, dec. 2017, doi: 10.1001/jama.2017.14585. [Online]. Beschikbaar op: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2017.14585. [Geraadpleegd: 17 oktober 2022]

[21] K. Ravichandran, N. Braman, A. Janowczyk, en A. Madabhushi, ‘A deep learning classifier for prediction of pathological complete response to neoadjuvant chemotherapy from baseline breast DCE-MRI’, in Medical Imaging 2018: Computer-Aided Diagnosis, Houston, United States, feb. 2018, p. 11, doi: 10.1117/12.2294056 [Online]. Beschikbaar op: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10575/2294056/A-deep-learning-classifier-for-prediction-of-pathological-complete-response/10.1117/12.2294056.full. [Geraadpleegd: 17 oktober 2022]

[22] Y. Zhang e.a., ‘Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centers’, Eur Radiol, vol. 31, nr. 4, pp. 2559–2567, apr. 2021, doi: 10.1007/s00330-020-07274-x. [Online]. Beschikbaar op: https://link.springer.com/10.1007/s00330-020-07274-x. [Geraadpleegd: 17 oktober 2022]

[23] Q. Sun e.a., ‘Deep Learning vs. Radiomics for Predicting Axillary Lymph Node Metastasis of Breast Cancer Using Ultrasound Images: Don’t Forget the Peritumoral Region’, Front. Oncol., vol. 10, p. 53, jan. 2020, doi: 10.3389/fonc.2020.00053. [Online]. Beschikbaar op: https://www.frontiersin.org/article/10.3389/fonc.2020.00053/full. [Geraadpleegd: 17 oktober 2022]

[24] S. Gupta, N. S. Punn, S. K. Sonbhadra, en S. Agarwal, ‘MAG-Net: Multi-task attention guided network for brain tumor segmentation and classification’, vol. 13147, 2021, pp. 3–15 [Online]. Beschikbaar op: http://arxiv.org/abs/2107.12321. [Geraadpleegd: 17 oktober 2022]

[25] R. M. Jimenez-Rodriguez e.a., ‘Organ Preservation in Patients with Rectal Cancer Treated with Total Neoadjuvant Therapy’, Diseases of the Colon & Rectum, vol. 64, nr. 12, pp. 1463–1470, dec. 2021, doi: 10.1097/DCR.0000000000002122. [Online]. Beschikbaar op: https://journals.lww.com/10.1097/DCR.0000000000002122. [Geraadpleegd: 17 oktober 2022]

[26] X. He, J. Ji, M. Tian, F. J. Esteva, G. N. Hortobagyi, en S.-C. J. Yeung, ‘Long-Term Survival Analysis of Adjuvant Chemotherapy with or without Trastuzumab in Patients with T1, Node-Negative HER2-Positive Breast Cancer’, Clinical Cancer Research, vol. 25, nr. 24, pp. 7388–7395, dec. 2019, doi: 10.1158/1078-0432.CCR-19-0463. [Online]. Beschikbaar op: https://aacrjournals.org/clincancerres/article/25/24/7388/82332/Long-Term-Survival-Analysis-of-Adjuvant. [Geraadpleegd: 17 oktober 2022]

[27] J. L. Katzman, U. Shaham, A. Cloninger, J. Bates, T. Jiang, en Y. Kluger, ‘DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network’, BMC Med Res Methodol, vol. 18, nr. 1, p. 24, dec. 2018, doi: 10.1186/s12874-018-0482-1. [Online]. Beschikbaar op: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-018-0482-1. [Geraadpleegd: 17 oktober 2022]

[28] S.-C. Huang, A. Pareek, S. Seyyedi, I. Banerjee, en M. P. Lungren, ‘Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines’, npj Digit. Med., vol. 3, nr. 1, p. 136, dec. 2020, doi: 10.1038/s41746-020-00341-z. [Online]. Beschikbaar op: https://www.nature.com/articles/s41746-020-00341-z. [Geraadpleegd: 17 oktober 2022]

[29] X. Wang e.a., ‘Combining Radiology and Pathology for Automatic Glioma Classification’, Front. Bioeng. Biotechnol., vol. 10, p. 841958, mrt. 2022, doi: 10.3389/fbioe.2022.841958. [Online]. Beschikbaar op: https://www.frontiersin.org/articles/10.3389/fbioe.2022.841958/full. [Geraadpleegd: 17 oktober 2022]

[30] R. S. Vanguri e.a., ‘Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer’, Nat Cancer, aug. 2022, doi: 10.1038/s43018-022-00416-8. [Online]. Beschikbaar op: https://www.nature.com/articles/s43018-022-00416-8. [Geraadpleegd: 17 oktober 2022]

Pre-requisites

Linear algebra, probability, statistics, experience with medical data formats desirable.

Short bio

I am a research scientist in the Radiology Department of the Netherlands Cancer Institute where I lead a team of doctoral researchers developing deep learning models using multi-modal medical imaging to predict treatment response and identify features that can be used to predict regrowth at the earliest possible opportunity. I received my PhD from the University of Edinburgh working on the LHCb experiment of CERN, where my research focused on statistical analyses of LHC data including the development of AI models to isolate interesting decay signatures. After CERN and Marie-Curie fellowships where a large part of my research centred on the development of real-time analysis infrastructure and the design of fast models for real-time evaluation, I worked as a Senior Data Scientist at KPMG where I developed and deployed explainable computer vision and natural language processing models before joining the Radiology Department of the Netherlands Cancer Institute.