Altan Çakır

[introductory] Introduction to Deep Learning with Apache Spark

Summary

Apache Spark, open-source cluster-computing framework providing a fast and general engine for large-scale processing, has been one of the exciting technologies in recent years for the big data development. The main idea behind this technology is to provide a memory abstraction, which allows us to efficiently share data across the different stages of a map-reduce job or provide in-memory data sharing. Our lecture starts with a brief introduction to Spark and its Hadoop related ecosystem, and then shows some common techniques – classification, collaborative filtering, and anomaly detection, among others, to fields scientific applications, social media analysis, web-analytics and finance. If you have an entry-level understanding of machine learning and statistics, and program in Python or Scala, you will find these subjects useful for working on your own big data challenges.

Syllabus

• Introduction to Data Analysis with Apache Spark
• Spark Programming Model with RDD objects and DataFrames
• Running Spark Applications on Hadoop / Cloud-based Cluster Systems
• Spark SQL
• Spark Streaming
• Machine Learning with Spark MLlib/ML
• Advanced Analytics Applications with Spark
• Anaysis of real world applications

References

https://spark.apache.org, Unified Analytics Engine for Big Data

Advanced Analytics with Spark: Patterns For Learning From Data at Scale, A. Teller, M. Pumperla, M. Malohlava

Mastering Machine Learning with Apache Spark 2.x, S. Amirgodshi, M. Rajendran, B. Hall, S. Mei

Pre-requisites

Python, Machine Learning, Distributed Computing.

Short bio

Altan Cakir has a M.Sc. degree in physics from Izmir Institute of Technology in 2006 and then went straight to graduate school at the Karlsruhe Institute of Technology, Germany. During his Ph.D., he was responsible for a scientific research based on new physics searches in the CMS detector at the Large Hadron Collider (LHC) at European Nuclear Research Laboratory (CERN). Thereafter he was granted as a post-doctoral research fellow at Deutsches Elektronen-Synchotron (DESY), a national nuclear research center in Hamburg, Germany, where he spent 5 years, and then recently got his present full professor position at Istanbul Technical University (ITU), Istanbul, Turkey. Currently, Altan Cakir is a group leader of ITU-CMS group at CERN leading a data analysis group at the CMS detector. Furthermore, he was a visiting faculty at Fermi National Accelerator Laboratory (Fermilab), Illinois, USA in 2017. His group’s expertise is focused around machine learning techniques in large scale data analysis. However, their research is very much interdisciplinary, with expertise in the group ranging from science and big data synthesis to economy, industrial applications and operations research. Today, he is consulting various companies worldwide, and sharing his expertise in big data application areas, strategies, skills and competencies based on the real-world scenarios.

Altan Cakir was involved in a large number of high-profile research projects at CERN, DESY and Fermilab in the last fifteen years. He enjoys being able to integrate his research and teaching key concepts of science and big data technologies. It’s rewarding to be part of the development of the next generation of scientists, engineers and help his students move on to careers all over the world, in academia, industry and government.

The following lectures on big data are periodically given by Assoc. Prof. Dr. Altan Cakir in Big Data and Business Analytics Program (http://bigdatamaster.itu.edu.tr) at Istanbul Technical University: Big Data Technologies and Applications, Machine Learning with Big Data. All in all, Altan Cakir is executive member of ITU AI Center (ai.itu.edu.tr) and one of the lecturers of Cambridge Big Data Program in BigDat2019, University of Cambridge, United Kingdom.