Daniel Cremers

[introductory/intermediate] Deep Networks for 3D Computer Vision

Summary

While Deep Learning has swept most areas of computer vision and replaced alternative approaches, extending their power to the realm of 3D reconstruction, visual SLAM and 3D shape analysis are still to some extent open challenges. In my course, I will give a glimpse into research on these topics. I will revisit the basic concepts of camera-based reconstruction, visual SLAM and 3D shape analysis. Subsequently, I will show how these can be tackled with suitably designed self-supervised learning approaches.

Syllabus

• Introduction to image-based 3D reconstruction
• Introduction to visual Simultaneous Localization and Mapping (SLAM)
• Development of self-supervised learning approaches to visual SLAM
• Development of hybrid methods that combine classical optimization with deep learning
• Introduction to challenges in 3D shape analysis
• Construction of both classical and deep learning approaches for 3D shape analysis

References

Hartley, Zisserman, “Multiple View Geometry”.

Ma, Soatto, Kosecka, Sastry, “An Invitation to 3D Vision”.

Engel, Schöps, Cremers, “LSD SLAM: Large-scale direct monocular SLAM”, ECCV 2014.

Engel, Koltun, Cremers, “Direct Sparse Odometry”, PAMI 2018.

Yang, Stumberg, Wang, Cremers, “D3VO: Deep depth, deep pose and deep uncertainty for monocular visual odometry”, CVPR 2020.

Eisenberger, Lähner, Cremers, “Smooth Shells: Multi-scale shape registration with functional maps”, CVPR 2020.

Eisenberger, Toker, Leal-Taixé, Cremers, “Deep Shells: Unsupervised shape correspondence with optimal transport”, NeurIPS 2020.

Eisenberger et al., “Neuromorph: Unsupervised shape interpolation and correspondence in one go”, CVPR 2021.

Pre-requisites

Basic mathematics. Basics of deep learning.

Short bio

Daniel Cremers holds the Chair of Computer Vision and Artificial Intelligence at TU Munich. He is Director of the Munich Center for Machine Learning, of the Munich Data Science Institute and of ELLIS Munich. For pioneering research, he has received five grants from the European Research Council. He is member of the Bavarian Academy of Sciences and Humanities. In December 2010 he was listed among “Germany’s top 40 researchers below 40” (Capital). In 2016, he received the Gottfried Wilhelm Leibniz Award, the biggest award in German academia. In 2022, he was listed among the top 10 most influential scholars in robotics of the last decade. He is co-founder and advisor of numerous startups. Since fall 2022, Prof. Cremers is on sabbatical at the University of Oxford.