DeepLearn 2022 Spring
5th International School
on Deep Learning
Guimarães, Portugal · April 18-22, 2022
Registration
Downloads
  • Call DeepLearn 2022 Spring
  • Poster DeepLearn 2022 Spring
  • Lecture Materials
  • Home
  • Schedule
  • Lecturers
  • Travel
  • Accommodation
  • Info
    • Sponsors
    • Code of conduct
    • Visa
    • News
  • Home
  • Schedule
  • Lecturers
  • Travel
  • Accommodation
  • Info
    • Sponsors
    • Code of conduct
    • Visa
    • News
deeplearn-speakers-roy

Kaushik Roy

Purdue University

[intermediate] Re-engineering Computing with Neuro-inspired Learning: Algorithms, Architecture, and Devices

Summary

Advances in machine learning, notably deep learning, have led computers to match or surpass human performance in several cognitive tasks including vision, speech and natural language processing. However, implementation of neural algorithms in conventional “von-Neumann” architectures are several orders of magnitude more area and power expensive than the biological brain. Hence, we need fundamentally new approaches to sustain the exponential growth in performance at high energy-efficiency. Exploring the new paradigm of computing necessitates a multi-disciplinary approach: exploration of new learning algorithms inspired from neuroscientific principles, developing network architectures best suited for such algorithms, new hardware techniques to achieve orders of improvement in energy consumption, and nanoscale devices that can closely mimic the neuronal and synaptic operations. In this talk, I will present our recent work on spike-based learning to achieve high energy efficiency with accuracy comparable to that of standard analog deep-learning techniques. Input coding from DVS cameras has been used to develop energy efficient hybrid SNN/ANN networks for optical flows, gesture recognition, and language translation. Additionally, we propose probabilistic neural and synaptic computing platforms that can leverage the underlying stochastic device physics of spin-devices. System-level simulations indicate ~100x improvement in energy consumption for such spintronic implementations over a corresponding CMOS implementation across different computing workloads. Complementary to the above device efforts, we have explored different local/global learning algorithms including stochastic learning with one-bit synapses that greatly reduces the storage/bandwidth requirement while maintaining competitive accuracy, and adaptive online learning that efficiently utilizes the limited memory and resource constraints to learn new information without catastrophically forgetting already learnt data.

Syllabus

  • Basics of neuromorphic computing; input representations
  • Training large scale spiking and non-spiking networks – local and global learning
  • Spiking networks for solving beyond perception centric tasks
  • Hardware implications and design of neuromorphic in-memory computing hardware for energy efficiency

References

Selected references – will add more later

  • K. Roy, P. Panda, A. Jaiswal, “Towards spike-based machine intelligence with neuromorphic computing,” Nature 575 (7784) 607-617.
  • A. Sengupta et. al. ,” Going deeper in spiking neural networks: Vgg and residual architectures,” Frontiers in neuroscience 13.
  • E.O. Neftci, H. Mostafa, F. Zenke, “Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks,” IEEE Signal Processing Magazine 36 (6), 51-63.
  • N. Rathi and K. Roy, “DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization”, IEEE Transactions on Neural Networks and Learning Systems.

Pre-requisites

Basics of regression and neural networks, basic computer architecture and logic design.

Short bio

Kaushik Roy is the Edward G. Tiedemann, Jr., Distinguished Professor of Electrical and Computer Engineering at Purdue University. He received his BTech from Indian Institute of Technology, Kharagpur, PhD from University of Illinois at Urbana-Champaign in 1990 and joined the Semiconductor Process and Design Center of Texas Instruments, Dallas, where he worked for three years on FPGA architecture development and low-power circuit design. His current research focuses on cognitive algorithms, circuits and architecture for energy-efficient neuromorphic computing/ machine learning, and neuro-mimetic devices. Kaushik has supervised 91 PhD dissertations and his students are well placed in universities and industry. He is the co-author of two books on Low Power CMOS VLSI Design (John Wiley & McGraw Hill).

Dr. Roy received the National Science Foundation Career Development Award in 1995, IBM faculty partnership award, ATT/Lucent Foundation award, 2005 SRC Technical Excellence Award, SRC Inventors Award, Purdue College of Engineering Research Excellence Award, Outstanding Mentor Award in 2021, Humboldt Research Award in 2010, 2010 IEEE Circuits and Systems Society Technical Achievement Award (Charles Desoer Award), IEEE TCVLSI Distinguished Research Award in 2021,  Distinguished Alumnus Award from Indian Institute of Technology (IIT), Kharagpur, Fulbright-Nehru Distinguished Chair, DoD Vannevar Bush Faculty Fellow (2014-2019), Semiconductor Research Corporation Aristotle award in 2015, and best paper awards at 1997 International Test Conference, IEEE 2000 International Symposium on Quality of IC Design, 2003 IEEE Latin American Test Workshop, 2003 IEEE Nano, 2004 IEEE International Conference on Computer Design, 2006 IEEE/ACM International Symposium on Low Power Electronics & Design, 2005 and 2019 IEEE Circuits and system society Outstanding Young Author Award (Chris Kim, Abhronil Sengupta), 2006 IEEE Transactions on VLSI Systems best paper award, 2012 ACM/IEEE International Symposium on Low Power Electronics and Design best paper award, 2013 IEEE Transactions on VLSI Best paper award. Dr. Roy was a Purdue University Faculty Scholar (1998-2003). He was a Research Visionary Board Member of Motorola Labs (2002) and held the M. Gandhi Distinguished Visiting faculty at Indian Institute of Technology (Bombay) and Global Foundries visiting Chair at National University of Singapore. He has been in the editorial board of IEEE Design and Test, IEEE Transactions on Circuits and Systems, IEEE Transactions on VLSI Systems, and IEEE Transactions on Electron Devices. He was Guest Editor for Special Issue on Low-Power VLSI in the IEEE Design and Test (1994) and IEEE Transactions on VLSI Systems (June 2000), IEE Proceedings — Computers and Digital Techniques (July 2002), and IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2011). Dr. Roy is a fellow of IEEE.

Other Courses

Eneko AgirreEneko Agirre
Altan CakirAltan Çakır
Rylan ConwayRylan Conway
deeplearn-speakers-gaoJianfeng Gao
deeplearn-speakers-hanBohyung Han
deeplearn-speakers-karamLina J. Karam
Kyle KeaneKyle Keane
deeplearn-speakers-xiaoming-liuXiaoming Liu
Jennifer NgadiubaJennifer Ngadiuba
Lucila Ohno-MachadoLucila Ohno-Machado
deeplearn-speakers-rajBhiksha Raj
Bart ter Haar RomenijBart ter Haar Romeny
deeplearn-speakers-saadWalid Saad
Yvan SaeysYvan Saeys
deeplearn-speakers-schultzMartin Schultz
deeplearn-speakers-smith-milesKate Smith-Miles
Mihai SurdeanuMihai Surdeanu
deeplearn-speakers-vallecorsaSofia Vallecorsa
deeplearn-speakers-vazirgiannisMichalis Vazirgiannis
Guowei WeiGuowei Wei
deeplearn-speakers-xiaowei-xuXiaowei Xu
deeplearn-speakers-zhaoGuoying Zhao
Zhongming ZhaoZhongming Zhao

DeepLearn 2022 Spring

CO-ORGANIZERS

Algoritmi Center, University of Minho, Guimarães

School of Engineering, University of Minho

Intelligent Systems Associate Laboratory, University of Minho

Rovira i Virgili University

Municipality of Guimarães

Institute for Research Development, Training and Advice – IRDTA, Brussels/London

Active links
  • DeepLearn 2022 Autumn – 7th International School on Deep Learning
  • DeepLearn 2022 Summer – 6th International School on Deep Learning
  • TPNC 2020 & 2021 – 9th-10th International Conference on the Theory and Practice of Natural Computing
  • SLSP 2020 & 2021 – 8th-9th International Conference on Statistical Language and Speech Processing
  • AlCoB 2020 & 2021 – 7th-8th International Conference on Algorithms for Computational Biology
  • LATA 2020 & 2021 – 14th-15th International Conference on Language and Automata Theory and Applications
Past links
  • DeepLearn 2021 Summer
  • DeepLearn 2019
  • DeepLearn 2018
  • DeepLearn 2017
© IRDTA 2021. All Rights Reserved.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Cookie SettingsAccept All
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-advertisement1 yearThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Advertisement".
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
PHPSESSIDsessionThis cookie is native to PHP applications. The cookie is used to store and identify a users' unique session ID for the purpose of managing user session on the website. The cookie is a session cookies and is deleted when all the browser windows are closed.
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
CookieDurationDescription
_ga2 yearsThis cookie is installed by Google Analytics. The cookie is used to calculate visitor, session, campaign data and keep track of site usage for the site's analytics report. The cookies store information anonymously and assign a randomly generated number to identify unique visitors.
_gat_gtag_UA_74880351_91 minuteThis cookie is set by Google and is used to distinguish users.
_gid1 dayThis cookie is installed by Google Analytics. The cookie is used to store information of how visitors use a website and helps in creating an analytics report of how the website is doing. The data collected including the number visitors, the source where they have come from, and the pages visted in an anonymous form.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT
Powered by CookieYes Logo