Ponnuthurai N. Suganthan

[introductory/intermediate] Randomization-Based Deep and Shallow Learning Algorithms and Architectures

Summary

This presentation will focus on randomization-based learning algorithms. Some of the algorithms have closed form solutions while some of the algorithms do not adjust the structures/parameters once constructed. The main algorithms considered in this talk are randomized neural network variants, and random forest variants. These methods have attracted attention of researchers due to their high performance in terms of accuracy. Randomized neural networks can be trained fast. The presentation will include the basic methods as well as their state-of-the-art variants. Deep randomized neural networks will also be extensively presented. Supervised, semi-supervised, continual, online, unsupervised learning approaches will be considered. These algorithms will be benchmarked using tabular classification, time series forecasting, medical data classification and visual tracking datasets. Future research directions will also be suggested.

Syllabus

• Origins of randomized neural networks.
• Deep randomization based neural networks and their variants.
• Random forest and their variants.
• Applications in supervised, semi-supervised, continual, online, unsupervised learning.
• Benchmarking of these methods using diverse datasets.

References

(Additional References will be included in the lecture materials.)

M Hu, JH Chion, PN Suganthan, RK Katuwal, Ensemble Deep Random Vector Functional Link Neural Network for Regression, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023.

R Gao, R Li, M Hu, PN Suganthan, KF Yuen, Dynamic ensemble deep echo state network for significant wave height forecasting, Applied Energy, 2023.

L Du, R Gao, PN Suganthan, DZW Wang, Graph ensemble deep random vector functional link network for traffic forecasting, Applied Soft Computing, 2022.

PN Suganthan, R Katuwal, On the origins of randomization-based feedforward neural networks, Applied Soft Computing 105, 107239, 2021

Q Shi, R Katuwal, PN Suganthan, M Tanveer, Random Vector Functional Link Neural Network based Ensemble Deep Learning, Pattern Recognition, 2021.

MA Ganaie, M Tanveer, PN Suganthan, V Snásel, Oblique and rotation double random forest, Neural Networks 153, 496-517, 2022.

R. Katuwal, PN Suganthan, L. Zhang, Heterogeneous Oblique Random Forest, Pattern Recognition, 2020.

L Zhang, PN Suganthan, Visual tracking with convolutional random vector functional link network, IEEE Transactions on Cybernetics 47 (10), 3243-3253, 2017.

Pre-requisites

Basic knowledge of neural networks, pattern classification, and decision trees will be advantageous.

Short bio

Ponnuthurai Nagaratnam Suganthan received the B.A degree and M.A degree in Electrical and Information Engineering from the University of Cambridge, UK in 1990, 1992 and 1994, respectively. He received an honorary doctorate (i.e. Doctor Honoris Causa) in 2020 from University of Maribor, Slovenia. After completing his PhD research in 1995, he served as a pre-doctoral Research Assistant in the Department of Electrical Engineering, University of Sydney in 1995–96 and a lecturer in the Department of Computer Science and Electrical Engineering, University of Queensland in 1996–99. Since August 2022, he has been with KINDI Centre for Computing Research, Qatar University, as a research professor. He was an Editorial Board Member of the Evolutionary Computation journal, MIT Press (2013–2018). He is/was an associate editor of Applied Soft Computing (Elsevier, 2018–), Neurocomputing (Elsevier, 2018–), IEEE Transactions on Cybernetics (2012–2018), IEEE Transactions on Evolutionary Computation (2005–2021), Information Sciences (Elsevier) (2009– ), Pattern Recognition (Elsevier) (2001– ) and IEEE Trans on SMC: Systems (2020– ). He is a founding co-editor-in-chief of Swarm and Evolutionary Computation (2010– ), an SCI Indexed Elsevier journal. His co-authored SaDE paper (published in April 2009) won the IEEE Transactions on Evolutionary Computation Outstanding Paper Award in 2012. His research interests include randomization-based learning methods, swarm and evolutionary algorithms, pattern recognition, deep learning and applications of swarm, evolutionary & machine learning algorithms. He was selected as one of the highly cited researchers by Thomson Reuters Science Citations yearly from 2015 to 2022 in computer science. He served as the General Chair of the IEEE SSCI 2013. He has been a member of the IEEE (S’91, M’92, SM’00, Fellow 2015) since 1991 and an elected AdCom member of the IEEE Computational Intelligence Society (CIS) in 2014–2016. He was an IEEE CIS distinguished lecturer (DLP) in 2018–2021.

Google Scholar:
http://scholar.google.com.sg/citations?hl=en&user=yZNzBU0AAAAJ&view_op=list_works&pagesize=100