PKU

See More

MATH 6380P. Advanced Topics in Deep Learning
Fall 2020

Course Information

Synopsis

This course is a continuition of Math 6380o, Spring 2018, inspired by Stanford Stats 385, Theories of Deep Learning, taught by Prof. Dave Donoho, Dr. Hatef Monajemi, and Dr. Vardan Papyan, as well as the Simons Institute program on Foundations of Deep Learning in the summer of 2019 and IAS@HKUST workshop on Mathematics of Deep Learning during Jan 8-12, 2018. The aim of this course is to provide graduate students who are interested in deep learning a variety of understandings on neural networks that are currently available to foster future research.
Prerequisite: there is no prerequisite, though mathematical maturity on approximation theory, harmonic analysis, optimization, and statistics will be helpful. Do-it-yourself (DIY) and critical thinking (CT) are the most important things in this course. Enrolled students should have some programming experience with modern neural networks, such as PyTorch, Tensorflow, MXNet, Theano, and Keras, etc. Otherwise, it is recommended to take some courses on Statistical Learning (Math 4432 or 5470), and Deep learning such as Stanford CS231n with assignments, or a similar course COMP4901J by Prof. CK TANG at HKUST.

Reference

Theories of Deep Learning, Stanford STATS385 by Dave Donoho, Hatef Monajemi, and Vardan Papyan

Foundations of Deep Learning, by Simons Institute for the Theory of Computing, UC Berkeley

On the Mathematical Theory of Deep Learning, by Gitta Kutyniok

Tutorials: preparation for beginners

Python-Numpy Tutorials by Justin Johnson

scikit-learn Tutorials: An Introduction of Machine Learning in Python

Jupyter Notebook Tutorials

PyTorch Tutorials

Deep Learning: Do-it-yourself with PyTorch, A course at ENS

Tensorflow Tutorials

MXNet Tutorials

Theano Tutorials

Manning: Deep Learning with Python, by Francois Chollet [GitHub source in Python 3.6 and Keras 2.0.8]

MIT: Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Instructors:

Yuan Yao

Time and Place:

Wed 3:00PM - 5:50PM, Zoom

Homework and Projects:

No exams, but extensive discussions and projects will be expected.

Schedule

Date Topic Instructor Scriber
09/09/2020, Wednesday Lecture 01: Overview I [ slides ]
Y.Y.
09/16/2020, Wednesday Lecture 02: Symmetry and Network Architectures: Wavelet Scattering Net, DCFnet, Frame Scattering, and Permutation Invariant/Equivariant Nets [ slides ] and Project 1.