PKU

See More

MATH 6380o. Deep Learning: Towards Deeper Understanding
Spring 2018

Course Information

Synopsis

This course is inspired by Stanford Stats 385, Theories of Deep Learning, taught by Prof. Dave Donoho, Dr. Hatef Monajemi, and Dr. Vardan Papyan, as well as the IAS-HKUST workshop on Mathematics of Deep Learning during Jan 8-12, 2018. The aim of this course is to provide graduate students who are interested in deep learning a variety of mathematical and theoretical studies on neural networks that are currently available, in addition to some preliminary tutorials, to foster deeper understanding in future research.
Prerequisite: There is no prerequisite, though mathematical maturity on approximation theory, harmonic analysis, optimization, and statistics will be helpful. Enrolled students should have some programming experience with modern neural networks, such as PyTorch, Tensorflow, MXNet, Theano, and Keras, etc. Otherwise, it is recommended to take some courses on Statistical Learning (Math 4432 or 5470), and Deep learning such as Stanford CS231n with assignments, or a similar course COMP4901J by Prof. CK TANG at HKUST.

Reference

Theories of Deep Learning, Stanford STATS385 by Dave Donoho, Hatef Monajemi, and Vardan Papyan

On the Mathematical Theory of Deep Learning, by Gitta Kutyniok

Tutorials: preparation for beginners

Python-Numpy Tutorials by Justin Johnson

scikit-learn Tutorials: An Introduction of Machine Learning in Python

Jupyter Notebook Tutorials

PyTorch Tutorials

Deep Learning: Do-it-yourself with PyTorch, A course at ENS

Tensorflow Tutorials

MXNet Tutorials

Theano Tutorials

Manning: Deep Learning with Python, by Francois Chollet [GitHub source in Python 3.6 and Keras 2.0.8]

MIT: Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Instructors:

Yuan Yao

Time and Place:

TuTh 3-4:20pm, Academic Bldg 2302 (Lift 17/18), HKUST
Venue changed: LTD from Feb 13, 2018.

Homework and Projects:

No exams, but extensive discussions and projects will be expected.

Teaching Assistant:


Email: Mr. Yifei Huang deeplearning.math (add "AT gmail DOT com" afterwards)

Schedule

Date Topic Instructor Scriber
02/01/2018, Thu Lecture 01: Overview [ Lecture01a.pdf ]
Y.Y.
02/06/2018, Tue Lecture 02: Invariance Wavelet Scattering Transform [ Lecture02.pdf ]