Course Information
Short Intro
The objective of this course is to provide a complete introduction to deep machine learning. General artificial intelligence and machine learning techniques will be quickly reviewed in a historical connections fashion, but the focus of this course is machine learning. So the main theme here is to understand modern techniques that specifically handle deep neural network: how to design it, how to train it and how to evaluate it on real problems.
Description
The course aims at providing an overview of existing approaches and methods, at teaching how to design and train a deep neural network for a given task, and (depending on the audience,) at providing the theoretical basis to go beyond the topics directly seen in the course.
The course will touch on the following topics:
- What is deep learning, and how is it derived from general artificial intelligence and machine learning.
- Tensors, multi-layer Perceptrons, gradient descent, back-propagation.
- Generalized networks, automatic differentiation, batch processing, convolutional networks.
- Initialization, optimization, and regularization.
- Drop-out, BatchNorm, ResNets.
- Deep models for computer vision.
- Deep models for sequence processing.
- Analysis and visualization of deep models, geometric explanation of deep learning.
- Auto-encoders, embeddings, and generative models.
Concepts will be illustrated with examples in the PyTorch framework.
Keywords: deep learning, neural networks, python, PyTorch
Prerequisites
We expect you to have the following skills before taking this course:
本课程建议掌握如下专业知识：
Required：
- Linear algebra (vector, matrix computation, Euclidean spaces).
- Differential calculus (Jacobian, Hessian, chain rule).
- Probabilities and statistics (common distributions, law of large numbers, conditional probabilities, Bayes)
- Python programming.
Recommended：
- Numerical optimization (notion of minima, gradient descent).
- Algorithm analysis (computational costs).
- Visual computing (image processing).
Teaching Plan
This course is organized into a 12-week session (4 hours per week). The main contents are listed below:
本课程计划安排12周内容，主要讲授内容分别如下：
- Introduction 导言：什么是深度学习
- Preliminaries 预备知识
- Linear Neural Networks 线性神经网络
- Multilayer Perceptrons 多层感知机
- Deep Learning Computation 深度学习计算
- Convolutional Neural Networks 卷积神经网络
- Modern Convolutional Neural Networks 现代卷积神经网络
- Computer Vision: Applications I 计算机视觉应用I
- Computer Vision: Applications II 计算机视觉应用II
- Recurrent Neural Networks 循环神经网络
- Modern Recurrent Neural Networks 现代循环神经网络
- Natural Language Processing: Applications I 自然语言处理应用I
For classes that also take Natural Language Processing 2022F, NLP part will be changed as follows:
对于同时学修《自然语言处理》的班级，课程关于NLP的部分将调整为如下内容：
- Attention Mechanisms 注意力机制
- Transformers and Pretraining Transformers及预训练
- Natural Language Processing: Applications II 自然语言处理应用II
Schedule
Monday, Wednesday; Multiple locations.
| Week | Date | Lecture | Handouts |
|---|---|---|---|
| 1 | 2022/02/21 | [导言] | [课程信息] [安装] |
| 2 | 2022/02/28 | [预备知识] | |
| 3 | 2022/03/07 | [线性模型] | |
| 4 | 2022/03/14 | [多层感知机] [过拟合] | |
| 5 | 2022/03/21 | [深度计算] | |
| 6 | 2022/03/28 | [卷积神经网络] | |
| 7 | 2022/04/04 | [现代卷积神经网络] | |
| 8 | 2022/04/11 | [计算机视觉I] | [课程考核说明] |
| 9 | 2022/04/18 | [计算机视觉II] | |
| 10 | 2022/04/25 | [循环神经网络] | |
| 11 | 2022/05/02 | [现代循环神经网络] [自然语言处理I] | |
| 12 | 2022/05/09 | [注意力机制] [Transformers] |
Methodology
Problem-solving oriented, equal emphasis on lecture and practice.
以解决实际问题为导向，教学与实践并重。
Evaluation
- Attendance & participation: 20%
- Understanding of topics: 20%
- Final project: 50%
- Honorable bonus: 10%
课程考核说明：[pdf]
- release data: 2022/04/11
- collect feedbacks: 1 week
- due: 2022/06/10 (2 weeks after courses).
Textbook
Not mandatory but recommended:
以《动手学深度学习》为主。其他辅助教材：
- Chollet, Deep Learning with Python
- Courville et al., Deep Learning
- 周志华, 机器学习
- Friedman et al., The Elements of Statistical Learning
 |
 |
 |
 |