Course Information
课程简介
The objective of this course is to provide a complete introduction to deep machine learning. General artificial intelligence and machine learning techniques will be quickly reviewed in a historical connections fashion, but the focus of this course is machine learning. So the main theme here is to understand modern techniques that specifically handle deep neural network: how to design it, how to train it and how to evaluate it on real problems.
课程内容
The course aims at providing an overview of existing approaches and methods, at teaching how to design and train a deep neural network for a given task, and (depending on the audience,) at providing the theoretical basis to go beyond the topics directly seen in the course.
The course will touch on the following topics:
- What is deep learning, and how is it derived from general artificial intelligence and machine learning.
- Tensors, multi-layer Perceptrons, gradient descent, back-propagation.
- Generalized networks, automatic differentiation, batch processing, convolutional networks.
- Initialization, optimization, and regularization.
- Drop-out, BatchNorm, ResNets.
- Deep models for computer vision.
- Deep models for sequence processing.
- Analysis and visualization of deep models, geometric explanation of deep learning.
- Auto-encoders, embeddings, and generative models.
Concepts will be illustrated with examples in the TensorFlow and Keras framework.
Keywords: deep learning, neural networks, python, TensorFlow, Keras
先修课程
本课程建议掌握如下专业知识：
必修：
- Linear algebra (vector, matrix computation, Euclidean spaces).
- Differential calculus (Jacobian, Hessian, chain rule).
- Probabilities and statistics (common distributions, law of large numbers, conditional probabilities, Bayes)
- Python programming.
选修：
- Numerical optimization (notion of minima, gradient descent).
- Algorithm analysis (computational costs).
- Visual computing (image processing).
课程计划
本课程计划安排12周内容，主要讲授内容分别如下：
- Introduction 导论：什么是深度学习
- 深度学习工作站的基本配置（实践）
- 神经网络的数学基础
- Python及深度学习编程基础（实践）
- 神经网络入门
- 二分类问题、多分类问题和回归问题（实践）
- 机器学习基础
- 过拟合与欠拟合（实践）
- 深度学习用于计算机视觉
- 卷积神经网络及其可视化（实践）
- 深度学习用于文本和序列
- 用卷积神经网络处理序列（实践）
- 生成式深度学习
- 深度学习的一些实践技巧与总结
Schedule
| Week | Date | Lecture | Handouts |
|---|---|---|---|
| 1 | 2021/03/01 | 导言 | slides pdf |
| 2 | 2021/03/08 | Tutorial: Getting started, MNIST classification | notes pdf code |
| 3 | 2021/03/15 | Math: tensor, geometric explaination; Tutorial: 2D affine transformation | slides pdf code |
| 4 | 2021/03/22 | Math: gradient descent, back-propagation, universal approximation theorem; Tutorial: basic activations | code |
| 5 | 2021/03/29 | Neural network; Tutorial: binary classification | slides pdf code |
| 6 | 2021/04/05 | - | - |
| 7 | 2021/04/12 | Tutorial: multi-class classification, regression | code |
| 8 | 2021/04/19 | Machine learning: model evaluation, feature engineering; Tutorial: data fitting | slides pdf code |
| 9 | 2021/04/26 | Machine learning: pipeline, overfitting & underfitting; Tutorial: spot & counter overfitting | code |
| 10 | 2021/05/03 | - | - |
| 11 | 2021/05/10 | Application: computer vision | slides pdf code |
| 12 | 2021/05/17 | Application: sequence processing | slides pdf code |
| 13 | 2021/05/24 | Advanced practices, conclusion | - |
教学方法
Problem-solving oriented, equal emphasis on lecture and practice. 以解决实际问题为导向，教学与实践并重。
考核标准
Attendance & participation: 40%, final project: 50%, project (bonus): 10%.
- Final project: due 3 weeks after the last lecture.
选用教材
以《Python 深度学习》为主。其他辅助教材：《深度学习》，《机器学习》。
