Course Information
Short Intro
The objective of this course is to provide a complete introduction to
deep machine learning. General artificial intelligence and machine
learning techniques will be quickly reviewed in a historical connections
fashion, but the focus of this course is machine learning. So the main
theme here is to understand modern techniques that specifically handle
deep neural network: how to design it, how to train it and how to
evaluate it on real problems.
Description
The course aims at providing an overview of existing approaches and
methods, at teaching how to design and train a deep neural network for a
given task, and (depending on the audience,) at providing the
theoretical basis to go beyond the topics directly seen in the
course.
The course will touch on the following topics:
- What is deep learning, and how is it derived from general artificial
intelligence and machine learning.
- Tensors, multi-layer Perceptrons, gradient descent,
back-propagation.
- Generalized networks, automatic differentiation, batch processing,
convolutional networks.
- Initialization, optimization, and regularization.
- Drop-out, BatchNorm, ResNets.
- Deep models for computer vision.
- Deep models for sequence processing.
- Analysis and visualization of deep models, geometric explanation of
deep learning.
- Auto-encoders, embeddings, and generative models.
Concepts will be illustrated with examples in the PyTorch
framework.
Keywords: deep learning, neural networks, python,
PyTorch
Course group
Previous offers: [2022], [DeepLearning21F]
Closely related: [Artificial
Intelligence], [Machine
Learning], [Natural Language
Processing]
Prerequisites
We expect you to have the following skills before taking this
course:
本课程建议掌握如下专业知识:
Required:
- Linear algebra (vector, matrix computation, Euclidean spaces).
- Differential calculus (Jacobian, Hessian, chain rule).
- Probabilities and statistics (common distributions, law of large
numbers, conditional probabilities, Bayes)
- Python programming.
Recommended:
- Numerical optimization (notion of minima, gradient descent).
- Algorithm analysis (computational costs).
- Visual computing (image processing).
Teaching Plan
This course is organized into a 12-week session (4 hours per week).
The main contents are listed below:
本课程计划安排12周内容,主要讲授内容分别如下:
- Introduction 导言:什么是深度学习
- Preliminaries 预备知识
- Linear Neural Networks 线性神经网络
- Multilayer Perceptrons 多层感知机
- Deep Learning Computation 深度学习计算
- Convolutional Neural Networks 卷积神经网络
- Modern Convolutional Neural Networks 现代卷积神经网络
- Computer Vision: Applications I 计算机视觉应用I
- Computer Vision: Applications II 计算机视觉应用II
- Recurrent Neural Networks 循环神经网络
- Modern Recurrent Neural Networks 现代循环神经网络
- Natural Language Processing: Applications I 自然语言处理应用I
For classes that also take Natural Language Processing
2023F, NLP part will be changed as follows:
对于同时学修《自然语言处理》的班级,课程关于NLP的部分将调整为如下内容:
- Attention Mechanisms 注意力机制
- Transformers and Pretraining Transformers及预训练
- Natural Language Processing: Applications II 自然语言处理应用II
Schedule
Multiple offerings; Multiple locations.
| Week |
Date |
Lecture |
Handouts |
| 1 |
2023/02/27 |
[导言] |
[课程信息] |
| 2 |
2023/03/06 |
[预备知识] |
[安装] |
| 3 |
2023/03/13 |
[线性回归] |
|
| 4 |
2023/03/20 |
[线性分类] |
|
| 5 |
2023/03/27 |
[前馈神经网络] |
|
| 6 |
2023/04/03 |
[深度计算] |
|
| 7 |
2023/04/10 |
[卷积神经网络] |
|
| 8 |
2023/04/17 |
[现代卷积神经网络] |
|
| 9 |
2023/04/24 |
[循环神经网络] |
|
| 10 |
2023/05/01 |
holiday |
|
| 11 |
2023/05/08 |
[现代循环神经网络] |
|
| 12 |
2023/05/15 |
review |
|
Methodology
Problem-solving oriented, equal emphasis on lecture and practice.
以解决实际问题为导向,教学与实践并重。
Each lecture is roughly organized into 3 progressive units:
- Core Concepts 核心概念: provides elementary knowledge of the
topic
- Advanced Discussion 进阶讨论: provides in-depth understanding,
mathematical formulations
- Practical Skills 实践技巧: provides problem-solving skills through
hands-on programming training
Evaluation
- Attendance & participation: 10%
- Understanding of topics: 40%
- Final exam/project: 50%
Textbook
Not mandatory but recommended:
以《动手学深度学习》为主。其他辅助教材:
- Chollet, Deep Learning with Python
- Courville et al., Deep Learning
- 周志华, 机器学习
- Friedman et al., The Elements of Statistical Learning
Resource
- 动手学深度学习
- HUNG-YI LEE (李宏毅), MACHINE LEARNING 2022 SPRING