Course Information

Short Intro

The objective of this course is to provide a complete introduction to machine learning, which discusses computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize faces, recommend music and movies, and drive autonomous robots).

Description

This course covers theoretical and practical algorithms for machine learning from different perspectives: theoretical concepts include induction bias, PAC learning framework, Bayesian learning methods, and Occam’s razor; programming assignments include hands-on experiments with various learning algorithms. This course is designed to provide a solid foundation for students who wish to master modern artificial intelligence techniques and to provide the necessary methods, techniques, mathematics, and algorithms for those who wish to pursue future research in machine learning.

The course will touch on the following topics:

• decision tree learning, linear models
• model selection and optimization, statistical learning methods
• neural networks, deep learning
• nonparametric models, ensemble learning

Keywords: machine learning, statistical learning, deep learning

Course group

Previous offers: [2024 Autumn], [2024 Spring], [2023 Spring]

Closely related: [Artificial Intelligence], [Deep Learning], [Natural Language Processing]

Prerequisites

We expect you to have the following skills before taking this course:

Required：

• Linear algebra (vector, matrix computation, Euclidean spaces).
• Differential calculus (Jacobian, Hessian, chain rule).
• Probabilities and statistics (common distributions, law of large numbers, conditional probabilities, Bayes)
• Proficient in one programming language (Python recommended).

Recommended：

• Numerical optimization (notion of minima, gradient descent).
• Algorithm analysis (computational costs).
• Specialized knowledge (visual computing, robotics, speech and language processing).

Teaching Plan

This course is organized into a 12-week session (4 hours per week). The main contents are listed below:

1. Introduction 导言，Preliminaries 预备知识
2. Decision Trees 决策树
3. Linear Models 线性模型
4. Model Selection 模型选择
5. (*)Nonparametric Models 非参数化模型
6. (*)Ensemble Learning 集成学习
7. (*)Statistical Learning Concepts 统计学习概念
8. (*)Probability Models: Complete Data 概率模型：完备数据
9. (*)Probability Models: Hidden Variables 概率模型：隐变量
10. Feedforward Neural Networks 前馈神经网络，Back Propagation 反向传播
11. Deep Learning Computation 深度学习计算
12. Convolutional Neural Networks 卷积神经网络
13. Modern Convolutional Neural Networks 现代卷积神经网络
14. Recurrent Neural Networks 循环神经网络
15. Modern Recurrent Neural Networks 现代循环神经网络

Tutorials are designed to consistent with lectures:

1. Getting Started 实验配置及基础操作
2. Decision Trees 决策树
3. Linear Regression 线性回归
4. Linear Classification 线性分类
5. (*)Nonparametric Models 非参数化模型
6. (*)Ensemble Learning 集成学习
7. Feedforward Neural Networks 前馈神经网络
8. Deep Learning Computation 深度学习计算
9. Convolutional Neural Networks 卷积神经网络
10. Recurrent Neural Networks 循环神经网络

Schedule

Several special topics:

Ref. [2024 Spring], [2023 Spring].

Methodology

Problem-solving oriented, equal emphasis on lecture and practice.

以解决实际问题为导向，教学与实践并重。

Each lecture is roughly organized into 3 progressive units:

1. Core Concepts 核心概念: provides elementary knowledge of the topic
2. Advanced Discussion 进阶讨论: provides in-depth understanding, mathematical formulations
3. Practical Skills 实践技巧: provides problem-solving skills through hands-on programming training

Evaluation

• Attendance & participation: 10%
• Understanding of topics: 20%
• Practical exercises: 20%
• Final exam/project: 50%

Project

Textbook

Not mandatory but recommended:

• Christopher Bishop, Pattern Recognition and Machine Learning, 2006, aka. PRML
  • Bishop, C.M. and Bishop, H. Deep Learning: Foundations and Concepts. Springer, 2023. aka. “New Bishop”
• Zhang et al., Dive into Deep Learning, 2023, 即《动手学深度学习》
• Hastie, T., Tibshirani, R., & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.). Stanford, 2009. 即《统计学习要素: 机器学习中的数据挖掘、推断和预测》
• S. Theodoridis, Machine Learning: A Bayesian and Optimization Perspective, 2nd Edition 2019. 即《机器学习：贝叶斯和优化方法》
• A. Gelman et al., Bayesian Data Analysis, 3rd Edition 2013.
• Murphy, Probabilistic Machine Learning: An Introduction & Advanced Topics, 2022

Resource

• 动手学深度学习¹
• Machine Learning with PyTorch and Scikit-Learn Book: GitHub², online courses³
• PRML Book⁴
• Deep Learning: Foundations and Concepts⁵
• The Elements of Statistical Learning⁶
• Bayesian Data Analysis⁷
• Probabilistic Machine Learning: An Introduction⁸, Advanced Topics⁹
• Mathematics for Machine Learning¹⁰
• 微软人工智能教育与学习共建社区（Microsoft AI Education Community, 简称AI-Edu）¹¹