Tentative Schedule
This is a tentative schedule. It will be updated according to the actual progress.
-
Week/DateDescriptionCourse MaterialRemark
-
Week 1
03/09/2025 (Wed)
Overview of Foundation Models in AI; Pre-Transformer Era of Deep Learning models[slides] -
Week 2/ Week 3
10/09/2025 (Wed)
17/09/2025 (Wed)
Transformer: Basic Architecture[slides] -
13/09/2025 23:59PM
SaturdayHomework 0 is due!Due -
21/09/2025 23:59PM
SundayReading Assignment 1 is due!Due -
Week 4
24/09/2025 (Wed)
Efficient Transformer Architectures[slides] -
29/09/2025 23:59PM
MondayHomework 1 is due!Due -
01/10/2025 (Wed)
National Day Holiday -
Week 5
04/10/2025 (Sat)
9:30am-12:30pm
Venue: SHB 801
Beyond Transformers: Alternative Architectures for LLMsReadings: [SSM@SimonInstitute] [SSM@icml24] [Post-TransformerArch] [Mamba] [introRWKV&WorldToken] [Eagle7B&RWKVv5] [RWKVv7]
Make-up lecture -
06/10/2025 23:59PM
MondayReading Assignment 2 is due!Due -
Week 6
08/10/2025 (Wed)
Distributed/ Parallel Training of Foundation ModelsReadings: [Megatron] [DeepSpeedZeRO] [ZeRO] Sec2.5 of [OPT]
-
Week 7
15/10/2025 (Wed)
Pre-training of Foundation Models and Data Preparation -
Week 8
22/10/2025 (Wed)
Post-training/ Adaptation of Foundation Models; -
29/10/2025 (Wed)
Double Ninth Festival Holiday -
Week 09/ Week 10
05/11/2025 (Wed)
12/11/2025 (Wed)
Retrieval Augmented Generation (RAG) and Tool Use for Foundation Models -
Week 11
19/11/2025 (Wed)
Teaching LLMs to Reason and Reasoning LLMs -
Week 12
26/11/2025 (Wed)
Agentic AI Applications -
Week 13
01/12/2025 (Mon)
14:30pm-17:30pm
Venue: TBD
AI Safety and SecurityMake-up lecture -
10/12/2025 (Wed)
The final exam will be held on Dec 10 (Wed) PM. Venue: TBDExam