Tentative Schedule
This is a tentative schedule. It will be updated according to the actual progress.
-
Week/DateDescriptionCourse MaterialRemark
-
Week 1
03/09/2025 (Wed)
Overview of Foundation Models in AI; Pre-Transformer Era of Deep Learning models[slides] -
Week 2/ Week 3
10/09/2025 (Wed)
17/09/2025 (Wed)
Transformer: Basic Architecture[slides] -
13/09/2025 23:59PM
SaturdayHomework 0 is due!Due -
21/09/2025 23:59PM
SundayReading Assignment 1 is due!Due -
Week 4
24/09/2025 (Wed)
Efficient Transformer Architectures[slides] -
29/09/2025 23:59PM
MondayHomework 1 is due!Due -
01/10/2025 (Wed)
National Day Holiday -
Week 5
04/10/2025 (Sat)
9:30am-12:30pm
Venue: SHB 801
Beyond Transformers: Alternative Architectures for LLMs[slides]Readings: [SSM@SimonInstitute] [SSM@icml24] [Post-TransformerArch] [Mamba] [introRWKV&WorldToken] [Eagle7B&RWKVv5] [RWKVv7]
Make-up lecture -
06/10/2025 23:59PM
MondayReading Assignment 2 is due!Due -
Week 6
08/10/2025 (Wed)
Distributed/ Parallel Training of Foundation Models[slides]Readings: [Megatron] [DeepSpeedZeRO] [ZeRO] Sec2.5 of [OPT]
-
13/10/2025 23:59PM
MondayReading Assignment 3 is due!Due -
Week 7
15/10/2025 (Wed)
Pre-training of Foundation Models and Data Preparation[slides] -
20/10/2025 23:59PM
MondayReading Assignment 4 is due!Due -
Week 8
22/10/2025 (Wed)
Post-training/ Adaptation of Foundation Models;[slides] -
29/10/2025 (Wed)
Double Ninth Festival Holiday -
03/11/2025 23:59PM
MondayHomework 2 is due!Due -
Week 09/ Week 10
05/11/2025 (Wed)
12/11/2025 (Wed)
Retrieval Augmented Generation (RAG) and Tool Use for Foundation Models[slides] -
05/11/2025 23:59PM
WednesdayReading Assignment 5 is due!Due -
09/11/2025 23:59PM
SundayProject proposal is due!Due -
12/11/2025 23:59PM
WednesdayReading Assignment 6 is due!Due -
Week 11
19/11/2025 (Wed)
Teaching LLMs to Reason and Reasoning LLMs[slides] [back up slides] -
19/11/2025 23:59PM
WednesdayReading Assignment 7 is due!Due -
Week 12
26/11/2025 (Wed)
Agentic AI Applications[slides] [back up slides] -
29/11/2025 23:59PM
SaturdayReading Assignment 8 is due!Due -
Week 13
01/12/2025 (Mon)
14:30pm-17:30pm
Venue: TYW_LT
AI Safety and Security[slides] [back up slides]Make-up lecture -
01/12/2025 23:59PM
MondayHomework 3 is due!Due -
05/12/2025 23:59PM
FridayReading Assignment 9 is due!Due -
10/12/2025 (Wed)
The final exam will be held on Dec 10 (Wed), 3:30pm to 5:30pm. Venue: TYW Lecture Theatre in Ho Sin Hang Engineering Building.Exam -
13/12/2025 23:59PM
SaturdayReading Assignment 10 is due!Due