IERG5050 AI Foundation Models, Systems and Applications / Fall 2025

Announcements

Released: [Homework 1]. Due: Mon, September 29, 11:59 PM.
Released: [Reading Assignment 1]. Due: Sun, September 21, 11:59 PM.
Released: [Homework 0]. Due: Sat, September 13, 11:59 PM.
The due date for HW0 is strict for all. Late-add student will NOT be granted extra time extension for submission.
Website account: AI, password: Fall2025AI
Please be noted that Late-add students MUST attend the lectures from the very beginning of the semester.

Course Description

Foundation Models in Artificial Intelligence (AI), such as GPT, Claude, Gemini and Llama, are large-scale neural networks that serve as a base for building a wide range of AI / Deep Learning applications. These models are pre-trained on massive amounts of data using distributed / parallel computing infrastructure. They are called foundation models because of their adaptability for many specific downstream tasks after fine-tuning or refinement. This graduate-level course explores the latest developments in AI foundation models, systems, and applications, providing students with a comprehensive understanding of how to design, implement, and deploy AI solutions in different domains by leveraging foundation models.

The course is designed for students who already have a background in deep learning and builds upon the knowledge gained in related introductory courses. It begins with a comprehensive coverage of the Transformer model, which has emerged as a universal and flexible learning architecture. Students will learn how state-of-the-art foundation models are constructed for different application domains using the Transformer as the basic building block. The course also covers how to leverage distributed / parallel computing infrastructure and methodologies to enable the training, serving, and deployment of foundation models in a scalable manner. Key aspects of foundation models including their emergent behavior, scaling laws, in-context learning ability, adaptation and augmentation are covered. The role of foundation models in the low-code, rapid development and adoption of new AI applications/ agents, as well as their societal considerations will also be discussed.

Throughout the course, students will gain hands-on experience through assignments, case studies, and a project, allowing them to develop a comprehensive understanding of AI foundation models and their applications in different domains.

Please check Blackboard for important announcements, assignment submissions, grades, etc.

Course Assessment

Homework and Programming Assignments (4 sets including HW0): 50%
Review/ Reflection write-ups for assigned Papers/ Recorded Talks: (2% each, more than 8 sets, count Top 8 only): 15%
Final Exam: 15%
Project: 20%

Student/Faculty Expectations on Teaching and Learning

http://mobitec.ie.cuhk.edu.hk/StaffStudentExpectations.pdf

Academic Honesty

You are expected to do your own work and acknowledge the use of anyone else’s words or ideas. You MUST put down in your submitted work the names of people with whom you have had discussions.

Refer to http://www.cuhk.edu.hk/policy/academichonesty for details

When scholastic dishonesty is suspected, the matter will be turned over to the University authority for action.

You MUST include the following signed statement in all of your submitted homework, project assignments and examinations. Submission without a signed statement will not be graded.

I declare that the assignment here submitted is original except for source material explicitly acknowledged, and that the same or related material has not been previously submitted for another course. I also acknowledge that I am aware of University policy and regulations on honesty in academic work, and of the disciplinary guidelines and procedures applicable to breaches of such policy and regulations, as contained in the website http://www.cuhk.edu.hk/policy/academichonesty/.

Academic Honesty Slides from Associate Dean of Faculty of Engineering

Generative AI Policy

You are NOT allowed to use any Generative AI (e.g., ChatGPT, Claude etc.) in this course, unless it is explicitly approved by the instructor in advance. Anyone who uses LLMs for completing the homework will be treated as cheating.