IERG5050 AI Foundation Models, Systems and Applications / Spring 2025

Course Description

Foundation Models in Artificial Intelligence (AI), such as GPT, Claude, Gemini and Llama, are large-scale neural networks that serve as a base for building a wide range of AI / Deep Learning applications. These models are pre-trained on massive amounts of data using distributed / parallel computing infrastructure. They are called foundation models because of their adaptability for many specific downstream tasks after fine-tuning or refinement. This graduate-level course explores the latest developments in AI foundation models, systems, and applications, providing students with a comprehensive understanding of how to design, implement, and deploy AI solutions in different domains by leveraging foundation models.

The course is designed for students who already have a background in deep learning and builds upon the knowledge gained in related introductory courses. It begins with a comprehensive coverage of the Transformer model, which has emerged as a universal and flexible learning architecture. Students will learn how state-of-the-art foundation models are constructed for different application domains using the Transformer as the basic building block. The course also covers how to leverage distributed / parallel computing infrastructure and methodologies to enable the training, serving, and deployment of foundation models in a scalable manner. Key aspects of foundation models including their emergent behavior, scaling laws, in-context learning ability, adaptation and augmentation are covered. The role of foundation models in the low-code, rapid development and adoption of new AI applications/ agents, as well as their societal considerations will also be discussed.

Throughout the course, students will gain hands-on experience through assignments, case studies, and a project, allowing them to develop a comprehensive understanding of AI foundation models and their applications in different domains.

Please check Blackboard for important announcements, assignment submissions, grades, etc.

Course Assessment

• Homework and Programming Assignments (3 sets): 60%
• Paper Reviews/Feedback Write-ups (3% each, more than 5 sets, count top 5): 15%
• Project: 25%

Student/Faculty Expectations on Teaching and Learning

http://mobitec.ie.cuhk.edu.hk/StaffStudentExpectations.pdf

Academic Honesty

You are expected to do your own work and acknowledge the use of anyone else’s words or ideas. You MUST put down in your submitted work the names of people with whom you have had discussions.

Refer to http://www.cuhk.edu.hk/policy/academichonesty for details

When scholastic dishonesty is suspected, the matter will be turned over to the University authority for action.

You MUST include the following signed statement in all of your submitted homework, project assignments and examinations. Submission without a signed statement will not be graded.

I declare that the assignment here submitted is original except for source material explicitly acknowledged, and that the same or related material has not been previously submitted for another course. I also acknowledge that I am aware of University policy and regulations on honesty in academic work, and of the disciplinary guidelines and procedures applicable to breaches of such policy and regulations, as contained in the website http://www.cuhk.edu.hk/policy/academichonesty/.

Academic Honesty Slides from Associate Dean of Faculty of Engineering

Generative AI Policy

You are NOT allowed to use any Generative AI (e.g., ChatGPT, Claude etc.) in this course, unless it is explicitly approved by the instructor in advance. Anyone who uses LLMs for completing the homework will be treated as cheating.

Instructor

Prof. Wing C. Lau

Email: wclau [at] ie.cuhk.edu.hk

Office hours: Wed 02:30PM - 03:30PM (SHB 818)

Teaching Assistants

Weiheng Tang

Email: tangweiheng [at] link.cuhk.edu.hk

Office hours: Tue 3:00PM - 04:00PM (SHB 803)

Muyi Wang

Email: muyi.wang [at] link.cuhk.edu.hk

Office hours: Tue 04:30PM - 05:30 (SHB 803)