| title | CSE 291 - AI Agents - UCSD CSE |
|---|---|
| layout | default |
| img | |
| active_tab | main_page |
This course will cover the basics of (1) what LLM-based AI Agents actually are; (2) where they can be useful (and where they are not); and (3) how to safely train and deploy an agent for a given virtual domain.
- What are agents? Starting in the 70s to today
- Simulation Environments, how they are built
- Search / Planning in Simulations
- Monte Carlo Methods
- Basics of Deep RL
- Search + RL
- LLM (Post) Training Basics: SFT, RLHF, Preference Reward, Closed Form Methods
- Deep RL for Language Agents
- Other LLM Agents Methods – Tool Use, In-context Learning, Inference Time Scaling
- Multi Agent + Human-AI Collab
- Agent Safety
Students should be familiar with basic CS concepts such as Search (A*, Monte Carlo Tree Search) and Deep Learning concepts such as Transformers (how self-attention works) and the basics of how Large Language Models are (pre-)trained. Students are expected to come into the class with the ability to implement these concepts from scratch (in Python/numpy) and also be able to use popular libraries such as Huggingface. Basic knowledge of Reinforcement Learning (what is a Markov Decision Process, differences between online and offline RL, RL from Human Feedback) is a plus but not required.
Undergrad Intro to AI/RL, and grad level Intro to Deep Learning / NLP types of courses are highly recommended
Instructors: Prithviraj Ammanabrolu
TAs: Bosung Kim
- Winter 2025, Tuesdays & Thursdays from 3:30 to 4:50 pm PT in CSE 2154
- First day of class is January 06, 2025
- Last day of class is March 13, 2025
Office Hours (see the Staff page for office hour locations)
There is no textbook for this course, but you will be required to puchase a varitey of materials including software and API credits. If any of these are prohibitively expensive for your budget, please let the instructor know.
- 40% - 4 Homeworks (including HW 0) - to be done individually
- 40% - Final Project – to be in groups of ~5
- 20% - Class participation - attendence and paper presentations
Over the course of the semester, as part of the class participation each final project group must prepare one or more ~15-20 minute presentations on a research paper relevant to the course. Since these presentation will be a substantial component of the learning experience in the class, slides must be prepared and emailed to us at least 72 hours in advance of the lecture they will be presented in (e.g., by 3PM on the Monday before the presenation date), so that we can provide feedback on them. Failure to send us the slides ahead of time will result in a grade penalty on the presentation.
Homeworks are expected to be done individually, and teams of ~5 are needed for the final project.
You may use LLMs for code completion on coding assignments but must submit a CREDITS.txt file noting the exact LLM you used along with prompt. All homeworks will also require explanations written of the code which must be done solely without an LLM. Writing on final projects can be edited but not entirely written with an LLM. Failure to comply with any of these policies will result in a 0 on that particular assignment.
Each student has five free "late days". Homeworks can be submitted at most two days late. If you are out of late days, then you will not be able to get credit for subsequent late assignments. One "day" is defined as anytime between 1 second and 24 hours after the homework deadline. The intent of the late day policy it to allow you to take extra time due to unforseen circumstances like illnesses or family emergencies, and for forseeable interruptions like on campus interviewing and religious holidays. You do not need to ask permission to use your late days. No additional late days are granted. Late days only apply to the homeworks. They cannot be used on the final project, which must be finished by the final day of class. Late days may not be used for paper presentations.