Important Details

  • Location: BH A51
  • Time: Tuesdays and Thursdays 2 PM - 3:20 PM
  • Instructor email: f2026-11-711@andrew.cmu.edu

Course Description

Advanced natural language processing is a graduate-level course on natural language processing aimed at students who are interested in doing cutting-edge research in the field. The course focuses on modern methods using neural networks, covering the fundamentals of generative models with a particular focus on the foundations of large language models. This includes the modeling, learning, and inference algorithms required for building cutting-edge NLP systems. The class culminates in an open-ended, creative research project on a cutting-edge topic in NLP.

The course covers key algorithmic foundations and applications of advanced natural language processing. While there are no hard course pre-requisites, programming experience in Python and knowledge of probability and linear algebra are expected. In practice, prior experience with deep learning is highly recommended.

Class Format

Classes will be in person, every Tuesday and Thursday.

Readings: Most classes will have associated reading material. You are expected to read the main readings. Topics within the main readings that are covered in the lecture are eligible for the exams. The Additional References provide additional content that is either related to the lecture or referenced in the lecture; you may find the additional references useful for gaining a better understanding of the lecture content.

Interactive Activities: There will be interactive activities interspersed through the lectures. These will include discussions of the course context and short quizzes. You will be given a participation grade for your participation in the course activities.

Code/Data Walkthrough: Some classes will involve looking through code or data.

Assessment

Your grade will be computed as:

  • 10%: Assignment 1
  • 10%: Assignment 2
  • 10%: Assignment 3
  • 20%: Assignment 4 (Final Project)
  • Midterm 20%
  • Final exam 20%
  • Participation 5%
  • Peer feedback 5%

Homeworks: There will be four homework assignments. Assignments 1 and 2 must be done individually, while Assignments 3 and 4 must be done in teams of 2-3 (individual submissions will not be accepted for these assignments). If you are having trouble finding a group, the instructor and TAs will help you find one after the first initial survey.

The aim of the assignments is to build basic understanding and advanced implementation skills needed to build cutting-edge systems or do cutting-edge research using neural networks for NLP, culminating in a project that demonstrates these abilities.

Exams: The class will have two in-person exams, a midterm exam and a final exam. The midterm exam will assess knowledge of the first half of the class while the final exam will focus on assessing knowledge of the second half.

The only acceptable reasons for requesting a makeup exam are:

  1. A medical emergency (with documentation)
  2. A conflicting exam from a different class
  3. Attendance at a conference in which you are presenting a paper you authored is an acceptable excuse for missing only the midterm, and only if you tell the instructors at least 30 days before the date of the midterm.

If you miss the midterm for any of the above reasons, your final exam will count for 40% of your grade. If you miss the final exam, you must work with the University Registrar’s Office to reschedule. Note, it is the Registrar’s Office, not the instructors of the class, who are responsible for scheduling final exams. Any student who misses the final exam without working with the Registrar’s Office to schedule a makeup exam will receive a 0% on the final exam, with no exceptions.

Participation: Your participation grade will be computed as followed. Let n equal the number of participation activities interspersed through the semester, and x be the number of activities you have participated in.

participation_grade = min(x / (0.8 * n), 1)

In other words, you may miss up to 20% of participation activities and still get full credit for participation. There will be no other accommodations made for missed participation.

Peer feedback: You will occasionally be asked to provide feedback to your peers on their projects. This feedback will be graded generously—we will assign full credit for earnest feedback that was written by you, the student, without the assistance of AI.

Late Policy

Each homework is due at 11:59 PM on its due date. For each unexcused day your homework is late, we will subtract 5% from your final grade for the homework. For example, if you submit your homework 1 minute late, and your grade would otherwise be a 97%, it will drop down to 92%. If you submit 26 hours late, and your grade would otherwise be 90%, it will drop down to 80%.

You may ask for an excused late day or days at least 168 hours (7 days) before the deadline. Any requests received closer than 168 hours before the deadline will automatically get denied, unless the request is medical (with documentation). If your excuse is approved, your grade will not be penalized. Valid excuses are (a) medical and personal health issues, (b) religious commitments, or (c) other extenuating life circumstances. Requests for excused late days may be emailed to f2026-11-711@andrew.cmu.edu.

In the event of a medical emergency, please make your personal health, physical and mental, your first priority. Seek help from medical and care providers such as University Health Services. Students can request medical extensions after the deadline with proof/note from providers.

Accommodations

If you have a disability and require accommodations, please contact Catherine Getchell, Director of Disability Resources, 412-268-6121, getchell@cmu.edu. If you have an accommodations letter from the Disability Resources office, we encourage you to discuss your accommodations and needs with us as early in the semester as possible. We will work with you to ensure that accommodations are provided as appropriate.

Policy on Missing Class

If you must miss class, we recommend securing notes from a friend in the class. We may make video recordings available, but we do not guarantee they will be available for every lecture. If you miss class for any reason, you will be counted as not having completed the interactive activities for that class session. Note, you may miss up to 10% of interactive activities without any penalty to your grade.

Academic Integrity

Please take some time to read through CMU’s Academic Integrity Policy. Students who violate this policy will be subject to the disciplinary actions described in the Student Handbook.

Code

In your code implementations for the assignments:

  • Code or pseudo-code provided by the TAs or instructor may be used freely without restriction.
  • For assignment 2, you may not just re-use an existing implementation written by someone else. The implementation should basically be your own. -Code written by other students in the class cannot be used (except, obviously, you can share code within your group for assignments 3 and 4).
  • If you are doing a similar project for a graded class at CMU (including independent studies or directed research), you must declare so on your report, and note which parts of the project are for 11-711, and which parts are for the other class. Consult with the Instructor during office hours or on Piazza if you are unsure.

Use of Language Models

Using a language model to generate any part of a written homework answer will be considered a violation of academic integrity. To reiterate, all written words you turn in for assessment should be written by yourself, without the use of AI (unless you are quoting AI outputs as part of your answer).

You may however use AI to learn more about the subject material of the class, help organize your thoughts, design figures, or to help you write code. However, you, the student, are ultimately responsible for the correctness of all parts of your homework. If you do use AI systems to help with the homework, please fill out the “Use of AI” question on the homework to help the instructors understand your usage. Use of AI without making an honest attempt at answering this question will be considered a violation of academic integrity.