Build a language model end to end: train a tokenizer, implement Llama-2, pretrain under a fixed compute budget, evaluate it, and adapt it to a downstream task. Individual assignment, 200 points.

Two Gradescope submissions, both due September 24 at 11:59 PM:

  • Assignment1: Code — submission.zip from scripts/zip-submission.sh, graded by the autograder.
  • Assignment1: Report — report.pdf from the provided LaTeX template.

Late submissions cost 5% per day (see the syllabus) and close on October 14 at 11:59 PM. Downloads: handout (PDF) · starter code (zip) · report template (zip)