Build a small language model from scratch: Pre-training and Inference

Dr. Raj Dandekar (MIT PhD) conducted a 7-hour small language model workshop. This is part 3 of that workshop.

In this, we cover the following four aspects:

1. 00:00 Recap of the SLM architecture
2. 18:30 Calculating the SLM Loss
3. 58:27 SLM Pre-training loop: Theory + Coding
4. 01:23:00 SLM Inference

If you want to get access to the detailed lecture notes, register here: https://vizuara.ai/courses/bui....ld-slm-from-scratch-