Notebooks – Understanding LLMs

LLM From Scratch: Part 2

LLM

In this notebook, we incorporate architectural improvements to the LLM from the previous part. While code changes are minor, understanding the rationale is crucial. We’ll…

LLM From Scratch: Part 1

LLM

To create a minimal GPT-style model, you need these core components:

Rotary Position Embedding (RoPE)

LLM

RoPE

After listening to Umar Jamil’s talk on GPU Mode, I was really inspired. I think it was one of the best talks debunking his mindset behind learning and producing super…