1 PartsIn Progress

LLMs From Scratch

Working through Stanford's CS336 and documenting what I learn at each step — from tokenization to training.

Implementing BPE

Building a Byte-Pair Encoding tokenizer from scratch

You can find the code here at ivanleomk/llms-from-scratch Every Large Language Model faces the same initial hurdle: converting unstructured text into the clean, numerical tokens it can process. The…

34 min read

Implementing BPE

Coming Soon