| Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training. The only prerequisite is being comfortable with learning through code & excel examples. 1. Sampling Large Language Models
https://go.justinangel.ai/video-1 2. Reverse Engineering Large Language Model
https://go.justinangel.ai/video-2 3. Perceptrons: wx+b
https://go.justinangel.ai/video-3 4. Activation Functions: ReLU, GELU, SwiGLU
https://go.justinangel.ai/video-4 5. GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton
https://go.justinangel.ai/video-5 6. MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks
https://go.justinangel.ai/video-6 7. Loss Functions: Residual errors, RMSE, Cross Entropy, Loss Landscapes
https://go.justinangel.ai/video-7 8. Backpropagation: Training loops, Optimizers, Learning Rate, Batch Size
https://go.justinangel.ai/video-8 9. Saving & Loading Models
https://go.justinangel.ai/video-9 10. Initialization: Kaiming, Glorot
https://go.justinangel.ai/video-10 11. Residuals: Addition, Scaling, Gated, Concatenation
https://go.justinangel.ai/video-11 12. Normalization: Pre-norm vs. Post-norm, RMSNorm, BatchNorm, LayerNorm
https://go.justinangel.ai/video-12 13. Regularization: Dropout, Gradient Clipping, Weight Decay
https://go.justinangel.ai/video-13 14. SoftMax
https://go.justinangel.ai/video-14 15. Tokenizers: By Character, By Word, BPE, SentencePiece
https://go.justinangel.ai/video-15 16. Embeddings: Absolute vs. Learned, Sinusoidal vs. RoPE
https://go.justinangel.ai/video-16 17. Attention: MHA, GQA, MQA, MLA
https://go.justinangel.ai/video-17 18. Transformers
https://go.justinangel.ai/video-18 19. Pre-training: Data Sources, Datasets, HTML Cleaning, Quality Filtering, Sharding
https://go.justinangel.ai/video-19 20. Evaluation: Leaderboards, Benchmarks, Verifiers vs LLM-as-Judge
https://go.justinangel.ai/video-20 21. Instruction Tuning: Alpaca & Other Formats, Self Instruct, Capabilities
https://go.justinangel.ai/video-21 22. Reinforcement Learning: Policy Optimization, SimPO
https://go.justinangel.ai/video-22 23. What We Didn't Cover: Scaling
https://go.justinangel.ai/video-23 Each section has slides teaching the concepts, followed by excel-by-hand developing intuition for the math, and then coding examples. The goal is able to grok all parts of modern LLM development. We did this workshop in-person in San Francisco last month and hopefully the spaciousness of watching online works for everyone.
https://emilyhk.com/llm-workshop/ If don't like watching videos, you can get the slides and exercises and work self-paced.
https://go.justinangel.ai/deck |