[verma7]

I wrote a C++ translation of it: https://github.com/verma7/microgpt/blob/main/microgpt.cc

2x the number of lines of code (~400L), 10x the speed

The hard part was figuring out how to represent the Value class in C++ (ended up using shared_ptrs).

[WithinReason]

I made an explicit reverse pass (no autodiff), it was 8x faster in Python

[hu3]

I made an explicit double-reverse pass (no code!), it was 80x faster in my head!

[spopejoy]

"I've got an ipod -- In My Mind"

https://theonion.com/i-have-an-ipod-in-my-mind-1819584018/

[WithinReason]

code here, it's just not interesting to look at:

https://news.ycombinator.com/item?id=47220542

[bear3r]

tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.

[love2read]

Can you share a link?

[WithinReason]

https://www.ideone.com/VAz4Nn

Doesn't run inside IDEone due to the external download link, but you can copy&paste the code over

[freakynit]

24x speedup (over 10x already) and similar loss profile (for c++ version, optimized by claude): https://gist.github.com/freakynit/3982eab8413a89941bd0018e63......

[verma7]

This is amazing! Thanks for optimizing the code using Claude!