Hacker News new | ask | show | jobs
by WithinReason 115 days ago
I made an explicit reverse pass (no autodiff), it was 8x faster in Python
3 comments

I made an explicit double-reverse pass (no code!), it was 80x faster in my head!
code here, it's just not interesting to look at:

https://news.ycombinator.com/item?id=47220542

tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.
Can you share a link?
https://www.ideone.com/VAz4Nn

Doesn't run inside IDEone due to the external download link, but you can copy&paste the code over