Hacker News new | ask | show | jobs
by big-chungus4 751 days ago
In `if i > x`, derivative with respect to x is mathematically 0 at all points. DiscoGrad gives you a useful smooth approximation that is not 0 and lets the function learn those conditional values.