Hacker News new | ask | show | jobs
by moralestapia 2318 days ago
>also play a huge role in automatic differentiation

Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.
2 comments
spacedome 2317 days ago
In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.
link
alexkcd 2317 days ago
Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff...

Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/

Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.

link