Hacker News new | ask | show | jobs
by ssivark 2079 days ago
Is this in essence the same as the relation between a differential equation and it’s adjoint equation, and how one could use the asking method to perform back propagation? (Eg: ref. Neural ODEs)
1 comments

NiLang requires every instruction being reversible, e.g. `SWAP` and `y += f(args...)`. The back-propagation is implemented on a limited instruction set. It is different with Neural ODE because neural ODE does not back propagate the program step by step. Some time reversible integrator can utilize the reversibility (with NiLang) to save memory. e.g. the leap frog integrator for symplectic systems.