[hdespiritu]

Your question "Has anyone converted stuff like gradient descent to set theory?" doesn't make sense from the perspective that gradient descent uses differential calculus to find min/max points of an objective function and differential calculus requires a lot of additional assumptions on top of set theory.

That being said, current deep learning libraries such as JAX and Pytorch use automatic differentiation to efficiently compute partial derivatives used for optimization algorithms such as gradient descent and it's not clear to me what the level of effort would be to convert that to something that could run efficiently in SQL?

[zackmorris]

Thank you, I knew that derivatives were used in gradient descent, but automatic differentiation is new to me. The Wikipedia article is fairly opaque compared to what I learned in school, but this tidbit stood out:

https://en.wikipedia.org/wiki/Automatic_differentiation#Impl...

Source code transformation (SCT): the compiler processes source code so that the derivatives are calculated alongside each instruction.

Operator overloading (OO): operators are overridden so that derivatives are calculated for numbers and vectors.

Based on the state of software these days, I'm guessing that OO (the "bare hands" method) is what's mainstream. It would be far better IMHO to use SCT, since it's a universal solution that doesn't require manually refactoring programs.

But stuff like SCT might be considered metaprogramming, which seems to have fallen out of fashion. I grew up with C++ macros and templates, so I feel that this is somewhat tragic, although readability and collaboration are much better today. A modern example might be something like aspect-oriented programming (AOP):

https://en.wikipedia.org/wiki/Aspect-oriented_programming

I once used the AOP library AspectJ to trace a Java app's execution, since Java made the (unfortunate) choice to focus on objects rather than functions, which makes it generally a poor fit for data processing, due to its high use of mutable state within objects (mutable state is what limits most object-oriented projects to around 1 million lines). Meaning that I couldn't remember the program's context as I was stepping through it, and had to analyze traces instead. AspectJ allows one to hook into the code without modifying it, sort of like a debugger, so that stuff like function calls and variable mutations can be watched:

https://en.wikipedia.org/wiki/AspectJ

https://www.eclipse.org/aspectj/doc/released/progguide/index...

Looks like this might still be an open problem in Python:

https://stackoverflow.com/questions/12356713/aspect-oriented...

https://docs.spring.io/spring-python/1.2.x/sphinx/html/aop.h...

But it seems like SQL would be a good candidate for AOP:

https://stackoverflow.com/questions/12271588/aspect-oriented...

https://technology.amis.nl/it/aspect-oriented-programming-ao...

If we had that, maybe we could automatically generate derivatives for the set operations. Then either access them as variables in stored procedures, or possibly as something like views or via metadata stored somewhere like MySQL's INFORMATION_SCHEMA.

I don't really know, but maybe these breadcrumbs could be helpful.