[DylanDmitri]

I wrote an "anything-regressor" in SymPy, where you provide an equation (like y=mx+b) and a dataset of x's and y's, and it solves for constants to get a line of best fit. Super simple to implement and worked for any equation (quartic, logistic, etc) and any dimension of input variables.

Eventual goal was trying to implement something like XGBoost, but by applying successive regression equations to the residuals instead of successive decision trees. E.g. figure out that a sin wave best fits the initial data, and then a linear best explains the remaining difference.

Worked pretty well on toy data sets, but was much too slow to scale to anything real. Still think the idea has promise for producing human interpretable ML models.

[nestorD]

This sounds like symbolic regression (where the space of possible equations is usually searched using genetic algorithm).