[improbable22]

> It's difficult to find usages for autodiff

I guess there are lots of uses in optimisation problems, and in sampling algorithms for statistics. I don't know how easy it will be to sell Swift to people who now use Stan or Stata (or R) and don't think of themselves as programmers.

> In a language with extensible syntax(ex: proc macros), this would sit in a library.

And this would allow easier iteration of different designs.

[Scipio_Afri]

Does autodiff exist in R? Then why not use R since it seems it also has much ML algo support due to its focus on statistics - its used by many statisticians.

Is the idea here that Swift is a more approachable language and thus this is to lower the barrier of entry to TF?

[6d65]

I think the main selling points of Swift to tensorflow are:

* Speed, ML learning pipelines are often bottlenecked by the data load and transformation. TF had new mechanisms (the last I've seen was TF.data). But a language compiled to native is much more flexible in that regard.

* Type safety. Sometimes issues with the models can pop long after they have been running. The hope is that typed API's will show simple errors at compile times

* Auto differentiation built into the language. If I'm not mistaken, this is more powerful than backpropagation in TF, wich also has autodiff. The idea is that this would allow for more custom models without a performance penalty. My knowledge here is limited, since it's been over 2 years since I've implemented back propagation. I've successfully forgotten most of the things I knew about ML/DL.

I don't have any experience with R, but from what I've heard, it was known to be slow. But that might have changed or I may have misunderstood the situation.

[socialdemocrat]

Does not need to be built into the language. Julia does it great without special built in support. But Julia is an extremely flexible language.

Number types are first class which is a big part of it and you can add extra custom passes to the JIT compiler in regular library code.

[socialdemocrat]

You cannot really do autodiff in a slow language. I mean you can but nobody wants to run large machine learning training algorithms on a slow language like Python or R.

You could write autodiff in say C++ but it is a user unfriendly language not well suited for machine learning and scientific computing.

Swift is a nicer high level language you can do autodiff in. But honestly I don’t see the point with Swift either.

Julia already does AutoDiff extremely well and outperforms Swift and pretty much everybody else.