[ninjin]

I think it is a bit unfair to say “I assume it was simply because they wanted to get Chris Lattner, and Swift was his baby”. Swift is an interesting programming language in its own right – and I am saying that as someone coding pretty much exclusively in Julia – and as I have stated before “If you are sitting on a team deeply familiar and passionate about a language – Swift – what kind of managerial fool would not let them take a stab at it? Especially with Lattner’s excellent track record” [1].

[1]: https://news.ycombinator.com/item?id=19717815

Only yesterday I gave a lecture to my cohort of MSc students on precisely this topic; there is history going back to the 60s, implementations alive since the 80s, and so much development over the last five years. As someone that cares deeply about the science (and as an engineer at heart), I simply can not be too sad that Googled picked up Lattner et al., poured money over them, and asked them to push the envelope of what is possible with differential programming. After all, what will stop us from lifting over advances to Julia et al. later down the line? Sure, I can wish that it would have been Bezanson et al. and not Lattner et al. that got the money poured over them, but that feels very petty. Let the “best” language win, and I still feel the same way that I felt in 2018 when Swift first planted their flag: “Swift has a non-existing scientific computing community […] they will have to build it entirely from scratch and community building is difficult. […] My decision to side with Julia is partially to stay my own course, partially a preference for ‘the bazaar’ development model, and partially because I have a hunch that Julia has a better chance to capture the scientific computing community as a whole which is likely to yield benefits down the line”.

[2]: https://news.ycombinator.com/item?id=16939525

[high_derivative]

I disagree, I don't think it's unfair to say at all. It is simply mismanagement to input resources into a language rework that is dead on arrival, community wise. This is of course par for the course for big tech research orgs where big names get a lot of free rein, but that does not mean it's not a strategic mistake here.

This is simply about focus as an org, and this is the reason why PyTorch is getting so popular.

There seems to be a massive lack of focus and direction in the TF org, too many egos wanting to put their stamp all over the APIs and subsystems (tf.keras anyone?).

TensorFlow eager with autograph or Pytorch solve all differentiation problems as far as researchers and practitioners are concerned.

[marmaduke]

> It is simply mismanagement to input resources into a language rework that is dead on arrival, community wise

how exactly might other features have a community of users prior to the feature being implemented?

> TensorFlow eager with autograph or Pytorch solve all differentiation problems as far as researchers and practitioners are concerned

I think this is a pretty narrow view of the world. From autograd to Stan to the cornucopia of implementations in Julia it's worth considering not everyone's going to be able to shoehorn their problem into the TF/PyTorch way of doing things.

[ninjin]

First of all, excellent points and thank you for the perspective on the direction of TensorFlow development. I usually settle for “It is opaque”, because that is as much as I know at this stage. Looking at all of your points, I agree with almost all of them. I would even add that I think a major reason for TensorFlow’s initial success was the fact that the machine learning community at large preferred (and was more familiar with) the overall Python echo system over the lackluster one in Lua land. However, I still feel that it is unfair to call Swift “dead on arrival” as I can imagine a future where its ecosystem becomes superior to that of Python – I would bet against it, as my comment history would suggest, but I can imagine it with some reasonably larger than zero probability.

Lastly, I would somewhat object to your statement that “TensorFlow eager with [AutoGraph] or [PyTorch] solve all differentiation problems as far as researchers and practitioners are concerned”. Yes, this is a very true statement at this exact point in time. Every single one of my PhD students prefer PyTorch over anything else on the market and I support them in their decision to pick the tool they see as best to accomplish their goals. However, my experience tells me that once you give researchers more powerful tools to express their models, they will find interesting ways to use that increase in expressive power to push the envelope in terms of what is possible. So, yes, as far as researchers and practitioners are currently concerned, what we have is sufficient, but what about the models of the 2020s? I am not so sure.