Hacker News new | ask | show | jobs
by amelius 2918 days ago
> Kind of reminds me of assembly programmers that thought C wasn't for l33t 10xx pwner programmers.

The problem with TensorFlow is mainly that you, as a user, have to build a data-dependency graph. This is something a C compiler can do very well, but Python is not so suitable for that.

So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their own language, where the whole data-flow graph determination could be implicit, and not a concern for the programmer.

However, computing a data-flow graph as-you-go (by the library, not the user), like (I think) is done in some libraries, is quite a good approach, since the overhead is quite small (percentage-wise) compared to the large tensor operations that can be performed in highly optimized code.
1 comments
sanxiyn 2918 days ago
> So, in my view, TensorFlow chose the wrong substrate for their "more efficient" library. Instead, they should have developed their own language, where the whole data-flow graph determination could be implicit, and not a concern for the programmer.

You just described Swift for TensorFlow.

link
amelius 2918 days ago
Well, apparently :) They say: We believe that machine learning tools are so important that they deserve a first-class language and a compiler.

However, I'd like to see some numbers on how more efficient it is to build a graph in advance, given that the lion's share of the computations will be in tensor math anyway (which can be heavily optimized, and is independent of the graph).

link
liuliu 2918 days ago
The problem is that if you build the graph as-you-go, dataflow graph optimizations cannot be done efficiently (some high level optimizations such as data layout optimization, automatic data / model parallelism etc.). Swift can do all these because the compiler can extract the graph out ahead of time.
link
codetrotter 2918 days ago
> You just described Swift for TensorFlow

This one? https://www.tensorflow.org/api_docs/swift/

link
SalimoS 2918 days ago
isn't work in progress ? https://github.com/tensorflow/swift/blob/master/docs/WhySwif...
link