[amkkma]

I believe we are currently at pytorch parity (and sometimes better) for speed. Memory usage depends....And this is without the extensive upcoming compiler improvements.

The reason for the lag is that Julia has been focusing on general composable compiler, codegen and metaprogramming infrastructure which isn't domain specific, whereas pytorch and friends has been putting lots of dev money into c++ ML focused optimizers.

Once the new compiler stuff is in place, it would be relatively trivial to write such optimizations, in user space, in pure Julia. Then exceeding that would be fairly simple also, plus things like static analysis of array shapes

[Royi]

Do you have updated comparison with some data of performance and memory vs. PyTorch / TF?

[amkkma]

I don't have a comprehensive suite of numbers at the moment (which is why I qualified it with "I believe"). My tentative conclusion is based on experience by a flux maintainer benchmarking forwards and backwards passes compared to pytorch, along with reports like this: https://discourse.julialang.org/t/flux-came-a-long-way/62288