[Narew]

Keras was already that some years ago. It supported tensorflow, theano, mxnet if my memory is right. And then they ditched everything for tensorflow. At the time it was really hard to use keras without calling backend directly for lots of optimisation, unsupported feature on they API etc... This make the use of Keras not agnostic at all. What's different now ?

[minimaxir]

> What's different now ?

PyTorch adoption: back when Keras went hard into TensorFlow in 2018, both TF and PyTorch adoption were about the same with TF having a bit more popularity. Now, most of the papers and models released are PyTorch-first.

[Narew]

Yes I understand why they do the move (they want to attract pytorch user). What's the benefit for the user instead of directly using pytorch for example ? I see we can maybe use tpu by switching to jax etc...

PS: sorry I'm a bit salty by my user experience of Keras.

[bootsmann]

Pytorch is an animal by itself when you try to put it into production. They have started addressing it with torch 2.0 but it still has lengths to go. With this you can switch to TFserve if you have usual architecture.

[cygn]

You can just use Triton which is basically TFserve for Tensorflow, Pytorch, Onnx and more.

[albertzeyer]

Can you explain that?

My understand of Triton is more that this is an alternative to CUDA, but instead you write it directly in Python, and on a slightly higher-level, and it does a lot of optimizations automatically. So basically: Python -> Triton-IR -> LLVM-IR -> PTX.

https://openai.com/research/triton

[chillee]

It's confusing, there's OpenAI Triton (what you're thinking of) and Nvidia Triton server (a different thing).

[m00x]

Tensorflow has some advantages, like being able to use tf-lite for embedded devices. JAX is amazing on the TPU, which AFAIK pytorch doesn't have great support for.

I assume most people will still research in PyTorch, but then move it over the Keras for production models if they need multi-platform support.

[minimaxir]

Keras has a cleaner API compared to base PyTorch, especially if you want to use the Sequential construction as demoed in the post.

[sva_]

How so? You can use torch.nn.Sequential pretty much equivalently?

https://pytorch.org/docs/stable/generated/torch.nn.Sequentia...

[minimaxir]

Huh, didn't realize base PyTorch had an equivalent Sequential API.

The point about the better API overall still stands (notably including the actual training part, as base PyTorch requires you to implement your own loop)

[albertzeyer]

Some people would say that this is an advantage of PyTorch, that it is very easy to write and play around with your own loop.

But if you don't want that, if you want to go a bit higher level w.r.t. training, there is e.g. PyTorch Lightning on top of PyTorch.

If you say the API is better in Keras, can you give any examples? They look kind of similar to me. Keras looks a bit more complicated, esp when looking at the internals (which I tend to do often, to better understand what's actually happening), which is a negative point.

[mochomocha]

Adding keras abstractions on top of pytorch seems like negative value-added to me. It only barely abstracts more the easy/mundane stuff, while creating yet another layer of indirection between ideas and machine code in the current Rube-Goldberg mess that people call modern ML.

Keras has always been "lipstick on a pig for tensorflow". Its value beyond that seems tenuous at best.