[DagsEoress]

Is any of this usable on hosted GPU notebooks (e.g. paperspace)?

I'm currently diving into Machine Learning using Python + Scikit-learn, and I'd love to one day replace Python with Ruby. But looking at the current ML ecosystem I don't see that happening. Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)?

[thibaut_barrere]

Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it, including:

- https://github.com/elixir-nx/axon Nx-powered Neural Networks

- https://github.com/elixir-nx/nx Multi-dimensional arrays (tensors) and numerical definitions for Elixir

- https://github.com/elixir-nx/scholar Traditional machine learning on top of Nx

- https://github.com/elixir-nx/bumblebee Pre-trained Neural Network models in Axon (+ Models integration)

- https://github.com/elixir-explorer/explorer Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir

- https://fly.io/blog/rethinking-serverless-with-flame/ (for offloading large work to remote containers)

- https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx

And of course Livebook (https://livebook.dev)

Old but interesting video https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2, Stable Diffusion, and more in Elixir)

A talk on the upcoming Elixir conference (https://2024.elixirconf.com/schedule/#schedules) is actually titled "Livebook in the cloud: GPUs and clustered workflows in seconds".

[DagsEoress]

Very interesting, thanks for sharing! I didn't know Elixir was so invested in Machine Learning. I have a background in Erlang so I'll definitely dive in. Do you know of any individuals / companies that have had success with Elixir+ML in production?

[thibaut_barrere]

You welcome. Afaik it has been a long-time bet done by José Valim (former Rubyist & author of e.g. Devise) and others, this is not something that is going away anytime.

Each library has been building on top of the previous libraries & abstractions (including transpiling Elixir instructions into GPU code, see "defn" etc).

Since you mention Erlang, there is even a Machine Learning Working Group at https://erlef.org/wg/machine-learning.

The most iconic & advertised case I'm aware of is the work done at Amplified https://www.amplified.ai ; this has been the topic of a Keynote at ElixirConf EU this year, which you can find at https://www.youtube.com/watch?v=5FlZHkc4Mq4.

I am also starting to use ML + Elixir in production and I'm aware of other individuals doing so.

I do not have a registry of companies doing so, but we're seeing more and more experienced ML practitioners mentioning they are coming from Python and willing to try something different (e.g. https://elixirforum.com/t/data-science-and-machine-learning-... and other posts on Elixir Forum).

Hope this helps!

[nextos]

I had looked into Elixir/Erlang + ML before, and I keep revisiting it for Python alternatives.

How developed is Elixir/Erlang in this area? What are the key advantages?

[xipho]

Perhaps not quite what you're looking for but Ankane does (a lot) of great work, e.g. https://github.com/ankane/torch.rb.

[michaelmior]

It's certainly possible, but the provider would need to add support for the kernel. That is, I'm pretty sure you can't install a new kernel in these environments yourself.

[qsort]

Nbviewer is just a service that lets you host notebooks, there is no computation there.

> Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)?

XGBoost/LightGBM have a C API and can be used from pretty much anything, deployment is not a problem. Practically building models is more about dealing with data, the ecosystem tends to revolve around Python and R for that reason.

[crabbone]

R and Julia have had integration with Jupyter for a long time.

Even so, trying to avoid Python in the world of Jupyter will put you in a very tough spot. In general, doesn't matter how much you dislike it, there's no real way around it. You'll have to face it in some capacity whether you like it or not.