[wppick]

If you don't need the isolation of of k8s then don't forget about erlang, which is another option to scale up to 1 million functions. Obviously k8s containers (which are fundamentally just isolated processes) and erlang processes are not interchangeable things, but when thinking about needing in the order of millions of processes erlang is pretty good prior art

[theptip]

This is 1m nodes, you typically run tens or hundreds of pods per node, each with one or more containers. So more like 100m+ functions if I follow the Erlang analogy correctly?

[reactordev]

This is not analogous. It’s just someone beating the Erlang drum. You can’t PyTorch in Erlang.

[wppick]

Kubernetes is way heavier than Erlang’s lightweight processes, so for millions of tasks at scale, a middle-ground solution could blend Erlang’s concurrency efficiency with k8s’ orchestration power, dodging containers’ overhead while keeping flexibility for diverse workloads. That's if you don't actually need the strict isolation of pods/containers and you're just trying to run something at massive scale. I don't get why so many people want to run everything as heavy container processes or pods vs coming up with a better solution. The point is we don't have to fit every problem into the shoe called kubernetes if it doesn't seem to fit, and we should look at other ways to spin up millions of processes

[victorbjorklund]

There are similar libraies in Elixir. Is the ecosystem for ML as developed as for python? Nope, but not every ML project needs the most obscure libraries etc.

(For the record I don't really see Erlang clusters as a replacement for k8s)

[reactordev]

You aren’t going to teach a research scientist Erlang in spite of Python and R. You aren’t going to win that fight, ever.

[victorbjorklund]

Oh totally, that is not anyone's goal I think. The goal is more about enabling teams already running Elixir to also do their ML stuff in Elixir or teams that needs to do something that would benefit from the BEAM actor model but also need to do some AI stuff to do it all in Elixir. Lots of neural networks created in Python or R can then run in Elixir.

[fcarraldo]

You can with Pyrlang or whatever other cursed implementation of Python on top of the Erlang VM you’d prefer.

[fcarraldo]

I don’t think there are very many k8s clusters running 100s of pods per node. The default maximum is 110. You can, of course, scale beyond this, but you’ll run into etcd performance issues, IP space issues, max connection, IOPS and networking limitations for most use cases.

At 1M nodes I’d still expect an average of a dozen or so pods per node.

[sally_glance]

Agree this is a consideration if your only workload is an existing or greenfield ErlangVM-compatible project.

From what I know basically everyone approaching this scale with k8s has different problems to solve, namely multi-tenancy (shared hosting/internal plattform providers) and compatibility with legacy or standard software.