[bialamusic]

I have a question.. As all these GPU cores are just used as computational acceleration for a CPU code, how the fuck they claim to have and control thousands of GPUs running one model inside a datacenter? Are they just using too big arrays :) - it makes no sense to me. How many GPUs one CPU code may need? How many CPUs they have and How then these CPUs are running in parallel? Do they at all?????

[joajoa]

Hey, thank you for your question. In fact it is not one CPU program driving thousands of GPUs but rather the opposite: independent processes, each driving one GPU, and all running the same program on different data. The host process (CPU) is just moving the data to the GPU and driving the network. The tensor (sub)tiles are allocated to individual nodes.

I also want to be clear and I hope I was upfront enough about it in the post: I haven't validated any of this at scale myself. Phobos is a learning project, and the numbers in the post are single-GPU. This is my understanding of how the big clusters work, not something I've run as I do not have access to a large cluster.