[0-_-0]

Well, it should be possible to crowdsource training a FOSS version, right? There should be a SETI-at-home for training neural networks. I would donate some GPU power for sure.

SETI@home achieved 50 times the computing power of the world's largest supercomputer [0], so it might actually be the only way to train the future GPT4 or GPT5.

[0]: https://en.wikipedia.org/wiki/SETI@home#Statistics

[hgial]

At the moment crowd-sourcing ML training doesn't work very well because updates to the model's weights have to be shared between all of the nodes in the computation periodically. Giant models tend to be trained on clusters with very fast interconnects for this reason.

[devetec]

Distributed GPT training doesn't really work (sadly). Maybe someone will get a good solution to that, though.