[yogthos]

This is precisely why we need more projects like this https://github.com/bigscience-workshop/petals

There's no fundamental reason why models couldn't be developed and trained using community efforts. It might not be as fast and efficient, but it's definitely possible.

[woctordho]

See the recent development of DiLoCo at Nous Research and Prime Intellect.

[yogthos]

oh that's very cool, exactly the kind of stuff we need