[a9h74j]

Can the training be parallelized in a manner similar to SETI-at-home?

[nodja]

Yes, hivemind trained a gpt 6B model like this.

General model training https://github.com/learning-at-home/hivemind

Stable diffusion specific https://github.com/chavinlo/distributed-diffusion

Inference only stable diffusion https://stablehorde.net/