[ablatt89]

And I guess we need to buy into their decentralized network so that this AGI compute can be done on their "clearly unbiased" crypto software that's running on the computers of all those who bought into the hype of their crypto coin. And then every participant who worships the feet of the crypto lords can get a unique NFT of their crypto lord. Maybe one person will get an NFT with the crypto CEO having silly green glasses, or another person who helped the CEO become a billionaire will get an NFT with the CEO having a pink mohawk. The endless possibilities of crypto and AGI :-)

[theGnuMe]

It's certainly a potential path. It's folding at home model + crypto. May offset electricity costs if you can sell the tokens...

I am interested in understanding why more training flops vs say more parameters. Surely bigger models perform better but perhaps there is a limit?

I don't quite understand how distributed subtrees will work.

[ablatt89]

I'm not an expert in ML theory, but from my perspective, I think more compute would cause better fitting for a model, but possibly overfitting if more parameters are not added. For some problem space, adding additional parameters introduces another degree of freedom, which should allow a larger domain for the inputs mapping to outputs (more answers to questions). And if we define AGI as a network that can answer questions for n > 2 domains (e.g. can we do image classification and a chat bot and synthesize them into a coherent system passes a Turing test), then more parameters makes sense to increase the range of outputs.

Interestingly, I don't think it's clear on how parameters in the network and compute would clearly find a domain within a combined problem space, where the mapping from question to answer will give sensible results. It seems like we need more tools to extend ML.

[theGnuMe]

We know more parameters improves generalization so that is why it's unclear to me what they mean by more training flops.

[ablatt89]

One new parameter = 1 new node on the crpto nutter's network to do additional compute for the parameter? Lol I can see many ways these crypto nutters can attempt to weakly justify purchasing into their network.