[epistasis]

As somebody who has used belief propagation a lot, before the current neural network renaissance, I'm not yet optimistic of that. (However, I have been wrong enough in the past to be somewhat humble about my ability to predict these things!)

As I see it, the current big machine learning models use data that can't fit well on a single machine, or training regimes that can't fit onto a single machine, but the end model still fits on a single GPU or box.

What I understand them to be advocating for here, is a model that is bigger than can fit on a single chip, that can use belief propagation to scale to that number of variables/size. I don't yet know what sort of applications that could address, so that's my primary point of skepticism about this sort of huge model being developed. BUT, my lack of imagination is not much of an argument on its own, it's merely the absence of an argument, not an actual argument against such large models being useful.