[magicalhippo]

Note, I'm not into ML though I've dabbled with NNs as a teen (before deep learning and all that).

The weights scale the output values from the previous layer, and the weighted values are summed. So it seems to me, instead of having a high-precision weight scale a single output, if you cloned the node in the previous layer M times, you could still have sqrt(M) bits of precision with 1-bit weights (or M bits, my brain is in weekend mode).

Thus a larger network with lower-precision weights should have the ability to have approximately the same precision as a smaller network with high-precision weights.

The larger network has more interconnects though, so seems like it could allow for more interesting space to explore during training, leading to better results.

Then again, I could be entirely wrong.