[elcritch]

Agreed, after writing that I realized that "bits" of training is a pretty poor metric. Especially in lossy NN as compared to normal computing. Likely researchers will be busy for decades defining and narrowing down the concepts in the field before useful values could be determined in terms of information theory.

A huge question I didn't even realize was "bits don't relate very directly to a NN ability to perform a task".