[earlyadopter2]

Shouldn’t the priors be updated and improved each step to be closer to a good prior, and that’s why inaccurate priors may be acceptable? (Do BNNs not iteratively update the (next step’s) prior with the previous step’s posterior?) I haven’t worked on BNNs, but since Bayesian are always talking about updating their priors I thought this would be the case.

[1e-9]

You are describing a recursive Bayesian approach, which can have significant computational and storage advantages for filtering (for example, Kalman filters). For this to work well, the prior must be able to adequately represent the learning of the posterior, which may be practical with a self-conjugate prior or a Monte Carlo approximation such as what particle filters use. In practice, for nontrivial machine learning applications, self-conjugate distributions rarely model the problem well and good approximations of the posterior into a concise prior are rarely practical.