I would add possibly polluted or adulterated dairy. Raw milk (anecdote) may not cause the same issues as processed dairy.
Also, I would add that most processed dairy (besides just plain old milk) is mixed with many other detrimental ingredients. So there may be a cause/effect relationship overlap with other foods.
I can't speak for other people's cases but in my case it's definitely not the dairy. The only time dairy ever had an effect on me was pre-elementary school when eating too many egg started giving me pimples, so I cut down on that and that fixed it. Never had a problem with it again.
Also, I would add that most processed dairy (besides just plain old milk) is mixed with many other detrimental ingredients. So there may be a cause/effect relationship overlap with other foods.