While watching a movie, look away - maybe. Get up and walk around and do chores - we always pause if needed. I think it's a matter of establishing that to watch a movie, one needs to set aside time and commit to focus just on it, but then it becomes yet another barrier.
Different story for TV shows which often are background though.
I don't think that's why. TVs without 3D are just cheaper, the early 3D tech just wasn't very good and took awhile to mature thus souring the market, 3D content was more expensive (or an extra expense, eg. buy the 3D and non-3D versions of a movie) and so people just went for the cheaper options overall. I've had an active 3D TV for 10+ years, and the 3D has not itself been a problem when I've watched with others.
The only time it's a problem is if someone currently experience a migraine is trying to watch, then they can get serious vertigo, but that's an issue caused by the migraine itself (visual auras and vertigo generally).
All of those reasons certainly contributed, but the reality is that most people don't watch movies at home the way they watch in a theater, where they dedicate 2+ hours to the experience with no distractions.
I do that, but I have to wait for everyone to go to bed first and then turn off my cell phone. Most people aren't willing to do that.
A lot of people wrote it off as unnecessary gimmick, I'd add that to list of reasons. VR 3D blows it out of water but then requires more effort to use.