I wouldn't work where I'm paid more unless I needed the money. I would rather work with a great team that I get along with, and love the product I work with.
When I lived in Australia, I did some contracting then decided to go back to full time. I narrowed my choices down to 2 jobs, one at 120k, and one at 80k.
In the end I picked the 80k one because during the interview they asked me to come in for the day and do a 'test' for a few hours, which was building a small project, then I got to sit with a couple of the members of the team and take them through it. I had a LOT of fun (more fun than I imagined) and ultimately decided to go there.
One of the best jobs I've had.
Obviously between 2 great jobs then the 20k would be the tipping point. But money isn't the sole reason I work. I love what I do for a living.
But that's all else being equal. A company with a toxic culture or a toxic reputation is not equal to one with a world-class culture and sterling reputation.
All of the stuff you mentioned is still part of "compensation".