I found that not having a driver's license tremendously limited some of my opportunities for recreation and travel. I drive way more when I'm on vacation than I do when I'm home.
If you're into things that requires you to be able to drive, sure. I prefer spending my times in cities, and I like walking. The combination means I've never felt constrained even when visiting the US. On the rare occasion where I want to go somewhere where it's a genuine challenge, I can justify blowing money on getting someone to drive me.