I use to live next to a highway with the closets exit being a few miles away and I would constantly get drivers on the highway as my closest driver. I always had to cancel since they were actually ~15 minutes away, not 3-5.
I have found that Uber's estimates get better with time. When Uber was newer in my city, it used to give ridiculous underestimates. They've become far more realistic now.