Cities have plenty of safe places to fly (over streets for instance; or over roofs) but the final contact issue is real. How about: don't come down until signaled by a smartphone, then home in on the gps location.
Over streets is one of the least safe place to fly - imagine a 10km brick falling at 40km/h into incoming traffic or a group of pedestrians. As for the final contact, your average quadcopter will have like 10-20 minutes of flight time tops; it can't just wait for you to signal it - being even a minute late could mean the drone won't make it back to a charging station.