About 18 months ago I removed support for iOS 3 and we got a huge pushback from existing users. I spent another month adding back iOS 3 support. I am sure things must be different now.
in other products which are very very non-tech consumer focused, i've seen almost zero iOS 3 deployment. You should feel incredibly comfortable deploying as iOS 5+ only at this point.