I've done this for years, and it's great! The way I see it is that a phone is a tool that you should use to your benefit. The tool should therefore not call attention upon itself. A weird analogous example would be having a hammer that constantly told you to hammer some nails for no good reason.
iPod touch or like Android product that doesnt have a phone but still has access to mobile data would be killer. I know you could carry a hotspot with you but thats another device...