personally I hate the idea of always needing a phone with me -- i leave mine at home most of the time just using my wallet and nfc transit card to get around
Difficult to overstate how much you're in an extreme, extreme minority position if you own a smartphone and don't carry it around pretty much everywhere.
that's one of the pirmary things for me too even as a guy with usually large pockets i find it annoying to walk around with them in my pocket espcially in the summer
plus on the bus or train I usually use a book or my kindle so it's easy to forget the phone