Well, but maybe on the other side of the phone there is another person, relationship with whom I value more? It's not like smartphones are all (or even mostly) about status updates.
That's what I'd do if you weren't interrupting and pressing me for face-to-face conversation. Also, it's you who invaded my space, so why don't you go somewhere else?
The point is - in my opinion, it's equally rude to pull out a smartphone in the middle of a face-to-face conversation, as is to approach someone "fiddling with their phone" and expect undivided attention.