Aren't these delis? At least when I used to live in Brooklyn we used to call them that. Often they had signs "Deli and Grocery" or something like that.
Milkbar in Australia because they used to sell milkshakes and the convenience store aspect was secondary. Over time the convenience store part took over but the name stuck.