Yes that's absolutely the case. So it just depends on how good the cooling system is. For a phone, it has no active cooling system. For an electric vehicle, almost all manufacturers (except Nissan Leaf) have adopted active cooling and you can expect them to do a good job activating adequate cooling for the battery.
It can affect battery longevity. If you charge lithium ion at cold enough temperatures, you get lithium plating which rapidly degrades capacity and can lead to internal shorts and fires.