The grey part of the icons make arrow-points that "point" in the opposite direction of the direction that is communication by the actual arrow-shaped icons, the latter being the actual direction required to produce the correct action.
The two points in the icons provide two exactly opposite messages, and the wrong one is larger than the correct one, thus GP's confusion.
You can indeed reason it out. That doesn't mean it's very good though.
You pretty much have to have a very specific image of a d-pad in mind. It helps that that d-pad is also the one in your hands, but you shouldn't have to look at the controller, or have its detailed look memorized, to figure it out.
Fundamentally the fact that there’s anything to think about is an abject failure of why they put the icons there in the first place.
I think what happened here is obvious: there’s a very longstanding software design pattern that was broken by a perfectly sensible hardware controller design. The solution is likely to just stop using that software pattern. Make analogous icons or whatnot.
The two points in the icons provide two exactly opposite messages, and the wrong one is larger than the correct one, thus GP's confusion.