Cmd+X is for cutting. You don't "cut" files. Cutting removes the original item, and puts it in a temporary location before pasting it in the new location. Moving files is not cutting. It's moving.
I get that that's intuitive for you but I think of these operations in text or image editors. If I copy and paste text it's now in two places. If I cut and paste it's moved from one place to the other.