[XorNot]

No I'd say it's that people are very bad at knowing what they want, and worse at knowing how to get it.

While it might be "moving the goal posts" the issue is that the goal posts were arbitrary to start with. In the context of the metaphor we put them on the field so there could be a game, despite the outcome literally not mattering anywhere else.

This isn't limited to AI: anyone dealing with customers knows that the worst thing you can do is take what the customer says their problem is at face value, replete with the proposed solution. What the customer knows is they have a problem, but it's very unlikely they want the solution they think they do.