I think is because they're going for a full machine learning approach, the neural network tries to learn the best solution but is unable to receive simple instructions like "go in a straight line"
They're not going for a full ML approach. That's just more Elon marketing language. Tesla reverse engineers have gotten their hands on the FSD beta and inspected the internals - they use ML for perception and scene understanding, but all of that information feeds into traditional robotics prediction, planning, and pathfinding algorithms.