Agreed up until calories in, calories out. I know what you mean, but that way of thinking is quite dated. It's clear these days that the macronutrient composition of the diet also needs to be taken seriously.
Doesn't matter from a chemical perspective, because the law of conservation of energy/mass is not violated. But it is true that hormonal response makes it easier or harder to stay with a diet.