Not exactly. What you heat is the inside of your rooms, and a significant part of that heat can be felt as infrared radiation. When you ventilate in a quick and intensive manner, you exchange the air, which becomes cold for a few minutes but you do not cool down the room - because that takes longer. And the cold air has very little mass, so it is easy to re-heat again, and will be warmed by the walls etc.
The energy stored in your walls is much higher than the energy stored in the air. Swapping out all the air for cold one will reduce the total energy stored in your house very little. The walls will reheat the air. Same with letting AC air escape for a minute or two.