the CPU temp can do that, but your typical desktop air tower has a lot of thermal capacity. if it took 5 seconds for the fan to react, that wouldn't be a big issue.
Desktop case fans also have a fair bit of inertia. They take at least a second or two to ramp up to full speed, and longer to coast back down to their minimum speed.