It seems like you might really be asking about a meltdown scenario.
The reaction can be slowed with control rods, which stops/minimizes the heat from being generated. The previously generated heat still needs to be handled, however (by evaporation).
However, those safety systems are obviously designed expressly to prevent such a disaster. For example, control rod systems are often designed to be fall (via gravity) into the reactor in the event power fails. These failsafe systems are typically very reliable, in the absence of other external events (such as the flooding and earthquake that took place in Fukashima.)
The control rods don't stop decay heat from being produced. That's what happened with Fukushima.
Also, nuclear poisons (neutron absorbers) build up after the reactor is shut down. After the control rods are withdrawn it takes a few days for the poisons to be burned up and the reactor can resume power production. I think they call that poisoning out the reactor.
When the Northeast power grid went down down in 2003, the Bruce Nuclear Power Development was almost entirely kicked off the grid. It was able to keep ticking over at a few percent of power output though because of a feeder line that went North. This prevented the reactors from poisoning out, and allowed them to come back faster than if they had poisoned out.
Instead of that energy being directed to the turbines to convert the thermal energy into rotational energy (and thus it to electricity), the reactor begins cooling from the primary reactions (but not completely since secondary reactions/decay still are going) since there is no reaction heat (reactor idled). All of that excess residual heat contained in the mass of the fuel and reactor water is carried away by the cooling water loop, and either out to the cooling tower where it evaporates and releases its energy, or through the next stages of the loop into the cooling supply, most likely a neighboring river or lake.
In general, any sort of "boil water to steam, use steam to drive turbines" system can vent the hot steam from the boilers to the atmosphere. There's a big reserve tank of cold boiler feed water, to replace the water you're no longer getting back from the steam condensers (attached to the "OUT" steam pipe on the turbine). So the boilers will keep soaking up just as much heat, while the engineers scramble to reduce the heat coming in from the burning wood, or burning coal, or burning oil, or fissioning atoms, or whatever.
The reactor is turned down and rapidly stops producing that much heat. On site backups keep the pumps running to continue cooling the residual heat. There's a long slow tail as lower energy fusion chains tail off towards lead but the vast majority of the power comes down quickly.
I would assume the cooling loop is still running on generator and/or backup utility feed power, so the heat from the reactor would go into the water of the cooling loop, and then the heat would be rejected into the atmosphere when the cooling loop water evaporates in the cooling towers, with the leftover heat being discharged into the river/lake/cooling water source.
The amount of heat that can be dissipated by evaporating water is kind of incredible.