This is a highly unusual situation (not to mention dangerous in the winter) to turn off your heat. Certainly not something you would worry about when trying to make a babysitter cozy.
Not all the way off, but there is a wide margin between comfortable and safe where it is generally cost-effective to let the temperature rest when you are not home.