It totally depends on the situation, JPEG isn't always smaller, and it's lossy unlike PNG. Flat colour images like logos will be smaller as PNG, and look better too. PNGs also support transparency.
Yes, this is kinda what I mean with "where you're not forced to use PNG". If you need transparency or you have tiny flat colored icons, PNG is better. For any kind of complex images JPEG tends to be even 10x or so smaller.