Carriers will still be able to track you via the cell towers you're connected to. I'm sure they can triangulate based upon signal strength, and that's strictly using your cellphone as a dumb phone.
Can we trust the GPS receiver to be powered down when we the OS tells us it's powered down? I know Android keeps listening for WiFi stations even if you tell it to turn off the antenna. Might it do the same thing with GPS?
It may help in regards to your exact location via GPS, but cell companies can still triangulate your location based off how strong your signal is to certain towers in the area and which towers you have connected to recently.