You skipped the "and pass information around as global state" part. It obviously depends on your language, but ideally an Exception object should be a specific class you can choose to catch or re-raise, and it should include a message. None of the calling state should be visible to you.
I give you that I skipped it. I skipped it intentionally to make a bit of hyperbole.
An exception is still essentially a goto that crosses stack frames. There is a whole class of common bugs, you will see a lot of talk of them in the C and C++ communities, where an early return or a throw at an unexpected time leads to resources not being cleaned up. A lot of good coding styles say avoid this stuff, it's dangerous, and they are not without cause. Whereas a goto cleanup block is often a lot more explicit and therefore can lead to more reliable code than a scheme that encourages not sweating details and not expecting failure at every step.