I think it might be more intuitive as an explicit stream modifier: hex, decimal, etc. No reason to turn anything “off”, which is actually the activation of a diffferent mode.
0 for octal was a huge mistake. Most sane languages use 0o now, or just forget octal. How often do you use octal anyway? Like, once a year for file permissions?
Was the first thing that stood out to me as well. Toggled modes like this are also quite annoying when shuffling around sections of the file. Copy one hex too many and suddenly your numbers in another part are all wrong. And due to the toggle you'll have a hard time finding the place where it all went wrong.
As far as I could tell it's been more or less the only mode. At that point I'd probably get rid of it because global state (which modes are) has to be remembered at all times and in this case its invisible until you see output. The suggestion to use different operators or modifiers for a single numbers was a good one in that direction IMO.
Edit: What I mean is