If you have a compact one that fits in a bag, you can bring it fine on most intercity trains (though ideally you'd want to check before with the particular line). In local transit it's mostly frowned upon, especially in denser urban areas and busy lines. But you can usually always find some form of accommodation accessible by bike along the route, wherever you're going.
Folding bikes are much more common there, which helps a bit. You can go to a bike shop or a used-stuff-store ("recycle shop") and pick one up for $100-ish. Not going to be "nice" at that price, but it'll be functional.