Very true. Depending on the modifications (like electrical), insurance could deny claims if they find unapproved/unpermitted work, even if it wasn't the cause, or the cause was unknown (depends on the policy).
Around here, if you don’t want to buy a house with un-permitted work, it’s no problem; there’s another buyer standing right behind you (often literally) who will.