Is it good hygiene to delete these records once they've served their purpose (i.e. validating control of the domain in question)? Or do these records server more than that use?
I think it's more about the opposite. If you are the new owner, you don't want the old owner to continue to have access to features connected to your new domain.
So as a new owner you would want to remove the tokens.