You can think of it as a contraction of, 'despite that having been said'.
I've always taken the phrase "that being said" to be a variation of the phrase "that said" that isn't as assertive of what was said, relegating it to only "having been said" rather than set in stone
I've always taken the phrase "that being said" to be a variation of the phrase "that said" that isn't as assertive of what was said, relegating it to only "having been said" rather than set in stone