The CPU probably works on just one endianess and convert the data format when reading from memory. The overhead is on kepping track when to do it. But Im speculating, havent looked into this.