If performance matters, TLV doesn't cut it either, you need fixed-offset fields. Self-describing binary formats like CBOR, BSON, bencode, DER, UBJSON, etc fall between two stools, being less hackable than JSON and slower than schema-driven binary formats, which is why they are all moribund today.
Everything has JSON because everything has a web browser with JS these days. Calculation can change if browsers stop shipping or stop shipping with JS.