There's no explanation as to how they achieved that speed up :( it would have been better if they also wrote a post on that

Something tells me they are calculating utilization in a non-standard way. While I have no doubt there is room for clever pipelining of data, to say you cut out all memory transfer overhead sounds ridiculous.

If they did, Zuckerberg will cut them a $5+ billion check today.

Ok nice clickbaiting, post your 3x speed up part and we will talk

Posts like these should have an [inbound] tag or something like that.

Sounds like some basic plumbing that an LLM should be able to do.