[ZunarJ5]

https://the-decoder.com/gpt-4-architecture-datasets-costs-an...

Not op, but this is where a cheeky google got me.

[1vuio0pswjnm7]

"The idea is nearly 30 years old and has been used for large language models before, such as Google's Switch Transformer."

Innovation! :)