[SwellJoe]

The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer

[alansaber]

Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).

[satvikpendem]

Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.

[vlian2088]

BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.

[kamranjon]

Where do these estimates come from?

[cyanydeez]

a dark anal cavity somewhere next to marketing.