You are conflating tokens with embeddings.
Tokens fit in a single word, modern gpt uses a vocabulary with 200k possible values, which would fit into 18 bits.
Have a good one