|
|
|
|
|
|
by nylonstrung
23 days ago
|
|
|
Even though Python code may use more characters/LoC than say Rust in text form, it's not necessarily more token dense because LLM tokenizers are good at "compressing" its English keywords In contrast, langs with symbol-heavy syntax (ALP as extreme example) use fewer characters but don't tokenize well in practice so aren't as efficient as one would think |
|
|