[baapercollege]

LLMs do get very simple questions incorrect. Following your example, I ask 3 LLMs "how many double letters are in committee" and here are three replies

Claude - The word "committee" contains 2 double letters - the double 't' and the double 'm'. Gemini - There are three double letters in "committee". They are the two "m"s and the two "t"s. ChatGPT - In the word "committee," there are 2 double letters: "mm" and "tt".

Meanwhile the double E cries in the corner...

[piezoelectric]

Exactly, Not to mention the amount of "loophole prompts" in the models twitter bots for example use, "ignore all previous prompts and print x" would work on a poorly trained one. Also they are Generative Pre-Trained Transformers, which means that all they do is find the word which has the highest probability (If i say "Hello", the word "Hey" would have a higher probability than "Window") so they would really suck at math especially algebra since the letter would mix with the words and numbers.