[lolinder]

The point is that this isn't even really useful because it's not a minimum reproduction of the problem they're actually interested in.

LLMs are bad at counting no matter what size of context is provided. If you're going to formulate a thought experiment to illustrate how an LLM stops paying attention well before the context limit, it should be an example that LLMs are known to be good at in smaller context sizes. Otherwise you might be entertaining but you're also misleading.

[araghuvanshi]

Well LLMs are claimed to be good at math too, and yet they can't count. Same point with the long contexts. And our actual use case (insurance) does need it to do both.

My hope from this article is to help non-AI experts figure out when they need to design around a flaw versus believe what's marketed.

[famouswaffles]

>Well LLMs are claimed to be good at math too, and yet they can't count.

You're putting a lot of weight into counting. I don't know anyone who wants to use a LLM after hearing "good at math" for counting of all things. Algebra, Calculus, Statistics, hell I used Claude 3 for Special Relativity. Those are the things people will care about when you say math, not counting.

Look, just test your use case and report that lol.

[araghuvanshi]

Look man, Claude 3, GPT4 etc didn't work for my startup out of the box. I thought it would be helpful to tell others what I went through. Why hate on the truth?

[famouswaffles]

Test the LLM on what you want it to do not what you think it should be able to do before what you want it to do. It's not hard to understand here and I'm not the only one telling you this.

Your article would have been very helpful if you'd simply did that but you didn't so it's not.

[stevenhuang]

But LLMs are good at math, they just aren't good at arithmetic.

https://www.lesswrong.com/posts/qy5dF7bQcFjSKaW58/bad-at-ari...