[helloplanets]

You need to be pushing much more data in than you're getting out. 40k tokens of input can result in 400 actual quality tokens of output. Not giving enough input to work off of will result in regressed output.

It's basically like a funnel, which can also be used the other way around if the user is okay with quirky side effects. It feels like a lot of people are using the funnel the wrong way around and complaining that it's not working.

[mjburgess]

Sure, if you have a high-quality starting point and need refinement.

The issue is that the vast majority of user-facing LLM use cases are where people don't have these high-quality starting points. They don't have 40k tokens to make 400.

[helloplanets]

You can just attach 40k of context directly into the Gemini, ChatGPT and Claude web interfaces afaik. If someone is using an LLM as a tool to actually be of help in an area they are already professionals in, conjuring good books, research, etc, as attachments shouldn't be an issue.

But yes, the default mode of LLMs is usually a WikiHow and content farm style answer. This is also a problem with Google: The content you get back from generic searches will often be riddled with inaccuracies and massive generalizations.

Not being able / bothering to come up with relevant context and throwing the dice on the LLM being able to do this out of the box is definitely a serious issue. I really think that is where the discussion should be: Focused more on how people use these tools. Just like you can tell quite a bit about someone's expertise based on the specific way in which they interface with Google (or any information on the internet) while they work.