[gradus_ad]

I think the best existing "product" analogy for LLM's is coffee.

Coffee is a universally available, productivity enhancing commodity. There are some varieties certainly, but at the end of the day, a bean is a bean. It will get the job done. Many love it, many need it, but it doesn't really cost all that much. Where people get fancy is in all the fancy but unnecessary accoutrements for the brewing of coffee. Some choose to spend a lot on appliances that let you brew at home rather than relying on some external provider. But the quality is really no different.

Apparently global coffee revenue comes out to around $500B. I would not be surprised if that is around what global AI revenue ends up being in a few years.

[teiferer]

> Coffee is a universally available, productivity enhancing commodity

The analogy carries further than you intended. If you have never reached addiction stage, then there is no factual productivity enhancement. "But I'm so much less productive if I haven't had my morning coffee" Yeah, because you have an addiction. It sounds worse than it is, if you just don't drink coffee for a few days the headaches will subside. But it doesn't actually enhance productivity beyond placebo.

[kortex]

It does objectively improve productivity though, beyond offsetting withdrawal.

https://www.sciencedirect.com/science/article/pii/S014976341...

[Balgair]

I'd be interested to know what the controls were for those studies. Were the participants already addicted or was the 30mg+ dosing done on non-addicted people? It's a lot of studies to pour through.

Also, that is a lot of metrics!

And it seems that the athletic performance increase to get statistical validation (for any person) is in the grams range. I ... I just can't see any reason to take that much caffeine unless I'm at the Olympics. I'd be jumping out of my skin!

[tehjoker]

I'm not so sure. The stimulation can also self-medicate for people with attention issues. I've tried quitting coffee before for weeks and I get so spacey it is difficult to work on major projects. I try coffee again and suddenly I feel quite capable. Perhaps I didn't quit long enough, but at this point after multiple attempts quitting with similar results, I've just accepted it.

[nchmy]

Because you haven't detoxed long enough. Try 1-2 months - you'll be shocked by the clarity, stability in energy, better sleep etc

[normie3000]

Enjoyable analogy.

> Some choose to spend a lot on appliances that let you brew at home rather than relying on some external provider.

This makes it sound like buying brewed coffee is the budget option. But the real budget option I've seen is to brew at home. Almost any household will have an appliance to boil water. Then add instant coffee.

I don't understand why, but in my experience instant coffee seems to be the baseline even in coffee-producing countries.

[arctic-true]

I think the idea is that there are higher startup costs to brew at home. Even a cheap coffee machine is going to cost more than a cup of coffee at a diner, in the same way that a computer that can run a local LLM is going to cost more than a bunch of API calls to a commercial model. Eventually, those diner coffees add up, but you’re stuck with them if you can’t afford the coffee machine.

[normie3000]

> Even a cheap coffee machine is going to cost more than a cup of coffee at a diner

I think I understood but disagree - the cheapest "coffee machine" is a kettle or cooking pot.

[nchmy]

Cowboy coffee can be excellent. It's almost literally what professional coffee tasters do when they do "cuppings".

A French press just adds fine mesh. You don't even need the glass jar - literally a jar or any other vessel would work

[nchmy]

They're not talking about instant coffee, which is awful. And coffee-producing countries are poor, hence they drink dirty sugar water while the good stuff gets exported.

[glitchcrab]

> But the quality is really no different.

Hard disagree. As someone who is somewhat into the home brewing rabbit hole, I can tell you that the gulf between what I can make at home and what you get in Starbucks is enormous. And I'm no expert in the field by any means.

The rest of your analogy holds up, but not that sentence.

[eudamoniac]

Not the best analogy because coffee has massive quality differences that anyone can tell in a real comparison. Brewing at home is also cheaper and requires nothing more than a $20 French press.

I have personally administered double blind taste tests to four unrelated coffee drinkers who believed their current coffee (Keurig, Nespresso, pre ground) was good. They all thought the actual good coffee was much better in the blind tasting, to the extent that they all permanently stopped drinking their previous coffee. Exclamations of shock occurred.

Coffee is a commodity only out of ignorance. It should be treated as fresh produce of numerous varietals.

Have there been any such "tastings" of LLMs?