[vanderZwan]

Even assuming all of what you said is true, none of it disproves the arguments in the article. You're talking about the technology, the article is about the marketing of the technology.

The LLM marketing exploits fear and sympathy. It pressures people into urgency. Those things can be shown and have been shown. Whether or not the actual LLM based tools genuinely help you has nothing to do with that.

[remus]

The point of the article is to paint LLMs as a confidence trick, the keyword being trick. If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick? If a street performer was doing the cup and ball scam, but I actually won and left with more money than I started with then I'd say that's a pretty bad trick!

Of course it is a little more nuanced than this and I would agree that some of the marketing hype around AI is overblown, but I think it is inarguable that AI can provide concrete benefits for many people.

[intended]

The marketing hype is economy defining at this point, so calling it overblown is an understatement.

Simplifying the hype into 2 threads, the first is that AI is an existential risk and the second is the promise of “reliable intelligence”.

The second is the bugbear, and the analogy I use is factories and assembly lines vs power tools.

LLMs are power tools. They are being hyped as factories of thoughts.

String the right tool calls, agents, and code together and you have an assembly line that manufactures research reports, gives advice, or whatever white collar work you need. No Holidays, HR, work hours, overhead etc.

I personally want everyone who can see why this second analogy does not work, to do their part in disabusing people of this notion.

LLMs are power tools, and impressive ones at that. In the right hands, they can do much. Power tools are wildly useful. But Power tools do not make automatically make someone a carpenter. They don’t ensure you’ve built a house to spec. Nor is a planar saw going to evolve into a robot.

The hype needs to be taken to task, preferably clinically, so that we know what we are working with, and can use them effectively.

[latexr]

> If LLMs do actually deliver very real, tangible benefits then can you say there is really a trick?

Yes, yes you can. As I’ve mentioned elsewhere on this thread:

> When a con man sells you a cheap watch for an high price, what you get is still useful—a watch that tells the time—but you were also still conned, because what you paid for is not what was advertised. You overpaid because you were tricked about what you were buying.

LLMs are being sold as miracle technology that does way more than it actually can.

[ozlikethewizard]

And at a cost im not sure most fully understand. We've allowed these companies to externalise all the negative outcomes. Now were seeing consumer electronics stock dry up, huge swaths of raw resources used, massive invasions of privacy, all so this one guy can do his corpo job 10x faster? Nah im good.

[megaBiteToEat]

A huge amount of tech is a confidence trick. Not one aimed at <50 year old crowd but aimed at innumerate and STEM ignorant political leaders.

It's not LLMs they care about, it's datacenter ownership. US political norms empower owners. If you think of a DC as a mega church and remote users the disciple, it makes the desired network effect obvious. That is leveraged to sway Congress and states.

These tech projects are not intended for users. They're designed to gain confidence of politicians, preferential political support.

Gen pop is not the market. DC is.

Most peoples individual data crunching problems can be resolved with a TI graphing calculator.

Big Tech convinced Congress that culture of helpless consumers of their data center outputs is simpler and will lead humanity to a forever growth future!... nevermind they will all be dead, unable to verify.

A con trick that worked great on older, more religious leaning Americans. One that's not working so well on the younger generation who know how these systems work.

[carpo]

But saying it's a confidence trick is saying it's a con. That they're trying to sell someone something that doesn't work. Th op is saying it makes then 10x more productive, so how is that a con?

[trimethylpurine]

The marketing says it does more than that. This isn't just a problem unique to LLMs either. We have laws about false advertising for a reason. It's going on all the time. In this case the tech is new so the lines are blurry. But to the technically inclined, it's very obvious where they are. LLMs are artificial, but they are not literally intelligent. Calling them "AI" is a scam. I hope that it's only a matter of time until that definition is clarified and we can stop the bullshit. The longer it goes, the worse it will be when the bubble bursts. Not to be overly dramatic, but economic downturns have real physical consequences. People somewhere will literally starve to death. That number of deaths depends on how well the marketers lied. Better lies lead to bigger bubbles, which when burst lead to more deaths. These are facts. (Just ask ChatGPT, it will surely agree with me, if it's intelligent. ;p)

[CamperBob2]

How does one go about competing at the IMO without "intelligence", exactly? At a minimum it seems we are forced to admit that the machines are smarter than the test authors.

[trimethylpurine]

"LLM" as a marketing term seems rational. "Machine learning" also does. We can describe the technology honestly without using a science fiction lexicon. Just because a calculator can do math faster than Isaac Newton doesn't mean it's intelligent. I wouldn't expect it to invent a new way of doing math like Isaac Newton, at least.

[CamperBob2]

Just because a calculator can do math faster than Isaac Newton doesn't mean it's intelligent.

Exactly, and that's the whole point. If you lack genuine mathematical reasoning skill, a calculator won't help you at the IMO. You might as well bring a house plant or a teddy bear.

But if you bring a GPT5-class LLM, you can walk away with a gold medal without having any idea what you're doing.

Consequently, analogies involving calculators are not valid. The burden of proof rests firmly on the shoulders of those who claim that an LLM couldn't invent new mathematical techniques in response to a problem that requires it.

In fact, that appears to have just happened (https://news.ycombinator.com/item?id=46664631), where an out-of-distribution proof for an older problem was found. (Meta: also note the vehement arguments in that thread regarding whether or not someone is using an LLM to post comments. That doesn't happen without intelligence, either.)

[trimethylpurine]

That doesn't appear to be what happened. But the marketing sure has a lot of people working quick to presume so.

I would guess it's only a matter of days before that proof, or one very similar, is found in the training data, if that hasn't happened already, just as has been the case every time.

No fundamental change in how the LLM functions has been made that would lead us to expect otherwise.

Similar "discoveries" occurred all the time with the dawn of the internet connecting the dots on a lot of existing knowledge. Many people found that someone had already solved many problems they were working on. We used to be able to search the web, if you can believe that.

The LLMs are bringing that back in a different way. It's functional internet search with an uncanny language model, that sadly obfuscates the underlying data while making guesswork to summarize it (which makes it harder to tell which of its findings are valuable, and which are not).

It's useful for some things, but that's not remotely what intelligence is. It doesn't literally understand.

>* if you bring a GPT5-class LLM, you can walk away with a gold medal without having any idea what you're doing.*

My money won't be betting on your GPT5-class business advice unless you have a really good idea what you're doing.

It requires some (a lot of) intelligence and experience to usefully operate an LLM in virtually every real world scenario. Think about what that implies. (It implies that it's not by itself intelligent.)

[latexr]

Exactly. It’s like if someone claimed to be selling magical fruit that cures cancer, and they’re just regular apples. Then people like your parent commenter say “that’s not a con, I eat apples and they’re both healthy and tasty”. Yes, apples do have great things about them, but not the exaggerations they were being sold as. Being conned doesn’t mean you get nothing, it means you don’t get what was advertised.

[JacoboJacobi]

The claims being made that are cited are not really in that camp though..

It may be extremely dangerous to release. True. Even search engines had the potential to be deemed too dangerous in the nuclear pandoras box arguments of modern times. Then there are high-speed phishing opportunities, etc.

It may be an essential failure to miss the boat. True. If calculators were upgraded/produced and disseminated at modern Internet speeds someone who did accounting by hand would have been fired if they refused to learn for a few years.

Its communication builds an unhealthy relationship that is parasitic. True. But the Internet and the way content is critiqued is a source of this even if it is not intentionally added.

I don't like many people involved and I don't think they will be financially successful on merit alone given that anyone can create a LLM. But LLM technology is being sold by organic "con" that is how all technology such as calculators end up spreading for individuals to evaluate and adopt. A technology everyone is primarily brutally honest about is a technology that has died because no one bothers to check if the brutal honesty has anything to do with their own possible uses.

[latexr]

> The claims being made that are cited are not really in that camp though..

They literally are. Sam Altman has literally said multiple times this tech will cure cancer.

[JacoboJacobi]

Such claims are not cited in the article. It may be possible to write a good article on the topic but this article could just as well be on the organic uptake of the PC and how most wealthy nontechnical people adopted a need for a PC on "cons" that preceded their ability to gain more worth than trouble.

[amelius]

Yeah, but it should have been in the title otherwise it uses in itself a centuries old trick.