[orangecat]

You can call your little doggy "AI" if it makes you happy.

Or you can keep calling them stochastic parrots as they solve decades-old open problems. The real question is how useful they are, and the answer "not at all" increasingly requires flat-earth levels of denial.

it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of

They sort of are. Think of Data from Star Trek TNG failing to understand figures of speech. Not that it's terribly relevant; humans regularly fall for tricks like "Paris in the the spring" or "where do you bury the survivors".

[vidarh]

If anything a large proportion of stories about AI in sci fi is about AI failing to understand humans in various ways.

[jacobgold]

> Or you can keep calling them stochastic parrots as they solve decades-old open problems.

I didn't use that phrase at all. But computers calculated digits of π to trillions of digits. With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades.

> The real question is how useful they are...

That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful.

> Data from Star Trek TNG failing to understand figures of speech.

These are just little instances of bad writing. Data is very much an attempt at displaying a human-like intelligence.

[Kim_Bruning]

> That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful.

Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to?

Is it Anthropomorphization?

I mean, sure, but watch out : when defending on that axis, it's easy to slip into Anthropodenial, right? Frans de Waal (from the same science that invented "Don't Anthropomorphize" ) can tell you about it.

> With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades.

Well, exactly. Whether any particular generation of AI or software is yes/no "Like A Human Being" is probably the least interesting question axis. It's all just anthropocentrism.

Is that the thing you're trying to lay your finger on?

[jacobgold]

What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences.

In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results.

The software developers who created these systems used sexy words like "reasoning" and "thinking" to describe this search process. They used words like these because they're trying to make money and it sounds cool, not because they've actually re-created human cognition.

[TeMPOraL]

That's the thing - they are more human-like intelligence than search engines. I'm gonna push back on your push-back here strongly. I'm not saying they are intelligent - but they're much more like human-like intelligences than like any kind of classical software systems, which is why it makes more sense to talk and think about them in these terms than as software system.

To do the opposite invites confused thinking like considering "lethal trifecta" a solvable programming problem.

[orangecat]

some people genuinely believe these LLM-based computer programs are human-like intelligences

"If LLMs were human-like intelligences they would do X, but they don't". What is X?

In reality, they're more like very good search engines that output relevant snippets of text.

What are the "relevant snippets" that contained the solutions for the unit distance and Jacobian conjectures?

[SpicyLemonZest]

It seems to me that they used words like these because the LLM-based computer programs they sell can solve problems which humans apply reasoning and thinking to solve. Why do you think it's more than that?

[jacobgold]

In the past, when people built programs to extract text from PDFs, they didn't wrap them in a chat interface which claimed it had the human cognitive ability to "read" human languages.

They could have done this. They could have claimed they'd recreated human vision and hyped it as the beginning of a full human brain, but they didn't.

Instead, they used real technical terms like "OCR" (optical character recognition), which gave people a much more accurate understanding of the technology and didn't encourage silly analogies to humans.

[lproven]

> The real question is how useful they are

No, it is not.

> and the answer "not at all" increasingly requires flat-earth levels of denial.

No, it does not.

For me, after ~25 years in the skeptics movement, I think the parallels with supplementary, complementary and alternative medicine are most useful.

I choose that term intentionally: its initials are S.C.A.M. and that's exactly what it is. As Tim Minchin and Alan Kay both noted, "we have a special term for alternative medicine that's been tested and shown to work. It's called 'medicine'."

If it worked, it'd be normal standard clinical medicine. But it doesn't work, and so it isn't.

And yet, SCAM is a multi-billion-dollar industry. People have ostensibly official qualifications like "ND", for "naturopathic doctor", even though that person is not a doctor and can't make you better from any kind of illness at all. Colleges teach it, millions use it, and yet, it does not work.

Which means we need to ask:

1. What does "It works! It's useful!" really mean?

2. How do we know it does not in fact work?

As a handy example, let's look at homeopathy.

Here's a quick list of things widely believed...

* It's traditional. It isn't. It was invented by Samuel Hahnemann in 1796. * It's a kind of herbal medicine. It isn't. One widely-used ingredient is duck's liver ("Oscillococcinum"). Ducks are not herbs and neither are their livers. * It's been proved to work. It hasn't.

We can go through the principles and prove it doesn't work even without going into a laboratory.

The principle is, "like cures like." A substance that causes symptoms like a given disease can treat that disease.

Fact: they can't.

Then we make that substance stronger by successive, succussive dilution.

Fact: it doesn't. That's why we say things are "watered down".

Succussive: you have to mix the diluted substance by banging the bottle against a copy of Hahnemann's book. Dude knew how to make money.

Fact: Dilution does not work.

That's why we call things "watered down." It makes them weaker.

Sufficiently high dilutions can be shown by statistics to have not a single molecule of the substance left, but that's OK because "water has a memory".

Fact: water does not have a memory.

We know from the principles it cannot work.

Relevance to AI: we know how the transformer algorithm works. It cannot think. Adding a few feedback loops for more plausible, but much more computationally expensive, answers does not miraculously add thinking, any more than banging a test tube of water and duck's liver magically mixes it better.

But people believe it, so it's been tested. It doesn't work. It doesn't work on people, or in vivo meaning when tested on animals, or in vitro meaning when tested in the lab on cell culture, or in silico which means in computational simulation.

*BUT!*

Most people get better from most things. This is called "reversion to the mean" and if it weren't so the first cold would have wiped out the cavemen.

What it can do, like all SCAM treatment, is make people feel better.

Being treated by a nice friendly doctor makes people feel better. It does not make them better -- it is only a state of mind.

That can sometimes marginally help gravely ill people rally, but only very rarely.

There is also the placebo effect, also much misunderstood.

This makes someone FEEL as if they'd had medicine if they think they've had medicine.

They do not get better. They just feel better for a bit. If they are ill, they remain ill. If they are dying, they still die.

But it might hurt less.

The placebo effect is very strong. Medicine from a person in a white coat works better than form the same person in street clothes.

Very big pills work better than smaller ones... but very small pills work better still, as a tiny pill suggests to people it's a very strong drug.

This is what "But AI works!" really means.

It makes people think they're doing less work -- in tests, they in fact do more, checking and fixing. Unless they don't check or fix, in which case, they are irresponsible fools.

It makes people think it can do amazing things because it can find prior art in its corpus they couldn't find -- or didn't look for, or know how to search for.

It does not save the need for skills.

Experienced practitioners can front-load the work with really detailed prompts which cover exceptions, edge cases, and things that novices don't know about. But the novices don't know that they don't know. (It enhances the illusion of competence. It helps the skilled more than it helps the unskilled, but neither realises, and it prevents the unskilled learning by trial and error. It reduces the supply of skilled workers.)

The reason AI works is the reason that people see the face of Jesus in slices of toast, as someone said recently.

[Kim_Bruning]

> The reason AI works is the reason that people see the face of Jesus in slices of toast, as someone said recently.

Apparently my unit tests can see faces in slices of toast.

[lproven]

Good for you.

Now, shall we discuss the ecological and commercial cost of that?

[Kim_Bruning]

So, I notice a https://rationalwiki.org/wiki/Gish_Gallop , followed by https://rationalwiki.org/wiki/Moving_the_goalposts

Let's just say I'm skeptical of your skepticism ;-)

[lproven]

No, it is categorically not a Gish gallop when I post single-line responses at an interval of days.

You are attempting to deflect the argument based on irrelevant side-claims. I'm sure there's a term for that, but I can't be bothered to look it up before my morning cup of tea is done.

[Kim_Bruning]

https://news.ycombinator.com/item?id=48991040 gish gallop

https://news.ycombinator.com/item?id=49012641 moving of goalposts once we started moving to empirical territory.

And I'll get back to getting my units to pass. Have a nice morning!