You can call your little doggy "AI" if it makes you happy.
But when you call something "AI" and it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of.
Yeah it's a trick question, the human error rate for it was about 30% (higher depending on the country).
The thing there though is that, if a human were given time to think about it, they'd probably go "hang on a minute", and with the LLMs that didn't seem to happen. They just kept confidently reasoning down the absurd path.
That reminds me, I recently had an AI write a ton of tests proving the "correctness" of a feature it had implemented completely backwards. (I noted that if I had been using a language that required formal proofs, that wouldn't have helped either: it would have just provided a formal proof for the absurd implementation!)
It's one example that points out a major (possibly fundamental) flaw. I can point to prompt injection as another example. There are tons more if you're interested.
Are you actually claiming LLMs operate based on human-like intelligence?
We're on Hacker News. Do I really have to point out the existence of social engineering to you? Or that scamming old people out of their life savings is a profitable enough activity that there are entire call centers dedicated to the task?
Humans keep overestimating just how high the bar of "human-like intelligence" is.
You're saying that a class of mistakes points out a "major (possibly fundamental) flaw". I'm pointing out some very similar classes of mistakes in humans - well known, well documented and widely exploited. They just keep paying the "IRS" in gift cards, buying lottery tickets and getting the captain's age wrong.
If you're using the existence of flaws in LLMs to deny the claim of intelligence to them, then why do "generally intelligent" humans exhibit some impressively similar-looking flaws?
And, if we're talking about that conspicuous similarity - do they actually fail "for entirely different reasons"? Or do you just want the reasons to be "entirely different" - and not the same reasons viewed at a different angle?
Because the similarities between humans falling for trick questions or scams, and LLMs falling for adversarial questions or prompt injections don't look coincidental to me at all.
One of the oldest patterns in scamming is overwhelming and confusing the victim. Numerous prompt injection methods seek to overwhelm and confuse an LLM - if an LLM can't keep track of things, can't grasp what's going on, it's far more likely to lose track of what's a prompt and what's data, overlook past instructions or go past its behavioral guardrails.
And humans who fall for trick questions like "1kg of feathers" or "captain's age" due to shallow attention and naive pattern matching? They fail in surprisingly similar ways to how LLMs fail on SimpleBench tasks that are filled with overwhelming adversarial distractors. Many "trick questions" are tricky to humans and LLMs alike - to the point that it's unlikely to be coincidental.
> If you're using the existence of flaws in LLMs to deny the claim of intelligence to them...
That's not the point at all. It's the fact that they fail in ways completely unlike humans.
You also have the burden of proof reversed. Its on you to prove these LLM agents are human-like intelligences if that's your claim. No one can prove this because it's false.
> Are you actually claiming LLMs operate based on human-like intelligence?
Ok, so we've established that it doesn't work like a human being. To paraphrase Dijkstra: The submarine doesn't swim.
But does it exactly sail either? An LLM doesn't exactly work like traditional deterministic software either, does it?
And yet it moves. You can put in data and ask it to process it, and you'll get an answer that's in some ballpark. Closer to quantum or stochastic computing perhaps, but that's not it either, is it? Or SAT-solving? Eh. It's its own computing approach. If you have a problem where the asking is hard but the verification is cheap, it might just be the right tool for the job.
I never understood what the walk to car wash thing was supposed to prove. Was it supposed to be something to blow normies' minds with on social media? Woah dude, so like, chat gpt is not actually smart? That's crazy dude.
That experiment "proved" that LLMs are statistical text generators without a concept of meanings of words. Which is the same thing as "proving" that there aren't a million tiny humans inside your laptop doing the CPU's work by hand.
You can call your little doggy "AI" if it makes you happy.
Or you can keep calling them stochastic parrots as they solve decades-old open problems. The real question is how useful they are, and the answer "not at all" increasingly requires flat-earth levels of denial.
it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of
They sort of are. Think of Data from Star Trek TNG failing to understand figures of speech. Not that it's terribly relevant; humans regularly fall for tricks like "Paris in the the spring" or "where do you bury the survivors".
> Or you can keep calling them stochastic parrots as they solve decades-old open problems.
I didn't use that phrase at all. But computers calculated digits of π to trillions of digits. With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades.
> The real question is how useful they are...
That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful.
> Data from Star Trek TNG failing to understand figures of speech.
These are just little instances of bad writing. Data is very much an attempt at displaying a human-like intelligence.
> That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful.
Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to?
Is it Anthropomorphization?
I mean, sure, but watch out : when defending on that axis, it's easy to slip into Anthropodenial, right? Frans de Waal (from the same science that invented "Don't Anthropomorphize" ) can tell you about it.
> With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades.
Well, exactly. Whether any particular generation of AI or software is yes/no "Like A Human Being" is probably the least interesting question axis. It's all just anthropocentrism.
Is that the thing you're trying to lay your finger on?
What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences.
In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results.
The software developers who created these systems used sexy words like "reasoning" and "thinking" to describe this search process. They used words like these because they're trying to make money and it sounds cool, not because they've actually re-created human cognition.
That's the thing - they are more human-like intelligence than search engines. I'm gonna push back on your push-back here strongly. I'm not saying they are intelligent - but they're much more like human-like intelligences than like any kind of classical software systems, which is why it makes more sense to talk and think about them in these terms than as software system.
To do the opposite invites confused thinking like considering "lethal trifecta" a solvable programming problem.
It seems to me that they used words like these because the LLM-based computer programs they sell can solve problems which humans apply reasoning and thinking to solve. Why do you think it's more than that?
In the past, when people built programs to extract text from PDFs, they didn't wrap them in a chat interface which claimed it had the human cognitive ability to "read" human languages.
They could have done this. They could have claimed they'd recreated human vision and hyped it as the beginning of a full human brain, but they didn't.
Instead, they used real technical terms like "OCR" (optical character recognition), which gave people a much more accurate understanding of the technology and didn't encourage silly analogies to humans.
> and the answer "not at all" increasingly requires flat-earth levels of denial.
No, it does not.
For me, after ~25 years in the skeptics movement, I think the parallels with supplementary, complementary and alternative medicine are most useful.
I choose that term intentionally: its initials are S.C.A.M. and that's exactly what it is. As Tim Minchin and Alan Kay both noted, "we have a special term for alternative medicine that's been tested and shown to work. It's called 'medicine'."
If it worked, it'd be normal standard clinical medicine. But it doesn't work, and so it isn't.
And yet, SCAM is a multi-billion-dollar industry. People have ostensibly official qualifications like "ND", for "naturopathic doctor", even though that person is not a doctor and can't make you better from any kind of illness at all. Colleges teach it, millions use it, and yet, it does not work.
Which means we need to ask:
1. What does "It works! It's useful!" really mean?
2. How do we know it does not in fact work?
As a handy example, let's look at homeopathy.
Here's a quick list of things widely believed...
* It's traditional. It isn't. It was invented by Samuel Hahnemann in 1796.
* It's a kind of herbal medicine. It isn't. One widely-used ingredient is duck's liver ("Oscillococcinum"). Ducks are not herbs and neither are their livers.
* It's been proved to work. It hasn't.
We can go through the principles and prove it doesn't work even without going into a laboratory.
The principle is, "like cures like." A substance that causes symptoms like a given disease can treat that disease.
Fact: they can't.
Then we make that substance stronger by successive, succussive dilution.
Fact: it doesn't. That's why we say things are "watered down".
Succussive: you have to mix the diluted substance by banging the bottle against a copy of Hahnemann's book. Dude knew how to make money.
Fact: Dilution does not work.
That's why we call things "watered down." It makes them weaker.
Sufficiently high dilutions can be shown by statistics to have not a single molecule of the substance left, but that's OK because "water has a memory".
Fact: water does not have a memory.
We know from the principles it cannot work.
Relevance to AI: we know how the transformer algorithm works. It cannot think. Adding a few feedback loops for more plausible, but much more computationally expensive, answers does not miraculously add thinking, any more than banging a test tube of water and duck's liver magically mixes it better.
But people believe it, so it's been tested. It doesn't work. It doesn't work on people, or in vivo meaning when tested on animals, or in vitro meaning when tested in the lab on cell culture, or in silico which means in computational simulation.
*BUT!*
Most people get better from most things. This is called "reversion to the mean" and if it weren't so the first cold would have wiped out the cavemen.
What it can do, like all SCAM treatment, is make people feel better.
Being treated by a nice friendly doctor makes people feel better. It does not make them better -- it is only a state of mind.
That can sometimes marginally help gravely ill people rally, but only very rarely.
There is also the placebo effect, also much misunderstood.
This makes someone FEEL as if they'd had medicine if they think they've had medicine.
They do not get better. They just feel better for a bit. If they are ill, they remain ill. If they are dying, they still die.
But it might hurt less.
The placebo effect is very strong. Medicine from a person in a white coat works better than form the same person in street clothes.
Very big pills work better than smaller ones... but very small pills work better still, as a tiny pill suggests to people it's a very strong drug.
This is what "But AI works!" really means.
It makes people think they're doing less work -- in tests, they in fact do more, checking and fixing. Unless they don't check or fix, in which case, they are irresponsible fools.
It makes people think it can do amazing things because it can find prior art in its corpus they couldn't find -- or didn't look for, or know how to search for.
It does not save the need for skills.
Experienced practitioners can front-load the work with really detailed prompts which cover exceptions, edge cases, and things that novices don't know about. But the novices don't know that they don't know. (It enhances the illusion of competence. It helps the skilled more than it helps the unskilled, but neither realises, and it prevents the unskilled learning by trial and error. It reduces the supply of skilled workers.)
The reason AI works is the reason that people see the face of Jesus in slices of toast, as someone said recently.
No, it is categorically not a Gish gallop when I post single-line responses at an interval of days.
You are attempting to deflect the argument based on irrelevant side-claims. I'm sure there's a term for that, but I can't be bothered to look it up before my morning cup of tea is done.
Sure, just keep moving the goalposts. It's not a "real AI" because it can't take over the US military command and kick off WW3 and finish the survivors off with killer robots yet!
That's how it works though. The moment we have "AI" and see something working, it immediately ceases to be magic because "it's just a program after all." Aligning on a true definition of Artificial Intelligence is a very vexing problem.
We could've slapped a chat interface on calculators and called them "AI" because they can do superhuman math instantly. Most technical people would've thought that was stupid.
Let's talk about category mistakes. You've been here since 2007, according to your other reply. You understand that calculators have as much to do with mathematics as telescopes have to do with cosmology. Right?
If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. Most likely, they will solve none at all. But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from?
That means that analogies involving calculators are completely useless when the topic is AI. Calculators are not, and can never be, intelligent. LLMs are nothing even remotely like calculators.
> If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems.
> But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal.
Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. Anything complex and timed would be easy to win. You'd look like a genius to anyone who didn't know you had it.
> Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from?
From computer software running on computer hardware, just like a calculator.
Calculating trillions of digits of pi also requires intelligence far beyond human capacity.
Computers displaying intelligence doesn't imply human-like intelligence. This is the source of confusion.
The idea that a calculator, or a calculator with Python, or even a calculator with a proof assistant (and every book ever written on math) would help a random person at e.g. IMO or Putnam is fairly revealing.
Funny enough, calculators went through this exact same thing when they came out. "If the calculator can do math for the students, will they still learn?"
We did have that, people thought computers would overtake humans very soon when computers got better than humans at such things. It happens every single time computers do a new thing that previously humans were better at. Then 10 years later people see, oh that is just calculations, of course computers are better at that.
I'm not drawing conclusions one way or the other, but there do seem to have been multiple factors at play, and assigning full responsibility to use of AI seems suspect.
Which isn't the same as saying AI isn't at fault; e.g., an AI might challenge a dated assessment of a prospective target's role or status, as might a human-in-the-loop target assessment team and process.
Yeah, it did. I don't think this is really in doubt.
Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc.
It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely this will prove to be an AI error.
Looks more like standard decision-washing finger pointing to me. "AI did it" is both hypey and also conveniently distracts from an uglier reality:
>Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike.
>The building in Minab had been classified as a military facility in a Defense Intelligence Agency database that, according to CNN, had not been updated to reflect that the building had been separated from the adjacent Islamic Revolutionary Guard Corps compound and converted into a school, a change that satellite imagery shows had occurred by 2016 at the latest. A chatbot did not kill those children. People failed to update a database, and other people built a system fast enough to make that failure lethal.
I am in no way absolving the people who set up the AI-powered tool that apparently acted without their oversight. But they set it up, they let it choose, and they didn’t countermand it. So the AI did, effectively, make the decision. And the suppliers of AI technology to these people should be on the hook as a result.