Hacker News new | ask | show | jobs
by coldpie 520 days ago
> Maybe it's like with Apple and I am using it wrong.

Well, yeah. You are. It's built to answer questions people actually ask, not solve new logic puzzles.

Despite what the marketing says, it's not a perfect-infinite-knowledge oracle. You should think of it more as a really, really big database with all of the Internet's "knowledge". When you ask it "2 + 2 = ?", it isn't parsing those into numbers and math operations, it's searching its database for occurrences on the Internet where someone answered the question "2 + 2 = ?" and filling in the closest answer it found. If you ask it what "120938120938120931 + 1209389120381208390" is, it'll probably get it wrong, because no one has asked that before. But you should probably be using a calculator instead.

If you ask it something it hasn't seen before such as your logic puzzle, it's not going to parse it like a person would and synthesize an answer. It's going to try to find something similar to what it's seen before and return that. Odds are good this will be a wrong answer, since it's not addressing what you actually asked.

However, if you ask it something it has seen before, like a programming problem, it will return something appropriate. It turns out the Internet is pretty big, so it has seen a lot of stuff, and so often works pretty well. Hence the success you're seeing from others who are using it as-intended, i.e., asking it real questions, not logic puzzles.

It's not very hard to come up with a scenario that has never been put on the Internet, so it's pretty easy to make it dig up the "wrong" answer and do something stupid, as you've found.

The real trouble is that it can't tell you whether it's guessing, or found an actual match. Hence the "confidently wrong" thing, which absolutely destroys user trust. If it's confidently wrong about this thing I know a lot about, how can I trust it to be accurate for something I know little about?

2 comments

Ok, maybe logic puzzles are "unfair", but there are other situations where you ask it a question that is similar (but different in an important way) to a problem it was trained on, and then it will provide the answer to what it "thinks" you asked, not what you actually asked. Hence, hallucinations.
Oh yeah it's got all sorts of problems. To be honest, I've personally never found it terribly useful. But, I find the "it can't answer this basic math/logic problem" criticism really dull. It's not built for that.
> Well, yeah. You are. It's built to answer questions people actually ask, not solve new logic puzzles.

Isn't the causality inverted here? It's trained on questions people have asked before, so that's what it's better at. New logic puzzles illustrate this flaw