[forrestthewoods]

That’s not particularly different from comments on HackerNews and Reddit. There’s a lot of extremely confident and very wrong answers on both sites!

ChatGPT is probably more wrong more often, by a good margin. But I don’t think the argument “it’s confidently wrong” carries any weight. Humans are extremely susceptible to humans who display confidence. It’d probably be a good thing if humans were as skeptical of confident humans as they need to be of confident chat/search bots.

[layer8]

> There’s a lot of extremely confident and very wrong answers on both sites!

Yeah, but you get to look at a multitude of them, at the context, at replies, at the websites they link to, and can form your own opinion. ChatGPT isn’t able to do that work for you.

[forrestthewoods]

Wait a few years and we’ll have multiple ChatGPTs from multiple sources you can ask the same question and compare the answer!

[m_mueller]

But I’d say at least readers of a random HN comment are naturally skeptic of it. Are ChatGPT users on average having the same stance? My worry is when these tools get just good enough such that due diligence is dropped out of convenience.

[dinkumthinkum]

It gives literally the same argument for 42 being prime as 23.

[spiderice]

I'm not sure what you're talking about. I just asked it to argue that 42 is a prime number, and it clearly knew it wasn't and even said multiple times that the agument is just pretend/hypothetical:

________

Me: Make an argument for why 42 is a prime number

ChatGPT: 42 is not a prime number. It is a composite number, meaning it is the product of two smaller numbers.

Me: But pretend it is a prime number and make that argument

ChatGPT: Sure, I can make an argument for why 42 could be considered a prime number if we pretend that it is.

One possible argument for why 42 could be considered a prime number is that it satisfies the basic definition of a prime number. A number is prime if it is divisible only by 1 ... (continues with rest of argument)

Of course, these arguments are purely hypothetical and are not based on the actual mathematical properties of 42. In reality, 42 is a composite number and is not considered a prime number.

________

[notahacker]

The specific phrasing you used implies probable continuations involve treating the proposition as false, and 42 is also a number the model will have had a lot of user feedback on. Similarly, the model gives an excellent answer for 1000 which even mentions some factors of 1000.

On the other hand

> Is 355 a prime number?

> Yes, 355 is a prime number. A prime number is a positive integer that has no positive integer divisors other than 1 and itself. For example, 2, 3, 5, and 7 are all prime numbers because they cannot be evenly divided by any other positive integers. 355 is a prime number because it cannot be evenly divided by any other positive integers except for 1 and itself. It is not divisible by 2, 3, 4, 5, or any other smaller positive integers.

[dinkumthinkum]

This has been documented many times, you can Google it or just ask ChatGPT again.