[gerdesj]

> Or, learn how to say “I don’t know”

It doesn't know that it doesn't know! It is, very roughly speaking, a model that is designed to print out the most likely word given its current input and training, and then the next word etc.

Whereas you or I might be mistaken about some of our faculties, memories and skills, ChatGPT cannot possibly "know" what its limitations are. It was never taught what it was not taught (obvs) nor can it conceive of what it was not taught about. It might come back with some useful chat in this regard but it cannot reason.

[okasaki]

It seems that you don't know what you don't know, really.

There's no way to definitively know what properties ChatGPT has. It does seem to reason to some extent and it does often say that some information isn't known/there's no data. And it almost obnoxiously often tells you that it's simplifying a complex and multifaceted situation.

[gerdesj]

Its a model that takes an input and spits out the most likely output given its training.

"There's no way to definitively know what properties ChatGPT has." - yes there is: ask it how the war in Ukraine is progressing or some other time based thing. It stops in 2021.

It is a really useful tool but it isn't sentient.

[okasaki]

I don't agree. It's like saying humans are just animated meat. You're massively simplifying the situation.

[gerdesj]

I never mentioned humans or meat yet you decided to pop those words in my mouth (mmm tasty).

I merely tried to very quickly give a hint as to what ChatGPT is - it's a LLM and that is something that takes a (normally textual) input and outputs an expected best guess word for an answer and then does it repeatedly until the full answer is returned.

[roflyear]

It doesn't know things. I promise.