Hacker News new | ask | show | jobs
by lsedgwick 972 days ago
Please don't paste ChatGPT or BARD answers as HN comments in general? In this specific case, no, LLMs don't reliably know about themselves. They're trained on a big corpus of internet text, then trained by rough reinforcement learning to say and not say certain things about themselves, then given a little more information about themselves in system prompts.
2 comments

A LLM that can use tools is probably going to know it can use tools. Either it's trained to do so or the information is embedded in the context.

3.5 can't browse, only 4 so the above is perfectly correct.

Fair, you're probably right in this case.
If you’re right, then this is probably the best proof that I’ve seen that ChatGPT isn’t even remotely conscious.
Imagine a distant future where there're so many crimes and so few judges you're cryofrozen awaiting trial.

If you were repeatedly flash frozen and flash unthawed to be asked questions about yourself, with the same memories upon thawing as when frozen, for those moments you weren't frozen would you be "remotely conscious"?

I think you'd think you think, don't you think?

Imagine your hand was in a cast and then you were cryogenically frozen. While frozen your cast was removed. Then you were unfrozen and asked if you could move your hand.

I think you’d be able to answer the question, don’t you?

Is this from a book? If not please write it