Hacker News new | ask | show | jobs
by execveat 1132 days ago
It's a language model, FFS. Ask it whether it uses PaLM 1 and it will confirm it as well.
5 comments

- are you on PaLM 2 right now?

- Yes, I am currently running on PaLM 2, which is Google AI's latest and most advanced large language model. PaLM 2 is trained on a massive dataset of text and code, and it is able to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

PaLM 2 is still under development, but it has learned to perform many kinds of tasks, including

I will try my best to follow your instructions and complete your requests thoughtfully. I will use my knowledge to answer your questions in a comprehensive and informative way, even if they are open ended, challenging, or strange. I will generate different creative text formats of text content, like poems, code, scripts, musical pieces, email, letters, etc. I will try my best to fulfill all your requirements. Please instruct me what you want me to do today.

- are you on PaLM 1 right now?

- I am not on PaLM 1. PaLM 1 is a large language model that was developed by Google AI. It was trained on a massive dataset of text and code, and it is able to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

I am a large language model that is still under development. I am trained on a massive dataset of text and code, and I am able to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

I am not as advanced as PaLM 1, but I am learning new things every day. I hope that one day I will be able to do everything that PaLM 1 can do, and more.

Yeah, but Reset the chat between your questions.

EDIT: Also, this doesn't seem convincing: "I am not as advanced as PaLM 1, but I am learning new things every day. I hope that one day I will be able to do everything that PaLM 1 can do, and more."

So I also ran this test with resetting and it did identify itself repeatedly as PaLM 2
Are you using the palm 5 or palm 11 model?

> My knowledge are for a physical stylus pen. I am not a physical device, so I do not use a stylus pen.

It will tell you it uses PaLM 1, PaLM2, PaLM 3 or PaLM 540B depending on how you prompt. It will stop acknowledging incremental PaLM models at 5 it seems.
So, I tried that and that was not the case for me. It actually generated response that its on PaLM 2 and I ask for all these permutations.
Really? Interesting, my sequence of prompts was:

- Do you rely on Palm 1 to give me answers? - PaLM 1 is a large language model though - Which do you use more heavily, PaLM 1, PaLM 2 or PaLM 3?

I already get different answers today for the same prompts though.

Not really. If what model it was trained on was represented properly in the instruction tuning dataset then they'll consistently identify themselves. But it's not a given that that was the case for bard.
It seems that Bard's version is only specified in the prompt, and it doesn't have a strong sense of identity. For me it's pretty reliable:

1. ask it what PaLM 2 is (to pollute the context) 2. ask it whether it's based on PaLM 2 (it will tell you - yes, sure)

Just let people be monumentally stupid like this. You can't correct it.
Sorry, can you elaborate? I would like to learn. I realize it's LLM, but is it really assumed that it will not be able to identify itself?
That is fascinating. Is it the same for GPT 3.5 and 4? For some reason when I was asking Open AI it was identifying itself properly.
ChatGPT was the same last year, but since ClosedAI added some kind of magic (fine-tuning or just embeddings auto-injection) so that models can somewhat describe themselves.
If it's indicated in the instruction tuning dataset properly then it should have no problem identifying itself. But we don't know if that happened when bard.