[bko]

I've been using ChatGPT a lot for various tasks and found the results very helpful. There are often errors but errors that I would make. With non programming tasks it can sometimes be word association and misstate the concept but it's almost always helpful and again the mistakes are reasonable.

When I asked Bard to list the months in alphabetical order, it failed to do so. I saw this on a tweet but confirmed that it can't sort the moths. Gpt passes. When you fail such a simple comprehension i lose faith in anything more complex where the incorrect results wouldn't be as obvious. I'm surprised they released this in such a state. My guess is they're panicking to reply. I can imagine simple queries like this weren't tested

[gorbypark]

That's pretty surprising it can't do the months in alphabetical order. I just tried with LLaMA 30B. This is running on my MacBook Air with llama.cpp, quantized to 4 bits, and is a "raw" model (not alpaca / instruction tuned at all). Obviously some models will be better at some things (and will have different outputs given the same inputs) and worse at other, but you'd think Bard would be able to do that pretty simple task.

> Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.

User: Hello, Bob.

Bob: Hello. How may I help you today?

User: Please tell me the largest city in Europe.

Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.

User:Can you list all the months in alphabetical order?

Bob: Yes. January February March April May June July August September October November December

edit Sorry for this useless post...months are in order, not alphabetical order.

[schroeding]

> Bob: Certainly. January, February, March, April, May, June, July, August, September, October, November, December.

> User: That's not in alphabetical order. Redo.

> Bob: Sorry about that. Here are the months again, this time in alphabetical order: April, August, December, February, July, June, March, May, November, October, September.

(Also LLaMA, 30B, llama.cpp on x86 Processor)

At least LLaMA sees its mistake and doesn't gaslight you (in this case). "I've been a good LLaMA." ;P

[jlarcombe]

er... "You missed out January, try again"?

[schroeding]

Ha, I didn't see that! :D

> User: Almost, but you missed January. Try again.

> Bob: Okay. Let's see...January, February, March, April, May, June, July, August, September, October, November, December.

Very brittle.

[ergl]

Both answers are wrong though?

[singularity2001]

gpt4:

months in alphabetical order

Here are the months in alphabetical order:

April August December February January July June March May November October September

[crazygringo]

I've never seen the months listed in alphabetical order before.

And it's absolutely breaking my brain. Which is a reaction I did not expect.

It looks so wrong.

[sorenjan]

It's a bit like the Stroop test.

https://en.wikipedia.org/wiki/Stroop_effect

[mcv]

I wonder if that's why it didn't want to put them in alphabetical order. There is only one valid order for the months.

[xpil]

Same here

[sixtram]

A google search returns about 10,200 results for this, gpt4 probably picked up this sequence as well during the training.

[schrodinger]

Seems like it. I asked it to sort by their second letter and it failed horribly:

Sorting months by the second letter, we get the following order:

1. April 2. January 3. June 4. July 5. March 6. May 7. February 8. December 9. November 10. September 11. August 12. October

[franze]

Months are not in alphabetical order.

And largest city is an ambiguous answer: per area, moskow is, by population its istanbul.

[gorbypark]

Oh man I am crazy, it spit out the months in order and not alphabetical order. Wow. I'll go home now.

[anonyfox]

> My guess is they're panicking to reply

this is what I don't believe anymore. Its strictly inferior and people should stop looking for defensive arguments and all that.

If it is simply a smaller/cheaper/less parametrized model than the competitor ones thats a fine explanation, but also admits being inferior in exchange for being cheaper to run.

[hgsgm]

"they launched a worse product out of desperation" isn't a defensive argument, it's an indictment.