[we_never_see_it]

Google is really trying to catch up to OpenAI & MS. The truth is they have never been in the race to begin with. All they had and still have is PR stunts. Let's see if their copying of MS model will produce anything useful.

[thewataccount]

> The truth is they have never been in the race to begin with.

Product race? My understanding is they've been so concerned with safety/harm that they've been slow to implement a lot of tools - then OpenAI made an attempt at it anyway.

Google has generally been ahead from a research perspective though. And honestly it's going to be really sad if they just stop releasing papers outright - hopefully the release their previous gen stuff as they go :/

[endisneigh]

I don't know how anyone can say this with a straight face when Google is the one who invented LLMs as used today to begin with.

Google has a product issue, not an AI research one.

[onlyrealcuzzo]

It's usually the least informed with the most self-assured sweeping opinions.

[cubefox]

DeepMind and Google invented many other things, but I think the first GPT style token predictor was actually ... GPT, a model by OpenAI. RLHF was also invented at OpenAI. They also had the first text-to-image model.

[andreyk]

Curious why you think this? PaLM2 looks great, and Google has been productizing cutting edge AI pretty fast for years.

[MacsHeadroom]

PaLM 2 can't even solve "Write three sentences ending in the word Apple."

It's worse than GPT-3.5. Go see for yourself at bard.google.com, which is running on PaLM 2 everywhere but the EU as of yesterday.

[Garrrrrr]

Ah yes, the famous benchmark for all LLMs. I just tried your novel example with GPT-3.5 and it couldn't solve it either:

> After lunch, I like to snack on a juicy and crisp apple to satisfy my sweet tooth.

> In the fall, many families enjoy going to apple orchards to pick their own apples and make homemade apple pies.

> The new MacBook Pro features a powerful M1 chip and a stunning Retina display, making it the perfect tool for creative professionals who work with Apple software.

[mustacheemperor]

Eh, I think as "human evaluated" metrics go, it's a decent test of how well it can parse a reasonably complex sentence and reply accurately.

For me:

GPT4 3/3: I couldn't resist the temptation to take a bite of the juicy, red apple. Her favorite fruit was not a pear, nor an orange, but an apple. When asked what type of tree to plant in our garden, we unanimously agreed on an apple.

GPT3.5 2/3: "After a long day of hiking, I sat under the shade of an apple tree, relishing the sweet crunch of a freshly picked apple." "As autumn approached, the air filled with the irresistible aroma of warm apple pie baking in the oven, teasing my taste buds." "The teacher asked the students to name a fruit that starts with the letter 'A,' and the eager student proudly exclaimed, 'Apple!'"

Bard 0/3: Sure, here are three sentences ending in the word "apple": I ate an apple for breakfast.The apple tree is in bloom. The apple pie was delicious. Is there anything else I can help you with?

Bard definitely seems to fumble the hardest, it's pretty funny how it brackets the response too. "Here's three sentences ending with the word apple!" nope.

Edit: Interesting enough, Bard seems to outperform GPT3.5 and at least match 4 on my pet test prompt, asking it "What’s that Dante quote that goes something like “before me there were no something, and only something something." 3.5 struggled to find it, 4 finds it relatively quickly, Bard initially told me that quote isn't in the poem but when I reiterated I couldn't remember the whole thing it found it immediately and sourced the right translation. It answered as if it were reading out of a specific translation too - "The source I used was..." Is there agent behavior under the hood of bard or is just how the model is trained to communicate?

[sebzim4500]

I guess PaLM2 is competitive with GPT-3.5 so for people not willing to pay it will be an attractive offering.

I'm not sure that counts as 'great' though.

[rsstack]

Based on what do you think it's comparable to GPT-3.5 and not to 4? Did we see a lot of public performance?

[sebzim4500]

They claim it is already being used in Bard, also if you read the paper it does much worse at the important benchmarks.

[kernal]

OpenAI is the Microsoft Explorer of AI.

[oars]

Google has multiple horses in this race.

They invested $300m in Anthropic in late 2022: https://www.ft.com/content/583ead66-467c-4bd5-84d0-ed5df7b5b...

(Non-paywall: https://archive.is/Y5A9B)