literally, how does anyone find this acceptable? It makes me feel dumb that I find this outrageous, like the emperor's new clothes: y'all can see this too right?
My strong suspicion is that, for, say, the CEO of Google, this is less about "this will improve the product", or even "LLMs are the future, so we should use them in the present even though they don't work", and more "this is what the markets, which are collectively kinda dumb, have decided they will reward this year, so we will appease them until such time as they move onto a new toy."
See Our Lord and Saviour the Blockchain; for a year or so practically every company was announcing some sort of blockchain thing. Did any of these come to anything? Of course not, but that was, when it comes to it, hardly the point.
LLMs are a particularly dramatic example, but I suspect the dynamic is, in reality, more or less the same as metaverses, blockchains, the _previous_ AI bubble (remember the year or so when everyone was announcing chatbots, until Microsoft Tay kinda scared everyone off it abruptly?), and so on.
Many in the tech community have high expectations for Google engineering quality and product development. The expectations are now too high, much like the executives who pushed this feature.
> Many in the tech community have high expectations for Google engineering quality and product development
Wait? Whom has these expectations? I presumed most in the tech industry scoff at anything Google launches, point at Google Graveyard or at all the failures they've been through.
Seriously: aside from search, ads and android, what product has Google developed ever that is of high quality and meeting the high quality? Maybe maps at some point? Gmail, maybe? Google docs? For each and every google product, I can name two competitors that meet much higher expectations of engineering.
Really the only 100% in-house products are Search and Gmail. The rest was bolted on piecemeal into the Borg: maps, android, double click, docs, YouTube, etc. - all acquired.
That’s not a bad thing, Google is really a story about a shrewd - even brilliant - acquisition and integration strategy.
However, they have systemically failed to grow new things in-house - be it space balloon internet, being an ISP, games network, etc etc.
They are likely better off just buying Anthropic at this point.
I think when it comes to success Google's products find success with either (or both) of the two factors.
1. Excellent integration with rest of the google ecosystem which ends up me and my friends using it and it is good enough so we continue using it. (E.g. Chrome, Photos, Drive, etc.)
2. Products are genuinely better and leagues ahead of nearest competition. (Gmail, Docs, Youtube).
I suspect Gemini is relying more on 1 rather than 2 at this point as it is not clear if it is indeed the best. But if it can play well with other Google products I am happy.
For example, I have Google One 2TB subscription to pay for all the storage and extra features, result of which I get Gemini Advanced for free and it also works in Docs. I rarely have the need to open ChatGPT even if ChatGPT is better than Gemini.
Search is where I am somewhat skeptical but this experiment is totally worth it in my opinion in Google's perspective and not doing it is stupid. Human beings adapt to the error rate of the Gen AI over time. So unless someone else delivers say 5x better accuracy than Gemini, I think it wont matter.
I am well aware of the situation, hence me searching her name.
But that doesn’t explain the nonsense word salad. I would actually be very skeptical to assume that these were taken verbatim from searchers.
You would have needed A LOT of people to have searched that exact phrase for it to be added to PAA.
Google does a good job with People Also Ask, and I have never seen this before. It might be nothing, but I am leaning towards one of their NLP systems leaking.
It's a great service for providing the wrong answers to things I didn't search for, but I admit I haven't yet figured out why I am supposed to desire this.
Yeah that sentence would literally be caught by a rudimentary grammar checker. This is what happen when you apply YOLO-driven design decisions on the basics of your core product.
Can you share the prompts you used to generate these results? I'm having a hard time replicating just by guessing what they were (full disclosure I work for Google but not on this)
It is frankly astonishing how poor Gemini continues to be in tasks even suggested answers, Assistant and in part, Google Now covered in the past.
Mistral, OpenAI, Anthropic and Meta all have their unique advantages and I have yet to find one LLM being superior in every task I use them for, but purely for such simple questions that should be in the training data, few are consistently as poor as Google efforts have been, first with Bard and now Gemini.
It doesn't matter. Most people are using Copilot because it's vetted,
"protects your data", and is paid for with office 365 (or whatever it is called now).
Google should have run at the enterprise when it had the chance fifteen years ago.
The answers aren’t deterministic. Which is part of the problem. While searching for these now may give you the right answers, I have personally encountered this level of bad responses enough times that none of these are surprising.
There’s no need to doctor screenshots to get these responses, at least twice now Google has shown bad answers in their own marketing materials.
It pretty much was deterministic before Google started using ML to "help" interpret your search queries, which was a decade or more ago. Not coincidentally, that was also when Google search results started getting noticeably worse for me.
Do you think anyone at Google even feels guilty or responsible about shipping this garbage or will they still receive a big fat bonus for shipping something? I understand that Google is probably under pressure due to all the AI-hype, but surely there must be someone on the inside that could call out this trash for what it is? Has the organization become so fully captured by psychopaths such that common sense is no longer able to prevail?
The hierarchy is too strong. Subordinate Googlers cannot really challenge anyone in their leadership chain.
That leaves Siblings to challenge each other, which doesn't really happen either. (VPs either can't check other VPs, or don't give a f** about doing so)
They just take the money at this point, I don't blame the low rang employees too much. Google isn't what it used to be and is not inhabited by the same spirit of the early days, most of those employees are long gone, the culture has changed, etc...
Jesus, that kidney stones one is bad :( It's funny when the LLM totally checks out of reality, but not at all funny when the statement seems plausible to ignorant people.
(A lot people seem to subscribe to an ideology of "dumb people get what they deserve." What this really means to me is "I have Dunning-Kruger syndrome," but I wonder how much of that gets filtered down into making excuses for AI that sucks so badly it becomes actively dangerous.)
If I already know the answer to something, why would I be looking it up? These answers from AI should assume the person on the other end doesn’t know anything and isn’t going to fact check. If they were going to fact check, they’d just look it up in the first place and not waste their time with AI.
To me, a real sign of intelligence and maturity is being able to say, “I don’t know.” I always lose a lot of respect for people when I catch them making up answers to questions they can’t answer. It means I can’t trust anything they say. If I’m not willing to accept this behavior from a person in my life, why would I accept it from a machine?
If you have a question about the answer have the AI review it to make sure it's correct. I have gotten it to pull it's head out of its ass numerous times. It's like a conversation with a human.
It's easy to hold it wrong if you believe things you see in print.
This is just completely disconnected from my original comment. What if someone reads "drink fruit juice to clear up kidney stones" from an AI and doesn't have a question about the answer because they don't fully understand what kidney stones are? The only response AI advocates seem to have is "not my problem." Or, far too often, the vindictive irresponsibility of "it was his fault for being so stupid that he trusted us."
"Flounder, you can't spend your whole life worrying about your mistakes. You fucked up! You trusted us! Hey, make the best of it... maybe we can help you."
