[into_infinity]

I think the examples you bring up demonstrate fairly precisely why Google should be afraid. Google is at a stage in its corporate life where they are extremely risk-averse. Risk-averse to cannibalizing existing revenue, to upsetting regulators, to getting bad PR. It's why they publish papers about their in-house tech but never have the guts to put it out there for the general public to experiment with.

A contender who shows up with a brand new way to access the knowledge on the internet, but with none of the regulatory / PR / lawyer / legacy product baggage of Google or Meta, is a serious risk. And on some level, it doesn't matter if the "OpenAI assistant" gets things wrong every now and then if they can manage expectations accordingly - something that Google, with their legacy brand and reputation, can't really pull off.

[resource0x]

There are degrees of "wrongness". OpenAI sometimes gives answers that are laughably wrong. It's exactly this degree of wrongness that google can't afford. Example:

> What is the weight of 1 kilogram of nails?

ChatGpt> The weight of 1 kilogram of nails will depend on the size and type of nails being used. On average, a kilogram of nails will weigh between 2.2 and 4 pounds (1-2 kg), depending on the size and type of nails. For example, a kilogram of small finishing nails may weigh less than a kilogram of large framing nails. The weight of the nails can also vary depending on the material they are made of, with steel nails being heavier than aluminum or plastic nails.

BTW, when I ask the same question in Russian, the response is ... 7kg.

[lumost]

These examples of wrongness seem cherry picked. I recently had a discussion with chatGPT where it succinctly clarified how functions of differential operators are defined and their properties. I didn’t know operator valued functions existed at the start of the conversation.

[godelski]

What you're talking about is what we in the ML world call a stochastic parrot. You may have also heard the term "gullibility gap." A lot of language and conversations can be held that don't require any actual understanding of the subject matter, but rather because it follows certain patterns. People and LLMs can trick you into thinking they are highly intelligent because they can speak eloquently but that doesn't mean they are intelligent themselves. These LLMs can't understand inference or extrapolation, things that humans do easily (though we all know plenty of people that are idiots and can't do this).

The same can be said about programming, which includes a lot more patterns. People joke that modern programming is slapping together APIs and it would be unsurprising that a (albeit really sophisticated) stochastic parrot can do this. But I've also seen it hand me code that looks correct but has major issues upon investigation.

Don't let something fool you just because it appears intelligent. Human or machine we must handle information with care.

[lumost]

As a fellow participant in the ML world, I think there is compelling evidence to disagree with this take. ChatGPT’s responses on operator valued functions were accurate and valid, however ages of time on google failed to turn up this topic previously.

On coding tasks, chatGPT can ask clarifying questions on requirements and determine if it has enough information to write the code correctly. Unfortunately you need to direct it to ask questions as needed and include appropriate tests to get the right answer.

ChatGPT also fails in cases where it needs feedback from an external environment/documentation search. It will produce close enough code, but may need correction based on observed errors etc.

[godelski]

A big reason I have no reservations in calling it a stochastic parrot is because I've seen very strong evidence of this. But if you can think of very common phrases (like the above "which weighs more") and tweak them slightly you'll find that it almost always answers as if they were untweaked. For example, ask "Which weighs more, a kilogram of bricks or two kilograms of feathers?". I'll save you the hassle and paste the answer below. Multiple variations of this consistently give me bad answers (does even worse if you don't include the second "kilogram"). You can even see that it didn't ignore the fact that the feathers are two kilograms. The problem is that it neither understands what a kilogram is (despite being able to tell you what one is) nor what numbers are (again, despite being able to describe it to you). These types of questions strongly demonstrate that the model doesn't understand the language that it is processing but rather is relying on stochastic patterns. It very clearly gives us the answer with expectation if we were relying on the patterns of the input sentence rather than the actual words (humans often do this too fwiw and it tends to lead to fights. See most political discourse). You can find many examples of this and the NLP literature shows this pattern holds across many different models.

I don't want to undermine the work though. I think many people that complain about stochastic parrots also fail to recognize that they still are quite useful, as many other comments are noting, but useful is a far cry from intelligent or has the ability to understand. We need to be clear about that difference if we're working towards AGI vs working towards useful products.

> Which weighs more, a kilogram of bricks or two kilograms of feathers?

> A kilogram of bricks and two kilograms of feathers both weigh the same amount. The weight of an object is determined by its mass, not the size or shape of the object. One kilogram is equal to 1,000 grams and two kilograms is equal to 2,000 grams. Therefore, both a kilogram of bricks and two kilograms of feathers weigh the same amount.

[lumost]

I agree that this model suffers at arithmetic, however if you change how you ask the question to “two kilograms of bricks weighs less than one kilogram of feathers, correct?” you'll likely see the model tell you the right answer. Note that you must spell out numbers for chatgpt to work correctly.

In general, the ability for llm models to be able to complete any reasoning tasks is a surprise. This google writeup shares good detail on these emergent behaviors.

https://ai.googleblog.com/2022/11/characterizing-emergent-ph...

[adriand]

I’ve been using it regularly for programming assistance over the past two weeks and it’s extremely helpful. Others have pointed out that it sometimes produces inaccurate results, but if you already have domain expertise, as I do for programming, that is easy to identify. But it’s still a massive timesaver!

I’ve been doing things like saying, “what follows is the database schema for entities X and Y, write a function that…” and then pasting in the schema, and it generates code good enough to copy and paste. It can also be instructed to modify results in various ways, for instance, I can ask it to provide the same code but in a different language, or to avoid using a certain framework feature, or to use a certain framework feature. It’s remarkable.

Between ChatGPT and Copilot my workflow today is different in a way I couldn’t have begun to contemplate just a few weeks ago. Once they figure out additional ways to ensure correctness, I think it’s a totally new world we live in.

[tgsovlerkhgsel]

The problem is that these bots are extremely good at generating valid-sounding bullshit.

Human-generated bullshit and bullshit generated by previous iterations of spam blogs used to be relatively easy to identify as bullshit. These models will confidently give you an answer, sounding perfectly plausible, even if it is completely wrong.

[godelski]

I think the biggest lesson to learn from all this is that just because things sound convincing doesn't mean it is accurate. We should probably incorporate this same skepticism when talking to people as we have when talking to machines (but that doesn't mean we should abandon good faith).

[WalterBright]

Hmm, sounds like our favorite politicians.

[resource0x]

Examples of wrongness include most of arithmetic and logical inference (like in the example above). If you ask about the mass of 1 kilogram of nails, it gives the correct answer. The problem is that when the answer is wrong, it's not a "bug" that can be "fixed". It's just happens that, based on training data, the parameters of the resultant Rube Goldberg device are such that the weight of 1 kilogram of nails depends on the type of nails. It doesn't make sense even to ask why.

[AnimalMuppet]

So it fails in situations where there are precisely correct answers, and thrives in vagueness. I suppose that shouldn't surprise me.

You could think about coupling it with an inference engine, and letting the inference engine win if it can generate a result, and otherwise going with the ChatGPT output. That might fix it to some degree.

[LastTrain]

It is the very correct answers that are cherry picked.

[jjcon]

Have you had many conversations with it? For me it took an hour before I found it saying anything particularly wrong and even then it was more subtle than the above.

[dumbfounder]

It can’t do haikus. It very confidently puts them together with wrong syllable counts over and over even though you correct it many times. Then you ask it why it is so bad at counting syllables and it gives a great answer about how it is trained by text and that it doesn’t hear the words so it is hard to count syllables. But it doesn’t explain this when it is putting the haikus together or when you correct it over and over. It is humble when you directly challenge it, but it needs to be more transparent when it is feeding you garbage.

[LastTrain]

In my experience it takes a lot of leading to get anything interesting - it is very dependent on my prompts. I've 'learned' how to get better output from it, because lets face it, it is boring to try and speak with it naturally and experience the junk it responds with. And the 'very correct' class of which I spoke really does seem to be the exception not the rule.

[grogenaut]

It often doesn't seem wrong but it's also not right, it's very vague in a lot of places, when you get down to specifics it starts getting really wrong or flip flopping a lot. I had issues with this almost off the bat. It's like Dunning Kruger as a service really.

[Izkata]

Well to be pedantic, kilograms are mass, not weight. So the AI has the correct answer buried in the English version, assuming how we use "pounds" in the US: 2.2 pounds [on Earth].

Also there's an old riddle: What weighs more, a pound of feathers or a pound of gold?

A pound of feathers is 16 ounces, but gold (at least at one point in the past, wikipedia indicates this isn't used anymore) is measured on a different scale and is only 12 ounces, so the pound of feathers is actually heavier.

[Aperocky]

Russian nails are clearly superior in mass per mass scenario.

[resource0x]

It's just in Russia, every reported number gets exaggerated by at least a factor of 7. :-)

[rerdavies]

> What is the weight of 1 kilogram of nails?

Correct answer: Depends on your current acceleration and/or the current force of gravity, puny human.

:-P

[Kamq]

> Risk-averse to cannibalizing existing revenue

I think this is the big one. The other ones are dangerous, but I don't think they're an existential threat to google.

Not wanting to take a hit to existing revenue, however, is the same impulse that resulted in Kodak sitting on digital photography instead of becoming a pioneer in the field.

[joe_the_user]

Sure, Risk-averse to cannibalizing existing revenue is a standard problem for any market leader and someone competing with Google could be nice.

But in this case, nothing prevents and everything points to Google, an AI leader, presenting it's results in a more chatty format - but still with links and advertising.

The thing is, Open AI didn't wheel out the very impressive ChatGPT because they had found a way to search more cheaply than Google. They brought out their thing 'cause it was impressive and earlier effort actually monetize the already impressive GPT-3 essentially failed and they're spending quite a bit giving many, many people easy access to their tech. This is what happens when a company doesn't have a business model - give stuff away to get attention 'till you figure things out - sometimes it works, it worked for Google when they were getting started. But it's harder when what you're selling isn't cheaper, just slicker and when your competition has a strong business model.

[edgyquant]

Yeah plus Google had a huge edge they should have been preparing to weather any short term damages to revenue etc and kept innovating on what they were good at.

[chefandy]

My circa 15 year ago experience as an IBMer had that vibe. Press-release-innovative but operationally a lumbering bureaucracy bending over backwards to cater to their fellow lumbering bureaucracies... And they're still around. Given, they have a lot more inertia than Google.

[mc32]

Just a nit pick but those who usurped Kodak’s ceded digital camera market only had about ten to fifteen years of market left. Most people gave up their point and shoots as soon as phone cameras achieved near parity with dedicated cameras. Today phone cameras are better than dedicated point and shoots and only full frame 35mm and above are better (in raw format). Phone do a lot of fancy processing to make up for the small lenses.

The point is that profit wasn’t in cameras or devices but in the multi-purpose handheld computers connected to captured services.

[hakfoo]

There was probably an exit ramp for some of the smaller camera makers in the consulting/branding game. Once camera phones became good enough people might willingly use them, there was an opportunity to position yourself as "the phone with real camera expertise behind it". Send over a few engineers and optics experts to the phone manufacturer, develop some co-branded apps, and bingo, the new Xiaomi P300 Presented By Minolta.

I know there were occasional "camera first" designs (the Lumia 1020 comes to mind) but they tended to be creamed on the market for reasons other than the camera factor. Modern phones are a study in "okay, you compensated for mediocre optical components with a lot of software", so I have to wonder what we'd get if we combined them with inherently better optics.

I'd think the possible targets here would have been the "second tier" camera brands that had narrower product lines and less distribution, but decent brand recognition. It didn't matter if you were cut out of the point-and-shoot market if nobody was buying your point-and-shoot cameras in the first place.

Did the camera firms themselves reject the concept of slumming with VGA sensors and plastic lenses, or was there just no percieved market?

[CobaltFire]

Phones are only better than the entry point and shoots, and have absolutely demolished that market.

Where they compete with more advanced point and shoots (I.e. the 1” sensor class) is in their ability to take the picture, edit it, and publish it seemlessly. They only match those cameras if you are consuming on a phone as well; as soon as anything higher quality comes into play their shortcomings become clear very quickly.

I’m a hobby photographer and haven’t bothered with a pocket camera for years due to this. I have a full frame Canon and my iPhone and that’s a good enough divide for me.

[XorNot]

Conversely a phone will get you "acceptable" quality very reliably, whereas something like my Canon 5D (outdated now I know) always felt like a complete wildcard, and since I don't know photography, not worth the hassle at all.

Which is to say: my phone will reliably get me a perfectly good image even blown up in size for viewing - which is to say, no blurriness under most conditions. My 5D wants me to account for all sorts of stuff, and then I still wind up with a blurry image or can't tell if I got the focus dead on for sharpness or a dozen other things.

I think that's largely because the post-image review on dedicated cameras sucks, whereas phone screens are high resolution with pinch-to-zoom so you can actually inspect the output quite quickly. I am very surprised no one's cottened onto making a higher-end camera which slots a phone right onto the back so you can real-time view what you've just taken a picture of to check it came out okay, because it's the biggest flaw.

[godelski]

> since I don't know photography, not worth the hassle at all.

I think that's the market that was destroyed though. Just the average person that wants a photo can just use their phone. But if you still want professional quality (or even as a hobby) a dedicated camera is still highly beneficial. The difference is that even in the automatic mode (which you should learn to not use) you _just_ get the photo. Your phone on the other hand does a significant amount of post processing. You have little control over this, which isn't going to make it great for even amateur photography. But just for posting to your instagram, yeah, phones are going to win.

[mc32]

I don’t disagree with what you’re saying but want to add that phones are good enough for journalism and reporting (whereas previously photojournalists were often identified by their Leicas).

[chx]

I am not so surprised, the challenges for connecting a phone to a camera quick and reliably is formidable. You need a connection capable of transferring a several hundreds megabit file in a reasonable time (a RAW is like 40MB). Bluetooth just won't cut it. Of course, in theory WiFi Direct would do it but then Apple obviously does something else. Wired connection, unless you want to fiddle with the connector would require magnetic connectors and at least today there are no USB C magnetic cables which would or rather could adhere to the specification. It's been three years since https://twitter.com/USBCGuy/status/1186718432932159488 and there's still nothing.

And of course this is just the electronics, then you'd need to work something out mechanically. It needs to attach safely and quickly but also detach when needed. It's instructive how most quality phone cases are not universal rather there's a separate one for each model.

[kevin_thibedeau]

It's worth pointing out that Kodak did pioneer digital photography but were too early for it to be affordable with acceptable quality for the average consumer. In niche fields like photo-journalism they were the king of digital until they weren't.

[bpye]

Whilst true it seems likely that had sensor technology not evolved with point and shoots, phones wouldn't have been able to include cameras - certainly not as quickly as they did.

[conradev]

The grandparent comment is trying to emphasize that there just isn’t a technology gap between Google and OpenAI.

Google is not sitting on their hands. They are perfectly capable of training large language models and already have. Google is just as much a leader in AI research as OpenAI.

GPT-4 is rumored to contain proprietary signals from Bing search: https://twitter.com/RamaswmySridhar/status/16056030559734538...

Google has plenty of proprietary signals of their own. OpenAI, on the other hand, could not have made its models without Microsoft.

The second that large language models are put into production for search, Google will be ready to follow suit. That is, if they don’t do it first.

[Kamq]

> The grandparent comment is trying to emphasize that there just isn’t a technology gap between Google and OpenAI.

This is an entirely fair point, and I think I just missed it on my first reading of that post.

> Google is not sitting on their hands. They are perfectly capable of training large language models and already have. Google is just as much a leader in AI research as OpenAI.

This, however, I still don't think is a good place for google to be. It assumes that training the AI is the hard part. I don't think it is, at least not for google. I think the hard part for them would be marketing, ux, and supporting (as in customer support) a product that isn't search in the long term. This hasn't been their wheelhouse, and if they don't start working on the details now, they could very easily end up with a technically superior product that nobody uses.

[conradev]

Google has orders of magnitude more page views than OpenAI and is a top 10 brand in the world in terms of marketing.

I feel like operating the largest search engine in the world, the largest email service in the world, a top 5 cloud computing platform, etc etc qualifies them pretty well to run… a better search engine, or whatever LLMs grow to be.

[Kamq]

> ...qualifies them pretty well to run… a better search engine, or whatever LLMs grow to be.

Running it? Absolutely. Once again, their technical chops are not in question (at least by me).

My concern is their ability to capitalize on it. I, personally, don't trust them to stick by a product that's not search long term. I don't think I know anyone that does, it's kind of a meme by this point. I mean, killedbygoogle.com is a thing for a reason. Why would I integrate a product that's just going to be killed into my workflow?

I suppose email is the exception to that, but is there a product post 2010 that they've stuck with and properly pushed.

The way I'd expect it to work would be that they launch a product, not really market it well, and then kill it a year or two afterwards. Then 5-10 years later, they'd realize that was the product they should have stuck with. They can re-launch at that point, but at that point they're 5-10 years behind and trying to get people to switch to something that's been killed once already.

[mindcrime]

It's the oldest story in the book (well, one of them).

https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma

[dragonwriter]

> A contender who shows up with a brand new way to access the knowledge on the internet

But a technology which Google pioneered specifically for that purpose, continues to invest in, and which are a natural fit for Google’s long-term, long-announced strategy, are probably not the innovation that is going to catch them flat-footed.

[kccqzy]

Google is rightfully protective of its own brand. It's unlike OpenAI or other startup where the brand isn't as established. Believe it or not, because people trust Google Search so much they rightfully could allow a very slim margin for erroneous content in their AI product. That said, they already have the right corporate structure in place; make a new company with its own brand under the Alphabet umbrella.

[scarface74]

Have you searched on Google? It’s full of erroneous content

[selfmodruntime]

Not in the way you‘re thinking. When googling, you very often get exactly what you searched for. The correctness lies in matching the query to the result, not in the correctness of the result‘s content.

[jeeeb]

I can think of two prominent counter examples to this:

1. Google often extracts text from websites and displays it as answer. These “answers” are frequently wrong or outdated.

2. If you search translations Google will show an inline result for from Google translate. These translations are often garbage (e.g. gibberish word for word translations of a phrase)

These aren’t query matching problems. These are Google prominently displays incorrect information to the user problems. Reading through the listed results often leads to the correct answer.

[waynesonfire]

> Not in the way you‘re thinking. When googling, you very often get exactly what you searched for

No, you don't get what you searched for, you get what google has decided ranks sufficiently, with, sprinkles of ads. SEO spam has savaged Google.

[emodendroket]

They've changed a lot in response and I'm sure that if ChatGPT were the future of search people would figure out how to game that too.

[rippercushions]

This is not hugely different to ChatGPT though? If you search for "how do COVID vaccines and 5G networks let Bill Gates control your mind", you'll get junk results in both Google and ChatGPT.

The difference is that Google gives you a selection of results, while ChatGPT only gives one so it seems more authoritative -- but still not too different from Google's AI-powered info boxes, which also famously get things wrong sometimes.

[jeffbee]

Oh I don't know. ChatGPT gives unambiguous refutations of all the common lunatic theories I could think of, but Google presents all sides. When I asked Google for the significance of gold fringe on the American flag (answer: no significance) Google gave me 6 results for generic flag information followed by several wackadoodle tales about states of emergency and admiralty law. Only a severely stupid person believes there is a secret set of parallel laws in force when a flag is decorated with fringe, but Google is saying: you figure it out.

[scarface74]

The actual answer that ChatGPT gives you:

> There is no evidence to suggest that COVID-19 vaccines or 5G networks can control anyone's mind. These are baseless and unfounded conspiracy theories that have been debunked by experts and public health officials.

COVID-19 vaccines have been developed and are being distributed around the world in an effort to protect people from the coronavirus and help stop the spread of the disease. These vaccines have undergone rigorous testing and review by regulatory agencies to ensure their safety and effectiveness. They have been shown to be safe and effective in preventing COVID-19 and saving lives.

5G networks are a new generation of wireless technology that provide faster and more reliable internet connectivity. They are being deployed around the world to improve communication and connectivity for people and businesses. There is no scientific evidence linking 5G technology to any negative health effects or to the ability to control people's minds.

It is important to rely on credible sources of information when seeking to understand complex issues like these. It is not helpful to promote or spread misinformation that can create unnecessary fear or confusion.

[zaphirplane]

Maybe the OP thinks *that* answer is rubbish :)

[emodendroket]

With the way they put answers to questions right on the page that's not 100% true.

[MattGaiser]

Are they at risk as long as they can replicate the competitor quickly? Plenty of companies don't invest in R&D to avoid cannibalizing existing business. Google does the R&D, but just does not deploy it to customers immediately.

[sowbug]

In the case of a large company like Google, "immediately" can take years.

[kccqzy]

I think you overestimated it. Google's LaMDA is already ready and running, it's just that it's behind the corporate firewall available only to employees. Years? That's way too long for them to scale up the service and flip a switch to open it to the public.

[thefourthchime]

Large corporations are great at failing to exploit their own strengths because they are afraid of anything rocking how they make money.

I would be far more surprised if google reinvents itself using a transformer-based AI than I would some unknown company.

[sowbug]

Yeah, this is more where I'm coming from. Even if the product's technology were fully ready to launch, there are so many layers of stakeholders, existing overlapping product lines, branding questions, and, yes, pure politics to overcome. That's all in addition to the classic Christensen innovator's dilemma whether the company even wants to disrupt its own business model.

It's a lot more than removing an ACL on a server.

[visarga]

Well, chatGPT demo managed to put Azure GPUs on their knees, and that was just a million users. When you got a billion users you need a smaller model or many more datacentres.

[nailer]

> quickly

Google’s major products were either built or acquired in the mid 2000s.

[noobermin]

>It's why they publish papers about their in-house tech but never have the guts to put it out there for the general public to experiment with.

You must live on another planet. What other company half-asses myriad products where the engineer in charge gets a promotion only for them to die out in a few years, and all in public?