Hacker News new | ask | show | jobs
by andai 31 days ago
When Bard (now known as Gemini) first came out in Europe, I think mid 2023, I tested it out. AI search was still a new thing in those days, and I was excited to see what Google's solution would be like. I had high hopes.

I asked it a question I knew the answer to. It searched the web, and told me the opposite of the truth. (Not nonsense, but a logical inversion of the actual fact. A common failure mode with earlier LLMs.)

Puzzled, I checked the sources. It cited two. Both AI SEO slop.

Bizarrely, I Googled it myself and couldn't even find those pages on Google. Maybe it was using a different search engine? ;)

1 comments

Fundamentally, yes, it is a different "search engine."

BTW, as critical as I can be to AI, using an argument that something didn't work 3 years ago, so it must be crap, doesn't work in this context. 3 years ago, AI could barely generate several lines of consistent code. Now, it generates working apps with a prompt (it's another discussion how good the code is, but still).

I guess 3 years ago, Gemini couldn't tell how many r's are in the word refrigerator.

Same for research. At some point, I switched from ChatGPT and Gemini to Perplexity as it promised AI-powered search. It worked visibly better. Until it didn't, as GPT and Gemini models made a leap.

Back to the point, as long as we understand that, for now, it's all just a probabilistic machine generating the most likely output, no one should expect bulletproof answers. Search was/is way more deterministic than LLMs.

> 3 years ago, AI could barely generate several lines of consistent code

Nah, not really. The same summer I made a self-modifying AI programmer. Or more precisely, GPT-4 made it. I copy pasted the classes from the chat one by one.

(I also told it what was wrong with the previous attempt at such a system, so it designed around that.)

That model was also scary good at deobfuscation.

I think it did have that issue about randomly saying the opposite of facts though (the fact but stated in the negative for some reason).