[mistercow]

I agree that the author is too dismissive of that, but I also think people are way too quick to oversimplify to "Bard has up to date information"

From what I can tell, Bard's access to recent information is through augmentation, not continuous training, and that's a really important difference. ChatGPT also has its "web browsing" alpha, which is a similar concept. The problem is that being able to search the web isn't the same as having a piece of knowledge integrated into your model of the world.

So, for example, if you ask contrived questions like "Why might Elon Musk have more time to focus on Tesla soon?", both Bard and ChatGPT+Browsing get a clue that they should search for Musk in the news, and they'll tell you about Twitter's new CEO pick. They'll then apply that competently to your question, and give you a reasonable answer.

But if you ask a question that requires more indirect inference, you immediately see the shortcomings of this kind of augmentation. For example, if you try "Is there hope for LGBTQ rights improving in Turkey?", neither model finds the extremely relevant point that Turkey's homophobic current president stands a significant chance of losing reelection. I can't see what Bard searched for in collecting its information (in fact, based on the answer I got, I'm not even sure it tried to go beyond its training data). I can see that ChatGPT searched for "LGBTQ rights in Turkey 2023". It's not surprising that that search didn't clue it in.

Now, obviously, I can't actually examine what would happen if each model was actually trained on the most recent news about Turkey's politics. But I'd have a very high expectation that GPT-4 would make that connection, and I would be fairly surprised if 3.5 didn't as well. But there's a bottleneck because the information isn't actually integrated.

Which isn't to say that that kind of augmentation is useless, of course. But the distinction is important.

[fakedang]

Honestly, your second question on LGBT rights in Turkey might not be an easy one for any LLM, unless they were fed an entire vault of information and allowed to connect the dots. For one, you would need Turkey's LGBTQ -specific coverage, which is comparatively low compared to coverage on Musk. Then you would need to know the opposition candidate's views on LGBTQ, which isn't so outwardly evident, especially since LGBTQ+ rights is essentially anathema outside Istanbul (or even in it). Then the model (or the individual) would need to be able to connect that knowledge with knowledge of the recent elections.

All in all, it's possible in due time, but very challenging. Even for a human for that matter - I wouldn't know how to answer that question myself, even after Googling about it.

[mistercow]

Well this kind of my point. Questions that are hard for an unfamiliar person to answer even after googling are where these chat LLMs really shine. Having a chat interface to simply search the web for related concepts is not all that useful. It's not totally useless, but there's not much value there if you know how to use a search engine well. "Connecting the dots" and tying the knowledge together is where a lot of the power is, and you don't get that through search augmentation.

[aatd86]

Well, if it starts giving aswers that are conjectures based on non-facts, it will be dangerous as it can now be gamed.

So maybe it's a feature here and not a bug?