[simonw]

Was this Claude Code or Claude.ai or some other tool that used Claude under the hood?

Different harnesses have different search capabilities.

If I'm doing something that benefits from search I tend to switch to ChatGPT because I know it has a really good search feature available to it. I don't trust Claude's as much.

[Razengan]

I used the Claude website and Mac desktop app for a relatively standard iOS SwiftUI project.

I used Claude Code with VS Code for some Godot stuff, and even there it sometimes gave outdated and outright made-up APIs (functions that seemed like they should exist but did not etc.)

[simonw]

Unfortunately LLMs mostly suck at Swift and SwiftUI from what I've heard - they still change pretty often and as a result there aren't enough fresh examples in the training data.

As primarily a Python/JavaScript programmer I don't have that problem!

[Razengan]

They're terrible at anything new, including knowing about THEMSELVES and their latest versions.

This is me asking ChatGPT 5 about ChatGPT 5: https://i.imgur.com/aT8C3qs.png

Asking about Nintendo Switch 2: https://i.imgur.com/OqmB9jG.png

This could be solved and LLMs could be a lot more useful if they could be a wrapper around live web search: Just search for this shit, scrap the top few results, and summarize the info to me.

But that's a stillborn dream, crippled because Google won't let 3rd-party AIs use their search willy nilly and websites don't want to be scrapped :(

Don't get me wrong: I see the potential in AIs/LLMs and I think they could be amazing for everything, but like every awesome thing, they're hampered by corporate (and government) idiocy.

[simonw]

Claude Code has a neat fix for that - it knows to look at its own documentation if you ask it questions about itself: https://simonwillison.net/2025/Oct/24/claude-code-docs-map/

I've had great results from ChatGPT running the "GPT-5 Thinking" model since that almost always opts to run a search before it attempts to answer a question.

Here's what I got from that for your Switch 2 question: https://chatgpt.com/share/69089028-db8c-8006-b238-1d6946e791...

Screenshot of the searches it ran here: https://gist.github.com/simonw/048ffb895dd6b94419f0b4e066143...

[Razengan]

A month ago when I asked Claude (on the website) about its privacy options and stuff, it always pointed me to the Antrhopic website to look it up myself.

Another annoying example: I thought Google's Gemini would be search-first since, well, they're Google.

I asked Gemini to search for Airbnb rooms in an area and give me a summarized list.

It told me it can't and I could do it myself.

I told it again.

Again it told me it can't, but here's how I could do it myself.

I told it it sucks and that ChatGPT etc. can do it for me.

Then it went and I don't know, scrapped Airbnb or used a previous search it must have had, to pull up rooms with an Airbnb link to each.

This could actually be THE absolute killer app for a lot of people, if AI could plan your trip from a single sentence: "I'm free next week. I'd like to go to A, B, or C for a couple days. What's a cheap flight and a room within this budget near X area?" and if it could also go and make a booking through your accounts it would be orgasmic. Finally we would have what people in the 1960s thought computers would be doing in 2000 :')

But as it is, in their current state you have to wade through quite a bit of dumbassery.