Hacker News new | ask | show | jobs
by honeybadger1 1199 days ago
No, I do not mean that. I provided it the URL and then asked it to use the contents of the CPU table from the invoke-request to strip away everything after the X and it not only did it, it even provided the example output of the array it modified.

Here is an excerpt from my chat:

" can you please modify the current array using the Microsoft listed CPUs in totality from the URL https://learn.microsoft.com/en-us/windows-hardware/design/mi...

Certainly! Here's the updated $compatibleCPUs array based on the list of supported Intel processors for Windows 11 version 21H2 from Microsoft's documentation:

"
2 comments
simonw 1199 days ago
It hallucinated the list of processors based on the URL.

Here's an experiment I just did to prove that it hallucinates content based on URLs: https://fedi.simonwillison.net/@simon/109998914078891574

Or try it yourself: Try these two prompts in separate ChatGPT sessions:

> List the processors on this page: https://bit.ly/422t51T0

Then:

> List the processors on this page: https://learn.microsoft.com/en-us/windows-hardware/design/mi...

That Bitly link is one I just created that redirects to the Microsoft page.

link
honeybadger1 1199 days ago
Interesting, and certainly convincing. I have a paid account, I wonder if this makes any difference? It would make sense to me that it is basing it's 'reality' off of the prevous 21h1 page(which would have existed) and it understands the goal I am trying to achieve.
link
simonw 1199 days ago
I have a paid account too. I'm very confident that it doesn't fetch URLs, despite doing an extremely convincing impression of being able to do so.

Try changing a few characters in that Microsoft URL and see if you still get the same results.

link
simonw 1199 days ago
Wrote more about that here: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...
link
IanCal 1199 days ago
It's not accessing that page live (or recently cached). That's not part of how it works. It's possible the model was updated and has learned from that, but it's also very possible it's just made up the answer. The old model cutoff was 2021 I think, I don't know if/what newer training has been done.
link