[talim]

> I have asked Kagi and cGPT to summarize my own post which Kagi did rather well (I don't have a screenshot), while cGPT spilled utter nonsense

> https://wojteksychut.com/pic/chatgpt_post_summarization.png

ChatGPT currently has no capability to access external sources, and its training data cuts off at the end of 2021, so there's no way for it to have ever succeeded at this task. It was essentially asked to summarize an article it has never seen, so it hallucinated the entire summary based off of the words in the URL as that is the only information the model was provided. For a more equivalent comparison you'd need to copy the text of the article into the chat input.

Of course that doesn't change the fact that it responded to your query with totally fabricated nonsense, which is a horrible failure mode that points to why it can be so hard to trust LLMs' responses, but I do believe the disclaimer popup that appears when you launch ChatGPT notes this particular limitation.

[voytec]

It spelled my last name correctly in 4th paragraph (although could have got it from domain name). This domain was registered in 2023 so it could not have data from 2021; but maybe has been fed from Bing later?

It currently states: "As an AI language model, I don't have direct access to real-world events that have occurred after my knowledge cutoff date of September 2021. However, I can access information and data from websites created after September 2021, as long as they are publicly available on the internet."

> it responded to your query with totally fabricated nonsense, which is a horrible failure mode

Agree. I would be fine with getting "I cannot summarize this post as I don't crawl external sources" but not this.