|
Those are my exact feelings since testing Kagi. I did not found it appealing as a plain search engine, nor private. But they have recently shown a tech preview of an "Universal Summarizer"[1] which I have found far superior to ChatGPT (cGPT is worse than some freely available pre-trained models in this regard). I have asked Kagi and cGPT to summarize my own post[2] which Kagi did rather well (I don't have a screenshot), while cGPT spilled utter nonsense[3], mentioning hardware and software that isn't mentioned in the post and which - in some cases - I have never used. While Kagi is unappealing to me at the moment, I will happily try it out the future again. tl;dr: I wrote about using: FreeBSD, Linux, Fail2ban, blacklistd, Terraform, Ansible, Postfix, Dovecot, Kubernetes, Jenkins, ArgoCD, Git and nginx. ChatGPT stated that I wrote about MacBook Pro, iPhone, AirPods, Kindle, Chrome, iTerm2, VSCode, Vimium, Adblock Plus, Dropbox, 1Password, Trello and Slack. [1] https://labs.kagi.com/ai/sum [2] https://wojteksychut.com/posts/work-tech-i-use-privately/ [3] https://wojteksychut.com/pic/chatgpt_post_summarization.png |
> https://wojteksychut.com/pic/chatgpt_post_summarization.png
ChatGPT currently has no capability to access external sources, and its training data cuts off at the end of 2021, so there's no way for it to have ever succeeded at this task. It was essentially asked to summarize an article it has never seen, so it hallucinated the entire summary based off of the words in the URL as that is the only information the model was provided. For a more equivalent comparison you'd need to copy the text of the article into the chat input.
Of course that doesn't change the fact that it responded to your query with totally fabricated nonsense, which is a horrible failure mode that points to why it can be so hard to trust LLMs' responses, but I do believe the disclaimer popup that appears when you launch ChatGPT notes this particular limitation.