Hacker News new | ask | show | jobs
by ImAnAmateur 981 days ago
There's a second wind to this story in the Mastodon replies. It sounds like the LLM appeared to be basing this output on a CVE that hadn't yet been made public, implying that it had access to text that wasn't public. I can't quite tell if that's an accurate interpretation of what I'm reading.

>> @bagder it’s all the weirder because they aren’t even trying to report a new vulnerability. Their complaint seems to be that detailed information about a “vulnerability” is public. But that’s how public disclosure works? And open source? Like are they going to start submitting blog posts of vulnerability analysis and ask curl maintainers to somehow get the posts taken down???

>> @derekheld they reported this before that vulnerability was made public though

>> @bagder oh as in saying the embargo was broken but with LLM hallucinations as the evidence?

>> @derekheld something like that yes
2 comments
jameshart 981 days ago
Took me a while to figure out from the toot thread and comment history, but it appears that the curl 8.4.0 release notes (https://daniel.haxx.se/blog/2023/10/11/curl-8-4-0/) referred to the fact that it included a fix for an undisclosed CVE (CVE-2023-38545); the reporter ‘searched in Bard’ for information about that CVE and was given hallucinated details utterly unrelated to the actual curl issue.

The reporter is complaining that they thought this constituted a premature leak of a predisclosure CVE, and was reporting this as a security issue to curl via HackerOne.

link
Arnavion 981 days ago
No, it's not that Bard was trained on information that wasn't public. It's that the author of the report thought that the information about the upcoming CVE was public somewhere because Bard was reproducing it, because the author thinks Bard is a search engine. So they filed a report that the curl devs should take that information offline until the embargo is lifted.
link
sfink 980 days ago
Which is a fair request. Perhaps Bard should be taken offline.

The curl devs might even be the right ones to do it, if they slipped a DDOS into the code...

link
INTPenis 980 days ago
Just what do you think you are doing sfink?
link