[tmpz22]

Isn’t the summarization of text like legal documents where the notion of hallucinations come in as a huge blocker?

Is the industry making progress on fixing such hallucinations? Or for that matter the privacy implications of sharing such documents with entities like OpenAI that don’t respect IP?

Until hallucinations and IP/PII are fixed I don’t want this technology anywhere near my legal or personal documents.

[famouswaffles]

Tasks like summarization and translation get extremely low hallucinations. The more a model "doesn't know" and "has to guess", the more it hallucinates. This isn't much of a problem with what i like to call "morphing" tasks.

>Until hallucinations and IP/PII are fixed I don’t want this technology anywhere near my legal or personal documents.

Good luck with that https://twitter.com/ai__pub/status/1644735555752853504

[tmpz22]

> Good luck with that. https://twitter.com/ai__pub/status/1644735555752853504

Is it fair to say these deals claimed to have been closed by the worlds largest law firms using OpenAI backed tooling double check all outputs at their own expense? Could this be a marketing stunt versus a real world usage that actually saved the firm money or time?

[mjr00]

I've been using the ChatGPT API to do summarization of text from free-form documents. Not in the legal domain though, so no real regulatory risks. It works very well. I didn't see any hallucinations when spot checking, though of course I can't rule it out. But even if it only gets things 98% correct, that accuracy is good enough for my use case, and being able to programmatically feed these documents in instead of hiring multiple contractors to read through and parse out the data is a massive, massive time and money saver.

> Or for that matter the privacy implications of sharing such documents with entities like OpenAI that don’t respect IP?

Their permissions/organization model is a mess, but ChatGPT does offer the ability to opt out of data collection, at least for corporate accounts.