[mjr00]

> Not one killer app has emerged. I for one am eager to be all hip and open minded and pretend like I use LLMs all the time for everything and they are "the future" but novelty aside it seems like so far we have a demented clippy and some sophomoric arguments about alignment and wrong think.

In my mind I divide LLM usage into two categories, creation and ingestion.

Creation is largely a parlor trick that blew the minds of some people because it was their first exposure to generative AI. Now that some time has passed, most people can pattern match GPT-generated content, especially one without sufficient "prompt engineering" to make it sound less like the default writing style. Nobody is impressed by "write a rap like a pirate" output anymore.

Ingestion is a lot less sexy and hasn't gotten nearly as much attention as creation. This is stuff like "summarize this document." And it's powerful. But people didn't get as hyped up on it because it's something that they felt like a computer was supposed to be able to do: transforming existing data from one format to another isn't revolutionary, after all.

But the world has a lot of unstructured, machine-inaccessible text. Legal documents saved in PDF format, consultant reports in Word, investor pitches in PowerPoint. And when I say "unstructured" I mean "there is data here that it is not easy for a machine to parse."

Being able to toss this stuff into ChatGPT (or other LLM) and prompt with things like "given the following legal document, give me the case number, the names of the lawyers, and the names of the defendants; the output must be JSON with the following schema..." and that save that information into a database is absolutely killer. Right now companies are recruiting armies of interns and contractors to do this sort of work, and it's time-consuming and awful.

[tmpz22]

Isn’t the summarization of text like legal documents where the notion of hallucinations come in as a huge blocker?

Is the industry making progress on fixing such hallucinations? Or for that matter the privacy implications of sharing such documents with entities like OpenAI that don’t respect IP?

Until hallucinations and IP/PII are fixed I don’t want this technology anywhere near my legal or personal documents.

[famouswaffles]

Tasks like summarization and translation get extremely low hallucinations. The more a model "doesn't know" and "has to guess", the more it hallucinates. This isn't much of a problem with what i like to call "morphing" tasks.

>Until hallucinations and IP/PII are fixed I don’t want this technology anywhere near my legal or personal documents.

Good luck with that https://twitter.com/ai__pub/status/1644735555752853504

[tmpz22]

> Good luck with that. https://twitter.com/ai__pub/status/1644735555752853504

Is it fair to say these deals claimed to have been closed by the worlds largest law firms using OpenAI backed tooling double check all outputs at their own expense? Could this be a marketing stunt versus a real world usage that actually saved the firm money or time?

[mjr00]

I've been using the ChatGPT API to do summarization of text from free-form documents. Not in the legal domain though, so no real regulatory risks. It works very well. I didn't see any hallucinations when spot checking, though of course I can't rule it out. But even if it only gets things 98% correct, that accuracy is good enough for my use case, and being able to programmatically feed these documents in instead of hiring multiple contractors to read through and parse out the data is a massive, massive time and money saver.

> Or for that matter the privacy implications of sharing such documents with entities like OpenAI that don’t respect IP?

Their permissions/organization model is a mess, but ChatGPT does offer the ability to opt out of data collection, at least for corporate accounts.

[Legend2440]

Image generators (midjourney, etc) are half LLM and still doing very impressive creation.

The big thing holding them back is legal/copyright concerns, but I expect that will be worked out eventually.