[dijit]

I dislike that analogy greatly.

Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.

[deeviant]

So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

[dijit]

>So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

No.

A LLM has more opportunity itself to replace a Lawyer, the person typing the prompt is not necessarily required to be as educated. Though a case can be made that you need to validate the information.

As it happens we have an opportunity to tell how this works. Software engineering has seen many abstractions of which each comes with its own complexity in verification.

What tends to happen is that people don't really do a lot of verification, we are just "mostly right" very fast and leave an immense amount of inefficiency and indirection behind us.

[p1esk]

the person typing the prompt is not necessarily required to be as educated.

If I need someone to help me interact with a legal LLM I will want to (and probably be able to, for 300k) hire someone with a law degree. In fact I anticipate many lawyers in the future will effectively become “prompt engineers” for legal LLMs.

[vba616]

A LLM is a generator of misinformation that is maximally difficult to distinguish from real information.

How do you use this as a lawyer?

I mean, as a stereotypical evil lawyer in a world of naïve people who don't learn from experience, you could maybe use it to win cases until you destroy the justice system.

But other than that...

[deeviant]

Speaking as somebody who spent thousands of dollars for a lawyer on a matter, with basically no results, and also who used chatGPT, personal research, and common sense to solve this same matter for free, I can only say if a LLM is "a generator of misinformation that is maximally difficult to distinguish from real information" a lawyer is simply "a human who has been trained to maximally drain your wallet without regard for any other matter", of course, neither is true and there exists far more nuance for both.

Sure, there are matters I would only trust a lawyer to handle, but there are a great many I wouldn't.

Further, the average quality of a human lawyer will likely remain the same tomorrow as it is today, while AI will only get better. LLM today, perhaps some hybrid stack tomorrow, it's only a matter of time before an AI lawyer is the way to go for just about any legal matter. And let me be clear, that time might be 10 years, or may be 100+, but it is coming.

[p1esk]

A LLM is a generator of misinformation

This is a strange statement. No one is training LLMs to generate “misinformation”. It’s the opposite - it’s trained to generate the most likely next word, given the preceding 2000 words - using billions of examples from a real world training corpus. So it will try to generate as much information as what’s present in the corpus. Maybe even more, but that’s debatable.

[vba616]

>No one is training LLMs to generate “misinformation”.

That is phrased like it is stating a fact about the training process, but it is a statement about the intent of the training, isn't it? So I don't see it as rebutting my comment.

>It’s the opposite - it’s trained to generate the most likely next word

Sure, of course, what else? But if you take any correct statement about something and modify it slightly, it's not very likely it will still be correct.

It seems intuitive to me that there are going to be a million billion (understatement) wrong things next to anything correct in the inputs. As a sort of combinatorial, mathematical thing. You just (in principle) count all the ways to be wrong that are similar to being right.

Nobody trained it to get anything right! It doesn't matter what people expect if they don't have a procedure to do it.

If a statement is adjacent to things that are also "correct", that almost implies a lack of information in the original statement. It seems born out in the impressive BS'ing - the key to BS'ing is saying things that can't really be wrong.

[p1esk]

To be an effective prompt engineer you need to have expertise in two different domains - large generative ML models, and in whatever it is people want to generate (e.g. art).