|
> Preload a city's, county's etc. entire set of laws You would also need to load an enormous amount of precedential case law, at least in the US and other common law jurisdictions. Synthesizing case law into rules of law applicable to a specific case requires complex analysis that is frequently sensitive to details of the factual context, where LLMs' lack of common sense can lead it to make false conclusions, particularly in situations where the available, on-point case law is thin on the ground and as a result directly analogous cases are not available. I don't see the utility at the current performance level of LLMs, though, as the OP article seems to confirm. LLMs may excel in restating or summarizing black letter or well-established law under narrow circumstances, but that's a vanishingly small percentage of the actual work involved in practicing law. Most cases are unremarkable, and the lawyers and judges involved do not need to conduct any research that would require something like consulting an AI assistant to resolve all the important questions. It's just routine, there's nothing special about any given DUI case, for example. Where actual research is required, the question is typically extremely nuanced, and that is precisely where LLMs tend to struggle the most to produce useful outputs. LLMs are also unlikely to identify such issues, because they are issues for which sufficient precedent does not exist and therefore the LLM will by definition have to engage in extrapolational, creative analysis rather than simply reproducing ideas or language from its training set. |
Very easily done. Is that it?
> lack of common sense, false conclusions
The AI tool doesn't replace the judge/DA/etc. it's just a very useful tool for them to use. Checkout the "RAG-based learning" section of this app I built (https://github.com/bennyschmidt/ragdoll-studio) there's a video that shows how you can effectively load new knowledge into it (I use LlamaIndex for RAG). For example, past cases that set legal precedents, and other information you want to be considered. It creates a database of the files you load in, so it's not making those assumptions like an LLM without RAG would. I think a human would be more error-prone than an LLM with vector DB of specific data + querying engine.
> I don't see the utility
Then you are not paying attention or haven't used LLMs that much. Maybe you're unfamiliar with the kind of work it's good at.
> actual work involved in practicing law
This is what it's best at, and what people are already using RAG for: Reading patient medical docs, technical documentation, etc. this is precisely what humans are bad at and will offload to technology.
> actual research is required
You have not tried RAG.
> LLMs struggle to produce useful outputs
You have not tried RAG.
> LLMs are unlikely to identify issues
You have not tried RAG.
> the LLM by definition is creative analysis
You have not tried RAG.
You can load an entire product catalog into LlamaIndex and the LLM will have perfect knowledge of pricing, inventory, etc. This specific domain knowledge of inventory allows you to have the accurate, transactional conversations that a regular LLM isn't designed for.