[avidiax]

Is there perhaps a training data problem?

Even if the LLM were trained on the entire legal case law corpus, legal cases are not structured in a way that an LLM can follow. They reference distant case law as a reason for a ruling, they likely don't explain specifically how presented evidence meets various bars. There are then cross-cutting legal concepts like spoliation that obviate the need for evidence or deductive reasoning in areas.

I think a similar issue likely exists in highly technical areas like protocol standards. I don't think that an LLM, given 15,000 pages of 5G specifications, can tell you why a particular part of the spec says something, or given an observed misbehavior of a system, which parts of the spec are likely violated.