[ethbr1]

I've always drawn the link between skill in memorization and in analysis as:

- Memorization requires you to retain the details of a large amount of material

- The most time-efficient analysis uses instant-recall of relevant general themes to guide research

- Ergo, if someone can memorize and recall a large number of details, they can probably also recall relevant general themes, and therefore quickly perform quality analysis

(Side note: memorization also proves you actually read the material in the first place)

[Jensson]

Problem is the LLM memorized the countless examples you can find of old BAR questions using extreme amounts of compute at training time, they don't have that ability to digest a specific case due to both lack of data and it doesn't retrain for new questions.

A human that can digest the general law can also digest a special case, but that isn't true for an LLM.

[anon373839]

I’m not sure why you’re being downvoted for this. I agree with you, fact recall is useful and necessary. If you have a larger and more tightly connected base of facts in your head, you can draw better connections.

And even though legal practice tends to be fairly slow and deliberative, there are settings (such as trial advocacy) where there is a real advantage to being able to cite a case or statute from memory.

All that said, I still maintain that it’s a poor way to compare humans with machines, for the same reason it would be poor to compare GPT-4 to a novelist on their tokens per second written.