[Digory]

They originally scored against a test usually taken by people who failed the bar.

So, GPT-4 scores closer to the bottom of people who pass the bar the first time. In other words, it matches the people who cull the rules from texts already written, but who cannot apply it imaginatively.

[speedgoose]

> In other words, it matches the people who cull the rules from texts already written, but who cannot apply it imaginatively.

Where did you find that in the article?

[Digory]

If you can recite the black letter law, you've got a good chance of passing the bar. The higher essay scores usually require creative arguments about resolving competing rules and policies.

It's easier to extract the formal statement of the rule against perpetuities from a reddit corpus, than to apply the rule to an artificially complex fact pattern in an essay question.