[chairhairair]

I have no idea what you are talking about now. You claimed to be able to write a program that can pass the LSAT. Now it sounds like you think the LSAT is a meaningless test because it... has answers?

I suspect that your own mind is attempting to do a lookup on a table entry that doesn't exist.

[phphphphp]

The original comment I replied to is scared for the future because GPT-4 passed the LSAT and other standardised tests — they described it as “terrifying”. The point I am making is that standardised tests are an invention to measure how people learn through our best attempt at a metric: information retention. You cannot measure technology in the same way because it’s an area where technology has been beating humans for decades — a spreadsheet will perform better than a human on information retention. If you want to beat the LSAT with technology you can use any number of solutions, an LLM is not required. I could score 100% on the LSAT today if I was allowed to use my computer.

What’s interesting about LLMs is their ability to do things that aren’t standardised. The ability for an LLM to pass the LSAT is orders of magnitude less interesting than its ability to respond to new and novel questions, or appear to engage in logical reasoning.

If you set aside the arbitrary meaning we’ve ascribed to “passing the LSAT” then all the LSAT is, is a list of questions… that are some of the most practiced and most answered in the world. More people have written and read about the LSAT than most other subjects, because there’s an entire industry dedicated to producing the perfect answers. It’s like celebrating Google’s ability to provide a result for “movies” — completely meaningless in 2023.

Standardised tests are the most uninteresting and uninspiring aspect of LLMs.

Anyway good joke ha ha ha I’m stupid ha ha ha. At least you’re not at risk of an LLM ever being able to author such a clever joke :)