The written portion or the driving portion? I'm actually surprised that you can't stuff whichever state's manual into the AI's context and have it out-perform the median human. Do you have a citation for that?
If you mean the driving portion, that falls somewhat outside the scope of an intelligence test since it relies on actually moving heavy machines in the real world.
That's YOUR definition of intelligence anyway. LLMs fundamentally aren't intelligent because they cannot solve novel problems. It's literally just a complex auto complete that interpolates data from problems already solved by humans. It has no capacity to actually reason about anything.
On a related note, I keep hearing that LLMs will never do X. As a teacher, I doubt that very many if any of my students can do X. LLMs may not be the perfect/God-like problem solver, but they are much smarter than anyone I work with at the community college.
you’re at community college, what do you expect? geniuses? in my basic math undergrad i encountered the limits of GPT daily, with a complete inability to solve anything above maybe freshman or sophomore level proofs. forget anything complex that hasn’t been answered extensively on stack overflow already.
This is a good point because “solving problems” and “generating convincing-sounding text” are the same thing. The fact that language models can answer heretofore unsolvable questions like “how many flimbops does it take to fill a fuzzlebugle to the halfway point?” means that Sam Altman has stolen fire from the gods as a modern day Prometheus