[sebzim4500]

That's a really interesting claim, can you give an example of a question that could reasonably be on a K-12 exam that a frontier model can't answer?

[criddell]

I learned to drive in a drivers ed course at school in tenth grade.

There's also home ec, wood shop, metal shop, auto shop, sculpture, choir, theater arts, marching band, phys ed.

[solumunus]

Surely the ability to drive or work wood is separate to the knowledge required to gain the ability? An LLM can explain how to drive, how a car works, wood working processes… Some knowledge is difficult to communicate through text, and consumption of text might be a poor way to gain the ability to use it, but the knowledge is there?

[criddell]

The knowledge could be contained in a book yet we would never say a book can drive a car.

I know how a violin works yet I would never claim to know how to play violin and if you put one in my hands it would become painfully clear that I know almost nothing about violins other than some trivia.