Hacker News new | ask | show | jobs
by MichaelMoser123 424 days ago
Are all of the ai model benchmarks just made up? https://epoch.ai/data/ai-benchmarking-dashboard https://ai.azure.com/explore/benchmarks
1 comments

The problem is that the benchmarks essentially test "is this data in the training set?" This methodology will never achieve any form of AI that is useful past a search engine.

A true AI shouldn't have any sort of informational knowledge about any domain, as it should be able to figure out how to get that knowledge (which includes building internal simulation if necessary), when asked a question.