Hacker News new | ask | show | jobs
by andoando 756 days ago
Well 1. Radiology is an insanely niche subject not indiciative of general intelligence, and 2. AI being at good radiology isn't about object recognition or spatial reasoning, its data analysis connecting features to outcomes.

Lastly, check out the ARC challenge or any other spatial reasoning tests for AI. Humans get ~80% on these challenges whereas the best AI is still at 25%
1 comments
CamperBob2 756 days ago
Can you point me towards a citation for the 25% figure? I'm seeing numbers like 96% ( https://paperswithcode.com/sota/common-sense-reasoning-on-ar... ) but I'm guessing that's just for a subset of the larger class of questions.

Also, are you familiar with this study? What are your thoughts on it? https://www.esmo.org/newsroom/press-and-media-hub/esmo-media... Seems like a valid case where AI is competitive with skilled humans at object/image recognition.

link
andoando 756 days ago
It seems theres multiple things by the name ARC. There is one by AI2 which is a text based science questions/word problems. The one Im referring to is this https://lab42.global/arc/

https://lab42.global/arcathon/leaderboard/

https://openreview.net/forum?id=E8m8oySvPJ

As to the study, I have the same objection as the radiology one. This isnt about object recognition and certainly not spatial reasoning, its the ability to predict cancer based on presence of visual features.

The "object recognition" part of this is super simple. Its a single, mostly 2D object in more or less the same angle, and the AI is trained on detecting just this.

link
CamperBob2 756 days ago
The "object recognition" part of this is super simple. Its a single, mostly 2D object in more or less the same angle, and the AI is trained on detecting just this.

And yet it outperforms human dermatologists.

link