[senordevnyc]

It's pretty amusing that we're now at the stage of AI denialism where the goalposts are "AI is only smart if it can get a PhD in an area it hasn't been trained in!"

Looking forward to where we move the next goalposts next. Perhaps AI isn't smart because it can't invent a cure for cancer in 24 hours? Or it can't challenge our core understanding of the laws of physics?

[_jab]

I think the reason the goalposts keep shifting is really that these AI labs are illustrating Goodhart's law in action: https://en.wikipedia.org/wiki/Goodhart%27s_law

We're at a point where AI labs are fairly reliably able to create AI systems that can achieve good performance on given benchmarks, via targeted training efforts. You argue that it's not reasonable to expect that performance to generalize across other domains. But given that these same companies are trumpeting that AGI is around the corner, I think that's a fair expectation for us to have, and AI has disappointed in that regard.

[jqpabc123]

Last time I checked, we are still at the stage of AI apologetics where an NBA player "throwing bricks" gets misinterpreted as an act of vandalism.

In other words, there is limited capacity for real "understanding" --- which McDonalds and IBM discovered after 3 years of trying to get AI to take orders at drive-thru windows.

https://www.cio.com/article/190888/5-famous-analytics-and-ai...

[ceejayoz]

I dunno if the title got changed, but it's now "ChatGPT Blows Mapmaking 101"… and this certainly isn't PhD-level mistakes happening. It's stuff you'd be docked points for in middle school.

[wrs]

You clearly didn’t read the article if you think that’s where the goalposts were. The title is understatement.

How about “ChatGPT can’t even keep its own maps consistent from one prompt to the next, much less get a PhD in geography”?