[gitfan86]

The debate around "What is AGI?" is becoming increasingly irrelevant. If in two iterations of DallE it can do 30% of graphic design work just as well as a human, who cares if it really "understands" art. It is going to start making an impact on the world.

Same thing with self driving. If the car doesn't "understand" a complex human interaction, but still achieves 10x safety at 5% of the cost of a human, it is going to have a huge impact on the world.

This is why you are seeing people like Scott change their tune. As AI tooling continue to get better and cheaper and Moore's law continue for a couple years, GTP will be better than humans at MANY tasks.

[humanistbot]

> If in two iterations of DallE it can do 30% of graphic design work just as well as a human, who cares if it really "understands" art. It is going to start making an impact on the world.

From an AI safety perspective, it is because understanding is a key step towards general-purpose AI that can improve / reprogram itself in any arbitrary way.

[theptip]

It’s worth being clear about what AI risk is. This has nothing to do with “AI may do some harm by putting lots of people out of work”.

The idea is that there is _existential risk_ (ie species-extinction) once an AI can self-modify to improve itself, therefore increasing its own power. A powerful AI can change the world however it wants, and if this AI is not aligned to human interests it can easily decide to make humans extinct.

Scott said in the OP that he now sees AGI as potentially close enough that one can do meaningful research into alignment, ie it’s plausible that this powerful AI could arrive in our lifetimes.

So he is claiming the opposite of you; AGI is more relevant than ever, hence the career change.

I agree with your premise that non-General AI will continue to improve and add lots of value, but I don’t think your conclusion follows from that premise.

[gitfan86]

I agree that putting lots of people out of work isn't the problem. The problem is that these Non-General models become very powerful and they can be programmed by humans to do very impactful things. So much so that even if AGI comes into existence just a few years later it will be of minimal impact to the world.

[preommr]

> "What is AGI?" is becoming increasingly irrelevant.

It's always been irrelevant in the practical sense. It's just an interesting conversation piece particularly among the general public where they're not going to discuss specific solutions like algorithms or techniques.

[abeppu]

Why does the question of defining AGI even need to enter into this?

Aaronson's post only sort of obliquely touches on AGI, via OpenAI's stated founding mission, and Yudkowsky's very dramatic views. Most of the post is on there being signs that the field is ready for real progress. AI safety can be an interesting, important, fruitful area without AI approaching AGI, or even surpassing human performance on some tasks. We would still like to be able to establish confidently that a pretty dumb delivery drone won't decide to mow down pedestrians to shorten its delivery time, right?