[kelnage]

Absolutely agree.

My biggest problem with AI safety is that, simply, the problem they envisage doesn't exist yet (generally, at a minimum, relying on the existence of "AGI"). Hence discussions about it have to make a huge amount of assumptions about a whole range of aspects of what the AI threat will be - what the AI will be capable of, what it's impact will be - before getting on to what possible solutions might be relevant to preventing it. But given the first two are so undefined, the later is pure speculation - one that is difficult to criticise directly, because any specific critisms can usually be easy deflected by adjusting any of the above assumptions without making a substantial change to the "inevitable" conclusion.

That's why it feels like an apocalypse cult to me - it's a conclusion, that has little strong evidence today, stacked on top of a constantly shifting set of assumptions, allowing adherents to avoid backing their arguments with evidence.

[shapefrog]

For the freseable future the real danger I see from AI is inferior basic statistical models being jammed into production pretending to be some sort of all seeing all knowing AGI - a class of product we are not really that close to.