[mpenick]

What’s wrong with being excited and working hard towards something? Hoping for nearly anyone’s demoralization is not good. I wish you a spirit of adventure that takes you many exciting places!

Also, what specific safety problems do you think need to be solved?

[threeseed]

Because 99% of them aren't working on anything.

They are the ones on the sidelines making money selling courses, going on podcasts, getting engagement money from X etc.

[dwaltrip]

Going back to the root comment of this thread…

Most of the guests on Dwarkesh’s podcast are working full-time in the industry, many of them in extremely senior.

[shermantanktop]

There’s a sudden market for a new flavor of bullshit, and brand-new bullshit providers have sprung up to fill the vacuum. But all the exec-webinar/thought-leader/hypetrain/pr stuff is a sideshow from people building actual value.

[talldayo]

> What’s wrong with being excited and working hard towards something?

Ahh, the old "how can cryptocurrency be harmful if people like buying it?" argument.

There are many realistic and amenable goals in life, like mapping the human genome or writing an Open Source microkernel. Declaring that you intend to surpass human intelligence via a statistical text generator is not one of those things. It does not have precedent, it does not have feasibility studies, it does not seemingly indicate in any way shape or form that it is possible. Nobody can even spell-out the intermediate steps to get us to AGI; every single "novel" solution involves scaling up our current, broken, concepts. It's ELIZA versus the Lisp pundits all over again.

Excitement and hard work go a long ways, but you won't know which way until you apply a little logic. The current "AGI" trend is practically non-existent outside the venture-capital sphere and OpenAI employees, both of whom would be bullish on AGI anyways since it's good for business. Once you discard the biased opinions, you're left with legitimately confused investors and nonsense opinions propagated by conspiracy theorists. Much like crypto, the concept of "AGI" is being used to confuse and exploit people who misunderstand technology and finance.

> Also, what specific safety problems do you think need to be solved?

I think you misread their comment. They said "safely incorporated", which is not a specific safety problem for AI but a holistic consideration that stops your product from sucking. Your computer vision model could be statistically perfect, but absolutely useless for self-driving tasks and multimodal robotic agents. It's not taste that separates these good implementations from the bad ones, it's logic. You have to be considerate when implementing AI in traditional systems, because inherently AI can be wrong and you must have a failure-mode for those situations.

Many people reject this idea, because it precludes the idea that someone could sell a cure-all to today's AI ills. But real AI safety cannot be baked-in to a model. It only exists when genuinely thoughtful humans anticipate every single fail-state; if that sounds like hard work, it's because it is. And nobody, nobody, sells it as "AGI".