[overgard]

> You can have something smarter than you which you use as a tool.

That sounds like how tech CEO's treat their employees!

Kidding aside, I think the notion that you could control something (legitimately) smarter than you is a pretty risky proposition. (Fortunately one that I think is actually far off)

[abendstolz]

> I think the notion that you could control something (legitimately) smarter than you is a pretty risky proposition.

But then think about the CEOs and their employees again.

We just need to invent something like money for an "AI" and then we can lead in on a stick ;-)

[codechicago277]

In how many companies or countries is the smartest person in charge? Raw intelligence is just one of many factors that affect a person’s abilities.

[cwbuilds]

I'm able to control my coding agent even though it writes code better and faster than I can.

[overgard]

The ultimate goal of this technology is to replace people -- if it's a tool and corporations still need to employ people to use the tool, then the valuations make no sense. So ultimately, you're going to need to hand the car keys to these agents more and more: here's access to production (we fired the devops people), here's access to all our internal messages and IP, here's all our source code (we fired the engineers).

While I think the discussions around alignment sound more like science fiction than science, it also hasn't been "solved", and the only "control" you have over what the agent does is by gating access. Even if you think the AI itself is aligned, handing complete control of your entire enterprise and all your data to a private unaccountable for-profit enterprise should give you a lot of pause.

The idea that you can create entire jobs where humans just act as reviewers or gate keepers seems very unlikely to me, just knowing human psychology. If the AI is right 99% of the time, and catastrophically wrong %1 of the time, but I've been condition to always hit "accept" because it's usually right, there's very little chance I'm going to catch that 1%. "yes, continue with the database migration". "yes, continue with the database migration." x 1000. (At button click 500, the user stopped reading). "yes, go ahead and delete the database. Shit, wait!!!"

[cwbuilds]

It already has created jobs where people just act as reviewers and gatekeepers though. There are people who vibecoded products and are now running decent-sized businesses.

[throwaway7356]

> I think the notion that you could control something (legitimately) smarter than you is a pretty risky proposition.

Well, Trump seems to control a lot of people given how afraid they are.

Trump is also called not the smartest person out there.

So...