[brookst]

Maybe this is peak AI panic?

It seems wild that someone could unironically talk about tools “disempowering” their users. Like, I get it, C disempowers programmers by shielding them from assembly language, and Cuisinarts disempower chefs, and airplanes disempower travelers by not making them walk through each territory.

But… isn’t that a pretty tortured interpretation of tool use? Doesn’t it lead to “the Stone Age was a mistake”, and right back to Douglas Adams’ “Many were increasingly of the opinion that they'd all made a big mistake in coming down from the trees in the first place”

I get that AI can be scary, and hey, it might kill us all and that would be bad, but this particular framing is just embarrassing.

[mrob]

Modern AI does not act like a tool. Tools behave predictably and deterministically. They act as a multiplier to human ability, not a replacement. Modern AI is a new class of thing that humanity has no experience with.

[tac19]

Human consultants have been a thing for a long time, they're a tool of business. They're meant as a multiplier to the corporate ability, not a replacement. It seems like a shift in economics and availability, rather than a shift in kind, if those consultants are virtual rather than human. No?

[mrob]

Calling humans a "tool" is metaphorical. Humans are not literally tools.

[tac19]

But they are in the abstract sense of something to employ to achieve a result. That's why consultants are hired, not for their humanity, but for their utility.

[mrob]

But people talking about tools usually mean tools in the concrete sense. Even very complicated tools in the concrete sense, e.g. compilers or CNC mills, only act as multipliers to the operator's ability. A consultant is an agent, not a tool, and this is a better comparison for modern AI. It's an important distinction because an agent can fail or act against your interests through no fault of your own. The ability multiplier is unpredictable and potentially negative.

[tac19]

Well I don't think getting hung up on such definitions will be fruitful. But here is the point i was trying to make: humans, as individuals and as collectives, do indeed have a lot of experience outsourcing intellectual jobs. They do this knowing full well that the "expert" they're employing is not a deterministic box, and may in fact be secretly working against their interests. None of those problems or potential issues is different if the expert is a human or a silicon agent.

[_vertigo]

All of your examples are tools that incrementally improve upon what came before and, importantly, still require the user to have an understanding of and experience with the underlying skill.

The risk being posed here is that AI may not land as an incremental improvement that still requires the user to maintain some understanding of the underlying skill.

We aren’t quite there yet with the current LLMs. You still need to have a base level of knowledge to use them effectively. But if they were just a little bit better, hallucinated just a little bit less, the value of actually knowing things goes way down.

What would the incentive be to learn the underlying skill or area if the LLM can handle things just fine on its own? Why not just let the LLM figure it out and do it? And at that point, it ceases to be a tool and starts to be something you are completely dependent on. That is a risk.