Hacker News new | ask | show | jobs
by eqvinox 43 days ago
> A lot of tools have affordances built in to make "right" things easy and "wrong" or unsafe things harder.

This is true for almost anything handed to laypeople, but not for a lot of professional tools. Even a plain battery powered drill has very few protections against misuse. A soldering iron has none. Neither do sewing needles; sewing machines barely do, in the sense that you can't stick your fingers in a gap too narrow. A chemist's chemicals certainly have no protections, only warning labels. Etc.

Also cf. the hierarchy of controls: https://www.cdc.gov/niosh/hierarchy-of-controls/about/index....

people don't seem to want to eliminate AI → replacing it doesn't improve things → isolating it - yup, people are trying to put it in containers and not give it access to delete the production database → changing how people work with it: that's where we are now → PPE: no such thing for AI, sadly → production database is deleted.
2 comments
BadBadJellyBean 43 days ago
Exactly this. I was talking about professionals. People who should know better. If we as professionals give away our agency and our accountability we make ourselves obsolete. If I just tell the LLM what to do and hope it doesn't go south then the Manager could probably do that as well.

And if a non professional did it they should ask themselves why we have professionals. Maybe there was a reason and maybe they do have value.

link
queenkjuul 43 days ago
An LLM is a large and complex machine, not a screwdriver. Large and complex [physical] machines are built with safeguards to prevent misuse, injury, etc by regulation.
link
nuancebydefault 43 days ago
LLM's are in principle text in / text out machines. If the user extends its capability to have agency over a production database or a machine, there's nothing that can safeguard the safety.

Imagine I ask an LLM to instruct left/right/speed up/slow down while driving. I can simply bypass any safeguard by stating i suddenly became blind while driving a car. While in fact i'm blindfolded and doing an experiment on a highway.

link
eqvinox 42 days ago
A bulldozer is a large and complex physical machine, yet it has (almost¹) no safeguards against misuse or injury. It's all operator training. Lathes tend to not have doors/enclosures, in particular large ones. You get taught where to not put your fingers, and to wear safety goggles. Cranes don't have a lot of safeguards either, you better know how to attach things; hardhats aren't gonna do sh*t if you get a ton of concrete dropped on you.

etc. pp.

I'm not sure where this "tools are made to be safe" belief comes from. This is only the case in "consumer" environments. Of course you don't intentionally make things unnecessarily unsafe, but — in a professional environment there is an expectiation that the operator had training and knows what they're doing.

Maybe that's what we're missing: training in safe AI use. With a certificate that has to be periodically renewed. At the current rate things are going, I'd say 3 months is a good renewal cycle ;D. </s>

(¹ it beeps when it goes backwards. Honestly, I'm not sure that counts for much.)

link