[locknitpicker]

> - a tool is activated with operator intent (at some point in the call-chain)

This again is where the simplistic assumption breaks down. Just because you can claim that a person kick started something, that does not mean that person is aware and responsible for all its doing.

Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system? Because with LLMs and agents you have even less understanding and control and awareness of what they are doing.

[engeljohnb]

>Just because you can claim that a person kick started something

Kick started what? If you decided to give an LLM access to your database, it's completely on you when you when it does something you don't want. You should've known better.

If all you "kickstart" is an LLM generating text that you can use however you decide, there will never be anything to worry about from the LLM.

> Let's put things in perspective: if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system?

Yes, and it bothers me that others don't feel the same. You vetted the app, you installed the app, and you gave it permission to do whatever on your system. Of course you're responsible.

[orphea]

  it bothers me that others don't feel the same

I bet these are the same people who don't admit they make mistakes; they are never wrong, something else is to blame.

[orion7]

I like to think every mistake I make at work is my manager's fault.

[bdangubic]

same, plus I also blame HR who gave me the contract to begin with

[locknitpicker]

> Kick started what? If you decided to give an LLM access to your database, it's completely on you when you when it does something you don't want. You should've known better.

You don't decide anything. You prompt a coding assistant to apply a change to a repository and without intervention it asserts there's a typo in a table name and renames it. The agent validates the change by running tests and integration tests fail because they are pointing to the old table name. The agent then fixes the issue by applying the change to the database.

Congratulations, you just dropped a table.

I don't think you fully understand how agents and coding assistants work. By design they are completely autonomous and work by reusing your own personal credentials. As they are completely autonomous, they can apply arbitrary changes. I mean, code assistants nowadays write their own tools on the fly. Why do you even presume that people explicitly grant permissions? That's not how it works at all.

If you wish to criticize a topic, the very least you must do is get acquainted with the topic. Otherwise you'll spend your time arguing with your misplaced beliefs instead if the actual problem.

> Yes, and it bothers me that others don't feel the same.

This is a problem you need to overcome, because you have clearly a distorted view of the whole problem domain and also personal responsibility. I recommend you spend a few minutes researching legal precedents associated with malware, because you will quickly learn that runninh arbitrary code you didn't explicitly authorized and acts against your best interests is widely considered a criminal act against the user.

[engeljohnb]

> You don't decide anything. You prompt

Right there. That's where you made the decision, and that's where you went wrong.

>I don't think you fully understand how agents and coding assistants work. By design they are completely autonomous and work by reusing your own personal credentials. As they are completely autonomous, they can apply arbitrary changes.

Yes, and someone somewhere decided to use a coding assistant that can apply arbitrary changes, knowing full well that LLMs are known to hallucinate and make mistakes, and not rarely.

> Why do you even presume that people explicitly grant permissions? That's not how it works at all.

How can you say this with a straight face? Did the LLM hack its way into your workflow? No, someone chose to use it. It doesn't matter that it's autonomous once you enter your prompt. That's actually all the more reason to not allow it to make changes.

> If you wish to criticize a topic, the very least you must do is get acquainted with the topic. Otherwise you'll spend your time arguing with your misplaced beliefs instead if the actual problem.

And if you want to argue with me, you need to actually read and understand what I'm saying.

Say you're staying in the hopsital, and instead of a human nurse making adjustments to your medication, the doctor has an LLM that interfaces directly with the pharmacy and your IV pump. It can make changes to your medication and your dosage without a human ever being involved.

If you overdose because the LLM hallucinated, would you consider an acceptable excuse if the doctor says

"I don't think you fully understand how agents and nursing assistants work. By design they are completely autonomous and work by reusing your own personal credentials. As they are completely autonomous, they can apply arbitrary changes. I mean, nursing assistants nowadays prescribe their own meds on the fly. Why do you even presume that people explicitly grant permissions? That's not how it works at all."

I wouldn't.

[BadBadJellyBean]

> if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system?

Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms.

Of course everyone is differently qualified to do this but my argument is more about professionals. Managers should know better than to blindly trust LLM companies. Engineers should take better care what they allow LLMs to do and what tools they give them.

There is a difference between "I couldn't have known" and "I didn't know". You can know that LLMs are not trustworthy. You couldn't have know what they do but you already knew that trusting them blindly might be bad.

You could know that giving a baby a razor blade is a bad idea. You can't know what exactly will happen but you might have a pretty good idea that it will probably be not good.

[locknitpicker]

> Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms.

No, you don't. If you install malware you are not suddenly held responsible for what has been done to you. Even EULAs you are forced to accept don't shift the responsibility away from bad actors.

[BadBadJellyBean]

I am talking about myself. I have to be careful with what I do. No EULA or any other legal framework protects me from my data be stolen. I have to be careful myself and not just blindly install crapware.

[52-6F-62]

Except what we have here is razor blade companies getting the government to heavily subsidize present razor blade production running massive advertising campaigns and intense intra-industry pressure to give said razor blades to babies under fear of losing your job or "falling behind" those not giving razor blades to babies.

Let's not forget all the razor blade enthusiasts just screaming at you that you are using babies with razor blades wrong and that it works totally fine for them.

[orphea]

  that does not mean that person is aware and responsible for all its doing.

If they are unaware or - worse - don't understand what they are doing, maybe they shouldn't do the thing in the first place?

[keerthiko]

There can be more than one person or entity to be held accountable, depending on the details of impact

If I install a powerful/dangerous app, and I come under harm, I have some accountability — most of it if it's due to user error (eg: I install termux and `rm -rf /`).

If it's malware, and Google/Apple approved said app to their store which is where I got it from, when their whole value proposition for walled-garden storefronts is protecting users, then they have significant accountability.

If the app requests more permissions than necessary for stated goals, and/or intentionally harms users via misrepresentation or misdirection (malware), the app publisher should also be held accountable (by the storefront, legally, etc).

I'm also unclear what angle you are arguing: are you stating that because tools have gotten so complicated that the end user may not understand how it all works, no one should be considered responsible or held accountable? Or that the tool (currently a non-entity) itself should be held accountable somehow? Or that no one other than the distributor of the tool should be accountable?*

[ButlerianJihad]

A few years back, I discovered my router had joined a botnet. The only reason I made this discovery was because of third-party external DNS logs.

Upon investigation, I also discovered that all 3 routers I owned were pwned. So I threw them out the window and tried making do with my ISP's equipment.

My ISP can't provide adequate service on theirs and it's worse than COTS routers, so I purchased a bleeding edge WiFi 7 router. Now there are the two literal black boxes on my network. They do their job and I don't know what else. I can't know.

It could be C2 or it could be a backdoor shell or some kind of server that collects illicit material, and torrents it out? Borrow your HDD for some CSAM sir? It could be a residential proxy that just steals part of my connection for some other paying customer. Are they infringing TOS? How would I know? Check their ID and verify their age??

I, and 99% of consumers with an ISP, have no way of telling when our routers or IoTs are pwned. A silent botnet or two is extremely likely. They're nigh undetectable, and can't be mitigated or defended, except by fastidious updates and upgrades.

My new router was literally triggering printouts on my old printer, because it was so damn "proactive" about "network security scans" and the old trusty printer couldn't tell the difference between a red-team intrusion, and a legit request to print something out!

Likewise even someone with a singular Windows or Mac directly plugged into their ISP could be in a botnet, and it's hard to know. Everyone who's got a smart TV or something with a Linux kernel and an Ethernet, could be doing more than was asked of it. It's the worst kind of malware that alerts the user to its presence. It's a shoddy install if your AV can detect and clean it. If it's stealthy enough then there's no telling.

It's because the vendors own these devices. They deploy the software. They control the builds. The vendors are responsible for what these machines are doing in our hands. Who really, really knows all that goes on when we click that green button? Was it a Joomla or a scam or a legit bank request? Who dafuq knows or cares anymore? Is it an apt analogy that they're selling us herds of animals and farms, and we know nothing of ranching? "Oh feed yourself; should be easy you got everything there" until the coyotes and locusts come? Or like having children who seem to be in school and doing alright, but where do they go at night? Sell drugs? Who knows, I'm not their father, they just live here?

Are they responsible for knowing and mitigating them? Our ISPs don't seem to care or notify us or disconnect us when it happens. Why should we? Why take responsibility?