[simon84]

It's not so much about velocity or quality, both of which LLM do (or will) provide.

The real question is about accountability and liability.

When a major data leak is going to happen, who will they sue or fire ? That is the value engineers provide. They understand, confirm, and take ownership.

[jalev]

This is what I'm wondering too. We've signed a confidentiality agreement with all the big players (as I'm sure all other companies have done), which is supposed to ensure our data is both segregated and not used for training. I don't trust these companies not to do just that; their business is in taking what we have and training their models.

[iterateoften]

Yeah, I always wonder if they do some type of obfuscation and transformation on the private data and find a way to backdoor the info without technically using it directly.

[afro88]

I wonder if there's a way to include data that's so unique you can prove it was trained on and sue later

[Chu4eeno]

Unique data like that is unlikely to have any impact on the learned/final weights after training. SGD, Adam and the other hillclimbing solvers abhor jagged edges from "novel" trade secrets and the like. Unless it turns out everyone had the same secret genius idea (and it became a pattern to learn).

[lanfeust6]

Ostensibly, due-diligence should not change. But people are lazy, just as they've always been around testing/QA/definition-of-done.

I'm not even certain that laziness gets them further along than it used to; I think it's that people have not had their overconfidence painfully corrected yet. Behaviors will re-align pretty fast when people realize that no, they're not going to get away with just pressing a button and saying everything is "good". That is happening right now.

[mexicocitinluez]

Just having this discussion with someone about AI in healthcare and how issues are going to be handled.

If a nurse does something incorrectly, they can lose their license. Ensuring that nurse will never be a nurse again. There is a very clear path of accountability and very clear ways to mitigate it.

For instance, if a nurse is drunk and you recognize there is a pattern of people showing up drunk, you institute drug tests and breathalyzers and move on.

While we probably won't have LLM's autonomously performing procedures, they are 100% parsing documentation, reading lab results, making suggestions, etc. And right now, the burden has been placed squarely on the clinicians themselves. It'll feed them them the data, ask if they approve/agree, and then essentially wash their hands of accountability. Let's say an LLM starts incorrectly reading lab results, how is that fixed/remedied? A prompt update? Additional safeguards? Adjusting the temperature? Changing a model?

This is a far different type of engineering that still feels pretty new. Granted, I'm still an amateur in this space (I use Claude Code a decent bit), but it feels really opaque to me right.

[verandaguy]

This question has been easily answered by many companies.

You, the IC, the developer prompting the code extruder, are ultimately responsible for its outputted code and its behaviour.

You may feel pressured to push out thousands of lines of code a day. You may see those thousands of lines refactored several times over the lifespan of a merge request. You may be asked to do this continue this in the long term with all the mental fatigue that entails.

When it's too much for you to sustainably deal with and you turn to using LLMs to review the code, that will still, presumably, fall on you at the end of the day.

The output is your responsibility.

[rvz]

> When a major data leak is going to happen, who will they sue or fire ? That is the value engineers provide. They understand, confirm, and take ownership.

This goes for serious incidents, disasters, outages and security breaches.

If there was an investigation and the answer was "a piece of software was vibe coded with AI" why would anyone trust the software vendor after that?

[marcosdumay]

When has any company ever faced consequences from atrociously bad code leaking data or negatively impacting their customers?

Even Solarwinds is still alive.

[simon84]

EU companies are judged guilty of negligence because backups were not totally disconnected (even though distant site) and ransomware did destroy them.

So that is starting to dig deeper than a plain mistake. I guess we will soon-ish witness the first AI slop trial going on, this will be interesting to follow

[mont_tag]

Knight Capital

[genxy]

Don't worry, we can throw in all in 55 gallon drums and dump it over a cliff when the time comes.