Hacker News new | ask | show | jobs
by xtajv 317 days ago
I dream that LLMs will be the disaster that finally convinces the software engineering field that code isn't so "soft" after all, and that software engineering should be licensed, bonded, and insured.

Every single other engineering field has gone through it. "Regulations are written in blood".

2 comments

Blood has been spilled. 737 MAX happened and it didn’t change the industry, so nothing will.
There isn't a single unified software industry. 737 MAX problems happened in a software engineering and software development context embedded within one of the nominally most rigorously regulated industries that already exist.

MCAS failures was not a failure of software per se, but a clear system engineering and management failure, and a failure of all engineers involved, including ones that actually are licensed.

If nothing else, MCAS shows the limits of regulation, particularly in the failure mode of regulatory capture (FAA delegated too much power back to Boeing).

It shows the broken incentives of the lobbied failure mode. It does not say anything about the limits of regulation. In particular, regulation can say: you need to perform 100,000 failure-less flights across the globe without any passengers to approve the aircraft. I’m not saying it’s practical to do so, rather that regulation always has a headroom.
Sadly, I’m not sure things change when blood is spilled. Only when a DRASTIC amount of money is lost. Multiple times.
Looking at https://en.wikipedia.org/wiki/List_of_software_bugs there has been several major bugs, e.g. in 2012 Knight Capital Says Trading Glitch Cost It $440 Million, or https://en.wikipedia.org/wiki/Northeast_blackout_of_2003 which lasted up to 4 days and affected 55 million people and resulted in almost 100 deaths.
History has sadly proved you right imo.

Don't think it'll change anytime soon either.

Not really, very few fields of engineering are that heavily regulated, and software in safety critical contexts is already heavily regulated. And having seen the sausage made, it's really not that much better. In fact the average web app probably has better quality software than most embedded medical devices, there's just a bare minimum bar of documentation and testing that hopefully stops them killing someone.