[matte_black]

Why don’t we require software engineers who work on self driving car software to go through licensing and certification.

And then, if their code results in a death, they are liable and can have their license completely revoked, and they would be unable to work on self driving cars again.

[superfrank]

- Expecting engineers to always write perfect code is insane. Mistakes happen.

- If bad code makes it into production, that is a systemic failure not an individual one (Why didn't the bug get caught in code review, QA, etc.)

- No one is going to want to work on a project where a single failure can taint their career.

- What if I use a 3rd party lib and that is where the bug is. Who is at fault then? What if the code isn't buggy, but I'm using it in an unexpected way because of a miscommunication? If I am only allowed use code that I (or someone certified has written) development is going to move at a snails pace.

- What if I consult with an engineer who doesn't have a certification on a design decision and the failure is there, who is at fault?

- What if the best engineer on the project makes a mistake and ends up banned? Does he/she leave the project and take all their tribal knowledge with them, or are they still allowed to consult? If they can consult, what stops them from developing by proxy by telling other engineers what to write?

Not to be a dick, but this is an awful idea that would basically kill the self driving car.

[davidgay]

> - Expecting engineers to always write perfect code is insane. Mistakes happen.

In safety-critical fields, setting a much higher quality bar than the regular 'it seems to works, the tests pass' seems perfectly rational to me. We can now write provably-correct C compilers (CompCert) and OS kernels (SeL4). There's no excuse for not putting similar levels of effort[0] into something as safety-critical as self-driving cars.

[0]: Note that I'm not advocating for "provably-correct self-driving car software" (that may not be the right approach, as a formal spec is likely unrealisable), but find the argument that "it's ok to write buggy spreadsheets, so it's ok to write buggy self-driving cars" to be morally unacceptable.

[PinguTS]

There is ISO26262 for automotive about safety.

Yes, there are reviews, QA, and all of that. So, yes, there is no single person responsible (exceptions apply).

But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.

That is the reason, why certifications are for. The same rules apply for medical and other areas.

[colatkinson]

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.

Wait, what? That goes against one of the core benefits of open source software--that having many eyes on a problem decreases the risk of bugs. I'm willing to bet that if Uber had to implement their own machine learning/vision libraries from the ground up, there would be significantly more issues.

[henrikeh]

Is there any evidence that this is the case? That simply putting more eyes on a system will reveal its problems?

Certification etc is about process. Open source code can be used in a safety critical product, but it must audited and confirmed against the system requirements.

[PinguTS]

Please do not get me started with the argument over "it is open source, so there are many eyes who have seen the code".

The problem with that is, nobody audits code, if it is working just for fun. And even if it is buggy, then most people look for bugs in their own software and then they work around, so that the original piece is not modified.

We have this seen in many open source projects. Remember all the obvious, mostly security related, bugs that weren't uncovered for years. They weren't uncovered because everybody thought: "huh, that is hard. I assume that other more experienced than I will have reviewed it, so I will trust it."

The thing with certification is, that it is required that it is really reviewed. That there is a guarantee that it is reviewed. That there were people with a different mind set, with different background have reviewed it and as such have brought in their own view.

Certification does not guarantee that something bug free. It guarantees only, it is reviewed. Open source has no guarantee that it is reviewed. There is only hope, that someone has reviewed it.

[Schwolop]

In ISO13485 medical grade software (certain levels of it anyway), the same concept applies. Anything not written in house is "SOUP"; Software of Unknown Provenance. You're required to pass that through a review process before using it, and in many instances it's not worth the effort to review compared to instead just re-writing it yourself.

[superfrank]

> But there is no excuse for using 3rd party libs. Just don't use it. If you not know: do not use it.

Pretty much everything in development relies on the work of other people. I used 3rd party lib just as an example, but what if it's in the framework or even the language that an engineer uses, who would be at fault then? You can't expect every developer to have gone through the entire source code for whatever language they are writing in.

Sciences build on each other and and after a certain period of time you have to take things for granted in order to keep moving forward.

> The same rules apply for medical and other areas.

No, they don't. Doctors kill patients all the time and they aren't banned from medicine for it. There is an investigation, they make sure it wasn't intentional and there wasn't any gross negligence and that this isn't a repeating pattern, if none of those are the case they see what they can learn from it and move forward in hopes that what they learn can help other doctors.

[Glawen]

That's why in iso 26262 you can only use certified compiler and tools. Rust compiler is not certified for example. Sure you can use 3rd party libs, but they must be certified.

I agree dith parent: never use copy paste from internet in safety critical SW, anyway it most probably isn't designed for your use cade. Personally I always have been disapointed by copy pasting stuff, it was always buggy somehow. In the end I always reimplemented it from scratch by reading the theory

[velobro]

Structural engineers are liable if the building they design collapses. I don't see why software engineers in safety-critical fields should be any different.

"The engineers employed by Jack D. Gillum and Associates who had "approved" the final drawings were found culpable of gross negligence, misconduct and unprofessional conduct in the practice of engineering by the Missouri Board of Architects, Professional Engineers, and Land Surveyors. Even though they were acquitted of all crimes that they were initially charged with, they all lost their respective engineering licenses in the states of Missouri, Kansas and Texas and their membership with ASCE.[22] Although the company of Jack D. Gillum and Associates was discharged of criminal negligence, it lost its license to be an engineering firm." - https://en.m.wikipedia.org/wiki/Hyatt_Regency_walkway_collap...

[simion314]

I think the company must pay a lot if a failure happens, if NASA makes a mistake that will costs so they make sure to think hard, have good processes and do a lot of testing.

From what we seen so far the Uber car failed to detect an obstacle, we also had Tesla crash where the car did not seen a truck so it is obvious that there are some major issues that are not tested for. They need to have better tests and maybe get better security drivers inside the cars so they don't text on the phone on the job.

[chopin]

So you can't be bothered to vet 3rd party software for safety critical uses? That sounds like a no-brainer to me. You can't just slap some NPM libraries together for that use case.

[InclinedPlane]

This is a straw man. Nobody is asking for perfection. What is needed are processes and regulations that ensure a high confidence that companies with self-driving cars have done the best that can be expected of them to ensure their vehicles are safe and that they have met a minimum bar in that regard as well. There are many industries where this already happens, there's no reason to single out self-driving automobiles as being impossible to regulate properly.

[always_good]

I can agree with this, but if a company wants to put self-driving cars on the street, I don't see how certificated engineers on their staff is the thing that matters.

Does that demonstrate that the cars are safe? Even a little bit?

To me it just demonstrates that a grunt is on the chopping block for what amounts to systemic failure.

[harshbutfair]

From many years developing safety critical software, I reckon culture and processes are more important than certification. There are various standards for developing safety systems in other industries (defence, aviation, etc) and these standards exist for a reason. Have Uber applied any standard for their automation software? Or equivalent development processes? "Move fast and break things" is fine for an app, but not fine for controlling a vehicle.

[lhorie]

My guess is that it's because the field is so new that there aren't really any experts that can define what are reasonable rules for said licensing and certifications

[ghaff]

They exist. They're required for some roles:

https://insight.ieeeusa.org/articles/professional-licensure-...

[Silhouette]

But what qualifies the people administering those examinations to judge how good someone is at writing reliable software in safety-critical environments?

[slavik81]

The examiners don't judge that. They just check that you meet the educational requirements, that you did an apprenticeship under supervisors who have a good safety record, and that those supervisors think you are qualified. Then you get your license. If there's a safety incident relating to your work and you were acting recklessly, you may lose your license.

That's the whole process. Repeat that over a long enough period of time and you tend to select for a more competent, safety-conscious group of engineers.

Over the long term, its survival of the fittest. Reckless engineers have safety incidents and get barred from both working and from supervising new engineers. So, it tends to be the safer engineers who get to pass on their work culture to the next generation of engineers.

[wsh]

The software engineering P.E. exam is being discontinued after the April 2019 administration, due to low turnout: only 81 candidates since 2013.

https://ncees.org/ncees-discontinuing-pe-software-engineerin...

[PinguTS]

We have this requirement, at least in Europe. There are even ISO standards to follow. The one related is ISO26262. But it seems, this does not apply to those permits issued for these cars by Uber.

[mr_toad]

Engineers aren’t in charge. Unlike lawyers, surgeons, even dental hygienists, they aren’t making the calls.

[ilaksh]

It may have been an executive that just said to turn off LIDAR for testing. Then the engineer probably mentioned it wasn't ready for live testing, was overruled, did it knowing it wasn't ready, because if he refused he may have been fired.

[bigger_cheese]

One of the things I was taught whilst studying engineering (in Australia) was if you, whilst acting in capacity as a professional engineer certify something knowing it is unsafe then you can be found personally liable.

Likewise if you knowingly observe anyone else in your company breaching safety/regulatory guidelines then as a professionally certified engineer you have a legal responsibility around ethical disclosure.

See: http://www.professionalengineers.org.au/rpeng/ethical-commit...

I do not know how things work in the US but in Australia these rights are protected by law. The company legally can not fire an engineer in this situation.

[tonysdg]

Professional Engineers (note the capital 'E') are protected in the U.S. by such laws.

Professional engineers (note the lowercase 'e') are usually not protected in the U.S. -- they're regular employees whose profession happens to be engineering.

[namelost]

Obligatory: https://www.fastcompany.com/28121/they-write-right-stuff

If you want error-free software you need a blameless culture based around process, not individual ownership of code. It should not even be possible for an error to be one individual's mistake, because by the time it hits the road it should have gone through endless code review and testing cycles.

[ubernostrum]

Uber will relocate the engineering team to a jurisdiction without those regulations.

Just like it relocated its testing to get a more "business friendly regulatory environment".

[davidgay]

That's trivially fixed by requiring the engineering to be certified in the jurisdiction in which its output operates.

For a mostly topical example of such rules: https://www.dmv.ca.gov/portal/dmv/detail/vr/checklists/outof...

[KKKKkkkk1]

There are existing laws on the books for this. Please google negligent homicide. Licensing and certification serve a different purpose.

[SamReidHughes]

Then we would never have self-driving cars.