[lelanthran]

> 80% of bugs come back to memory safety… don’t think it’s overselling it.

I think you are misremembering the statistic.

It's 70% of security bugs that are related to memory safety: https://www.zdnet.com/article/chrome-70-of-all-security-bugs...

The way you say it, it means that out of every 100 bugs, 80 are due to memory safety.

The reality is that out of every 100 security bugs, 70 are due to memory safety.

For example, a codebase with a thousand bugs might only have 10 that are security bugs, of which 7 are due to memory safety.

You imply that a codebase with a thousand bugs have 800 memory safety issues.

[majewsky]

The framing in the last two lines only makes sense if you consider the amount of total bugs as a prior. In practice, the more realistic prior is something that's more observable, e.g. the number or rate of published CVEs or of security breaches. If something like that can be reduced by 70%, that's much more significant than your framing of "0.7% of all bugs in this example are security bugs due to memory safety" makes it out to be.

[lelanthran]

I don't disagree with that, I was disagreeing with OP whose claim implies that out of every 1000 bugs, 800 of them are due to memory issues.

OP was wildly off the mark.

[c_crank]

This would be an argument in favor of using a GC'ed language, because that would reduce those bugs by 99.9%.

[tialaramex]

Bugs mean the software's behaviour is outside what was specified, but for security we're allowed to depend on any aspect of the specified behaviour. So, all bugs are security bugs.

Some bugs aren't as important as others, but they're all just bugs.

[spacechild1]

> So, all bugs are security bugs.

That's an unreasonably wide and rather useless definition of "security bug".

> Some bugs aren't as important as others, but they're all just bugs.

I think nobody questioned that bugs are bugs :)

[lelanthran]

> So, all bugs are security bugs.

For security software, maybe.

For everything else? I think you have not worked on much software touched by the end-user.

A few years ago, on a security-critical product (chipcard/swipecard terminals)[1] written in C++ I specifically did an analysis on tickets logged as a bug over a 3 year period.

It's a fairly mature product with the original devs who built it still working on it.

Very large surface area - 100s of thousands of deployments at merchants, sales ringing up at all those merchants almost constantly, with a variety of different inputs that the terminal had to handle[2]. The environment can be so random in its input at times it's halfway to fuzzing-in-the-field.

IIRC (I am no longer there), there were maybe a few hundred tickets out of a dozen thousand or so that were actual bugs.

Of those bugs exactly three were memory safety issues.

So, your statement:

> So, all bugs are security bugs.

appears to me to be entirely imaginary and fantastical. To me, who has been developing professionally since the min-90s, that statement appears to be purely fiction. I've not worked on any system where even close to 10% of the bugs turned out to be security issues.

"Missing postcode field in address input" is not a security issue. "Retry not attempted when host is down" is not a security issue. "Incorrectly rejects valid ID number" is not a security issue.

The clear majority of bugs are not security issues, even in a product which is intended to be a security product.

[1] I was an EMV developer.

[2] In case you think this is a simple thing, think Foreign Currency purchase, foreign issuer, Fallback from Tap->Chip->Swipe due to corrupt cards or floor/merchant/bank/cardholder/issuer limits, cancellations of transactions that may/may not have completed due to network errors, power cycles in the middle of a transaction, card switch-out attempts, PoS errors, EMI causing random data corruption in serial transmission lines (those chillers next to the till that hold Red Bull and things have compressors come on and go off intermittently, causing spikes).

Then there's also the fact that the terminal sold cellphone airtime for various SPs, PAYG electric vouchers, scanned QR codes, fingerprints. They also, when deployed to forecourts, had code to handle fuel prices and quantities automatically, split out receipts for oil separate from petrol and accept rewards cards linked to retailers.

When deployed at a restaurant, it had code to allow atomic multi-party payments, even when each party was on a different switch, to a different bank, to a different issuer.

It had code to talk to the most popular PoS systems (each one using a different application layer protocol even if they were all using BER/DER at the base layer).

That's not counting the health monitoring code, or the OTA updates code, or the TLS code (yes, it had its own because certifying OpenSSL or similar was just as expensive as rolling your own with the bare minimum necessary, and certifying that), or code specific to each bank, or key injection code, or cryptography code.

It's anything but small or simple. The average YC startup almost certainly has nowhere near the same amount of complexity, nor does the code you are used to working with have anywhere near the same amount of criticality or attempts to exploit.

In other words, this is a fucking complicated source codebase, with constant attackers who aren't interested in DDoSing you, and severe financial repercussions per second when it is down, while profiting successful exploits immensely.

[tialaramex]

> I was an EMV developer.

I almost couldn't ask for a better example. EMV is so broken that arguably even if systems did behave as specified that's a security bug anyway, because it doesn't fulfil people's basic expectations of how such a system should work.

For people tuning in at home, EMV stands for Europay, Mastercard, Visa, it's the standard for modern "chip and PIN" type payment cards, including "tap to pay" designs and these days is controlled by a consortium or something which allows all the various payment card outfits to dick each other over equally rather than just those three.

Anyway, you seem to just assert that there weren't security bugs, but with no evidence whatsoever for your claim. I have no doubt that you confidently believed that things which actually were serious security bugs in your EMV terminal were fine. After all, why would this standard be ludicrously fragile and have key security requirements that you might destroy by mistake ? Only something designed by idiots would do that...

I think the main thrust of your view is that you deemed the tickets in your ticket system (which may have been labelled "bugs") not to really be bugs at all. That's a very different semantic argument, and mainly I'd say you should take it up with management. At my current employer we distinguish requests, incidents, cards, tickets, bugs and work items, and it feels like that's maybe too much, but whatever.

[nullc]

In security software like sudo you could argue that nearly all bugs are security bugs. But there are plenty of bugs in non-primarily-security software where it's really hard to argue that there is any security implication.

In any case the reason for the figure is because when there is a buffer overflow that allows overwriting the stack its almost always a security bug because it usually allows untrusted input to make the program do anything. So of all the kinds of bugs a program can have the memory safety ones are usually security bugs, while other logic bugs may or may not have meaningful security implications.

In a suid program that is designed to launch other programs (e.g. sudo), significant logic bugs are very likely to have security implications too.

This is supported by sudo's CVE list-- the vast majority appear to be logic bugs with no memory safety involvement. One argument made for rust is that with memory safety "in the bag" developer time can be better invested in other areas. But it isn't safe to assume that: Rust is a very different language and other properties of it such as its far greater complexity than C may make logic bugs more likely. As far as I know the defect rate in rust software has never been seriously studied.

It would be a huge relief if there were some serious analysis that could provide some confidence that rust wouldn't make the software quality situation worse. Especially since I'm not convinced that trading memory safety bugs for other bugs alone would be an improvement: At least memory safety flaws are always bugs, so when valgrind, memcheck, stack protector, etc tell us about them during testing or formal methods tell us they exist (which hardly can exist for rust given that the language isn't even formally specified) we always know we have an issue. With general logic bugs it's much harder to be confident that a behavior is a bug.

And, of course, memory safety issues can arise in rust, due to compiler/library bugs or because other restrictions or performance issues have caused users to use unsafe or depend on external library code... so it's unclear how much we can really say that memory safety is in-the-bag. I think to be confident that a critical piece of rust code lacks memory safety bugs we still must test comprehensively, including with fuzzing and we must review its dependencies which are almost always far far far far worse than in C (creating a whole issue for sneaking in bad code).

Informally, my experience with software written in Rust is that it is far more likely to crash (panic) on the spot when I attempt to use it than software written in C-- usually because the author just hasn't bothered with error handling due of some mixture of the language making it somewhat harder to do, out of a culture that doesn't regard code that panics unexpectedly as shoddy work (because unlike a crash in C, it's unlikely to be the sign of a potential vulnerability), or in some cases because the goal was just "rewrite in rust" and the programmer didn't take much care or pride in their work or lacked the subject matter expertise to do it competently.

It's heartening to see this effort has a test suite-- Of course (AFAIK) sudo doesn't have one itself but arguably sudo's test suite is the decades of use it's had in production. A newly written program has a lot more need for a test suite than an established one.

I think rust programs are more likely to have some amount of tests compared to really old free software, but usually not enough to compensate for their newness.

[tialaramex]

> It's heartening to see this effort has a test suite

A test suite which sudo (here named "ogsudo" although I'm not sure how "gangsta" sudo really is) failed, and as a result sudo's authors had to fix sudo as described in the article we're discussing right ?

> Informally, my experience with software written in Rust is that it is far more likely to crash (panic) on the spot when I attempt to use it than software written in C

Sure, in the C when the author wrote no error handling, it just presses on. Is its internal state now incoherent? Who cares, error handling is for pussies, YOLO. In the Rust it panics. I'd argue that for many cases, including sudo, the panic is much better though still not the ideal outcome of course.

The canonical "Hello, world" program in C and in Rust when given a stdout that claims to be full behaves differently. In C it just exits successfully. Sure, it couldn't actually output the "Hello, world" message, but who checks errors anyway? In Rust it panics, nobody told it how to handle the condition where the output device is full.

[nullc]

> A test suite which sudo (here named "ogsudo" although I'm not sure how "gangsta" sudo really is) failed, and as a result sudo's authors had to fix sudo as described in the article we're discussing right ?

Yup which is table stakes, as the original software didn't have tests (but did have decades of use in production, which is why it has less need for tests than the rewrite).

> Sure, in the C when the author wrote no error handling, it just presses on. Is its internal state now incoherent? Who cares, error handling is for pussies, YOLO. I'd argue that for many cases, including sudo, the panic is much better though still not the ideal outcome of course.

My concern there is that because bad things happen the C program has the error handling, while in rust it may be "don't handle that case? who cares? rust is safe(tm)".

It's not necessarily the case that both are yolo. I'm sure you've heard of risk compensation. https://en.wikipedia.org/wiki/Risk_compensation

The kind of experience I've had with rust code isn't a case where the c-analog is yolo. You just don't expect widely used C code to crash on some command-line argument misuse or when a file is missing, it's not entirely unheard of but it's not common. In my experience it's extremely common in rustlandia. In that sense, rust culturally has really managed to give me a very strong yolo vibe.

"It can just panic" is no so good when the code is in a library that called from the motor controller for a forklift holding a pallet over your head. ... and in plenty of other less drastic situations.

I agree for sudo most of the time panicing is probably not directly unsafe (though it might produce an outage that causes harm). For software that deals with more complex external state, panicing can still be pretty bad-- like leaving the system in a vulnerable state, leaving confidential information laying around, etc. There are plenty of examples where a simple DOS attack can be used to compromise a system, e.g. DOS a master system to cause a fall over to a slave.

[tialaramex]

> no so good when the code is in a library that called from the motor controller for a forklift holding a pallet over your head

There should be a mechanical fail safe. The Therac-25's first important difference from prior hardware was that they removed the physical fail safes because hey, nobody will write a program that's incorrect. Yes they will, and it killed people - so stop designing hardware which assumes unreasonable things about software. With an appropriate fail safe if the code panics, the forklift locks up, maybe there's some sort of mechanical reset needed - this is annoying but nobody is dead.

When the firmware in your elevator freaks out, which isn't even a rare occurrence, nobody dies because the hardware people don't trust that firmware. If you let the firmware control it directly, you'd be lucky if anybody used them because they'd be too dangerous. I've watched Third Year undergraduates in electronics learning with (toy) elevators how to write real time software and they were not good at this. The toys were designed so that they weren't damaged when (not if, when) the students smashed the elevator into the top or bottom of the shaft and still kept the motors running.

> My concern there is that because bad things happen the C program has the error handling,

The example we just looked at is common, and in fact the C program doesn't have error handling, it's just ignoring the error because caring about errors is more trouble in C so why bother.

> The kind of experience I've had with rust code isn't a case where the c-analog is yolo

You're going to have to give a lot more details. It's so easy to YOLO in C that it's harder to think of cases you can't just YOLO than cases where you can. C loves functions where if your inputs are invalid you get invalid outputs, which you can then blindly supply to another function where likewise the invalid inputs result in invalid outputs. YOLO all the way.

[nullc]

On mechanical fail safes, they're sometimes impossible, wildly expensive, or simply less safe. In some devices the safe region is a complex polytope defined over a dozen degrees of freedom and may not even be static.

Perhaps for cases where someone will literally be squished when it fails you can draw a bright line rule, but you can turn the safety requirements down some-- instead of being crushed they might just lose a finger, or a million dollar float glass batch will be lost. At some point the risk isn't worth the elaborate hardware failsafe and software will be all you've got. There really isn't a safe threshold in any case: maybe it's just a word processor but its failure costs someone days of work and leads to their suicide. Software reliability matters.

Panicing on exceptional situations is often only slightly more acceptable than crashing or corrupting in other ways (and sometimes less, commonly the corruption is inconsequential).

> caring about errors is more trouble in C

That hasn't matched my experience. YMMV I guess. It's my experience that it's the exception rather than the rule that rust programs won't just panic instead of handling errors sensibly, in particularly because wrapping types to express failure and explicitly handling it along with all the required lifetime management adds a lot of complexity.

[PrimeMcFly]

> here named "ogsudo" although I'm not sure how "gangsta" sudo really is

It's a joke.