[tptacek]

NTPsec advocates keep saying "eliminated 50% of vulnerabilities <<<before they were disclosed>>>", as if there were another meaningful way to eliminate vulnerabilities from a codebase.

Can you provide a breakdown of the vulnerabilities NTPsec HAS and HAS NOT been vulnerable to, along with their severity (low: degrades time service, medium: provides a practical vector for corrupting integrity of time service, high: compromises integrity of the server itself) and whether they're exposed (a) in the default configuration, (b) in a configuration run widely on the Internet, or (c) in no configuration actually known to the project maintainers?

You clearly have the list somewhere, because everyone involved in the project has this statistic ready to quote.

If you don't have the severity and exposure breakdowns, that's OK. Post the list anyways. Maybe it'll be obvious what the severity and exposure is.

This business of counting vulnerabilities and claiming victories has been a problem for software security for two decades now. Ops people don't care about the vulnerability count, if the vulnerabilities left exposed in the codebase are the ones that get their servers popped.

[HedgeMage]

I'm sorry if I wasn't clear, I meant "before they were disclosed to NTP classic or NTPSec". In other words, by simply improving on the software engineering practice, we eliminated classes of vulnerabilities without having to track them down individually. This is pretty common with ailing code bases, though often overlooked. I'm at work right now, so I don't have a comprehensive list handy. Going through NTP classic vulns and seeing how many never impacted NTPsec would recreate such a list.

The severity varies (many weren't that big, some were)... the point of claiming the victory is to demonstrate that I'm not just having a fuss about testing code, using static analysis tools, using an accessible code repository, refactoring for lower attack surface and better separation of concerns because they are beautiful in abstract. I like results. NTPSec, and before it the temporary "rescue" team, have been slowly chipping away at the big picture mess, making the code safer and more maintainable, because it's likely to remain in service for another decade or two.

Every time 14 vulns are disclosed and we are already immune to half of them, we get to put twice the effort on the half we do need to deal with, if even we need that much. We aren't just firefighting, NTPSec can develop proactively. That means something for our users.

lots of personnel overlap here...the main difference being pre- and post- fork and where the funding came from, probably not interesting to most people.

[tptacek]

No, I understood your meaning. I'm saying: that's what every code refactoring does. I'm saying that since you can't claim credit for eliminating vulnerabilities that are already disclosed, the emphasis you place on precluding vulnerabilities is strange.

Can you provide that list of vulnerabilities now? You're obviously keeping track of them, that being part of the premise of the project. I know you don't have them broken down, but we can help with that.

[tptacek]

How about this: before I put the effort in to generating the list myself, can you at least promise to confirm that I have the complete an accurate list once I do, and to fill in any gaps?

[HedgeMage]

As a side note, I'd like to add a point that I highlighted in my O'Reilly Security Conference talk but previously forgot to mention here...

One of the coolest after-effects of this whole thing was that, after the fork, when NTP classic began feeling the pressure of competition, their speed in addressing security vulnerabilities increased incredibly. While I was sorry that it didn't happen on its own, I was pleased and impressed to discover what Mr. Stenn was capable of once his competitive hackles were raised.

Many people experience hurt feelings during a fork, and a fork represents a frustrating duplication of effort that I'd usually rather avoid. However, forking is a central tenet of the open source ethos for a reason. Competition can do incredible things. <3

[tptacek]

If a primary purpose of forking ntpd was to give the original project a kick in the ass about fixing vulnerabilities, could it not be argued that your project has now served its purpose, and dollars could be better spent on building from the success of "NTP Classic" --- which, after all, is the version of NTP most likely to be deployed?

[HedgeMage]

I would agree with you if NTP classic had fixed the total of its social and technological problems. Unfortunately, this is not the case. "Patching faster" is one small victory.

[tptacek]

What percentage of "NTP Classic"'s problems are managerial/social and what percentage are raw technical?