[acqq]

His post announcing his "from the scratch" effort:

http://phk.freebsd.dk/time/20140926.html

The problem with all such initiatives ("we start clean") is that they typically never even reach the feature set of the previous software (implementing features takes orders of magnitude more time than some simple "proof of concept") and that they often make the same errors the older software already solved. JWZ wrote nicely somewhere about that effect (something like "it's not 'hard to read code,' those are the actual features implemented") but I can't find the link.

[spectre256]

Joel Spolsky has one of the most convincing discussions on this topic: http://www.joelonsoftware.com/articles/fog0000000069.html

"Back to that two page function. Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. ...

Each of these bugs took weeks of real-world usage before they were found. The programmer might have spent a couple of days reproducing the bug in the lab and fixing it. If it's like a lot of bugs, the fix might be one line of code, or it might even be a couple of characters, but a lot of work and time went into those two characters.

When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work."

[mgkimsal]

It doesn't convince me. There's no one answer that's correct for all situations.

Sometimes, had it been written well from the beginning, there wouldn't have been as many bugs in the first place, though.

I've seen codebases that really shouldn't be redone, just maintained and slightly patched over time - the risks are too high, the maintenance is low, patches are at least understood (mostly). I've also seen other codebases that should be scrapped and restarted. It really depends on the skills, commitment and expectations of the parties involved, and there's no one answer that fits all situations.

[acqq]

This time, PHK wasn't able to claim that he recognized any serious issues, but still started from scratch, leaving out even the idea to support all that NTPd already has fully implemented. And he received money to find and fix the bugs, not to make one more proof of concept.

[mgkimsal]

Wasn't specifically defending the PHK decision, more just against the Spolsky article. I've seen it quoted as gospel over the past decade, and ... it just doesn't hold true in all situations.

[msandford]

It depends on the initial quality of the code. Spolsky happens to work with very talented people who tend not to write bad code, so if something looks weird and you don't know why it's there it's easy to give someone the benefit of the doubt.

But if you don't work with talented/skilled/experienced people and there's "weird stuff" in the codebase it might well be for no good reason at all.

You can only invoke the Spolsky "it's bug-fixes!" argument if you're not cleaning up someone's horrific mess.

[mgkimsal]

Exactly, but that nuance is lost on many people. I've lost count of how many people have quoted "don't restart from scratch!" and cited that Spolsky article as some sort of irrefutable wisdom of the ages.

Is it the boy scouts that promote "leave the campground cleaner than you found it"? That's the attitude I try to bring to projects, but there are limits, both time and effort. I can leave you (client, employer) a much better system (by whatever metrics you want to establish) by rebuilding from scratch when what I'm starting with is a broken, unstable mess. not always, but the idea shouldn't be dismissed out of hand because of something Joel Spolsky wrote about Netscape in 2000.

[acqq]

The short (but perfect) JWZ text is here:

http://www.jwz.org/doc/cadt.html

"that's what happens when there is no incentive for people to do the parts of programming that aren't fun. Fixing bugs isn't fun; going through the bug list isn't fun; but rewriting everything from scratch is fun (because "this time it will be done right", ha ha) and so that's what happens, over and over again."

But the longer one, containing more or less the quote I first approximated, I just can't find. If I remeber he wrote about Netscape, the code for FTP and how long it took to get it right in all edge cases, and then it was thrown away.

[themusicgod1]

This is written in 2003: I do not think it is a coincidence that Kent Beck is credited ( http://en.wikipedia.org/wiki/Test-driven_development ) with re/discovering Test Driven Development in that year.

TDD can go horribly wrong, too -- but if your tests are on high-enough-level functionality, and you maintain them, then they can encompass the lessons you learn from fucking up every time you do it. Writing & Maintaining tests isn't fun, just as fixing bugs isn't fun -- so in spirit these kinds of arguments hold just as much sway: there are fun parts of programming, and there are parts of programming that are significantly less fun, but even if you fork and start from scratch, it is feasible to have a checklist of bugs/features developed in the order that they are developed initially.

Do most open source projects do this, even the more well-maintained? Of course not. However, if people are seriously worried about this phenomenon, that's probably one of the ways developed in the decade since this essay was published to approach it.

[wglb]

I hardly think PHK is ADT at all. There are often good engineering reasons to rewrite software. I think Joel's point is that it is more often undertaken for wrong reasons.

[acqq]

There's the software that works, he gets the money to find the bugs and fix them, he decides to write from the scratch something that certainly isn't the replacement of the existing software except for some specific users. That is very ADT, exactly to JWZ's definition:

"This is, I think, the most common way for my bug reports to open source software projects to ever become closed. I report bugs; they go unread for a year, sometimes two; and then (surprise!) that module is rewritten from scratch -- and the new maintainer can't be bothered to check whether his new version has actually solved any of the known problems that existed in the previous version."

http://www.jwz.org/doc/cadt.html

He got the money to look for the bugs and to fix them (hard). He instead goes to make the fully new undiscoverd bugs in fully new (his own) code (easy).

[noselasd]

Indeed. e.g. ntpd has support for quite a lot of different hardware - which is important, someone must be a stratum 1 server.

[nullc]

Stratum 1 support doesn't require a lot of different hardware, arguably it requires less: you need only a PPS device; which has become increasingly generic due to the kernel support needed for precise timestamping. (okay sure, you also need to initially name the second, but you can have another NTP server do that).

[vidarh]

But perhaps it does not make sense for all of these to considerations to be taken care of by a single package.

E.g. perhaps it makes sense to have an entirely separately client-only package with a separate codebase.

[gpvos]

That is exactly what PHK plans to do with ntimed: split it up into separate packages for clients, time distribution servers and authoritative time transmitters.