First take the Netscape/Mozilla project. The article addressed that one and said that the decision to rewrite was an unmitigated disaster. And implies strongly that if they had decided to work with the existing code-base, they would have had better results. Well the way that I remember it, they were getting eaten alive by IE, and development was crippled by having to deal with and work around layers of bug fixes on bug fixes. The fact that one set of nightmares came true doesn't mean that the other would not have. Hindsight isn't 20/20. Rather it is speculation with the comfort of knowing you will never find out if you are wrong.

Take next Perl 6. Perl 6 had many major goals. The most important was to reinvigorate the Perl community. Others included making it easier to get into the internals and easier to port to different platforms (eg the JVM, C#, or a more aggressively optimized binary). Note that Perl 5 was not doing too well at these tasks despite much energy and interest. Well Perl 6 has done quite well at the first, and I am fairly confident that it will be able to succeed in the others. It just won't do it on an aggressive schedule.

But I said I don't like to argue from failure. How about a success? Take a look at perl. Scan for the words "version 5". Perl 5 is a complete rewrite of Perl 4. According to Joel that was a horrible mistake, and Perl 5 was bound to fail. Didn't fail that I see. In fact when I take a look at what it resulted in, I don't think that features like lexical scope, removing 2/3 of the reserved keywords, adding references, etc, etc, etc would have happened in the same timeframe doing incremental refactoring. Furthermore I give Larry Wall due credit, he has probably been writing influential free software projects longer than both of us have been writing software combined. Given his trail of successes, when he thinks a rewrite is doable, it probably is. If he thinks that it is a good idea to get where he wants Perl to get, well he is the one whose vision got it where it was.

Now this is not to say that refactoring is a bad idea. When it works, it works well. It is a useful tool. I am glad that it helped you on File::Find. But I think that most big projects can profitably use multiple modes. For instance the Linux project does both. Most of the time you do incremental ongoing development. But I think that ESR made the right choice when he made CML 2 a complete rewrite. Sometimes you incrementally adapt a component. Sometimes you replace it. You are doing something wrong if you need to replace big components very often.

And one final thing. Ovid is dealing with a system, one of whose problems being that it had a bunch of features added without much rhyme or reason because an ex-employee thought they were "cool". It does not have a large user base. I don't think, therefore, that he should build tests based on the current behaviour, enshrining the misfeatures in tests. Rather he should do some research about how the system is actually used, and only test for what people use from it. Whether or not he rewrites from scratch, blindly refactoring based on the current behaviour will not solve one of the problems that he wants to solve. And, whether or not he rewrites, he should think about how to solve the business problem. Perl 4 did not stop development just because Larry was working on Perl 5. Perl 5 is not stopping active development just because Perl 6 is being worked on. Creating great software is one thing. But you need to survive to actually do it...