I agree wholeheartedly, and think this approach is one doomed to failure on any large-scale project. Add functionality as you build (Along with making sure you have an up to date and comprehensive suite of unit tests), and then refactor to make the code tidier without changing functionality (And also whilst making sure your test suite keeps working). Rinse and repeat.
It does seem kind of ironic that the top-level thread started with the observation that people throw words like refactoring around with no real idea of what they are, and then proceeds to do exactly that.
I think I'm slowly being indoctrinated by the Patterns/Refactoring Cult. It took some of them a considerable amount of time to crowbar some of the ideas into my skull, and a lot of patience to help keep me on the right track (For that methodology, anyway), but now I keep my unit tests fresh and am at least able to talk patternese whilst designed architecture, which helps when dealing with others who're more than able to do such.