Has that actually worked for you on a structural bases? Is it worth the time it takes? (Cause not only does the developer-to-be write the code, you need people to judge it). How do you deal with the fact many candidates are under stress? Are you giving them new, non-trivial tasks, or do you give them (part of your) existing code base, and ask to modify it - the latter is for many companies a far more realistic scenario.

From the articles I've read, even a trivial programming problem is useful. It proves that they can code, not just that they say they can. If you want to go detailed and dig into how well they can code and if they can work with your coding style all the more power to you, but that does (as you note) take time and effort on the part of the interviewers. But even a simple 'take two numbers from the user, switch which variables they are in, add them together, and print the result' is better than hiring them and then finding out they are totally incompetent at anything coding related.

I will often ask candidates to write down a few lines off code on a white board, or a piece of paper, but in those cases, I usually don't really care about the details of the code, as I will ask them to explain what they are doing.

But I wasn't getting the impression that was the kind of coding being referred to by the OP. I've heard from other companies giving coding tasks that exceed the size of a whiteboard - but I've never heard of any research done about the value of this. We've discussed this at our company (that is, giving potential hires a coding task), but we're rejected it; the cost of creating a task that's representative of the work we are doing is just too great. (And since we get in lots of candidates, we'd need to cycle between tasks, as people do pass around the questions that have been asked). We care more about people fitting in the development methodology than about their exact level of Perl coding (which can be trained).

But I wasn't getting the impression that was the kind of coding being referred to by the OP. I've heard from other companies giving coding tasks that exceed the size of a whiteboard - but I've never heard of any research done about the value of this.

I've never asked them to write large projects in an interview, though I've sometimes asked to see some large projects they have written (e.g. a CPAN module).

As indicated in the root node, I feel the cost of hiring the wrong person is enormous and so I feel it is well worth doing much more than a simple one hour job interview. I prefer multiple one-on-one interviews with multiple interviewers. During these interviews, candidates will be asked to write code and solve problems. Also, cultural fit is assessed (often by asking behavorial questions). BTW, Google typically perform five one-on-one 45 minute interviews after you get through screening. To offset the high cost of multiple interviews, you need to find cheap ways to screen out the glaringly incompetent and the resume liars and simple coding questions are a good way to do that in my experience. I cannot rigorously prove that all this effort is worth the cost. More details of my preferred approach to interviewing can be found in On Interviewing and Interview Questions.

I think the fallacy here is to assume that anyone working on the project needs to be an as top-notch programmer as you are. Usually, that's not the case. Many projects only need "expert" knowledge for a fraction of the project, but the rest is just "grunt" work.

Now, adding new "below average" developers to perform highly skilled work, such as finishing the Synopses or improving the Perl 6 parser seems counter-productive to me, a classic case where Brooks' Law holds. In which of the following areas would adding more people help finish the project sooner?

• Writing the Perl 6 specification
• Perl 6 development of parsing, compiling, ...
• Perl 6 runtime development (run machine, garbage collector, introspection, foreign code, parrot, ...)
• Perl 6 library development
• Writing Perl 6 documentation
• Development of installers
• Development of tools (e.g p5 to p6 translator, pod2html, ...)
• Development of Perl 6 CPAN
• Writing test suites
• Using Perl 6 and reporting bugs and issues found
• Maintaining the bug database, issue tracking, ...
• System administration
• Licensing
• Marketing, maintaining Perl 6 web sites, wikis, ...
• Release engineering
• Project management

I suspect that most open source projects (such as Perl 6) are well ahead of most commercial ones in terms of documentation and in partitioning the system into independent components. Accordingly, I feel Brooks' Law is less applicable to open source projects than to commercial ones.