Whether or not Perl is "scalable" can depend upon how you defined "scalable". Do you mean "large projects can run using Perl as the primary language"? I would argue that they can. If you mean "many programmers can work on the same Perl project", then scalability can depend upon the strength of your organization. Do you have strong programmers and good standards in place? Do you have strong project management? These issues, from what I can see, cause problems to crop up quicker in Perl shops than others. However, it's important to note that these problems occur in any shop regardless of the language.

Perl, however, has a unique approach. Rather than increase the power of a team to work together (which Java can do), it can increase the power of the individual programmer allowing a smaller team to be more productive in Perl than a larger team working with another language. This can lower overall costs and increase communication (because there are fewer communication paths) and this is a benefit to the success, and ultimately the scalability, of almost any project.

I'll finish this by posting an email reply that I made to someone who was concerned about this issue.

I don't know all that much about programming. I am under the impression that Perl is fine for fairly low traffic CGI, but C would be needed when the site scaled beyond a certain amount of traffic. Please correct me if I am wrong.

That depends upon exactly what you are trying to do and what technologies you are trying to apply to it. For example, I know of a company (that I cannot name) where the programming team was working in C++ to develop a large Web site to manage financial data for customers. A friend of mine was their QA director. After I learned a bit more about the company, I discovered that they had quality management, a sharp QA team and solid financial backing. Further, they were offering their employees generous stock options. I urged my friend to quit the company.

This sounds a bit ridiculous, but the problem was due to their choice of programming language (languages like C and Java often suffer the same problem). It takes too long to develop in them. Quite frequently, using a language like Perl allows you to finish your program and move it into production before competitors have even gotten their first compile. With my friend's company, it took them two years to get their prototype working. This means that if it was successful, competitors using a language such as Perl (or Python, Lisp, or other language that's quick to develop in) would likely be able to duplicate their results in only a few months. Further, any new features that my friends company could produce could be duplicated in a much shorter time frame at a fraction of the cost.

My friend is now looking for a job. He was convinced they would succeed and every time I talked to him, I was impressed with what they were trying to do, but I was convinced that they made the wrong technology choices. The company has gone bankrupt because the technologies they chose simply took too long to develop in. (side note, not in the email: there was nothing that this company was doing that required C++. It was vanilla CGI with a powerful database backend that DBI would have handled nicely. And yes, I know that the DBDs are not pure perl :)).

For more information about Perl's success, you can read Perl Success Stories.

If you find that scalability is an issue with your application, switching to mod_perl will likely remove the bottleneck. Further, because of the way that mod_perl is designed, mod_perl programs often run faster than their C counterparts. You also have the option of embedding C in Perl for very time-sensitive tasks.

Note that the Perl Success Stories link above has wonderful examples of how scalable Perl can be with the right team using it.

Regarding Perl vs. Java: one of my favorite quotes is by merlyn (pardon me if I am misquoting). He was facing an audience of students and there was some discussion of both Perl and Java. One of the students asked him how he dealt with the lack of strong typing in Perl. merlyn replied, "I just smile and move my programs into production before the Java programmer has his first compile."

Cheers,
Ovid

New address of my CGI Course.
Silence is Evil (feel free to copy and distribute widely - note copyright text)