First of all I don't know whether Perl is declining. Certainly I know that some of the Perl 6 effort has done exactly what it was intended to do, and attracted effort and interest in Perl. I know that at my job we have replaced the vast majority of our work with Perl, and the directions we are considering away from Perl are not exactly popularly publicized ones.

Secondly I am personally not overly concerned with what the popular language of the day is. As I commented ages ago at RE (tilly) 1: Java vs. Perl from the CB, the dream that is sold to PHBs of programmers as interchangable monkeys doesn't appeal to me, and is a proven recipe for IT disasters. See Choose the most powerful language for further discussion, and a link to an excellent article by Paul Graham. As long as I have freedom to be productive, I will make the best choice for me. Often that is Perl.

Third I don't see it as a huge disaster if Perl at some point falls by the wayside. Perl is not magically great to push just because it is Perl. Perl is good because it does things very well. But other languages can adopt some of Perl's good ideas and do what Perl already does. Indeed languages like Python and Ruby borrow some of Perl's good ideas, and make them conveniently available to people who want some of the power of Perl, but who didn't happen to click with Perl. I think this is a good thing in the end. Trying different approaches allows people to figure out what they like and why they like it, leading to better languages later.

Perhaps I am being narrowminded in focussing so much on what makes for good personal productivity, but I don't think so. Lacking excellent marketing, Perl can't win in the hype game. It has to win by actually being better for solving problems. Sure, you don't see Perl advertised everywhere. But smart management understands that something is up when a small team of Perl programmers in 3 months manages to match what a fair sized team of Java programmers had done in 2 years. And when the Perl programmers come back again a year later and in a similar time frame do what the Java programmers had planned to do over the next 5 years...

Well, to start off, I think you're equating "popularity" with "success". It's true that Perl awareness is not as widespread as the other languages you mention. (By the way, I notice that two of them, C# and Java, are products of corporations who would like to run away with the market regardless of the technical merit of their solution). But when I first read the title of your note, I thought a lot of other things. To me, "decline" means disuse or death, neither of which I think apply to Perl today.

The fact that there isn't a company with a lot of money standing behind Perl is probably the cause of the phenomena you observe. The same applies to other software that is produced and maintained by enthusiasts rather than companies. Linux is a very good example. Up to recently, Linux was perceived as just a hobbyist's toy. Now there are small glimmers of its acceptance in the corporate world, mainly from IBM stepping up and putting together a marketing package that non-technical people can understand. (I'm thinking of recent TV ad campaigns.) But does that make Linux "better"? Now there's a way to start an argument.

I agree that except in certain cases (look at The Top Perl Shops for some examples), most companies don't "get" Perl. Last year at my current client, I suggested that a new application be prototyped using a Perl backend. My suggestion was met with something between ridicule and disbelief ("We don't want a bunch of scripts, we want a real program". That one stung.) To give them credit -- and this lends credence to one of your points -- they had very few people that could program Perl. And none of then were very good at it.

So has this lack of knowledge among some people made Perl any worse, or any less useful? No, definitely not. I think the ongoing work with Perl 6 is some of the most interesting and exciting stuff around. I think the language continues to be used in fascinating leading-edge application areas, such as bioinformatics. The state of Perl definitely doesn't fit my definition of "decline".

Nonetheless, I think your point of "corporate acceptance" is well-taken. It's not that the language is declining, it's that it's not making inroads in the average boardroom. How do you get past that barrier? For my part, I think the p5ee project is a step in the right direction. We need to simplify Perl training, which is one of the goals of the standardization, and provide something for a corporate executive to hold on to -- which is a topic of discussion in the mailing list right now. And the nice part is that the standardized framework doesn't stop all the wonderful and creative Cool uses for Perl that we've become accustomed to. If the lack of corporate acceptance is of concern to you, then join the group. "Dont' curse the darkness, light a candle" is an old Chinese proverb.

There was a post on the mod_perl list later the same day that I wrote this, that shows a steady rise in the use of mod_perl based servers. The graph.
Maybe a better title should be, Top Hindrances in Selling Perl Solutions

I agree with your timeline and general ideas about what brought perl into focus. I also agree with perl being adrift in a sea of choices and with the rush subsiding in 2000. I whole heartedly disagree with your arguments about why perl will decline. Let's not look at your summary but at the pillars of your summary.

1. Perl is thought of as a UNIX administrative tool.

You really don't have much support for this summary. You state that system administrators love it but so do a whole lot of developers!

2. There are not enough professional organizations or guilds to bolster confidence with corporations that investing in a Perl solution is a good long-term plan.

Well you're at one of the most professional guilds right now. I don't see a cold fusion monks site? What do you want? Certification? I think more of my BS and BA then I do of any certification. As for "good long-term plans" - very few business see past the quarter. While I think this is generally bad, I think it's going to work wonders for open software. Where can you trim the budget to ensure profitability - cut those huge $$$$ software licenses down to nothing.

3. Perl doesn't have large scale advertising and full time advocates that keep Perl in major computing publications and remind companies that when they chose, chose Perl.

Hmmm ... not sure I want to base my IT selection on what the mags have to say -- I've seen a whole lot of shelf-ware that was bought due to what some wags say in the latest issue of some Ziff Davis trash.

3. There is no official certification. I have seen Larry's comments on this and I agree with him, but lack of certification hurts in the corporate world.

The are only two certifications that count - one is years of experience and the other is a sheep skin. Anything else is pure garbage. As long as you have the fundamentals of algorithm design down - then who cares what the cert is.

4. Lack of college or university Perl class, or maybe better-stated lack of Perl promotion by colleges.

I wouldn't expect anyone right out of college to be productive in any language. I would expect them to know what makes a good algorithm - and that my friend is language agnostic. Be it VB, C, C++, or perl - you have to know big-o.

It sounds like you're a little worried about a corportations perception about our language of choice. I wouldn't be. Given the current perception of corporate management (ala Enron), I think the people who make the (ehh) long-range plans may be around a lot less than us tech weenies. Bring it in back doors if you want. Rename it if you have to - an SOAP enabled back-end XML processor may be more appealling then an apache/mod_perl engine (that's where the BA comes in).

It also sound like you're worried about overseas chop shops. Ed Yourdan rang that bell about ten years ago with "The Decline and the Fall of the American Programmer." I must say I lost a lot of respect for Ed on that one. Farming out development to India has proven to be more of a lead egg than the golden hen - time zone headaches and culture clash has proved very hard to overcome.

Perl is moving on ... it seemed static because everyone was catching up to it. That being said, some day other OSS languages may overtake it but Python and Ruby are still in perl's review mirror.

Just like loved ones, we tend to ignore those which are around us everyday. For this Valentines day, do us all a favor and by some chocolates and a few flowers for our hard-working and beloved partner - We love you perl.

-derby

Perl may have gained ground initially in System Administration, and since the Web came along, Perl now is though of more as the language of CGIs.

My previous employer also had a subsidiary in India, and my distributed project included a team there, working on a large (80K line) web application in Perl. Folks coming out of University in India are more likely to have been trained in C++ or Java, but Perl isn't unknown.

On Certification: Be very careful with this. HR departments might like to see certifications on resumes, but in the development teams I've worked in, a Very Big Flag is raised by someone claiming to have a certification. The implication, fair or not, is "this person is a Bozo."

   Here's the thing: to some extent, you're right, but at the same time, you're still way off base. Now let's say, just for a second, that the only thing Perl can / should be used for is system administration and web programming (please don't flame me, guys. I know that is by no means the extent of Perl's capabilities, but I don't have time to write a post covering all the bases right now.) Even assuming that's true, you're still wrong.

   Considering the web side of it, yes, Perl is being used in far fewer places now. There are two major reasons for this: one is the abundance of other languages (PHP and ::shudder:: ASP, for example). Another is the fact that a lot of sites backed by Perl programming crashed and burned when the dot-coms did. You know what? I don't see this as a big deal. The techies who wrote those sites are still around, likely still using Perl, and hopefully not hurting its reputation by using it to support companies that will never, ever, make any money or do anything useful...ever. (Not to say that all dot-coms were this way, but c'mon, there were quite a few useless companies out there.) These sites were thrown together (in many cases) to make a quick buck. Granted, Perl can be great for quick-and-dirty code, but do we really want it making up the majority of the volume of the code out there?

   System administration: I still think Perl is one of the finest languages ever created for system administration, especially cross-platform system admin, for those of us who don't want to learn the ins and outs of shell scripting 100x over. I really don't think there'll be much argument there, so I'll move on.

   The community: when was the last time Perl had organizations as devoted as the Perl Mongers and Yet Another Society? Do you see a website as popular and helpful as Perlmonks for PHP? Before last year, I don't remember ever having programmers whose sole job is to improve and evangelize Perl. Do you? I can't remember ever before having an argument on the Linux kernel mailing list on whether Linux should switch to a quasi-Perl method of patch management. (This past week, for those who haven't been reading the kernel mailing list or Slashdot.)

   To be honest, I have no idea what you're talking about. As far as I'm concerned, Perl is as strong as it's ever been. If not, then it's purely a matter of evangelization. I know I've impressed more than one boss by getting what would've been a several-day-long job done in an hour with Perl. Have you?

I'm not sure how you backup the statement that Perl is declining, but anyway.

There's a huge difference between writing sysadmin tools and writing business oriented applications. (Unless your business is to provide sysadmin tools. ;-)

In my shop, the sysadmins are free to use almost any tools, languages, etc. to get their work done. OTOH, when it comes to business supporting applications, with end user interface, the situation is very different. This is when our middle level management starts to worry.

My personal experience is that Perl does have a very short and low learning curve when it comes to writing different 'tools' to be used by IT folks.

The learning curve may quickly become long and steep when you want to create a business oriented application, very often combining techniques like CGI, DBI (or SybPerl), using non standard libraries written as OO or not plus adding your own modules to cover your shop-specific topics. Especially if you want to create some shared business objects to be reused by other Perl applications. Add to this that the CGI application shall create JavaScript to help the user orient through the application, sometimes even JS that eval()'s more JS and it becomes tricky. (Did someone mention 'security' as well?)

Furthermore, there is (at least in Central Europe) a huge lack of good training. There are commercial training courses, but those that I've found around where I live are all beginners courses covering the first chapters in the llama. Which is good, but not enough.
Because after the introduction course, my colleagues ask me how to proceed. When I tell them to go on-line, read books and otherwise participate, they are unhappy. Yes, many of them still haven't really learnt how to take advantage of the 'Net. And yes again, not enough people (fewer and fewer?) appreciates to RTFM. They all want quick solutions. And no, they don't appreciate to spend their evenings reading the Cookbook or the Camel. (Some odd colleagues, I admit. ;-)

You can repeatedly have quick solutions using Perl, but that requires efforts to learn. And this learning stage (where I'm still at) is not quick if you need to do something big.

Too many people (around where I make my living) want quick solutions to everything with no or little investment.
(Do I sound old? Yeah, I guess I am. That's how you become after 20+ years in this business. ;-)

Conclusion: In my shop, the middle management are worried what will happen if I leave.
Questions like: "Who will take over all this Perl stuff?" and "How can we get a new colleague up to speed on this Perl thingy within a week or two?" are commonplace here. Which in a way creates a resistance.

I'd say: Momentum! When there is a hord of Perl programmers available the middle management will sleep well during the nights. (At least concerning "The Perl Problem". ;-)
I'm working very hard to create the momentum that is necessary in our shop and I hope you do the same. (Biker points a finger at the reader.)

"Livet är hårt" sa bonden.
"Grymt" sa grisen...

I agree with most of this, but a major problem that I have noticed is that with the recent 'dot.gone burn', the (job) market is literally flooded with experienced Perl programmers, but there are no positions for them. Mostly this is due to Perl being free. To learn C++, C#, Java you've got to spend a lot of money. Courses do not come cheap, the software doesn't come cheap and certification (where it is offered) isn't cheap.

However, if anybody with just programming knowledge in anything from BASIC upwards and the willingness to learn can get hold for software that is free to use, free to download, free to 'learn by example' (the quickest and easiest way to learn IMHO) - you can probably have a Perl programmer that can make a basic script in next to no time. He runs his own website off it and other scripts he wrote, learns, looks at other scripts and before you know it, he's writing a complete Intranet based Management Information System in it... If that person had to get C++, learn by reading books and going to courses (and the associated costs - there isn't that much code available to read and learn by), it would have taken so much longer and if you are on a budget - it isn't an option.

Compare Linux+MySQL+Apache+Perl to (shudder) Windows 2000 Advanced Server+MS SQL Server+IIS+ASP. Which costs a lot more in the set up costs, staff costs and maintenance. Let alone security holes? But which do big corporations go for (even though ever techy knows which is the best one to go for). Why? Because 'oh, things are free for a reason - if we've got to pay lots of money for it it has got to be damn good - just look at all those ASP programmers asking 60,000UKP upwards, it must be good if they are charging that much'.

All in all, if Perl6+ had a licence fee charge for it and a 'certification' was made available AND Perl programmers put up their 'minimum wage', Perl would take off again big time. Of course, it's all IMHO, but you did ask for my opinion :-)

If you really think that by asking money and shutting up Perl, you would make it more popular and profitable, then I challenge you to go out and try to do it.

If you read the licensing terms, you can take Perl, take advantage of the artistic license, rename it slightly, and make your own version which can be proprietary if you want. (See oraperl and sybperl for examples where this was done with Perl 4.)

My prediction based on both theory and observation of past examples (particularly examples of what people in the Lisp world do wrong time after time again) is that you will put in a lot of energy, lose money, and never achieve popularity. For some of the theory, the usually referred to starting place is The Cathedral and the Bazaar.

Of course if you want to charge money for something and can get away with it, go ahead. No less than Larry Wall has said that, It's almost like we're doing Windows users a favor by charging them money for something they could get for free, because they get confused otherwise. But I think that as time goes by it is becoming more mainstream to accept that it is possible for software to be both free and good at the same time.

I know, but that's how head of departments and corporate management think: these are the people that believe the FUD that Microsoft put out ('XP is more secure - we best get it then', never mind that Linux is more secure and the Windows 2000 machines are also secure). Sometimes it's just the brand name which also helps - I know of a certain sausage manufacture who makes sausages for two major supermarkets. People say 'Oh, that's from Supermarket X' upon tasting, although it is just the same sausage.

All in all - it comes down to packaging. 'Tart' something up by packaging, brand names and high prices are, despite the rival product being better in every respect, the 'well packaged' product will win.