In summary, if you code more than one app of a type, you owe it to yourselves to factor out the common parts into some sort of framework. Using a framework already on CPAN gives you others to help you with both your individual apps and with the framework internals. It's much easier to come up to speed on a public framework, for which there are tutorials on line, talks at conferences, getting started movies, and more, than it is to pick up a framework that was created in a single shop and never given the documentation (and code review) which are the inevitable result of external users.

Please don't think this is only a shameless plug for Gantry. (It is that, of coures.) These benefits come from having groups of users in more than one setting.

A very good response, and it did in fact open my eyes that little bit more. :o)

Although I do keep thinking that if I walked into a job I'd wanna be hands on with what it's all doing behind the scenes, maybe it's the geek in me.

I'd also want others coming in to the production team in a few years time to have that same exposure. Not just doing a bit of code in the middle that doesn't expose them to the raw elements.

At this rate we'd all end up as VB developers simply doing a bit of point-and-click to earn a wage.

for ( qw( HTMLparser Templater ORM Framework) ){
   # reinvent as a rite of passage if you must
   # but, shudder, don't use that in production
}
[download]

update : fixed spelling error (and, as a cultural anthropologist, you would have thought I would have known the difference between "right" and "rite" without rhesa's help)

Especially if it'll be a team of 10 developing with me, and 8 may leave in a month, then trying to find people with all those skills, which are certainly a bit hard to come by in my part of the world.

• think the other way around. every new person you hire might know one or the other framework, or maybe not. that doesn't matter, they must *all* learn *your* framework. if you were using a common cpan framework, some of them might already know it, and if not, they can learn it and they'll find help on perlmonks.org, for example, because they're not the only ones using it. yes, programmers can learn, and some of them even enjoy it.
• again, think in the perspective of the programmer. if you're hiring, they might not know catalyst, DBIx::Class, but that's no reason not to hire them because they can learn it. if you insist on your own framework and for some reason they leave your company after a while they have learnt a framework which isn't used *anywhere* else in the world.
• think about *your* perspective in the future - in some years maybe most programmers know catalyst and you can't hire them because they don't want to work in a company where everything is proprietary.
• don't think "it has worked for me all the time, why should i use something additional?" this *is* how some people think who haven't used better things. if you have used ORM or a web framework it is hard to go back to plain DBI or plain CGI scripts. of course an ORM might not fit everywhere, but then some other technique that saves the developer time.
• i'm not saying never ever roll your own, but you should at least know a bit about the existing modules out there before saying i don't want that.

tinita is right. Rolling your own often makes it harder to bring new people in. And I'm always suspicious when I interview at a company and they tell me they wrote their own proprietary versions of things available on CPAN. It has the appearance of a "not invented here" attitude.

Once upon a time (for years actually) I invented my own templating system. Then I discovered HTML::Template and gradually moved to that finding it saved time. I scoffed at Template::Toolkit and Mason. Now I still scoff at Mason :) because I don't feel like learning it (I may learn and stop scoffing at it one day though) but I do use Template Toolkit 2.

I selected TT2 because I selected Catalyst, but I like it very much (not my ideal but quite useful). Honestly I can say TT2 is a big time-saver. I fumbled a little until I was able to use it to make an email template as I used to use HTML::Template, but I know that it is capable and I just have to learn more to unlock more features. For example I discovered belatedly that it can do some very useful filtering (FILTER command). I also wrote a wrapper to make plaintext templates but realized this also was already available. With DBIx::Class you can throw a resultset at a template through the Catalyst stash and quickly make nice-looking reports. It is so easy that you are tempted to do it over and over but probably you could save even more time by making some generic report template (I think this should be a Catalyst extension myself). And you can even access objects related to those retrieved from within the TT2 template itself, without looking them up in your program even.

So I do recommend TT2 as a big timesaver and quite powerful and easy to update tool. Personally in catalyst I do a regex on the template file name and use that to choose which CSS file to use (i.e. "public" or "portal").

As for databases, I used to use hand-rolled DBI. Worked great, but then I moved to Class::DBI which I liked a lot.. although it seemed slow and a bit wierd or hard to find out how to do something, sometimes. Now I used DBIx::Class (because of Catalyst) and I like it. However I found there is a little bit of a lack in describing philosophy and recommended approaches to newbies. If you already know a lot about databases then it is great. The user community is great too. I think there is far too little written about ORMs, and also about just how much performance is given up. Also there is too little written I think about how a perl programmer should approach designing an app (when he is thinking in OO terms to start with) when a database and ORM is involved. In particular (I posted on the DBIx::Class ML recently about this) docs about how to design a db with the DBIx::Class::Relationships module could be improved. So I am not the person to say anything definitive about ORMs but I certainly like using DBIx::Class better than the Class::DBI code I used to write. It feels more robust, there is more extensive functionality, documentation, and support I think, and I think you can do more without thinking/writing SQL. For example I used to write a number of custom SQL searches but this is more perlish now.

I used to invent my own form code over and over, tried some different modules, then moved (after using Catalyst) to HTML::Widget which felt clunky and obscure (and is being obsoleted by a new module made by its authors) and settled on FormBuilder which I like very much, it even makes Javascript popups automatically for you. There are more data validation modules out there to try if you wish too. I definitely recommend using FormBuilder (the CGI version or the Catalyst version) since you just define you form and it builds it for you, and just tweaking a single setting will make it editable or not. I no longer need a designer to make nice forms.

Now as for frameworks, there are several and I haven't tried them all. I wish someone would write about experiences with all of them and characterize each one.

So once upon a time I built my own framework. I've built my own encrypted persistent forms, my own sessions, etc. Then I moved to CGI::Application and that seemed quite useful. It still took a long time to make big apps but there are useful plugins for it and it organizes things, which was a plus. (I haven't used it in a while but it is probably great.) Then I moved to Catalyst, which I like a lot.

It is a massive time saver, and has many plugins that make many things just work, like authentication, role-based authorization, sessions, breadcrumbs, etc. I think here too it could use more docs but it is extremely useful I found, and it works well with TT2 and DBIx::Class. There is plenty to continue to learn about with it but it also works well.. I had a standalone perl demon for testing right away and it was a snap to install a FastCGI persistent server too. A quick start doc would have been useful to get past the initial trepidation and actually understanding how it works. Also with Catalyst there is a tension I think between perl hacker types wanting to stuff everything into the Catalyst Controller (the module holding subroutines that are keyed directly to URLs) and rolling code into the Model modules which would allow you to call them from the command line or other kinds of interfaces. Some docs about that would be useful too, I get it from the mailing list.

So I think if you are making a simple CGI app you can use the CGI:: modules as they are I think, since some of them seem to just be updated for Catalyst separately, but it would be a very good idea to try something very simple in Catalyst, or in a few other frameworks, to get your feet wet and see what it's like. I think Catalyst would be very good for a team of your size since you can break things apart into modules for different people.

In particular, I have found with Catalyst that I can quickly prototype a function and then push through alpha, beta and live stages on a per-function (per-url) basis and feel very safe that the whole thing won't explode. I don't use tests much yet in it yet but there are hooks for them too. Anyway this means I was able to have very short cycles and was able to include the client in a dialog about each function. I used OpenOffice Draw to make the most important flowcharts and a limited number of screen layout mockups, but stopped that and just moved to prototyping with TT2 after a while. So I can say that Catalyst makes it easy to keep subroutines small and modular, and I think it even reduces the time to develop. In particular it was quite easy to add ACL rules (to control whether public users or logged in customer or admin can access certain URL paths) and tweak the dispatcher (in Root.pm) using regex tests on requested urls too, making an exception or an error page if I needed to.

So anyway I've turned into a Catalyst fanboy and while I'm interested in others (Jifty? Gantry?) that might be even more streamlined, I think as long as you get it to build with automatic follow using the cpan utility (since there are a LOT of modules involved), Catalyst will save you time, keep you sane, and provide fertile ground for further learning and development. In particular it is quite easy to make a proof of concept of a new function and then quickly implement a demo, then roll it into the live site. Built-in Catalyst modules enable many such functions to be done with minimum coding. It is naturally suited to short, incremental development cycles and promotes organic development and upgrading of functionality as you go. Most of the time I find I am documenting too much or spending too much time reading/fretting when just diving in and seeing how it feels will save time.

Catalyst

Well the ORM side is about the same kind of philosophy Phil is talking about. Use only a common denominator subset of many database engines, add it a few best practices and you get closer to the goal of an ORM. Also once you have started hiding the actual database (to the point it does not matter what it is!), you can start thinking about even more powerful abstractions.

The other day I came across DBIx::Perlish and wondered if it was another full-fledged ORM...still there was that 'perlish'. The author addresses just that in the docs.

One thought that I believe is that you should only really use such tools/modules once you've done it fully yourself manually.

Have you written your own programming language then? Do you only hire people who've done so?

I think both sides of the argument are being overstated here. The truth, as happens so often, lies somewhere (IMO) in the middle. To use your analogy, whereas it shouldn't be a go-nogo criteria to have written a language, I remember that the first time I did it it completely transformed my coding style from single diatribes with lots of ifs to tight multi-level routines of one or two lines each. It meant that instead of development and testing for periods of arbitrary duration I could suddenly write high quality work to a deadline. So yes, I for one would be influenced by that in a CV, though of course there are more usually more direct criteria relating to the job in hand that are going to take precedence more often than not. So yes, I agree with the OP insofar as it is a huge advantage to have done something from scratch - modules and other tools can often replace effort, but they can't replace knowledge and experience.
__________________________________________________________________________________

^M Free your mind!

Although there are some great replies that I'm sure have helped you narrow down your choice a bit, I'd like to throw in my 2 bits.

What i've found is through combining CGI::Application, Template::Toolkit and DBIx::Class i ended up reinventing the wheel and writing login methods, session tracking methods, authorization methods, etc., that had already been done, tested, tried and true by the likes of Jifty, and Catalyst. I worked with both before I decided that Catalyst was my true calling, even after writing Catalyst off as too complicated and "expensive". I chose Catalyst because of configurability, speed, and the ability to get a pretty substantial app up and running in a very very small window of time.

Perhaps you'll have to go through the same rite of passage as I did and realize that the most common web methods such as logging in and CRUD/BREAD have already been written for you via a very well tested MVC.

Hope this helps!

In my building web applications I have taught myself through much pain and suffering .. *one* good way (perhaps of many) of working.

My way (likely a shared view with some others):

The "web application" is an oxymoron. There is no web application- there is only a web interface.

Why? I keep all my procedures, database things, interaction, methods, whatever- as much as possible (usually this is just about everything)- completely appart from *any* interface but the API itself.

Then my tests must interact with the code.

That works? Then I may do a cli interface, *then* I use CGI::Application and it's minions of plugins to interface.

Because of the nature of the web and multiple hordes asking for the same junk everytime, I may resort to Cache::File.

I'm sure my mantra of a web app is not unique and it's not the only good one out there. It doesn't hold imperative for me if I face a simple need such as a picture gallery. But growing into it has really helped me out with larger things.

The web application is an oxymoron. There is no web application. There is only a web interface.

On the other hand, ORM fans praise their "speed of development", and if you don't expect to need to scale up to heavy database use, you might go with the ORM approach instead.

Something I keep meaning to experiment with some time, is developing with Catalyst, then benchmarking/profiling to look for bottle necks (preferably under simulated heavy loads), and then try to refactor around the bottle-necks with the hand-crafted SQL approach. It's at least concievable that there's a best-of-both-worlds hybrid that might work well.

What I'd actually like to see though is a framework that starts with a database schema and creates code around it. I don't understand why I haven't run across one of these yet... it seems like a pretty obvious notion.

Have you seen Bigtop?


holli, /regexed monk/