Well since I just had a huge debate with my boss about reinventing good wheels, I may be a little on edge about this subject so please try not to take this as an attack, it's not.

reinventing good wheels is a waste of time. Why is it a waste of time?:

Now notice I did not say that CGI.pm is the be all, end all, nor did say improving the wheel was bad (who would want to live in a world with out sporty rims), but rewriting a perfectly good piece of code and in the process introducing security holes (your admission) and broken functionality (another admission) is not only a waste of time, but a sure way to find yourself in a heap of trouble.

your points (as I see them):

In line with grep's comments, I prefer using mature modules because they do things I don't know about, may have forgotten, or didn't know that I didn't know.

What I don't know can kill me, and what I don't know that I don't know will send me to hell afterwards. :)

Not that you don't have a right or freedom to reinvent anything you like, it truly is a valuable learning experience - if you learn.

0.02

I know how you feel - for your specific purposes, it seems much easier to do "just the job at hand", instead of bringing in the big guns that do much more than you ever thought you wanted.

I'm currently exploring the Coro module and I'm adapting other modules, for example HTTP::Daemon to use nonblocking sockets via some glue socket code. It would be much easier to code my own HTTPd (and I already have done so). So why do I spend my time writing a IO::Socket subclass instead of simply writing a new http daemon ? I do this because the Perl modules give me leverage - the possible errors I will make by misreading the RFCs and the possible incompatibilities I would encurr with the vast varieties of web browsers have already been worked out with HTTP::Daemon, and I don't have to worry about the majority of them - plus, HTTP::Daemon is maintained by someone other than me, so all my dumb questions about it go to that person instead of myself.

The points you criticize aren't really that valid, as CGI.pm already can deliver your parameters as a hash, and where is the sense in having several identically named parameters, if you only care about the value of one of these parameters ?

If you bring in security, there is only one answer - there never is enough. You don't want to roll your own without having studied the existing solutions and knowing where they fall short. The approach of fixing a hole when it's been found does not apply in security, as it may then be already too late, and your mailserver already has sent out several thousand mails in your name all over the world - something you might want to avoid.

Of course, the learning experience is there with rolling your own version, but for my part, I'm content that CGI.pm provides me with everything I need in an easy fashion (and the documentation for CGI.pm answers all questions I have about it except why they used ugly AUTOLOAD style).

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web
[download]

I'm content that CGI.pm provides me with everything I need in an easy fashion (and the documentation for CGI.pm answers all questions I have about it

Well, with >3K lines of output from "perldoc CGI", yeah, I hope that would answer all possible questions, but it does present a daunting hurdle for anyone who hasn't used it yet. This is a problem inherent in any big module that does everything in a big domain. For a module like DBI, this is perhaps unavoidable; if it doesn't cover everything, it won't usable at all. But I wonder whether CGI.pm might be better if it could be broken into smaller modules that could be documented more clearly and learned more easily. (Can it be?)

I've only used CGI.pm once myself, and because I'm comfortable with Perl and big man pages, the module saved me a good bit of effort and grief. Still, my trial-and-error cycles tended to dominate over getting things right based on the documentation. Next time I need to do this sort of thing, I'll try CGI::Lite (or something of that nature) first, and hope that will meed the need.

If I end up having to use CGI.pm a lot, and easier versions don't work out, I'll probably write a wrapper module for myself -- something that will "use CGI;" and provide just the methods that I really need, in terms that are easy for me to use and remember. Then, hopefully, I'll never need to look at the CGI man page again. This is a good exercise for a newbie: write something that makes it easier for you to use a monster module.

CGI::Simple vs CGI.pm - Is twice as fast good enough?
RFC CGI.pm refactoring

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

I've written exactly 2 programs with CGI (I do mostly other things with Perl than Web programming). I used CGI, because I know very little CGI, and Lincoln Stein seems to know a great deal of it. Most of what I post here is from typing <samp>perldoc CGI</samp>. I never read the source for CGI.pm.

Some answers:

How do i tell what was sent via 'get' and what was sent via 'post'?

See CGI's request_method method. There's even a section on "Mixing POST and URL parameters", if you really want to go that way.

What exactly is cgi.pm doing to the vars it slurps in?

This I couldn't understand, probably due to my lack of CGI experience. What vars is it "slurping in"? Script parameters? In other words, what would you like it to do, that it doesn't?

Accessing parameters in a hash

This is a true lack in CGI.pm, at least in the docs I have on a non-serving machine (2.46). (But a fix (using the param method, both without and with a parameter) should be easy to code.)See Corion's note on this topic; it's been addressed.

[I]f you have say, 'name=soemthing&name=blah', wont cgi.pm return an array for param('name')? What if your code doesnt want an array, what if you just want a nice little scalar value?

If you want a nice scalar value, you need to decide which of the possible scalar values you'd like to have. The param method returns different values in list context and in scalar context, so you could use "my $name = $q->param('name')". If, for some reason, you needed to get the middle value, you'd probably say something like

my @names = $q->param('name');
my $name = $names[$#names/2];
[download]

Think of it this way: if you want to use a list as a scalar, you need to decide how you want to do it; there is no "natural" way to do it. But you'd also need to make the same decision in your own code.

using your own solution may be less secure, but you can fix that!

True. But you're forgetting:

1. To fix it, you need to find it. If the bad guys find it first... In my case, I'd need to learn a lot about possible exploits against CGI scripts. CGI.pm might well contain security flaws, but Ariels_CGI.pm is guaranteed to contain them...
2. Laziness. Why should I chase down security flaws which have already been nailed down? What about new flaws? If there's a security flaw in Ariels_CGI.pm, I have to fix it. In CGI.pm, I stand a good chance of someone else fixing it for me.

HTML-generating methods

It's a free country (I hope). You don't have to use them if you don't want to. In one of my scripts, I didn't bother to use all of them, either.

However, regarding the difficulty finding unclosed brackets, a good editor and/or indenter (I use XEmacs) can give excellent feedback for Perl syntax errors; correcting nesting of HTML code embedded in Perl is harder.

Learning about CGI

Definitely an advantage to rolling your own. (If I had to write CGI in a language without the purty functions, I'd probably just wish I could use Perl...)</pp>

I'd mostly be worried about security. It's not so much about fixing the holes as about finding them in the first place.

But as you rightly note, TIMTOWTDI. As a programmer, it's your job also to evaluate the pros and cons of different approaches, and solve the problem the right way.

So let's see, basically you don't want to use CGI.pm because you did not write it, hence it doesn't behave exactly as you want, plus you don't use all of it.

Now why don't you apply that same line of thought to Perl, after all you did not write it, and maybe you don't like the $#array construct, after all no one seems to like it that much. Plus I would be ready to bet that you don't use all of it, like blessing colsures into objects, or anything that Dominus or TheDamian do routinely in their abject perversions of the language. Plus the code is huge, and hard to read, so really, you should not be using it I think.

And what about the web server, that's the piece of code that calls your CGI script, remember, if it's Apache, man what a huge software, hard to read, do you use all of its features? I hope not! And if it is IIS you can't even read the code..

No really, logically you should code the web server and your CGI script, all in assembly language.

No? Doesn't make sense? Then maybe you should realize that using trusted code is not so bad after all. I know if I was working with I'd feel _very_ uncomfortable if I heard you explain why you don't want to use the best tools available.

chromatic has a nice piece on it in his journal on Use Perl.</p.

I quickly learnt CGI, but found that Pascal/Delphi was more effort than it was worth to use with CGI, so I went off and got a Perl book and learnt that instead. Perl seemed popular, and there were plenty of scripts lying about free to use.

Though I had done programming before Perl was very new to me, and it took a while to find the correct resources. I found an old Perl4 CGI library "cgi-lib" and played with that. Back then CGI.pm wasn't distributed with copy of Perl I had, Matt Write didn't use it and anyway I didn't understand it.

I wrote some scripts based upon a customised version of cgi-lib and all was well. I was happy and it did "exactly what i want".

I quickly found that it was not ideal:

• If you distribute your scripts then you have to distribute your library
• It didn't do everything, so you have to for ever expand it
• It wasn't documented, so other people couldn't use it
• It was very unsafe

I had a problem, several script based on old and obsolte code, that was getting messy, difficult to distribute, and had no future.

In steps CGI.pm. Even in the days before ActiveState's PPM, I was able to install it from CPAN on a NT3.51 box. I believe it's shipped as standard with all versions of Perl now. Okay I now had to change my scripts, but I got the following:

• Properly tested code, Perl is easy, CGI is easy, but getting Perl/CGI right is hard. I'm not an expert in CGI, Perl or security, so I use code that's better than mine, and it'll be as good as possible.
• You can buy a book for this module (ISBN 0471247448), and the built-in POD is excellent. It's very well documnted, you don't have to read the source to figure out what it's doing...
• You don't have to use all of it, it goes out of it's way to load only the bits you want. Don't like it's HTML generation methods, don't use them.
• Comes in traditional and OO modes, pick the one you like best.
• If you read the docs you will find it's very flexible, and very easy to get it to do most of the things you may want it to do.
• Most modern scripts and good books will assume it's thre and use it. It is not perfect, but it'e very good, and very available.

Since switching to CGI.pm, my ego may have been deflated, but my code is simpler, more secure, and easy to distribute now.

My humble 2p

What exactly is cgi.pm doing to the vars it slurps in?

I realize i could look at the source of cgi.pm, but since its something like Six Thousand lines, its a little hard to find exactly what is doing what.

Also, what is with the param('paramname') usage? It just doesnt make sense to me, i prefer a hash.

You may not, but isnt the motto 'timtowtdi' (or however thats spelled).

And (unless im mistaken) if you have say, 'name=soemthing&name=blah', wont cgi.pm return an array for param('name')?

What if your code doesnt want an array, what if you just want a nice little scalar value? Again this comes under the heading of 'what is it doing that i dont know about?'.

#How's this for a nice scalar value?
perl -MCGI -e"print scalar CGI->new(qq{1=2;1=3;})->param(1)"
[download]

Alot of arguements against using your 'home rolled' input parsing libs revolve around security, and lack thereof.

Yes, using your own solution may be less secure, but you can fix that!. Find an exploit (such as the null character..), and fix it. Its simple. Theres no reason (if you work at it) you couldnt write a function thats just as secure as cgi.pms. Its all perl.

"Using functions to output wellformed html". Right, like h1('text'); is any easier then print "<h1>text</h1>";? Its just as hard to remember to close your parens and so forth as it is to add closing tags, not to mention the fact that not closing your html tags just makes the page look kinda ugly, whereas not closing your parens breaks your script. (you could argue that then the compiler could catch your error, which is a small advantage, but you could easily see and find your error once you via it in a renderer. *shrug* i dont think that its that big of a deal, but ymmv).

Making my own functions to output cookies and parse incoming formdata (of various sorts) also provided a valuable learning oppurtunity, as i got to learn a lot more about how webclients work with headers and submit files in forms and so forth. This is a small benefit, but a benefit none the less, say, if i wanted to go write a cgi program in some other language that doesnt have nice purty functions to do it all for you.

Around the beginning of this post you said "seeing that 'use cgi.pm' is such a favored mantra here, i thought i might introduce a different perspective on the matter" ~ your perspective is that of every cow!bie who didn't want to use CGI, and simply don't apply (learn all you want, but perlfaq is there for a reason, cause it's already been done well).

--superman

However CGI.pm is heavily recommended because it is probably the best tool to do the job for lots of people who do a lot of CGI work, and it has been hammered on alot so we know it is robust. Professionals using it should understand the underlying protocol relatively well but not waste their time on things which are not central to the task at hand.

One of the most powerful things about Perl is the CPAN module network and being able to leverage off of the countless hours of time invested in this community resource by people around the world. While CPAN does not always have the answer, it often has some very good stuff so I often look there first, among other places. I learn and look at source code, usually, and do look for all tools available to enhance security because I have a responsibility to the person who is paying me to do it.

So while I applaud your enthusiasm for learning perl, I'd have to say that in general learning is one thing. Intentional ignorance is not defensible. There are very good reasons why seasoned programmers who are totally capable of doing what you did (and likely have done so many times) make a point of not doing so. Read tons, make your mistakes on your own time, and become a great programmer! We'll all use your modules if they do the job.

I remember when I first learnt CGI.pm. The first thing I did was stare into the perldoc. I must have pressed enter about a thousand times. But I read it. And I saw in the examples that it was so much easier to use it than anything I could do on my own.

Why was this? Mainly because you can use as much or as little as you like. It goes with Larry's Perl philosophy in this way - it's very easy to speak a subset of CGI.pm. And most people do.

This is why I fail to see the fascination in writing your own param parser. They all end up looking the same anyway - some fat regex to decode the parameters, and setting a hash element to them as they arrive. It's not unmarked territory. It's not even fun.

With CGI.pm, you can use your own methods of HTML generation. It's up to you. As is everything. I don't see why you wouldn't want it to "slurp up STDIN". That's how you would have done your param parsing anyway, isn't it?

Writing a CGI.pm alternative is simply a waste of time, time that could be better spent analysing and augmenting the design of your script itself. So you don't know exactly how it works. I bet you don't know how a lot of things work. I don't think you know what your OS is doing every single second, but does that make you want to go and write your own OS? (Rhetorical.) Even if it did, would you go down that path?

Anyway, not meant to be harsh, just some things to ponder on.

...my "home rolled" cgi-parser may not be the best, fastest, most secure function in the world, but it does exactly what i want and tell it do...

• The functionality of all CGI is to slurp up stdin on a post. Is this where you are having problems?
• Arrays are returned for parameters in case multiple values are returned for a single parameter. With your question, what value for name would be returned in a scalar? How else do you handle two values with one variable> Also, it would be returned as an array in Java and PHP, I believe.
• "Using functions to output wellformed html" - Don't assume there is one way of doing things in perl. Many monks use a templating system of some sort to do the HTML output. You can do it with CGI.pm if you wish, but I wouldn't suggest it as a final solution.
• You can get the request method by checking $ENV{REQUEST_METHOD}
• I'm also sure that a module could be written as secure as CGI.pm, but why chance it. CGI.pm has been used for a long time, and if there were a major security issue with it still, it would be known by now.

Rather than bash a module simply because its not doing what you think it should do in this forum, why not just ask the questions, or do some of the research on your own? I'm sure many of us would be glad to answer your questions, and I, for one, would definately suggest that you look into a copy of CGI Programming with Perl.

I'm amused by that "what's wrong with the wheel" commant so much I may just have to re-use it mysqlf someday

Once upon a time I wrote code with the same mindset as the original poster. But two years ago I stopped using Windows for anything serious. And since then I've learned to trust the work of others, so I'm now able to take that little extra effort to ensure others can trust my code.

So I think the reinventing mindset stems from the fact that with Windows you can never be assured something outside your control will do it's job properly, and that writing what you need is simpler than learning all there is to know.

It's not about trusting other people's work. Admittedly, my feeling is that many coders don't want to use Module::X because they can't figure out what the hell it does.

On the other hand, different people think in different terms. Maybe I want to rewrite Module::X because it doesn't fit the way I think, or to address problems in my situation that are not a problem for the majority of users of Module::X. Or maybe I think I can do a much better job or just want to sharpen my skills.

I started off writing games for the 286/386 mostly on Extended DOS using Rational 4G. I had to create everything myself. Every video card chip with modes above 320x240 worked in a different freaking way (planar/banked/etc). You had to write assembler to drive them all. (Over 8 chipsets when I stopped.) I know the hell of having to write bimodal interrupt handlers that can handle the context switch from real mode to protected mode. It's not a place I every want to go again.

When I found Perl and the huge amount of code on CPAN, I was amazed at how simple it was to build complex and diverse apps in no time flat. So for me, it isn't a matter of trust.

-Lee

"To be civilized is to deny one's nature."

1. var=1&var=2 would of course make var=2; (hey, it makes sense to me)
2. a lot of my problems was that i didnt know there was sucha nice spec page (Here) And if i had known it, i probably would of ended up using cgi.pm. Nothing i saw in my various researching about it ever said 'go look at perldocs.com/cgi to learn alot more about it'.
3. i wont show you my code because i *know* its broken, what good does having you tell me do me?
4. 'what is it doing to the vars it slurps up' by that i mean, when it grabs all the input from STDIN and parses it, what exactly does it do? Does it convert the escaped %xx characters? Does it convert other things?
5. no, i wasnt paid for it, it was solely for my own usage.
6. Yes, perl may be doing things i dont know about, but i can read lots and lots of documentation about it (i didnt know there was a perl docs site!)
7. After reading up on the perldocs site, i think i will switch my code over to cgi.pm (maybe...), But i am glad that i took the time to learn more about how the cgi protocol actually works.

As a side note, if perl's binary directory is in your PATH you can usually say perldoc CGI to your shell and read the documentation offline. :)

--
amoe

Laziness, impatience and hubris, they say, are things that make a good programmer, or at the very least, a good Perl. Hubris, though, should not mean arrogance. These questions you ask are both intelligent and ignorant. Which is to say that they are good questions to be asking, but they are easily answered by reading the documentation for CGI.pm.

First, 'var=1&var=2' means that you have checked off two selections from a list or from checkboxes and that 'var' has both values. Of course, CGI.pm handles this elementary condition flawlessly.

Second, CGI.pm de-escapes all input so that even if it came in encoded in RFC-23190 ROT-13 EBCDIC, you wouldn't know it, but then, why would you care? You can re-escape at any time, at your discretion.

These two questions are merely the tip of the proverbial iceberg, and there is a whole lot of work going on under the waterline inside CGI.pm. It is probably better that you don't know exactly what it's doing, because some of the things that is has to do are quite unfortunate and unsettling, but that is how portability is achieved, especially with things like IIS.

Simply, it will take far less time to understand how CGI.pm works than to develop a replacement that is even functionally equivalent in the most basic sense, regardless of the skill of the programmer. In fact, the better the programmer, the faster they will likely understand how CGI.pm works, and how it can be utilized effectively, which is a lot less to think about than the myriad of details and obscure workarounds involved in reimplementing it.

They say that when re-engineering something, you can do 80% of the work in 10% of the time, but that the last 20% can seem like an Archimedian struggle, where you are moving closer and closer to completion but are never able to get all the way there.

Of course, this does not prevent experimentation or pseudo-academic reimplementations for the purposes of discovery. However, using these academic versions in a production environment is irresponsible and naive.

Two points:
1. Writing your own CGI.pm or Text::Template or Data::Dumper or whatever is always a good idea, unless you're pressed for time. I didn't bother with CGI.pm for the first dozen CGIs I wrote. (I also didn't make those CGIs available where anyone remotely unfriendly might use them.) I even started writing my own module. And I refuse to agree that it was a bad idea, because I now appreciate CGI.pm more than I ever would have if I had mindlessly followed the "writing a CGI? Use CGI.pm!" advice. I know more about the sorts of problems it's handling for me, and when things break it's likely that I'll be able to immediately infer the root cause of the symptoms I observe.

   Note that I am not advising publicly deploying anything that relies on a home-grown module in an area you're not an expert. But if your aspirations are anywhere above being a mindless code flunky, you will have to reimplement a whole set of wheels. Any experienced programmer will tell you, usually with a self-deprecating laugh, of a whole set of lumpy wheels they carved themselves even though in retrospect it seems like a waste of time.

2. The assertion that it doesn't matter whether you understand what's going on internally is ridiculous. If you follow that path, you'll find yourself in a room full of monkeys typing frantically on keyboards, changing things randomly until the compiler stops giving those damn syntax errors. We all started out that way, remember? I've inherited a lot of code from people who are still like that in a figurative sense, and that shit ain't Shakespeare.
I know I'm reading too much into the comments I'm responding to, but I just think it's a bad idea to chastise someone for reinventing the wheel. If you manage to convince everyone, there'll be nobody left who really understands how things work. And don't worry -- they'll eventually figure out the advantages of reuse for themselves. In fact, they'll understand the advantages better because they'll know what a pain it must have been to write the stuff they're reusing. As long as it doesn't clutter up CPAN or end up on an accessible (exploitable) server, it's purely a good thing.

Just a couple notes, from a W3C Secret KGB Police member:

• No support for any variant of XHTML (currently 1.1)
• No support for CSS of any kind

It says though, right at the bottom of the CGI manpage, that in the future CGI.pm will be broken up into more smaller projects.

DakeDesu teh Confused Werewolf.
"Apparent confusion is a product of good order" -- Sun Tzu, The Art of War, Chapter 5.
In perl, only those who are bless'd are treated as objects
(Webpages under construction)

No support for CSS of any kind

You might be looking at an old version of the CGI.pm docs. Here is a newer version

grep

grep> cd /pub grep> more beer

I looked at that, and I have never been scared so much by code... somebody... hold me.

But seriously, you cannot put a h# inside a span, and I do not even want to know what using -style in start_html() does... my nightmares have been covered since my mother asked me for my opinion on the Victoria's Secret catelog she got in the mail (One of her, is very sadistic)

DakeDesu teh Confused Werewolf.
"Apparent confusion is a product of good order" -- Sun Tzu, The Art of War, Chapter 5.
In perl, only those who are bless'd are treated as objects
(Webpages under construction)