Rule of thumb: code it in Perl. If it's fast enough, job done. If it's too slow, profile then either fix the algorithm, or push the slow bits into C using XS.

Perl is very fast for prototyping, and if the result is fast enough then you can spend the time you saved tossing back beers or whatever. Very often you can stick with Perl for the vast majority of the code and concentrate on speeding up the bottle neck chunks in C and still have time to swill a few beers at the end of the day. Even if you end up rewriting most or all of the code in C, at least you have a good tested prototype to work from.

I would correct a little bit on your rule of thumb: before coding in Perl, search CPAN for solutions that you can integrate. More often than not IME there will be a nice, optimized, XSed module that does the slow. YMMV, of course :-)
[]s, HTH, Massa (κς,πμ,πλ)

That is simply part of "Code it in Perl" is it not? In like fashion one might long ago have said "Code it in FORTRAN" then optimise in assembly language, or "Code it in C" (which is almost assembly language anyway). (I've written in machine code, assembly language, FORTRAN and C, among other languages btw.)

---

Perl reduces RSI - it saves typing

1. Start with Perl
2. If it's too slow, think about optimizations
3. If there are no obvious optimizations, either incorporate a fast XS module that does the most work, or write in C instead

The default choice of perl is simply because I know Perl much better than C, because I don't like segfaults, because string handling is so cumbersome in C, and because I'm just more productive in Perl.

I've done quite some small scientific computations for my studies, and I only needed C's speed very rarely.

As a side note there's currently a Google Summer of Code project on its way to bring GSL to perl. (GSL is the GNU Scientific Library).

BTW most of the time when you need raw speed it's best to use some well-optimized libraries, like the lapack or atlas libs. Even Matlab can be surprisingly fast when you formulate all your operations as vector operations instead of loops.

It very much depends on * what * exactly you want to do. I have done some number crunching in Perl (processing many GB's of data) and it worked fine for me. You would expect C to have better performance though.

Some matrix operations can be notoriously tricky, especially if the matrices get big and/or the numbers have high precision. A lot of stuff was developed in mathematics to tackle this.

So you typically want to use some C-library or Perl module tailored for this. You can take a look at Math::Matrix (Perl) or try your luck with some C math library. Check for example http://www.ibm.com/developerworks/opensource/library/l-matrix.html

(This is actually a great interview question.) Having spent about 15 years doing C before discovering awk and then Perl, I feel qualified to say that you should be able to do just about everything you want to in Perl, with the exception being code that needs to run significantly faster than Perl -- and at that point you may be even better off by writing it in assembler.

An alternative that lies somewhere in the middle is to use Inline::C which will (or should) give you the best of both worlds -- the blazing speed of C, with the convenience of Perl. Although I can't say I've used this module, as far as I know, no makefile is required. It should take you just an hour or two to confirm that it works as advertised, and if you have problems, c'mon back here.

I agree with what others are saying, Perl first and then apply C as needed.

One point I think others have missed is that speed may not be the only reason. Memory consumption may also be a good reason to go with a C implementation.

Most of the time this is not really an issue. But in some cases it can be a major factor.

Although C wins out the gate when it comes to memory usage, never forget to consider the flexibility that Perl provides. Sometimes you have so much data that even switching to C wouldn't solve your memory woes. Enter BerkeleyDB: by simply tying a hash or array to a file, you immediately drop memory usage to almost nothing while leaving your implementation otherwise completely intact.

Perl gives you the flexibility to try this, test if the performance is adequate, and have a completed, tested solution, all with the inclusion of a single line of code. Even in matters of memory, Perl can have a leg up on C.

There absolutely are ways to reduce memory usage in Perl programs. And in most cases they are sufficient.

However in cases where the volume of data to be handled is large (eg processing dna sequences) or the system is resource constrained (eg a small ARM linux based appliance) it makes sense to consider a C implementation.

Let's consider an example where I have confronted this issue. I've been working on software for an ARM system that has only 32 MB of RAM and no virtual memory. One can write a simple socket based server in C that has much smaller memory needs than the equivalent perl server. The downside, is that it is harder make the C server do anything useful. I wrote my server application in Perl first, and it fits nicely on the system. However, I know that I can get rid of a fair amount of overhead if I rewrite the server in C. I won't do that unless it becomes absolutely necessary. Perl is much easier to debug, extend and maintain.

In the case above, adding the BerkelyDB to the mix would just make things worse. In data intensive applications, a dbfile is often the answer. Other times simple things like using the ternary for loop instead of for (0..$#bigArray) can make a big difference.

C has it's own problems. Did I forget to free that chunk of memory? Oops--memory leak. Just because it is possible to write more efficient code in C does not mean that C code will always be more efficient. So, you've got to be selective with what you try to do in C.

My basic position is that premature optimization is a bad thing. Writing code in C instead of Perl is an acceptable, if costly (in terms of labor) speed and memory optimization. So, before applying that optimization, make damn sure there aren't any less expensive tricks you can try that will yeild acceptable results AND be sure that your C code will, in fact be smaller/faster than the Perl it replaces.


TGI says moo

Most of the time speed is not absolutely critical. Most of the time it is not more important than a) doing the job *right*, b) being maintainable, c) being a bit portable, and other such priorities.

I might use C out of memory concerns. Perl uses a *lot* of memory in its data types.

I might use C because I want a finer level of control. A lot of byte-by-byte or bit-by-bit work is very straightforward in C, while I wouldn't want to do so much packing in Perl. There are things about C that are syntactically very convenient, like its structs, its pointers, and doing bitwise operations in C.

Perl is a heavy layer of abstraction over C. Most of the time this is a very nice thing. But sometimes you want to work under that abstraction, and you simply would rather not have it the way. In that case, I choose C over Perl. C is very simple, and really puts very little in your way. The standard library is very simple and concise, and the language is standardized. Sometimes less is more.

I also tend to use C when I want to have a close connection with my operating system or with my processor, in the way that Dennis Ritchie referred to C as a "portable assembly language". C is just more convenient than Perl in these cases. Perl is nice at making things the same (and easy) across OSes or processors, not at being really particular to them when you want to be.

I guess I didn't really explain any of that very well. I just felt I could explain why I use C about as much as I use Perl, and how speed isn't usually the reason.

Be careful with Inline::C, if you have thousand of data Inline::C needs to convert the entire C structure to Perl structure, and this is slow.

I don't test yet, but i think that XS fix that problem :)

So, if Inline::C sometimes seems slow to you, try XS before anything.