Re: Perl vs Java in Heavyweight Filesystem Processing
by chromatic (Archbishop) on May 22, 2003 at 22:32 UTC
|
What would happen if you were to write a mockup in Perl and let people play with it for a few days? It'd likely be faster to develop than a mockup in Java. Having something concrete and working would give everyone a feel for how the technology works.
| [reply] [Watch: Dir/Any] |
|
Yes, and if the mockup is reasonably complete, the bosses may ask how long it will take to finish. This has happened to me more than once and my 'proof of concept' CGI scripts are still running in production :)
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by perrin (Chancellor) on May 22, 2003 at 22:15 UTC
|
There are some benchmarks of Perl and Java for reading files on this site. The main things they will tell you is that Perl is plenty fast, and it takes tons more code to do a decent job of it in Java. You already said that the performance of the application is irrelevant in this case, so it comes down to the cost of development and maintenance. If you can build and maintain this app twice as fast in Perl as in Java, you will save your company money. It's that simple. | [reply] [Watch: Dir/Any] |
|
You already said that the performance of the application is irrelevant in this case, so it comes down to the cost of development and maintenance.
From the sound of it, it seems that while the OP realizes this, his PHBs may care deeply about performance. Perl is at a natural disadvantage in this case, since it sounds like their running a persistant environment for Java, but not for Perl.
---- I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer
Note: All code is untested, unless otherwise stated
| [reply] [Watch: Dir/Any] |
|
Actually, I suspect Perl will mop the floor with Java in this case, unless someone goes to great lengths to use the new direct io classes in the latest Java release. This is not a case where a small script is being launched over and over (which would favor a persistent environment over CGI), but rather a very heavy script that is run infrequently. In that situation, the actual file-processing speed is much more important than the startup time.
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by Abigail-II (Bishop) on May 22, 2003 at 22:15 UTC
|
Of what you are posting, I only find one thing that might be
relevant in your quest. You say that Java is the wrong tool
for the job. Whether that is true or not, I cannot judge from
your posting, but that, IMO, is the only thing you have you
can use as a pivot point.
What you have against you is that your platform (iPlanet)
isn't geared towards Perl, but to Java, that you don't have
benchmarks (which would only be useful if the Java solution
was slow anyway), and that you seem to be the only person in
your team wanting (or perhaps able) to program in Perl.
If I were a manager, I'd weight the latter very heavily.
Abigail | [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by submersible_toaster (Chaplain) on May 23, 2003 at 03:25 UTC
|
While I don't have quite the same case to relate, let me tell you about
a daemon I wrote once. I worked with a company that used a java , web-faced system to track big
lumps of data around the building (with versioning and various bells). This system become interactive enough
that users could create a work order to copy chunk-of-data-BLAH from someplace to someOTHERplace.
I was tasked to write the poor little slave process to do the work orders. My initial try was with
perl , it was quick, smart and IMO very cool. But it was deemed that rather than a CGI perl service, this
was to be done using Tomcat instead. Not knowing java , I ported the worker to python, then jythonized
it to run with Tomcat.
The developer descends on me, howling about the slowness of this new service. Files were taking
10 times longer to transfer , no network or disk IO bottlenecks could be found. I went straight for the
.py file , changed some shutil.copyfile()'s with actual system calls to cp and magically everything
went a LOT faster , but still not as quick as perl.
I doubt this gives you much ammunition, but it's a Java vs perl story, where java won (but sucked) and perl lost (but got the job done). I am suspicious of other
things pushing the bias toward java , certainly it was seen as more acceptable to use a python/java hack than a clever perl CGI. Why? I just dont know.
I can't believe it's not psellchecked
| [reply] [Watch: Dir/Any] [d/l] [select] |
|
This is completely irrelevant as you didn't program anything
in Java, you ported a perl script to a python script, and run the python script via a Python interpreter implemented in Java. That harldy qualifies a Java "program".
| [reply] [Watch: Dir/Any] |
|
Correct - I programmed nothing in java, but ultimately the python
ran in a python interpreter, implemented in java. So what java classes does
the j-python interpreter call for file IO?
For those unfamiliar with Java's java.io.File package, consider yourselves lucky. In
comparison to Perl's File modules, Java's API is clumsy, slow, and incomplete. The text
processing piece is a no brainer; clearly Perl has the upper hand.
Irrelevant ? I can't really say, I don't program in java.
hunting /. grammar nazis since when?
| [reply] [Watch: Dir/Any] |
|
When you were hired, did they require Java skills? I'd have fired you for jythonization
| [reply] [Watch: Dir/Any] |
|
- I was never hired as a programmer
- My employ pre-dated the use of java for internal services
- I programmed in perl to avoid sysadmin fear and loathing
- Tomcat was not my idea.
- The perl solution solved the problem.
- The perl solution did not create a problem
- You'd fire me for jythonising? Like a care
I can't believe it's not psellchecked
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by adamk (Chaplain) on May 23, 2003 at 15:33 UTC
|
I've worked in a company where we had to do time budgets for jobs. Our general rule of thumb was writing typical code, from scratch, in perl was about 2.5 times more efficient. That was normal code, and basic HTML fluff.
For every major API you have to interact with, CPAN vs Java stuff, add another factor of 2.
So for the typical example of working with 1. A database API, and 2. Filesystem search/modify/replace, you get 2.5 x 2 x 2 == about 10 times faster. A Java guru I work with agrees with these figures. I've written automation systems in about a month, and then waited 18 months for an ( admittedly better ) Java version to replace it.
With respect to CGI, remember the process creation/teardown is only relavent above about 1-2 hits per second per CPU. Below that rate, it hardly matters.
Also, CGI applications have zero resource usage when not in use. I'm not a Java person, but from memory most of the speed from Java comes from having a permanently running VM, kinda like mod_perl. I used to work for Cisco, and on their big main worldwide E10000 intranet, they had thousands of perl scripts. They could do this because each script wasn't using up permanent resources.
Cisco later switched to Java corporation wide, with "ALL NEW APPLICATIONS IN JAVA!". They filled up a dedicated E10000 in less than 2 months just from NEW apps, because they could no longer add all the VMs required by all these apps.
Hope some of this helps | [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by hardburn (Abbot) on May 23, 2003 at 14:05 UTC
|
Present a challenge to your superiors. Two teams, one writing the Java version, the other writing the Perl version. If the people on the Perl team are decent with the langague, they'll have the Perl version done while the Java people are still hagaling over the design.
The Perl team could probably be as small as one person. The success of Perl in this regard will depend heavily on that person's familiartiy with Perl and CPAN (which is true for any Perl project).
Your description of the actual program was a bit vague, but it sounds like something that I could whip up in an afternoon, including debugging. There might be details I'm not aware of, though.
Your superiors will no doubt think you're setting yourself up for disaster by having a team of only yourself against the company's Best and Brightest Java programmers. Again, the details of the application matter a lot here, but if you have more than passing familiarity with Perl and CPAN, you'll likely have your project done in at least 1/10th the time of the Java team.
---- I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer
Note: All code is untested, unless otherwise stated
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by higle (Chaplain) on May 23, 2003 at 18:12 UTC
|
Thank you, everyone, for your comments, suggestions, and/or support.
I've been mulling this over for quite some time; this isn't the first time that I've gotten some flak for using or wanting to use a "scripting" language in an enterprise application. I suppose that my problem really consists of two things:
- My coworkers are more concerned with the "latest and greatest industry standard" rather than leveraging the effectiveness of an existing tool or platform. To them Perl is old news, only good for writing throwaway scripts. Java is Big Business, and we've invested (wasted?) Big Money in our existing platform, which changes every year, continues to bloat, and eats our budget like candy corn.
- I've never stood my ground, pointed out the wasteful spending, or generally argued this point to anyone in our company who's in a place to do something about it until now.
What I'm actually fighting is a mindset. It's not that they don't know how or can't learn to code in Perl; they're just the kind of people who'll buy the first thing the vendor drops on the table and insist we use it, for better or worse.
I've decided to press the issue further, perhaps using subversion ("Oh, uh, I've already written a prototype in Perl... is that OK?"), or perhaps going the business route and making a dollars & cents case, or maybe both.
It matters to me on a somewhat personal level, not because I'm a proud Perl zealot or Java basher, but because I've recently agreed to move for the company and I've signed a contract obligating me for the next year. I personally don't want to live with and maintain crappy, troublesome software, de facto, for the next year.
Is this pursuit dangerous? Maybe. Is it worth it? Maybe.
Only those who will risk going too far can possibly find out how far one can go. -- T.S. Elliot
higle
Can I crash at your place if this backfires? -- Higle | [reply] [Watch: Dir/Any] |
|
Just arrived at this node via "Selected Best Nodes" and I wonder ... how did this end? If you are still around I think it would be interesting to hear your further adventures in Javaungle.
Jenda
Always code as if the guy who ends up maintaining your code
will be a violent psychopath who knows where you live.
-- Rick Osborne
Edit by castaway: Closed small tag in signature
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by defyance (Curate) on May 23, 2003 at 16:14 UTC
|
hig, As a fellow employee, I feel compelled to say something here.
In recent years our company have always been open for ideas and suggestions. We were a smaller company, and trying to find our niche. Hell, they even let me write a few scripts and things for them, remember? Now that we have gone "metro", things have really changed, I'm sure you've noticed. I'd have to say that due to the recent acquisitions and our growth spurt, uniformity is key for organization. Basically, it looks like they found a niche in Java. I'd say that right now it would be in your best interest to lay low until things get settled. This is just a suggestion, but I'm sure you've noticed tensions have been high in our building recently. I'm not sure if your going to be doing this for the PHB in Corporate, but I'm sure that things are changing up there as well..
So, in short, I'd just lay low.
I could be really, really wrong here, but in nearly 5 years with our company, I've tried to pick up on how things work around here...
Peace, Love, and Chicken Grease.
--
A conclusion is simply the place where someone got tired of thinking. | [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by linux454 (Pilgrim) on May 23, 2003 at 18:07 UTC
|
I've been programming in perl for quite some time, and I've picked Java up in the past several years.
And I've found that I love both languages equally, but for different things. Perl can do any job that Java can do, somethings it does better and somethings it doesn't and vice versa. The way I see your situation is that perl may be the more efficient tool, but that does not necessarily mean that it is the best tool for the job. Meaning that if you are the only developer who knows perl, in an all java environment, then what happens to that little sliver of perl in the sea of java if you kick the bucket, are fired, or when you move on to bigger and better things? Well it will either have to be replaced or someone will have to learn perl. Both are in my opinion unacceptable from a business perspective. On the other side of things, it's sounds like you aren't going to have to process file contents just file meta-data (filename, size, etc.). If that is indeed the case then there should really be no argument java is not going to be that much slower (if any) when performing those types of operations. If however, you will have to operate on a large amount of file contents (i.e. parse text from inside the file etc.) then I would hate to use java, perl is the king of file slurping. But if speed is not of paramount concern, I say just suck it up, take one for the team, and right the thing in Java. Because java really is a great language if you give it a chance. ((This whole post is written by a converted perl bigot)) | [reply] [Watch: Dir/Any] |
|
If Perl allows the employees to finish their tasks sooner and better, then it's good in the business perspective if a few more are forced to learn it.
Jenda
Always code as if the guy who ends up maintaining your code
will be a violent psychopath who knows where you live.
-- Rick Osborne
Edit by castaway: Closed small tag in signature
| [reply] [Watch: Dir/Any] |
Re: Perl vs Java in Heavyweight Filesystem Processing
by petesmiley (Friar) on May 23, 2003 at 16:49 UTC
|
The thing that truly revolts me about this question (++) is that you could probably write the damn thing in C (how's that for speed) almost as fast as you could write it in Jyva. The benefits of using Jyva probably wouldn't come in to play much with a project like this, especially if you are on this project by yourself. The whole world seems to think that you have to kill knats with a sledge hammer that's painful to lift.
I vote for the "write a test version in perl". It seems like you could write something in no time flat. If you want perl to work for you, you have to work for perl. Work late one day if you can't get it done during your regular work schedule. Then tell them how long it took. If you're worried about there being repercussions, just say it was a working model for the java version ;) Remember, it's better to ask for forgiveness than to ask for permission.
Or if you think you're going down the toilet anyway, just write it in perl, hand it to them and say nothing. See how long it takes them to notice the difference ;) (just kidding)
smiles
(thankfully I no longer program in C for most things)
PS I have never worried about there being other perl programmers around to maintain my work. Currently I'm the only perl programmer where I work. Like most work places, mine likes to make me feel like I'm easily replaceable. So I act like I am ;) Oddly enough, most of my programs have never had problems, and usually they are retired before they ever do (the oldest running program I have is 3 years old). All because I have the time to test my programs because I'm working in perl. | [reply] [Watch: Dir/Any] |
|
Currently I'm the only perl programmer where I work.
/me tries not to take offence to that comment.
:)
-silent11
| [reply] [Watch: Dir/Any] |
|
Hehe, let me rephrase. I'm the only perl programmer that's been hired to program. There are other slackers where I work that are perl programmers.
smiles
| [reply] [Watch: Dir/Any] |
A reply falls below the community's threshold of quality. You may see it by logging in. |