Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Re: Re: Re: Confirming what we already knew

by AssFace (Pilgrim)
on Mar 05, 2003 at 14:15 UTC ( [id://240581]=note: print w/replies, xml ) Need Help??


in reply to Re: Re: Re: Confirming what we already knew
in thread Confirming what we already knew

That is a great point. I suppose I should clarify that statement by saying that my idea of "nearly exactly" in regards to translating to C was that it is much easier than say to forth - the math logic is the same and the functions are setup in a very similar manner.

In terms of length difference, I agree when dealing with strings, C ends up taking multiple lines to do what Perl can do in one line. Aside from that, the line count is IMO largely due to comments and I'm not really sure how many lines are just comments and haven't bothered to really determine that exactly. Were I to strip out all comment blocks from the C and Perl code - I would imagine the Perl to be shorter - I likely just estimated how many they have of each to be wrong.

Whereas if you can pare your code down so that the work gets done by map, foreach, tr, regexes etc then the execution time should drop.

I'm certainly interested in speeding it up just out of curiosity (since I already have a C version now that takes 1.4sec, which I don't foree the Perl version ever doing - but I am curious where I made decisions that slowed the Perl down).
I converted every for loop to a foreach loop instead - that slowed the code down by 6 seconds. I assume I didn't get the optimal use of it because I still need to be aware of where I am in the array and not just what $_ is, so I kept a counter along with it - which I would assume would be pretty much just like a for loop in that case.

Perhaps I'm just being dense, but the only place I see where map would really apply to my stuff is when I'm originally manipulating my arrays - and I've already benchmarked that to take less than a second, and it is only done once - so that isn't the speed concern there.

I'm only doing math inside the loops, so the tr is useless to me in that case - again, the only text manipulation I do is at the start and at the end of the program - outside of any of the loops. Other than that, it is just math.

Again, I don't see how a regex will help me when I just want to iterate N days of values and add them together, and then divide by some number.


Were I parsing text and seeing this huge a time difference, I would be very suspicious of it all - but this is nearly all math inside the loops (3000+ times it iterates through the outer loop, and inside of that is a large 1000+ loop iteration with a few smaller 5-20 iterations that depenind on what algorithm is used is done on each 1000 as well).

I understand that I'm being pretty useless in not putting up the actual code so that people can point out "you did this and THIS would be way faster" - but I think in the end I mainly wanted to show another example with some numbers of what speed differences for various applications exist. When I was orginally going to port it over to C, I looked around for numbers of what sort of difference I might see, and very few people talk about doing it.
So now there is at least one more node about it :)

Is there a resource out there that lets me know that "doing XYZ is more efficient in Perl than doing ABC"? I'm curious which things I'm doing that might be sped up, but the bulk of what is inside the loops is if/else statements, and just basic math ((a+b)/c)
  • Comment on Re: Re: Re: Re: Confirming what we already knew

Replies are listed 'Best First'.
Re: Re: Re: Re: Re: Confirming what we already knew
by l2kashe (Deacon) on Mar 05, 2003 at 14:50 UTC
    You might be able to achieve a huge performance gain by taking your algorithms and memoizing them. I.e
    my %multiply = (); $str = "$num1 $num2"; if (!$multiply{$str}) { $result = $num1 * $num2; $multiply{$str) = "$result"; } else { $result = "$multiply{$str}"; } # use result to do something else below
    You could even turn this into a sub function, which takes a string as an argument, shifts it as it goes, determines what to do with the different expressions on the line, and then returns the final result. Thinking about that though.. You would need to add logic to deal with precedence, and parentheses. Also it would be very expensive to start, but once you are through a fair portion of your data, it will really help

    This is probably a very poor example, and could be cleaned/optimized, but you see the point I hope. Floating point math is always leaps and bounds slower then interger math. On top of that you moved from a windows machine to a FreeBSD machine. The kernel architecture, memory management, threading mechanisms, as well as application overhead all play massive roles in how your code will perform, especially in regards to loops such as you are talking about.

    Also as long as the data sets do not rely on each other, you could take all your data, parse it, store it in a hash or array, then take slices of those structures and fork off seperate processes which can do the computations in parallel, and then collect the results from files, or even share memory space, and have them all report back when they are done if you have ithreads enabled, and know how to use them.

    I really am interested to see your code, much like other monks. I have started a few threads about speed, and squeezing the last bit I can out of perl. I've received great help and optimizations, which have also allowed me to peer a little deeper into how perl works and what it likes to see. It really does worry me, just like the other monks have stated, that your perl code looks like your C code. You could be missing out on some really great stuff there.

    O well just my 2 cents..

    /* And the Creator, against his better judgement, wrote man.c */
      in regards to what system it is running on - I have run the Perl code on a win2K P4 2G box, a WinXP Home Athlon Mobile 1G box, a dual PIII 667 RH8 Linux box (didn't do anything to take into account the dual processors, so it just ran effectively on one), a PII 300 RH Linux box, and they all seem to scale based on the processor speed - although the Athlon Mobile 1G seemed to be slightly faster than what its rating would make one think - but that is what AMD is known for (the P4 2G runs it in around 195 seconds and the Athlon Mobile 1G runs it in 300 seconds).

      Things are kind of busy at work right now - if I can get a chance - I will try to strip out parts of my code that I am wary of (what the data looks like and the comments) and then I will try to post it here later - maybe tonight if I can get to it.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://240581]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2024-04-19 04:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found