Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re^5: Module uses loads of CPU.. or is it me

by redhotpenguin (Deacon)
on Dec 11, 2007 at 17:22 UTC ( [id://656436]=note: print w/replies, xml ) Need Help??


in reply to Re^4: Module uses loads of CPU.. or is it me
in thread Module uses loads of CPU.. or is it me

Well I think you should take a serious look at some of the lower level methods in Net::S3::Amazon and try to develop a parallelized application. It isn't likely that you will be able to increase the efficiency of the parser by double, but I think with a few hours hacking you could get a parallelized version of your program that you can make Nx speedups with.

  • Comment on Re^5: Module uses loads of CPU.. or is it me

Replies are listed 'Best First'.
Re^6: Module uses loads of CPU.. or is it me
by hsinclai (Deacon) on Dec 11, 2007 at 18:58 UTC
    Without hacking too deeply, some parallelization might be acheived by using the "marker" method already in the module itself, and having parallel fetches in batches. Looking further into the module I think I found an error or bug (maybe it's just a documentation bug) http://rt.cpan.org/Public/Bug/Display.html?id=31381.

    Going much further is going to be a question of (my) available time right now -- but I really do appreciate the push!

    -Harold

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://656436]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (2)
As of 2024-04-25 02:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found