Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

You mention that your program is 4-5 seconds per minute slower. Depending on the total number of minutes, that may not be a lot of time in the long view. Compare the amount of time you will save over the lifetime of the programs use to the amount of time it will take to find the true source of the slow down and optimize it away. Unless you are running this program over and over, need real-time responsiveness or you have a marketing reason to look fast and slick, you may want to ask whether optimization is even necessary.

If you still feel you need to optimize, you might be able to confirm your suspicions about the cause by running your script with the -d:DProf option. This will generate a file named tmon.out in the current directory. This file contains raw profiling data. You can analyzed the contents of the file, by typing the command dprofpp, which should be part of your Perl installation. This will give you an idea of how much time is taken by each subroutine. Thus your shell would look something like this:

$ perl -d:DProf myscript.pl $ dprofpp

If you are still convinced that buffering is the source of your problem, you may want to consider using the "sys" family of functions. These are low level functions that work directly with the file and will allow you to define your own buffer size and buffering strategy:

  • sysopen to open the file
  • syswrite to dump out your buffer
  • sysread to read
  • sysseek to move the cursor to some place other than the end of the previous read or write.

If you go this route, do not use the normal Perl file io (open, read, print, seek tell) on the same file handle or your file handle will give you confusing and not terribly helpful results.

Before you make any changes, you should restructure your program so that the old file buffering code is encapsulated in a subroutine. The new code should also be in a subroutine. That way you can swap out the old and new code at will to compare them. Encapsulating the two approaches (Perl buffering vs. custom buffering) will also let you use a wonderful optimization tool, a core module built into Perl: Benchmark. This module lets you compare two subroutines to see which is faster. If you are going to put in the effort to optimize, it pays to make sure that you are actually making an improvement.

Finally, you may want to take a look at this Unix Review article, which goes over some of the basic tools for optimizing programs: Speeding up your Perl programs.


In reply to Re: temp hold logfiles in memory? by ELISHEVA
in thread temp hold logfiles in memory? by TRoderic

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (3)
As of 2021-10-26 06:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (90 votes). Check out past polls.

    Notices?