What's going on may depend on the OS and filesystem, but creating an 81G file by printing out line by line is going to be a killer. You're going to be constantly extending it piece by piece, probably ending up with a badly fragmented file. And even with the disk likely being a RAID volume of some sort (As it's got to be at least a 170G volume there) you're going to find the reads and writes competing for the disk heads a lot.
This is as much, or more, a system administration issue as a perl one. Find your sysadmin, get him/her to fill you in on the characteristics of the system, and work with that. You may find that pre-extending the file is a good thing. There may be system parameters that can be set that govern the size of the new chunk your OS allocates when it extends a file. There may be cache settings that can be tweaked on the disk controller or individual files. This is obviously an adequately beefy machine, so you just need to do some performance tuning, and probably at the OS level.
Hopefully you're not accessing the file via NFS or some other networked file system. Those will tend to kill your performance dead.
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|
