Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: Re: split and sysread()

by relaxed137 (Acolyte)
on Apr 18, 2003 at 23:58 UTC ( [id://251585]=note: print w/replies, xml ) Need Help??


in reply to Re: split and sysread()
in thread split and sysread()

They are terminated by new lines. The issue with the above is it still reads in the file one line at a time, right?. That's what I am trying to get away from.

Replies are listed 'Best First'.
Re: Re: Re: split and sysread()
by Limbic~Region (Chancellor) on Apr 19, 2003 at 00:19 UTC
    The issue with the above is it still reads in the file one line at a time, right?.

    Wrong! Perl reads from the file in a buffer and only goes back out to disk when the buffer is empty. What <FILEHANDLE> does is read from the buffer up to the newline. I think you are falling prey to premature optimization.

    while (<INPUT>) { my @fields = split /\|/ , $_; #Do stuff with particular field }

    Cheers - L~R

    Update: See this node by chromatic as he was setting me straight on the very same matter

      Hmmm... That's what I had before I started to use sysread... I've noticed that sysread will read a file much faster though... Maybe I can make the read buffer bigger instead to tune "while <input>" ?
      open (INFILE, "/sentry/ecisock/ecisock.rpt"); while (<INFILE>) { chomp; ($serverhostname, $execdate, $cicstranid, $sentryapi, $elapsedtime, $errorcode, $corphdr, $channelcode, $clientprodcode, $clientipaddr, $sentrytermid, $signonid, $accountnum, $branch, $campaign, $segment, $custid, $localsysid, $remotesysid, $sentrytarget, $hfb, $sentryver, $corphdrfailreason, $corphdrfailtype, $corphdrbussvc, $corphdrclientcode, $mfcicstasknum, $sentrycicstasknum, $foo1, $foo2, $cpsm)=split(/\|/,$_); ########################################### # Counts total calls ########################################### $totalsentrycalls++; (.. Etcetera ..)
        Just a note: rather than split into a zillion variables, IMO it's more maintainable to list your field names in an array, then split into a hash variable:
        my @fields = qw( serverhostname execdate etc. ); while (<INFILE>) { chomp; my %stuff; @stuff{@fields} = split /\|/; # Easy iteration over fields (e.g. for debugging) print "[$_][$stuff{$_}]\n" for @fields; }
Re: Re: Re: split and sysread()
by dws (Chancellor) on Apr 18, 2003 at 23:59 UTC
    The issue with the above is it still reads in the file one line at a time, right?. That's what I am trying to get away from.

    But in order to pick out the right fields, you have to know where a line starts and stops. There's no getting around that.

    You can either use sysread() to pull stuff in by blocks, then juggle blocks to handle lines that span blocks, or you can uses Perl's line-by-line IO. Your call.

Re: Re: Re: split and sysread()
by pfaut (Priest) on Apr 19, 2003 at 00:12 UTC

    Well, what you had would read the next 16K of data from the file and append it to $buffer each time through the loop. Was your intent to process the file 16K at a time? In that case, you would still have to remove the fourth argument to sysread as it was causing new data to be written to the end of $buffer.

    Perl will allow you to read line-at-a-time or to slurp all of the file into an array with each element containing one line from the file. If you use sysread, breaking the data into lines is up to you. You would first have to break the data into lines by splitting on '\n' and then split the record into fields by splitting on '|'. It would be highly likely that your 16K read won't end at a record boundary so you would have to add code to merge the remaining data from one read with the beginning of the next. I'm not sure whatever you came up with would be more efficient than perl's line mode buffering.

    90% of every Perl application is already written.
    dragonchild

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://251585]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others contemplating the Monastery: (3)
As of 2024-04-25 06:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found