Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Does perl read the entire pl file into memory?

by harangzsolt33 (Chaplain)
on May 21, 2019 at 15:21 UTC ( [id://11100320]=perlquestion: print w/replies, xml ) Need Help??

harangzsolt33 has asked for the wisdom of the Perl Monks concerning the following question:

I have thought about putting a large comment section into my perl script after the __END__ mark, and I am wondering if this is okay. How does perl parse the files? Does it read the entire script into memory FIRST even if it's over 1MB, or does it read pl files one buffer at a time and when it encounters the __END__ then it stops reading? If it reads the entire script into memory, then I guess it would be wiser for me to put the comments into a separate text file..

Replies are listed 'Best First'.
Re: Does perl read the entire pl file into memory?
by haukex (Archbishop) on May 21, 2019 at 15:27 UTC

    AFAIK Perl treats __END__ like an EOF, i.e. it will stop reading the file at that point, so the amount of data after an __END__ shouldn't make a difference. Everything before that does have to be read before it is parsed and executed, of course. If you have large amounts of data, it's probably better to put that in an external file, though. It is fairly common to put large POD documents after an __END__, though.

Re: Does perl read the entire pl file into memory?
by davido (Cardinal) on May 21, 2019 at 16:55 UTC

    In perldata it mentions that __END__ behaves just like __DATA__ in top level scripts. So it's pretty easy to put an __END__ in a script with a bunch of junk after it, and then print "Byte offset: ", tell(DATA), "\n"; in the script prior to the __END__ tag. If you do that, you'll see that the DATA filehandle is sitting at the offset to the first line after the __END__ tag.

    Now whether or not Perl is doing trickery behind the scenes that we don't know about, I can't say. It's possible that it's keeping track of where the __END__ was seen but continuing to read through to the end of the file anyway, and then seeking back to where __END__ was found. But I can't think of any reason why it would go to all that trouble. That's probably not happening; the simplest explanation, that it isn't reading the entire file, is probably correct.

    Here's an example:

    #!/usr/bin/env perl my $string = do { open my $fh, '<', $0 or die "Cannot open $0: $!\n"; local $/ = undef; <$fh>; }; if ($string =~ m/^__END__\n/gm) { print '__END__ tag found at ', pos($string), "\n"; } print 'Entire file length: ', length($string), "\n"; END { print 'DATA filehandle tell offset: ', tell(DATA), "\n"; } __END__ Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country. Now is the time for all good men to come to the aid of their country.

    This produces the following output:

    __END__ tag found at 351 Entire file length: 1892 DATA filehandle tell offset: 351

    That seems to indicate that Perl stops reading after it has finished reading the line that contains the __END__.


    Dave

Re: Does perl read the entire pl file into memory?
by Your Mother (Archbishop) on May 21, 2019 at 15:38 UTC

    You already got good comments, just want to mention it's a semi-common practice to have a .pod file companion where the documentation goes. Usually this is for modules, though, .pm files. So MyStufs.pm would have the code and MyStufs.pod, or even something like MyStufs/Manual.pod, the documentation.

Re: Does perl read the entire pl file into memory?
by hippo (Bishop) on May 21, 2019 at 15:30 UTC
    Does it read the entire script into memory FIRST even if it's over 1MB

    No idea but to me that's not the over-riding factor. If your comment section is 1MB and the mean line length is 60 bytes then that's coming in at over 17k lines of comment block. I'd say that's too large. Pity the poor monk who has to do a search+replace in all of that only to find that most of it is comments. Move it out into a separate file and you'll keep everyone happy: both the POD editors and the code editors.

Re: Does perl read the entire pl file into memory?
by LanX (Saint) on May 21, 2019 at 16:30 UTC
    > when it encounters the __END__ then it stops reading?

    Right this happens at compile time and DATA° is exactly that filehandle which points there.

    That's why you can continue reading from <DATA> or use seek to set DATA back and read the source again. ( better not at compile time though ;)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

    °) there is a minor difference between DATA and END which doesn't matter in this case.

    See perldata#Special-Literals

      Thank you for all your replies! You were all very helpful and great point. I'll probably keep the comments section at the end separate from my perl script. ;)

        Let me add: please do not use literal comments. Use POD (and there are several other tutorials around). It's easy, it's relatively powerful, it reads fine raw, and various formatters know how to display it nicely.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11100320]
Approved by haukex
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2024-03-28 21:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found