Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
I have a question about the most efficient (in both memory and processing) way to read Apache log files. I realize that there are many many packages on CPAN that can read log files from Apache servers, but I am curious about what may be the best way to parse these files.

I am a beginner PERL programmer who is curious about the inner workings of both these packages and PERL itself.

I was thinking that it may be possible to use pack and unpack alongside File::Map somehow to be able to parse and process the log lines directly without loading the entire file into memory.

Is my line of thinking flawed? Is it possible to use pack and unpack in this manner with variable width files? I have read perlpacktut, but it only explains the capabilities of text files in a fixed width format.

Likewise, for the All-In-One packages for log parsing on CPAN, how do they parse these kinds of files and why?


In reply to Question about the most efficient way to read Apache log files without All-In-One Modules from CPAN (personal learning exercise) by lulz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others browsing the Monastery: (2)
    As of 2020-08-13 00:25 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?
      Which rocket would you take to Mars?










      Results (68 votes). Check out past polls.

      Notices?