http://qs321.pair.com?node_id=653300

TStanley has asked for the wisdom of the Perl Monks concerning the following question:

As part of my company's PCI requirements, I have been assigned a task to parse a log file that is updated continually through the day. I figure that I can key off the date entry to get just the ones for the previous day, so that I don't duplicate anything, but my big issue is with extracting a specific piece of data. A small sample of the log is below:
2007-11-16 16:04:33 Local1.Alert 128.29.29.40 id=firewall tim +e="2007-11-16 16:04:08" fw=WS2000-Store 29 pri=1 proto=6(tcp) src=128 +.29.29.200 dst=128.29.100.102 mid= 1013 mtp= 2 msg=TCP connection re +quest received is invalid, dropping packet Src 23 Dst 4412 from EXT n +/w agent=Firewall 2007-11-16 16:05:05 Local1.Alert 128.24.24.40 id=firewall tim +e="2007-11-16 16:03:25" fw=WS2000-Store 24 pri=1 proto=6(tcp) src=128 +.24.24.200 dst=128.24.100.101 mid= 1013 mtp= 2 msg=TCP connection re +quest received is invalid, dropping packet Src 23 Dst 4344 from EXT n +/w agent=Firewall 2007-11-16 16:05:34 Local1.Alert 128.29.29.40 id=firewall tim +e="2007-11-16 16:05:09" fw=WS2000-Store 29 pri=1 proto=6(tcp) src=128 +.29.29.200 dst=128.29.100.102 mid= 1013 mtp= 2 msg=TCP connection re +quest received is invalid, dropping packet Src 23 Dst 4412 from EXT n +/w agent=Firewall 2007-11-16 16:05:39 Local1.Alert 128.2.2.40 id=firewall time= +"2007-11-16 16:03:36" fw=WS2000-Store 02 pri=1 proto=6(tcp) src=128.2 +.2.200 dst=128.2.100.106 mid= 1013 mtp= 2 msg=TCP connection request + received is invalid, dropping packet Src 23 Dst 4631 from EXT n/w ag +ent=Firewall 2007-11-16 16:05:40 Local1.Alert 128.2.2.40 id=firewall time= +"2007-11-16 16:03:36" fw=WS2000-Store 02 pri=1 proto=6(tcp) src=128.2 +.2.200 dst=128.2.100.106 mid= 1013 mtp= 2 msg=TCP connection request + received is invalid, dropping packet Src 23 Dst 4631 from EXT n/w ag +ent=Firewall 2007-11-16 16:05:40 Local1.Alert 128.2.2.40 id=firewall time= +"2007-11-16 16:03:37" fw=WS2000-Store 02 pri=1 proto=6(tcp) src=128.2 +.2.200 dst=128.2.100.106 mid= 1013 mtp= 2 msg=TCP connection request + received is invalid, dropping packet Src 23 Dst 4631 from EXT n/w ag +ent=Firewall
I need to extract the data starting with "msg=" and ending just before "Src". The code I am currently using to put the above data into a csv file that I later import to an Excel spreadsheet is below
#!perl use strict; open INPUT,"<","input_file.txt"||die "Can not open input_file: $!\n"; open CSV,">","OUTPUT.csv"||die "Can not open OUTPUT.csv: $!\n"; print CSV "Date,Time,WS 2000,FW Date,FW Time,Store,Src IP,Src Port,Dst + IP,Dst Port,Type,Agent\n"; while(<INPUT>){ my @line = split /\s+/; my $Date = $line[0]; my $Time = $line[1]; my $ws2k = $line[3]; my $FW_Date = $line[5]; my $FW_Time = $line[6]; my $store = $line[8]; my $src_ip = $line[11]; my $dst_ip = $line[12]; my $src_prt = $line[$#line - 6]; my $dst_prt = $line[$#line - 4]; my $type = $line[$#line - 2]; my $agent = $line[$#line]; chomp $agent; $agent=~s/agent=//; $FW_Date=~s/time="//; $FW_Time=~s/"//; $src_ip=~s/src=//; $dst_ip=~s/dst=//; print CSV "$Date,$Time,$ws2k,$FW_Date,$FW_Time,$store,$src_ip,$src_p +rt,$dst_ip,$dst_prt,$type,$agent\n"; } close INPUT; close CSV;
I am guessing that I need to do something along the lines of a regular expression before I actually split the line out into the array. Would something like the following even be close to what I am looking for, or am I heading in the wrong direction with this?
while(<INPUT>){ $_=~m/msg=(.*) Src/; my $msg=$1; my @line = split /\s+/; ## rest of code
As always, thanks for any suggestions.

TStanley
--------
People sleep peaceably in their beds at night only because rough men stand ready to do violence on their behalf. -- George Orwell