Re: Parsing Data from a PHP Socket Server

I once had to try something similar getting v3 data in a stream from a phone system, the following may help you organize the stream of values coming from the socket server, you can break out the stream into a hash of fieldnames->values using unpack() as in the following code.. to get your final values it looks like you have to do more, like reverse the string or do some more math on it, but at least you can organize the stream into a hash to work with it?

#!/usr/bin/perl -w

use strict;

my %fields;
my @field_names       =   qw/FIELD1 FIELD2 FIELD3 FIELD4 FIELD5 FIELD6
+/;

# $pack_definition derived from the format in the Packet Layout Guide,
+ e.g.
# ignore T18
# FF15 1B 8 A028 3043 D2070000
my $pack_definition   =   'a4 a2 a1 a4 a4 a8';

while ( <DATA> )
{
   @fields{@field_names} =  unpack($pack_definition,$_);
}

#this is just to display what we got, in hex
while (my($k,$v) =  each %fields)
{
   print sprintf("% 9s",$k) . '   =>   ' .  hex($v) . $/;
}

__DATA__
FF151B8A0283043D2070000
[download]

Note how the FIELD2=27 as it says in packet layout guide. Maybe you already found a better way but if not, HTH

-Harold

Comment on Re: Parsing Data from a PHP Socket Server Download Code

Replies are listed 'Best First'.
Re^2: Parsing Data from a PHP Socket Server by lmrd_1 (Initiate) on Sep 26, 2008 at 12:57 UTC
Harold Thanks for that. I'll definitely be giving that a try shortly. Right now I have it working with the static file with this: `#!/usr/bin/perl use warnings; use strict; &readfile; sub readfile{ #open the file my $source = 'test.txt'; my $dest = 'temp1.txt'; chomp $source; open SOURCE, '<', $source or die "Cannot open '$source' $!"; open DEST, '>', $dest or die "Cannot open '$dest' $!"; while (<SOURCE>){ s/(T18)//g; print DEST; } close SOURCE; close DEST; } __DATA__ FF151B8A0283043DB070000 FF161B83264643AAD4370FF FF171B8581BE0010000FFFF __END__` [download] I was originally doing it all with regex, but found this to be much easier:-)	[reply] [d/l]
Re^3: Parsing Data from a PHP Socket Server by lmrd_1 (Initiate) on May 11, 2009 at 19:23 UTC
first of all....sorry to dig up an old thread:-) Now on to my question. This project got put on the back burner for a few months and now it's back in play. What I'm wondering is how can I make this program run directly on the stream instead of first writing to a tmp file running the script then writing out a csv file for insertion into a database? Here's what I have so far ( I wind up with two tmp files after this is run): #vim set sw=2 ts=2 #!/usr/bin/perl use warnings; use strict; use Data::Dumper; &convert; #open the source file, strip out any unneeded chars, reformat the data sub convert{ my $source = "$ARGV[0]"; my $dest = "$ARGV[0]"."_tmp.txt"; chomp $source; open SOURCE, '<', $source or die "Cannot open '$source' $!"; open DEST, '>', $dest or die "Could not open '$dest' $!"; # move the data from the source file into an array my %fields; my @field_names = qw/FIELD1 FIELD2 FIELD3 FIELD4 FIELD5 FIELD6/; my $pack_definition = 'a4 a2 a1 a4 a4 a8A'; while(<SOURCE>){ # strip out the packet header s#T18##g; s#T00##g; s/^(FF14).$//g; s/^(FF18).$//g; s/(^\|\n)[\n\s]/$1/g; # arrange the data into the necessary order @fields{@field_names} = unpack($pack_definition, $_); s#(FF15)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2 +})#$1\t$2\t$3\t$5$4\t$7$6\t$11$10$9$8#g; s#(FF16)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2})(.{2 +})#$1\t$2\t$3\t$4\t$5\t$6\t$7\t$8\t$9\t$10\t$11#g; s#(FF17)(.{2})(.{1})(.{2})(.{2})(.{2})(.{2})(.{4})(.{4})#$1\t$2\t$ +3\t$5$4\t$7$6\t$8\t$9#g; my @spds = /(FF15)\t(.{2})\t(.{1})\t(.{4})\t(.{4})\t(.{8})/; # convert the data from hex to ascii foreach my $data (@spds){ my $replacement = hex($data); s#$data#$replacement#g; } my @psis = /(FF16)\t(.{2})\t(.{1})\t(.{2})\t(.{2})\t(.{2})\t(.{2}) +\t(.{2})\t(.{2})\t(.{2})\t(.{2})/; foreach my $data1(@psis){ my $replacement1 = hex($data1); s#$data1#$replacement1#g; } my @rates= /(FF17)\t(.{2})\t(.{1})\t(.{4})\t(.{1,4})\t(.{4})\t(.{ +4})/; foreach my $data2 (@rates){ my $replacement2 = hex($data2); s#$data2#$replacement2#g; } # print the converted data to the destination file print DEST; } # close the files close SOURCE; close DEST; } &create_vals; # perform conversion from raw values to human readable output sub create_vals{ my $source = "$ARGV[0]"."_tmp.txt"; my $dest = "$ARGV[0]"."_converted.txt"; chomp $source; open SOURCE, '<', $source or die "Cannot open '$source' $!"; open DEST, '>', $dest or die "Cannot open '$dest' for writing $!"; while(<SOURCE>){ s#(65301)\t(.{2})\t(8)\t(.{1,5})\t(.{1,5})\t(.{1,4})#"'".$1."','". + $2."','". $3."','". ($4/8)."','". ($5/8)."','". ($6/20)."'"#eg; s#(65302)\t(.{2})\t(8)\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})\t(. +{1,3})\t(.{1,3})\t(.{1,3})\t(.{1,3})#"'".$1."','". $2."','". $3."','" +. $4."','". (($51.8)-40)."','". (($61.8)-40)."','". ($7.58)."','". +($8.58)."','".($9.29008)."','".(($101.8)-40)."','".$11."'"#eg; s#(65303)\t(.{2})\t(8)\t(.{1,5})\t(.{1,5})\t(.{1,5})\t(.{1,5})#"'" +.$1."','". $2."','". $3."','". ($40.014)."','". ($5.05)."'"#eg; print DEST; } } close SOURCE; close DEST; __END__ [download] Also, If anyone has any pointers on how to clean this up it would be appreciated as I'm sure that this isn't the most efficient way to handle this (I am still very much a Perl n00b ;-))	[reply] [d/l]


"be consistent"
	PerlMonks