Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^2: Parsing EDI messages

by Miguel (Friar)
on Oct 09, 2005 at 16:16 UTC ( [id://498594]=note: print w/replies, xml ) Need Help??


in reply to Re: Parsing EDI messages
in thread Parsing EDI messages

Thnk you for your interest Thor.
Altought I don't have all the answers for your questions, here is what I have now:

My message looks like (I changed some codes):

UNH+789+INVOIC:D:93A:UN:EAN007' BGM+380+1234567890' DTM+137:20040726:102' RFF+DQ:9087654321' RFF+ON:01234567' NAD+BY+9182736453627::9' RFF+VA:123987465' RFF+API:123' NAD+SU+123987658XXXX::9' RFF+VA:200XXXXXX' NAD+IV+1928374650981::9' CUX+2:EUR:4' LIN+1++230XXXXXXXXXX:EN' IMD+F+M+:::Pencils' QTY+47:70' MOA+66:114.45' PRI+AAB:5.36' PRI+AAA:1.64' ALC+A+++1+TD' PCD+1:69.495' UNS+S' CNT+2:1' MOA+139:138.48' MOA+79:114.45' MOA+125:114.45' MOA+176:24.03' MOA+98:375.20' TAX+7+VAT+++:::21' MOA+176:24.03' UNT+30+798'
Now, what I want to achieve is:
  1. Validate this message against standard rules (and possibly some set of rules defined by the receiver);
  2. If is valid, create a Perl data structure, something better than this (I'm using a Data segment index from an external file):
    $VAR1 = { 'PDS' => { '11' => { 'SEG' => [ [ 'IV' ], [ '1928374650981', '', '9' ] ], 'CODE' => { 'DES' => 'Name and address' +, 'TAG' => 'NAD' } }, '21' => { 'SEG' => [ [ 'S' ] ], 'CODE' => { 'DES' => 'Section control', 'TAG' => 'UNS' } }, '7' => { 'SEG' => [ [ 'VA', '123987465' ] ], 'CODE' => { 'DES' => 'Reference', 'TAG' => 'RFF' } }, '26' => { 'SEG' => [ [ '176', '24.03' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '17' => { 'SEG' => [ [ 'AAB', '5.36' ] ], 'CODE' => { 'DES' => 'Price details', 'TAG' => 'PRI' } }, '2' => { 'SEG' => [ [ '380' ], [ '1234567890' ] ], 'CODE' => { 'DES' => 'Beginning of messa +ge', 'TAG' => 'BGM' } }, '22' => { 'SEG' => [ [ '2', '1' ] ], 'CODE' => { 'DES' => 'Control total', 'TAG' => 'CNT' } }, '1' => { 'SEG' => [ [ '789' ], [ 'INVOIC', 'D', '93A', 'UN', 'EAN007' ] ], 'CODE' => { 'DES' => 'Message header', 'TAG' => 'UNH' } }, '18' => { 'SEG' => [ [ 'AAA', '1.64' ] ], 'CODE' => { 'DES' => 'Price details', 'TAG' => 'PRI' } }, '30' => { 'SEG' => [ [ '30' ], [ '798' ] ], 'CODE' => { 'DES' => 'Message trailer', 'TAG' => 'UNT' } }, '23' => { 'SEG' => [ [ '139', '138.48' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '16' => { 'SEG' => [ [ '66', '114.45' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '13' => { 'SEG' => [ [ '1' ], [], [ '230XXXXXXXXXX', 'EN' ] ], 'CODE' => { 'DES' => 'Line item', 'TAG' => 'LIN' } }, '29' => { 'SEG' => [ [ '176', '24.03' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '27' => { 'SEG' => [ [ '98', '375.20' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '25' => { 'SEG' => [ [ '125', '114.45' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '6' => { 'SEG' => [ [ 'BY' ], [ '9182736453627', '', '9' ] ], 'CODE' => { 'DES' => 'Name and address', 'TAG' => 'NAD' } }, '28' => { 'SEG' => [ [ '7' ], [ 'VAT' ], [], [], [ '', '', '', '21' ] ], 'CODE' => { 'DES' => 'Duty/tax/fee deta +ils', 'TAG' => 'TAX' } }, '3' => { 'SEG' => [ [ '137', '20040726', '102' ] ], 'CODE' => { 'DES' => 'Date/time/period', 'TAG' => 'DTM' } }, '9' => { 'SEG' => [ [ 'SU' ], [ '123987658XXXX', '', '9' ] ], 'CODE' => { 'DES' => 'Name and address', 'TAG' => 'NAD' } }, '12' => { 'SEG' => [ [ '2', 'EUR', '4' ] ], 'CODE' => { 'DES' => 'Currencies', 'TAG' => 'CUX' } }, '20' => { 'SEG' => [ [ '1', '69.495' ] ], 'CODE' => { 'DES' => 'Percentage detail +s', 'TAG' => 'PCD' } }, '14' => { 'SEG' => [ [ 'F' ], [ 'M' ], [ '', '', '', 'Pencils' ] ], 'CODE' => { 'DES' => 'Item description' +, 'TAG' => 'IMD' } }, '15' => { 'SEG' => [ [ '47', '70' ] ], 'CODE' => { 'DES' => 'Quantity', 'TAG' => 'QTY' } }, '8' => { 'SEG' => [ [ 'API', '123' ] ], 'CODE' => { 'DES' => 'Reference', 'TAG' => 'RFF' } }, '4' => { 'SEG' => [ [ 'DQ', '9087654321' ] ], 'CODE' => { 'DES' => 'Reference', 'TAG' => 'RFF' } }, '24' => { 'SEG' => [ [ '79', '114.45' ] ], 'CODE' => { 'DES' => 'Monetary amount', 'TAG' => 'MOA' } }, '19' => { 'SEG' => [ [ 'A' ], [], [], [ '1' ], [ 'TD' ] ], 'CODE' => { 'DES' => 'Allowance or char +ge', 'TAG' => 'ALC' } }, '10' => { 'SEG' => [ [ 'VA', '200XXXXXX' ] ], 'CODE' => { 'DES' => 'Reference', 'TAG' => 'RFF' } }, '5' => { 'SEG' => [ [ 'ON', '01234567' ] ], 'CODE' => { 'DES' => 'Reference', 'TAG' => 'RFF' } } }, 'EDI' => 'UNH+789+INVOIC:D:93A:UN:EAN007\' BGM+380+1234567890\' DTM+137:20040726:102\' RFF+DQ:9087654321\' RFF+ON:01234567\' NAD+BY+9182736453627::9\' RFF+VA:123987465\' RFF+API:123\' NAD+SU+123987658XXXX::9\' RFF+VA:200XXXXXX\' NAD+IV+1928374650981::9\' CUX+2:EUR:4\' LIN+1++230XXXXXXXXXX:EN\' IMD+F+M+:::Pencils\' QTY+47:70\' MOA+66:114.45\' PRI+AAB:5.36\' PRI+AAA:1.64\' ALC+A+++1+TD\' PCD+1:69.495\' UNS+S\' CNT+2:1\' MOA+139:138.48\' MOA+79:114.45\' MOA+125:114.45\' MOA+176:24.03\' MOA+98:375.20\' TAX+7+VAT+++:::21\' MOA+176:24.03\' UNT+30+798\' ' };
    I wrote a small script to do this part:
    #!/usr/bin/perl -w use strict; use CGI; use CGI::Carp qw/fatalsToBrowser/; use Data::Dumper; my $q = new CGI; my $result = ParseEDI( { EDI => '/path/to/edis/invoice.edi', DIC => '/path/to/dics/d93a_invoic.dat', } ); print $q->header, $q->start_html, $q->start_pre; print Dumper $result; print $q->end_pre, $q->end_html; sub ParseEDI { my $self = shift; my $output = { EDI => undef, # Original message PDS => undef, # Perl Data Structure }; open ( EDI, "<", $self->{EDI} ) or die ( "Error opening EDI messag +e"); my $i = 1; while ( <EDI> ) { $output->{EDI} .= $_; my ($CODE, $VARS) = ($_ =~/^(\w{3})\+(.*)'$/); $output->{PDS}->{$i} = { CODE => { TAG => $CODE, DES => DataIndex( { DIC => $self->{DIC} } )->{$CODE}, }, SEG => [map ([map ($_ , split /:/)] , split /\+/ , $VARS)] +, }; $i++; } close ( EDI ); return $output; } sub DataIndex { my $self = shift; open (DIC, "<", $self->{DIC} ) or die "Error opening the data inde +x"; my $res = {}; while (<DIC>) { my ($CODE, $DESC) = ($_ =~/^(\w{3}) (.*)$/ ); $res->{$CODE} = $DESC; } close DIC; return $res; }
  3. Show some informations to the user.

My problems are:

  1. How to apply the rules (the 'standard' and others defined by the receiver)? There are 100s (perhaps 1000s) of constraints to check. How to do the mappings?
  2. How to construct a solid, fast and accurate data structure that allows me to extract any kind of information from an EDI message?
Hope this helps.
Miguel

Edit: g0n - Added readmore tags

Replies are listed 'Best First'.
Re^3: Parsing EDI messages
by thor (Priest) on Oct 10, 2005 at 19:44 UTC
    My problems are:
    1. How to apply the rules (the 'standard' and others defined by the receiver)? There are 100s (perhaps 1000s) of constraints to check. How to do the mappings?
    2. How to construct a solid, fast and accurate data structure that allows me to extract any kind of information from an EDI message?
    I can't speak to the first (showing an example or three of a "rule" might help), but the second almost says database to me. Data structures are geared well towards doing things in a certain way. For instance, if you create a data structure based on people, you could structure it as a hash of hashes, with family name as the primary key and individuals as the sub-keys. Finding how many people are males over 25 from this structure isn't pretty. It seems that you want more flexiblity in the way that you get data out of the parsed message. As it stands, your initial structure didn't look that bad. The only complaint that I'd have against it is the existance of non-descriptive keys at the first level (they're all numbers which don't convey any information to me). After that, everything starts to fall into place.

    thor

    Feel the white light, the light within
    Be your own disciple, fan the sparks of will
    For all of us waiting, your kingdom will come

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://498594]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (6)
As of 2024-04-19 12:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found