Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

EBCDIC & COMP-3 *shriek*

by Grygonos (Chaplain)
on Jan 22, 2004 at 22:01 UTC ( [id://323332]=perlquestion: print w/replies, xml ) Need Help??

Grygonos has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

The need has arisen for me to have a method of converting COMP-3 packed zoned EBCDIC fields to ASCII fields. I have written some tests with Convert::EBCDIC and found that it works for translating EBCDIC to ASCII

However, converting an EBCDIC field to ASCII, will not take care of the decoding of the fields. I'm wanting to learn this and not buy some product that does the conversion, mainly because those are expensive ;)

Does anyone have any good information about how to decode a COMP-3 packed/zoned field? i.e. an explanation of how comp-3 packed/zoned is performed when putting it into that format, so that I can simply reverse the process and then convert the data to ASCII via the module listed above.

I have googled this a bit but since the technology is pretty much abandoned in modern day teaching and tech(with the exception of legacy mainframes) I am having trouble finding good information. Even a book would be good at this point...

Thanks monks,

Grygonos

Replies are listed 'Best First'.
Re: EBCDIC & COMP-3 *shriek*
by Old_Gray_Bear (Bishop) on Jan 22, 2004 at 23:37 UTC
    This is a blast from my COBOL past.

    Comp-3 (aka 'packed decimal') is a compression technique for numbers on the IBM mainframes. In EBCDIC, the digits are x'F0', x'F1', etc. (Called 'zoned decimal' or display.) It was noted very early on (late 50's? certainly by the early 60's when OS/360 came out) that the string representation of digits containd a lot of redundant bits, from a computational point of view. You could compress out the 'noise' and reduce the memory foot print. COMP-3 was the COBOL name for this compressed form.

    For example -- '1234' is x'F1F2F3F4' as a string, of length four bytes. The packed decimal representation is x'0123C4', only three bytes long; net saving 1 byte of memory and disk storage. Not so much, you say, but remember -- this technique was developed at the time when 24K of main memory was the norm. (That 'K' was not a typo. The first 'production' machine I wrote code for had 4096 bytes of memory!) Packed decimal will always be no longer than (length of source/2)+1, and could provide a considerable space savings for commercial programs. Scientific programming used floating point (either 32, 64, or 128 bits wide) and didn't concern itself with such trivia as memory footprint.

    The process is to strip out the high order four bits of each number, concatenate all but the last digit together, insert a 'zone' to indicate the sign (odd zones (A, C, E) are positive; evens (B, D) are negative; if memory serves me. My green card is at home in the memorabilia drawer), followed by the last digit. If you started with an even number of digits in your string, you add a leading byte of zeros to make up a full byte. There are a full set of instructions that manage and compute with packed decimal format (including my favorite: Zero and ADD, Packed -- ZAP).

    That out of the way, I suspect that a modicum of C<pack> and C<unpack> logic will get you where you want to go.

    Best of luck,

    ----
    I Go Back to Sleep, Now.

    OGB

Re: EBCDIC & COMP-3 *shriek*
by Old_Gray_Bear (Bishop) on Jan 23, 2004 at 03:26 UTC
Re: EBCDIC & COMP-3 *shriek*
by dbush (Deacon) on Jan 23, 2004 at 10:58 UTC

    Hi Grygonos,

    Have a look at Convert::IBM390. This module has conversion functions for data in those formats. I have successfully used it to unpack COMP-3 packed fields.

    Regards,
    Dom.

      Thanks for all the replies monks. What I'm doing is translating a data dump from our clients mainframe. I can't get them to convert it...I'm by no means in a position of authority in this case. This IBM convert module may work. I REALLY hope it does. Thanks for all the great information.
        Sorry the above AM is me.. forgot to login.

        Grygonos
        Be very careful with that module. I have an example of where a client made the zipcode packed decimal. Convert::IBM390 loses leadind zeros. So, 02354 becomes 2354. Not what you want.
      Grygonos,

      Convert::IBM390 works very well at converting COMP-3 to the ASCII world. I'm currently writing a generic ebcdic2ascii.pl translator, which given a map file will spit out comma separated value files.

      But, if you need to convert lots of EBCDIC string data to ASCII, a simple "dd conv=ascii if=infile of=ofile" will serve you lots faster.

      Cheers,

      jhorner

Re: EBCDIC & COMP-3 *shriek*
by pilgrim (Monk) on Jan 22, 2004 at 22:47 UTC

    I believe this explanation should be a step in the right direction.

    --pilgrim
Re: EBCDIC & COMP-3 *shriek*
by sts (Initiate) on Jan 22, 2004 at 22:42 UTC
Re: EBCDIC & COMP-3 *shriek*
by pboin (Deacon) on Jan 23, 2004 at 03:26 UTC
    Are you by chance trying to decipher an Oracle dump?

    I have a quick-n-dirty around here somewhere to reverse that, and believe me, the sneaky bastards went out of their way to make it difficult to read. (It really chaps my ass when companies forget whose data it is. Code, well that's fine, but it's *my data*, period.)

    I'm so tired I'm ready to fall over, and I'd like to review said code before I post it. Let me know if that's what you're working on, and I'll bump it up on my list tommorrow morning.

Easy three-step method for decoding EBCDIC COMP-3 data
by jonadab (Parson) on Jan 23, 2004 at 05:41 UTC
    1. Find out who encoded the data in this format in the first place.
    2. Tie said person to a workbench.
    3. Beat said person's feet with a rubber hose until an agreement is reached wherein said person will retype all of the data in a sensible format.

    This method is sometimes known as "rubber hose cryptanalysis".


    $;=sub{$/};@;=map{my($a,$b)=($_,$;);$;=sub{$a.$b->()}} split//,".rekcah lreP rehtona tsuJ";$\=$ ;->();print$/
Re: EBCDIC & COMP-3 *shriek*
by sth (Priest) on Jan 23, 2004 at 19:14 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://323332]
Approved by VSarkiss
Front-paged by diotalevi
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (4)
As of 2024-03-28 21:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found