Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: Text::xSV

by princepawn (Parson)
on Apr 17, 2001 at 01:41 UTC ( #72969=note: print w/replies, xml ) Need Help??

in reply to Text::xSV

A related module(s):
  • Text::FixedLength
  • Text::FixedLength::Extra
  • Replies are listed 'Best First'.
    Re (tilly) 2: Text::xSV
    by tilly (Archbishop) on Apr 17, 2001 at 04:17 UTC
      I believe they are orthogonal to my module.

      Besides which I would not encourage the proliferation of file formats which are by design not extensible or self-documenting. See the note in the documentation that I provided about exactly that. Fixed width file formats are not exactly on my hit parade of great ideas to encourage.

      An incidental question. Even if I did want to use fixed width file formats, what is your value add over the better-known pack and unpack plus Perl's native support for formats? Perl started life dealing with lots of ASCII reports and has quite a few now little-used facilities for doing just that.

        If one is designing from scratch, then you are right, fixed-width is not extensible nor self-documenting. But I know that Valley Media, the company who does fulfillment for and CDNOW and some other people I have worked for, does use fixed-width format and they have too much technology behind it too change in midstream. I would be surprised if they are the only such place using this admittedly inferior format.

        Regarding use pack and unpack instead of Parse::FixedLength, Text::FixedLength, and Text::FixedLength::Extra, if you read "Data Munging with Perl", the answer would be none, because he did not mention the FixedLength module set in his book, but instead focused on pack and unpack.

        I think pack and unpack are excellent for implementation of fixed-width data processing, but in terms of making a human-level description, a higher level interface is needed. For example, let's say I have this:
        field name field width justifiction numeric?
        catalog id 12 L no
        price 6 L yes -- with 2 decimal places

        Here is a high-level description that one could expect a data-entry expert to convert the english above into:

        catalog_id = '12L', price = '6L*2'
        and to turn that into the programmatic representaiton of Parse::FixedLength is a one-step transformation:
        package Business::Logic::Orders; our $line_data = [ { catalog_id => '12L' } , { price => '6L*2' } ]; 1;
        And then the script which plugs into the data description is quite clean, with no low-level Perl pack/unpack:
        ### open D, 'data-file.dat'; while (<D>) { Parse::FixedLength::Parse(\%line_parse, $Business::Logic::Orders); DBIx::Recordset->Insert({ \%dsn }, \%line_parse} ); }
        And of course, internally Parse::FixedLength can implement it's high-level API with pack, unpack, or sprintf as need be.
          Sorry, but I don't buy it.

          I think you are overestimating the difficulties of using pack/unpack, underestimating the abilities of the people who are likely to deal with the format and overestimating again the naturalness of interfaces you have grown for other people.

          You have also discounted entirely the cost of learning your interface (particularly if someone doesn't know any Perl) and there is a loss of flexibility. You see, abstractions require domain specific knowledge, and they are only worthwhile when they either significantly simplify things, or when you will have the opportunity to use that information over and over again. Neither appears to be the case here.

          Plus looking at your example snippet I have to wonder at the amount of pass by reference magic you seem to be fond of. That is not the kind of API I like to write or use.

          In short, I don't deal with a lot of fixed position formats. But if I found a need to, I would spend time getting more familiar with perlform and Perl's other native facilities before reaching for these modules.

          Trust me they are _definitely_ not the only ones that use fixed-width file formats. Fixed-width file formats are big in the financial/banking industry. Look up ACH files for example.

    Log In?

    What's my password?
    Create A New User
    Node Status?
    node history
    Node Type: note [id://72969]
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others drinking their drinks and smoking their pipes about the Monastery: (6)
    As of 2020-11-24 10:32 GMT
    Find Nodes?
      Voting Booth?

      No recent polls found