Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Perl record types

by naveed010 (Acolyte)
on Jan 17, 2008 at 13:11 UTC ( [id://662863]=perlquestion: print w/replies, xml ) Need Help??

naveed010 has asked for the wisdom of the Perl Monks concerning the following question:

Good morning most gracious of monks.

Being a relative noob, I'm looking on some good information on Perl records, I've looked around google and this site, only coming up with the basics ie.

@records = ( {field1 => 'row1-1', field2 =>'row1-2'}, {field1 => 'row2-1', field2 =>'row2-2)); foreach my $row (\@records) { #..Do some stuff.. }

Specifically, I'm trying to load the contents of a pipe-delimited file into a record. The end goal of what I'm doing is creating an FTP service, the config file is what stores various things such as frequency of polling, etc etc.

So, here's my questions:

  • How do I pre-define the record type, so I can use it throughout my scripts (I am aware I can create it in a "require my_library.pl" file.)
  • Is there a method to suck the entire contents of the file into the @records variable at once, putting everything in it's place?

I should also add, I can't really add new modules, the sys admins take weeks here to add a new modules, the approval process is simply brutal (public sector).

If someone has a link that I've overlooked, a small example, or something relevant they think would be a help, I would greatly appreciate it.

Thanks again most brilliant of monks.

20080119 Janitored by Corion: Fixed UL tag, as per Writeup Formatting Tips

Replies are listed 'Best First'.
Re: Perl record types
by amarquis (Curate) on Jan 17, 2008 at 14:14 UTC

    When you say "pre-defined" record type, are you looking for something like a C struct or an Object?

    perldoc perlfaq4 might have what you are looking for, under "How-can-I-make-the-Perl-equivalent-of-a-C-structure C++-class hash-or-array-of-hashes-or-arrays". Also, perldoc perldsc is the best place to go to see examples of simple compound data structures in use in Perl. perldoc perltoot, besides having a cute name, is also a good "objects in Perl" starter kit.

    Is there something pre-built to do your work for you? In general, it depends on how standard your input and output are. I'm thinking for this (And I think they might be core, they come with my distro of ActivePerl, at least) that you can use one of the CSV modules set to use the pipe as the delimiter to slurp your data in.

    If you are totally sure your data will not contain any pipe characters, you can feed each line to the split built-in, which can return a list of each pipe-delimited value.

      Hi amarquis. Thanks for the response. I've been browsing that sinse you pointed it out. Great help!

      This is sort of what I am looking for. What I would do if I was doing this inside of Oracle (PL/SQL) (which is where my background is) would be to create a record type such as:

      type example_rec is record ( IP varchar2(10), Username vachar2(56), get_files varchar2(15));
      and subsequently create a table of those records such as:
      ex_table is TABLE of example_rec index by binary_integer;
      Subsequently opening the text file, looping through it and filling the table with the rows in the text file.

      I hope this gives you a better understanding of what I'm trying to do.

      I'm certain my data won't contain any pipes, I'm the only one that will be using it. I'm trying to create multiple ftp transfers for various files that need to be moved around here. So it was either multiple scripts, or try and create one to do all using a configuration file.

      Eventually, my plan is to use an Oracle external table to allow users to access it, but that's further down the road.

Re: Perl record types
by naChoZ (Curate) on Jan 17, 2008 at 14:28 UTC

    Parse::CSV would really fit the bill, I even demonstrated using a pipe sep over here.

    But if you're stuck without using modules, you could probably rig sub up to do what you want easily. This just reads a text file of values in the form of:

    some_key_param=value

    my $config = read_config({ filename => 'some_config_file' }); # {{{ read_config # sub read_config { my $args = shift; die "No configuration file specified\n" unless defined $args->{filename}; open my $conf_fh, '<', $args->{filename} or die "Error opening conf file...$!\n\n"; my $conf = {}; for (<$conf_fh>) { chomp; # use '=' as the separator if ( $_ =~ m/[=]/ ) { # skip these lines s/#.*//; s/^\s+//; s/\s+$//; next unless length $_; my ( $key, $value ) = split( /\s*=\s*/, $_, 2 ); $config->{$key} = $value; } } close $conf_fh; return $conf; } # }}}

    --
    naChoZ

    Therapy is expensive. Popping bubble wrap is cheap. You choose.

Re: Perl record types
by dwm042 (Priest) on Jan 17, 2008 at 16:56 UTC
    Ok, these are my recommendations. I'd recommend not using a pipe delimited file at all, but look closely at the CPAN module Config::Simple. Why?

    Parsing a pipe delimited file usually gets done something like this:
    my $data = `grep -i $account_name /config/data.file`; my @parms = split /|/, $data;
    Which leads to writing code like this:

    unless ( -d $parms[3] ) { # # do some ftp stuff here. # }
    And this kind of code, which relies on the context of the fields of the pipe delimited file, is hard to read and hard to maintain, and even harder to extend.

    The advantage in Config::Simple is that you have context, the keys and the values. The code no longer becomes dependent on the order of data in your data file. You won't create a massive mess if you insert another value in the middle of an entry in a Config::Simple file, whereas the maintenance issues you'll create with careless use of field position in a pipe delimited file will be huge.

    If you have to go the pipe delimited method, I'd either have a separate file that map the configuration of the pipe file, and then I'd insert your pipe values into a hash (by mapping hash keys to positions of your parameters) or have a header at the top of the pipe delimited file that does the same; one that is read by the program and used to set the names of values.

    You /do not/ want to have to find your FTP program has become a SFTP and oh, by the way, they want you to transmit on port 2222 rather than 22 after you've hard coded your pipe delimited field positions into your Perl.

      Hi dwm.

      Config::Simple looks like an ideal solution for me.

      Thanks for your suggestion!

      Why not just:
      use constant FILE_NAME => 3; # ... unless ( -d $parms[FILE_NAME] ) { # # do some ftp stuff here. # }
      Gives you compile-time checks and has all the advantages of the hash solution.
        plobsing, your suggestion improves readability, but the code is still directly tied to the data file and its internal representation. Consider this scenario:

        1. You write a piece of code tied to the data representation.

        2. You leave the company.

        3. It becomes very popular.

        4. Dozens of people want to use it, but they want to each make lots of small changes to the data representation. Rather than rewriting the code to handle all cases, they cut and paste unique versions of the code and change the constant mapping to fit their unique cases.

        And all I'll say is, I've been there, at step 5, when someone gets asked to refactor all those scripts into 1 script.

        So I beg to differ: as long as your code is tied directly to data representation, it isn't as good as indirect solutions, where the representation can be modified without affecting running code.

        Update: typo fix
Re: Perl record types
by mr_mischief (Monsignor) on Jan 17, 2008 at 16:20 UTC
    Any module that's pure Perl code can just be put inside your project's directory with a (file manager|FTP client|cp command). You might want to look at use which offers more niceties than require for the type of thing you're trying to do.
Re: Perl record types
by Anonymous Monk on Jan 17, 2008 at 15:46 UTC
Re: Perl record types
by Joost (Canon) on Jan 18, 2008 at 20:08 UTC
    I'm not clear on what you want exactly. How does your record type relate to the input file?

    If all you want is to assign names to columns the type could be as easy as:

    my $c = 0; our @Columns = qw(field1 field2 ...); our %Column_numbers = map { $_ => $c++ } @Columns;
    Where I use a package variable to indicate that @Columns etc are "global" and so that it can be accessed when that code is loaded from another file / package.

    Any utility subs (like parsing the input file) can go in the same module, so they can be re-used.

    As for existing CPAN modules, you should be aware that you can generally install perl modules locally (i.e. in a sub directory of your choice, so no root permissions necessary), That also usually means you can distribute them with the rest of the program tree without any issues. The most common drawback is that this won't work for XS/C based modules across architectures.

    Text::CSV and Text::xSV can take some work out of your hands for this specific problem.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://662863]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (1)
As of 2024-04-19 18:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found