Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

csv to flat-file

by Cockneyphil (Acolyte)
on Nov 28, 2002 at 08:39 UTC ( [id://216257]=perlquestion: print w/replies, xml ) Need Help??

Cockneyphil has asked for the wisdom of the Perl Monks concerning the following question:

Is there any "quick" way to ocnvert a one line csv file into a flat-file ie from xxx,yyy,zzz into xxx yyy zzz

Replies are listed 'Best First'.
Re: csv to flat-file
by BrowserUk (Patriarch) on Nov 28, 2002 at 09:50 UTC

    It just struck me that Perl already knows how to deal with csv data without needing any modules or clever regexes.

    C:\test>type junk.txt 1, 1.2, -3000, fred, "bill & ben", "there's no business like show busi +ness", 'stuff, with, commas' C:\test>perl -ne "my @a; eval qq[\@a = ($_);];print qq[@a],$/" junk.tx +t 1 1.2 -3000 fred bill & ben there's no business like show business stu +ff, with, commas C:\test>

    A list assignment is a very proficient parser for csv data.

    Update:Having seen the update that you want the fields newline delimited, this should handle most forms of csv file.

    C:\test>perl -ne "my @a; eval qq[\@a = ($_);];print $_, $/ for @a" jun +k.txt 1 1.2 -3000 fred bill & ben there's no business like show business stuff, with, commas C:\test>

    Okay you lot, get your wings on the left, halos on the right. It's one size fits all, and "No!", you can't have a different color.
    Pick up your cloud down the end and "Yes" if you get allocated a grey one they are a bit damp under foot, but someone has to get them.
    Get used to the wings fast cos its an 8 hour day...unless the Govenor calls for a cyclone or hurricane, in which case 16 hour shifts are mandatory.
    Just be grateful that you arrived just as the tornado season finished. Them buggers are real work.

      That's just scary. If you're going to solve things that way then just use something like perl -naF, -e 'INIT{ $, = $/ } print @F'. It's going to be more direct and not make tons of eval STRING calls.

      Update: There aren't any quoted commas in the original example - this still works fine for normal comma delimited data anyway. (no quoting/escaping allowed)

      __SIG__ use B; printf "You are here %08x\n", unpack "L!", unpack "P4", pack "L!", B::svref_2object(sub{})->OUTSIDE;

        Then main reason is because your version doesn't work. Given this file

        C:\test>type junk.txt 1, 1.2, -3000, fred, "bill & ben", "there's no business like show busi +ness", 'stuff, with, commas'

        Your version does this.

        C:\test>perl -naF, -e"INIT{ $,=$/} print @F" junk.txt 1 1.2 -3000 fred "bill & ben" "there's no business like show business" 'stuff with commas'

        Note

        • the extra space at the beginning of every line except the first.
        • the unstripped quotes
        • the quoted string with embedded commas split into seperate fields.

        Whereas my version does this.

        C:\test>perl -ne "my @a; eval qq[\@a = ($_);];print $_, $/ for @a" jun +k.txt 1 1.2 -3000 fred bill & ben there's no business like show business stuff, with, commas

        I'm not sure what you have against using eval? Your exercising the very same parser everytime you do a list assignment in your program code.


        Okay you lot, get your wings on the left, halos on the right. It's one size fits all, and "No!", you can't have a different color.
        Pick up your cloud down the end and "Yes" if you get allocated a grey one they are a bit damp under foot, but someone has to get them.
        Get used to the wings fast cos its an 8 hour day...unless the Govenor calls for a cyclone or hurricane, in which case 16 hour shifts are mandatory.
        Just be grateful that you arrived just as the tornado season finished. Them buggers are real work.

Re: csv to flat-file
by Bilbo (Pilgrim) on Nov 28, 2002 at 08:59 UTC

    Why not just use a substitution to convert commas to spaces:

    $line =~ s/,/ /g;

    Alternatively split the fields up and join them back together:

    my @tmp_array = split(',',$line); $line = join(' ',@tmp_array);
    or more compactly but less readably
    $line = join(' ', split(',',$line));

      All of these solutions are great, but they all assume that you can't have a comma as part of the field value.

      Given that this is almost always possible, you have two choices:

      1. If commas as part of the field are escaped, then you can use a negative look-behind assertion:split(/(?<!\\),/, $line)
      2. If commas are not escaped, but can form part of a field delimitted with quotation-marks of some kind (" or ') then things get a lot uglier. I believe that there's a regex that could do it, but I could never figure it out and had to switch to a while loop and essentially chew through a line.

      HTH

      The question hasn't displayed properly (it did on the preview though) the output i'm after is xxx yyy (on a new line) zzz (on a new line)

        OK. In the substitution replace the comma witha newline character: s/,/\n/g;

        The split solution could become:

        $, = "\n"; print split(',',$line);

        ($, = "\n"; sets the character used to separate entries when print is used to print an array, as returned by split)

        As pointed out, all these solutions fail if any entries can contain a comma, in which case you should investigate the modules suggested elsewhere. I was assuming that since the format of your example was simple you would not have such complex data.

Re: csv to flat-file
by Jenda (Abbot) on Nov 28, 2002 at 13:36 UTC

    If you want to be safe you should use Text::CSV_XS:

    ... use Text::CSV_XS; my $reader = new Text::CSV_XS { 'sep_char' => ',', 'quote_char' => '"', 'escape_char' => '\\', 'binary' => 1, }; $data = $reader->getline(IN); print OUT "$data->[0] $data->[1]\n$data->[2]\n"; ...

    Jenda

Re: csv to flat-file
by robartes (Priest) on Nov 28, 2002 at 09:02 UTC
    The quick and dirty way to do what you imply in your question:
    perl -nep 's/,/ /;' untested.csv
    However, if you need to do anything more intricate with CSV files, have a look at Text::CSV for all your csv manipulation needs.

    CU
    Robartes-

      You need the 'g' on the end of the regexp, ie s/,/ /g;otherwise only the first comma is replaced (g for global). Also I think this should have read:

      perl -pe 's/,/ /;' untested.csv

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://216257]
Approved by JaWi
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (2)
As of 2024-04-26 07:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found