Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Extracting multiple column from csv file and adding/appending onto new csv file with first columns already existing

by bks (Novice)
on Apr 17, 2014 at 08:45 UTC ( #1082609=perlquestion: print w/replies, xml ) Need Help??

bks has asked for the wisdom of the Perl Monks concerning the following question:

Hi All

My requirement is to connect to multiple network devices(cisco,juniper) and pull some data to csv file and further filtering it (column wise) and putting it in different csv.

However i was able to extract single column and display it onto console, but as said i wanted multiple columns to be filtered.

Sample command file output and csv file

Command output

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

2.2.2.2 4 100 273 274 2 0 0 04:30:05 0

3.3.3.3 4 100 273 274 2 0 0 04:30:05 0

4.4.4.4 4 100 273 273 2 0 0 04:30:00 0

CSV FILE

Neighbor,V,AS,MsgRcvd,MsgSent,TblVer,InQ,OutQ,Up/Down,State/PfxRcd

2.2.2.2,4,100,273,274,2,0,0,04:30:05,0

3.3.3.3,4,100,273,274,2,0,0,04:30:05,0

4.4.4.4,4,100,273,273,2,0,0,04:30:00,0

Required output

Host/Device IP,Neighbor,Up/Down

192.168.10.3,2.2.2.2,4:30:05

192.168.10.3,3.3.3.3,4:30:05

192.168.10.3,4.4.4.4,4:30:00

I would be getting similar output from different devices and i would like append the output to same csv file

I tried installing text::csv_xs, but unfortunately i wasnt able extract *.tgz as it gave some checksum error. Please guide me or point me in right direction about how to install modules that are in *.tgz format

Note: Running perl on solaris 10

Please post some sample code to achieve my requirement that would really be of great help

My code

#!/usr/bin/perl use Net::Telnet::Cisco; use Text::CSV; ######### Command line argument while executing the script ########### +#### if ($#ARGV < 0) { print "Please run the script with correct arguments\n\n"; print "Usage Details:perl bgp_summary_xls.pl <Device IP>\n"; print "Ex: perl bgp_summary_xls.pl 192.168.3.10\n"; exit; } $ip = $ARGV[0]; print "$ip\n"; #Connecting to Cisco Device my $session = Net::Telnet::Cisco->new(Host => $ip); $session->login('cisco', 'cisco'); # Execute a command my @output = $session->cmd('show ip bgp summary | begin Neighbor'); #my $file = /AutomationScripts/Korea_health_checks/test; open my $fh, ">test" or die "Cant open file died unexpectedly"; foreach (@output) { #@output = split(/\t/, $_); print $fh $_; } close $fh; ###### To write to csv ###### open (TABFILE, 'test') or die "test: $!"; open (CSV,">test.csv") or die "test csv: $!"; my @fld; while (<TABFILE>) { chomp; my @fld = split('\t', $_); #print "Before @fld"; for (@fld) { s/\+//g; s/ +/,/g; } print CSV "@fld\n"; } close CSV; ############### To extract column ################ my $csv = Text::CSV->new ({ binary => 1 }); open (CSV, "<test.csv") or die "Can't open $file: $!\n"; while (<CSV>) { if ($csv->parse($_)) { my @column = $csv->fields(); print "$column[8]\n"; } else { my $err = $csv->error_input; print "ERROR: Failed to parse line: $err"; } } close CSV;
  • Comment on Extracting multiple column from csv file and adding/appending onto new csv file with first columns already existing
  • Download Code

Replies are listed 'Best First'.
Re: Extracting multiple column from csv file and adding/appending onto new csv file with first columns already existing
by Yaerox (Scribe) on Apr 17, 2014 at 09:08 UTC

    I can't follow where Host/Device IP is cooming from. I guess this shows that you want to merge two csv files?

    I never worked on Solaris 10, but why not looking for .tar.gz and then using tar xzf (or something like that) to extract? A second way if installed would be cpan. get module, make module, test module, install module.

    Anyway, if you're able to extract one column, you just have to add the same part again in your loop by checking for another column ... you're creating an array or append on a string all your results (formatted) for both csv-files then writing a third loop printing into csv-file.

    Following code is not tested and should be more like a kind of Pseudo Code

    my @aColumns1; my @aColumns2; open ( my $sData, '<', $hFile ) or die "Could not open '$hFile' $!\n"; while ( my $sLine = <$sData> ) { chomp $sLine; my @aFields = split "," , $sLine; push ( @aColumns1, $aField[1] ); push ( @aColumns2, $aField[3] ); } close ( $hFile ); my @aColumns3; my @aColumns4; @aFields = ''; open ( $sData, '<', $hFile ) or die "Could not open '$hFile' $!\n"; while ( $sLine = <$sData> ) { chomp $sLine; @aFields = split "," , $sLine; push ( @aColumns3, $aField[2] ); push ( @aColumns4, $aField[4] ); } close ( $hFile ); open ( '>', $hFile ) or die "Could not open '$hFile' $!\n"; for (...) { print $hFile "$aColumns1[x];$aColumns2[x];$aColumns3[x];$aColumns4 +[x]\n"; } close ( $hFile );

    If you would like you could probably use 2D-Arrays too.

    Regards

      Yaerox, I think you'll find that the host IP is initially a script argument and is then subsequently returned by the invoked CISCO show command (IIRC :-).

      A user level that continues to overstate my experience :-))

        Yeah you're right. Found it ;-) But I think the way how to get this done can be pretty the same ... if not feel free to tell me :-)

        Thank you Sir.

Re: Extracting multiple column from csv file and adding/appending onto new csv file with first columns already existing
by kcott (Archbishop) on Apr 18, 2014 at 01:06 UTC

    G'day bks,

    Welcome to the monastery.

    "I would be getting similar output from different devices and i would like append the output to same csv file"

    I don't know which of the two CSV files this refers to. Use a mode of '>' to write to a new file; and a mode of '>>' to append to an existing file. See open for details. I've used append mode for both files in the code below.

    "I tried installing text::csv_xs, but unfortunately i wasnt able extract *.tgz as it gave some checksum error. Please guide me or point me in right direction about how to install modules that are in *.tgz format"

    You'll need to show us what you did and what output you got. The guidelines in "How do I post a question effectively?" explain the sort of information we need and how to present it.

    "Please post some sample code to achieve my requirement that would really be of great help"

    The code below shows how to use Text::CSV for all your I/O.

    #!/usr/bin/env perl -l use strict; use warnings; use autodie; use Text::CSV; my ($all_csv, $sel_csv) = qw{pm_1082609_all.csv pm_1082609_sel.csv}; my $ip = '192.168.10.3'; chomp( my @output = <DATA> ); my $csv = Text::CSV::->new(); open my $out_all_fh, '>>', $all_csv; $csv->print($out_all_fh => [ split /\t/ ]) for @output; close $out_all_fh; open my $in_all_fh, '<', $all_csv; open my $out_sel_fh, '>>', $sel_csv; while (my $row = $csv->getline($in_all_fh)) { $csv->print($out_sel_fh, [ $ip, @$row[0,8] ]); } close $in_all_fh; close $out_sel_fh; __DATA__ 2.2.2.2 4 100 273 274 2 0 0 04:30:05 0 3.3.3.3 4 100 273 274 2 0 0 04:30:05 0 4.4.4.4 4 100 273 273 2 0 0 04:30:00 0

    Here's the file contents after running that script.

    $ cat pm_1082609_all.csv 2.2.2.2,4,100,273,274,2,0,0,04:30:05,0 3.3.3.3,4,100,273,274,2,0,0,04:30:05,0 4.4.4.4,4,100,273,273,2,0,0,04:30:00,0
    $ cat pm_1082609_sel.csv 192.168.10.3,2.2.2.2,04:30:05 192.168.10.3,3.3.3.3,04:30:05 192.168.10.3,4.4.4.4,04:30:00

    -- Ken

      Thanks a lot Ken. By using your sample script i was able to extract the required columns.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1082609]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (3)
As of 2022-07-01 10:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My most frequent journeys are powered by:









    Results (98 votes). Check out past polls.

    Notices?