Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Need some data structures cooked

by ibanix (Hermit)
on Nov 19, 2002 at 20:50 UTC ( [id://214252]=perlquestion: print w/replies, xml ) Need Help??

ibanix has asked for the wisdom of the Perl Monks concerning the following question:

I got some good replies to my other (first!) posting, so I thought I'd post my whole script here.

This script collects connection information from a F5 BigIP load-balancer (runs BSDi) and generates some nice graphs.

The data structures are really ugly, and I'd love to have some Wise Monks show me how to make this more efficent and clean.

Note: SimpleGraph is a homebrew module which I can not release due to licensing.

#!/usr/bin/perl use simplegraph; # %client_ip = (); # %frontend = (); # %total_conn = (); %vips = (); %scale = (); %name = (); $iteration = 0; # Set the graph max scale per server $scale{'64.210.209.50'} = 3500; $scale{'64.210.209.51'} = 2000; $scale{'64.210.209.54'} = 3500; $scale{'64.210.209.56'} = 100; $scale{'64.210.209.61'} = 3000; $scale{'64.210.209.132'} = 500; # Set VIPs -> names mapping $name{'64.210.209.50'} = 'www.zwire.com'; $name{'64.210.209.51'} = 'www.adquest3d.com'; $name{'64.210.209.54'} = 'bannerads.zwire.com'; $name{'64.210.209.56'} = 'sitemanager.zwire.com'; $name{'64.210.209.61'} = 'imagesource.zwire.com'; $name{'64.210.209.132'} = 'www.carcast.com'; # Loop forever while(1) { $iteration++; # Reset data @connections = `bigpipe conn dump`; foreach $line (@connections) { if ($line =~ /^\s*(\d+\.\d+\.\d+\.\d+):\d+ \s*->\s*(\d+\.\d+\.\d+\.\d+):\d+ \s*->\s*(\d+\.\d+\.\d+\.\d+)/x) { # $client_ip{$1}{$iteration}++; $vips{$2}{$iteration}++; # $frontend{$3}{$iteration}++; } } # Push our new data onto the stack, but remove from the queue if we ha +ve more than 50 foreach $k (keys %vips) { push @{$vips{$k}{'stack'}}, $vips{$k}{$iteration}; if ( $#{ $vips{$k}{'stack'} } > 50 ) { shift @{$vips{$k}{'stack'}}; } } # Make a graph for each VIP foreach $k (keys %vips) { if ($name{$k}) { $label = $name{$k}." connections"; } else { # Skip VIPs that aren't in %name, we don't care about +them next; } my $image = SimpleGraph->new( width => 230, height => 120, yscale => $scale{$k}, marginwidth => 5, borderright => 5, borderleft => 30, gridlabels => yes, gridlines => ($scale{$k} / 4), label => $label, ); my $temp = $vips{$k}{'stack'}; $image->PlotSeries( color => "0,0,255", data => $temp, ); $image->Draw(file => "/usr/local/www/local/connections$k.tmp") +; `mv -f /usr/local/www/local/connections$k.tmp /usr/local/www/l +ocal/connections$k.gif`; } sleep 10; }
<-> In general, we find that those who disparage a given operating system, language, or philosophy have never had to use it in pratice. <->

Replies are listed 'Best First'.
Re: Need some data structures cooked
by dreadpiratepeter (Priest) on Nov 19, 2002 at 21:14 UTC
    #1. You aren't using strict.
    #2. You aren't using strict.
    #3. You are calling external programs without checking for errors.
    #4. You aren't using strict.


    -pete
    "Worry is like a rocking chair. It gives you something to do, but it doesn't get you anywhere."
Re: Need some data structures cooked
by dws (Chancellor) on Nov 19, 2002 at 21:30 UTC
    I don't get why you care about $iteration. You use it to accumulate counts, pull the counts out of it to add them to {'stack'}, then never throw it away. Looks like a memory leak to me.

    You could simplify your data structures by using separate hashes for per-iteration and accumulated data. Then you would write

    $vips{$2}++; ... push @{$vipstack{$k}}, $vips{$k}; shift @{$vipstack{$k}} if @{$vipstack{$k}} > 50; ... my $temp = $vipstack{$k};
    Oh, and

    #5. You aren't using strict.

Re: Need some data structures cooked
by UnderMine (Friar) on Nov 19, 2002 at 21:43 UTC
    #!/usr/bin/perl use strict; use simplegraph; $iteration = 0; $servers={'64.210.209.50' => { domain => 'www.zwire.com', scale => 3500, vips => [], }, '64.210.209.51' => {}, };
    Trap your errors .. and link/unlink.

    splice (@{$vips{$k}{'stack'}},0,$#{$vips{$k}{'stack'}}-50) if ($#{$vip +s{$k}{'stack'}}>50);
    Hope this helps
    UnderMine
      Well cooked!  Oddly enough, all you need is:    splice @{$vips{$k}{'stack'}}, 0, -50; It will leave the array untouched if it is not greater than 50.
      $ perldoc -f splice . . . If LENGTH is negative, leaves that many elements off the end of the + array.
      (I guess they mean "off the deleted array" -- at all events, that's how it works (and also what makes sense).  It's explained clearly here!)

        p
Re: Need some data structures cooked
by BrowserUk (Patriarch) on Nov 20, 2002 at 00:52 UTC

    Update: As dws points out below, I completly missed the hash growth problem. I added a line below to deal with that (and corrected my obligatory typo also pointed out to me by dws Thanks!)

    I didn't think that there was too much wrong with your data structures. You could squash the 3 hashes into 1 but they seem okay as they are. Actually I didn't see to much wrong at all with your code, though I do use warning and strict religously, they aren't obligatory. You really should be checking the return codes from your backtick commands, I left this as comments as I don't know what values would be correct for those two programs. You could probably recode the ... unless $? == 0 to be ... if $?; but you'll need to check the docs for the right values.

    Given your code as a starting point there are a few things that I would do differently but most of them are a matter of taste rather than any real benefit.

    I offer my lightly reworked version of your code which you may or may not find useful.

    #!/usr/bin/perl -w use strict; #use simplegraph; my %vips = (); # Set the graph max scale per server my %scale = ( '64.210.209.50' => 3500, '64.210.209.51' => 2000, '64.210.209.54' => 3500, '64.210.209.56' => 100, '64.210.209.61' => 3000, '64.210.209.132' => 500 ); # Set VIPs -> names mapping my %name = ( '64.210.209.50' => 'www.zwire.com', '64.210.209.51' => 'www.adquest3d.com', '64.210.209.54' => 'bannerads.zwire.com', '64.210.209.56' => 'sitemanager.zwire.com', '64.210.209.61' => 'imagesource.zwire.com', '64.210.209.132' => 'www.carcast.com', ); my $re_ip = qr/(\d+\.\d+\.\d+\.\d+):\d+/; my $re_line = qr/ ^ \s* $re_ip \s* -> \s* $re_ip \s* -> \s* $re_ip /x; my $iteration = 0; # Loop forever do { $iteration++; # Reset data my @connections = `bigpipe conn dump`; # You'll have to check docs for bigpipe to decide what constitutes + a failure code # warn "bigpipe failed with rc:$?\n" and next unless $? >> 8 == 0; +# or whatever foreach my $line (@connections) { $vips{$2}{$iteration}++ if ($line =~ /$re_line/); } # Push our new data onto the stack, but remove from the queue if w +e have more than 50 foreach my $k (keys %vips) { push @{$vips{$k}{'stack'}}, $vips{$k}{$iteration}; shift @{$vips{$k}{'stack'}} if $#{$vips{$k}{'stack'}} > 50; ## THIS LINE SHOULD BE ADDED!! See [dws]'s post below. delete $vips{$k}{$iteration}; } # Make a graph for each VIP foreach my $k (keys %vips) { # Skip VIPs that aren't in %name, we don't care about them next unless $name{$k}; my $image = SimpleGraph->new( width => 230, height => 120, yscale => $scale{$k}, marginwidth => 5, borderright => 5, borderleft => 30, gridlabels => 'yes', gridlines => ($scale{$k} / 4), label => $name{$k}." connections", ); $image->PlotSeries( color => "0,0,255", data => $vips{$k}{'st +ack'} ); $image->Draw( file => "/usr/local/www/local/connections +$k.tmp" ); `mv -f /usr/local/www/local/connections$k.tmp /usr/local/www/l +ocal/connections$k.gif`; # warn "mv failed with rc:$?\n" unless $? >> 8 == 0; # or whate +ver? } } while(sleep 10);

    Okay you lot, get your wings on the left, halos on the right. It's one size fits all, and "No!", you can't have a different color.
    Pick up your cloud down the end and "Yes" if you get allocated a grey one they are a bit damp under foot, but someone has to get them.
    Get used to the wings fast cos its an 8 hour day...unless the Govenor calls for a cyclone or hurricane, in which case 16 hour shifts are mandatory.
    Just be grateful that you arrived just as the tornado season finished. Them buggers are real work.

      I didn't think that there was too much wrong with your data structures.

      Then how does $vips{$k}{$iteration} ever get deleted when $iteration moves on to $iteration++ ? Looks like out-of-control HoH growth to me.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://214252]
Approved by VSarkiss
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (2)
As of 2024-04-20 04:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found