Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

comment on
( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Yeah, me too. But I'm a capitalist, so I like to see what people are writing checks for. As I'm fed up with the various unsubstantiated claims, I've whipped up a little graphic that will hopefully cheer you up. I'll try to keep it updated regularly.

If nothing else, it'll give us all something to watch as we're overtaken by our Ruby and Python overlords.

Update:
Hmm, based on my weblogs, this seems to have generated a little buzz. Anyway, after a night to sleep on it, I've made some minor changes to better indicate my motivation. I'm not just trying to pimp Perl. I really want to know what the situation is, without all the FUD and religious debate. After all, the chart isn't all good news for Perl: the upticks in Ruby and Python in the past 10 months are important to watch when plotting a career path or making project decisions. It'll be interesting to see the direction of the trend (thnx to perrin for the indeed.com link below).

I've been asked to provide the code, so here it is; be advised the DICE screen scrape is pretty brute force, and highly subject to any CGI or format changes DICE might apply in future; and the FTP code has been altered:

Update 2: posted revised code that produces a better looking chart.

Update 3: yet another code revision:

  • added bar for "titles only" search to indicate more focused job postings
  • added link to DICE 

use Socket;
use Net::FTP;
use LWP::Simple;
use DBI;

use strict;
use warnings;

my %queries = (
'Perl',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl&Ntx=mode+
+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTR
+Y=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034
+&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=3
+3&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl+and+%28ph
+p+or+python+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl&Ntx=mode+
+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIU
+S=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.787
+15899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10
+&NUM_PER_PAGE=30&x=40&y=8',
],

'PHP',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php&Ntx=mode+m
+atchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY
+=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&
+TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=33
+&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php+and+%28per
+l+or+python+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php&Ntx=mode+m
+atchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS
+=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.7871
+5899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&
+NUM_PER_PAGE=30&x=40&y=8',
],

'Python',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python&Ntx=mod
+e+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUN
+TRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.391640
+34&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x
+=33&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python+and+%28
+php+or+perl+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python&Ntx=mod
+e+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RAD
+IUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.7
+8715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=
+10&NUM_PER_PAGE=30&x=40&y=8',
],

'Ruby',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby&Ntx=mode+
+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTR
+Y=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034
+&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=3
+3&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby+and+%28ph
+p+or+python+or+perl%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby&Ntx=mode+
+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIU
+S=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.787
+15899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10
+&NUM_PER_PAGE=30&x=40&y=8',
],
);

my %baseline = (qw/
Perl 4150
PHP 1073
Python 665
Ruby 232 
/);

my @ltime = split /\s+/, scalar localtime();
my $ltime = join('-', $ltime[-1], $ltime[1], $ltime[2]);

my %results = ();
my ($k, $url);

my $localin = 0;
my $localout = 0;
foreach (@ARGV) {
    $localin = 1, next if ($_ eq '-f');
    $localout = 1 if ($_ eq '-n');
}

if ($localin) {
    open INF, "langjobs.csv" or die $!;
    my @jobs = <INF>;
    close INF;
    chomp $jobs[-1];
    pop @jobs and
    chomp $jobs[-1]
        while (@jobs && ($jobs[-1] eq ''));
    ($ltime, $results{Perl}[0], $results{Perl}[1], $results{Perl}[2],
        $results{PHP}[0], $results{PHP}[1], $results{PHP}[2], 
        $results{Python}[0], $results{Python}[1], $results{Python}[2],
+ 
        $results{Ruby}[0], $results{Ruby}[1], $results{Ruby}[2]) = 
        split ';', $jobs[-1]; 
}
else {
    $results{$k} = [ 
        fetchCount("$k only", $url->[0]), 
        fetchCount("$k plus", $url->[1]),
        fetchCount("$k titles", $url->[2]),
        ]
        while (($k, $url) = each %queries);
#
#    now append the results to the CSV file
#
    open INF, ">>langjobs.csv" or die $!;
    print INF join(';', $ltime, @{$results{Perl}}, @{$results{PHP}}, @
+{$results{Python}}, @{$results{Ruby}}), "\n";
    close INF;
}

my $dbh = DBI->connect('dbi:Chart:', undef, undef) 
    or die "Can't connect for charting: " . $DBI::errstr;

$dbh->do('create chart currjobs (language varchar(60), total integer)'
+);
$dbh->do('create chart basejobs (language varchar(60), total integer)'
+);
$dbh->do('create chart titles_only (language varchar(60), total intege
+r)');
$dbh->do('create chart mixedjobs (language varchar(60), total integer)
+');
my $sth = $dbh->prepare('insert into currjobs values(?, ?)') or die $d
+bh->errstr;
$sth->execute($_, $results{$_}[0])
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into basejobs values(?, ?)') or die $dbh-
+>errstr;
$sth->execute($_, $baseline{$_})
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into titles_only values(?, ?)') or die $d
+bh->errstr;
$sth->execute($_, $results{$_}[2])
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into mixedjobs values(?, ?)') or die $dbh
+->errstr;
$sth->execute($_, $results{$_}[1])
    foreach (sort keys %results);

$sth = $dbh->prepare("select image, imagemap from 
    (select barchart from currjobs 
    where COLORS IN ('gold', 'lblue', 'lgreen', 'lred')
    AND SHOWVALUES=1) current,
    (select barchart from basejobs 
    where COLORS IN ('gray', 'gray', 'gray', 'gray')
    AND SHOWVALUES=1) baseline,
    (select barchart from titles_only
    where COLORS IN ('lorange', 'blue', 'green', 'red')
    AND SHOWVALUES=1) titles_only,
    (select barchart from mixedjobs 
    where COLORS IN ('orange', 'dblue', 'dgreen',    'dred')
    AND SHOWVALUES=1) mixed
where WIDTH=550
AND HEIGHT=450
AND X_AXIS=' '
AND Y_AXIS='Total Jobs'
AND TITLE='DICE Results For $ltime (10 Days/No restrict/all locs)'
AND FORMAT='PNG'
AND SHOWGRID=1
AND MAPNAME='langjobs'
AND MAPTYPE='HTML'
AND X_ORIENT='HORIZONTAL'
AND KEEPORIGIN=1
")
or die $dbh->errstr;
$sth->execute;
my $row = $sth->fetchrow_arrayref;

open HTMLF , ">langjobs.html" or die $!;
print HTMLF <<"EOHTML";
<html>
<head><title>Dynamic Language Jobs Barometer for $ltime</title></head>
<body>
<table border=0>
<tr><td align=left>
<i>The rumors of my death have been greatly exaggerated.</i></td></tr>
<tr><td align=right>- Mark Twain</td></tr>
</table>
<p>
<h2>Dynamic Languages Jobs Barometer</h2>
<h3>Total Jobs For Dynamic Languages on <a href='http://www.dice.com'>
+DICE</a> for $ltime</h3>
Each language has 
<ul>
<li>a light bar for its overall total, 
<li>a gray bar for a baseline value from Nov. 15, 2006
<li>a medium bar for titles-only search results
<li>a dark bar for listings which overlap any of the other languages.
</ul>
<p>
<img src='langjobs.png' usemap='#langjobs' title='Barchart of job list
+ings by language' border=0>
<p>

A comparative trendline is available at
<a href='http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cphp%2Cruby&
+l='>indeed.com</a>
<i>(thanks to <a href='http://www.perlmonks.com/?node_id=104919'>perri
+n</a> for the link)</i>.
<p>
<p>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://www.presicie
+nt.com'>[ Presicient Home ]</a>

<!-- BEGIN AREAMAP -->
$row->[1]

</body>
</html>

EOHTML
close HTMLF;

open IMAGEF, ">langjobs.png" or die $!;
binmode IMAGEF;
print IMAGEF $row->[0];
close IMAGEF;

unless ($localout) {
#
#    now ftp it up
#
my $ftp = Net::FTP->new('ftp.somedomain.com', Passive => 1)
    or die "Cannot connect: $@";

$ftp->login('userid','password')
    or die "Cannot login ", $ftp->message;

print "Connected, sending HTML\n";
$ftp->ascii();
$ftp->put('langjobs.html');
print "Sending image\n";
$ftp->binary();
$ftp->put('langjobs.png');
$ftp->quit;
}

sub fetchCount {
    my ($k, $url) = @_;
    sleep 3;
    my $result = get $url;
    warn "** No result for $k" and return 0
        unless $result;

    $result=~tr/\n/ /;
    $result=~s/\s+/ /gs;
    my ($count) = ($result=~/\b1\s*-\s*\d+\s+of\s+(\d+)\s+jobs/);
    warn "Can't find count for $k\n" and return 0
        unless defined $count;
    print "***$k: $count\n";
    return $count;
}

[download]

Perl Contrarian & SQL fanboy

In reply to Tired of "Perl is dead" FUD ? by renodino

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":


  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others taking refuge in the Monastery: (5)
As of 2024-04-19 22:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found