How do I compute the difference of two arrays? How do I compute the intersect ion of two arrays?

Use a hash. Here's code to do both and more. It assumes that each element is unique in a given array:

@union = @intersection = @difference = ();
%count = ();
foreach $element (@array1, @array2) { $count{$element}++ }

foreach $element (keys %count) {
  push @union, $element;
  push @{ $count{$element} > 1 ? \@intersection : \@difference
  }, $element;
}
[download]

Note that this is the *symmetric difference*, that is, all elements in either A or in B but not in both. Think of it as an xor operation.

The example computes the symmetric difference, but most likely you will only be interested in the pages that are new on the web and missing in your local copy, so you will want to modify the check as follows so it only gives you the locally missing items:

use strict;

my (@local) = get_local_ids();
my (@remote) = get_remote_ids();

my %have_local = ();
foreach $element (@local) { $have_local{$element}++ };

foreach $id (@remote) {
  next if $have_local{$id};
  retrieve($id);
  $have_local{$id}++;
}
[download]