OK, let me rephrase your problem to see if I understood you correctly. I assume that you want to compare several (other) files to one base file. You want to calculate a similarity rating between those files based on the number of words that are same (=intersection) and the total number of words (=total) in each of the other files - this last point I'm not sure about, it could also be total number of words in base and other file together.

Does this do what you want?

#!/usr/local/bin/perl -w
use strict ;

my $stopfile = 'stopwords';
my $base= shift @ARGV;

# read in stopwords
open STOP, "<$stopfile" or die $!;
chomp( my @stop= <STOP> );
close STOP;

my %stopwords=();
# add the empty string '' to the stopwords as well
@stopwords{@stop,''} = ();

# read in basefile
my %base_filterwords=();

open BASETEXT, "<$base" or die $!;
while ( <BASETEXT> ) {
   @base_filterwords{ map { my $l = lc; exists $stopwords{$l}?():$l } 
+split /\W+/ } = ();
}
close BASETEXT;

# read in other files
my %other_filterwords=();
while ( <> ) {
   @other_filterwords{ map { my $l = lc; exists $stopwords{$l}?():$l }
+ split /\W+/ } = ();
}
continue {
   if (eof) {
      my $total = scalar keys %other_filterwords;
      my $intersect = 0;

      for my $key (keys %other_filterwords) {
         $intersect++ if exists $base_filterwords{$key};
      }

      my $sim = 2*$intersect/$total;

      print "Similarity between $base and $ARGV = $sim\n";
      print "\t@{[keys %other_filterwords]}\n";
      print "\t@{[keys %base_filterwords]}\n";

      # reset other filter words
      %other_filterwords = ();
   }
}
[download]

Update Fixed some small bugs in the code ...

-- Hofmator

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.