Decouple the article import from detecting duplicates - the latter process could be quite involved, and it will allow you to play with the detection algorithm without interrupting your import process.

Have a look at this article: Building a Vector Space Search Engine in Perl.

What you need to do could be seen as an instance of the document classification problem. A search engine would help find documents would could be duplicates of a target document, and then you could use something like Text::Similarity to perform a more robust comparison (or flag them for manual review.)