I can't answer the Test::TCP-related questions, but with regard to "is it enough to run the test of Perl modules just once or how many times are they needed to be run?" I can venture an opinion.

The fact that a test passes once, does not prove it will always pass. Take a look at this...

$ date
Tue Mar  5 09:30:54 GMT 2013
$ perl ~/tmp/scratch.pl 
1..1
ok 1 - DateTime returns a value for "second" method
$ perl ~/tmp/scratch.pl 
1..1
ok 1 - DateTime returns a value for "second" method
$ perl ~/tmp/scratch.pl 
1..1
not ok 1 - DateTime returns a value for "second" method
#   Failed test 'DateTime returns a value for "second" method'
#   at /home/tai/tmp/scratch.pl line 7.
# Looks like you failed 1 test of 1.
[download]

What was the test case? Spot the obvious error...

use strict;
use warnings;
use Test::More tests => 1;

use DateTime;

ok(
    DateTime->now->second,
    'DateTime returns a value for "second" method',
);
[download]

That's right, the test doesn't check the definedness of DateTime->now->second; it tests its truthfulness. When the clock hits an even minute, the second method correctly returns 0.

However, datetime-specific issues and race conditions can be very unpredictable. There are also all kinds of subtle bugs that can crop up running test cases in parallel with each other or in a random order.

These kind of errors are quite rare, and you may need to run the test suite hundreds of times to catch them. So in most cases the cost-benefit ratio doesn't justify running the test suite multiple times as part of the installation process.

Usually better would be a continuous integration policy, where the test suite is run before each code checkin, with test case regressions noted in the commit message. (And ideally should pass the suite before checkins are merged into the master/default/trunk codebase.) A policy like this should catch a large number of these sporadic errors.